Compare commits

...
97 Commits
Author SHA1 Message Date
peter 90523284ab release: 0.86.0 (feature/e11.5-download-failures)
gate / frontend (push) Failing after 3m17s
gate / backend (push) Successful in 4m17s
2026-09-14 01:53:06 +02:00
peter 17bfea8754 release: 0.86.0 notes 2026-09-14 01:52:34 +02:00
peter 2a9ddb78ac fix(downloads): E11.5 epic-end review round 2
Pause now re-reads the job under the same asset-then-job row lock as resume,
cancel, delete and redownload. Its unlocked write waited for a concurrent
cancel's lock and then set the released job back to paused, a holding status,
so a later delete released the same hold again and could free a file another
user's job still held (measured: the overlapping-request test ends 'paused'
instead of 'canceled' with the old read). The lock test now calls the route
handlers, so it also pins which helper each route uses.

Comments follow the code: resolve_url's contract names both verdicts and that a
failed page read is raised as itself; the inherited-failure diagnosis asks the
cookie gate as it stands when the job is claimed, so its advice fits the cookie
that job's Retry would use; the undecryptable-cookie warning no longer claims a
download goes ahead, since the worker parent reaches it only to diagnose.
2026-09-14 01:42:57 +02:00
peter e887233210 fix(downloads): E11.5 epic-end review round 1
A refused Facebook cookie on a group post is no longer "no video in this post":
measured on dev, Facebook answers with its login page (no post id at all), so the
adapter now raises its own POST_NOT_SHOWN verdict and the catalog explains it as
the refused cookie. The adapter owns both post verdicts and the "cannot parse
data" wall wording, so the Chrome retry and the catalog read one constant. The
group-post "no formats" entry is cookie-scoped (with a cookie yt-dlp only gets the
post's video link) and uses the adapter's own group-post test instead of a
"/groups/" substring.

Job transitions that move an asset's ref count (resume, cancel, delete,
redownload) re-read the job under a row lock, asset first, so two overlapping
requests can no longer release the same hold. get_or_create_asset loses an insert
race inside a SAVEPOINT instead of rolling back the caller's pending writes (Retry
lost its cleared error). Every worker-side job failure goes through
finalize_if_running and writes error_detail, so a cancel that landed first is kept
and a failure never shows an earlier one's copy. The worker asks the download
child's own cookie gate (credentials.usable_cookie) when a job inherits a failed
asset.

UI: the Facebook copy no longer points at a Retry button the failure window does
not have, says "paste" like Settings, and the HU wording is fixed
("Újraletöltés" on the re-download confirm). A category this client has no copy
for falls back to the generic sentence instead of a raw key.

Tests: route serialization of error_detail, the probed duration through a real
_download, the page-read cap and a failed read, a Retry off a pending asset, the
confirm-button labels, the lock and savepoint races.
2026-09-14 00:50:05 +02:00
peter b17d92d7c2 fix(downloads): confirm dialogs name the action on their confirm button
The download center's confirm button repeated the dialog's question
("Re-download this file?"). It now reads the action: Stop download,
Re-download, Remove, Remove from my list.
2026-09-13 23:38:11 +02:00
peter 90ee34a1b4 feat(downloads): a Facebook group post downloads its own video, or says it has none
Given a group post's link, yt-dlp either found no formats or took another
video from the page's feed data, so a "successful" retry of such a post was a
wrong download. With the user's cookie the Facebook adapter now reads the post
page, takes the single video node in the script blocks that carry the post id,
and downloads that video's watch link (the Chrome retry does the rest). No
single video in the post fails with its own message and the group-post copy,
reworded to say the post has no video; the post link is never handed to yt-dlp
as a fallback. Without a cookie nothing changes.
2026-09-13 23:38:11 +02:00
peter 118fc70e08 fix(downloads): take the duration from the file when yt-dlp gives none
A Facebook or Instagram reel extracts with no duration, so the asset kept a
NULL duration_s although the file plays for minutes, and the Library row had
no trim action. The finalize probe now returns the duration with the codecs
(the file wins, info fills a gap), and runs before the sidecars so the .nfo
runtime gets it too.
2026-09-13 23:12:16 +02:00
peter 636372511b chore: remove the unreferenced backend/Dockerfile
Nothing builds from it: both compose files build the root Dockerfile, and the
test image has its own Dockerfile.test. It installed the unpinned
requirements.txt, contradicting the lock header that every image installs
requirements.lock.
2026-09-13 23:12:16 +02:00
peter 1354f5b503 feat(downloads): a Facebook cookie download stuck at "Cannot parse data" is retried as Chrome
Measured on dev: with the user's cookie some reels still answer "Cannot parse
data" and only extract when the request impersonates Chrome. Impersonation is
a second attempt behind that one error, only when a cookie is attached — links
that already download with the cookie keep their plain path, and without a
cookie the same error is the login wall.

curl_cffi is pinned inside yt-dlp's supported range (0.10-0.15; outside it
every target silently reports "unavailable"); the lock only gains the new
packages. A test fails the gate if the locked pair can no longer impersonate.
2026-09-13 22:48:21 +02:00
peter 98ec1ffcab fix(downloads): Retry and Re-download take a cookie added after the failure
The login-wall copy tells the user to add their cookies and press Retry, but
Retry kept the job on the shared asset the worker never attaches a cookie to,
so it failed on the same wall. Resume now moves a pending/errored job onto the
asset enqueue would pick with the cookie (never off a file or a download in
flight).

Re-download salted its fresh asset with the job's previous asset id, which the
cookie gate cannot recompute, so even a cookie download rebuilt without the
cookie. The rebuild salt is now the new asset's own id (plus the user's cookie
salt when one applies), and is_authed_asset recognises it.
2026-09-13 22:31:49 +02:00
peter 18c11ca437 feat(downloads): a per-site failure catalog — Facebook's walls say why and what to do
A failed Facebook download read "Download failed": yt-dlp words the login wall
(18+ reel, closed group) as "Cannot parse data", which no category matched.
Measured on dev with the owner's cookie before writing a rule — the same text is
the login wall without a cookie and a refused cookie with one.

- errors.diagnose(msg, url, cookie_attached) tries a host-scoped catalog first
  (all-of markers, cookie state, link fragment) and falls back to the category.
  First entries: facebook.login_wall, facebook.cookie_rejected,
  facebook.group_post_video. The module doc is the recipe for the next entry.
- download_jobs.error_detail (0084) + backfill of the stored Facebook parse
  failures that were unknown.
- The pipeline's failure branch passes the link and whether the cookie went
  with it; resume and redownload clear code and detail (redownload never
  cleared error_code).
- A job meeting an already-failed asset got the text but no code, so it read
  "Download failed" where the first job said why. It is diagnosed again, with
  the cookie state from credentials.is_authed_asset (now shared with the
  cookie gate); the redacted network text classifies back to network.
- FE: failCopyKeys gives the row and the failure window one lookup — catalog
  copy, else the category. Copy in EN and HU; a contract test pins every
  backend entry to both locales.
2026-09-13 21:53:25 +02:00
peter ca5e446fbd release: 0.85.0 (feature/e11-power-navigation)
gate / frontend (push) Failing after 3m13s
gate / backend (push) Successful in 4m36s
2026-09-13 21:02:22 +02:00
peter 4c7ab12a3e release: 0.85.0 — E11 Power Navigation
Back, dialogs and menus keep your place, and the keyboard walks every list: five sprints plus three epic-end review rounds (19 → 14, 18 → 12, 14 → 14). Minor: j/k list navigation, s and g-chords, and Back-to-item are features.
2026-09-13 21:02:22 +02:00
peter c6b552e75f fix(keys): epic-end review round 3 — the part-way clip count shows, busy buttons keep focus, cards say their names
Targeted round over 3cd0c525's fix surfaces: 14 raised, 14 fixed.

- VideoEditor: the "only X of Y clips" toast went silent exactly on the job-cap/quota refusal it was
  written for (req had already reported the error); it now always shows. The cut list holds still while
  Create queues, and a retry continues the "(i/N)" series.
- ChannelPage: Subscribe/Unsubscribe use Button `loading`, "Load more" and ChannelDiscovery's Subscribe
  block the click with aria-disabled instead of native disabled; busy Block/Hide no longer light up on
  hover.
- listNav: an item can name itself (`navItemAttrs(i, name)`) — Plex posters, seasons, episodes, message
  rows and download rows pass their title; focus on a toast's Undo keeps the removed card's slot.
- AnchoredPopover: the hand-back only for focus-managed surfaces (a hover card no longer pulls focus
  back); the dead panel-ref check is gone.
- AccountPanel: no confirmation mail offered to the demo account; send_failed copy no longer blames
  the message for an unreachable server. Pager comment points at scrollParent.
2026-09-13 20:47:13 +02:00
peter 3cd0c525e2 fix(keys): epic-end review round 2 — focus comes back to the card, the right rail and the reason
Whole-epic deep-review of E11 (18 raised, 12 kept; B and G refuted, four latent test-hardening notes dropped),
plus the admin refusal copy gap round 1 surfaced.

- A context menu opened from a card the keyboard is on gives the focus back to the card. floating-ui returns
  focus through the opener's first tabbable descendant even when handed a ref, so for a non-tabbable opener the
  surface restores it itself (measured in Chrome: j, Shift+F10, Escape left it on the thumbnail link).
- Back finds a named list (a rail) by its name: a rail that appeared above it after the cache expired sent the
  focus into the wrong rail. An unnamed list is matched only when the page has the same number of lists.
- The slot kept for a removed item is forgotten once focus moves on to page content (search, a chip), so j after
  new results starts at the first visible item; a menu or dialog opened from the item keeps it.
- The list keys reach the cheat sheet only while a list has an item.
- A j-focused card root is a named group while it holds the focus.
- The channel page's Block and Hide toggles keep the focus while their request runs.
- A split export that fails part-way says how many clips were added and deselects them, so a retry adds only
  the rest.
- Config save, the one-click share link and "log out other sessions" report failures with their reason.
- VirtualGrid uses the one scroll-parent walk listNav measures against.
- Admin refusals (activate, resend, invite, set-password link) say why in HU and EN instead of "try again";
  the email-off special cases collapse into that one reason. The contract reads the codes from localizeDetail.
  HU copy points to Konfiguráció → E-mail / SMTP; HU "unreachable" matches the EN change of round 1.
2026-09-13 19:18:04 +02:00
peter f130a1c05a fix(keys): epic-end review round 1 — Back, menus and dialogs keep the focus where it belongs
Deep-review of E11 (19 raised, 14 confirmed; 2 refuted, 3 latent test-hardening notes dropped).

- Back from a sub-view (a Plex show page, a Messages thread) restores the item it was opened from
  even when focus was still inside the view: the first attempt now waits one task, past the
  sub-view's own popstate listener (measured in Chrome: it left focus on <body>).
- The restore listener is installed at load, so the first Back after a reload finds it; a push
  cancels a restore still waiting for the last landing's lists.
- Navigating over an overlay asks it again, a render apart, while its entry stays: the Plex
  player peels one panel per Back step, so Shift+W over info + stats closed one panel and dropped
  the navigation.
- j/k: a native <button> item keeps its own Tab stop; landing twice on the same item no longer
  leaks its tabindex; after the focused item is removed (a hidden card) the next j goes on from
  its slot when that slot is on screen.
- An abandoned `g` no longer swallows a modifier chord (g, then Mod+K opened the browser search).
- A hover card above a dialog no longer switches its Tab trap off (player, Modal, wizard,
  Lightbox): new layerStack `anyBlockingLayerAbove`.
- "Manage views…" closes its menu before opening the dialog, through a shared `closeThenAct`
  that ActionMenu now uses too.
- Copy: the EN welcome says "Feed"; the EN unreachable reason no longer claims a change;
  stale comments about PlayerModal's scroll lock and dialog contract.
2026-09-13 17:17:49 +02:00
peter bbd31c7143 feat(keys): Back returns focus to the item a view was opened from; "Feed" everywhere; every backend id named
Owner answers to the E11 S5 raises (2026-09-13): 1 → A, 2 → "Feed" in every language, 3 → translate the
proxy label and fix every config key missing in BOTH languages ("maintenance", "tmdb" — old bugs).

Focus across a view change (1-A). Opening a Plex title from Home's rail, a channel from the Channels
table or a title from a Plex grid switches the page or the sub-view, which unmounts the list; Back used
to hand focus to <body>, so the next `j` started over at the top. The YouTube player (an overlay)
already restored onto the card.
- `lib/listNav`: `navPosition()` (list's place in reading order, the page's list count, item index)
  and `restoreNavPosition()`: focuses the item once the page has as many lists on screen as when it was
  left, polls on a timer (a hidden tab runs no animation frames), gives up after 2 s or at once when
  something already holds focus, focuses without scrolling (the scroll restore owns the position).
- `lib/history`: `rememberNavPosition()` stamps `sfNav` onto the entry being left and returns the state
  without it (a new entry must not inherit it); one popstate listener restores and spends the stamp,
  so a later landing on the same entry (an overlay closing over it) cannot pull focus to an old item.
- Called before every view push: `setPage`, `openChannel`, `enterYtSearch`, `useHistorySubview.open`,
  the Channels tab push. `navPosition.wiring.contract.test` fails on any `history.pushState(` without
  it (the overlay stack's own push is the one exemption).

"Feed" (2). The module is "Feed" in both languages: every "hírfolyam" in the Hungarian UI (≈45 strings,
suffixed: Feedből, Feeded, Feedben…) and the prose "feed" in English that means the module ("your
Feed", "Hide from Feed"); "RSS feed" keeps its meaning. The welcome email says it too.

Backend ids without a name (3). The Configuration page showed `maintenance`, `tmdb`,
`maintenance_revalidate_batch`, `download_upload_max_bytes`, `audit_retention_days`, `plex_overlay_map`,
`tmdb_api_key` and `tmdb_language` as raw identifiers — the lookup falls back to the id, and the i18n
parity test stayed green because both languages missed the same keys. Added labels + hints (EN/HU),
the two groups (and their tab order), the missing `scheduler.jobDesc.demo_reset`, removed the stale
`plex_sync_interval_min` field, and "YouTube API kimenő proxy". `i18n/backendIds.contract.test` reads
`sysconfig.SPECS`, `scheduler.JOB_INTERVALS` and the audit/quota action enums from the backend sources
and fails on an id without its name in either language, or a name left for an id the backend dropped.
2026-09-13 07:36:41 +02:00
peter 8731508645 feat(keys): the conversation list walks with j/k; the cheat sheet names keys in the reader's language
E11 S5 — the HU/EN sweep and the keyboard-only pass over every module.

- The keyboard cheat sheet printed "Space" and "Menu" on a Hungarian sheet: a keycap that carries a
  WORD (not a legend every layout prints alike, like Esc, Ctrl or Enter) is now named by the caller in
  the reader's language (`formatChord(chord, words)`, `keys.cap.*`: Szóköz / Menü).
- The Plex player's ⌫/Esc line said "Stop & back to feed", but the player goes back to wherever it
  was opened from (Home, a Plex page): "Stop & go back" / "Leállítás és vissza".
- Messages: the conversation list is a keyboard list (NavGrid + navItemAttrs, the name button is the
  open element; the pop-out button keeps its own Enter). D1 covers every list whose rows open one
  thing; this one was missed in S4. Notification cards have no single open action and stay out, like
  the download queue.

Sweep, measured: every static `t("…")` key used in the source exists in both languages (the only
misses are template prefixes); the HU values equal to EN are proper nouns and loanwords (Admin, Plex,
Trakt, Shorts). Palette in Hungarian: "téma" makes "Téma: Midnight" the active row ahead of the channel
"Topik - mindig témában", live region "7 találat".

Keyboard-only pass in Chrome, Hungarian UI — 1024×768: every `g` chord lands on its module, `j` rings
the first item (solid 2px accent) on Home, Feed, Channels, Playlists, Plex and Downloads, `s` focuses
the search on Feed, Channels, Playlists and Plex, no page scrolls sideways. 3440×1440: `j` → → ↓
walks Home, Feed (8 columns: 0 → 1 → 9), Channels, Playlists, Plex and Downloads, no sideways scroll.
The help sheet lists every line translated; inside the Plex player "Szóköz / K" and "⌫ / Esc
Leállítás és vissza". Messages: a real `j` rings the row, Enter opens the thread.
2026-09-13 07:00:51 +02:00
peter b5d37ca142 feat(keys): j/k and the arrows walk the channel, playlist and download lists
Third and last slice of the E11 keyboard layer (S4c). S4a/S4b covered every card grid and rail; the
table-shaped pages — the channel manager, a playlist, the download center — were still Tab-stop
country: a channel row carries a priority stepper, four action buttons and a tag picker before the next
row's name.

- `ui/DataTable keyboardNav`: the rows are the items, Enter clicks the element a column marks
  `data-nav-open`. DataTable renders its rows twice (a table, and a card list below `md`); both register
  and `lib/listNav` now walks only lists on screen (`checkVisibility`), so `j` past the last row never
  carries on into the hidden copy.
- `lib/listNav` leaves a key something before it already handled (`defaultPrevented`): during a keyboard
  drag a playlist row's grip moves the row with the arrows, and dnd-kit takes the key on the document
  before listNav's window listener. Without the guard ↓ moved the focus off the grip and dropped the drag.
- `ui/NavGrid as="tbody"` for a table whose body is the list (the playlist, under dnd-kit's
  SortableContext).
- Wired: Subscriptions and Discovery tables, the card layout (`ChannelLayoutGrid` → VirtualGrid
  `keyboardNav`), Hidden/Blocked cards (the ChannelLink name is a channel's open element); a playlist's
  rows (the title plays; the grip keeps Space/Enter); the download queue, library and shared lists (the
  play button opens, `IconBtn open`; a row without one opens nothing).
- `listNav.wiring.contract.test`: `VirtualGrid` and `DataTable` must say `keyboardNav` like `Carousel`,
  `DownloadRow` is a card; the audit log's table is the one exemption (its rows open nothing), and an
  exemption that no longer matches a tag fails the test.

Measured in Chrome: Channels table j → row, ring round the whole row; a row walked down keeps 12px above
the viewport edge, walked up clears the sticky header by 35px; Enter opens the channel. Cards 3 columns:
→ 0→1, ↓ 1→4. Hidden and Discovery: j + Enter open the channel. Playlist: j + Enter plays, Escape returns
focus to the row; a keyboard drag (Enter on the grip, ↓) moves the row with focus kept on the grip, and
Escape cancels with the order unchanged. Downloads library: Enter plays a video row, does nothing on a
plain file row.
2026-09-13 06:08:05 +02:00
peter eb0507f942 fix(keys): a card reached by Tab rings the whole card, not just its poster
Owner report: on the Plex browse grid, Tab onto the first card ringed only the poster, and the first
arrow key then moved the ring round the whole card. The poster is the card's own role=button Tab stop,
so Tab gave it the app-wide button ring, while listNav's arrows focus the card root (D2: the ring goes
round the whole item).

The card's "open" element (`data-nav-open` — the Plex poster, a video card's title link) now stands
for the card: when it holds keyboard focus the card root wears the ring and the element's own ring is
transparent. Controls inside the card (watched toggle, actions) keep their own ring.

Measured in Chrome (jsdom evaluates neither :focus-visible nor :has): Plex browse, Tab from the sort
control → poster focused, card outline 2px accent, poster outline transparent; → moves to card 1 with
the same ring. Feed, Tab onto a card's title link → the card is ringed, the link is not.
2026-09-13 05:47:43 +02:00
peter 6d09b94bbb feat(keys): j/k and the arrows walk Home's rails and every Plex grid and rail
Second slice of the E11 keyboard layer (S4b). S4a gave the feed, the library and the channel page
`j`/`k`, the arrows and Enter; Home's shelves and the whole Plex module were still Tab-stop country —
a Plex poster is its own Tab stop, and a rail's cards sat behind every control of the cards before.

- `lib/listNav`: a list that mounts every item no longer declares a count or columns — both are
  measured off the DOM when a key is pressed, so an auto-fill grid's columns follow its width and a
  rail is one row. ↑ on a list's top row and ↓ on its bottom row go on to the neighbouring list, onto
  the item horizontally nearest the one left (owner decision D4: ←→ within a rail, ↑↓ to the rail
  above or below). The first `j` skips cards a rail has scrolled out to the left.
- `Carousel keyboardNav` registers the scroller; the scroller gets a 4px inset (against a -4px margin,
  layout unchanged) so the focus ring is not clipped, and `[data-nav-index]` a 2.5rem inline scroll
  margin so a card stepped into view clears the rail's edge fade and paging arrow.
- `ui/NavGrid`: the plain, fully mounted grid that registers itself.
- `navIndex` on PlexPosterCard / SeasonCard / EpisodeCard (the card root is the item, the poster the
  open element — the ring goes round poster and title, D2), PosterTile and Home's ShelfTile (the button
  is the item); cast, related and collection-strip buttons carry the attrs directly.
- Wired: Home's comfort and resume rails; Plex Browse titles + episodes, Discover rails + genres,
  collection view, playlists/collections browse, person known-for + filmography, show seasons, season
  episodes, cast/related strips, info-page cast + collection strips.
- `listNav.wiring.contract.test`: every `Carousel` says `keyboardNav`, every card says `navIndex`, and a
  card with `navIndex` sits inside a NavGrid / Carousel / VirtualGrid — both halves are opt-in, and
  either one missing leaves a list the keyboard silently skips.
2026-09-13 05:38:36 +02:00
peter 5274216981 feat(ui): PageUp/PageDown page through every dropdown list, not just the palette
The palette got PageUp/PageDown, and the same crawl stayed everywhere else a list opens under a
control: the sidebar's channel filter, the Plex genre/studio/collection pickers, the filter and sort
menus, submenus of every playlist you own. floating-ui's list navigation has only the arrows and
Home/End.

- `lib/listPaging`: the one definition of a page (the rows wholly in view minus one, a fixed 10 with
  no layout) and of where it lands (stops at both ends, never on a row it may not take, looking back
  toward the start rather than past it). The palette now uses it instead of its own copy, and
  `listNav` takes its scroll-parent helper from here.
- `AnchoredPopover`: one keydown handler on the reference AND the surface, so a combobox (keys on its
  input, `aria-activedescendant`) and a roving menu (keys on the focused row) both page. It runs after
  floating-ui's own, which has by then switched to keyboard modality and scrolls the new row in. A
  disabled row is never the target; a submenu's key does not also page the menu around it; with no
  mounted rows (closed, a hover card, a dialog) the key keeps its native meaning.

Measured in the browser: the channel filter shows 8 rows and pages by 7 with the active row in view
after the scroll; a menu pages to its last row and back, the page behind it does not scroll; the
palette still pages by 11 of its 12 visible rows.
2026-09-13 05:20:53 +02:00
peter 75d89613d0 feat(palette): PageUp/PageDown move a page of results at a time
The jump-to palette lists every module, channel, playlist, view and action — 461 rows on the
owner's account, ~11 of them in view — and the arrows were the only way through it: forty presses to
cross the channel group.

- PageDown/PageUp step by the rows that fit in the list right now, minus one, so the row you paged
  from stays in sight at the new page's edge. Measured per press (the list is 50vh and group headings
  eat into it); a fixed 10 when there is no layout to measure.
- Unlike the arrows they stop at the ends instead of wrapping: a page-sized jump that lands back at
  the top reads as having lost your place.
- Home/End stay the text field's own.

Measured in the browser: 0 → 10 → 20 rows, stops on row 461, PageUp from the end steps 9 (a group
heading is in view), the selected row stays visible throughout.
2026-09-13 05:03:35 +02:00
peter 8fb276aa90 feat(keys): j/k walk the feed, s finds the search box, g jumps to a module
A feed card cost ~10 Tab stops, so the tenth card sat ~135 Tabs from the top of the page, and the
only keyboard focus ring (a 60% accent outline on the thumbnail) all but vanished on the dark glass.
First slice of the E11 keyboard layer (S4a); Home, Plex and the tables follow in S4b/S4c.

- `lib/listNav`: one window listener over every registered list, walked in document order. `j`/`k`
  step item to item (on into the next list — owner decision D4), the arrows move around a grid, Enter
  clicks the item's own `data-nav-open` element, so it opens exactly like a click. An item root is
  never a Tab stop: it gets `tabindex=-1` only while the keyboard is on it, and keeps it until the next
  item takes it, so the player's focus restore lands back on the card. The first `j` takes the first
  card you can see, not the top of the list.
- `VirtualGrid` takes part with `keyboardNav` (column count + scrolling an unmounted row in);
  `VirtualFeed` turns it on, so the feed, the library and the channel page all have it.
- Keymap: two-step sequences (`g h`) whose pending step belongs to the scope that started it; the
  page-level scopes (`page`, `search`, `list`) are quiet while a layer that owns the keyboard is open,
  enforced in `dispatch`. A test pins the owner's rule that none of them needs a modifier on the
  Hungarian layout.
- `g h/f/c/p/x/n/m/d/s/,` jump to a module this account can reach; `s` focuses the header search box
  and selects the old query; Escape in it hands the keyboard back to the page (Plex's typeahead spends
  its first Escape on its panel).
- Hover popovers are passive layers: an open tooltip still takes Escape first but no longer silences
  the page keys.
- Focus ring (owner decision D2 = B): solid 2px accent around the whole item; the app-wide button/
  link/row ring goes from 60% to solid too. The item keeps a transparent resting outline because the
  video card transitions `all` — without it every `j` flashed a 3px white ring (measured in Chrome).
- Help sheet: the new keys, in a fixed order after the surface's own; a `g` chord only for a reachable
  module; opened over a player or dialog it leaves out the page-level keys and the context-menu key,
  which nothing can reach there (the latter pre-existing since E10).
- CLAUDE.md: a canonical-patterns row for list/grid keys (`useListNav` + `navItemAttrs` +
  `data-nav-open`, not a private arrow-key handler).
2026-09-13 04:16:20 +02:00
peter 78a9324381 fix(ui): the palette ranks what you typed, and lib stops leaning on components
Typing "theme" put the channel "The Meltzone Podcast" (matched only through its handle
`@themeltzone`) above every "Theme: …" row, and Enter opened the channel — the palette listed
matches in pure group order and always highlighted the first one.

- CommandPalette ranks each word: label starts with it > elsewhere in the label > only the hint,
  keywords or group name; an item ranks by its worst word, then the sum. The group order stays
  fixed, rows are ranked inside each group, and the best row overall starts highlighted. The
  highlight is held by id and follows the best match until the arrows or the pointer pick a row,
  so a channel list landing mid-typing no longer shifts it.
- A11y: result groups are `role="group"` named by their heading; `aria-expanded` is false with
  nothing to choose; the listbox stays mounted (hidden) so `aria-controls` never names a missing
  element, and "Nothing matches" sits outside it; a polite status announces the count once the
  typing pauses.
- `Column`/`FilterMap`/`filterRows`/`isActive` move from `ui/tableHeader` to `lib/tableColumns`,
  and the player's auto/loop modes from `playerModalControls` to `lib/playerModes` — three lib
  helpers imported the column contract from a ui component, and `playerModalHooks` its types from a
  feature component.
- eslint: `lib/` may not import from `components/` at any depth, type-only and re-exports included;
  the boundary contract test proves it fires. CLAUDE.md and the ui README say so.
- keymap: `dispatch`'s docblock sat stacked on `matchesShortcut`'s; moved onto `dispatch`.
2026-09-13 02:14:07 +02:00
peter 8a50538275 fix(ui): a write that fails says what failed and why, wherever it fails
`req` opens its blocking dialog for 400/409/422/5xx only. A 403/404/429 was left to the caller and a
502-504 or lost connection got just the self-clearing connection notice, so 34 mutations without a
reporting onError and 15 direct api writes with a bare or missing catch failed in silence: a hidden
channel that snapped back, a share link that never revoked, a confirmed account delete that ended
with nothing said.

- lib/queryClient: one MutationCache reporter. react-query awaits it BEFORE the mutation's own
  onError, so it waits a task and skips a failure the site already reported; the toast title comes
  from `meta.failTitle`, `meta.errorShownInline` opts out.
- notifyActionError: title names WHAT failed, the line under it says WHY (`failureReason`: a
  structured localized detail, else the status). It marks the error reported (`markReported`), so
  nothing reports one failure twice; `notifyKnownFailure` for a site's own specific sentence.
- Every silent site named (45 new HU/EN titles, 4 reasons). Stale "the global dialog shows it"
  comments corrected on the job actions, channel block, Plex sync, cookies, download enqueue,
  account delete and usePref.
- Channels passed the tag hook's channel id to the reporter as the error; the tag and priority hooks
  now report themselves, with the real error.
- A contract test keeps every useMutation naming its failure; CLAUDE.md gains the pattern row.
- history.dom.test: the "navigates only once the entries are back" assertion raced an async act's
  macrotask flush (1 in 5 runs); it now asserts from a later popstate listener.
2026-09-13 01:45:28 +02:00
peter b4aea7fa5b fix(ui): dialogs, menus and busy buttons keep the keyboard focus where it belongs
Measured in Chrome before each change:

- A dialog opened from a menu lost its focus: the menu row ran its action
  and closed in one batch, and floating-ui's focus return (a microtask)
  moved the focus to the row BEHIND the confirm dialog's scrim 7 ms after
  it opened, where Enter activated the row. MenuItemList now closes first
  and runs the action a task later; the dialog also captures the row, not
  a dying menu item, as its restore target.
- A side panel opened as a flyout collapsed the moment a dialog opened
  from inside it, unmounting the row the dialog hands the focus back to.
  useHoverFocusWithin holds while the focus is inside an aria-modal dialog.
- ui/Button's loading state set native disabled; Chrome moved the focus of
  the pressed button to <body> and never gave it back. Loading is now
  aria-disabled with a click guard that also cancels implicit submission.
- Escape/Enter in the playlist rail's rename field dropped the focus on
  <body>. The row refocuses from its ref callback when it re-attaches (a
  post-commit focus sticks; the earlier attempt ran before the button
  existed).
- PlayerModal was the one aria-modal shell without the dialog contract:
  it is now named by its title, traps Tab, restores the focus to its
  opener and shares one counted body scroll lock with every dialog.
2026-09-13 01:04:11 +02:00
peter 92c305bab9 fix(nav): a navigation closes every overlay before it pushes
The overlay history stack let a navigation land ABOVE open overlays and
"buried" their entries with a counter. Measured in Chrome: Shift+W over
About navigated under the still-open dialog, the next Back both undid the
navigation and closed About, and the Back after that did nothing; Shift+W
on Home with the player open did nothing at all. The counter could also be
spent by a different overlay closing first, and callers that could not
report whether they pushed were assumed to have.

setPage and openChannel now close every open overlay and push only once
the history is back in sync (one entry per live overlay, no programmatic
back() in flight, no reconcile queued), so nothing is ever buried and the
whole mechanism goes: navigateOverOverlays, closeThenNavigate, buryUnder,
setPage's boolean, and the three test files that pinned them. The palette
closes only itself before running a row, so a theme picked over the player
keeps the player. Shift+W on Home with the player open now closes it
(owner decision 2026-09-09).

The wait is on state, not on "a popstate": the old 250 ms fallback could
run with back()s still in flight. An overlay that refuses to close within
1.5 s drops the command instead of being navigated out from under.
2026-09-13 00:17:09 +02:00
peter 32fe0768b6 release: 0.84.0 (bug/hunt-2026-09)
gate / frontend (push) Failing after 2m45s
gate / backend (push) Successful in 4m5s
2026-09-12 16:44:01 +02:00
peter 415a6c16f9 release: 0.84.0 — the owner's sixteen-item bug hunt
Sixteen items written from real use: eleven fixed, one measured and closed without a code change
(the micro-stutter is a 23.976 fps film on a 120.0148 Hz display, reproduced identically outside
Siftlode), two parked for a reproducible case, one scoped out to its own sprint. Two of the eleven
were destroying user state — a YouTube video that rewound itself on reopen, and a "clear resume
point" that told Plex nothing about an in-progress item.

Then seven deep-review rounds over the fixes and over each other's fixes: 32 → 24 → 43 → 31 → 22 →
26 → 18 raised. Minor, not patch: item 14 added an admin knob, item 10 a sort control, and items
2/4/5/7 are features.
2026-09-12 16:43:54 +02:00
peter 8c51062e5b fix(api,ui): the demo refusal says why at every site, and the contract guard cannot fail open
Round 6 converted three of four `is_demo` 403s to a structured detail and wrote a docstring saying it
had converted all the sites there are. The account-delete route in `routes/me.py` still answered a
hand-written English sentence, which `notifyActionError` discards in favour of the caller's
translated fallback — so the demo user gets no hint that the demo account is the reason. It is
UI-gated today, so the string is unreachable; what was reachable was the false completeness claim,
which is precisely what stops the next reader from looking. The site is converted, and the list is
now DERIVED from the source: a new test walks `app/` for every `is_demo`-guarded 403, asserts each
raises a structured detail, and fails when the count moves — so this file cannot fall behind the code
the way it just did.

The same round's frontend contract guard was failing open. Its pattern required a bare identifier
before `.detail` and excluded `.detail` used as a last call argument, so `setErr(e.detail)`,
`(e as HttpError).detail`, `String(e.detail)` and four more idiomatic forms were invisible — and
because the file set is compared with `toEqual`, a file the pattern cannot see is simply never listed
and the gate stays green over a fresh copy of the contract. Widened to cover cast receivers, optional
chains and call-argument reads; the pinned set is unchanged, so the narrowing had removed no false
positive. A destructured `detail` stays out of reach (a pattern loose enough for it fires on
react-query's own `const { data: detail }`), and that hole is written into the test rather than
papered over with two innocent files.

Its anti-vacuity guard also did not guard anything: the comment says the glob must have matched
something, but the assertion under it read a sibling file by its own URL, which resolves whether or
not the glob enumerated anything. That only mattered while the pinned list was non-empty — the moment
the deferral sweep lands and it becomes [], the whole file would pass over a glob matching zero
files. The scan size is asserted now.
2026-09-12 16:22:27 +02:00
peter 63bf1bca78 fix(plex): an accepted reorder refetches the detail it just wrote
`commit`'s success handler writes the dragged list into the detail cache — round 6's fix, so a
re-entry inside the 30s staleTime renders the accepted order without a request — but then invalidated
only the LIST key, while the YouTube sibling it cites invalidates both. `next` is the membership of
the render that started the drag, and `reorder_playlist` only touches the rating_keys it is handed,
so it never ratifies that list: anything that landed meanwhile was clobbered by the write with
nothing scheduled to correct it. `commit` sets no busy flag either, so a removal really can complete
while a reorder is in flight, and with staleTime 30s and refetchOnWindowFocus off the phantom row
stayed on screen for as long as the view was mounted — and the next drag persisted an order built
from it.

Round 6's test had to be re-pointed, because it was written in a world with no refetch: its mock
answered the pre-reorder order forever, which a server that just accepted the write cannot do. It now
holds the refetch open instead, so it still pins what round 6 actually fixed — the cache is right
WITHOUT waiting for a request — and would fail if the write were deleted. Both halves are
mutation-checked against each other.
2026-09-12 16:22:27 +02:00
peter c7a377396b fix(ui): a closing dialog leaves the focus to the dialog standing above it
Round 6 widened dialog focus ownership to count focus inside ANY `aria-modal` subtree as ours, so a
dialog and the dialog stacked on it closing together would still hand the focus back. That clause has
no way to tell a layer closing WITH us from one that stays open, so a dialog closing on its own — an
in-flight mutation's `onSuccess`, not a click, since a scrim covers it — under a modal raised above it
pulled the focus onto its own pre-open trigger, behind that scrim, where Enter still activated it.
Three verifiers confirmed the mechanism and two corrected the severity down: it self-heals on the
next Tab, which is why this is a containment violation rather than a broken end state.

`lib/layerStack` cannot refine it. Measured against the real hook: at the instant the lower dialog's
focus cleanup runs, the both-closing case and the sibling-stays-open case give identical readings —
`anyLayerAbove` true, the upper layer topmost — because passive cleanups run parent-first and
`useDialogChrome` is declared before the effect that calls `removeLayer`. Gating on the stack turns
round 6's own test red. What separates them is the DOM: React detaches a deleted subtree before
passive cleanups, so a layer closing with us is already disconnected while one that stays open is
not. Both tests are green, and each was mutation-checked against the other.

Verified in Chrome on localdev: opening the cheat sheet from a sidebar button and closing it restores
that button; with the command palette stacked over the cheat sheet, closing the top one leaves the
focus neither on the background trigger nor on <body>, and closing both returns it to the trigger.

Welcome's Lightbox got the trap and the restore in round 6 but not the layer membership the other
shells have, so its Escape and Tab fired from a raw window listener whatever was on top — the shape
the wizard's comment records being bitten by, unreachable today only by a mount ordering nothing
enforces. It joins the stack. Its comment also read as though the four-shell sweep were finished:
`PlayerModal` is still an `aria-modal` surface with no name, no trap and no restore, which is on
ROADMAP.md now rather than implied to be done.
2026-09-12 16:22:06 +02:00
peter 8036251589 fix(plex): the reclaim window rolls, and a refused finish is not performed through a third route
Two ways round 6's own fixes did not bound what their comments said they bounded.

The reclaim budget's window anchor was written only where the window RESET, so it marked the window's
start rather than the last loss — a tumbling window, not the rolling one the constant is named for.
Any loss cadence slower than half the window satisfied the reset on every second loss, so the budget
never reached its bound: measured over 200 loss cycles, 61/70/90/119s apart all gave 200 reclaims and
never an error, while 5s and 30s apart correctly gave two and then the honest answer. The failing
band is exactly the case the ERROR handler names — a transcode that falls behind realtime — and both
committed tests spaced their losses 0s and 10min apart, i.e. outside it. The anchor now moves with
every loss.

And `finish`'s refusal handler lowered the play-through latch, which is the only gate on the 10s
interval, the page-hide beacon and `settle`. All three post `absRef`, which on the play-to-the-end
path IS the end of the media, and the backend reads a position within `_FINISH_MARGIN_S` of the
duration as a finish — so within ten seconds of the toast saying the mark had failed, the item was
marked watched, its position cleared and a scrobble sent to Plex. Round 6 closed the explicit
rollback write and left the other three open; the test it added set `upNext: false`, which tears the
effect down before any of them can fire. The refusal now takes back the CLAIM rather than the latch:
a second flag stops `settle` re-asserting watched, and playing again clears both.

`FINISH_MARGIN_S` duplicated the backend's constant in a second language with nothing relating them,
and its comment had the asymmetry backwards — if the BACKEND's margin grows, this guard starts
admitting a rollback the backend reads as a finish, i.e. re-opens the defect above. Pinned by a
contract test that reads the Python, and the comment corrected.
2026-09-12 16:21:51 +02:00
peter 71d7646bf1 fix(plex): a name this session cannot write fails at once, and the layout has one spelling
Round 6 turned `artifact_may_arrive` into a positive rule, but keyed it to the HLS vocabulary in
general rather than to what THIS session produces: the playlist regex accepted master/index/stream_*
for every session and the `seq == 0` branch accepted any variant token unconditionally. The two argv
branches are disjoint — a single-audio session is given `seg_%d.ts` + `index.m3u8` and no `%v` at
all, so `master.m3u8`, `stream_*.m3u8` and every `seg_<variant>_<n>.ts` are names it can never write,
and vice versa — so each of those was read as "an HLS name, so wait and see" and held a threadpool
worker, plus its pooled DB connection, for the full timeout. That is verbatim the round-5 outage the
route's own comment claims it prevents.

Measured through the real route against a live single-audio session, before: master.m3u8 404 after
30.0s, stream_q7z.m3u8 404 after 30.0s, seg_a0_0.ts / seg_zzzz_0.ts / seg_0_0.ts 404 after 20.1s
each. After, on the running localdev API: 404 in 44-316ms for all of them, while index.m3u8,
seg_0.ts and seg_1.ts still answer 200 — nothing is over-refused.

The session now carries the rendition names its ffmpeg was told to write, and `entry` is derived from
them instead of passed. And "one place" now means the argv BUILDS its patterns from the same
functions the parser reads back (`segment_name("%d", "%v")`, `variant_playlist_name("%v")`): the
layout was still spelled out independently in six places, with `_VIDEO_VARIANT` read by no consumer,
which is the drift that caused round 6's HIGH. A test asserts the argv's own patterns are what those
functions produce, and that every name they expand to is one the guard accepts.

The load-bearing MEASURED comment cited ffmpeg 8.1.1 — the Windows host's. The binary that writes
these files is the API container's 7.1.5; re-measured there, the conclusion holds.
2026-09-12 16:21:37 +02:00
peter 3eb0f41962 fix(plex): a Plex reorder the server accepted is the order the cache holds
The YouTube list got this rule in round 5 — one accepted write, one cache entry
that matches it — and the Plex list got only the rollback half of that fix. Its
success path invalidated `qk.plexPlaylists()`, a disjoint key root, and never
touched `qk.plexPlaylist(id)`: the key `putBack` rolls back to, and the one a
re-entry renders from without refetching.

So after an accepted drag the detail cache still held the pre-reorder order.
Leaving the playlist and coming back inside the 30 s staleTime showed the
reorder undone, and the next drag from there persisted that — silently undoing
an order the server had taken. Measured in jsdom: server 2,1,3, cache 1,2,3, no
refetch.

1 test, mutation-checked.
2026-09-12 05:50:40 +02:00
peter a0114deff1 fix(api,ui): a refused write says why, wherever the refusal happens
Round 5 taught `notifyActionError` to discard a plain-string `detail` in favour
of the caller's translated fallback, because `req` leaves 403/404/429 to the
caller and those routes hand-write English. Three places did not get the
message.

**The demo account lost its explanation.** `require_human` refused with a plain
string, so from that commit on the shared demo account got its caller's generic
"couldn't do that" and no hint that the demo account was the reason — and the
gate is not backed by a UI gate everywhere: the comfort shelf has no `is_demo`
check, so the pin button is live and simply fails. A structured `{code: "demo"}`
and an `errors.demo` string in both locales. The two auth routes that hand-roll
the same check were swept with it.

**`ChatThread` carried a second copy of the contract.** It filtered 404/429
itself and then did `e.detail || t(…)`, the pre-fix order, so the central fix
could not reach it and "That member hasn't set up messaging yet." stayed English
under a Hungarian UI. It goes through the shared helper now, and a contract test
pins the remaining raw-`detail` consumers as an explicit SET — never a count —
so a new copy fails the gate instead of shipping. The five on that list are
inline surfaces (a form field's error line, a setup banner) and are deferred
with the ~60 backend routes whose details they print.

**The Plex "Add to playlist" dialog was still silent.** Same shape as the
collection editor round 5 fixed and the YouTube twin that already had the fix:
no catch, so a playlist deleted on another device answered a structured 404 that
nothing reported — the tick never flipped, the gone row stayed, and the floating
promise became an unhandled rejection.

Also: the `ABORTED` sentinel's VALUE was asserted nowhere. The consumer test
hand-built the error it expected, so replacing `xhr.onabort`'s rejection with a
friendlier message left the lane green — and every upload the user cancels would
then raise a red "the upload failed" toast, which is what the sentinel prevents.
Both producer tests assert through `isAborted` now.

And two backend tests were pinning fewer sites than their own docstrings claimed:
the channel-404 loop said "all five" and issued three requests (explore and
subscribe unexercised, the excuse for subscribe naming a different call site
with a different reason slug), and the recipient/job test named three refusals
and requested two — `share_download`'s `not_found("recipient")` had no test in
the suite at all, nor did the locale keys added beside it. Every site named now
has a request behind it.

6 tests, mutation-checked against 7 mutations: the demo detail at both the gate
and an auth route, a re-hand-rolled ChatThread, the silent Plex dialog, the
abort sentinel, and each unpinned 404 — all red on the test that names them.
2026-09-12 05:50:31 +02:00
peter d89d887a9b fix(plex): the reclaim budget bounds what it says it bounds, and a refused finish is not performed anyway
`MAX_SESSION_RECLAIMS` was cleared by the first fragment of any session, and a
reclaimed session essentially always delivers one — `start_session` does not
return until its first segment exists. So the budget only ever bit where a
session parsed its manifest and then died before a single fragment, which is not
either case its own comment names. Measured: 40 loss cycles produced 41 sessions
and never an error. A transcode falling behind realtime, or two viewers of one
rating_key taking turns, respawned ffmpeg indefinitely — the round-5 behaviour,
still there behind the guard added to stop it.

It counts within a rolling window now. Two losses inside two minutes is a
session that keeps being lost; one an hour apart is not. And the window is reset
per item, because the player stays mounted across a queue advance — otherwise a
budget spent on one episode would deny the next its first legitimate reclaim.

Separately: `finish`'s rollback re-performed the write it was taking back. On
the play-to-the-end path the position it restores IS the end of the media, and
the backend reads a position within its finish margin as a finish — so the
branch named "take the finish back" set watched, cleared the position and
scrobbled to Plex through a second route, while the caller's error toast was
already on screen saying the mark had failed. There is no resume point at the
end of the media to restore; the cache drop is the whole of the rollback there.
The credits-skip caller captures a mid-episode position and still restores it,
which is the case that needs it.

7 tests, mutation-checked: the FRAG_LOADED reset, the missing window, and the
unguarded rollback each red on the test that names them.
2026-09-12 05:50:11 +02:00
peter f8a31d48cf fix(ui): a dialog keeps the focus it was given, and hands it back where it took it from
Round 5 extracted `ui/dialogFocus` so no shell would hand-roll the focus
contract, then introduced four ways to lose the focus anyway — all four measured
in jsdom against the real components.

**Ownership was narrower than the trap.** `ui/Modal` traps Tab over the whole
dialog on purpose, so a toast's Undo or ✕ stays reachable; ownership was tracked
over the panel alone. Tabbing to that ✕ therefore read as "the user left the
dialog", and since removing a toast fires no `focusin` to re-arm, closing
afterwards skipped the restore and left the focus on <body>. Before this hook
existed the restore was unconditional, so it was a regression.

**A dialog stacked above us is still ours.** A nested `ui/Modal` portals into its
own host, so it is outside the outer dialog's root however wide that scope is
drawn — widening alone could not fix it. Both closing in one commit left the
focus on <body>: the outer skipped its restore, and the child's own target (a
control inside the outer panel) was already detached, because passive cleanups
run parent-first. Ownership now means "inside my dialog, or inside a modal layer
above it", which is what the rule always intended.

**The wizard disagreed with `ui/Modal` on its first line.** `trapTab(e, panel)`,
with neither the dialog-root scope (so its own toast host was outside the cycle
— the connection notice never auto-dismisses, and its ✕ was unreachable from the
keyboard) nor the `container` argument the JSDoc exists for: `panel.contains(
panel)` is true, so focus sitting ON the tabIndex=-1 panel matched no branch and
a Shift+Tab walked into the app behind the scrim. Both call sites now pass the
same three arguments.

**The toast handover was a render-phase read-modify-write.** A `useMemo` read a
module global that the next line overwrote; React re-invokes a component body
and every `useMemo` factory in it on a StrictMode double render, and the LAST
invocation commits — so the second pass re-read what the first had just written,
the takeover host classified its OWN new toast as inherited, and the dev build
behaved exactly like the code round 5 replaced. Snapshotted through a ref keyed
on `isTop`, so a repeated invocation of one render is a no-op.

And the fourth `aria-modal` shell, `Welcome`'s screenshot lightbox, is finally
swept into the same contract: it declared itself a modal dialog while leaving
the focus on the thumbnail behind an 80%-black scrim and letting Tab walk the
landing page underneath.

The headline fix of the previous commit — StrictMode's re-mount taking the focus
off an `autoFocus` field — had no test at all; deleting it left all 1009 green.
It has one now.

9 tests, mutation-checked against 8 mutations, each red on the test that names it.
2026-09-12 05:49:58 +02:00
peter 7f17b17c13 fix(plex): an HLS session's file names are one vocabulary, and a name it cannot produce fails at once
Round 5 added a guard so a request for a segment beyond the produced region
answers instead of polling out the full timeout — the route is a SYNC one, so
every such wait parks a threadpool worker and, through `current_user`, a pooled
DB connection. Measured against real ffmpeg, the guard could not fire where it
mattered.

`-hls_segment_filename seg_%v_%d.ts` substitutes the VARIANT first: a dual-audio
session writes seg_0_0.ts (video) and seg_a0_0.ts / seg_a1_0.ts (the `name:aN`
renditions). The regex read those as sequence-first, so every video segment
parsed as segment 0 and every audio segment did not parse at all — the guard was
inert on every multi-audio item, which is most of this library. `_seg()` carried
the same inversion: the warm-up waited 8 s for `seg_1_0.ts`, a name ffmpeg never
writes, on every multi-audio session start AND every seek-restart.

The layout is now one constant plus one parser (`segment_name` /
`_SEG_NAME_RE`), with a test pinning them to each other, instead of three
comments that could drift — and they had drifted, all three ways.

The guard is also a positive rule now. It was a blocklist, so anything that was
not a single-audio segment name was waved through: `GET …/anything.m3u8` still
parked a worker for 30 s, while the route's comment claimed a flood of hopeless
names was what it prevented. A session produces its playlists and its segments
and nothing else. `.vtt` leaves `_HLS_CT` with it — ffmpeg runs with `-sn` and
subtitles are a separate route, so that extension could only ever name a file
that will not arrive.

Also here:
- A 5000-digit segment index reached the client as a 500: `int()` on an
  unbounded digit run raises past 4300 digits and only OSError was caught. The
  name is bounded on the route and the index in the regex — the same bug class
  the subtitle routes were bounded for.
- A session torn down between the wait and the read is a 404, not a 500. `_kill`
  rmtrees the directory and no lock spans those two steps; the player's recovery
  is keyed on the STATUS, so a 500 skipped the reclaim and ended playback with
  "couldn't play" instead of restarting at the playhead. The trigger is ordinary
  — `loadSession` awaits the new session before destroying the old hls instance.
- Two comments claimed an open player checkpoints every 10 s "whether or not it
  is playing", which is the stated reason the 30 s finished-idle reap is safe.
  `flush` sends nothing once the finish latch is up, or before a position is
  known. The states it does cover — playing or paused mid-item — are the ones
  the rule needs; the comments now say that instead of more.
- The route's tests leave the DB lane: the route reads nothing from the database
  since the auto-start was removed, and there those four regressions SKIP on a
  machine with no Postgres while pytest prints green.

14 tests, mutation-checked: the old parse, the old warm-up name, the blocklist,
both bounds off the absurd name, the unguarded read, and the route dropping
`proc` — each red on the test that names it.
2026-09-12 05:49:38 +02:00
peter 1f81684b6e fix(ui,plex): a dialog keeps the focus its child asked for, and a hopeless segment fails at once
Both found by UAT in a real browser, after the tests for them were already green — which is the
whole reason this repo settles focus and timing claims by running them.

- **The autoFocus fix worked, and then the dialog took the focus back.** React 18's StrictMode runs
  every effect mount→cleanup→mount in development: the cleanup handed focus back to the trigger, and
  the second mount, finding the focus outside the dialog, moved it to the first focusable — off the
  `autoFocus` field a beat after the browser had put it there. Chrome measured it: INPUT (commitMount)
  → BODY (cleanup) → ✕ (re-mount). The effect is idempotent now — it remembers what the dialog was
  focused on and restores THAT on a re-mount — and the close-time restore is conditional on the
  dialog still holding the focus, which it tracks as it changes rather than computing at cleanup,
  where the panel is already out of the document. A dialog no longer yanks focus back from wherever
  the user has since moved it.

- **Passing ffmpeg's handle to the segment wait only shortened the FINISHED case.** Measured against
  the running localdev API: a request for `seg_99999.ts` on a live session still took 20 195 ms,
  because that ffmpeg has not exited and the wait has no other reason to stop. ffmpeg writes segments
  in order, so a segment whose predecessor does not exist is not the next one and cannot arrive —
  and that is knowable at once. Same request now: 45 ms.

Verified on localdev against the real library (Westworld S01E06, remux, multi-audio): Mod+K opens the
command palette with the focus in its search field and typing reaches it; playback runs on the new
session-scoped HLS URL; a request naming a stale session id is a 404; and a viewer paused for longer
than the finished-session rule keeps their 2.4 GB session across two reaper passes — the progress
checkpoint touches it — and resumes cleanly.

vitest 1009, pytest 1127.
2026-09-12 04:31:51 +02:00
peter 33fddacee2 test(plex): the over-long-name guard is asserted the same way on every OS
Deep-review round 5. The assertion behind `subtitles._exists` was only red where pathlib re-raises
ENAMETOOLONG: on Linux it does, on Windows the same stat fails with ERROR_INVALID_NAME, which
pathlib's `exists()` already swallows — so the assertion passed with the fix reverted. It forces the
errno now instead of relying on the filesystem. (The gate's pytest lane is the Linux container, so
the branch was never actually unguarded; the test was just dishonest about why it passed.)
2026-09-12 04:18:35 +02:00
peter 12121255dd fix(playlists): a rejected reorder goes back to the order the server accepted
Deep-review round 5, over round 4's own fixes.

- Round 4 made a failed reorder roll the rows back to the CACHED detail, because a resync cannot do
  it (the server's list is unchanged, so the refetch is deep-equal, react-query keeps the same
  `data` object and the effect keyed on it never runs). But nothing wrote an ACCEPTED order into
  that cache — only the refetch that follows a successful reorder did. So a second reorder rejected
  while that refetch was still in flight rolled the rows back past the first one, which the server
  had accepted; the membership guard cannot correct an order (its key is the sorted id set), so the
  wrong order stayed on screen for good and the next drag persisted it. `onApply` now writes the
  accepted order into the cache before invalidating.

- `PlexPlaylistView.rename` had a `catch` no test could reach, and `commit` (reorder) was not gated
  by `busy` the way its siblings are — a reorder in flight while a removal completes is what makes a
  rollback ambiguous in the first place. The rename failure path is now covered; `commit` defers.

- `putBack` read `q.data` from the render that bound the failing handler instead of the cache at
  rollback time, unlike the sibling `restoreFromCache`. Aligned. Stated plainly: I could NOT drive
  that staleness in jsdom — a mutation that restores the old shape leaves every candidate test green
  — so this one ships as an alignment with the proven sibling, not as a fix with a red test behind
  it. The scenario and the harness limit are on ROADMAP.md.

Two regression tests, both mutation-checked red (the cache write, the rename catch). The
drag-reorder failure path still has no test: dnd-kit's pointer and keyboard sensors do not activate
under jsdom even with geometry and scrollIntoView stubbed, so a test there would have pinned the
harness, not the code. Logged, not faked.

vitest 1009.
2026-09-12 04:15:15 +02:00
peter 4a7b65d187 fix(api): a refused write is named, in the reader's language
Deep-review round 5, over round 4's own fixes.

- `notifyActionError` preferred the server's `detail` over the caller's translated fallback, and
  `reportFailure` deliberately leaves 403/404/429 to the caller — so for exactly those statuses the
  dozen call sites round 4 converted started printing whatever English a route hand-wrote into an
  otherwise Hungarian UI ("No user with that email.", "Not subscribed to this channel", "Unknown
  job"). A STRUCTURED detail is still preferred: `req` has already run it through `localizeDetail`,
  so it is both localized and more specific. A plain string is not.

- The other half of the same defect, on the backend: round 4's `get_or_404(what="channel")` sweep
  reached exactly ONE of the five identical sites in channels.py and shipped the HU/EN string with
  it, so the Block button said it in Hungarian and the four controls beside it in English. All five
  are converted, and the subscription / share-recipient / scheduler-job 404s with them (two new
  reasons in both locales).

- `PlexCollectionEditor`'s add/remove, take-over and delete were try/finally with no catch — in the
  very file round 4 edited to give `createNew` one. 403 and 404 are the caller's to report, so the
  tick simply never appeared and a CONFIRMED "delete collection" did nothing, silently.

- `UploadDialog` is the only `uploadReq` caller and kept an empty catch whose comment still claimed
  the global error modal had it. It had not since `reportFailure` became shared: a proxy 502 raised
  only the self-clearing connection notice and a 413/403/404 nothing at all, so the dialog snapped
  back to its pre-upload state with no error named. The user's own Cancel stays silent — and since
  a cancel and a network drop both carry status 0, that difference is now a named `isAborted`
  rather than a string literal at each end.

Regression tests for each (5 vitest, 2 pytest, plus two existing assertions corrected), all
mutation-checked: 6 mutations, each red on the assertion it is named after. vitest 1007,
pytest 1127.
2026-09-12 04:07:25 +02:00
peter 47af4d5c05 fix(plex): an HLS artifact names the session it belongs to, and a paused viewer says so
Deep-review round 5, over round 4's own fixes.

- Sessions are keyed by rating_key alone while a session's DIRECTORY is per (key, offset, tag), and
  the artifact URL carried no session identity — so once a seek-restart or a second viewer of the
  same item took the registry slot, the old viewer's request for `seg_12.ts` was answered 200 with
  the new session's twelfth segment: bytes from a different `-ss` offset under `-copyts`, on a
  timeline the client is not on. Wrong content with a success status is the one failure a player
  cannot notice, and round 4's reclaim-on-404 could never fire on it. The id is in the path now (so
  the playlist's relative names resolve back into the same session), a mismatch is a 404, and the
  player reclaims at the playhead. The entry playlist's auto-start is gone with it: it could only
  ever begin at 0, silently discarding the viewer's position.

- Round 4 exempted paused viewers from the reaper by keeping EVERY finished session for the full
  ten-minute idle window — one whole segmented copy of a film per item, on the volume the download
  center shares. The signal it needed already existed: an open player checkpoints its position every
  10 s whether or not it is playing, so that route now touches the session and the periodic pass is
  back on the short rule.

- POST /session's opportunistic reap was never scoped to the key being started: it walked the whole
  registry with the 30 s finished rule, so pressing play on ANY item — even a direct-play one, the
  call ran before that early return — killed another viewer's paused session. It is gone; the
  periodic pass owns reaping.

- A segment that can never arrive no longer polls out the full 20 s: the producer's handle is passed
  to `wait_for`, so a finished ffmpeg answers at once. This is a sync route, so each such wait parked
  a threadpool worker AND a pooled DB connection, and an ordinary seek past the produced region was
  enough to do it.

- The DataError handler's re-raise branch — the half that keeps a genuine DataError a 500 — is
  pinned, so simplifying it to an unconditional 422 cannot pass in silence.

Six regression tests (two route-level, in the DB lane), each mutation-checked red on the assertion
it is named after. pytest 1124, vitest 999.
2026-09-12 03:57:52 +02:00
peter 8a83bc0e9f fix(plex): a finished item is finished when the server says so, and a lost session is reclaimed a bounded number of times
Deep-review round 5, over round 4's own fixes.

- Round 4 added a `finish` latch so a played-through item records no checkpoint at the credits.
  On a NATURAL play-through it was raised a beat too late: per the HTML spec the browser sets
  `paused` and fires 'pause' BEFORE 'ended', so the pause checkpoint went out while the latch was
  still down and raced the watched mark — both requests read "not watched yet" from their own
  snapshot, so Plex counted one watch twice, and where Plex's reported duration runs more than the
  finish margin past the real media the checkpoint wrote a resume position AFTER the scrobble. The
  pause that ends the media no longer flushes. The regression test that was meant to cover this
  modelled the end as a bare 'ended' event — a browser that does not exist — and is now the real
  order.

- The watched mark was fired and forgotten (`.catch(() => {})`) while the latch patched the cached
  detail to watched / position 0 regardless, and round 4's once-per-id start guard meant the
  corrected detail arriving later was never used: an episode whose mark failed reopened at 0:00
  with a Watched badge the server disagreed with. `finish` now takes the in-flight write and takes
  itself back if the server refuses it — the real position is reported, the lying cache entry is
  dropped, and the failure is said out loud instead of swallowed. Both callers (the natural end
  and the credits auto-skip) hand it over.

- The reclaim of a 404'd HLS session had no bound. `parsed` cannot provide one — it belongs to one
  hls instance and every restart builds a fresh one — and the server answers 404 both for "the
  session is gone" and for "that segment is not there yet", so a transcode falling behind realtime,
  or two viewers of one item taking turns, respawned ffmpeg forever behind a spinner that never
  became an error. Two consecutive reclaims, reset by the first fragment that actually loads, then
  the honest failure.

Five regression tests, each mutation-checked red on the assertion it is named after.
2026-09-12 03:47:42 +02:00
peter 41b652f8fd fix(ui): a dialog opens focused where it said it would, and the wizard is a dialog
Deep-review round 5, over round 4's own fixes.

- Round 4 gave `ui/Modal` a persistent portal host so a fullscreen transition would stop tearing
  the dialog down. The host was attached in a `useLayoutEffect` — one commit phase AFTER React
  runs `autoFocus` in `commitMount`, and `focus()` on a disconnected node is a no-op. Every
  autoFocus in every dialog was dead: Mod+K opened the command palette with the focus on the ✕,
  so typing searched nothing and Enter closed it; the same for the download rename field and
  every non-danger confirm. The host is attached in a `useInsertionEffect` now, a whole phase
  earlier, and the whole re-parent — including the focus save/restore across the move, which
  `appendChild` needs because it is remove-then-insert — lives in one `useFullscreenPortalHost`.
  `ui/AnchoredPopover` had a second copy of that re-parent WITHOUT the focus restore, and a
  comment claiming focus survived it; both now come from the hook.

- `ui/Toaster` classified a toast raised in the SAME commit as a host takeover among the ones it
  inherits, because it read the store live at the takeover render. A dialog that notifies and
  closes in one handler (UploadDialog) therefore put its toast up with no role, into a region
  still `aria-live="off"` — announced by neither path. "Inherited" is now what the outgoing host
  last had on screen, and a toast that is NOT inherited is announced even inside the unarmed
  window.

- The onboarding wizard renders its own shell, and Escape ordering was all of the dialog contract
  it had: no role, no label, no initial focus, no trap, no restore. A keyboard user tabbed out of
  the first-login wizard into the app behind the scrim. The focus half of the contract is now one
  shared `ui/dialogFocus` that both `ui/Modal` and the wizard use.

Every fix has a regression test, each mutation-checked (6 mutations, all red on the assertion
they are named after).
2026-09-12 03:41:37 +02:00
peter 545996f807 fix(ui): a toast host ranks by when its overlay opened, and an overlay keeps its Escape, its dialog and its navigation straight
Deep-review round 4, the overlays.

- Toast hosts were ranked by mount time, and PlayerModal swaps its host on every fullscreen change:
  leaving true fullscreen with the download dialog open put the player's host back on top, so toasts
  were drawn under that dialog's scrim and a click on one closed the dialog. Each overlay takes a
  rank once, when it opens (`useOverlayOrder`), and hands it to every host it renders.
- A dialog open while fullscreen is left keeps a stable portal container (moved, not re-created):
  React tore the portal down when `fsRoot` changed, resetting a half-filled form and dropping focus
  to <body>. Focus is put back after the move.
- The onboarding wizard is a layer, not a raw document listener: one Escape used to dismiss the
  error dialog over it AND end onboarding, skipping the write-scope step.
- `navigateTo` (a toast's "View", clicked inside the player's own host) goes over the overlays it
  unmounts, so the player's Back-to-close cleanup no longer pops the page it just opened — the same
  fix Shift+W already had. The access-requests toast action too.
- The live region arms itself a task after it takes over inherited toasts: React flushes effects in
  the same task as that render, so the "silent inheritance" arming was a no-op.
- The overlay-host guard matches class tokens in any order (PlexPlayer's root was never checked),
  globs `ui/` too (ui/Modal was not checked either), and requires the rank.
2026-09-11 23:43:23 +02:00
peter 267fd7f70f fix(playlists): a refused edit is taken back on screen, and a gone list or tag is said, not swallowed
Deep-review round 4, the playlist surfaces.

- A failed move or reorder relied on the resync to put the rows back — but the server's list is
  unchanged, so the refetch comes back deep-equal, react-query keeps the same `data` object, and the
  re-seed effect keyed on it never ran: the moved row stayed gone, the rejected order stayed on
  screen. Both restore from the cached detail now.
- A failed removal puts its row back itself: when the server is unreachable the resync fails too,
  and round 3's "keep the list on a failed refetch" guard then kept showing it as removed.
- The Plex playlist page (remove / rename / delete / reorder), the add-to-playlist popover and the
  tag manager had no failure handling at all: a list or tag deleted elsewhere (a 404, now a
  structured `not_found` the request layer leaves to the caller) failed in silence and an
  optimistic removal stayed removed. Each says it and resyncs; deleting a Plex playlist that is
  already gone just leaves the page.
- Playlists' own loading/error branches use the shared `LoadingState` / `StateMessage` (the loading
  line had no live region).
- The "player stays mounted" tests count mounts: a remount ended with the same flag as staying up.
2026-09-11 23:34:33 +02:00
peter 6d49da17fe fix(api): a write that hit a connection failure is still reported, and every failure is reported once
Deep-review round 4, the "one report per failure" contract round 3 introduced.

- `req` marked gateway (502/503/504) and network failures "already reported" because they raise
  the connection notice. But that notice is about connectivity and removes itself on the very next
  answer — which every caller's own resync refetch fetches at once — so a delete that hit a
  keepalive reset flashed the notice and was then reported nowhere, the list still there. Only the
  durable reports (the error dialog, the 401 sign-out) count now; the caller says the action failed.
- A connection notice the user closed swallowed every later one for the rest of the visit; it is
  raised again.
- `req` and `uploadReq` share one failure policy (`reportFailure`). The upload copy had drifted: a
  502 opened the blocking "Server error (502)" dialog and cleared a notice that was still true.
- A taken Plex collection name is a structured 409, localized, and the create/rename calls are
  quiet: it was an English dialog plus a Hungarian toast. `notifyCollectionEditError` is gone.
- The mutation onErrors that toasted unconditionally after a non-quiet request (invite approve /
  deny / add, demo whitelist, audit clear, scheduler, share, full history, edit, config tests, tag
  attach) go through `notifyActionError`; the playlist delete's hand-rolled copy of it too (it takes
  a level now). The user role / suspend / delete actions had no onError at all: a user deleted in
  another tab failed silently and stayed listed — now said, and the list refetched.
2026-09-11 23:27:48 +02:00
peter 6a92585098 fix(plex): the player restarts a reaped session, finishes a played-through item as watched, and starts each item once
Deep-review round 4, the Plex player.

- A 404 on an HLS session that WAS playing means the server reaped it (a long pause, a deploy);
  hls.js never retries a 4xx, so playback ended with "couldn't play". A new session at the playhead
  now takes its place (the frontend half of the paused-viewer reap fix). A session that 404s before
  its manifest is still a plain failure, so it cannot restart itself in a loop.
- Leaving an item that ended, or that the credits auto-skip left, sent a final checkpoint at the
  credits (racing the watched mark, and pushing a resume point to Plex after the scrobble) and
  patched "new, at 49:55" into the cache — Prev reopened it at its last seconds. `finish` settles it
  as watched at 0, said before the pause whose own checkpoint could land first.
- An audio switch, an A/V offset change and every seek control read `abs`, which is 0 until
  playback ticks: a resume at 20:00 restarted from 0:00. They use the shared `playhead`.
- A seek while a restart is pending always restarts (the old media's `seekable` described the old
  session, and the pending answer overrode the newer seek).
- A superseded direct-play session's loadedmetadata/canplay are generation-gated and detached (they
  seeked the next item to the previous one's resume point).
- The session starts once per item: a reconnect refetch, the info panel's watched toggle or a
  language switch no longer restarts the film at the last checkpoint.
- Tests for the session generation's failure paths and the item-change bump, which had none.
2026-09-11 23:18:33 +02:00
peter 31dc4184f5 fix(api): a paused viewer keeps its HLS session, a refused subtitle publish backs off, and a NUL is a 422 everywhere
Deep-review round 4, backend half.

- The periodic HLS reaper (round 3) applied the "finished and unfetched for 30 s" rule that only a
  NEW session request may apply. A remux finishes long before the film, and a paused player with a
  full buffer fetches nothing, so a one-minute pause killed the session and playback died ~30 s after
  resuming. The periodic pass is `reap_abandoned` now (the long idle limit, finished or not).
- A subtitle `publish` that could not keep its output (full / read-only cache volume) returned
  quietly, so every poll queued the window and the whole-container pass again. It reports failure
  and the pass is backed off like a failed read; a cached VTT that cannot be READ is answered
  `unavailable` (`cache_unreadable`) instead of scheduling reads that can never land.
- The subtitle routes bound `rating_key` / `sub_ord` (they become a cache file name: a 250-digit
  one was ENAMETOOLONG, which pathlib's exists() re-raises -> a 500), and the cache helpers no longer
  throw on such a name.
- One app-wide handler turns psycopg's "text fields cannot contain NUL" DataError into a 422: round 3
  guarded only the by-rating_key lookup, and `contains=`, `contains_group=`, reorder/bulk key lists
  and the collection search still 500'd.
- The hand-written `db.get(...) is None -> 404 "Unknown ..."` siblings use `get_or_404` with a
  `what` (channel / video / user), so the Hungarian UI no longer shows English for a gone row.
2026-09-11 23:08:11 +02:00
peter 86971c574d fix(api): progress checkpoints past the int4 column are a 422, and idle HLS sessions are reaped on a timer
Deep-review round 3 of bug hunt 2026-09.

- The YouTube and Plex progress bodies are bounded by POSITION_MAX_S (the int4 position columns): a
  finite position of 2^31 or more was a DataError, i.e. a 500 — the sibling of round 2's non-finite
  fix.
- plex/stream.py promised "a periodic reaper", but reap_idle ran only inside the NEXT session
  request: after the last playback an abandoned session (its ffmpeg still working, its directory a
  whole segmented copy of the film) lived until someone pressed play again. A lifespan task reaps
  every 60 s.
- Tests for what round 2 left untested: the 422's body keeps the list shape the UI localizes, and
  stream_session refuses non-finite start/aoff.
2026-09-11 18:05:30 +02:00
peter 785080f3da fix(plex): a Surprise card whose watched toggle failed is put back
Deep-review round 3 of bug hunt 2026-09 — the sibling of the collection page's fix (d349bc90).
Discover's toggle flips the Surprise draw too, but its resync refetches only the rails (refetching the
draw would reshuffle it under the viewer), so a refused toggle stayed flipped there. toggleWatchState
takes an optional `revert`, called on a failure only; the queryKeys comment that claimed the watched
toggle invalidates the Surprise key now says what the code does.
2026-09-11 18:05:30 +02:00
peter 76b5f7c93e fix(playlists): a failed background refetch keeps the list on screen, and the player outlives every branch
Deep-review round 3 of bug hunt 2026-09.

- The detail's error branch applies only when there is nothing to show (`isError && <no data>`, the
  QueryState convention): one failed refetch — on focus, after a move or a removal — replaced a
  loaded list with "couldn't load". The error line is announced (role=alert).
- Tests for the two branches of "the player survives a change of selection" that round 2 left
  untested (a list that fails to load, no selection).
2026-09-11 18:05:19 +02:00
peter 1cfe6a7f59 fix(api): a gone row and a refused playlist action are said in the reader's language, and a failure is reported once
Deep-review round 3 of bug hunt 2026-09.

- get_or_404 / owned_or_404 answer a structured {code: "not_found", reason: <what>}
  (errors.notFound.* in HU+EN, generic fallback). The structured-404 fix of round 2 had gone into the
  ad-hoc `_own_playlist` (CLAUDE.md's "Not" pattern) while the shared helpers stayed string-only, so
  views, tags, pins, Plex playlists, invites and users still 404'd in English. `_own_playlist` now
  goes through owned_or_404; admin's hand-rolled user lookups use get_or_404.
- A move into a list that is gone names the TARGET (reason "move_target") — "this playlist doesn't
  exist any more" was read as the list on screen, which still exists.
- Every refusal in the playlists router is a {code: "playlist", reason} (playlists.errors.* HU+EN)
  instead of a bare English string shown verbatim in the Hungarian UI.
- `req` / `uploadReq` mark the errors they already put on screen (the error dialog, the connection
  notice, the 401 sign-out), and notifyActionError skips those. The watched toggles, the saved views
  and a playlist delete toasted a fault the dialog was already showing; Playlists' two hand-written
  status filters now go through the same check.
2026-09-11 18:05:19 +02:00
peter 33fe895539 fix(ui): every full-screen overlay hosts its own toasts, above the players' controls, without re-announcing them
Deep-review round 3 of bug hunt 2026-09 (the round-2 overlay toast host, finished).

- Inside an overlay the toast stack sits above the bottom band (bottom-24, the UpNextCard rung) and
  lets clicks between cards through: at the app's own spot it covered the Plex transport row and
  PlayerModal's docked bar and took their clicks while a toast was up (measured: play/pause, ±10,
  stop, prev/next, mute all hit the toast).
- ui/Modal and the onboarding wizard mount an overlay host too, so a toast raised under any dialog
  is readable and a click on it no longer closes the dialog; the Modal's Tab trap spans the whole
  dialog, so a toast's action and dismiss are keyboard-reachable. A test now enforces the
  convention: a component that covers the viewport at z-overlay must mount <Toaster overlay />.
- PlayerModal's host moves into the stage in true fullscreen (outside the element holding the lock it
  was painted under it).
- A host that takes over shows the toasts it inherits silently (no second role=alert, its region
  aria-live=off until they are in), and each countdown bar starts as far in as its toast already is
  instead of restarting while the real dismiss timer does not.
- The overlay-host registry uses lib/store instead of a hand-rolled listener triad.
2026-09-11 18:04:54 +02:00
peter 5c5cabbe5a fix(plex): the player drops a session answer that lands late, and Next neither loses nor lends a position
Deep-review round 3 of bug hunt 2026-09.

- loadSession checks, after its await, that it is still the newest call and the player is still
  mounted. A late answer used to replace what was playing (Next twice quickly: the slower transcode
  landed last and played the middle item under the last one's title, recording its progress there)
  or attach a stream to the detached <video> after Back, never destroyed.
- The <video>'s clock is trusted only once the new session has answered. go()'s pause() fires a
  timeupdate AFTER the reset to 0 (measured in Chrome), so the previous item's position was sent as
  the next item's subtitle position and saved as its progress (final, pushed to Plex).
- Next settles the item being left (final checkpoint + reopen-cache patch) before switching: the
  progress cleanup ran with the position already reset and skipped, so every Next dropped the last
  stretch watched.
- A seek-restart puts its target into the position state too; the ref alone was re-synced back to
  the pre-seek spot by the restart's own render.
- The unmount teardown captures the <video> at mount: React detaches refs before an unmount's
  cleanups, so the stop/detach never ran.
- The tracks panel exposes the selected audio/subtitle track via aria-pressed.

Verified in real Chrome on localdev (Westworld S1E6 → Next): the item left was settled at 783 s, the
next asked for subtitles at its own start (0), and no progress was written for it.
2026-09-11 18:04:41 +02:00
peter 4f37c41021 fix(plex): a subtitle failure about one position no longer ends the track, and windows are read ahead and never wait behind a full pass
Deep-review round 3 of bug hunt 2026-09.

- `failed` is now final only for a track that can never be text (bitmap, no item/track/file). A Plex
  hiccup or a stretch ffmpeg could not read answers `unavailable`: said, polled again every 5 s, and
  the <track> stays mounted when cues exist elsewhere (gen > 0). Re-picking the selected track asks
  the server again at once.
- Within LOOKAHEAD_S of a covered stretch's end the next window is read ahead, instead of only once
  the playhead has fallen off it (subtitles dropped out at every edge while the full pass was backed
  off or slower than a window lasts).
- Windows and full passes run on separate pools with separate in-flight caps, so a window never
  queues behind (or is refused by) minute-long full passes — the pool comment promised that and
  nothing enforced it.
- A pass that times out is TERMinated and keeps what it reached (published as partial) instead of
  being killed and discarded.
- Lapsed failure memos are swept on every write (they were only pruned when the same key was read
  again, so polling fresh keys grew the process without bound).
- `publish` no longer merges into a sidecar with no VTT behind it, or claims a region whose VTT write
  failed, or overwrites an unreadable track with one window.
- A NUL in a rating_key names no row (it was a DataError, a 500, on every Plex route) and no longer
  500s the status poll.

Tests: 16 new, each mutation-checked (18 mutations, all red on the test that names them).
2026-09-11 18:04:30 +02:00
peter f7faa19ee2 fix(plex): the player starts, and resumes after a session-restart seek, when React mounts it twice
The player's "still alive" flag — every play() is gated on it, so a late one cannot start a detached
<video> after Back — was only ever LOWERED, in an effect's cleanup. React may run a cleanup and mount
again (StrictMode does exactly that in development), and then the flag stayed false for the life of
the player: it never started by itself, and a seek back past the loaded region, which restarts the
session, left it paused. Found on localdev (Westworld S01E06: playing → seek back → the picture
settles, paused). Production does not double-mount, which is why only dev showed it — and why
several earlier localdev measurements "stayed paused".

The effect now raises the flag in its setup. A new test renders the real player under StrictMode and
checks it plays once the media is ready; red without the fix. Verified on localdev: it starts by
itself, and keeps playing through a backward session-restart seek.
2026-09-11 03:47:36 +02:00
peter 0b4b268678 test(feed): pin the sort options' default directions as literals
The feedSortOptions test compared each option's default direction against SORT_DEFAULT_DIR — the
implementation's own table, so a wrong entry passed — and that import was the export's only user
outside its module, which kept knip from seeing it had become dead after F3. The test now states the
directions, the table is private again.
2026-09-11 03:19:17 +02:00
peter d349bc9084 fix(plex): a collection's failed watched toggle is put back the way it was
C1 moved the optimistic flip onto the key the page reads — and so made visible that nothing corrects
it: the resync only invalidated `plex-library`, so a failed toggle stayed "watched" beside its own
error toast, and the collection's other cached sort orders stayed stale after a good one. The
resync now takes this page's `detail` prefix too, as the Discover and person pages do for theirs.
The C1 test holds the request open, so it still pins the optimistic write on its own.
2026-09-11 03:19:17 +02:00
peter dcd8a9566a fix(playlists): the player's Remove follows the list it was launched from, the player survives a change of selection, and a gone list is said in the reader's language
- D3 snapshotted the removal's TARGET, but whether the player offered Remove still followed the
  selected list: launched from Watch later (no Remove, by design), selecting another list under it
  made Remove appear — and delete from Watch later. The snapshot now carries the launch list's
  editability too.
- Selecting another list under an open player (the command palette works over it) put the page into
  "Loading…", and the player was rendered only in the loaded branch — it vanished mid-playback and
  came back at its launch video. It now sits at one place in the tree whatever the page is doing.
- A removal takes the video out of the CACHED list as well as the rows: the player's progress
  checkpoint rewrote that entry before the refetch landed, handing the membership guard a list that
  still held the video — the row came back and the undo history went.
- The watched-state callback refreshed the selected list, not the launch list (D3's sibling).
- `_own_playlist`'s 404 is structured (`{"code": "playlist", "reason": "not_found"}`) and localised:
  the bare "Unknown playlist" reached the Hungarian UI in English, which also made round 1's new
  `removeFailed` string unreachable. Both Remove and Move report through `notifyActionError` now.
- The "viewer moved on under the confirm" test stepped back only once, which lands on the same video
  whether or not the removed one left the queue; it now takes the second step that tells them apart.
2026-09-11 03:19:07 +02:00
peter 6de2cd950d fix(ui): toasts raised while a full-screen player is open show inside it, not under its scrim
The app's Toaster lives in the content column, which is a stacking context (its `@container` implies
layout containment), so both full-screen players — portalled to <body> at the same z-index — painted
over every toast. Round 1's "a failed Remove is reported" fix was therefore still silent inside the
player: its toast sat under an 80% black, blurred scrim (elementFromPoint returned the scrim), and a
click on it closed the player.

An overlay now mounts its own `<Toaster overlay />`, and exactly one host shows the toasts: the most
recently mounted overlay, else the app's. Same component, same look — and inside the aria-modal
dialog, where a screen reader can still reach it. A click on a toast inside an overlay stays there.
Both players get one (the Plex player's also works in native fullscreen, where nothing outside the
fullscreen element is drawn). Verified on localdev: the removal failure reads above the scrim,
bottom-left, and clicking it leaves the player open.
2026-09-11 03:18:55 +02:00
peter 645d14f45d fix(plex): a subtitle track's failed load is heard and retried with a pause, and the poll asks about the right item, position and failure
- The <track>'s `error` never reached `onSubTrackError`: it does not bubble, and React 18 attaches no
  listener of its own to a <track> (measured in Chrome and jsdom — `onError` 0 calls, `onErrorCapture`
  1). Round 1's remount-and-give-up was dead code behind it. The hook now hands the player
  `onErrorCapture`, and the test fires a real non-bubbling `error` at a rendered <track>.
- Once reachable, the retry burned its whole budget within milliseconds of an outage (a remount
  fetches at once) and the give-up outlived turning subtitles off and on. Retries now wait (2 s,
  doubling), running out is reported at once, and picking the track again starts a fresh budget.
- Until the first timeupdate the poll sent `sessionK` — still 0 while the session request was out,
  and the PREVIOUS item's start after a move to the next episode. It now sends where the session was
  asked to start, and the track choice belongs to the item it was made for, so the next item is not
  asked about the old ordinal at the old position.
- A failing poll over a stale `partial` answer was silent: react-query keeps the last good data, the
  track stayed "showable", and the notice only rendered for an unmounted track. The hook now decides
  what the player says (`subNotice`), independently of whether a window is mounted.
2026-09-11 03:18:42 +02:00
peter 7360339039 fix(api): a non-finite number in a JSON body is refused with a 422, not a 500
The JSON parser under FastAPI accepts `Infinity`/`NaN`, and both progress checkpoints truncate the
position with `int()` — a hand-written checkpoint came back as a 500 with a traceback. Their bodies
now refuse non-finite floats, and the app's validation handler — FastAPI's own shape — echoes a
non-finite input by name: the default one failed to serialise its own error (JSON has no NaN), which
was a 500 by another route. Sibling of the subtitle routes' non-finite query parameters.
2026-09-11 03:18:17 +02:00
peter 473d9f02f2 fix(plex): subtitle windows land where the full pass does, and the extractor stops re-reading, respawning and outliving the process
Deep-review round 2 of the 2026-09 bug hunt, over the round-1 fix diff.

- A window was LATE by the distance to the keyframe its input seek landed on, and then doubled for
  good once the full pass merged in: a cue already running at the window's start came out at a
  negative time, and the webvtt muxer's default shifted the whole output so it sat at 0. Measured on
  the real Westworld S01E06 at 25:00: every window cue +5.005 s, 682 cues merged instead of 608.
  `-avoid_negative_ts disabled` keeps the times exact (the parser reads ffmpeg's signed-field
  spelling of a negative time); now 608/608 on that file, and a real-ffmpeg test pins it.
- A full pass that failed was re-armed every minute while the viewer watched what it did publish —
  a whole-container read per minute. It now backs off for an hour; windows keep running meanwhile.
- A failed WINDOW set the track's failure memo, so any uncovered position read "failed" — a final
  answer that ends the player's poll, even with the full pass about to deliver, and one any account
  could plant with an absurd `at`. It now marks only the span it failed to read; "failed" is given
  once the full pass AND a window there have both failed.
- Windows join by what they COVER, not a bucketed identity: a backward seek no longer waits behind a
  window that cannot cover it, and a playhead crossing a bucket edge no longer queues a second read.
  The queue is bounded per track and overall.
- shutdown() sets a flag a worker checks before it spawns ffmpeg (one still in its Plex lookup used to
  start a read after the kill sweep); queued-job cancellation and the lifespan call are now tested.
- A window starts where ffmpeg is told to seek (rounded to the ms it gets), a rounded-up millisecond
  carries into the minute (no more `00:00:60.000`), a key's first publish claims nothing before its
  VTT lands, and `at`/`offset`/`start`/`aoff` refuse non-finite values (`at` also an absurd one).
2026-09-11 03:18:08 +02:00
peter d2e225666b test(playlists): let react-query's notify tick run inside act before the reopen click
The D1 regression test patched the cache and clicked at once; react-query delivers a cache write to
its observers on a setTimeout(0), so under the gate's parallel load the click could beat the page's
re-render and read the old row (748, not 830). Test-only race — a human reopen comes seconds later.
Stable across three full-suite runs; the D1 mutation still turns it red.
2026-09-11 01:45:53 +02:00
peter efd95fca01 fix(ui): the card's channel name is a real link too; call sites the bug hunt left untested now have tests
Deep-review R6 — the small items:

- The video card's channel name was still a <button> (pre-existing): ctrl/middle-click did nothing
  there while the title above it and the player's channel name honoured them. It renders through
  ChannelNameLink now, which grew a forwarded ref (the clamp measurement), pass-through attributes,
  a `tooltip`, and an opt-in `stopPropagation` so no click reaches the card's row.
- ChannelNameLink rendered an empty, still-tabbable <a> for a channel with no title yet (a stub first
  seen through a live search; pre-existing at this site). It falls back to the translated "Channel".
- The feed's SortKey→SortOption mapping existed twice in Feed.tsx; one `feedSortOptions` in
  feedSort.ts serves both modes (DIRECTIONLESS is module-private again).
- The backfill refresh's comment claimed a one-page cost. It refetches every loaded page of the
  grid, and that is the right price: backfilled videos are older, so they land below the head.

Tests for call sites that could be deleted with every lane green: the channel page running the
backfill refresh (knip had been blinded by the hook test's import), Feed handing the grid the live
results in the chosen key AND direction, the card using the link. Plus the alt-click arm of the shared
click predicate, and a plex filter-chip test that claimed it made no request while firing a real one.
Verified on localdev: the feed's channel names are anchors (same 14 px muted look), a plain click
opens the channel in-app. 7 mutations, each red on its assertion.
2026-09-11 01:38:31 +02:00
peter 060deb89d2 fix(playlists): "remove" inside the player targets the list it came from, reports failure, and keeps the cursor and focus honest
Deep-review R5 — five defects in the Remove button f76a2afb put inside the YT player:

- It removed from whichever playlist was selected by the time it was pressed, not the one the player
  was launched from; the selection can change under an open player (the command palette opens over
  it). The launch list is now part of the player snapshot, and the optimistic row removal only
  touches the list actually on screen.
- A 403/404 failed in total silence: the catch claimed the global dialog had reported it, but `req`
  deliberately leaves those to the caller. It now notifies, as moveItemTo already did (the silent
  catch predates this round; the full-screen consumer did not). New key playlists.removeFailed, HU+EN.
- It computed the next item from the cursor as it was when the button was pressed, but prev/next and
  auto-advance keep working under the confirm dialog: the viewer was thrown back onto a video they had
  moved past. It reads the live cursor, and if they moved on it just drops the item from the queue.
- The button was disabled for the confirm's lifetime, so the dialog's focus restore hit a disabled
  element and a keyboard user who cancelled was left on <body>, outside the aria-modal player. A ref
  guards the double press instead.
- Its accessible name was a bare "Remove"; it now names its object, like the row control.

The call-site wiring (step on, drop from the queue, close on the last item, leave everything alone
when the removal did not happen) finally has tests. Verified on localdev in the real YT player: a row
reopened at the 136 s it had just stored; Cancel returned focus to the Remove button; a confirmed
removal stepped 1/3 -> 1/2. 10 mutations, each red on its assertion. Scratch list and positions reset.
2026-09-11 01:28:27 +02:00
peter 4869d6c108 fix(player): the resume position reaches the playlist page and live search, not just the feed
Deep-review R4. 1630a3df wrote the YT player's stored position into the react-query cache so a
reopen could not rewind it — but only the feed actually read it back:

- Playlists renders from its own undoable copy of the list, re-seeded only when the SET of ids
  changes, so the patched position never reached it: close a video at 830 s, reopen it from the
  playlist, and it resumed at the old spot and the next checkpoint wrote that over the 830. A detail
  with the same membership now rebases its fresh rows into every snapshot, keeping order and undo
  history. (The ids-only guard is older than this round; the write that depended on it was new.)
- Live YouTube search results carry the same per-user position and open the same player, but their
  cache was never patched. It is now — patched only, never invalidated, since refetching an
  API-sourced search page spends 100 quota units.

Tests for what had none: a Playlists harness (PlayerModal stubbed to record its launch props) and a
PlayerModal harness with a fake IFrame API that records where each item is started — the latter pins
that stepping back to an item watched earlier in the same session resumes at THIS session's position.
3 mutations, each red on its assertion.
2026-09-11 01:19:14 +02:00
peter d20317e0f1 fix(plex): the collection page's watched toggle and Sort control survive the sort joining the query key
Deep-review R3. When 889d2acf put the sort into the collection detail's query key, two things on
the page did not follow it:

- The watched toggle's optimistic write kept the key's old spelling. setQueryData matches exactly,
  so it landed on an entry nothing reads, and its resync only invalidates the library root — the
  click succeeded server-side and changed nothing on screen (badge, icon, aria-label) until a
  reload; a second click toggled it back. One `detailKey` now serves the read and the write.
- Every sort change was a key with no data yet, so the whole page — hero, grid, and the Sort control
  with its open panel — was swapped for a spinner, and focus fell to <body>: key and direction could
  not be set in one visit. The previous page now stays on screen while the new order loads, for the
  SAME collection only (stepping to another must not show this one's hero meanwhile).

Verified on localdev (Rocky Collection): the toggle flips the poster; IMDb rating then Descending set
in one open panel, the hero never blanked, order reversed. 3 mutations, each red on its test.
2026-09-11 01:12:51 +02:00
peter d2ba65e383 fix(plex): the subtitle poll backs off on errors, a failed <track> is replaced, and a resume asks for the right window
Deep-review R2, plus one defect found while verifying it on localdev.

- The status poll re-armed every second after a FAILED request: `data` stays empty on an error,
  so it read as "still preparing" for as long as the player was open — and the request was not
  quiet, so a persistent 500 re-opened the global error dialog every second. It now backs off to
  10 s (a blip still heals on its own) and is quiet; the player already shows the failure inline.
- A <track> whose own VTT fetch failed never fetched again: with the server's answer unchanged its
  key was unchanged, so React kept the dead element — silently blank subtitles. The key (now built
  in the hook, where it is tested) carries a failed-load count; after two remounts the track is
  reported as unavailable instead.
- Found in UAT: until the first timeupdate `abs` is 0, so after a resume every poll said at=0 and
  the first window was cut around the head of the film (a resume at 20:00 polled at=0 for as long
  as it stayed paused). The poll reports the session's start until playback ticks. And the window's
  dedup identity is the track plus a window-length bucket of its start — keyed on the track alone
  (R1), the right window queued behind that wrong one.

The poll's start/stop rules finally have tests (every earlier one resolved a single fixed answer).
Verified on localdev against Westworld S1E6 resumed at 20:00: the polls carried at=1197, one
subtitle line on screen at 21:02, the poll stopped at `ready`; the real file's window (56 cues) +
full pass (608) merged to 608 — the old merge would have served 664. 11 mutations, all red.
2026-09-11 01:08:27 +02:00
peter cf7ec5e9ec fix(plex): the subtitle extractor's seams — doubled cues, a truncated track cached as whole, a poll that respawned ffmpeg
Deep-review of the bug-hunt round (R1). The background extractor from ed4ab4e1 held, but its seams
did not:

- Window + full pass doubled every cue in the first hour. merge_vtt de-duplicated on the raw timing
  text, and the two passes spell a timestamp differently (shift_vtt writes HH:MM:SS.mmm, ffmpeg's
  webvtt muxer drops the hour below 1 h; shift_vtt(-0.0) never normalised the full pass). Timing is
  now parsed and re-emitted canonically, keyed with the cue text so two cues sharing a time survive.
- A full pass that exited non-zero was published as complete: "ready" forever, subtitles ending
  where ffmpeg died, and no eviction to ever retry. It now covers only up to its last cue, is never
  complete, and records the failure memo.
- The window's dedup identity held the live playhead, so every 1 s poll queued another read of the
  file. It is keyed on the track now, and a queued job re-checks coverage before reading anything.
- A sidecar track waited behind an unrelated embedded full pass (the full job is per item). Its
  window job now fetches it whole — one small request.
- The <track> fetch ran the scheduler at position 0: an unrequested window over the head of the file,
  and a refusal of a window that was on disk. It is a pure read of the cache now.
- publish() is serialised (a lost update became permanent once complete was set); the pool is shut
  down and running ffmpeg children killed at app shutdown (they held interpreter exit for minutes);
  ?at=inf is a 422, not a 500; and both subtitle routes use current_user instead of the cookie-only
  poster gate, which let a suspended or epoch-revoked session keep scheduling reads (pre-existing).

_extract_and_publish finally has tests of its own — every earlier test stubbed it out. 19 mutations,
each red on the assertion that names it.
2026-09-11 00:52:59 +02:00
peter f76a2afbf1 feat(ui): the four small gaps — player delete, repeat-move, a real channel link, YT search sorting
Items 2, 4, 5 and 7 of the bug hunt, one round. They are independent, but they are all the same
kind of thing: an action that exists somewhere else in the app and is missing where you actually
want it.

**5 — the channel name in the YT player is a link again.** It was a `<button>`, so middle-click and
Ctrl/Cmd-click did nothing on a thing that looks exactly like a link (the YouTube icon beside it IS
an anchor and opens a tab). It is now a real `<a>` whose href is the same `?chan=<id>` URL
`openChannel` pushes, with a plain left click still taking the in-app path — that path does more
than navigate, since the player must close and reclaim its own Back-to-close history entry first.
The "is this click ours or the browser's" test is now ONE predicate (`isPlainLeftClick`), which
VideoCard's card link also uses, so a second such link cannot forget the middle-click arm.

**2 — "Remove" from inside the player.** The parent owns the removal (the SAME `removeItem` the row
menu and the inline ✕ use: one confirm, one optimistic update, one undo-stack rebase), and the
player owns what happens next, because it is the only side that knows which item is playing — its
launch queue is frozen at mount. It drops the item, plays what followed, and closes when the list
runs out. That forced a real fix in Playlists: the player's mount was derived from `shown` every
render, so anything that dropped the launch row yanked the player off the screen mid-playback. That
was a latent edge (change the view and it happens) and the normal case for this feature. The parent
now snapshots what it launched with, and only onClose takes the player down.

**4 — "Move to <the last list you moved to>".** The submenu is two levels deep and moving a run of
videos into the same list is the shape this gets used in. The remembered id is a per-account pref;
the row is derived from `moveTargets`, which already excludes the list you are looking at, so it
cannot offer a move to where the video already is. Remembered only on a SUCCESSFUL move — a
shortcut to a move that failed is a shortcut to failing again. Absent, not disabled, when there is
nothing to repeat: there is nothing to explain.

**7 — the live-YouTube results can be sorted.** Client-side, over the page already loaded. Not a
new `order=` param, and this is the decision worth recording: the owner's case is "even one page of
results", i.e. reorder what is on screen; a re-query would REPLACE the results, which is not what
"sort these" means; and the zero-quota scrape source has no order parameter at all, so a
server-side control would silently do nothing (or quietly cost 100 quota units) depending on which
source served the page. Only keys a search hit carries are offered — `subscribers`/`priority`
describe a subscription, which a search hit is not, and `shuffle` on a ranked list is noise. Its
own pref, so sorting a search page never re-sorts the library behind it.

Verified in the browser on localdev against the real account: the channel name renders as
`<a href="…&chan=…">` and a ctrl-click / middle-click is left to the browser while a plain click
still opens the channel in-app; removing from the player stepped 1/3 → 1/2 onto the next video and
closed on the last one; the row menu grew "Move to “…”" only after a move and only on a list that
is not the target; and the results sort reordered by views 41.7M → 29.8M → 24.2M → 15.8M. (The
pane's own ctrl-click arrives without the modifier — a harness limit, so the modifier cases were
driven as real MouseEvents.) Scratch playlists and the touched prefs were cleaned up afterwards.

22 vitest, mutation-checked against 14 mutations (no href at all · href without the rest of the
query · every click hijacked in-app · the middle-click arm dropped from the predicate · plain click
no longer in-app · always jumping to the top of the list · the removed item left in the queue · no
clamp when the last item goes · an empty queue not reported as "nothing left" · sort-then-reverse
flipping ties · unknown values first · titles compared numerically · shuffle offered on search
results · direction ignored) — each red on exactly the test that names it. Item 4 has no test of
its own: its rule is enforced by construction, not by a check.
2026-09-10 22:34:12 +02:00
peter 889d2acf52 feat(plex): the collection page sorts, and a linked collection has a name
Two halves of the same report ("Collectionben nem működik a rendezés?"). The pasted browse URL
sorts correctly — that part was working. What did not:

The Collection INFO page had no sort control at all and a hardcoded release-year-ascending order.
It now shows the browse grid's Sort control, with the browse grid's vocabulary, because a second
sort vocabulary on a neighbouring surface is exactly the kind of drift this repo keeps paying for:
`PLEX_SORT_KEYS` is now one tuple in `plex/query.py`, and a test pins BOTH implementations to it —
browse's key→column map (extracted from the handler to be reachable) and the collection page's
key→member map. A key added to the vocabulary without teaching one of them fails the gate instead
of putting a dead option in the menu.

Ordering a collection cannot be a single ORDER BY: its two legs are different tables, so the merged
member list is sorted in Python. Members with no value for the key sink to the END in both
directions (a movie with no rating is not "the worst rated" — the browse grid's `nullslast()`), and
ties always resolve title-ascending, so flipping the direction reverses the ranking rather than the
alphabet inside it. `lastwatched` needed the show leg's `resumed_at`, so the page now takes ONE
rollup pass that yields both the badge status and the recency instead of the status alone — without
it that menu entry would have ordered every show as never-watched. The default stays release-year
ascending: a franchise reads in release order, and that is what the page did before.

The other half: a shared Plex link carries the collection filter as bare rating_keys, so a link
opened cold had no name for them — the sidebar token rendered `17045` and the chip could only say
"Collection". `GET /api/plex/collections` takes a `keys=` CSV now, and a display-only hook resolves
just the missing ids. It deliberately does NOT write them into `collectionTitles`: that map is
hashed into the Plex query keys, so storing a resolved label there would split the cache from the
identical state and refetch the grid for a piece of text.

Verified in the browser on localdev against the real Plex library: the movie collection's menu
offers Length and the show collection's does not (the same scope rule as browse); rating-ascending
reorders 8.1/7.4/7.2/6.6/6.5 correctly; `lastwatched` on a show collection brings the in-progress
shows up; the choice survives a reload (it is a server pref, on its own key so opening a collection
never re-sorts the library behind it); and the reported URL now reads "Pirates of the Caribbean
Collection" in both the sidebar and the chip.

12 pytest + 8 vitest, mutation-checked against 14 mutations (direction ignored, nulls first, no
stable tiebreak, unknown key untouched, wrong default, show recency dropped, empty `keys=` falling
through to everything, a key missing from either sort map, every id refetched, ids unsorted,
fetched titles unmerged, the chip ignoring them, the hook's result never reaching the builder) —
each red on exactly the test that names it.
2026-09-10 22:05:39 +02:00
peter 782362b808 fix(channels): the video grid follows a running backfill instead of freezing until F5
"Get full history" on a channel page left the list on the cards the channel already had. The
count above it climbed into the thousands, the grid did not move, and only a browser F5 brought
the rest in.

The mechanism: the backfill runs in a BackgroundTask, so the PATCH that starts it returns before
a single row is written — and the mutation invalidates `qk.feed()` on that RESPONSE, which just
refetches the OLD catalogue. Nothing afterwards refetches it: the feed's infinite query has no
interval, and the app-wide defaults are `staleTime: 30s` + `refetchOnWindowFocus: false`. The job
emits no event either (the realtime layer only carries messages today), so the channel detail's
4s live poll is the only thing in the app that sees the job progress — and it refreshed only
`qk.channel`.

So the poll now drives the grid too: on a tick where the stored count moved (or the backfill
finished), it invalidates `qk.feed()` and the `qk.feedSummary()` behind the count pill. The window
is the poll's own (`deep_requested && !backfill_done` on the previous reading), so a detail
refetch caused by some other mutation — each of which invalidates the feed itself — cannot come
through here twice.

Measured on localdev against the real account, same instrument, `document.visibilityState`
forced visible (the poll deliberately pauses on a hidden tab, and a headless pane reads hidden —
without that the first measurement was about the harness, not the fix):

  before — CASCADA 19 -> 176 stored: the header updated live to "162 normal", the grid and its
           count pill sat on "5 items" for 50s.
  after  — fern 54 -> 139 stored: "54 items" -> "139 items" by ~12s, no reload.

6 vitest, mutation-checked against six mutations (no invalidation at all; grid only, count pill
forgotten; predicate ignored; no channel-identity guard; completion with an unchanged count
dropped; poll-window guard dropped) — each one red on exactly the test that names it.
2026-09-10 20:56:16 +02:00
peter 5a42d76e8a feat(plex): an off-by-default loudness normalisation toggle for playback
Bug hunt 2026-09, item 14 — "how does the multichannel downmix work, and is it
normalised?" It isn't: `plex/stream.py` has always encoded `-c:a aac -ac 2 -b:a
192k`, i.e. fixed stereo with ffmpeg's default downmix matrix and no loudness
handling at all, which is why a film mastered for a cinema arrives with whispered
dialogue and deafening action.

New admin knob `plex_audio_loudnorm` (Configuration → Plex), DEFAULT OFF: it makes
a film markedly louder and narrows its dynamic range, which is a taste call rather
than a fix. It sits at the end of the Plex group, after the software-transcode
knobs — it is an audio concern, not one of the libx264 ones.

  -af aformat=channel_layouts=stereo,loudnorm=I=-16:TP=-1.5:LRA=11,aresample=48000

Three things about that chain are load-bearing and each has a test:

  * the downmix is FORCED FIRST rather than left to `-ac 2` — measuring 5.1 and
    folding it down afterwards would normalise something other than what is heard;
  * the trailing resample is not decoration: `loudnorm` runs its own resampler at
    192 kHz internally, which the AAC encoder will not take;
  * one `-af`, no stream index, so EVERY audio rendition of a multi-audio file is
    normalised — not just the first.

The knob is also part of the session directory name (`…_multi-ln_+0.00`), because
it changes the audio the segments contain, and a session left over from before the
admin flipped it must not be mistaken for one that honours it. The suffix is
appended with a DASH: `_SESSION_DIR_RE` treats the tag as one underscore-free
field, and that regex is the only thing keeping the orphan sweep off what may be a
shared scratch path. A test pins both halves — the name changes, and the sweep
still recognises it.

Measured end-to-end on the app's own HLS segments (a real 5.1 AC3 file, same item,
same 160 s offset, same ten segments): OFF -35.2 LUFS, ON -17.2 LUFS. On a 60 s
bench: I -27.6 → -16.6 LUFS with LRA 13.3 → 9.8 LU. A/V sync is untouched — under
this pipeline's own `-copyts` the audio start PTS is identical with and without the
filter, so the 3 s lookahead desyncs nothing.

Direct-played files are served untouched and stay so; a 5.1 mix is always `remux`,
so the reported case is fully covered.
2026-09-10 20:02:06 +02:00
peter ed4ab4e13c fix(plex): subtitles stop blocking the request, and stop re-reading the file
Bug hunt 2026-09, items 8 and 12 — the two things the Plex player was measurably
lying about.

Item 8: picking a subtitle track showed nothing for a long time, and the reported
workaround was to stop/start until it suddenly worked. Measured on prod against a
17.06 GB MKV with two embedded SubRip tracks:

  full pass, one track                154 s
  full pass, both tracks in ONE run   111 s
  -c:s copy (no WebVTT conversion)    143 s
  from 1:00:00 to the end of file      66 s
  a 630 s window at 1:00:00, both       8 s

So the conversion was never the cost: ffmpeg streams every byte of the container
between its seek point and its stop point, and the array delivers ~140 MB/s. The
only lever is reading less. Three changes follow from that:

  * a window around the playhead is extracted and published FIRST (seconds), and
    the full pass completes behind it;
  * every embedded text track rides that same single read — picking Hungarian used
    to cost a full pass and then switching to English cost another one;
  * none of it happens inside a request any more. The endpoint answers 202 while a
    background worker extracts, the player polls /status (carrying its position, so
    the window follows the viewer), and one ffmpeg run finally has a timeout.

-copyts is deliberately absent from a windowed pass: with it ffmpeg ignores -t and
reads the whole container anyway (measured — a [0, 600 s] window still took the
full 145 s), so a window comes back on a zero-based clock and is shifted to
absolute time here. Cached tracks written by older builds have no coverage sidecar
and count as complete, so nothing is re-extracted after the upgrade.

The <track> is no longer mounted before the server has something to put in it; a
"Preparing subtitles: …" notice stands in, and a track that can never be shown says
why instead of leaving the viewer to conclude the picker does nothing.

Item 12: "Bitrate" read a constant 211 kbps on every remux, because it was the
master playlist's BANDWIDTH — which ffmpeg derives from the one stream whose rate
it knows, the -b:a 192k AAC track, plus 10%. It is a measurement now: bytes per
second of MEDIA from hls.js FRAG_LOADED, accumulated per fragment type and summed
(a multi-audio session's video and audio renditions cover the same media seconds,
so pooling them would report half the truth), and an honest dash until the first
fragment lands. The panel also finally names the audio and subtitle track in play.

Verified in the browser against the real Plex library, and every new test
mutation-checked.
2026-09-10 19:31:05 +02:00
peter b8bff504d8 fix(plex): "Clear resume point" now tells Plex about an in-progress item
The button deletes the local row and mirrors the un-mark to Plex, but it decided whether
to push from the previous STATUS alone (prev_status != "new") — and an in-progress item's
row is status="new" carrying a position, which is exactly how the importer maps a bare
viewOffset. So for the single case the button exists for, nothing was ever sent to Plex:
the offset stayed, and the next pull imported it straight back. The user cleared Protector
twice and it returned to Continue watching at 92s both times.

Push now when Plex holds anything to clear — a view count, or a resume point. "hidden"
stays Siftlode-only and pushes nothing, which is what this endpoint's docstring always
claimed and the status-only guard contradicted.

One unscrobble clears BOTH viewCount and viewOffset — measured against the live server from
a watched+offset state and an offset-only one, so the push itself needed no change. The
client docstring that claimed "clears viewCount" is what sent the first diagnosis down a
wrong path (a timeline write with time=0, which does not clear an offset at all); it now
records what the server actually does.

Verified end to end on the real Plex server: the button leaves viewCount=None viewOffset=None,
and a full 44,885-item re-import brings back no row for it.
2026-09-09 23:55:51 +02:00
peter 1630a3df3c fix(player): the YT progress checkpoint reaches the cache, so a reopen cannot rewind it
The player checkpointed every 5s and the server stored it, but nothing wrote the new
position back into react-query: the close path only marked qk.feed() stale with
refetchType "none", and nothing refetches the feed until it remounts (staleTime 30s,
refetchOnWindowFocus off). So the resume read the position the cache had been FETCHED
with, and the next checkpoint then persisted that older value over the newer one.

Measured on localdev: played to 830s, closed (server had 830), reopened the same card
without a reload -> the row was back to 751s. 79 seconds of progress destroyed by a
reopen, and the card's progress bar never moved either.

The Plex player already solved this for its detail query, with a comment describing the
identical failure; this is the feed-side twin that was missed in that sweep. patchCachedProgress
writes the position the SERVER stored (the endpoint clears trivially-early and near-finished
checkpoints to 0, so mirroring our own reading would disagree with the row) into every cached
feed page and playlist detail, keeping array identity where nothing changed. A session map
covers the second half: the launch queue is frozen at mount, so stepping next->prev back to an
item played earlier in the same session resumed from that snapshot.

Verified in the browser: 811 -> close 816 -> reopen 825, and the card now shows Continue/Restart.
2026-09-09 23:55:51 +02:00
peter 14b5fe2e31 release: 0.83.0 (feature/e24-anti-paralysis)
gate / frontend (push) Failing after 2m5s
gate / backend (push) Successful in 7m0s
2026-09-09 22:04:14 +02:00
peter d260aabfd7 chore(release): 0.83.0 — the Home surface 2026-09-09 22:01:33 +02:00
peter d832bb19ce fix(e24): review round 4 — a test that pinned nothing, and prose still ahead of the code
Round 4 reviewed round 3's commit. Nothing in production behaviour was wrong; what was wrong
was the round's own test and its comments, which is the failure class this loop keeps
finding.

- The new NavigationProvider case named after the `&& !channelViewRef.current` clause never
  reached it. With storage cleared and a bare URL the provider lands on "home" (E24's own
  default), not "feed", so asking for "feed" was an ordinary page change and the guard's
  first conjunct short-circuited. Deleting the clause left all 828 tests green — a dead nav
  control (the rail's own module stops closing an open channel page) with no red anywhere.
  The test now asserts the precondition and calls `setPage` with the page that IS current.
- Both new Home cases rejected a promise and then asserted after one fixed macrotask.
  React-query notifies through `notifyManager`'s own `setTimeout(0)`, registered after any
  timer the test schedules, so the assertion raced it; two reviewers measured intermittent
  reds. They poll for a signal that the rejection LANDED instead — the reroll button
  releasing, which is also the honest one for the empty-answer case, where the empty state is
  already on screen while the press is in flight and asserting it early would pass either way.
- `buryUnder`'s comment still opened with "one rule for both entry points" ten lines above the
  paragraph saying the rule is complete only for one of them, listed the non-reporting palette
  rows without `openChannel` — which has its own `replaceState` no-push branch and is reached
  from PlayerModal too — and pointed at a ROADMAP.md this repository does not contain.
- `history.dom.test.ts`'s header claimed its layers close by genuine Backs only, which two of
  its own tests contradict; and `CommandPalette.ordering.dom.test.tsx` still claimed to be
  "the only assertion in the suite" doing what the file corrected in round 3 also does. Round
  3's message said that pair was handled and only one side had been.

The provider test was re-mutation-checked (dropping the clause now goes red), and both
hardened Home tests still go red against their own reverts. Seven consecutive full runs green.
Gate: ruff, ruff format, 3x tsc, eslint (0 errors), prettier, knip, vitest 828, pytest 1005.
2026-09-09 01:02:22 +02:00
peter 59efc8f2e8 fix(e24): review round 3 — the holes round 2's own fixes left
Round 3 reviewed the round-2 commit across six dimensions. Fifteen findings, five survived
three adversarial refuters each; three died on 2+ refutations (a keyboard-focus claim that
is a property of `ui/Button` at all six of its `loading` call sites, a contract-test
false-pass that needs a deliberate revert plus a self-incriminating comment, and a
MAX_EXCLUDE coupling whose slice is unreachable from the real client).

- `rerolling` was still wrong for a FAILED press. `data === undefined` is true forever for a
  key whose request failed (`retry: false`), so the next unrelated mutation re-disabled the
  reroll button — the same defect round 2 reported, one branch over. `!isFetched` is the
  flag that remembers a key settled; `!isError` would not have worked, because query-core
  resets a data-less query to "pending" for the duration of a refetch, hiding the error
  exactly when it matters.
- The hero fallback used `??`, which cannot tell "this budget's last pick resolved to NO
  hero" from "nothing remembered". So a press that failed after an empty one resurrected the
  card the page had already excluded as seen and just said it had nothing beyond. The tagged
  slot is now authoritative for its own budget.
- `setPage`'s boolean — the input the whole burial rule reads — was asserted by nothing.
  The type fences the SHAPE, so deleting a `return` fails tsc, but swapping the two values
  passed typecheck and all 822 tests. NavigationProvider gets its first render test, on the
  value AND on the effects the early return skips.
- `closeThenNavigate`'s door into `buryUnder` was equally unpinned: reverting that one call
  site to the old unconditional burial left the suite green. It has the mirror case now.
- The comment claiming one rule serves both entry points was overstated. `buryUnder` fails
  OPEN, and `CommandItem.run` is typed `() => void`, so only the palette's bare
  `() => setPage(page)` rows forward the boolean at all. Nothing regressed — that path buried
  unconditionally before — but the prose said more than the code does, so it now says what is
  true and points at the deferred fix. Same for a "the only assertion in the suite" claim
  that the file it cites as precedent also makes.

Every new test mutation-checked red for the right reason, both edges where there are two.
Gate: ruff, ruff format, 3x tsc, eslint (0 errors), prettier, knip, vitest 828, pytest 1005.
2026-09-09 00:39:37 +02:00
peter e4ffd59ca7 fix(e24): review round 2 — a buried history entry, a stale hero, and a spinner that was not ours
Nine of the round's ten surviving findings. The tenth (a failed shelf reorder leaving an
order the server rejected on screen) is on ROADMAP.md: its obvious remedy reintroduces the
overlapping-move defect the code already documents, so it wants a design, not a review edit.

- `navigateOverOverlays` buried the live overlay entries before knowing whether the
  navigation pushed one. `setPage` no-ops when the page is already current and Home is the
  default landing page, so Shift+W over a player ON Home marked its entry buried under a
  push that never happened: the unmount consumed the mark instead of reclaiming the entry,
  and the next browser Back was a press that did nothing. `setPage` now reports whether it
  navigated, and one `buryUnder` rule serves both entry points — `closeThenNavigate` takes
  the same no-op navigation from a palette command for the page you are already on.
- The hero could answer a new time budget with the previous one's pick. The chip row resets
  seed/seen/press but cannot reset an answer that already resolved, and `keepPreviousData`
  chained through the disabled seed-0 query — so a reroll under "10 perc" showed the film
  picked for "Bármennyi" while it loaded, and kept it if the request failed. `lastPick` now
  carries the server's own budget echo, and the placeholder is scoped to the budget.
- The reroll button spun and swallowed clicks after mark-watched, bookmark, pin and shelf
  moves: the pick key sits under the invalidated `discover` prefix on purpose, so raw
  `isFetching` was true for background refetches. It now means "a press of mine is
  outstanding" — which still covers the first press, where there is no placeholder.
- Failed writes on Home toasted the page's LOAD-failure string over a fully loaded page;
  they get their own key (HU+EN). The empty-state reroll had no `loading`, so the press was
  unacknowledged and a second click skipped a tier. `danger` moved into the shared unpin row
  so the same command is not red on the shelf and neutral in the rail below it.
- Two comments claimed things the code does not do: the `exclude` cap guards no "unbounded
  scan" (it never reaches SQL), and the route test that named the cap sends three keys.

Tests: the burial rule gets its own file (it is the one assertion that lets a reconcile
issue a real programmatic back()); App's keymap wiring gets a source contract, because
dropping it left all 818 tests green; `add_pin`'s IntegrityError arm was reached by nothing.
Every new test mutation-checked red, both edges where the predicate has two.

Gate: ruff, ruff format, 3x tsc, eslint (0 errors), prettier, knip, vitest 822, pytest 1005.
2026-09-09 00:12:53 +02:00
peter 3709b71a94 fix(e24): the review's low findings — dead constants, honest labels, a 500 that should be a 404
`add_pin`'s IntegrityError handler assumed the unique constraint was the only way in and
re-raised otherwise, so a target row pruned between `resolve_ref` confirming it and the
INSERT (a Plex sync dropping a removed item) tripped the FK and handed the user a 500.
Semantically that is the same "no such thing" the pre-check already returns. Untested: the
race needs two concurrent transactions.

The five per-kind unique constraints had no test at all — the route's own dedupe returns
before reaching the DB, and the migration drift check filters constraint diffs out as noise,
so they could go missing with every lane green.

Frontend, all from the same review: the shelf's unpin row is now `danger` like every other
destructive row in the app; the hero's artwork button says what it will actually do and
names the item instead of always announcing "Play"; the empty-budget message offers a reroll
rather than `StateMessage`'s "Retry", which promised a re-run of something that had not
failed; `setState` and the watch-later toggle catch their rejections instead of leaving an
unhandled promise and a click that silently does nothing.

`StateMessage` gained a `compact` rung, which retires the frontend's only two `!important`
overrides — they existed to beat the primitive's own padding from the call site, and were
the copyable idiom for the next surface that wanted a compact empty state.

Dead vocabulary removed or made load-bearing: `DISCOVER_BUDGETS` was exported, re-exported
and consumed by nothing while `Home.tsx` re-typed the same four bands (a `readonly
DiscoverBudget[]` accepts any subset, so a fifth band would have typechecked green and
rendered no chip); `PIN_KINDS`' value export had no consumer at all; `useComfortPin` no
longer publishes `pinIdOf`/`toggle`, which nothing outside the module calls.

Three comments corrected against the code they describe: a pointer to a `resolve.py` that
was never written, the file header's claim that Home defines no card component of its own,
and the `seen` window still quoting the 8 that was measured and abandoned for 32.

The rest of the review's survivors are logged on ROADMAP.md under "Deferred — E24 epic-end
deep-review (2026-09-08)" — the ones whose dissent was substantive, and the ones whose fix
is wider than the finding.
2026-09-08 03:23:59 +02:00
peter cedebe0ffb test(e24): close the review's coverage gaps, and correct a comment that was untrue
`test_only_a_show_row_pays_for_a_resolution` seeded no watch state, so every rail query
inner-joined its way to zero rows and the loop body — with it the assertion — never ran. It
now seeds a video, a movie and an episode, and asserts all three kinds are present before
the loop, so it cannot go vacuous again.

The Plex half of the surprise rail had no test for either of its promises: every test that
reached `plex_surprise` passed budget="any" against a library with no state rows, so the
duration bounds and the already-started exclusion could both be deleted with the suite
green. The YouTube half has had tests for the same two rules since S1 — the asymmetric half
of one principle.

`plex_item` and `plex_show` — two of the five pinnable kinds — were pinned by no test at
all, leaving the rating_key → internal-id translation and both shelf arms uncovered,
including the self-reference cycle that once 500'd the shelf. And `GET /api/discover/pick`
was exercised only through its domain function, so the route's own query-string translation
could regress silently; the new test excludes all three candidates, which is what proves the
whole list parsed rather than one unsplit string.

`PIN_COLUMNS`/`PIN_KINDS` claimed in a comment to be the one place the pin vocabulary is
spoken and had no readers anywhere: `resolve_ref` re-spelled the column names and the route
re-typed the kind list. Both now drive off the constants.

The `_NO_TS` sentinel was introduced as a crash fix and is not one. The key it replaced,
`(ts is not None, ts)`, cannot raise — tuple comparison finds the first UNEQUAL element via
`==`, so two null keys compare equal and `None < None` is never reached; three independent
runs confirmed it over every permutation. The rationale now says what the sentinel actually
buys, lives once in `plex.rollup`, and the third rail that was still on the older spelling
has been swept to match. Its test is renamed and re-scoped to the mutation it does catch.

Frontend: the reroll's second press asserted `expect.any(Array)`, which cannot tell an
accumulating exclusion window from a replacing one — both that and SEEN_WINDOW = 1 (each
reinstating the measured nine-card cycle) left the suite green. And `onInfo`'s jump type was
untested while `onPlay`'s was pinned twice, so pointing the ⓘ affordance at the player
typechecked and shipped green.

All eight fixes mutation-checked; every mutant died on its intended test.
2026-09-08 03:14:21 +02:00
peter fdc1eabbe0 fix(e24): the review's seven frontend defects on the Home surface
Shift+W was dead whenever anything was open. The global handler called `setPage` directly,
so the page change unmounted the player (it lives inside the module column) and that
unmount's reconcile reclaimed the entry the navigation had just pushed — the player closed
and the page snapped back. `closeThenNavigate` cannot serve here: a global key does not
know which overlay is open, or whether one is. `navigateOverOverlays` is the same fix one
step shorter — bury the live overlay entries first, so the unmount consumes the marks
instead of issuing `history.back()`.

`q.isError` had no `&& !data` guard, alone among 33 call sites and against the convention
`ui/QueryState` documents: react-query flips status to error on a failed REFETCH while
keeping the payload, and every mutation here invalidates the discover prefix, so one blip
while marking a card watched blanked a fully populated landing page.

The reroll's key rooted at "discover-pick", outside the prefix every mutation invalidates,
so a rerolled hero outlived the page: play it, finish it, and it was still the answer with
a zeroed progress bar. It now sits under the same root. A FAILED reroll also rewound the
hero to the card the page loaded with, several presses back, because a new key per press
means `keepPreviousData` (pending-only) cannot cover an error; the last resolved pick is
held instead.

The bookmark on a rail tile never updated — `VideoCard` reads `video.saved` from this
payload and keeps no optimistic state — so the star stayed empty and the next click
re-sent the same call.

The shelf reorder wrote a captured snapshot back on failure rather than re-asking the
server, which its own comment promised: two overlapping moves each reverted to their own
snapshot and settled on an order neither side held, and since `before` is the whole
payload a rollback also discarded any rail refresh that had landed meanwhile. It now
invalidates the discover PREFIX on settle, which also fixes the sibling budgets keeping
the pre-move order.

The hero/rail de-duplication matched one key, so for a Plex series — the rail row is the
show, the hero is the episode it resolves to — the same series printed twice; it now
matches both keys, as `hero.pick`'s own exclusion already did. And when the hero empties
the resume rail, the rail is hidden rather than announcing "nothing half-finished right
now" under a hero captioned "where you left off".

Ten regression tests, all mutation-checked. Three of them were rewritten after the first
mutation round showed they could not go red: two asserted before the state under test had
settled, one matched a title the Plex tile does not render.
2026-09-08 03:03:45 +02:00
peter 84f4b20e5a fix(e24): the review's three backend defects — blocklist, shelf order, next episode
The resume rail built its own SELECT instead of going through `filtered_query` and copied
only three of its predicates, so it dropped the per-user channel blocklist the module
docstring claims it inherits. Blocking a channel never touches `video_states`, so a
half-watched video from it kept its resume position and headed the rail — and, at the
default budget with the resume tier leading, became the landing page's hero: the app
answering "what should I watch" with the thing the user had just banned. The clause is now
`feedquery.not_blocked_cond`, shared with `filtered_query` rather than copied a second
time; the feed's SCOPE clauses stay deliberately unapplied, for the reason `yt_video_row`
already gives.

`next_position` returned the shelf's row COUNT while nothing ever renumbers positions —
`remove_pin` deletes, and every FK cascade removes a pin when its target disappears. Past
the first hole the count under-counts, so a new pin landed in the middle of the shelf and,
past the second, on a position another pin already held. `max(position) + 1`, the idiom
`routes/playlists` already uses.

`next_episode` took the FIRST in-progress episode in broadcast order while `plex.rollup`,
which drives the show page's Resume button, takes the last. One show therefore had two
different "continue watching" answers, with the Home tile ranking it by its most recent
touch and then offering the episode abandoned weeks earlier. The new test asserts the two
implementations against each other rather than against a literal.

All three mutation-checked: each new test goes red for the defect it names.
2026-09-08 02:32:08 +02:00
peter 828dad015c feat(e24): reach the Home surface — the landing toggle, a palette command, Shift+W
S4's reachability half. Home already led the nav order; what was missing was every
way IN other than the rail.

- Settings → Appearance → Startup: "Start on Home", the documented escape hatch for
  the landing override. Deliberately still localStorage and not a server pref —
  `loadInitialPage` answers before /api/me lands, and a landing module decided after
  the first paint is a visible jump. Its two stored values are now ONE exported
  constant, so the writer and the boot-time reader cannot drift on a magic string.
- The command palette gets "Meglepetés" / "Surprise me": it navigates to Home and
  rerolls the hero on arrival. No second "Mit nézzek?" row beside it — the Home
  module row already matches that phrase through its descriptive title.
- A global Shift+W jumps to Home. A chord, not a bare `w`: the Plex player binds bare
  letters and `w` sits on `s`, so a blind reach for the subtitle key would have thrown
  a viewer out of the film.

The palette's request crosses a lazy module boundary, so it is a token that waits
(lib/homeIntent) rather than an event fired into an empty room. Browser-measured:
taken with a compare-and-clear, because StrictMode's double-invoked mount effect
otherwise rerolled twice and landed on the page two picks deep.

HU/EN parity kept; UAT on localdev in both languages, including the empty shelf.
2026-09-08 00:06:34 +02:00
peter 82914b56ca fix(e24): the Home surface's three UAT findings — Back, the reroll, and a show card
All three were measured before they were touched, and the diagnosis moved twice.

**Back from a Plex title played off Home landed on the Plex module's root.** One
click pushed three history entries: the page navigation, the sub-view opened by
the `jumpTo` hand-off, and — under StrictMode, whose effect body runs twice
before `setJumpTo(null)` re-renders — that same sub-view a second time. So the
first Back visibly did nothing and the second stranded the user on a Discover
page they had never visited.

A cross-module hand-off is the CONTINUATION of the navigation that brought you
there, not a second navigation, so it now REPLACES the page entry instead of
stacking on it (`useHistorySubview.open(view, {replace})`, asked for by the jump
target). Replacing is also idempotent, which is what disarms the StrictMode
double-run. The intent rides on `PlexJumpTarget` rather than being inferred,
because the header typeahead jumps from INSIDE the module, where the entry being
replaced would be a sub-view the user really was looking at.

**"Mást kérek" cycled nine cards and never left the resume tier.** Measured:
twelve presses, nine distinct titles, period nine, every one `tier: "resume"` —
the shelf and the surprise tier were unreachable by pressing the button. Two
causes multiplied: the tiers were tried in a fixed priority order over a 20-row
resume rail, so the button just walked down it; and the client's rolling
exclusion window of 8 guaranteed the wrap.

The press count now rotates which tier answers first (`start`), so each press
offers a different KIND of answer — the user's call, given the choice between
that, a wider window, and a surprise-only reroll. The fallthrough is kept, so a
press does not come back empty because the tier it led with had nothing. The
window is matched to the server's own cap rather than a smaller number of ours.

**A half-watched series on Home opened the show page.** A movie already went
straight to the player; only shows did not — and the comfort shelf and the hero
both resolved a show to its next episode while this rail alone did not, handing
back six seasons to choose between. The rail row now carries `next`, and the
client already played `next ?? card`, so this is a backend fix.

Affording it needed `queries.next_episode` to stop loading every episode of the
show into Python to pick one of them (236 rows for Friends, 227 for JAG) — three
ordered LIMIT 1 probes instead. The show→episode rule now lives in ONE function
(`rails.playable`), which is what the shelf, the hero and the rail all call; two
implementations are how they came to disagree.

Ten mutations run, all ten red for the right test — including a first pass where
the press count reached `api.discoverPick` and stopped there, invisible because
every Home test mocks that module. The slice now has its own URL test.

Verified in a clean browser tab against the live account: one Back returns to
Home, the rotation gives resume/comfort/surprise across nine presses with no
repeat, and Westworld opens at S1E1 rather than at its show page.
2026-09-06 05:38:28 +02:00
peter ccd5f3739a feat(e24): pin to the comfort shelf from every card and row
S3 wires the shelf's editor. The gesture is the E10 right-click menu, so the
decision "this is comfort media" can be made where the thing is — the feed, a
Plex grid, a channel row, a playlist — rather than only on the page that lists
the shelf.

One shared piece, `components/comfortPin.tsx`: the five pin-kind constructors,
the shelf query, and ONE `ActionMenuItem` that switches Pin/Unpin honestly (and
says "not known yet" with a disabled row until the shelf has answered, since a
confident "Pin" over something already pinned is exactly the quiet lie the house
rules forbid). Surfaces that already have a row menu compose the item in
(`Playlists`, `PlaylistsRail`); the rest mount `ComfortPinMenu`.

Two structural notes:

- The feed's menu lives in `VirtualFeed`, not in `Feed` — that is the list, and
  it is also what gives the channel page the gesture for free, since it renders
  `<Feed channelScoped>` all the way down.
- Cards take `menuProps` as a FUNCTION of the row, not the built props object.
  `triggerProps` is stable across renders while its return value is not, and a
  fresh object prop would blow past `VideoCard`'s memo for every card in the
  virtualized feed on every parent render.

`DataTable` and `ChannelLayoutGrid` grow a generic `rowProps` so a channel row
answers the same gesture in both of the manager's layouts; the table stays
domain-free and never learns what a menu is.

On Home the shelf tile's menu is the whole editor: four move rows (front / left
/ right / end, reusing the playlist menu's ordering idiom) plus the unpin, over
menu rows rather than drag & drop — dragging inside a horizontal, scrolling
Carousel fights the scroll, and menu rows are keyboard-reachable for free. The
reorder is optimistic on the cached payload and re-asks the server on failure
rather than showing an order that was never accepted.

The exhausted shelf tile (a pinned channel with nothing unwatched left) is
`aria-disabled`, not `disabled`: measured in Chromium, a disabled button still
receives contextmenu and pointerdown, but it loses focus — and with it Shift+F10,
the only keyboard route to the menu it most needs.

Verified end to end in the browser against the live account: all five pin kinds,
the reorder (persisted across a reload), the unpin, and the honest Unpin label on
a resume tile that is already on the shelf.
2026-09-06 05:07:06 +02:00
peter f3c0b7be23 feat(e24): the Home surface — one question, one answer, both sources
E24 S2. A new `home` module in layout C+ (the mockup the owner picked): the
time-budget chips on top, one large hero that answers them, then the comfort
shelf and a cross-source resume rail. Home is the landing — deliberately OVER
the remembered last module, since landing on whatever you happened to close the
app on is the habit this epic exists to replace; `LS.startOnHome = "0"` opts an
account back out (Settings toggle lands in S4).

No third card component: each unified card carries the native feed / Plex
payload, so the rails render through the existing VideoCard (`cardsSmall`) and
PlexPosterCard, and `aspect` decides the tile shape without branching on source.
Plex playback hands off to the Plex module via a new `jumpTo` type — the player
lives there with its lazy hls.js and its provider tree, and mounting a second one
on the landing page to save one hand-off would be the expensive way round.

Four defects found by driving it in a real browser against the live account, each
now pinned by a mutation-checked test:

- **The reroll gave the same card back.** A new seed only re-draws the SURPRISE
  tier; the resume tier deterministically returns the most recently progressed
  thing that fits, so anyone with something half-finished pressed a dead button.
  `/pick` now takes an `exclude` list of `<source>:<id>` keys and the client keeps
  a rolling window of 8. Excluding a SHOW also excludes the episode it resolves
  to — otherwise the same suggestion comes back wearing a hat.
- **"Nothing to suggest yet" flashed over a full library** whenever a reroll was
  in flight: the new query key has no data for a beat. An empty state standing in
  for a loading one is the dishonesty the house rules name outright. The hero now
  falls back to the loaded payload while a pick is pending, and only trusts a null
  once the pick has RESOLVED.
- **A budget change replaced the page — chips included — with a spinner**, so the
  control you just clicked vanished under your cursor. `keepPreviousData`, the
  same reason the feed uses it.
- **The hero printed twice**, once as itself and once as the first resume tile.

Two smaller UI calls made from the same session: the budget caption sits NEXT to
its chips rather than in SegmentedToggle's justify-between mode, which on an 80rem
page throws the answers to the far edge; and the feed takes the YouTube mark its
own toolbar already uses, since Home now owns the house icon.

Gate green: ruff, ruff format, 3x tsc, eslint, prettier, knip, vitest 780,
pytest 989. Verified end to end on localdev against the real account: budget
chips re-answer (matched on REMAINING time — a 51-minute video with 2:46 left is
a ten-minute answer), the reroll walks three distinct heroes, and a Plex show
tile opens its drill-down in the Plex module.
2026-09-06 04:14:14 +02:00
peter 53b50231ff feat(e24): the Home surface read-model — hero, resume rail, comfort shelf
E24 S1. The anti-paralysis answer layer, served from our own Postgres with no
upstream call: one hero suggestion, one CROSS-SOURCE resume rail, and the user's
own comfort shelf.

Survey first, because the roadmap line was wrong about the starting point: Plex
already ships the whole trio (E5's Discover rails, including a reshuffling
Surprise me), the YouTube side ships none of it, and nothing anywhere answers
the question across both. The data already existed on both sides — VideoState
and PlexState are structurally identical, the feed has an `in_progress` filter
and a seeded shuffle, and the YT player already resumes position.

- app/discover/budget.py — the time bands. "How much time do you have"
  (10 perc / 30 perc / egy este / mindegy) replaces 15/D's mood questionnaire:
  deterministic, answerable from a column we have, and it picks the source for
  free. Matched against REMAINING time, so a film with 8 minutes left is a
  ten-minute answer. An unknown runtime fits only the unbounded band.
- app/discover/cards.py — a thin unified card that carries the NATIVE feed /
  Plex payload, so the page renders with the existing card components instead of
  a third one. `aspect` is the one new field: a rail needs to know poster from
  16:9, and that is a property of the artwork, not of the module.
- app/discover/queries.py — the SQL, reusing feedquery and plex.rollup rather
  than the Plex Discover ROUTE, whose LibraryFilters narrowing belongs to a
  surface that has a sidebar. One seed drives both halves so a reroll moves
  them together.
- app/discover/hero.py — resume -> comfort -> surprise, with the chosen tier in
  the payload so the client labels it honestly instead of guessing.
- comfort_pins (0083): five nullable CASCADE-ing FKs under num_nonnulls(...) = 1
  rather than a flag or a polymorphic string ref. A flag cannot order a shelf or
  hold a channel; a string ref would let a deleted library item leave a dead tile
  on the one surface whose whole job is to be reliable.

Two honest-state rules the tests pin: the surprise draw excludes anything
already started (the resume tier may have skipped it for not fitting, and
offering it back as "Meglepetés" would contradict the question), and which
source leads the draw follows the seed, so the Plex library can actually answer
"egy este".

Gate green: ruff, ruff format, 3x tsc, eslint, prettier, knip, vitest 770,
pytest 987.
2026-09-06 03:31:26 +02:00
peter 5703c4d713 fix(plex): a null progress timestamp must sort a rail row last, not raise
The Continue-watching and Surprise rails sorted on `(ts is not None, ts)`. That
key LOOKS null-safe and is not: two rows that both lack a timestamp compare
equal on the flag and then compare `None < None`, which raises — so a show whose
rollup has no `resumed_at`, or a movie state written before
`progress_updated_at` existed, could 500 the rail instead of sorting last.

Replaced with an explicit epoch sentinel, and named it beside the other rail
constants so the sibling rail E24 adds cannot re-derive the same broken key.
2026-09-06 03:31:09 +02:00
peter 013f19faca refactor(plex): move content_libs down into the query layer
The browse scope is a pure DB query with no request state, but it lived in
routes/plex/_shared.py. E24's Home surface needs the same scope from a domain
package, and a domain package importing from routes/ inverts the one-way
dependency CLAUDE.md draws.

Moved to app/plex/query.py and renamed to content_libs (public — it is a domain
helper now). Swept all five call sites and both tests in one go rather than
leaving a re-export shim: the canonical-patterns table already names the
routes.admin re-export as the anti-pattern.
2026-09-06 03:31:02 +02:00
354 changed files with 25612 additions and 2075 deletions
+102
View File
@@ -7,6 +7,108 @@ edit the release notes and re-run the generator.
Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading).
## 0.86.0 — 2026-09-14
A failed download now tells you why it failed and what you can do about it, instead of repeating yt-dlp's raw error. Facebook comes first: downloads that used to die on “Cannot parse data” go through, and a Facebook group post downloads its own video instead of a different one from the page.
### Features
- A failure says why, in your language: the row names the cause (“Facebook needs you to be signed in”, “Facebook did not accept your cookies”, “No video found in this post”) and its details window says what to do next. The raw error text is still there below it.
- A Facebook download with your cookies that stops at “Cannot parse data” is tried again as Chrome, which gets past it.
- A Facebook group post link downloads the post's own video. Before, yt-dlp either found nothing or quietly downloaded another video from the page; a post without a video now says so.
- Retry and Re-download use a cookie you added after the failure, so the Retry the message asks for actually goes out signed in.
### Fixes
- A Re-download of a download that needed your cookies used to rebuild without them.
- A job queued behind a download that already failed shows that failure's explanation instead of a bare “Download failed”.
- A download yt-dlp reports no length for takes its length from the file, so its Edit / trim action appears.
- Pressing Pause, Stop or Remove twice, or in two tabs, can no longer free a file another download still uses.
- The confirm buttons in Downloads name the action (Stop download, Re-download, Remove) instead of repeating the question.
### Internal
- Four sprints, each Facebook behaviour measured on the development server with a real cookie before it was built. Two epic-end review rounds (20 → 17 fixed, then a targeted 4 → 4), every regression test mutation-checked.
## 0.85.0 — 2026-09-13
Siftlode can now be driven from the keyboard, and Back, dialogs and menus stop losing your place. j/k and the arrow keys walk every list and grid, Enter opens what you are on, and Back puts you on the item you opened a view from instead of at the top of the page.
### Features
- j/k and the arrow keys walk the feed, the library, a channel page, the Home rails, every Plex grid and rail, the Channels, Playlists and Downloads tables and the conversation list; Enter opens the item you are on. After you hide or delete an item, the next j goes on from where it was.
- Back returns you to the item a view was opened from — the Plex title you opened from a Home rail, the channel you opened from its row — with the keyboard still on it.
- Shortcuts to get around: s jumps to the search field, g-chords jump to a module, and the cheat sheet (h) names every key in the language you are reading. PageUp/PageDown move a page at a time in the command palette and in every dropdown list.
- The command palette ranks what you typed first: “theme” now puts the Theme rows above a channel whose handle happens to contain it.
- A card reached with Tab or j shows one ring around the whole card, and a screen reader calls it by its title.
### Fixes
- Back and overlays keep their books straight: Back after navigating over About also closes About, a late clean-up no longer undoes the navigation you just made, and Shift+W on Home closes the player.
- Dialogs and menus hand the keyboard back where it came from. Escape from a card’s context menu returns to the card, not its thumbnail link; closing the player under another dialog no longer leaves the page unscrollable; a button that is busy with a request keeps the focus instead of dropping it to the page.
- A write that fails says so, with the reason — several saves, share links and admin actions used to fail silently or in English under the Hungarian interface.
- A split video export that fails part-way (the job limit) tells you how many clips were added and leaves only the rest selected, so trying again does not queue the first ones twice.
- “Feed” is called Feed everywhere, in both languages.
### Internal
- Five sprints: history integrity, one dialog/focus contract, palette ranking and accessibility, the keyboard layer, and a Hungarian/English keyboard-only pass. Focus and event-order questions were measured in a real browser before each fix, not argued.
- Three epic-end review rounds — 19 → 14, 18 → 12 and a targeted 14 → 14 — each regression test mutation-checked. The last round's findings all sat inside the previous round's own fixes, which is where the loop was closed.
## 0.84.0 — 2026-09-12
A bug-fix release, from a list you wrote while using Siftlode. Two of these were destroying your own data: a YouTube video rewound itself when you reopened it, and clearing a resume point on a half-watched Plex item told Plex nothing. Subtitles now appear in seconds instead of minutes, the Plex player's bitrate is a real measurement rather than a guess, and a channel you turn full history on for fills in front of you instead of looking frozen.
### Features
- Subtitles arrive in seconds. Picking an embedded subtitle track used to take about two and a half minutes on a large film, and switching from Hungarian to English cost the same again — because the cost was never the conversion, it was reading the whole file. It now reads a window around where you are, shows that first, and finishes the rest behind it; every text track rides the SAME read, so switching tracks is instant. On a 17 GB film: 154 s before, 8 s for the window now. Nothing blocks while it works, there is a “Preparing subtitles” pill instead of a frozen player, and a track that can never be shown finally says so.
- The Plex player's info panel tells you the truth. Bitrate is measured from the bytes actually arriving, not copied off the manifest — and it shows a dash until there is something to measure rather than a confident wrong number. It also lists the audio and the subtitle track you are on, which it never did.
- A new admin switch: loudness normalisation for Plex playback (Configuration → Plex). Off by default, because it makes a film markedly louder and narrows its dynamic range — that is a taste call, not a fix. On a 5.1 film it moved the measured loudness from −35.2 to −17.2 LUFS, and it does not touch A/V sync.
- Collections can be sorted. The collection page had no sort control at all and always ordered by release year — it now carries the same Sort control as the library grid, with the same options, and your choice survives a reload without re-sorting the library behind it. A collection opened from a shared link also shows its real name instead of a raw id.
- Remove a video from a playlist without leaving the player, and the player steps to whatever followed instead of closing. Above the move-to submenu there is now a one-click “move to the list you last moved to”.
- Live YouTube search results can be sorted — by views, date or length, over the results already on screen, so sorting rearranges what you are looking at instead of running a different search.
- The channel name in the player is a real link again: middle-click and Ctrl-click open it in a new tab, the way the YouTube icon beside it always did, while a plain click still takes the in-app route.
### Fixes
- A YouTube video no longer rewinds when you reopen it. Closing at 13:50 and coming back put you at 12:31 — the checkpoint reached the server but not the page's own cache, so the stale position won. It now goes forward: 811 → 816 → 825 s across reopens.
- “Clear resume point” works on a half-watched Plex item. The push to Plex was skipped for exactly the case you would use it on, so Plex kept the old offset and handed it straight back on the next import. Verified against the real server, including a full 44,885-item re-import.
- Turning full history on for a channel fills the page in front of you. The grid and the item count stayed frozen for the best part of a minute while the header counted up, because the request that starts the backfill returns before anything is stored. Measured: the count now moves 54 → 139 within about twelve seconds, no reload.
- Starting or seeking a Plex film is about eight seconds faster. Every session start and every seek waited out a dead eight-second timer for a file the transcoder never writes — on a film with more than one audio track, which is most of them. A multi-audio session now starts in 0.9 s cold and 0.7 s on a seek.
- A Plex session the server no longer has now recovers or says so, instead of spinning forever. A stream that kept being lost could restart itself indefinitely behind a spinner that never became an error.
- A Plex stream request for something that cannot exist answers immediately rather than holding the server for 20–30 seconds. Enough of those at once could stall the whole app.
- The Plex player no longer counts one viewing twice, and a failed “watched” mark no longer half-happens: it used to leave the item looking watched in the app while the server disagreed, or quietly mark it watched anyway a few seconds after telling you it had failed.
- Dragging a Plex playlist into a new order sticks. Leaving and coming straight back showed the old order, and the next drag saved that.
- A refused action says WHY, in the language you are reading. Several places showed a generic “couldn't do that” — or nothing at all — where the server had explained itself, including the Plex “Add to playlist” dialog, the collection editor and the upload dialog. The demo account gets its reason back too.
- Dialogs behave for the keyboard. Opening one puts the cursor where you would expect (the jump-to box opens on its search field, not its close button), closing one hands the focus back to what you opened it from, and a dialog that closes underneath another one no longer steals the focus out of the one still on screen.
- The regular micro-stutter in the Plex player is NOT Siftlode's, and this release does not pretend to fix it. Measured per frame: the player delivers every source frame with nothing dropped; the hold comes from a 23.976 fps film on a 120.0148 Hz display, and it reproduces identically outside Siftlode. Setting the display to 119.88 Hz removes it for every application, not just this one.
### Internal
- The round started from sixteen items written from real use, not from an audit. Eleven were fixed, one was measured and closed without a code change (the display cadence above), two are parked for a reproducible case, and one — playlist multi-select and bulk actions — was scoped out to its own sprint. The measurements are the point: item 8's “35 s” turned out to be one file's number, and the real law is bytes-ffmpeg-must-read divided by ~140 MB/s, which is why the fix is a window rather than a faster conversion.
- Seven deep-review rounds followed, each reviewing the previous round's own fix commits: 32 → 24 → 43 → 31 → 22 → 26 → 18 raised. Round 7 was run TARGETED rather than full — four reviewers on the six surfaces round 6 had changed, and adversarial verification spent only where a wrong fix could re-break the previous round's — which is seven agents instead of about thirty-six for two HIGH findings. Every regression test added in a fix round was mutation-checked: reverted, run, and confirmed red for the defect it is named after. Thirteen of round 7's mutations were, and two were instructive — one guard no test could drive, and one round-6 test whose mock modelled a server that accepts a write and then serves the pre-write state.
- What the eight-lane gate structurally cannot catch, and what this round taught about it: every real defect of the last three rounds was a wrong belief about an external system, not broken logic. ffmpeg substitutes the VARIANT before the sequence in its segment filenames, so a guard written the other way round was inert on every multi-audio file and a warm-up waited eight seconds for a name that is never written. A reclaim budget's window was anchored at the first failure rather than the last, so any failure cadence between one and two minutes reset it forever — invisible to two tests that spaced their cases at zero and ten minutes. jsdom performs no default Tab, so a focus test can be green with no focus trap installed at all. The conclusion carried forward is to measure the boundary when writing the code, not when reviewing it: one ffmpeg run costs pennies, a review round costs a session.
## 0.83.0 — 2026-09-09
Siftlode now opens on a page that answers one question: what should I watch right now. You say how much time you have — ten minutes, half an hour, a whole evening — and it gives you one suggestion, from YouTube or from Plex, with a button that picks up where you left off. Under it sits a shelf of the things you always come back to, which you fill yourself.
### Features
- A new Home page, and it is where Siftlode starts. It asks how much time you have — 10 minutes, 30 minutes, an evening, or it does not matter — and answers with ONE suggestion rather than a wall of thumbnails. The question is on top, the answer directly under it. If you would rather land on the feed as before, Settings → Appearance → Startup turns it off.
- The time you have picks the source, not just the length. Ten minutes gets you a short video; an evening is allowed to offer a film. It matches on the time you have LEFT in something, so a series episode you are four minutes from finishing counts as a four-minute answer.
- The suggestion is one of three things, and it says which: something you left half-finished, something off your comfort shelf, or a surprise. Continue takes you back to the second you stopped. Something else rerolls it — there is no separate surprise-me page, the reroll is it.
- A comfort shelf you fill yourself, for the media you put on without thinking. It takes a whole channel, a whole show, a film, a single video or a playlist — because comfort viewing is put on TheVR, not play this one episode. One click resolves it: a show gives you the next unwatched episode, a channel its newest unwatched video. The shelf can be reordered into the order you want.
- Pin from wherever you are: the same ⋯ menu — or a right-click — on a feed video, a channel row, a playlist, a Plex poster or a Plex row now carries a Comfort shelf entry. Feed video cards answer a right-click for the first time.
- Continue watching, on both sides at once. The rail merges what you left half-watched on YouTube with what you left half-watched on Plex, most recently touched first — the YouTube half of that did not exist before.
- Shift+W jumps to Home from anywhere, including out of a player, and it is in the keyboard sheet. The jump-to box (Ctrl+K) gained a Surprise me row, and typing what should I watch finds Home itself.
### Internal
- E24, four sprints. S1 built a discover read-model as its own backend package — the unified card shape maps the EXISTING Plex and feed serializers rather than adding a base serializer, the rails reuse the feed query layer and the Plex rollup, and the tiered hero rule is one shared function that returns the tier it chose, so the UI prints an honest kicker instead of guessing. The shelf is a typed comfort_pins table with a one-of check constraint and a real foreign key per kind, not a polymorphic string reference: a deleted Plex item must not leave a dead pin on the shelf. S2 built the page on the existing carousel and the two existing card components rather than a third. S3 wired pinning through the context-menu input path E10 shipped, which absorbed that epic's parked video-card item. S4 added the ways in.
- The closing review ran four rounds, 46 → 23 → 19 → 15 → 14, each round reviewing the previous round's own fix diff, with three adversarial verifiers per finding in rounds one to three. Four of round one's findings died at 2-of-3 and are recorded so they are not raised again. What the eight-lane gate could not have caught: the resume rail built its own query and dropped the per-user channel blocklist, so a video from a channel you had just banned could become the landing page's suggestion; the shelf numbered a new pin by row COUNT while nothing renumbers, so after any non-tail unpin a pin landed mid-shelf and could collide; the Home tile and the show page disagreed about which episode comes next, one taking the first in-progress episode and the other the last; and Shift+W was dead whenever anything was open, because the page change unmounted the player and its cleanup reclaimed the history entry the navigation had just pushed. Rounds three and four stopped finding defects in the epic and started finding them in the review's own fix commits — a test named after a branch it never reached, comments claiming more than the code does — which is the signal to stop.
- Two process notes worth keeping. A commit in this epic fixed a bug that does not exist: its message and two comments claimed a sort key raises when two rows both lack a timestamp, which tuple comparison never reaches — three verifiers executed it independently, and the rationale is now corrected in place rather than quietly deleted. And three of the regression tests written during the fix rounds could not go red for the defect they named until they were rewritten, which is why every test added in a fix round on this epic was mutation-checked.
## 0.82.0 — 2026-09-06
Siftlode gets a keyboard. There is one place that lists every shortcut, a jump-to box that takes you anywhere by name, and a right-click menu on playlist rows — plus the whole app now holds together at a tablet's width and stops thinning itself out on a very wide screen.
+6 -2
View File
@@ -63,8 +63,9 @@ Frontend: `main.tsx` provider tree → `App.tsx` shell → `moduleRegistry` rout
components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
its menus, `Modal`/`Overlay`, `Button`/`form`, `DataTable`/`VirtualGrid`, `QueryState`, …). That
boundary is one-way and **lint-enforced**, not just described: a platform component may not import a
feature component or a domain API slice (`no-restricted-imports` in `eslint.config.js`, with its own
test). A new generic component goes in `ui/`; see [`frontend/src/components/ui/README.md`](frontend/src/components/ui/README.md).
feature component or a domain API slice, and one layer down `lib/` may not import from `components/` at
all, not even a type (`no-restricted-imports` in `eslint.config.js`, with its own test). A new generic
component goes in `ui/`; see [`frontend/src/components/ui/README.md`](frontend/src/components/ui/README.md).
Data via react-query (`qk` factory), per-account `localStorage`, and server prefs; API through
`lib/api/*` slices over `core.req<T>`.
@@ -89,8 +90,11 @@ CQRS, a repository/DI framework, a router library, a generic CRUD-router factory
| FE per-account state | `usePref` | raw `readAccount`/`writeAccount` or bespoke hooks |
| FE action button | `ui/Button` — pick a `SIZES` rung | a hand-rolled `<button className="px-3 py-2 …">` |
| FE keyboard shortcut | a row in `lib/keymap.ts` + `dispatch()` | a private `keydown` listener with its own key literals |
| FE list/grid keys | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open` | a private arrow-key / roving-tabindex handler |
| FE view push (history) | `rememberNavPosition()` right before `history.pushState` | a push that forgets: Back strands focus on `<body>` |
| FE control | the chip/segmented primitives (`Select` only for >6 options) | a native `<select>` |
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)` | a bare `catch {}` or `notify()` with no error attached |
## House rules that bite (learned from real bugs)
+1 -1
View File
@@ -1 +1 @@
0.82.0
0.86.0
-19
View File
@@ -1,19 +0,0 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
RUN adduser --disabled-password --gecos "" appuser && chown -R appuser /app
USER appuser
EXPOSE 8000
CMD ["sh", "entrypoint.sh"]
@@ -0,0 +1,78 @@
"""comfort_pins — the shelf you fall back to when you cannot decide
Revision ID: 0083_comfort_pins
Revises: 0082_invite_lang
Create Date: 2026-09-06
E24's one new table. The shelf holds five different things — a video, a YouTube channel, a Plex show,
a Plex movie, a playlist — because comfort media is usually "put on X", a series or a channel, and
almost never one particular video.
Three shapes were possible and two are wrong here:
- a boolean on `video_states` / `plex_states` cannot ORDER a shelf, and has nowhere to put a channel
or a show (neither has a per-user state row at all);
- a polymorphic `(kind, ref)` string pair orders and holds anything, but drops the foreign key — and a
Plex item removed from the library, or a channel the user unsubscribes and prunes, would then leave
a dead tile on the one surface whose entire job is to be reliable when nothing else is.
So: one nullable CASCADE-ing FK per kind, plus `num_nonnulls(...) = 1`. The union invariant lives in
the schema rather than in a comment at every insert site, which is the same rule that the
`cancelled`/`canceled` bug taught this codebase.
NULLs are DISTINCT in a Postgres unique index, so the five per-kind `(user_id, <ref>)` uniques each
stop a double-pin without colliding with the other four kinds' NULL rows.
No backfill: an empty shelf is the honest state for every existing account. The surface renders its
own empty state and invites the first pin.
"""
import sqlalchemy as sa
from alembic import op
revision: str = "0083_comfort_pins"
down_revision: str | None = "0082_invite_lang"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.create_table(
"comfort_pins",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column(
"user_id",
sa.Integer(),
sa.ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("video_id", sa.String(length=16), sa.ForeignKey("videos.id", ondelete="CASCADE")),
sa.Column(
"channel_id", sa.String(length=32), sa.ForeignKey("channels.id", ondelete="CASCADE")
),
sa.Column("plex_item_id", sa.Integer(), sa.ForeignKey("plex_items.id", ondelete="CASCADE")),
sa.Column("plex_show_id", sa.Integer(), sa.ForeignKey("plex_shows.id", ondelete="CASCADE")),
sa.Column("playlist_id", sa.Integer(), sa.ForeignKey("playlists.id", ondelete="CASCADE")),
sa.Column("position", sa.Integer(), nullable=False, server_default="0"),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.func.now(),
nullable=False,
),
sa.CheckConstraint(
"num_nonnulls(video_id, channel_id, plex_item_id, plex_show_id, playlist_id) = 1",
name="ck_comfort_pin_one_ref",
),
sa.UniqueConstraint("user_id", "video_id", name="uq_comfort_pin_video"),
sa.UniqueConstraint("user_id", "channel_id", name="uq_comfort_pin_channel"),
sa.UniqueConstraint("user_id", "plex_item_id", name="uq_comfort_pin_plex_item"),
sa.UniqueConstraint("user_id", "plex_show_id", name="uq_comfort_pin_plex_show"),
sa.UniqueConstraint("user_id", "playlist_id", name="uq_comfort_pin_playlist"),
)
op.create_index("ix_comfort_pins_user_id", "comfort_pins", ["user_id"])
def downgrade() -> None:
op.drop_index("ix_comfort_pins_user_id", table_name="comfort_pins")
op.drop_table("comfort_pins")
@@ -0,0 +1,41 @@
"""add download_jobs.error_detail — the domain-specific entry of the failure catalog (E11.5 S1)
Revision ID: 0084_dl_job_error_detail
Revises: 0083_comfort_pins
Create Date: 2026-09-13
`error_code` is the generic category (it decides whether a Retry is offered). `error_detail` names
the host-specific catalog entry `downloads.errors.classify` matched on top of it — `facebook.login_wall`
and its siblings — so the UI can say what went wrong ON THAT SITE and what to do about it, instead of
the category's generic sentence. Nullable: most failures have no host-specific entry.
Backfill: a job's code is written once, at failure time, so a new rule never reaches the rows that
failed before it. The one shape measured on prod (2026-09-13, jobs 9/66/75/145/156/157/182/256/257) is
yt-dlp's `[facebook] <id>: Cannot parse data`, stored as `unknown`. Without a cookie that is Facebook's
login wall. Whether a cookie was attached at the time is not recorded anywhere, so every old row gets
the no-cookie entry — on prod no account had a Facebook cookie, so that is what they all were.
"""
import sqlalchemy as sa
from alembic import op
revision: str = "0084_dl_job_error_detail"
down_revision: str | None = "0083_comfort_pins"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.add_column("download_jobs", sa.Column("error_detail", sa.String(length=48), nullable=True))
op.execute(
sa.text(
"UPDATE download_jobs SET error_code = 'login_required', "
"error_detail = 'facebook.login_wall' "
"WHERE (error_code IS NULL OR error_code = 'unknown') "
"AND error ILIKE '%[facebook]%cannot parse data%'"
)
)
def downgrade() -> None:
op.drop_column("download_jobs", "error_detail")
+5 -1
View File
@@ -341,7 +341,11 @@ def require_human(user: User = Depends(current_user)) -> User:
no token, so has_read_scope/has_write_scope are False); this makes the boundary explicit
where there's no scope check to lean on."""
if user.is_demo:
raise HTTPException(status_code=403, detail="Not available in the demo account.")
# Structured, because `req` leaves 403 to the caller and `notifyActionError` only shows a
# STRUCTURED detail — a plain string here reached the demo user as their caller's generic
# "couldn't do that", with no hint that the demo account is the reason (round 6). Affordances
# this gate sits behind are not all UI-gated on `is_demo` (the comfort shelf is not).
raise HTTPException(status_code=403, detail={"code": "demo"})
return user
+5
View File
@@ -271,6 +271,11 @@ class Settings(BaseSettings):
# trades quality for size (lower = better + bigger). `max_height` caps the SOFTWARE transcode's output
# height (0 = keep source) so a CPU re-encode of a tall source stays affordable — the GPU path always
# encodes at source resolution.
# Loudness normalisation of the downmixed stereo audio on the HLS path (EBU R128 `loudnorm`).
# OFF by default: it makes a film ~11 dB louder and narrows its dynamic range, which is a taste
# call, not a fix. Only affects files that go through remux/transcode — a direct-played file is
# served untouched, but those are browser-native stereo anyway, not the 5.1 mixes this is for.
plex_audio_loudnorm: bool = False
plex_transcode_preset: str = "veryfast"
plex_transcode_crf: int = 20
plex_transcode_max_height: int = 1080
+10
View File
@@ -0,0 +1,10 @@
"""E24 — the read-model behind the Home surface ("Mit nézzek most?").
The anti-paralysis answer layer: one hero suggestion, a cross-source resume rail, and the user's own
comfort shelf, all served from our own Postgres with no upstream call at serve time (the same rule
the Plex Discover surface follows).
Layering, mirroring `app/plex/`: `budget` (the time bands) and `cards` (the unified row shape) are
pure; `queries` holds the SQL; `rails`, `pins` and `hero` compose. Nothing here imports from
`app.routes` — `routes/discover.py` is a thin HTTP shell over these.
"""
+62
View File
@@ -0,0 +1,62 @@
"""The time budget — E24's replacement for "what mood are you in?".
`random-gondolatok` 15/D asked for a mood questionnaire before the recommendations. The same file's
later BA-interview note (#23) says the mood half is not what makes this work: the friction is
deciding at all. A mood→genre mapping would also be invented data — we store no mood signal and never
will — whereas "how much time do you have" is a question the library can actually answer, from a
column we already have.
So the surface asks about TIME, and the answer is deterministic: four bands, no model, no latency.
The band also picks the source for free — ten minutes is a YouTube video, an evening is a movie —
which is exactly the cross-source behaviour the Home surface exists to provide.
The bands overlap on purpose. "I have 30 minutes" should not be answered with a two-minute Short just
because it fits; a lower bound is as much a part of the promise as the upper one.
"""
from typing import Literal
Budget = Literal["short", "medium", "long", "any"]
DEFAULT_BUDGET: Budget = "any"
# budget -> (min_seconds, max_seconds); None = unbounded on that side.
BANDS: dict[str, tuple[int | None, int | None]] = {
"short": (None, 12 * 60), # "10 perc" — a bit of slack over the round number the chip shows
"medium": (8 * 60, 40 * 60), # "30 perc"
"long": (40 * 60, None), # "Egy este"
"any": (None, None), # "Mindegy"
}
def band(budget: str) -> tuple[int | None, int | None]:
"""The (min, max) seconds for a budget. Unknown budgets fall back to the unbounded band rather
than raising: the route validates the vocabulary with a Literal, so anything reaching here with a
bad value is our own bug, and an unfiltered shelf is a better failure than a 500 on the landing
page."""
return BANDS.get(budget, BANDS["any"])
def remaining_seconds(duration_seconds: int | None, position_seconds: int) -> int | None:
"""What is actually LEFT to watch. The question is "how much time do you have", so a 90-minute
film with 8 minutes to go belongs in the ten-minute band, not the evening one. Unknown duration
stays unknown — see `fits`."""
if duration_seconds is None:
return None
return max(0, duration_seconds - max(0, position_seconds))
def fits(seconds: int | None, budget: str) -> bool:
"""Does a runtime fit the budget?
An unknown duration fits only the unbounded band. We cannot promise an item finishes inside ten
minutes when we do not know how long it is, and quietly including it would make the one control
on this page dishonest — the failure mode the house rules call an un-honest state."""
lo, hi = band(budget)
if seconds is None:
return lo is None and hi is None
if lo is not None and seconds < lo:
return False
if hi is not None and seconds > hi:
return False
return True
+141
View File
@@ -0,0 +1,141 @@
"""The unified discover card: one row shape the Home surface can render without knowing which
module it came from.
This is deliberately a THIN mapping, not a new serializer layer. The per-source projections already
exist and are already the shapes the frontend's `VideoCard` and `PlexPosterCard` consume —
`feedquery.serialize` for YouTube, `plex.cards.movie_card`/`show_card`/`leaf_card` for Plex — so each
unified card carries the native payload under `video` / `plex` alongside the handful of fields the
rail chrome itself needs (title, thumbnail, progress, runtime). The Home page therefore renders with
the existing card components rather than a third one, and a field added to a feed row or a Plex card
still gets added in exactly one place. CLAUDE.md's "not wanted" list names a base serializer for
precisely this reason.
`aspect` is the one genuinely new field. A rail has to know whether a tile is a 16:9 still or a 2:3
poster, and that is a property of the ARTWORK, not of the module: a Plex episode ships a landscape
still exactly like a YouTube video does, while a movie and a show ship posters. Emitting it lets the
rail layer stay domain-free instead of branching on `source`.
`subtitle` only ever carries DATA — a channel name, a show title, a year. Nothing here composes an
English phrase: this repo is HU+EN with zero key drift, and a sentence assembled in the backend is a
string no translation file can reach. Anything that needs wording (season counts, "N videó") is left
to the client, which has both the native payload and the translations.
"""
from app.discover.budget import remaining_seconds
from app.models import Channel, Playlist
# Plex artwork is a poster for the things you browse and a landscape still for the things you play.
_POSTER_KINDS = ("movie", "show")
def _base(
*,
source: str, # "youtube" | "plex"
kind: str, # "video" | "movie" | "show" | "episode" | "channel" | "playlist"
id: str,
title: str | None,
aspect: str, # "wide" (16:9) | "poster" (2:3)
subtitle: str | None = None,
thumb: str | None = None,
duration_seconds: int | None = None,
position_seconds: int = 0,
status: str = "new",
) -> dict:
return {
"source": source,
"kind": kind,
"id": id,
"title": title or "",
"subtitle": subtitle,
"thumb": thumb,
"aspect": aspect,
"duration_seconds": duration_seconds,
"position_seconds": position_seconds,
"remaining_seconds": remaining_seconds(duration_seconds, position_seconds),
"status": status,
# Native payloads — at most one is ever set. The client opens the player from these.
"video": None,
"plex": None,
}
def key(card: dict) -> str:
"""The identity of a card ACROSS sources — `<source>:<id>`. A Plex rating_key and a YouTube video
id could in principle collide as bare strings, and the reroll's exclusion list has to be able to
say "not that one" about either."""
return f"{card['source']}:{card['id']}"
def from_video(row: dict) -> dict:
"""`row` is a `feedquery.serialize` dict — the same object the feed grid renders."""
card = _base(
source="youtube",
kind="video",
id=row["id"],
title=row["title"],
aspect="wide",
subtitle=row.get("channel_title"),
thumb=row.get("thumbnail_url"),
duration_seconds=row.get("duration_seconds"),
position_seconds=row.get("position_seconds") or 0,
status=row.get("status") or "new",
)
card["video"] = row
return card
def from_plex(row: dict) -> dict:
"""`row` is a `plex.cards` dict (movie / show / leaf). Its own `type` is the kind."""
kind = row.get("type") or "movie"
if kind == "episode":
subtitle = row.get("show_title")
elif kind == "show":
subtitle = None # season count lives in the native payload; the client words it
else:
subtitle = str(row["year"]) if row.get("year") else None
card = _base(
source="plex",
kind=kind,
id=row["id"],
title=row.get("title"),
aspect="poster" if kind in _POSTER_KINDS else "wide",
subtitle=subtitle,
thumb=row.get("thumb"),
duration_seconds=row.get("duration_seconds"),
position_seconds=row.get("position_seconds") or 0,
status=row.get("status") or "new",
)
card["plex"] = row
return card
def from_channel(ch: Channel, unwatched: int) -> dict:
"""A pinned channel. It is not itself playable — the shelf resolves it to its newest unwatched
video on click (see `pins._shelf_card`) — so the tile advertises what is waiting there instead of a
runtime it does not have."""
card = _base(
source="youtube",
kind="channel",
id=ch.id,
title=ch.title,
aspect="wide",
thumb=ch.thumbnail_url,
)
card["unwatched_count"] = unwatched
return card
def from_playlist(pl: Playlist, item_count: int, thumb: str | None) -> dict:
"""A pinned playlist. `thumb` is its first item's thumbnail — a shelf of blank tiles would defeat
the point of a surface you are supposed to be able to scan at a glance."""
card = _base(
source="youtube",
kind="playlist",
id=str(pl.id),
title=pl.name,
aspect="wide",
thumb=thumb,
)
card["item_count"] = item_count
return card
+104
View File
@@ -0,0 +1,104 @@
"""The hero: the single suggestion the Home surface leads with.
One rule, in one function, because the page's entire promise is that it answers rather than offers.
Three tiers, tried in order:
1. **resume** — something already started whose REMAINING time fits the budget. Almost always the
right answer: it is the thing the user themselves chose most recently, and finishing it costs no
decision at all.
2. **comfort** — a pinned shelf entry that fits. The user's own declared fallback, which beats
anything we could infer.
3. **surprise** — a seeded draw from what is unseen and fits.
That order is the PAGE's answer. "Mást kérek" rotates it (`start`), so each press offers a different
KIND of answer rather than the next row of the same list — the user's call, 2026-09-06, after the
button was measured cycling nine resume cards and never once reaching the shelf.
The tier travels with the card so the client can label it honestly ("Ott hagytad abba" vs "A
polcodról" vs "Meglepetés") instead of guessing from the shape of the payload — a kicker that says
"where you left off" over a film the user has never opened is exactly the kind of quiet lie the house
rules call a dishonest state.
Budgets are matched against REMAINING time throughout (`budget.remaining_seconds`, which equals the
runtime for anything not yet started): with eight minutes left, a 90-minute film is a ten-minute
answer.
"""
from sqlalchemy.orm import Session
from app.discover import cards, pins, rails
from app.discover.budget import fits
from app.models import User
HERO_TIERS = ("resume", "comfort", "surprise")
# How deep the surprise tier draws before giving up. Deliberately deeper than one, so an excluded
# pick moves the tier along instead of emptying it (the shuffle is seeded — the same seed always
# hands back the same order, so without the headroom "already shown" would mean "nothing left").
SURPRISE_DRAW = 12
def pick(
db: Session,
user: User,
budget: str,
seed: int,
*,
resume: list[dict] | None = None,
shelf: list[dict] | None = None,
exclude: set[str] | None = None,
start: int = 0,
) -> dict | None:
"""`{"tier": ..., "card": ...}`, or None when nothing at all fits.
`resume` and `shelf` may be passed pre-computed so the consolidated `/api/discover` response does
not build the same two rails twice; the reroll endpoint passes neither and pays only for what it
reaches.
`exclude` is what makes "Mást kérek" mean something. A new seed alone only re-draws the SURPRISE
tier: the resume tier deterministically returns the most recently progressed thing that fits, so
pressing reroll on a user who has something half-finished handed back the identical card and the
button looked dead (measured in the browser, 2026-09-06). Skipping the cards already shown makes
every press produce a different answer, at every tier.
`start` ROTATES which tier is tried first — 0 keeps the priority order (the page's own first
answer), 1 leads with comfort, 2 with surprise. Exclusion alone was not enough: the resume rail
holds up to 20 rows, so the button walked down it and never reached the shelf or a surprise at
all (measured, 2026-09-06: twelve presses returned nine cards, every one of them `resume`). The
fallthrough is kept, so a press does not come back empty just because the tier it led with has
nothing to offer."""
skip = exclude or set()
def _resume() -> dict | None:
for card in rails.continue_watching(db, user) if resume is None else resume:
playable = rails.playable(db, user.id, card)
# Excluded on the card the USER saw (the show), not only on what it resolves to, so a
# rerolled-past show does not come straight back as its own next episode.
if cards.key(card) in skip or (playable is not None and cards.key(playable) in skip):
continue
if playable is not None and fits(playable["remaining_seconds"], budget):
return playable
return None
def _comfort() -> dict | None:
for card in pins.playables(pins.shelf(db, user) if shelf is None else shelf):
if cards.key(card) not in skip and fits(card["remaining_seconds"], budget):
return card
return None
def _surprise() -> dict | None:
# A wider draw than the one card we need: the shuffle is seeded, so without the headroom an
# excluded pick would leave the tier empty rather than move past it.
for card in rails.surprise(db, user, budget, seed, limit=SURPRISE_DRAW):
if cards.key(card) not in skip:
return card
return None
tiers = {"resume": _resume, "comfort": _comfort, "surprise": _surprise}
offset = start % len(HERO_TIERS)
for tier in HERO_TIERS[offset:] + HERO_TIERS[:offset]:
card = tiers[tier]()
if card is not None:
return {"tier": tier, "card": card}
return None
+173
View File
@@ -0,0 +1,173 @@
"""The comfort shelf: pin a thing you can always fall back to, and have it be one click from playing.
Five kinds are pinnable — a video, a YouTube channel, a Plex show, a Plex movie, a playlist — because
comfort media is almost always "put on X", a series or a channel, and only sometimes one particular
video. `models.ComfortPin` carries that union as five nullable FKs under a `num_nonnulls(...) = 1`
check; this module is where the union is spoken about in one vocabulary, so no call site invents a
second one.
Every shelf card carries a resolved `next`: the thing that actually plays when you click the tile —
a show's in-progress-or-next episode, a channel's newest unwatched video, a playlist's first
unwatched item. It costs a query per non-leaf pin, which is affordable precisely because this shelf is
short and hand-curated, and it buys the property the whole surface exists for: one click, no
intermediate page, no second decision.
Domain module, so nothing here raises HTTP. `resolve_ref` returns None for "no such thing, or not
yours" and the route turns that into its 404 — core never imports routes.
"""
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.discover import cards, queries
from app.models import Channel, ComfortPin, PlexItem, PlexShow, Playlist, User, Video
from app.plex import rollup
from app.plex.cards import leaf_card, show_card
from app.plex.lookups import by_rating_key
# kind -> the ComfortPin column that holds it. The client's vocabulary and the schema's, in one place.
PIN_COLUMNS: dict[str, str] = {
"video": "video_id",
"channel": "channel_id",
"plex_item": "plex_item_id",
"plex_show": "plex_show_id",
"playlist": "playlist_id",
}
PIN_KINDS = tuple(PIN_COLUMNS)
def resolve_ref(db: Session, user: User, kind: str, ref: str) -> tuple[str, object] | None:
"""Translate the client's (kind, ref) into the (column, value) the row stores, or None.
The client speaks the ids the rest of the API speaks: a video id, a channel id, a Plex
`rating_key`, a playlist id. Plex's internal integer keys never leave the backend, so the
translation belongs here rather than in five call sites.
The COLUMN half of every answer comes from `PIN_COLUMNS`; only the LOOKUP differs per kind
(a direct `db.get`, a rating-key translation, an int parse plus an ownership check). Before the
E24 review the branches spelled the column names out again and `PIN_COLUMNS` had no readers at
all — a constant that called itself the one vocabulary while nothing spoke it."""
col = PIN_COLUMNS.get(kind)
if col is None:
return None
if kind == "video":
return (col, ref) if db.get(Video, ref) is not None else None
if kind == "channel":
return (col, ref) if db.get(Channel, ref) is not None else None
if kind == "plex_item":
it = by_rating_key(db, PlexItem, ref)
return (col, it.id) if it is not None else None
if kind == "plex_show":
sh = by_rating_key(db, PlexShow, ref)
return (col, sh.id) if sh is not None else None
try:
pid = int(ref)
except ValueError:
return None
pl = db.get(Playlist, pid)
# A playlist is per-user, unlike the other four (which are shared catalog rows whose PRIVACY
# lives in the state tables). Someone else's list must read as "no such thing", not as a 403 —
# the same reason `owned_or_404` 404s rather than 403s.
return (col, pl.id) if pl is not None and pl.user_id == user.id else None
def list_pins(db: Session, user: User) -> list[ComfortPin]:
return list(
db.execute(
select(ComfortPin)
.where(ComfortPin.user_id == user.id)
.order_by(ComfortPin.position.asc(), ComfortPin.id.asc())
)
.scalars()
.all()
)
def next_position(db: Session, user: User) -> int:
"""New pins land at the end of the shelf.
`max(position) + 1`, not the row count. Positions are never renumbered on removal — `remove_pin`
deletes the row, and every FK cascade silently removes a pin when the thing it points at goes
away — so past the first hole a count under-counts and would drop the new pin into the MIDDLE of
the shelf, or onto a position another pin already holds (nothing in the schema forbids that).
Same idiom as `routes/playlists._next_playlist_position`."""
return (
db.scalar(
select(func.coalesce(func.max(ComfortPin.position), -1)).where(
ComfortPin.user_id == user.id
)
)
+ 1
)
def _shelf_card(db: Session, user: User, pin: ComfortPin) -> dict | None:
"""One pin as a unified card with its resolved `next`. None when the pinned row has gone (the FK
cascade removes the pin itself, so this is only the window between a delete and a refetch)."""
card: dict | None = None
nxt: dict | None = None
if pin.video_id is not None:
row = queries.yt_video_row(db, user, pin.video_id)
if row is None:
return None
card = cards.from_video(row)
nxt = dict(card) # a video plays itself
elif pin.channel_id is not None:
ch = db.get(Channel, pin.channel_id)
if ch is None:
return None
card = cards.from_channel(ch, queries.channel_unwatched_count(db, user, ch.id))
row = queries.channel_next_video(db, user, ch.id)
nxt = cards.from_video(row) if row else None
elif pin.plex_item_id is not None:
it = db.get(PlexItem, pin.plex_item_id)
if it is None:
return None
st = rollup.item_state_map(db, user.id, [it.id]).get(it.id)
card = cards.from_plex(leaf_card(it, st))
nxt = dict(card) # a movie or a single episode plays itself
elif pin.plex_show_id is not None:
sh = db.get(PlexShow, pin.plex_show_id)
if sh is None:
return None
status = rollup.state_map(db, user.id, [sh.id]).get(sh.id, "new")
card = cards.from_plex(show_card(sh, status))
ep = queries.next_episode(db, user.id, sh.id)
nxt = cards.from_plex(ep) if ep else None
elif pin.playlist_id is not None:
pl = db.get(Playlist, pin.playlist_id)
if pl is None:
return None
count, row = queries.playlist_summary(db, user, pl.id)
card = cards.from_playlist(pl, count, row.get("thumbnail_url") if row else None)
nxt = cards.from_video(row) if row else None
if card is None:
return None
# A COPY, never the card itself: a self-playing pin (a video, a movie) would otherwise make
# `card["next"] is card`, which serializes as a circular reference and 500s the whole shelf. The
# copy is taken before `pin_id`/`next` are attached, so `next` stays a plain playable card.
card["pin_id"] = pin.id
card["next"] = nxt
return card
def shelf(db: Session, user: User) -> list[dict]:
"""The whole shelf, in the user's own order."""
out = []
for pin in list_pins(db, user):
card = _shelf_card(db, user, pin)
if card is not None:
out.append(card)
return out
def playables(shelf_cards: list[dict]) -> list[dict]:
"""The `next` of every shelf entry that has one — what the hero may draw from. A pinned channel
with nothing unwatched left, or an empty playlist, simply is not an answer right now."""
return [c["next"] for c in shelf_cards if c.get("next")]
+348
View File
@@ -0,0 +1,348 @@
"""The row fetchers behind the Home surface — one place where "what is startable right now" is
expressed as SQL, for both sources.
Two rules shape this module:
**Reuse the query layer, not the route.** The YouTube half calls `feedquery.filtered_query` /
`sort_keys`, so the Home surface inherits the feed's own definitions of subscribed, blocked,
explore-visible and in-progress for free, and cannot drift from them. The Plex half calls
`plex.rollup` and the `plex.cards` serializers rather than `routes/plex/discover.py`: that route
narrows EVERY rail by the Plex sidebar's `LibraryFilters`, which is right for a surface that HAS a
sidebar and wrong for a landing page that does not.
**One seed drives both halves.** The feed orders a shuffle by `md5(id || seed)` so a keyset cursor
stays valid across pages; the Plex surprise rail uses `ORDER BY random()`, which is fine for a single
un-paged draw but cannot be reproduced. A cross-source reroll needs both halves to move together —
otherwise "Mást kérek" re-draws the Plex picks while the YouTube ones stand still — so the Plex side
here takes the feed's seeded-md5 idiom instead of `random()`.
"""
from sqlalchemy import and_, func, or_, select
from sqlalchemy.orm import Session, aliased
from app.discover.budget import band
from app.feedquery import (
feed_columns,
filtered_query,
not_blocked_cond,
order_by,
serialize,
sort_keys,
)
from app.models import (
Channel,
PlaylistItem,
PlexItem,
PlexShow,
PlexState,
User,
Video,
VideoState,
)
from app.plex import rollup
from app.plex.cards import leaf_card, movie_card, show_card
from app.plex.query import apply_show_state_filter, content_libs
# How many rows a single rail may carry. Matches `routes/plex/discover.py`'s RAIL_LIMIT on purpose —
# the two surfaces should feel like the same app, and a rail is a rail.
RAIL_LIMIT = 20
# The feed parameters a Home rail runs with. Spelled out rather than borrowed from the route's
# `_feed_params` defaults, because those are an HTTP contract that may change for the feed's own
# reasons; the landing page's scope should not move with them silently.
_FEED_BASE: dict = {
"tags": [],
"tag_modes": {},
"untagged": False,
"channel_ids": [],
"q": None,
"min_duration": None,
"max_duration": None,
"max_age_days": None,
"published_after": None,
"published_before": None,
"show_normal": True,
"include_shorts": False,
"include_live": False,
"show": "unwatched",
"scope": "my",
"library_source": "organic",
}
def _seeded(col, seed: int):
"""The feed's deterministic shuffle expression, reused for Plex so one seed moves both halves."""
return func.md5(func.concat(col, str(seed)))
# ── YouTube ───────────────────────────────────────────────────────────────────────────────────────
def yt_in_progress(db: Session, user: User, limit: int = RAIL_LIMIT) -> list[tuple[object, dict]]:
"""Started-but-not-finished videos, most recently progressed first, as (progressed_at, row).
The predicate is `feedquery`'s `show="in_progress"` verbatim (a stored resume position, status not
watched/hidden) — but the query is built here rather than through `filtered_query` because this
rail wants an ordering by `progress_updated_at`, and that column lives on the state alias the feed
query keeps private. Driving from `video_states` is also the faster plan: it is a per-user table of
a few hundred rows, against a catalog of many thousands.
Building it here means the feed's OTHER predicates have to be chosen deliberately rather than
inherited. `not_blocked_cond` is shared, because "never show me this channel" outranks "you
started it" — blocking leaves the state row untouched, so without this the video the user just
banned heads the resume rail. The feed's SCOPE clauses (subscribed / via_explore accessibility)
are deliberately NOT applied, for the reason `yt_video_row` gives: something you have already
started must stay resumable even after you unsubscribe or an exploration expires."""
st = aliased(VideoState)
q = (
select(*feed_columns(user, st), st.progress_updated_at.label("progressed_at"))
.join(Channel, Channel.id == Video.channel_id)
.join(st, and_(st.video_id == Video.id, st.user_id == user.id))
.where(
st.position_seconds > 0,
st.status.notin_(("watched", "hidden")),
Video.unavailable_since.is_(None),
not_blocked_cond(user),
)
.order_by(st.progress_updated_at.desc().nullslast())
.limit(limit)
)
return [(r.progressed_at, serialize(r)) for r in db.execute(q).all()]
def yt_surprise(db: Session, user: User, budget: str, seed: int, limit: int) -> list[dict]:
"""Unwatched videos from the user's own feed scope, in seeded-shuffle order, inside the budget.
Shorts are opened up only for the shortest band: "I have ten minutes" is the one answer a Short is
actually a good response to, and the feed's own default keeps them out otherwise."""
lo, hi = band(budget)
params = dict(_FEED_BASE, min_duration=lo, max_duration=hi, include_shorts=budget == "short")
query, rank_expr = filtered_query(db, user, **params)
keys = sort_keys("shuffle", seed, rank_expr)
# Over-fetch, then drop anything already started. The feed's `unwatched` scope keeps watched and
# hidden out but deliberately keeps in-progress rows in (they belong in the feed). Here they must
# go: the resume tier has already had its chance at them, and it may have passed precisely because
# the REMAINING time did not fit the budget — offering the same item back as a surprise, whose
# runtime does fit, would label a half-finished thing "Meglepetés" and contradict the question.
rows = db.execute(query.order_by(*order_by(keys)).limit(limit * 4)).all()
out = [serialize(r) for r in rows]
return [r for r in out if not r["position_seconds"]][:limit]
def channel_next_video(db: Session, user: User, channel_id: str) -> dict | None:
"""The newest unwatched video of a pinned channel — what "put on that channel" resolves to."""
# `library_source="all"`: a pinned channel means the whole channel, including videos that only
# ever arrived through a search. The feed's "organic" default is a feed-scoping choice, not a
# statement about what belongs to a channel.
params = dict(_FEED_BASE, channel_ids=[channel_id], scope="all", library_source="all")
query, rank_expr = filtered_query(db, user, **params)
keys = sort_keys("newest", 0, rank_expr)
row = db.execute(query.order_by(*order_by(keys)).limit(1)).first()
return serialize(row) if row is not None else None
def yt_video_row(db: Session, user: User, video_id: str) -> dict | None:
"""One video as a feed row, with this user's watch state.
Built directly rather than through `filtered_query` on purpose: a PINNED video must keep working
even when it falls outside the feed's scope — the user unsubscribed from the channel, or blocked
it, or the video only ever arrived through a search. The shelf is the surface you fall back to;
it silently losing an entry because a filter moved is exactly the failure it must not have."""
st = aliased(VideoState)
row = db.execute(
select(*feed_columns(user, st))
.join(Channel, Channel.id == Video.channel_id)
.outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
.where(Video.id == video_id)
).first()
return serialize(row) if row is not None else None
def channel_unwatched_count(db: Session, user: User, channel_id: str) -> int:
"""How many unwatched videos a pinned channel is holding — the tile's only honest promise, since a
channel has no runtime of its own."""
params = dict(_FEED_BASE, channel_ids=[channel_id], scope="all", library_source="all")
query, _ = filtered_query(db, user, **params)
return int(db.execute(select(func.count()).select_from(query.subquery())).scalar() or 0)
def playlist_summary(db: Session, user: User, playlist_id: int) -> tuple[int, dict | None]:
"""(item count, the row to play next) for a pinned playlist — its first not-yet-watched item, or
its first item once the whole list is watched. The same row doubles as the tile's artwork, so a
shelf of playlists is scannable instead of a row of blank boxes."""
st = aliased(VideoState)
rows = db.execute(
select(*feed_columns(user, st), PlaylistItem.position)
.join(PlaylistItem, PlaylistItem.video_id == Video.id)
.join(Channel, Channel.id == Video.channel_id)
.outerjoin(st, and_(st.video_id == Video.id, st.user_id == user.id))
.where(PlaylistItem.playlist_id == playlist_id)
.order_by(PlaylistItem.position.asc())
).all()
if not rows:
return 0, None
nxt = next((r for r in rows if (r.status or "new") not in ("watched", "hidden")), rows[0])
return len(rows), serialize(nxt)
# ── Plex ──────────────────────────────────────────────────────────────────────────────────────────
def _unseen_movie_conds(uid: int):
"""A movie the user has neither finished nor hidden. A LEFT-JOIN-IS-NULL would also drop rows that
merely carry a resume position, which are exactly the ones Continue watching wants back."""
seen = (
select(1)
.select_from(PlexState)
.where(
PlexState.item_id == PlexItem.id,
PlexState.user_id == uid,
or_(
PlexState.status.in_(("watched", "hidden")),
# Started counts as "not a surprise" too — see the note in `yt_surprise`.
PlexState.position_seconds > 0,
),
)
.exists()
)
return ~seen
def plex_in_progress(db: Session, uid: int, limit: int = RAIL_LIMIT) -> list[tuple[object, dict]]:
"""In-progress movies + partially-watched shows as (progressed_at, card).
Mirrors `routes/plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
NULL status still counts as started, so `notin_` alone would drop it), and the same shared
`rollup.agg_subquery` + `apply_show_state_filter` for shows, so the two rails cannot disagree on
what "in progress" means. What it does NOT inherit is that route's `LibraryFilters` narrowing:
this surface has no Plex sidebar to read a filter from."""
movie_libs, show_libs = content_libs(db)
out: list[tuple[object, dict]] = []
if movie_libs:
mq = (
select(PlexItem, PlexState)
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
.where(
PlexItem.kind == "movie",
PlexItem.library_id.in_(movie_libs),
PlexState.position_seconds > 0,
or_(
PlexState.status.is_(None),
PlexState.status.notin_(("watched", "hidden")),
),
)
.order_by(PlexState.progress_updated_at.desc().nullslast())
.limit(limit)
)
for it, st in db.execute(mq).all():
out.append((st.progress_updated_at, movie_card(it, st)))
if show_libs:
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
sq = apply_show_state_filter(
select(PlexShow, prog.c.resumed_at).join(prog, prog.c.show_id == PlexShow.id),
prog.c.total,
prog.c.watched,
prog.c.inprog,
"in_progress",
).where(PlexShow.library_id.in_(show_libs))
for sh, resumed_at in db.execute(
sq.order_by(prog.c.resumed_at.desc().nullslast()).limit(limit)
).all():
out.append((resumed_at, show_card(sh, "in_progress")))
return out
def plex_surprise(db: Session, uid: int, budget: str, seed: int, limit: int) -> list[dict]:
"""Unseen playable movies inside the budget, in seeded-shuffle order.
Movies only. A show is not a thing you can start playing, and resolving every candidate show to
its next episode just to draw three cards would be a query per row for a rail whose whole value is
that it answers instantly — shows reach this surface through Continue watching and the comfort
shelf, which is where "put the next episode on" actually belongs.
No `playable` filter, matching the Plex surprise rail: `PlexItem.playable` is a playback CLASS
("direct" / "remux" / "transcode"), not a yes/no, and NULL there means "not probed yet" rather than
"will not play". Excluding un-probed items would quietly hide a fresh library from the one surface
that is supposed to have an answer. The class travels on the card, so the client still shows it."""
movie_libs, _ = content_libs(db)
if not movie_libs:
return []
lo, hi = band(budget)
q = (
select(PlexItem)
.where(
PlexItem.kind == "movie",
PlexItem.library_id.in_(movie_libs),
_unseen_movie_conds(uid),
)
.order_by(_seeded(PlexItem.rating_key, seed))
.limit(limit)
)
if lo is not None:
q = q.where(PlexItem.duration_s >= lo)
if hi is not None:
q = q.where(PlexItem.duration_s <= hi)
items = db.execute(q).scalars().all()
states = rollup.item_state_map(db, uid, [it.id for it in items])
return [movie_card(it, states.get(it.id)) for it in items]
# Watch states that take an episode out of the running for "what would play next".
_DONE = ("watched", "hidden")
def next_episode(db: Session, uid: int, show_id: int) -> dict | None:
"""What "put that show on" means: the episode in progress, else the first unwatched one, else the
first — over the same `(season, episode)` broadcast ordering `detail.py::_episode_nav` uses.
Three ordered `LIMIT 1` probes rather than one scan picked apart in Python. The Python version
loaded EVERY episode of the show to choose one of them — 236 rows for Friends, 227 for JAG — which
the comfort shelf could just about afford (a handful of hand-picked pins) and the resume rail
could not, since it resolves a show per rail row on the landing page.
Returned as a `leaf_card` so it carries the show title the episode row itself does not."""
asc = (
PlexItem.season_number.asc().nullsfirst(),
PlexItem.episode_number.asc().nullsfirst(),
)
base = (
select(PlexItem, PlexState)
.outerjoin(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
.where(PlexItem.show_id == show_id, PlexItem.kind == "episode")
.limit(1)
)
# In progress — the LAST one in broadcast order, which is `plex.rollup.rollup`'s `inprog[-1]`,
# i.e. what the show page's own Resume button plays. Two episodes of one show are in progress at
# once more often than it sounds (`watch_sync._reconcile_inprogress_movies` documents that a
# scrub to the end leaves a stale `status='new', position>0` row until the daily reconcile), and
# taking opposite ends of that set is how one show came to have two different "continue" answers:
# the Home tile ranked it by its LATEST touch and then offered the EARLIEST started episode.
# `position_seconds > 0` already requires the state row to exist, so no null guard.
pick = db.execute(
base.where(PlexState.position_seconds > 0, PlexState.status.notin_(_DONE)).order_by(
PlexItem.season_number.desc().nullslast(),
PlexItem.episode_number.desc().nullslast(),
)
).first()
# Unwatched — INCLUDING an episode with no state row at all, which is the common case and the one
# a bare `status NOT IN (...)` silently drops (NULL is not false, it is unknown, so the row fails
# the filter).
pick = (
pick
or db.execute(
base.where(or_(PlexState.id.is_(None), PlexState.status.notin_(_DONE))).order_by(*asc)
).first()
)
# Fully watched show: start it again from the top.
pick = pick or db.execute(base.order_by(*asc)).first()
if pick is None:
return None
ep, st = pick
show = db.get(PlexShow, show_id)
return leaf_card(
ep, st, show_title=show.title if show else None, show_rk=show.rating_key if show else None
)
+97
View File
@@ -0,0 +1,97 @@
"""The Home surface's rails: cross-source lists, assembled from `queries.py` and emitted as unified
cards.
The whole point of this module is that a rail is ONE list, not a YouTube list beside a Plex list. Two
side-by-side rails would just move the choice the user could not make ("which half do I look at?") one
level up, which is the opposite of what this epic is for.
"""
from sqlalchemy.orm import Session
from app import sysconfig
from app.discover import cards, queries
from app.discover.queries import RAIL_LIMIT
from app.models import PlexShow, User
from app.plex.lookups import by_rating_key
from app.plex.rollup import NO_TS
def plex_available(db: Session) -> bool:
"""Plex is a deployment-level toggle, not a per-user one (see `routes/me.py`). With it off, the
Home rails are YouTube-only and say nothing about Plex."""
return sysconfig.get_bool(db, "plex_enabled")
def playable(db: Session, uid: int, card: dict) -> dict | None:
"""The thing that would actually START from this card — itself for anything with a runtime, and
for a Plex SHOW the episode that is next (in progress, else the first unwatched, else the first).
ONE function, because three surfaces need the same answer and a second implementation is how the
shelf and the rail came to disagree in the first place: the hero calls it to decide what it is
offering, `continue_watching` calls it to fill each show tile's `next`, and the comfort shelf
resolves the same way in `pins`. None when the show has no episodes mirrored yet.
A card that already carries a resolved `next` (a rail row) is taken at its word rather than
re-queried — the hero reads the rail it was handed."""
if not (card.get("source") == "plex" and card.get("kind") == "show"):
return card
if "next" in card:
return card["next"]
sh = by_rating_key(db, PlexShow, card["id"])
if sh is None:
return None
ep = queries.next_episode(db, uid, sh.id)
return cards.from_plex(ep) if ep else None
def continue_watching(db: Session, user: User, limit: int = RAIL_LIMIT) -> list[dict]:
"""One resume rail across both sources, most recently progressed first.
A Plex SHOW enters as a show CARD — that is what the Plex rail shows, and it is the artwork the
user recognises — but it carries `next`: the episode that would actually start. Without it a
click on a half-watched series landed on the show page with six seasons to choose between, which
is the decision this whole surface exists to remove (the comfort shelf and the hero already
resolved; only this rail did not). The client plays `next ?? card`, so the tile stays a poster and
the click stays one click."""
rows: list[tuple[object, dict]] = [
(ts, cards.from_video(r)) for ts, r in queries.yt_in_progress(db, user, limit)
]
if plex_available(db):
rows += [(ts, cards.from_plex(c)) for ts, c in queries.plex_in_progress(db, user.id, limit)]
rows.sort(key=lambda r: r[0] or NO_TS, reverse=True)
out = [card for _, card in rows[:limit]]
# Resolve AFTER the slice: only the rows that survive it are worth a query.
for card in out:
if card["kind"] == "show":
card["next"] = playable(db, user.id, card)
return out
def surprise(db: Session, user: User, budget: str, seed: int, limit: int = 3) -> list[dict]:
"""The hero's draw pool: unseen things that fit the budget, from both sources under ONE seed.
Interleaved rather than concatenated — the same alternation `routes/plex/discover.py` uses for its
movie/show split — so a short pool never comes back all-YouTube or all-Plex just because one side
happened to be queried first.
WHICH source leads is decided by the seed, not by a hardcoded order. The hero draws a pool of ONE,
so a fixed order would mean the Plex library could never answer "egy este" while a single YouTube
video also fit — the reroll would look broken and the cross-source promise would quietly be a
YouTube promise. Parity on the same seed that orders the rows keeps it deterministic: the same
seed always gives the same answer, and pressing "Mást kérek" moves both halves together."""
halves = [
(cards.from_video, queries.yt_surprise(db, user, budget, seed, limit)),
(
cards.from_plex,
queries.plex_surprise(db, user.id, budget, seed, limit) if plex_available(db) else [],
),
]
if seed % 2:
halves.reverse()
out: list[dict] = []
for i in range(max(len(rows) for _, rows in halves)):
for to_card, rows in halves:
if i < len(rows):
out.append(to_card(rows[i]))
return out[:limit]
+101 -4
View File
@@ -11,10 +11,22 @@ Resolution works both before extraction (from source_kind / URL host) and after
`extractor_key` in the info dict), so a non-YouTube "url" job still lands on the right adapter.
"""
import re
from urllib.parse import urlparse
import yt_dlp
from yt_dlp.networking.impersonate import ImpersonateTarget
from app.config import settings
# Facebook's "Cannot parse data": the page yt-dlp got was a wall, not the content. Both the Chrome retry
# below and the failure catalog (errors.py) decide on it, so the wording exists once.
FACEBOOK_PARSE_WALL = "cannot parse data"
# A source's own verdicts on a link, raised from `resolve_url`. errors.py classifies them; it imports
# them from here because this module must not import errors (errors → credentials → adapters).
POST_WITHOUT_VIDEO_MESSAGE = "No single video was found in this post."
POST_NOT_SHOWN_MESSAGE = "Facebook did not show this post to the signed-in session."
class SourceAdapter:
"""Generic source: yt-dlp handles everything and the ref is already a fetchable URL."""
@@ -22,9 +34,21 @@ class SourceAdapter:
key = "generic"
# --- extraction (worker._extract_with_fallback / backfill) ---------------------------------
def client_sets(self) -> list:
"""Player-client sets to attempt, in order. A generic source needs no client override, so a
single default attempt (`[None]`) — the per-client fallback is a YouTube-only concept."""
def resolve_url(self, url: str, ydl_opts: dict) -> str:
"""The link to hand yt-dlp for `url`, resolved BEFORE the first attempt. `ydl_opts` are the
download's own options (cookie, timeouts, address family) for any page this needs to read.
Raises a RuntimeError carrying one of this module's verdicts when the page read shows the
download cannot go ahead: the content is there but holds no single video
(`POST_WITHOUT_VIDEO_MESSAGE`), or the page is not the content at all — the login page a
refused cookie gets (`POST_NOT_SHOWN_MESSAGE`), never the no-video verdict. A failed page read
is raised as itself, so it is classified like any other download error. Generic: `url`."""
return url
def client_sets(self, authed: bool = False) -> list:
"""Attempts to make, in order — each entry is handed back to `ydl_opts_extra`, and the next one
is tried when `retriable` says so. A generic source makes a single default attempt (`[None]`).
`authed` is whether the acting user's cookie file goes with the download."""
return [None]
def ydl_opts_extra(self, clients) -> dict:
@@ -67,7 +91,7 @@ class YouTubeAdapter(SourceAdapter):
"Sign in to confirm",
)
def client_sets(self) -> list:
def client_sets(self, authed: bool = False) -> list:
return [
c for c in (settings.player_client_list, settings.player_client_fallback_list) if c
] or [None]
@@ -93,6 +117,46 @@ class YouTubeAdapter(SourceAdapter):
class FacebookAdapter(SourceAdapter):
key = "facebook"
# Measured (E11.5, yt-dlp 2026.07.04): with the user's cookie, some reels answer a plain request
# with "Cannot parse data" and parse only when the request looks like Chrome. Impersonation is a
# SECOND attempt behind that one error, not the default: the links that already download with the
# cookie keep their proven path, and only the ones stuck at the wall go through curl_cffi. Without
# a cookie the same error is the login wall, which no disguise gets past.
_IMPERSONATE = "impersonate"
# A post page is ~4–5 MB of inline JSON; anything far past that is not a post page.
_POST_PAGE_MAX_BYTES = 16 * 1024 * 1024
def resolve_url(self, url: str, ydl_opts: dict) -> str:
# Measured (E11.5 S4): given a group post's link, yt-dlp either finds no formats or — worse —
# downloads ANOTHER video from the page's feed data. The post's own video is the one Video node
# in the page scripts that carry the post id; its watch link downloads (plain first, then the
# Chrome retry above). Needs the cookie: signed out, the page is the login wall.
post_id = facebook_group_post_id(url)
if post_id is None or not ydl_opts.get("cookiefile"):
return url
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
raw = ydl.urlopen(url).read(self._POST_PAGE_MAX_BYTES)
page = raw.decode("utf-8", "replace")
# Measured (E11.5 review): a refused or expired cookie gets the login page (~450 KB, a login
# form, no post id at all), which is not a post without a video — the user's cookie is the cause.
if _quoted(post_id) not in page:
raise RuntimeError(POST_NOT_SHOWN_MESSAGE)
video_id = facebook_post_video_id(page, post_id)
if video_id is None:
# Never fall back to the post link: that is exactly where the wrong video comes from.
raise RuntimeError(POST_WITHOUT_VIDEO_MESSAGE)
return f"https://www.facebook.com/watch/?v={video_id}"
def client_sets(self, authed: bool = False) -> list:
return [None, self._IMPERSONATE] if authed else [None]
def ydl_opts_extra(self, clients) -> dict:
if clients == self._IMPERSONATE:
return {"impersonate": ImpersonateTarget.from_str("chrome")}
return {}
def retriable(self, error_text: str) -> bool:
return FACEBOOK_PARSE_WALL in error_text.lower()
def uploader_url(self, info: dict) -> str | None:
# Facebook gives neither channel_url nor uploader_url, but exposes a numeric `uploader_id`
@@ -107,6 +171,39 @@ class FacebookAdapter(SourceAdapter):
return True
_FB_GROUP_POST_PATH = re.compile(r"^/groups/[^/]+/(?:permalink|posts)/(\d+)/?$")
_SCRIPT_BLOCK = re.compile(r"<script[^>]*>(.*?)</script>", re.S)
_VIDEO_NODE = re.compile(r'"__typename":"Video","id":"(\d+)"')
def _quoted(post_id: str) -> str:
# How the page JSON carries the id: quoted, so a longer id that merely starts with it never counts.
return f'"{post_id}"'
def facebook_group_post_id(url: str) -> str | None:
"""The post id of a Facebook group post link (`/groups/<group>/permalink|posts/<id>/`), else None."""
parsed = urlparse(url)
if not _host_is(parsed.netloc.lower(), "facebook.com"):
return None
match = _FB_GROUP_POST_PATH.match(parsed.path)
return match.group(1) if match else None
def facebook_post_video_id(page: str, post_id: str) -> str | None:
"""The id of the one video attached to `post_id`, read from its page: the Video nodes of the script
blocks that carry the post id (the page's other ~10 videos — feed, reels — sit in other blocks).
None for no video AND for more than one: guessing between two would be the wrong-video bug again."""
quoted = _quoted(post_id)
ids = {
vid
for block in _SCRIPT_BLOCK.findall(page)
if quoted in block
for vid in _VIDEO_NODE.findall(block)
}
return ids.pop() if len(ids) == 1 else None
class RedditAdapter(SourceAdapter):
key = "reddit"
+53 -16
View File
@@ -1,10 +1,12 @@
"""Per-user, per-host download cookies — the single source of truth for "does the acting user's
cookie apply to this download, and how does that keep the result private".
Two call sites depend on ONE consistent answer (house rule: an invariant lives in shared code, not
Two sides depend on ONE consistent answer (house rule: an invariant lives in shared code, not
in prose):
* `enqueue` salts the MediaAsset cache signature with the user id WHEN a cookie applies, so the
authed download becomes a PRIVATE asset that is never cache-served to another user.
authed download becomes a PRIVATE asset that is never cache-served to another user. Retry
(`service.resume`) and Re-download (`service.redownload`) ask the same question again, so a
cookie the user added after a failure goes with the next attempt.
* the worker attaches the cookie file to yt-dlp ONLY when a cookie still applies AND the asset it
is downloading into is that private, salted identity — so a cookie can never poison a shared
asset (the load-bearing cross-user privacy fix).
@@ -30,7 +32,7 @@ from sqlalchemy.orm import Session
from app.downloads import formats
from app.downloads.adapters import _host_is
from app.models import DownloadCredential
from app.models import DownloadCredential, MediaAsset
from app.security import decrypt_optional, encrypt
log = logging.getLogger("siftlode.downloads.credentials")
@@ -104,6 +106,29 @@ def authed_format_sig(spec: dict, user_id: int, host_key: str) -> str:
return formats.format_sig(spec, salt=cookie_salt(user_id, host_key))
def rebuild_salt(asset_id: int, cookie: str | None) -> str:
"""The salt of a Re-download's fresh asset: its OWN row id (so every rebuild is a new identity no
enqueue can ever compute), prefixed with the user's `cookie_salt` when a cookie applies. The id is
what makes it recognisable — the worker has the asset row, so `is_authed_asset` can recompute this
one, where a salt built from the job's previous asset id could not be."""
return f"{cookie}:rebuild:{asset_id}" if cookie else f"rebuild:{asset_id}"
def is_authed_asset(asset: MediaAsset, spec: dict, user_id: int, url: str | None) -> bool:
"""Whether an asset is `user_id`'s private, cookie-salted identity for `url`'s host — the one
kind of asset a cookie is ever attached to: the one enqueue creates (`authed_format_sig`), or a
Re-download's rebuild of it (`rebuild_salt`). The worker's cookie gate asks it before a download,
and the failure diagnosis asks it after one (a failure on this asset happened WITH the cookie)."""
host_key = host_key_for_url(url)
if host_key is None:
return False
if asset.format_sig == authed_format_sig(spec, user_id, host_key):
return True
return asset.id is not None and asset.format_sig == formats.format_sig(
spec, salt=rebuild_salt(asset.id, cookie_salt(user_id, host_key))
)
def salt_for_enqueue(db: Session, user_id: int, url: str | None) -> str | None:
"""At enqueue: the cache salt to apply if this user has a cookie for `url`'s host, else None (a
normal shared download). Returns the salt string so the caller passes it straight to
@@ -199,20 +224,19 @@ def status_list(db: Session, user_id: int) -> list[dict]:
# --- worker: resolve + write the cookie file --------------------------------------------------
def resolve_cookiefile(
db: Session,
user_id: int,
url: str | None,
asset_format_sig: str,
spec: dict,
staging: Path,
) -> Path | None:
"""In the worker child: the cookies.txt to hand yt-dlp for this download, or None.
def usable_cookie(
db: Session, user_id: int, url: str | None, asset: MediaAsset, spec: dict
) -> str | None:
"""The user's cookie text a download of `url` into `asset` goes with, or None — the ONE decision
whether a download is signed in. The worker child writes it out for yt-dlp (`resolve_cookiefile`);
the worker asks it again for a job that inherits a failed asset. That second ask sees the
credential as it is when the job is claimed: a cookie the user removed or added since the failed
download changes this job's explanation to fit the cookie its Retry would go with.
The double gate that prevents cross-user leakage:
1. a credential must still exist for `url`'s host (host-scoped via `host_key_for_url`), AND
2. the asset being downloaded into must BE this user's private, salted identity
(`asset_format_sig == authed_format_sig(...)`).
(`is_authed_asset`).
If (2) fails the asset is the SHARED (unsalted) one — attaching the cookie would poison the
global cache with a private-group video, so we attach nothing. An unreadable blob (rotated key →
@@ -224,18 +248,31 @@ def resolve_cookiefile(
cred = get_credential(db, user_id, host_key)
if cred is None:
return None
if asset_format_sig != authed_format_sig(spec, user_id, host_key):
if not is_authed_asset(asset, spec, user_id, url):
return None # shared asset — never attach a cookie to a non-private identity
raw = decrypt_optional(cred.cookies_enc)
if not raw:
log.warning(
"download cookie for user %s host %s could not be decrypted (rotated key?) — "
"proceeding without it",
"treated as no cookie",
user_id,
host_key,
)
return None
return _write_cookiefile(raw, staging)
return raw
def resolve_cookiefile(
db: Session,
user_id: int,
url: str | None,
asset: MediaAsset,
spec: dict,
staging: Path,
) -> Path | None:
"""In the worker child: the cookies.txt to hand yt-dlp for this download (`usable_cookie`), or None."""
raw = usable_cookie(db, user_id, url, asset, spec)
return _write_cookiefile(raw, staging) if raw else None
def _write_cookiefile(raw: str, staging: Path) -> Path:
+140 -5
View File
@@ -1,4 +1,5 @@
"""Classify a raw yt-dlp / ffmpeg failure string into a stable category code.
"""Classify a raw yt-dlp / ffmpeg failure string into a stable category code — and, where a site's
failure has been measured, into that site's entry of the failure catalog.
The worker stores the raw exception text (truncated) for technical detail, but a raw
`ERROR: [youtube] <id>: Private video. Sign in…` is noise to a user. `classify()` maps that text
@@ -6,21 +7,51 @@ to one of a small set of category codes; the frontend turns the code into a loca
explanation (and decides whether a retry makes sense). The raw text stays available behind a
"details" disclosure. This is the same string-matching approach the worker already trusts for
`_RETRIABLE_MARKERS` — kept here as the single source of truth for "what kind of failure is this".
**The failure catalog** (`_CATALOG`, E11.5). A category is generic: "Sign-in required" cannot tell a
Facebook user that the cookie they already gave was refused, and yt-dlp words several different
Facebook walls identically (`Cannot parse data`). A catalog entry is scoped to ONE host and may also
depend on whether a cookie was attached and on the kind of link, and names a `detail` code the UI has
its own sentence for — why it failed on that site, and what the user can do. `diagnose()` tries the
catalog first and falls back to the category. How an entry is added, every time a failure is reported:
1. reproduce it on dev (with and without the user's cookie) and keep the verbatim yt-dlp text;
2. add the entry below, scoped as narrowly as the measurement allows (a marker that is yt-dlp's
generic wording for every site must never match without its host; a marker another module also
decides on — the Facebook Chrome retry's FACEBOOK_PARSE_WALL — is imported, never retyped);
3. pin the real text in `tests/test_download_errors.py`;
4. add `downloads.errors.fail.catalog.<detail>.{short,body}` in EN and HU (a contract test checks it;
the group is not called `detail`, which `actionErrors.contract` reads as a raw `HttpError.detail`).
"""
# Two worker-authored strings that this module owns because both are coupled to `_RULES`.
from collections.abc import Callable
from dataclasses import dataclass
from typing import NamedTuple
from app.downloads.adapters import (
FACEBOOK_PARSE_WALL,
POST_NOT_SHOWN_MESSAGE,
POST_WITHOUT_VIDEO_MESSAGE,
facebook_group_post_id,
)
from app.downloads.credentials import host_key_for_url
# Worker-authored strings that this module owns because each is coupled to `_RULES`.
#
# QUOTA_FULL_MESSAGE is raised by the worker and must classify back to `quota_exceeded`; its rule
# below matches a fragment of it, and test_download_errors pins the pair, because a message and its
# rule drifting apart is exactly how a category silently becomes unreachable.
#
# NETWORK_REDACTED_MESSAGE is the opposite direction: it REPLACES a message after classification, so
# it deliberately has no rule. Nothing re-classifies a stored `job.error` today; if anything ever
# does, it must carry the original code rather than re-deriving one from this text.
# NETWORK_REDACTED_MESSAGE REPLACES a message after classification (`user_error_text`). It is written
# to the shared asset too, and a second job that inherits that failed asset re-diagnoses the stored
# text (`worker._process_job`), so it has a rule of its own that gives the category back.
#
# Other worker-raised texts are still inline literals at their raise sites. The ffmpeg ones and
# "produced no output file" have markers here; the urlguard refusal deliberately has none and
# classifies as `unknown` — it is a refusal, not a diagnosis, and no category would say more.
#
# A source adapter's verdicts on a link (`SourceAdapter.resolve_url`, E11.5 S4) are adapters.py's
# constants, imported above; each has a rule below that gives its category back, and a catalog entry
# that gives the site's sentence.
QUOTA_FULL_MESSAGE = "Your download storage is full — free up space, then try again."
NETWORK_REDACTED_MESSAGE = "The server could not reach that address."
@@ -30,6 +61,12 @@ _RULES: tuple[tuple[str, str], ...] = (
# Our own refusal: the worker found the user already at their storage cap (SEC-13). Placed
# first — it is our wording, so it can never be a false match on an upstream message.
("download storage is full", "quota_exceeded"),
# Our own redaction of a transport failure (see NETWORK_REDACTED_MESSAGE) — our wording too.
(NETWORK_REDACTED_MESSAGE.lower(), "network"),
# Our own verdicts on a post link (adapters.py) — our wording too: its page holds no single video,
# or the site answered the signed-in read with something that is not the post at all.
(POST_WITHOUT_VIDEO_MESSAGE.lower(), "format_unavailable"),
(POST_NOT_SHOWN_MESSAGE.lower(), "login_required"),
# Login / age / members / bot-check — the user must be signed in (we deliberately are not).
("sign in to confirm your age", "login_required"),
("confirm your age", "login_required"),
@@ -101,6 +138,83 @@ _RULES: tuple[tuple[str, str], ...] = (
)
class Diagnosis(NamedTuple):
"""What a failure was: the generic `code` (category) and, when a catalog entry matched, its
host-specific `detail`."""
code: str
detail: str | None = None
@dataclass(frozen=True)
class _Entry:
"""One measured failure of one site. It matches when the download URL belongs to `host`
(`credentials.SUPPORTED_HOSTS`), every one of `markers` is in the message, the cookie state is
`cookie` (None: either), and — when `link` is given — it recognises the URL (a truthy result). A
link predicate is the adapter's own, so the catalog and the download agree on what a link is."""
host: str
markers: tuple[str, ...]
code: str
detail: str
cookie: bool | None = None
link: Callable[[str], object] | None = None
_CATALOG: tuple[_Entry, ...] = (
# Facebook answers a signed-out client with a page yt-dlp cannot parse — an 18+ reel, a closed
# group's post (measured 2026-09-13: prod jobs 145, 156, 256 download once the owner's cookie is
# attached). "Cannot parse data" is yt-dlp's generic wording for a broken extractor on EVERY site,
# which is why the entry is scoped to Facebook and never becomes a general rule.
_Entry(
host="facebook",
markers=("[facebook]", FACEBOOK_PARSE_WALL),
code="login_required",
detail="facebook.login_wall",
cookie=False,
),
# The same text WITH a cookie: an expired / signed-out cookie, or a page that also needs
# impersonation (prod job 257). Telling this user to add cookies would be wrong — they have.
_Entry(
host="facebook",
markers=("[facebook]", FACEBOOK_PARSE_WALL),
code="login_required",
detail="facebook.cookie_rejected",
cookie=True,
),
# A group post read with a cookie that Facebook answered with its login page (measured with a
# refused cookie: no post id anywhere on the page) — the same refused cookie, found by the adapter.
_Entry(
host="facebook",
markers=(POST_NOT_SHOWN_MESSAGE.lower(),),
code="login_required",
detail="facebook.cookie_rejected",
cookie=True,
),
# A group post with no single video of its own. With a cookie the adapter reads the post's page
# and says so itself (POST_WITHOUT_VIDEO_MESSAGE); without one yt-dlp is given the post link and
# answers "No video formats found" (measured on prod jobs 66 and 145, whose videos are reshared
# clips). With a cookie yt-dlp only ever gets the post's VIDEO link, so the same text there is about
# that video, not the post — the category says it. Only for group post links (the adapter's own
# test of one): a reel with no formats is a different failure.
_Entry(
host="facebook",
markers=(POST_WITHOUT_VIDEO_MESSAGE.lower(),),
code="format_unavailable",
detail="facebook.group_post_video",
link=facebook_group_post_id,
),
_Entry(
host="facebook",
markers=("[facebook]", "no video formats found"),
code="format_unavailable",
detail="facebook.group_post_video",
cookie=False,
link=facebook_group_post_id,
),
)
def classify(msg: str | None) -> str:
"""Return a category code for a raw failure message (``"unknown"`` if nothing matches)."""
if not msg:
@@ -112,6 +226,27 @@ def classify(msg: str | None) -> str:
return "unknown"
def diagnose(
msg: str | None, *, url: str | None = None, cookie_attached: bool = False
) -> Diagnosis:
"""The catalog entry for this failure of this link, else just its category (see the module doc).
`url` is the address yt-dlp was given; `cookie_attached` whether the acting user's cookie went
with it. Without a URL only the category can be known."""
host = host_key_for_url(url) if msg and url else None
if host is not None:
low = msg.lower() # type: ignore[union-attr] — `host` is only set when `msg` is truthy
for e in _CATALOG:
if (
e.host == host
and all(m in low for m in e.markers)
and (e.cookie is None or e.cookie == cookie_attached)
and (e.link is None or e.link(url)) # type: ignore[arg-type] — as `msg` above
):
return Diagnosis(e.code, e.detail)
return Diagnosis(classify(msg))
def user_error_text(source_kind: str, msg: str, code: str) -> str:
"""The failure text to STORE on the job, given the raw message and its category.
+45 -19
View File
@@ -8,7 +8,6 @@ from datetime import date, datetime, timedelta, timezone
from pathlib import Path
import yt_dlp
from sqlalchemy import select
from app import sysconfig
from app.config import settings
@@ -323,14 +322,33 @@ def _extract_with_fallback(
):
"""Download via yt-dlp using the source `adapter`'s client sets + options. For YouTube the adapter
supplies the primary player-client set then a POT-backed fallback (retried on a bot/DRM/format
error); other sources make a single attempt. Returns the info dict of the successful attempt.
error); a Facebook download WITH a cookie is retried as Chrome behind "Cannot parse data"; other
sources make a single attempt. Returns the info dict of the successful attempt.
Before the first attempt the adapter may resolve `url` to the link that really holds the video (a
Facebook group post → its video's watch link); when the page shows no single video there, or is not
the post at all (a refused cookie's login page), the adapter raises its own verdict
(adapters.POST_*_MESSAGE) instead of letting yt-dlp guess, and a failed page read fails the
download as itself.
`cookiefile` (S6) is the acting user's per-host cookies.txt, already gated by
`credentials.resolve_cookiefile`; passed to every attempt so the fallback stays authenticated.
`max_bytes` (SEC-13) is the owner's remaining storage budget, applied to every attempt."""
client_sets = adapter.client_sets()
client_sets = adapter.client_sets(authed=cookiefile is not None)
outtmpl = str(staging / "%(id)s.%(ext)s")
# Resolving may read a page first — network time the stall watchdog must not see as a stall.
_touch_job(job_id)
target = adapter.resolve_url(
url,
formats.build_ydl_opts(
spec,
outtmpl,
_make_progress_hook(job_id, asset_id),
cookiefile=cookiefile,
max_bytes=max_bytes,
),
)
last_exc: Exception | None = None
for i, clients in enumerate(client_sets):
opts = formats.build_ydl_opts(
@@ -348,7 +366,7 @@ def _extract_with_fallback(
# killed — acceptable, as socket_timeout + extractor_retries bound one attempt well under it.)
try:
with yt_dlp.YoutubeDL(opts) as ydl:
return ydl.extract_info(url, download=True)
return ydl.extract_info(target, download=True)
except yt_dlp.utils.DownloadError as exc:
last_exc = exc
if i < len(client_sets) - 1 and adapter.retriable(str(exc)):
@@ -425,6 +443,7 @@ def _download(
) -> None:
staging = _staging_dir(asset_id)
cookiefile: Path | None = None
url: str | None = None # read by the failure branch, which can run before it is resolved
try:
if staging.exists():
shutil.rmtree(staging, ignore_errors=True)
@@ -445,12 +464,10 @@ def _download(
# this asset is that user's private, salted identity (the double gate lives in credentials).
remaining_bytes: int | None = None
with SessionLocal() as cdb:
asset_sig = cdb.execute(
select(MediaAsset.format_sig).where(MediaAsset.id == asset_id)
).scalar_one_or_none()
if asset_sig is not None:
target = cdb.get(MediaAsset, asset_id)
if target is not None:
cookiefile = credentials.resolve_cookiefile(
cdb, user_id, url, asset_sig, spec, staging
cdb, user_id, url, target, spec, staging
)
# SEC-13: what this user has left. `check_enqueue` only compared the footprint as it
# stood when the job was QUEUED — a download's size is unknown until it finishes, so a
@@ -484,13 +501,19 @@ def _download(
produced = _produced_file(info, staging)
produced = _ensure_browser_playable(job_id, produced, spec, info)
ext = produced.suffix.lstrip(".").lower()
# Codecs/dims/duration come from a PROBE of the produced file — the ground truth — NOT yt-dlp's
# `info` dict, whose fields are absent for some sources (Facebook among them): deriving
# media_kind from those nulls misclassified a real H.264 video as a download-only 'file', and a
# missing duration hid the trim action. Probed before the sidecars so the .nfo runtime has it
# too. Same probe the upload ingest uses.
vcodec, acodec, width, height, duration_s = probe.resolve_media_facts(produced, info)
meta = storage.MediaMeta(
video_id=info.get("id") or source_ref,
title=normalize_title(info.get("title")) or source_ref,
uploader=info.get("uploader") or info.get("channel") or "Unknown Channel",
upload_date=_parse_upload_date(info.get("upload_date")),
uniqueid_type=adapter.nfo_uniqueid_type(),
duration_s=int(info["duration"]) if info.get("duration") else None,
duration_s=duration_s,
description=info.get("description"),
thumbnail_url=info.get("thumbnail"),
)
@@ -516,12 +539,11 @@ def _download(
asset.rel_path = rel
asset.size_bytes = size
asset.duration_s = meta.duration_s
# Codecs/dims come from a PROBE of the finished file — the ground truth — NOT yt-dlp's
# `info` dict, whose codec fields are absent for some sources (Facebook among them):
# deriving media_kind from those nulls misclassified a real H.264 video as a
# download-only 'file' with no in-app play/edit. Same probe the upload ingest uses.
asset.vcodec, asset.acodec, asset.width, asset.height = probe.resolve_media_facts(
final, info
asset.vcodec, asset.acodec, asset.width, asset.height = (
vcodec,
acodec,
width,
height,
)
asset.container = ext
asset.media_kind = mediakind.derive(asset.vcodec, asset.acodec)
@@ -578,6 +600,7 @@ def _download(
status=JobStatus.ERROR,
error=msg,
error_code=dlerrors.classify(msg),
error_detail=None,
phase=None,
)
db.commit()
@@ -588,9 +611,11 @@ def _download(
log.info("job %s refused for its owner: %s", job_id, msg)
except Exception as exc: # noqa: BLE001 — surface any failure to the user, keep worker alive
msg = str(exc)[:500]
code = dlerrors.classify(msg)
# The catalog needs the link and whether the user's cookie went with it: the same yt-dlp
# text is Facebook's login wall without one and a refused cookie with one.
diagnosis = dlerrors.diagnose(msg, url=url, cookie_attached=cookiefile is not None)
# SEC-12: redact a transport-level readout for user-supplied URLs (see user_error_text).
msg = dlerrors.user_error_text(source_kind, msg, code)
msg = dlerrors.user_error_text(source_kind, msg, diagnosis.code)
with SessionLocal() as db:
asset = db.get(MediaAsset, asset_id)
if asset:
@@ -603,7 +628,8 @@ def _download(
job_id,
status=JobStatus.ERROR,
error=msg,
error_code=code,
error_code=diagnosis.code,
error_detail=diagnosis.detail,
phase=None,
)
db.commit()
+18 -9
View File
@@ -81,10 +81,12 @@ def probe_media(path: Path) -> tuple[str | None, str | None, int | None, int | N
def resolve_media_facts(
path: Path, info: dict
) -> tuple[str | None, str | None, int | None, int | None]:
"""(vcodec, acodec, width, height) for the stored download asset: PROBE the finished file on disk
(the ground truth). yt-dlp's `info` omits codec/dimension fields for some sources (Facebook among
them), so deriving media_kind from it there labelled a real H.264 video a download-only 'file'.
) -> tuple[str | None, str | None, int | None, int | None, int | None]:
"""(vcodec, acodec, width, height, duration_s) for the stored download asset: PROBE the finished
file on disk (the ground truth). yt-dlp's `info` omits codec/dimension fields for some sources
(Facebook among them), so deriving media_kind from it there labelled a real H.264 video a
download-only 'file'. It omits the duration too (a Facebook reel, an Instagram reel), and a NULL
duration hides the Library row's trim action although the file plays for minutes.
Precedence is by whether the probe RAN, not per-field: if ffprobe read the file, its codec verdict
is authoritative — INCLUDING a deliberate None (cover-art-only audio, a still image). A per-field
@@ -92,16 +94,23 @@ def resolve_media_facts(
have (yt-dlp reports a progressive `vcodec` for an audio-mode download whose video it then strips),
mislabelling an audio download as a broken in-app 'video'. Only when ffprobe could NOT read the
file at all do we fall back to `info` (the pre-probe behaviour, a safety net); the cosmetic
dimensions still gap-fill from `info` in either case."""
dimensions and the duration still gap-fill from `info` in either case."""
data = editmod.run_ffprobe_json(path, show_streams=True, show_format=True)
info_v = info.get("vcodec") if info.get("vcodec") not in (None, "none") else None
info_a = info.get("acodec") if info.get("acodec") not in (None, "none") else None
info_w = int(info["width"]) if info.get("width") else None
info_h = int(info["height"]) if info.get("height") else None
info_d = int(info["duration"]) if info.get("duration") else None
if data is None:
# ffprobe could not read the file — fall back to the extractor's info entirely.
return info_v, info_a, info_w, info_h
pv, pa, pw, ph, _dur = _facts_from_probe(data)
return info_v, info_a, info_w, info_h, info_d
pv, pa, pw, ph, pd = _facts_from_probe(data)
# Probe ran → its codecs win outright (a None means "no such track", not "unknown"); info only
# gap-fills the cosmetic width/height the probe couldn't read.
return pv, pa, pw if pw is not None else info_w, ph if ph is not None else info_h
# gap-fills the width/height/duration the probe couldn't read.
return (
pv,
pa,
pw if pw is not None else info_w,
ph if ph is not None else info_h,
pd if pd is not None else info_d,
)
+64 -8
View File
@@ -7,6 +7,7 @@ handles everything else. State transitions used by the routes (pause/resume/canc
the worker live in `worker.py` / the routes; this module owns enqueue + asset bookkeeping.
"""
import uuid
from datetime import datetime, timezone
from sqlalchemy import func, select, update
@@ -80,11 +81,13 @@ def get_or_create_asset(
status="pending",
ref_count=0,
)
db.add(asset)
try:
db.flush()
# A SAVEPOINT: losing the race undoes only this insert, never the caller's own pending writes
# (Retry clears the job's error before it gets here; a whole-session rollback threw that away).
with db.begin_nested():
db.add(asset)
db.flush()
except IntegrityError:
db.rollback()
asset = db.execute(stmt).scalar_one()
return asset
@@ -341,13 +344,20 @@ def pause(db: Session, job: DownloadJob) -> None:
def resume(db: Session, job: DownloadJob) -> None:
"""Requeue a paused/errored job for a fresh attempt, clearing its error. If the SHARED asset itself
failed, reset it to pending so the worker actually re-downloads — otherwise the requeued job would
instantly re-inherit the asset's stale error (the worker short-circuits an errored asset)."""
instantly re-inherit the asset's stale error (the worker short-circuits an errored asset).
A cookie the user added AFTER the job was queued moves the job onto their private asset first
(`_follow_the_cookie`): the worker only ever attaches a cookie to that identity, so without the move
the Retry a login-wall message asks for would fail on the same wall."""
if job.status in (JobStatus.PAUSED, JobStatus.ERROR):
job.progress = 0
job.error = None
job.error_code = None
job.error_detail = None
if job.asset_id:
asset = db.get(MediaAsset, job.asset_id)
if asset is not None:
asset = _follow_the_cookie(db, job, asset)
if asset and asset.status == "error": # MediaAsset.status (no enum), not a JobStatus
asset.status = "pending"
asset.error = None
@@ -355,6 +365,32 @@ def resume(db: Session, job: DownloadJob) -> None:
db.commit()
def _follow_the_cookie(db: Session, job: DownloadJob, asset: MediaAsset) -> MediaAsset:
"""Re-point a requeued job at the asset enqueue would pick NOW, when the user has a cookie for the
link's host but the job still holds an asset the worker won't attach it to (queued before the
cookie existed). Returns the asset the job holds afterwards.
Only a `pending` / `error` asset moves — neither has a file nor a download in flight. A `ready` one
already has the file, and a `downloading` one may be this job's own child still being stopped by
the pause; the next Retry after it settles moves the job then. Removing a cookie never moves a job
back: the worker attaches nothing without a credential, so the private asset is as good as any."""
if asset.status not in ("pending", "error"):
return asset
url = source_url(job.source_kind, job.source_ref)
salt = credentials.salt_for_enqueue(db, job.user_id, url)
spec = formats.normalize(job.profile_snapshot)
if salt is None or credentials.is_authed_asset(asset, spec, job.user_id, url):
return asset
target = get_or_create_asset(
db, job.source_kind, job.source_ref, formats.format_sig(spec, salt=salt)
)
populate_from_catalog(db, target)
release_asset(db, job) # drop the old hold BEFORE re-pointing — it keys off job.asset_id
incr_ref_count(db, target.id)
job.asset_id = target.id
return target
def cancel(db: Session, job: DownloadJob) -> None:
"""Cancel a job unless it's already done/canceled, releasing its asset hold WHILE it still counts
as holding — `canceled` ∉ HOLDING, so a later delete won't double-release."""
@@ -373,13 +409,31 @@ def delete(db: Session, job: DownloadJob) -> None:
def redownload(db: Session, job: DownloadJob) -> None:
"""Rebuild a finished/failed job's file into a FRESH, private asset (the cache sig is salted with
the job id + current asset id, so it never reuses the dedup pool) and requeue the SAME job row —
the new asset's own id, so it never reuses the dedup pool) and requeue the SAME job row —
so any public share link (DownloadLink -> job_id) keeps resolving to the newly-downloaded file.
The old hold is dropped first (freeing the old file iff this job was its last holder). The caller
has already rejected edit clips + still-in-progress jobs (HTTP 409)."""
has already rejected edit clips + still-in-progress jobs (HTTP 409).
When the user has a cookie for the link's host, the salt carries it (`credentials.rebuild_salt`),
so the worker recognises the rebuild as their private identity and attaches the cookie — a cookie
download, or one the user added after a failure, rebuilds with the cookie."""
spec = formats.normalize(job.profile_snapshot)
sig = formats.format_sig(spec, salt=f"{job.id}:{job.asset_id}")
asset = get_or_create_asset(db, job.source_kind, job.source_ref, sig)
cookie = credentials.salt_for_enqueue(
db, job.user_id, source_url(job.source_kind, job.source_ref)
)
# The salt names the row's own id, which exists only once the row does: insert it under a unique
# placeholder, then write the real signature in the same transaction (nobody sees the placeholder).
asset = MediaAsset(
source_kind=job.source_kind,
source_ref=job.source_ref,
format_sig=f"rebuild:{uuid.uuid4().hex}",
status="pending",
ref_count=0,
)
db.add(asset)
db.flush()
asset.format_sig = formats.format_sig(spec, salt=credentials.rebuild_salt(asset.id, cookie))
db.flush()
populate_from_catalog(db, asset)
# Drop the old hold BEFORE re-pointing the job — release_asset keys off the job's current asset.
release_asset(db, job)
@@ -389,6 +443,8 @@ def redownload(db: Session, job: DownloadJob) -> None:
job.progress = 0
job.phase = None
job.error = None
job.error_code = None # a rebuild that fails again gets its own diagnosis, never the old one
job.error_detail = None
job.speed_bps = None
job.eta_s = None
# Re-derive the display name from the freshly cleaned title on completion (else an older, un-cleaned
+14 -7
View File
@@ -117,6 +117,19 @@ def tags_by_category(db: Session, tag_ids: list[int]) -> dict[str, list[int]]:
return out
def not_blocked_cond(user: User):
"""Per-user channel blocklist: never show this user videos from a channel they blocked
(applies to feed, Library, explore, the channel page and the Home resume rail alike).
A shared expression rather than a clause each surface writes for itself. Blocking does not touch
`video_states`, so a half-watched video from a newly blocked channel survives with its resume
position — and the one query that copied only PART of this predicate set is exactly how that
video came back as the landing page's suggestion after the user had said "never again"."""
return Video.channel_id.notin_(
select(BlockedChannel.channel_id).where(BlockedChannel.user_id == user.id)
)
def filtered_query(
db: Session,
user: User,
@@ -207,13 +220,7 @@ def filtered_query(
)
)
# Per-user channel blocklist: never show this user videos from a channel they blocked
# (applies to feed, Library, explore, and the channel page alike).
query = query.where(
Video.channel_id.notin_(
select(BlockedChannel.channel_id).where(BlockedChannel.user_id == user.id)
)
)
query = query.where(not_blocked_cond(user))
# Source filter. In the shared Library ("all" scope) it uses the GLOBAL via_search flag
# (search-discovered by anyone). In "my" scope it's the user's OWN provenance:
+2 -2
View File
@@ -175,7 +175,7 @@ MAIL_TEXT: dict[str, dict[str, str]] = {
"en": (
"Hi,\n\n"
"Someone has opened up Siftlode for you — a quiet, self-hosted reader for your YouTube "
"subscriptions. Every upload from the channels you follow lands in one feed you can "
"subscriptions. Every upload from the channels you follow lands in one Feed you can "
"actually filter and sort, with no algorithm deciding what you see, and no Shorts or "
"livestreams unless you want them.\n\n"
"{next_step}\n\n"
@@ -186,7 +186,7 @@ MAIL_TEXT: dict[str, dict[str, str]] = {
"Szia,\n\n"
"Valaki megnyitotta előtted a Siftlode-ot — ez egy csendes, saját üzemeltetésű olvasó "
"a YouTube-feliratkozásaidhoz. A követett csatornák minden feltöltése egyetlen "
"hírfolyamba érkezik, amit tényleg tudsz szűrni és rendezni; nincs algoritmus, ami "
"Feedbe érkezik, amit tényleg tudsz szűrni és rendezni; nincs algoritmus, ami "
"eldönti, mit látsz, és nincs Shorts vagy élő adás, hacsak nem kéred.\n\n"
"{next_step}\n\n"
"Nem sürget semmi — az ajtó nyitva marad.\n\n"
+66 -4
View File
@@ -1,10 +1,13 @@
import asyncio
import logging
import math
import sys
from contextlib import asynccontextmanager, suppress
from pathlib import Path
from fastapi import FastAPI, HTTPException
from fastapi import FastAPI, HTTPException, Request
from fastapi.encoders import jsonable_encoder
from fastapi.exceptions import RequestValidationError
# When started via uvicorn --log-config the "siftlode" logger is already configured
# (see log_config.json). This block is a timestamped fallback for other entrypoints
@@ -21,18 +24,21 @@ log = logging.getLogger("siftlode.app")
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse
from fastapi.staticfiles import StaticFiles
from sqlalchemy.exc import DataError
from starlette.middleware.sessions import SessionMiddleware
from app import state
from app.config import settings
from app.db import SessionLocal
from app.plex import stream as plex_stream
from app.plex import subtitles as plex_subtitles
from app.routes import (
admin,
audit as audit_routes,
auth as auth_routes,
channels,
config as config_routes,
discover as discover_routes,
downloads,
feed,
health,
@@ -76,6 +82,7 @@ async def lifespan(app: FastAPI):
# alone doesn't buy that; not waiting does. The sweep only touches directories no live session
# owns, so racing it against early playback is safe.
sweep = asyncio.create_task(_sweep_hls_orphans())
reaper = asyncio.create_task(_reap_hls_sessions())
start_scheduler()
try:
yield
@@ -85,10 +92,36 @@ async def lifespan(app: FastAPI):
# pending ("Task was destroyed but it is pending!" — noise that reads like a bug during
# triage). Cancelling does NOT interrupt the rmtree thread itself; this only makes our own
# bookkeeping honest.
sweep.cancel()
with suppress(asyncio.CancelledError):
await sweep
for task in (sweep, reaper):
task.cancel()
with suppress(asyncio.CancelledError):
await task
shutdown_scheduler()
# Kills a running subtitle extraction: its worker would otherwise hold interpreter exit for
# the rest of the read (see plex_subtitles.shutdown).
plex_subtitles.shutdown()
# How often idle HLS sessions are reaped (see plex/stream.py `reap_idle`).
_HLS_REAP_EVERY_S = 60.0
async def _reap_hls_sessions() -> None:
"""The periodic reaper plex/stream.py's docstring promises. `reap_idle` used to run only inside
the NEXT session request, so after the last playback of the day an abandoned session — its
ffmpeg still working through the film, its directory a whole segmented copy of it — lived until
somebody pressed play again or the process restarted.
A FINISHED session is reaped on the short rule here, not kept for the full idle window: an open
player checkpoints every 10 s and that route touches the session, so a paused viewer is no longer
invisible to this pass (see `reap_idle`). Keeping every finished session for ten minutes instead
held one whole segmented copy of a film per item on the download volume (round 5)."""
while True:
await asyncio.sleep(_HLS_REAP_EVERY_S)
try:
await asyncio.to_thread(plex_stream.reap_idle)
except Exception:
log.exception("HLS session reap failed")
async def _sweep_hls_orphans() -> None:
@@ -107,6 +140,34 @@ async def _sweep_hls_orphans() -> None:
app = FastAPI(title=settings.app_name, lifespan=lifespan)
@app.exception_handler(RequestValidationError)
async def _validation_error(request: Request, exc: RequestValidationError) -> JSONResponse:
"""FastAPI's own 422, in its own shape — except that it can be ENCODED. The JSON parser accepts
`NaN`/`Infinity`, a float field refuses them, and the refusal echoes the value back: the default
handler then failed to serialise its own error (JSON has no NaN) and the request became a 500
with a traceback. A non-finite value is echoed as its name instead."""
finite = {float: lambda x: x if math.isfinite(x) else str(x)}
return JSONResponse(
status_code=422, content={"detail": jsonable_encoder(exc.errors(), custom_encoder=finite)}
)
@app.exception_handler(DataError)
async def _nul_in_text(request: Request, exc: DataError) -> JSONResponse:
"""A NUL byte in a string the request sent is a 422, on every route at once. Postgres cannot hold
NUL in text, so the driver refuses to even send the query — and a `%00` in a query parameter or
an escaped NUL in a JSON body (a playlist's `contains=`, a reorder's key list, a collection
search) was a 500 with a traceback wherever it reached a comparison. Guarding each site did not
hold: round 3 guarded the by-rating_key lookup and missed the rest. Any other DataError is left
alone — re-raised, it stays the 500 it always was."""
if "NUL (0x00)" not in str(exc.orig):
raise exc
return JSONResponse(
status_code=422, content={"detail": {"code": "invalid_input", "reason": "nul"}}
)
app.add_middleware(
SessionMiddleware,
secret_key=settings.secret_key,
@@ -187,6 +248,7 @@ app.include_router(auth_routes.router)
app.include_router(sync.router)
app.include_router(tags.router)
app.include_router(feed.router)
app.include_router(discover_routes.router)
app.include_router(search_routes.router)
app.include_router(me.router)
app.include_router(notifications.router)
+58
View File
@@ -3,6 +3,7 @@ from datetime import date, datetime
from sqlalchemy import (
BigInteger,
Boolean,
CheckConstraint,
Computed,
Date,
DateTime,
@@ -23,6 +24,11 @@ from app.db import Base
from app.utils import canonical_email
# The resume-position columns (VideoState / PlexState.position_seconds) are int4. The checkpoint
# routes bound their bodies by this, so an out-of-range position is a 422 rather than a 500.
POSITION_MAX_S = 2**31 - 1
class TimestampMixin:
"""Adds a server-defaulted `created_at`. Models that index created_at or order it
differently (QuotaEvent, Notification, Message) keep their own column on purpose."""
@@ -422,6 +428,7 @@ class VideoState(Base, UpdatedAtMixin):
# status: a partially-watched video keeps status "new" but a non-zero position, so
# it can render a progress bar and match the "in progress" feed filter. 0 = not
# started / finished (trivially-early and near-finished positions are not stored).
# int4: a checkpoint body must stay under POSITION_MAX_S, or the flush is a DataError (a 500).
position_seconds: Mapped[int] = mapped_column(
Integer, default=0, server_default="0", nullable=False
)
@@ -864,6 +871,8 @@ class DownloadJob(Base, TimestampMixin, UpdatedAtMixin):
error_code: Mapped[str | None] = mapped_column(
String(24)
) # category, see downloads.errors.classify
# The host-specific failure-catalog entry on top of the category (`facebook.login_wall`), or NULL.
error_detail: Mapped[str | None] = mapped_column(String(48)) # see downloads.errors.diagnose
queue_pos: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
@@ -1406,3 +1415,52 @@ class PlexPlaylistItem(Base):
ForeignKey("plex_items.id", ondelete="CASCADE"), index=True
)
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
class ComfortPin(Base, TimestampMixin):
"""One entry on a user's comfort shelf — the anti-paralysis fallback: "just put on X" (E24).
Deliberately NOT a boolean on `VideoState`/`PlexState`, and NOT a polymorphic `(kind, ref)` string
pair. A flag cannot ORDER a shelf and cannot hold a channel or a show — and comfort media is
usually "put on X", a series or a channel, not one particular video. A string ref would order and
hold anything, but it would also lose the foreign key: a Plex item dropped from the library, or an
unsubscribed channel, would leave a dead tile on the one surface whose whole job is to be
reliable. So instead: one nullable CASCADE-ing FK per pinnable kind, and a check constraint that
exactly one of them is set.
That constraint is the house rule applied to a five-way union — the invariant lives in the schema,
not in a comment every insert site has to remember. `kind`/`ref` below derive FROM the columns, so
there is no second vocabulary to keep in sync.
NULLs are DISTINCT in a Postgres unique index, so the five per-kind unique constraints each stop a
double-pin of the same thing without colliding across kinds."""
__tablename__ = "comfort_pins"
__table_args__ = (
CheckConstraint(
"num_nonnulls(video_id, channel_id, plex_item_id, plex_show_id, playlist_id) = 1",
name="ck_comfort_pin_one_ref",
),
UniqueConstraint("user_id", "video_id", name="uq_comfort_pin_video"),
UniqueConstraint("user_id", "channel_id", name="uq_comfort_pin_channel"),
UniqueConstraint("user_id", "plex_item_id", name="uq_comfort_pin_plex_item"),
UniqueConstraint("user_id", "plex_show_id", name="uq_comfort_pin_plex_show"),
UniqueConstraint("user_id", "playlist_id", name="uq_comfort_pin_playlist"),
)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
video_id: Mapped[str | None] = mapped_column(ForeignKey("videos.id", ondelete="CASCADE"))
channel_id: Mapped[str | None] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
plex_item_id: Mapped[int | None] = mapped_column(
ForeignKey("plex_items.id", ondelete="CASCADE")
)
plex_show_id: Mapped[int | None] = mapped_column(
ForeignKey("plex_shows.id", ondelete="CASCADE")
)
playlist_id: Mapped[int | None] = mapped_column(ForeignKey("playlists.id", ondelete="CASCADE"))
# 0-based shelf order. The shelf is short and hand-curated, so a dense integer the reorder
# endpoint rewrites wholesale beats a fractional index nobody would ever exercise here.
position: Mapped[int] = mapped_column(Integer, default=0, server_default="0", nullable=False)
+6 -1
View File
@@ -197,7 +197,12 @@ class PlexClient:
)
def unscrobble(self, rating_key: str) -> None:
"""Mark the item *unwatched* on the token's Plex account (clears viewCount)."""
"""Mark the item *unwatched* on the token's Plex account.
Clears the view count AND the resume point (`viewOffset`) — measured against the live server
on 2026-09-09, from both a watched+offset state and an offset-only one. This one call is the
whole un-mark; `/:/timeline?time=0` and `/:/progress?time=0` were both tried there and do
NOT clear an offset, so do not reach for them."""
self._write(
"GET",
f"/:/unscrobble?identifier={self._LIBRARY_IDENTIFIER}"
+7 -1
View File
@@ -11,5 +11,11 @@ _M = TypeVar("_M")
def by_rating_key(db: Session, model: type[_M], rating_key: str) -> _M | None:
"""The mirrored row for a Plex rating_key, or None. `model` is any of the rating_key-keyed Plex
tables (PlexItem / PlexShow / PlexSeason / PlexCollection); the return type is that model, so call
sites keep their `Model | None` static type."""
sites keep their `Model | None` static type.
A key with a NUL in it (`%00` in a path segment) names no row — a 404, like any unknown key.
Postgres refuses NUL in text, so asking would be a DataError; the app-wide handler turns the
places that DO ask (lists and filters compared to rating_key) into a 422 (see main.py)."""
if "\x00" in str(rating_key):
return None
return db.execute(select(model).where(model.rating_key == str(rating_key))).scalars().first()
+41 -1
View File
@@ -8,11 +8,28 @@ from datetime import datetime, timedelta, timezone
from sqlalchemy import and_, case, func, or_, select, type_coerce
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Session
from app.models import PlexItem
from app.models import PlexItem, PlexLibrary
TS_CONFIG = "public.unaccent_simple"
# The ONE Plex sort vocabulary, shared by every surface that offers the Sort control (the browse grid
# and the collection info page) and mirrored by the frontend's `plexSort.ts`. It lives here rather than
# inside a router because the two surfaces order DIFFERENT things — browse orders a union subquery in
# SQL, a collection orders its already-loaded member list in Python — and a key that one of them cannot
# honour would put an option in the UI that silently does nothing. Tests pin both implementations to
# this tuple, so adding a key here fails the gate until every surface can order by it.
PLEX_SORT_KEYS: tuple[str, ...] = (
"added",
"lastwatched",
"release",
"year",
"rating",
"duration",
"title",
)
# --- Language-aware column pickers (E5 S6) ----------------------------------
#
# The HU-overlay twin (E5 S5) stores Hungarian metadata in `*_hu` columns on the SAME EN row. When
@@ -342,3 +359,26 @@ def apply_show_state_filter(query, tot, wat, inp, show: str):
if show == "unwatched":
return query.where(wat == 0, inp == 0)
return query
def content_libs(db: Session) -> tuple[list[int], list[int]]:
"""The BROWSABLE content libraries, partitioned into (movie_lib_ids, show_lib_ids) — the single
source of the browse scope. Excludes HU overlay libraries (`overlay_of` set): those are never
browsed on their own, they only lend their Hungarian metadata to the EN twin's rows (E5 S5).
Lives here, in the query layer, rather than in `routes/plex/_shared.py` where it started: E24's
Home surface needs the same scope, and a domain package importing from `routes/` would invert the
one-way dependency CLAUDE.md draws. It never touched request state, so nothing was lost by the
move."""
libs = (
db.execute(
select(PlexLibrary).where(
PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
)
)
.scalars()
.all()
)
movie_lib_ids = [lb.id for lb in libs if lb.kind == "movie"]
show_lib_ids = [lb.id for lb in libs if lb.kind == "show"]
return movie_lib_ids, show_lib_ids
+14
View File
@@ -3,11 +3,25 @@
Drives the show/season grid badges and the Resume / Play-from-start / mark-all buttons.
"""
from datetime import datetime, timezone
from sqlalchemy import and_, case, func, or_, select
from sqlalchemy.orm import Session, aliased
from app.models import PlexItem, PlexState
# Sort sentinel for a rail row whose timestamp is missing, so `key=lambda r: r[0] or NO_TS` reads the
# same at every rail that merges two type-homogeneous lists in Python.
#
# It is NOT a crash fix, whatever the E24 commit that introduced it says. The `(ts is not None, ts)`
# key it replaced cannot raise: tuple comparison finds the first UNEQUAL element via `==`, so two
# `(False, None)` keys compare equal and `None < None` is never reached — verified by execution
# during the E24 epic-end review, over every permutation of null/non-null rows. What the sentinel
# buys is one idiom instead of two, and a key that says "missing sorts last" outright rather than
# encoding it in a boolean flag. A BARE `r[0]` would genuinely raise, which is the mutation the
# rails' tests do guard against.
NO_TS = datetime.min.replace(tzinfo=timezone.utc)
def show_status(total: int, watched: int, inprog: int) -> str:
"""Roll a show's episodes up into a single watch state: fully watched, partially started, or new."""
+206 -16
View File
@@ -21,10 +21,12 @@ CPU-only prod host; only P3's full transcode is CPU-heavy.
import logging
import os
import re
import secrets
import shutil
import subprocess
import threading
import time
from collections.abc import Sequence
from pathlib import Path
from sqlalchemy.orm import Session as DbSession
@@ -55,6 +57,10 @@ _SEG_SECONDS = 6
# worse than the old zero-buffer behaviour).
_WARMUP_SEGMENTS = 3
_WARMUP_SEG_TIMEOUT_S = 8.0
# The HLS variant ffmpeg gives the VIDEO rendition of a multi-audio session. `-var_stream_map` names
# each audio rendition (`name:a0`, `a1`, …) but leaves the video one unnamed, so ffmpeg falls back to
# its index for that one. See `segment_name` for why this is a constant and not a comment.
_VIDEO_VARIANT = "0"
_SESSION_IDLE_S = 600 # reap a session with no access for this long
# A session directory is named `{rating_key}_{int(start_s)}_{tag}_{aoff:+.2f}` (see start_session);
# before multi-audio it was just `{rating_key}_{int(start_s)}`. `sweep_orphans` deletes ONLY names
@@ -94,11 +100,19 @@ class HlsSession:
directory: Path,
proc: subprocess.Popen,
start_s: float,
entry: str,
variants: "Sequence[str]",
transcode: bool = False,
hw: bool = False,
tonemap: bool = False,
):
# Unguessable per-session id, carried in the URLs the client is handed. Sessions are keyed
# by rating_key alone, so a seek-restart or a SECOND viewer of the same item replaces the
# registered one — and the old viewer's next request for `seg_12.ts`, resolved by rating_key,
# was answered 200 with the NEW session's twelfth segment, produced from a different offset
# under `-copyts`. Wrong bytes, not an error, so the client's reclaim never fired on it
# (deep-review round 5). A request naming a session that is no longer registered is a 404,
# which the player recovers from by restarting at the playhead.
self.id = secrets.token_urlsafe(9)
self.key = key
self.dir = directory
self.proc = proc
@@ -113,7 +127,13 @@ class HlsSession:
# subtitle cue-shift both key off the true content position — no drift. Provisional = X until
# the first segment is probed.
self.media_start_s = start_s
self.entry = entry # the playlist filename hls.js should load (master.m3u8 with subs, else index.m3u8)
# The rendition names this session's ffmpeg was told to write — () for a single-audio
# session. Held on the session because it is what `artifact_may_arrive` needs to tell a name
# this session CAN produce from one belonging to the other argv branch (see it).
self.variants = tuple(variants)
# The playlist filename hls.js should load: the master of a multi-audio session, else the
# single index. Derived, not passed, so it cannot disagree with the argv.
self.entry = _MASTER_PL if self.variants else _INDEX_PL
self.last_access = time.time()
@@ -271,6 +291,30 @@ def _hwaccel_in_opts(hw: bool, tonemap: bool = False) -> list[str]:
return ["-hwaccel", "vaapi", "-hwaccel_output_format", "vaapi", "-vaapi_device", _VAAPI_DEVICE]
# Audio is ALWAYS re-encoded on the HLS path: the source is typically AC3/DTS 5.1, which no browser
# plays, so it is downmixed to stereo AAC. `-ac 2` uses ffmpeg's standard downmix matrix; nothing is
# normalised, which is why a film mastered for a cinema arrives with whispered dialogue and deafening
# action (bug hunt 2026-09, item 14).
_AUDIO_ENC = ["-c:a", "aac", "-ac", "2", "-b:a", "192k"]
# Optional loudness normalisation (admin knob `plex_audio_loudnorm`, default OFF). EBU R128 single-pass
# `loudnorm` — two-pass is impossible here, since a seek-restart session must start emitting segments
# immediately and cannot analyse the file first.
#
# The downmix is FORCED FIRST (`aformat=channel_layouts=stereo`) rather than left to `-ac 2`: measuring
# 5.1 and then folding it down would normalise something other than what the viewer hears. `loudnorm`
# runs its own resampler at 192 kHz internally, so the chain resamples back to 48 kHz for the AAC
# encoder. Measured on a real 5.1 AC3 film (Westworld S1E6, 60 s from 15:00): I -27.6 → -16.6 LUFS,
# LRA 13.3 → 9.8 LU — the quiet/loud gap narrows, and everything gets ~11 dB louder, so expect to turn
# the volume down after switching this on. The filter's 3 s lookahead does NOT desync A/V: measured
# under this pipeline's own `-copyts`, the audio start PTS is identical with and without it.
_LOUDNORM_AF = "aformat=channel_layouts=stereo,loudnorm=I=-16:TP=-1.5:LRA=11,aresample=48000"
def _aenc_opts(loudnorm: bool) -> list[str]:
"""Audio encoder args for one output, with the normalisation filter when the admin enabled it."""
return [*_AUDIO_ENC, *(["-af", _LOUDNORM_AF] if loudnorm else [])]
def _venc_opts(
transcode: bool,
hw: bool,
@@ -446,6 +490,7 @@ def _ffmpeg_cmd(
preset: str = "veryfast",
crf: int = 20,
max_height: int = 0,
loudnorm: bool = False,
) -> list[str]:
# For a transcode session the video is re-encoded (VAAPI or libx264) instead of stream-copied; the
# hwaccel decode options go before -i, the encoder args replace `-c:v copy` below. Everything else is
@@ -469,11 +514,15 @@ def _ffmpeg_cmd(
if audio_langs:
# MULTI-RENDITION: map EVERY audio track as an alternate HLS rendition in one group, so hls.js
# switches audio CLIENT-SIDE on the same timeline (no session restart, no drift) — the premium
# audio-switch path. Produces master.m3u8 + stream_%v.m3u8 (v0=video, v1..=audio) + seg_%v_%d.ts.
# audio-switch path. Produces master.m3u8 + one stream_%v.m3u8 and seg_%v_*.ts per variant,
# where %v is the video rendition's index ("0") and each audio rendition's `name:` (a0, a1,
# …). Those names are spelled by `segment_name` / `variant_playlist_name` below, which the
# guard and the warm-up read back — see them for the measurement behind the ordering.
args += ["-map", "0:v:0"]
for i in range(len(audio_langs)):
args += ["-map", f"{ai}:a:{i}?"]
args += ["-sn", *venc, "-c:a", "aac", "-ac", "2", "-b:a", "192k"]
# One `-af` covers EVERY audio output of this file, so each rendition is normalised.
args += ["-sn", *venc, *_aenc_opts(loudnorm)]
var = ["v:0,agroup:aud"]
for i, lang in enumerate(audio_langs):
entry = f"a:{i},agroup:aud,name:a{i}"
@@ -482,23 +531,25 @@ def _ffmpeg_cmd(
if i == 0:
entry += ",default:yes"
var.append(entry)
# The name patterns come from the vocabulary functions, with ffmpeg's own placeholders passed
# through them — so the producer and the parser cannot spell the layout differently (round 7).
args += _HLS_TAIL + [
"-master_pl_name",
"master.m3u8",
_MASTER_PL,
"-var_stream_map",
" ".join(var),
"-hls_segment_filename",
str(out_dir / "seg_%v_%d.ts"),
str(out_dir / "stream_%v.m3u8"),
str(out_dir / segment_name("%d", "%v")),
str(out_dir / variant_playlist_name("%v")),
]
else:
ao = audio_ord if audio_ord is not None else 0
args += ["-map", "0:v:0", "-map", f"{ai}:a:{ao}?", "-sn"]
args += [*venc, "-c:a", "aac", "-ac", "2", "-b:a", "192k"]
args += [*venc, *_aenc_opts(loudnorm)]
args += _HLS_TAIL + [
"-hls_segment_filename",
str(out_dir / "seg_%d.ts"),
str(out_dir / "index.m3u8"),
str(out_dir / segment_name("%d")),
str(out_dir / _INDEX_PL),
]
return args
@@ -579,16 +630,26 @@ def start_session(
src = paths.local_media_path(db, item.file_path)
if src is None:
return None
# Read once, here: the knob has to be identical for the directory name and the ffmpeg argv, and a
# session respawn (the HW->SW fallback) must not pick up a mid-playback config change.
loudnorm = sysconfig.get_bool(db, "plex_audio_loudnorm")
key = item.rating_key
start_s = max(0.0, float(start_s))
audio_langs = _probe_audio_langs(src) if multi else []
is_multi = len(audio_langs) > 1 # only worth a master playlist when there really are ≥2 tracks
tag = "multi" if is_multi else str(audio_ord)
# The rendition names the argv below will produce — one source for the argv, the warm-up's
# segment names, the entry playlist and the artifact guard.
variants = session_variants(audio_langs if is_multi else None)
# Loudness normalisation changes the AUDIO THE SEGMENTS CONTAIN, so it belongs in the directory
# name beside the other parameters that do (offset, audio track, A/V offset): a session left over
# from before the admin flipped the knob must not be mistaken for one that honours it. Appended to
# the tag with a DASH — `_SESSION_DIR_RE` treats the tag as one underscore-free field, and that
# regex is the only thing keeping the orphan sweep off a shared scratch path (see its comment).
tag = ("multi" if is_multi else str(audio_ord)) + ("-ln" if loudnorm else "")
directory = _HLS_ROOT / f"{key}_{int(start_s)}_{tag}_{aoff:+.2f}"
def _seg(i: int) -> Path:
# multi-audio: the video rendition is variant v0 → seg_{i}_0.ts; single-audio → seg_{i}.ts
return directory / (f"seg_{i}_0.ts" if is_multi else f"seg_{i}.ts")
return directory / segment_name(i, _VIDEO_VARIANT if variants else None)
def _spawn(
hw: bool, tonemap: bool = False, *, replace_only: "HlsSession | None" = None
@@ -632,6 +693,7 @@ def start_session(
preset=preset,
crf=crf,
max_height=max_height,
loudnorm=loudnorm,
),
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
@@ -642,7 +704,7 @@ def start_session(
directory,
proc,
start_s,
"master.m3u8" if is_multi else "index.m3u8",
variants,
transcode=transcode,
hw=hw,
tonemap=tonemap,
@@ -714,6 +776,101 @@ def current_session(key: str) -> HlsSession | None:
return s
# The names ffmpeg writes into a session directory, in ONE place — and "one place" means the argv
# BUILDS its patterns from these same functions (`segment_name("%d", "%v")`), not that a comment
# says so. Getting this wrong is invisible in both directions: a name we failed to anticipate used
# to fall back to "wait and see" (round 6's HIGH), and now that the rule is positive it falls back
# to a 404 instead — so the producer and the parser have to be literally the same spelling.
#
# MEASURED (ffmpeg 7.1.5 in the API container — the binary that actually writes these files — and
# re-checked on 8.1.1): `-hls_segment_filename seg_%v_%d.ts` substitutes `%v` with the VARIANT and
# `%d` with the SEQUENCE — variant FIRST. A dual-audio session writes seg_0_0.ts… (video, the
# unnamed variant) plus seg_a0_0.ts… / seg_a1_0.ts… (the `name:aN` audio renditions), and the
# playlists beside them are master.m3u8 + stream_0/a0/a1.m3u8. Single-audio uses `seg_%d.ts` +
# index.m3u8, with no variant field at all.
#
# The two argv branches are DISJOINT vocabularies, which is why the guard below is keyed to the
# SESSION and not to HLS in general: a single-audio session can never write `master.m3u8`,
# `stream_*.m3u8` or any `seg_<variant>_<n>.ts`, and a multi-audio one can never write `index.m3u8`
# or `seg_<n>.ts`. Accepting the union meant a hopeless name of the other shape was waited out for
# the full playlist/segment timeout — the threadpool parking this guard exists to end (round 7).
_MASTER_PL = "master.m3u8"
_INDEX_PL = "index.m3u8"
_SEG_NAME_RE = re.compile(r"^seg_(?:([0-9a-z]{1,4})_)?(\d{1,9})\.ts$")
def session_variants(audio_langs: "Sequence[str | None] | None") -> tuple[str, ...]:
"""The `-var_stream_map` rendition names a session will write, or () for a single-audio one.
The video rendition is left unnamed in the map, so ffmpeg falls back to its index
(`_VIDEO_VARIANT`); each audio rendition is named `aN` in track order."""
if not audio_langs:
return ()
return (_VIDEO_VARIANT, *(f"a{i}" for i in range(len(audio_langs))))
def segment_name(seq: "int | str", variant: "str | None" = None) -> str:
"""The file ffmpeg writes for the `seq`-th segment of `variant`.
`variant=None` is a single-audio session, whose argv carries no `%v` field at all. `seq` and
`variant` are typed loosely because `_ffmpeg_cmd` builds its own `-hls_segment_filename` pattern
through this function, passing ffmpeg's `"%d"` / `"%v"` placeholders — that is what keeps the
producer and the parser from drifting, rather than a comment asking the next editor not to."""
return f"seg_{variant}_{seq}.ts" if variant else f"seg_{seq}.ts"
def variant_playlist_name(variant: str) -> str:
"""The sub-playlist ffmpeg writes for one rendition of a multi-audio session."""
return f"stream_{variant}.m3u8"
def playlist_names(variants: "Sequence[str]") -> frozenset[str]:
"""Every playlist name this session's argv can write — the master plus one per rendition, or
just `index.m3u8` for a single-audio session."""
if not variants:
return frozenset({_INDEX_PL})
return frozenset({_MASTER_PL, *(variant_playlist_name(v) for v in variants)})
def artifact_may_arrive(s: "HlsSession", filename: str) -> bool:
"""Whether waiting for `filename` can possibly end in a file OF THIS SESSION.
Positive rule, not a blocklist: a session produces segments and playlists and NOTHING else, so
any other name is refused at once. ffmpeg also produces segments strictly in order, so a segment
whose PREDECESSOR does not exist yet is not the next one — it is beyond the generated region,
and no amount of waiting will conjure it. Playlists are the one thing that must still be waited
for: they are rewritten continuously and have no ordering to read.
Keyed to the SESSION's own vocabulary, not to HLS's: the two argv branches are disjoint, so a
name belonging to the other shape is as hopeless here as a random string. Reading those as "an
HLS name, so wait and see" left the whole class reachable — measured through the real route
against a live single-audio session, `master.m3u8` and `stream_q7z.m3u8` were 404s after 30.0 s
and `seg_a0_0.ts` / `seg_zzzz_0.ts` after 20.1 s each (deep-review round 7).
Saying no at once matters because the route that waits is a SYNC one: it holds a thread off the
~40-slot pool, and a pooled DB connection with it, for the whole timeout. An ordinary seek past
the produced region was enough to park one for 20 s, and forty concurrent requests for a name
that will never exist parked the whole API (deep-review round 5). Round 5's guard read only
single-audio segment names, so both halves of that were still fully reachable: every multi-audio
name fell through, and so did any `.m3u8`/`.vtt` name at all."""
if filename in playlist_names(s.variants):
return True
m = _SEG_NAME_RE.match(filename)
if m is None:
return False
variant, seq = m.group(1), int(m.group(2))
# The variant field must be one THIS session writes: present and known for a multi-audio
# session, absent for a single-audio one.
if variant not in (s.variants or (None,)):
return False
if seq == 0:
return True
try:
return (s.dir / segment_name(seq - 1, variant)).exists()
except OSError:
return False
def wait_for(path: Path, timeout: float = 20.0, proc: subprocess.Popen | None = None) -> bool:
"""Wait until a session file (playlist / segment) exists and is non-empty. Segments are
produced ~faster than realtime, so a segment just ahead of playback appears quickly; a segment
@@ -785,14 +942,47 @@ def sweep_orphans(older_than: float | None = None) -> int:
return dropped
def reap_idle() -> int:
# How long a session whose ffmpeg has FINISHED may go unfetched before a new session request reaps
# it (see reap_idle).
_FINISHED_IDLE_S = 30
def reap_idle(finished_idle_s: float = _FINISHED_IDLE_S) -> int:
"""Kill sessions nobody has said they still want for `_SESSION_IDLE_S` — or, once their ffmpeg
has finished, for `finished_idle_s`.
A finished session is also what a PAUSED viewer has: a remux runs far faster than realtime, so
ffmpeg is done a minute or two in and writes ENDLIST; hls.js then stops asking for the playlist,
and with its forward buffer full a paused player fetches nothing at all. With the 30 s rule that
killed a one-minute pause, and playback died ~30 s after resuming (deep-review round 4).
What makes the short rule safe again is that fetching is no longer the only signal: a player
parked MID-ITEM checkpoints its position every 10 s, and that route calls `touch_session`, so a
viewer who is merely paused says so far more often than this rule asks (round 5). The heartbeat
stops once the item is finished or before a position is known (`usePlexProgress.flush`) — both
states where reaping is correct, and where the player's 404 → reclaim recovers a viewer who
scrubs back in anyway. Do not restate this as "every 10 s whether or not it is playing": round 6
found that claim in two comments and not in the code. Round 4's blanket
`reap_abandoned` bought the same safety by keeping every finished session for ten minutes — each
one a whole segmented copy of the film on a volume shared with the download center."""
now = time.time()
dropped = 0
with _lock:
for key, s in list(_sessions.items()):
done = s.proc.poll() is not None
if now - s.last_access > _SESSION_IDLE_S or (done and now - s.last_access > 30):
idle = now - s.last_access
if idle > _SESSION_IDLE_S or (done and idle > finished_idle_s):
_kill(s)
_sessions.pop(key, None)
dropped += 1
return dropped
def touch_session(key: str) -> None:
"""Say a viewer is still there for `key`, without fetching anything. `last_access` used to move
only on an HLS file fetch, which a paused player does not do — so the only way to keep a paused
viewer's session alive was to keep EVERY abandoned one just as long."""
with _lock:
s = _sessions.get(key)
if s is not None:
s.last_access = time.time()
+730 -14
View File
@@ -1,15 +1,57 @@
"""WebVTT conversion, HLS-clock time-shift, and on-disk cache for Plex subtitle streams.
"""WebVTT conversion, HLS-clock time-shift, and the on-disk cache + background extractor for Plex
subtitle streams.
Text subtitles (embedded or external) are turned into WebVTT so the browser <track> can render them;
bitmap subtitles (Blu-ray/DVD) can't be (would need burn-in). Converted VTTs are cached on the shared
Plex on-disk cache root (``cache.CACHE_DIR``).
WHY THERE IS A BACKGROUND EXTRACTOR HERE — measured on prod, 2026-09-10, against a 17.06 GB 1080p
MKV with two embedded SubRip tracks on the 81 TB array:
=========================================================== ========
full pass, ONE track (what this module used to do) 154 s
full pass, BOTH text tracks in a SINGLE ffmpeg run 111 s
`-c:s copy` instead of converting to WebVTT 143 s
from 1:00:00 to the end of the file 66 s
a 630 s WINDOW at 1:00:00, both tracks 8 s
=========================================================== ========
The conversion is not the cost (dropping it saved 7%): ffmpeg has to stream every byte of the
container between its seek point and its stop point, and the array delivers ~140 MB/s — so the time
is (bytes read) / 140 MB/s, and the ONLY lever is reading less. Hence:
* extract a WINDOW around the playhead first — the viewer sees subtitles in seconds — then complete
the whole track in the background so every later seek is instant and the next session is free;
* extract EVERY embedded text track in ONE pass, because the read is shared: picking Hungarian used
to cost a full read and then switching to English cost another one (the "hektikus" report);
* never do any of it inside a request (it froze the <track> with no feedback AND held an API worker
thread for the whole read).
`-copyts` CANNOT be used for a window: it makes ffmpeg ignore `-t`/`-to` (measured — a [0, 600 s]
window still took the full 145 s and emitted the whole track), so windows are extracted on a
zero-based clock and shifted back to absolute time here with `shift_vtt`.
"""
import json
import logging
import re
import subprocess
import tempfile
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from sqlalchemy.orm import Session
from app.db import SessionLocal
from app.models import PlexItem
from app.plex import paths as plex_paths
from app.plex.cache import CACHE_DIR
from app.plex.client import PlexError
from app.plex.client import PlexClient, PlexError
from app.plex.lookups import by_rating_key
log = logging.getLogger("siftlode.plex")
# Image-based subtitle codecs (Blu-ray/DVD) — can't be turned into text WebVTT (would need burn-in).
BITMAP_SUBS = {
@@ -31,6 +73,9 @@ def srt_to_vtt(text: str) -> str:
def _vtt_ts_to_s(ts: str) -> float:
# Also reads ffmpeg's spelling of a NEGATIVE time (a windowed pass, see `_ffmpeg_extract`): its
# webvtt muxer splits the milliseconds with C's truncating division, so every field carries the
# sign — -5.005 s is `00:-5.-05`, -65.3 s is `-1:-5.-300` — and summing the signed fields is exact.
parts = ts.split(":")
h = "0" if len(parts) == 2 else parts[0]
m, rest = parts[-2], parts[-1]
@@ -39,16 +84,12 @@ def _vtt_ts_to_s(ts: str) -> float:
def _vtt_s_to_ts(x: float) -> str:
x = max(0.0, x)
h = int(x // 3600)
x -= h * 3600
m = int(x // 60)
x -= m * 60
s = int(x)
ms = int(round((x - s) * 1000))
if ms >= 1000:
s += 1
ms -= 1000
# Rounded to whole milliseconds FIRST, then split: rounding the fraction last carried a rounded-up
# millisecond into the seconds and no further, so 59.9996 came out as `00:00:60.000` — a timestamp
# the WebVTT parser rejects, silently dropping that cue.
h, rest = divmod(int(round(max(0.0, x) * 1000)), 3_600_000)
m, rest = divmod(rest, 60_000)
s, ms = divmod(rest, 1000)
return f"{h:02d}:{m:02d}:{s:02d}.{ms:03d}"
@@ -109,7 +150,7 @@ def ffmpeg_to_vtt(src: str, sub_ord: int | None) -> str:
"webvtt",
"pipe:1",
]
r = subprocess.run(cmd, capture_output=True)
r = subprocess.run(cmd, capture_output=True, timeout=EXTRACT_TIMEOUT_S)
if r.returncode != 0 or not r.stdout:
raise PlexError(
(r.stderr or b"").decode("utf-8", "replace")[:200] or "ffmpeg subtitle convert failed"
@@ -125,11 +166,686 @@ def cache_read(key: str) -> str | None:
return None
def cache_write(key: str, vtt: str) -> None:
def cache_write(key: str, vtt: str) -> bool:
"""Atomically replace the cached VTT; False when it could not be written (`publish` must then
not claim the region it was carrying)."""
try:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
tmp = CACHE_DIR / f"{key}.vtt.tmp"
tmp.write_text(vtt, encoding="utf-8")
tmp.replace(CACHE_DIR / f"{key}.vtt") # atomic publish
return True
except OSError:
return False
# --- cache identity -----------------------------------------------------------------------------
# The cached VTT holds ABSOLUTE cue times (per rating_key + subtitle ordinal); the per-session shift
# is applied on the way out, so a resume/seek to any offset reuses the same file. Beside it, a small
# JSON sidecar records WHICH absolute-time ranges the file actually covers — a windowed extraction
# publishes a usable but incomplete track, and "incomplete" must never be mistaken for "all there is".
_FULL_END_S = 1_000_000_000.0 # stand-in for "to the end of the file" (JSON can't hold infinity)
def cache_key(rating_key: str, sub_ord: int) -> str:
return f"sub_{rating_key}_{sub_ord}"
def _vtt_path(key: str) -> Path:
return CACHE_DIR / f"{key}.vtt"
def _meta_path(key: str) -> Path:
return CACHE_DIR / f"{key}.json"
def read_meta(key: str) -> tuple[list[list[float]], bool]:
"""Covered absolute-time ranges + whether the track is fully extracted. A VTT with no sidecar
(written by an older build, which only ever cached complete tracks) counts as complete."""
try:
raw = json.loads(_meta_path(key).read_text(encoding="utf-8"))
return [[float(a), float(b)] for a, b in raw.get("ranges", [])], bool(raw.get("complete"))
except (OSError, ValueError, TypeError):
return ([[0.0, _FULL_END_S]], True) if _exists(_vtt_path(key)) else ([], False)
def _exists(p: Path) -> bool:
"""`Path.exists` that answers False for a path the OS refuses outright: pathlib swallows only
ENOENT-like errors, so an over-long name (ENAMETOOLONG, from an absurd rating_key or ordinal)
escaped as a 500 from the status route."""
try:
return p.exists()
except (OSError, ValueError):
return False
def _write_meta(key: str, ranges: list[list[float]], complete: bool) -> None:
try:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
tmp = CACHE_DIR / f"{key}.json.tmp"
tmp.write_text(json.dumps({"ranges": ranges, "complete": complete}), encoding="utf-8")
tmp.replace(_meta_path(key))
except OSError:
pass
def generation(key: str) -> int:
"""Monotonic id of the cached file's current content. It goes into the <track> URL so that when a
window is superseded by a wider one the browser refetches instead of keeping the short VTT."""
try:
return _vtt_path(key).stat().st_mtime_ns
except (OSError, ValueError): # ValueError: a NUL in the key — `stat` refuses the path outright
return 0
def merge_ranges(ranges: list[list[float]]) -> list[list[float]]:
"""Sort + coalesce touching/overlapping covered ranges."""
out: list[list[float]] = []
for a, b in sorted((r for r in ranges if r[1] > r[0]), key=lambda r: r[0]):
if out and a <= out[-1][1]:
out[-1][1] = max(out[-1][1], b)
else:
out.append([a, b])
return out
def covers(ranges: list[list[float]], at: float) -> bool:
return any(a <= at <= b for a, b in ranges)
def merge_vtt(*sources: str) -> str:
"""Union of the cue blocks of several absolute-time VTTs, ordered by start time.
Two extraction passes over the same file emit identical cues for the region they share, so they
are de-duplicated — but NOT on the raw timing text. The two passes do not spell a timestamp the
same way: a window comes back through `shift_vtt`, which always writes `HH:MM:SS.mmm`, while a
full pass is ffmpeg's own output, and its webvtt muxer drops the hour below one hour
(`10:00.123`). Keyed on the text, every cue of a window in the first hour survived twice and the
viewer saw each line stacked on itself. So each timing line is parsed and re-emitted in one
canonical form, and the key is that canonical timing PLUS the cue text: two different cues that
happen to share a time (two speakers, a sign over dialogue) are both kept."""
seen: set[str] = set()
cues: list[tuple[float, str]] = []
for src in sources:
for block in re.split(r"\n[ \t]*\n", src or ""):
canon = _canonical_cue(block.strip())
if canon is None:
continue # header / NOTE / STYLE — the merged file gets its own single header
start, key, text = canon
if key in seen:
continue
seen.add(key)
cues.append((start, text))
cues.sort(key=lambda c: c[0])
return "WEBVTT\n\n" + "\n\n".join(b for _, b in cues) + "\n"
def _canonical_cue(block: str) -> tuple[float, str, str] | None:
"""(start seconds, identity, the block with its timing line canonicalised) — or None for a block
that is not a cue. The identity leaves out anything before the timing line (an SRT-style cue
number), which is a label, not content."""
lines = block.split("\n")
for i, ln in enumerate(lines):
if "-->" not in ln:
continue
left, _, right = ln.partition("-->")
rparts = right.strip().split(None, 1)
settings = (" " + rparts[1]) if len(rparts) > 1 else ""
try:
st = _vtt_ts_to_s(left.strip())
en = _vtt_ts_to_s(rparts[0])
except (ValueError, IndexError):
return None
lines[i] = f"{_vtt_s_to_ts(st)} --> {_vtt_s_to_ts(en)}{settings}"
return st, "\n".join(lines[i:]), "\n".join(lines)
return None
def _last_cue_end(vtt: str) -> float:
"""Where the last cue of a VTT ends — how far a pass that died early provably got."""
end = 0.0
for ln in vtt.split("\n"):
if "-->" not in ln:
continue
try:
end = max(end, _vtt_ts_to_s(ln.partition("-->")[2].strip().split(None, 1)[0]))
except (ValueError, IndexError):
continue
return end
def publish(key: str, vtt: str, covered: tuple[float, float], complete: bool) -> bool:
"""Merge a freshly extracted region into the cached track and widen its covered ranges. False
when the region did not land (the cache is unwritable, or the VTT there cannot be read) — the
caller must then record a FAILED pass: returning quietly let the next poll schedule the same read
again, a whole-container pass every time, while the player said "preparing" for good.
A read-merge-write, so it runs under `_PUBLISH_LOCK`: two jobs publish into the same keys (a
window and the full pass, and every embedded track of the file rides each of them), and without
the lock the later writer drops whatever the earlier one merged — permanently, once `complete` is
set. It is also the only writer of the cache files, which is what keeps their fixed `.tmp`
staging names safe."""
with _PUBLISH_LOCK:
old = cache_read(key)
if old is None and _vtt_path(key).exists():
# There, but unreadable (`cache_read` swallows the OSError). Merging into "" would replace
# a whole track with this one region while its sidecar still claimed all of it.
log.warning("subtitle cache %s unreadable; not publishing into it", key)
return False
old_ranges, was_complete = read_meta(key)
if old is None:
# A sidecar with no VTT behind it claims cues that are not there, so it is not merged
# into. And a VTT with no sidecar reads as a COMPLETE track (see read_meta — how older
# builds cached), while a publish is two renames: so before a key's first VTT lands, its
# sidecar is made to claim nothing. Dying between the two then leaves a window that is
# extracted again, never one taken for the whole track for good.
old_ranges, was_complete = [], False
_write_meta(key, [], False)
if not cache_write(key, merge_vtt(old or "", vtt)):
log.warning("subtitle cache %s could not be written", key)
return False # the region never landed, so the sidecar must not claim it
_write_meta(key, merge_ranges(old_ranges + [list(covered)]), complete or was_complete)
return True
# --- background extraction ----------------------------------------------------------------------
# How much of the track to extract before the viewer sees anything. WINDOW_PRE_S starts a little
# before the playhead so a cue that is already on screen is not missed; WINDOW_LEN_S is the span
# published as the first, provisional VTT (~4 s of work on the measured file). It only has to hold
# until the full pass lands behind it — 111 s for that file — so it is deliberately much longer than
# any realistic full pass.
WINDOW_PRE_S = 30.0
WINDOW_LEN_S = 300.0
# …but it does not always land: a full pass can be backed off (FULL_RETRY_S) or simply slower than
# the window lasts (a file past ~40 GB, or several passes sharing the array). A window was only asked
# for once the playhead had LEFT coverage, so every edge unmounted the track for a poll plus an
# extraction. Within this distance of a covered stretch's end, the next window is read ahead.
LOOKAHEAD_S = 60.0
# Ceiling on one ffmpeg run so a wedged extraction can't hold a pool thread forever (the old
# in-request extraction had no timeout at all).
EXTRACT_TIMEOUT_S = 900.0
# How long a timed-out ffmpeg gets, after a TERM, to close its outputs before it is killed.
_TERM_GRACE_S = 10.0
# A failed track (or window) is remembered briefly so a 1 s status poll doesn't respawn ffmpeg every
# second, and forgotten again soon after so fixing the file (or Plex coming back) doesn't need a
# restart.
FAIL_TTL_S = 60.0
# …except a FULL pass that failed. That is a read of the whole container (minutes), a failure that
# reproduces does so every time, and the viewer is typically inside what it did publish — so the
# short memo re-armed a whole-file read every minute for as long as they watched. Windows around the
# playhead still run meanwhile; only the full pass waits this out.
FULL_RETRY_S = 3600.0
# The furthest position the status poll accepts. No film is this long, and an absurd one used to
# reach ffmpeg's `-ss` (which rejects anything past its int64 microseconds).
MAX_POSITION_S = 100_000.0
# Windows and full passes run on SEPARATE pools, so a window never waits behind a full pass. They
# shared one FIFO pool of three whose comment promised "room for a window" that nothing enforced:
# stepping through three episodes with subtitles on left three full passes (minutes each) holding every
# worker, and a seek window on the third queued behind them. Two of each: bounded, so a burst can't
# thrash the array.
_WIN_POOL = ThreadPoolExecutor(max_workers=2, thread_name_prefix="plex-subs-win")
_FULL_POOL = ThreadPoolExecutor(max_workers=2, thread_name_prefix="plex-subs-full")
# Bounds on what may wait for those pools — separate too, so held full-pass slots (minutes each)
# cannot refuse a window (seconds). The queues are unbounded and every job is a read of the library:
# a viewer scrubbing through a film queued a window per stop (the one where they stopped ran LAST),
# and a script could queue thousands. Past a bound nothing is queued — the next poll asks again, by
# which time something has finished.
_MAX_WINDOWS_INFLIGHT = 8
_MAX_FULL_INFLIGHT = 6
_MAX_TRACK_WINDOWS = 2
_LOCK = threading.Lock()
# Queued or running jobs → the absolute span a window job reads (None for a full pass).
_INFLIGHT: dict[tuple, tuple[float, float] | None] = {}
# Track-level failures: a track that cannot be served at all (no item, a bitmap track, Plex down…).
_FAILURES: dict[str, tuple[str, float]] = {}
# rating_key → when its full pass last failed (see FULL_RETRY_S).
_FULL_FAILED: dict[str, float] = {}
# cache key → the spans windows failed to read, with when. A failed window says nothing about the rest
# of the track — the full pass may be about to deliver it — so it is remembered as a SPAN: it must be
# neither respawned every poll nor taken as a failure of positions it never tried. (It used to set the
# track-level memo, which told every viewer at any uncovered position "failed" — a final answer that
# ends the player's poll — including one planted by a request for an absurd position.)
_WIN_FAILED: dict[str, list[tuple[float, float, float]]] = {}
_PUBLISH_LOCK = threading.Lock()
# The ffmpeg children currently running, so a shutdown can kill them (see `shutdown`), and whether it
# has: a worker that reaches its Popen after the kill sweep must not start a read.
_PROCS: set[subprocess.Popen] = set()
_STOPPING = False
class SubtitleUnavailable(Exception):
"""A track that cannot be served as text, carrying the stable code the route maps to a status."""
def __init__(self, code: str):
super().__init__(code)
self.code = code
# code -> HTTP status. An extraction/upstream failure is 422 with a structured detail (502 is
# reserved for the raw byte proxies); the rest are genuine "no such thing" / "not text".
UNAVAILABLE_STATUS = {
"no_item": 404,
"no_track": 404,
"no_file": 404,
"not_fetchable": 404,
"bitmap": 415,
"extract_failed": 422,
"plex_error": 422,
}
# The codes that are a FINAL answer about the track: it can never be served as text, so the player
# stops asking. The others (`plex_error`, `extract_failed`) are about now or about one stretch — a
# memo that lapses, a window that failed where the rest is fine — and answered as `unavailable`: said,
# but asked again. Given as `failed`, one Plex hiccup ended the subtitles for the rest of the session,
# at cached positions too, and after the full pass had completed behind it.
FINAL_CODES = frozenset({"no_item", "no_track", "no_file", "not_fetchable", "bitmap"})
def reset_state() -> None:
"""Drop the in-flight/failure registries (tests; nothing in the app needs it)."""
global _STOPPING
with _LOCK:
_INFLIGHT.clear()
_FAILURES.clear()
_FULL_FAILED.clear()
_WIN_FAILED.clear()
_STOPPING = False
def shutdown() -> None:
"""Stop extracting, for the app's shutdown. The pool's workers are non-daemon threads and Python
joins them at interpreter exit — so a worker still inside a full pass (minutes on a big file, up
to EXTRACT_TIMEOUT_S) held the process open after uvicorn had stopped, until the container runtime
SIGKILLed it. Queued jobs are dropped and running ffmpeg children are killed, which returns their
workers at once — and a worker still BEFORE its ffmpeg (the Plex lookup, the DB) finds the flag
and never starts one. Nothing that matters is lost: an interrupted publish cannot pass for a whole
track (see `publish`), and an interrupted track is simply extracted again on the next poll."""
global _STOPPING
with _LOCK:
_STOPPING = True
procs = list(_PROCS)
for pool in (_WIN_POOL, _FULL_POOL):
pool.shutdown(wait=False, cancel_futures=True)
for p in procs:
p.kill()
def _sweep_memos() -> None:
"""(Caller holds _LOCK.) Drop every lapsed failure memo. They were only ever pruned when the SAME
key was read again, and the status route takes any rating_key/ordinal — so a script polling fresh
keys grew the API process without bound. Swept on every write, they hold at most a TTL's worth."""
now = time.monotonic()
for k in [k for k, (_, t) in _FAILURES.items() if now - t >= FAIL_TTL_S]:
del _FAILURES[k]
for k in [k for k, t in _FULL_FAILED.items() if now - t >= FULL_RETRY_S]:
del _FULL_FAILED[k]
for k in list(_WIN_FAILED):
_live_window_failures(k)
def _fail(key: str, code: str) -> None:
with _LOCK:
_sweep_memos()
_FAILURES[key] = (code, time.monotonic())
def _recent_failure(key: str) -> str | None:
with _LOCK:
hit = _FAILURES.get(key)
if hit and time.monotonic() - hit[1] < FAIL_TTL_S:
return hit[0]
_FAILURES.pop(key, None)
return None
def _full_failed(rating_key: str) -> bool:
with _LOCK:
when = _FULL_FAILED.get(rating_key)
if when is not None and time.monotonic() - when < FULL_RETRY_S:
return True
_FULL_FAILED.pop(rating_key, None)
return False
def _live_window_failures(key: str) -> list[tuple[float, float, float]]:
"""(Caller holds _LOCK.) The key's failed-window spans that are still remembered."""
now = time.monotonic()
live = [f for f in _WIN_FAILED.get(key, []) if now - f[2] < FAIL_TTL_S]
if live:
_WIN_FAILED[key] = live
else:
_WIN_FAILED.pop(key, None)
return live
def _window_fail(key: str, span: tuple[float, float]) -> None:
with _LOCK:
_sweep_memos()
_WIN_FAILED[key] = [*_live_window_failures(key), (span[0], span[1], time.monotonic())]
def _window_failed(key: str, at: float) -> bool:
with _LOCK:
return any(a <= at <= b for a, b, _ in _live_window_failures(key))
def ensure(rating_key: str, sub_ord: int, at: float = 0.0) -> dict:
"""State of one subtitle track, scheduling whatever extraction is still missing. NEVER blocks.
``ready`` = fully extracted · ``partial`` = usable, covers ``at``, still growing · ``preparing``
= nothing to show for ``at`` yet · ``unavailable`` = nothing for ``at`` and nothing being read
for it right now (``code`` says why), ask again · ``failed`` = the track can never be shown
(``code`` says why) — the only final answer. ``gen`` changes whenever the cached file grows, so
the client can refetch a window that has been superseded; 0 = nothing is cached at all."""
key = cache_key(rating_key, sub_ord)
ranges, complete = read_meta(key)
have = cache_read(key) is not None
covered = have and covers(ranges, at)
gen = generation(key)
if complete and have:
return {"state": "ready", "code": None, "gen": gen}
if not have and _exists(_vtt_path(key)):
# On disk but unreadable (permissions, a bad volume): nothing can serve it, and no extraction
# can help — `publish` refuses to merge into it, and a job whose sidecar covers the position
# skips as already done — so scheduling only kept the player on "preparing" for good. `gen` 0:
# there is nothing a <track> could fetch.
return {"state": "unavailable", "code": "cache_unreadable", "gen": 0}
code = _recent_failure(key)
if code and not covered:
return {
"state": "failed" if code in FINAL_CODES else "unavailable",
"code": code,
"gen": gen,
}
# Nothing left that could cover the position for now: the full pass failed, AND so did a window
# here. Still not final — the memos lapse, and the rest of the track may be right there.
full_failed = _full_failed(rating_key)
window_failed = not covered and _window_failed(key, at)
if full_failed and window_failed:
return {"state": "unavailable", "code": "extract_failed", "gen": gen}
if not code:
probe, want_window = at, not covered and not window_failed
if covered:
# Read ahead of an edge instead of waiting to fall off it (see LOOKAHEAD_S). The probe is
# just PAST the edge: `covers` is inclusive, so the edge itself reads as already done.
edge = max(b for a, b in ranges if a <= at <= b)
if edge - at <= LOOKAHEAD_S and edge < _FULL_END_S:
probe = edge + 1.0
want_window = not _window_failed(key, probe)
_schedule(rating_key, sub_ord, probe, want_window=want_window, want_full=not full_failed)
return {"state": "partial" if covered else "preparing", "code": None, "gen": gen}
def cached_track(rating_key: str, sub_ord: int) -> tuple[str | None, str | None]:
"""What the <track> fetch can serve RIGHT NOW, scheduling nothing: the cached VTT (whatever it
covers), else the recent failure that explains why there is none.
Scheduling belongs to `ensure` — the status poll, which knows where the viewer is. The fetch does
not: when it scheduled too, it judged readiness at position 0, so a viewer who resumed an hour in
got an unrequested window over the head of the file (plus a remount when it landed), and with a
failure memo alive it was refused a window that WAS on disk while the poll called it showable."""
key = cache_key(rating_key, sub_ord)
hit = cache_read(key)
return (hit, None) if hit is not None else (None, _recent_failure(key))
def _schedule(
rating_key: str, sub_ord: int, at: float, want_window: bool, want_full: bool = True
) -> None:
"""Kick off the jobs this track still needs: the window that makes it watchable now, and the one
full pass that makes it complete. A 1 s status poll (or a second viewer) joins a job already
queued or running instead of starting another ffmpeg.
A window is joined when one of this track's in-flight windows will COVER the position asked about
— decided by what it reads, not by an identity derived from the position. The poll sends the live
playhead, so a position-keyed window was a new job every second; bucketing the position instead
still queued a second read when the playhead crossed a bucket edge, and joined a backward seek to
a window that could not cover it. Nor may it be one window per track: a window cut around the
wrong spot (the first poll after a resume can still report 0) made the right one wait for it. The
full pass is per ITEM, because one read fans out to every embedded text track of the file."""
jobs: list[tuple[tuple, float | None]] = []
with _LOCK:
if _STOPPING:
return
n_windows = sum(1 for span in _INFLIGHT.values() if span is not None)
n_full = len(_INFLIGHT) - n_windows
if want_window and n_windows < _MAX_WINDOWS_INFLIGHT:
spans = [
span
for ident, span in _INFLIGHT.items()
if span is not None and ident[1:3] == (rating_key, sub_ord)
]
if not any(a <= at <= b for a, b in spans) and len(spans) < _MAX_TRACK_WINDOWS:
# Rounded to what ffmpeg is told (`-ss %.3f`): the cues come back on ffmpeg's zero,
# and shifting them by an unrounded start put them up to half a millisecond off —
# enough to spell a timestamp differently from the full pass and double the cue.
w_start = float(f"{max(0.0, at - WINDOW_PRE_S):.3f}")
ident = ("win", rating_key, sub_ord, w_start)
_INFLIGHT[ident] = (w_start, w_start + WINDOW_PRE_S + WINDOW_LEN_S)
jobs.append((ident, w_start))
if want_full and n_full < _MAX_FULL_INFLIGHT and ("full", rating_key) not in _INFLIGHT:
_INFLIGHT[("full", rating_key)] = None
jobs.append((("full", rating_key), None))
for ident, window_start in jobs:
pool = _FULL_POOL if window_start is None else _WIN_POOL
pool.submit(_run, ident, rating_key, sub_ord, window_start, at)
def _already_done(key: str, window_start: float | None, at: float) -> bool:
"""A queued job looks again before it reads anything. While it waited, another job may have
covered it: every embedded track rides each pass over its file, so a window for one language also
publishes the others."""
if not _exists(_vtt_path(key)):
return False
ranges, complete = read_meta(key)
return complete or (window_start is not None and covers(ranges, at))
def _run(
ident: tuple, rating_key: str, sub_ord: int, window_start: float | None, at: float
) -> None:
key = cache_key(rating_key, sub_ord)
try:
if _STOPPING or _already_done(key, window_start, at):
return
_extract_and_publish(rating_key, sub_ord, window_start)
except SubtitleUnavailable as e:
if e.code == "extract_failed":
_pass_failed(rating_key, [key], window_start)
else:
_fail(key, e.code)
except PlexError as e:
log.warning("subtitle extraction failed for %s/%s: %s", rating_key, sub_ord, e)
_fail(key, "plex_error")
except (
Exception
): # a worker thread has nobody to raise to — record it, let the next poll see it
log.exception("subtitle extraction crashed for %s/%s", rating_key, sub_ord)
_pass_failed(rating_key, [key], window_start)
finally:
with _LOCK:
_INFLIGHT.pop(ident, None)
def _pass_failed(
rating_key: str,
keys: list[str],
window_start: float | None,
span: tuple[float, float] | None = None,
) -> None:
"""Record an ffmpeg pass that failed: a full pass backs off the whole item (FULL_RETRY_S); a
window marks only the stretch it failed to read, for the tracks it was reading."""
if window_start is None:
with _LOCK:
_sweep_memos()
_FULL_FAILED[rating_key] = time.monotonic()
return
for key in keys:
_window_fail(key, span or (window_start, window_start + WINDOW_PRE_S + WINDOW_LEN_S))
def _text_streams(db: Session, rating_key: str) -> tuple[PlexItem, list[dict]]:
"""The item plus its subtitle streams, in the ordinal order the API and the UI both use."""
it = by_rating_key(db, PlexItem, rating_key)
if it is None:
raise SubtitleUnavailable("no_item")
with PlexClient(db) as plex:
part = ((plex.metadata(it.rating_key) or {}).get("Media") or [{}])[0].get("Part") or [{}]
return it, [s for s in (part[0].get("Stream") or []) if s.get("streamType") == 3]
def _is_bitmap(stream: dict) -> bool:
return (stream.get("codec") or stream.get("format") or "").lower() in BITMAP_SUBS
def _extract_and_publish(rating_key: str, sub_ord: int, window_start: float | None) -> None:
"""One extraction job. A sidecar track is fetched whole from Plex (cheap, no ffmpeg); an embedded
one is extracted together with EVERY other embedded text track in the file, because they all ride
on the same single read — the whole point of the measurement at the top of this module."""
with SessionLocal() as db:
it, subs = _text_streams(db, rating_key)
if sub_ord < 0 or sub_ord >= len(subs):
raise SubtitleUnavailable("no_track")
want = subs[sub_ord]
if _is_bitmap(want):
raise SubtitleUnavailable("bitmap")
if want.get("key"):
# An external sidecar is one small fetch from Plex, never a read of the media file, so a
# window job fetches the whole thing too. It must not be left to the full job: that
# identity is per ITEM (the embedded fan-out shares it), so a sidecar picked while an
# embedded track's full pass was running sat behind that pass for minutes.
vtt = _sidecar_vtt(db, want)
if not publish(cache_key(rating_key, sub_ord), vtt, (0.0, _FULL_END_S), True):
raise SubtitleUnavailable("extract_failed")
return
if want.get("index") is None:
raise SubtitleUnavailable("not_fetchable")
src = plex_paths.local_media_path(db, it.file_path)
if src is None:
raise SubtitleUnavailable("no_file")
# Ordinal among the file's EMBEDDED subtitle streams — what `-map 0:s:N` counts. Sidecars sit
# in the same list but not in the file, so they must not shift the mapping.
targets = {
sum(1 for x in subs[:i] if x.get("index") is not None): i
for i, s in enumerate(subs)
if s.get("index") is not None and not _is_bitmap(s)
}
start = window_start or 0.0
length = (WINDOW_LEN_S + WINDOW_PRE_S) if window_start is not None else None
end = _FULL_END_S if length is None else start + length
texts, clean = _ffmpeg_extract(str(src), sorted(targets), window_start, length)
for emb_ord, vtt in texts.items():
key = cache_key(rating_key, targets[emb_ord])
# A pass that exited non-zero holds only what ffmpeg reached before it died. Claim no more
# than its last cue, never call it complete, and remember the failure: published as complete,
# the track read "ready" for good while its subtitles stopped wherever ffmpeg did (the cache
# has no eviction, so nothing would ever have extracted it again).
covered_end = end if clean else start + _last_cue_end(vtt)
landed = publish(
key,
shift_vtt(vtt, -start), # zero-based window clock → absolute cue times
(start, covered_end),
clean and length is None,
)
if not landed:
# A read that could not be kept is a failed read (see publish) — backed off like one.
_pass_failed(rating_key, [key], window_start, (start, end))
elif not clean:
_pass_failed(rating_key, [key], window_start, (covered_end, end))
def _sidecar_vtt(db: Session, stream: dict) -> str:
"""An external subtitle file, fetched through Plex and converted to WebVTT."""
with PlexClient(db) as plex:
raw = plex.raw_get(stream["key"])
codec = (stream.get("codec") or stream.get("format") or "").lower()
if codec in ("srt", "subrip", ""):
return srt_to_vtt(raw.decode("utf-8", "replace"))
with tempfile.NamedTemporaryFile(suffix="." + (codec or "sub"), delete=False) as fh:
fh.write(raw)
tmp = fh.name
try:
return ffmpeg_to_vtt(tmp, None)
finally:
Path(tmp).unlink(missing_ok=True)
def _ffmpeg_extract(
src: str, emb_ords: list[int], start: float | None, length: float | None
) -> tuple[dict[int, str], bool]:
"""Extract several embedded subtitle streams to WebVTT in ONE ffmpeg run (= one read of the file).
Returns the texts and whether ffmpeg exited cleanly — a non-zero exit can still leave partial
output, which is worth publishing but must never be taken for the whole track.
`start`/`length` bound the read to a window. Without `-copyts` the cues come out on a zero-based
clock (the caller shifts them back) — which is exactly why `-copyts` is absent: with it ffmpeg
ignores `-t` and reads the whole container anyway (measured)."""
if not emb_ords:
return {}, True
with tempfile.TemporaryDirectory(prefix="plexsub-") as tmpdir:
cmd = ["ffmpeg", "-nostdin", "-loglevel", "error"]
if start:
cmd += ["-ss", f"{start:.3f}"]
cmd += ["-i", src]
outs = {o: Path(tmpdir) / f"{o}.vtt" for o in emb_ords}
for emb_ord, out in outs.items():
cmd += ["-map", f"0:s:{emb_ord}"]
if length is not None:
cmd += ["-t", f"{length:.3f}"]
# The input seek lands on the keyframe at or BEFORE `start`, so a cue already running
# at `start` comes out at a negative time — and by default the muxer then shifts the
# WHOLE output so that cue sits at 0: every cue of the window late by the distance to
# that keyframe (1–5 s measured on a real library file, a whole GOP on a synthetic
# one), and doubled for good once the full pass merged in beside it. Kept negative, the
# times are exact on the zero-based clock (`_vtt_ts_to_s` reads the sign).
cmd += ["-avoid_negative_ts", "disabled"]
cmd += ["-f", "webvtt", "-y", str(out)]
# Popen rather than run(): the child is registered so `shutdown` can kill it — and one that
# would start after the shutdown's kill sweep never does, or is killed right here.
with _LOCK:
if _STOPPING:
raise SubtitleUnavailable("extract_failed")
proc = subprocess.Popen(
cmd, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE
)
with _LOCK:
_PROCS.add(proc)
if _STOPPING:
proc.kill()
timed_out = False
try:
try:
_, err = proc.communicate(timeout=EXTRACT_TIMEOUT_S)
except subprocess.TimeoutExpired:
# A pass that ran out of time is a pass that died partway: what it reached is kept and
# published as partial (the caller's unclean path), like any other non-zero exit. It
# used to be killed and its outputs deleted unread — and with the full-pass backoff, the
# same doomed read came back every hour. TERM first: ffmpeg closes its outputs on it,
# whereas a KILL leaves only what was already flushed in whole blocks (measured: none).
timed_out = True
proc.terminate()
try:
_, err = proc.communicate(timeout=_TERM_GRACE_S)
except subprocess.TimeoutExpired:
proc.kill()
_, err = proc.communicate()
finally:
with _LOCK:
_PROCS.discard(proc)
got = {
o: p.read_text(encoding="utf-8", errors="replace")
for o, p in outs.items()
if p.exists()
}
if not got:
raise SubtitleUnavailable("extract_failed")
clean = proc.returncode == 0 and not timed_out
if not clean:
log.warning(
"ffmpeg subtitle pass %s: %s",
"timed out" if timed_out else f"exited {proc.returncode}",
(err or b"")[:200],
)
return got, clean
+13 -5
View File
@@ -17,12 +17,20 @@ from app.models import User
M = TypeVar("M", bound=Base)
def get_or_404(db: Session, model: type[M], id_: Any, *, detail: str = "Not found") -> M:
def not_found(what: str = "generic") -> HTTPException:
"""The 404 for a row that is gone — structured, so the bilingual UI says it in the reader's
language (`errors.notFound.<what>`, generic fallback). This is the 404 a user actually meets: the
thing was deleted in another tab or on another device. It used to be a bare English string
("Unknown view", "Not found"), which the Hungarian UI toasted verbatim."""
return HTTPException(status_code=404, detail={"code": "not_found", "reason": what})
def get_or_404(db: Session, model: type[M], id_: Any, *, what: str = "generic") -> M:
"""Fetch a row by primary key or raise 404. Replaces the `db.get(Model, id); if None: 404`
idiom repeated across routes."""
obj = db.get(model, id_)
if obj is None:
raise HTTPException(status_code=404, detail=detail)
raise not_found(what)
return obj
@@ -33,13 +41,13 @@ def owned_or_404(
user: User,
*,
owner_attr: str = "user_id",
detail: str = "Not found",
what: str = "generic",
) -> M:
"""Fetch a row by primary key that `user` owns, or raise 404. A row owned by someone
else 404s (NOT 403) on purpose — a 403 would confirm the id exists to a stranger.
Generalizes the near-identical per-route ownership helpers (e.g. plex's
`_own_playlist_or_404`, the saved-views owner check)."""
obj = get_or_404(db, model, id_, detail=detail)
obj = get_or_404(db, model, id_, what=what)
if getattr(obj, owner_attr) != user.id:
raise HTTPException(status_code=404, detail=detail)
raise not_found(what)
return obj
+10 -16
View File
@@ -173,7 +173,7 @@ def _activate_user_by_email(db: Session, email: str) -> None:
def _decide(db: Session, invite_id: int, admin: User, approved: bool) -> Invite:
inv = get_or_404(db, Invite, invite_id, detail="No such invite")
inv = get_or_404(db, Invite, invite_id, what="invite")
# Idempotency guard: only a still-pending invite can be decided. Without this a repeat/stale
# approve or deny re-runs its side effects — re-activating a user, or (now that deny emails)
# sending a duplicate decision mail, even flipping an already-approved invite to "denied" and
@@ -291,7 +291,7 @@ def send_invitation(
Only an APPROVED row can be invited: there is nothing to invite someone to while their request is
undecided, and inviting a denied address would contradict the decision."""
inv = get_or_404(db, Invite, invite_id, detail="No such invite")
inv = get_or_404(db, Invite, invite_id, what="invite")
if inv.status != "approved":
raise HTTPException(
status_code=409,
@@ -364,7 +364,7 @@ def delete_invite(
falls back to the env ALLOWED_EMAILS / ADMIN_EMAILS bootstrap lists once no invite exists, so for
an address on those the deletion leaves access intact (and an ADMIN_EMAILS address is still
promoted to admin on its next Google sign-in). The admin has to hear that, not guess it."""
inv = get_or_404(db, Invite, invite_id, detail="No such invite")
inv = get_or_404(db, Invite, invite_id, what="invite")
if _accounts_for(db, [inv.email]):
raise HTTPException(
status_code=409,
@@ -434,9 +434,7 @@ def set_user_role(
role = body.role
if role not in ("user", "admin"):
raise HTTPException(status_code=400, detail="role must be 'user' or 'admin'")
target = db.get(User, user_id)
if target is None:
raise HTTPException(status_code=404, detail="No such user")
target = get_or_404(db, User, user_id, what="user")
if target.is_demo:
raise HTTPException(status_code=400, detail="The demo account's role can't be changed.")
if target.id == admin.id:
@@ -491,9 +489,7 @@ def set_user_suspended(
un-suspend the user is emailed they can return (suspend is announced reactively, on a blocked
sign-in)."""
suspended = body.suspended
target = db.get(User, user_id)
if target is None:
raise HTTPException(status_code=404, detail="No such user")
target = get_or_404(db, User, user_id, what="user")
if target.is_demo:
raise HTTPException(status_code=400, detail="The demo account can't be suspended.")
if target.id == admin.id:
@@ -537,9 +533,7 @@ def admin_delete_user(
"""Admin-triggered account deletion — the same full erasure a user can perform on themselves
(cascade delete of all personal data + access-request cleanup + Google-grant revoke). Guards:
not the demo account, not yourself (use Settings → Account), and not the last admin."""
target = db.get(User, user_id)
if target is None:
raise HTTPException(status_code=404, detail="No such user")
target = get_or_404(db, User, user_id, what="user")
if target.is_demo:
raise HTTPException(status_code=400, detail="The demo account can't be deleted.")
if target.id == admin.id:
@@ -632,7 +626,7 @@ def user_access(
Its own endpoint rather than extra columns on /users: `is_allowed` and `can_activate` each run a
query, and the panel opens for ONE user at a time — folding them into the list would pay that cost
per row on every load to answer a question nobody asked yet."""
target = get_or_404(db, User, user_id, detail="No such user")
target = get_or_404(db, User, user_id, what="user")
return _serialize_user(target) | {
"is_allowed": is_allowed(db, target.email),
"can_activate": can_activate(db, target),
@@ -657,7 +651,7 @@ def create_set_password_link(
an instance with no mail — narrowing it to one of those would only make the admin's real case the
one it refuses. The demo account is the single exception: it is shared, so handing anyone a way to
change its password is handing them everyone else's session."""
target = get_or_404(db, User, user_id, detail="No such user")
target = get_or_404(db, User, user_id, what="user")
if target.is_demo:
raise HTTPException(
status_code=400, detail={"code": "setpw", "reason": "demo", "email": target.email}
@@ -693,7 +687,7 @@ def activate_user(
This endpoint changes nothing about what an unauthenticated caller can cause — the admin could
already do exactly this by re-entering the address in the whitelist box (`add_invite` →
`_activate_user_by_email`). What was missing was any way to know they had to."""
target = get_or_404(db, User, user_id, detail="No such user")
target = get_or_404(db, User, user_id, what="user")
if target.is_demo:
raise HTTPException(status_code=400, detail="The demo account can't be activated.")
if target.is_active:
@@ -732,7 +726,7 @@ def resend_user_verification(
Not rate-limited: an admin is not a threat model to enumerate against, and the deliberate silence
of the public endpoint is exactly what this exists to escape."""
target = get_or_404(db, User, user_id, detail="No such user")
target = get_or_404(db, User, user_id, what="user")
if target.is_demo or target.email_verified:
raise HTTPException(
status_code=409,
+2 -2
View File
@@ -1246,7 +1246,7 @@ def logout_others(
are rejected, then re-stamp THIS cookie so the caller stays signed in. For "I left myself logged
in somewhere" / a suspected session compromise. The demo account (shared) is excluded."""
if user.is_demo:
raise HTTPException(status_code=403, detail="Not available in the demo account.")
raise HTTPException(status_code=403, detail={"code": "demo"})
# Commit the bump BEFORE re-stamping this cookie, so a failed commit can't strand the current
# session at a newer epoch than the DB (see set_password).
bump_session_epoch(user)
@@ -1293,7 +1293,7 @@ def set_password(
account) needs no current password — the session already proves identity. Changing an
existing one requires the current password, so a hijacked session can't silently rotate it."""
if user.is_demo:
raise HTTPException(status_code=403, detail="Not available in the demo account.")
raise HTTPException(status_code=403, detail={"code": "demo"})
new_password = body.password or ""
if len(new_password) < PASSWORD_MIN_LEN:
raise HTTPException(
+8 -17
View File
@@ -26,7 +26,7 @@ from app.models import (
User,
Video,
)
from app.routes._common import owned_or_404
from app.routes._common import get_or_404, not_found, owned_or_404
from app.sync.explore import explore_ingest_page
from app.sync.runner import backfill_channel_now, run_recent_backfill
from app.sync.subscriptions import apply_channel_details
@@ -287,9 +287,7 @@ def channel_detail(
needed — a stub, or a channel last enriched before those fields existed (published_at is a
reliable sentinel: every real channel has a creation date). The enrich uses the API key
(1 unit, cached after), so it's skipped for the quota-less demo account."""
channel = db.get(Channel, channel_id)
if channel is None:
raise HTTPException(status_code=404, detail="Unknown channel")
channel = get_or_404(db, Channel, channel_id, what="channel")
if channel.published_at is None and not user.is_demo:
try:
with (
@@ -366,9 +364,7 @@ def explore_channel(
it, then ingests one page of its uploads (newest-first) marked via_explore + enriched.
Returns the next page token so the page can "load more". Guarded against the shared
backfill quota reserve; blocked for the demo account (it spends shared quota)."""
channel = db.get(Channel, channel_id)
if channel is None:
raise HTTPException(status_code=404, detail="Unknown channel")
channel = get_or_404(db, Channel, channel_id, what="channel")
if _is_blocked(db, user, channel_id):
raise HTTPException(status_code=403, detail="You've blocked this channel.")
if quota.backfill_reserve_reached():
@@ -426,7 +422,7 @@ def _user_subscription(db: Session, user: User, channel_id: str) -> Subscription
)
).scalar_one_or_none()
if sub is None:
raise HTTPException(status_code=404, detail="Not subscribed to this channel")
raise not_found("subscription")
return sub
@@ -512,9 +508,7 @@ def reset_backfill(
state (a "reset" trigger). Clears the channel's backfill markers, opts it back into deep
backfill, and re-runs its recent pull immediately; the deep scheduler re-pages the full
back-catalog on its next run. Idempotent — videos upsert by id, so nothing duplicates."""
channel = db.get(Channel, channel_id)
if channel is None:
raise HTTPException(status_code=404, detail="Unknown channel")
channel = get_or_404(db, Channel, channel_id, what="channel")
channel.backfill_done = False
channel.backfill_cursor = None
channel.recent_synced_at = None
@@ -544,9 +538,7 @@ def subscribe(
local subscription with YouTube's returned resource id and enriches the channel's metadata
if it's still a stub, so the new row renders properly in the manager. It does NOT pull the
channel's videos — the scheduler picks those up on its next run, like any subscribed channel."""
channel = db.get(Channel, channel_id)
if channel is None:
raise HTTPException(status_code=404, detail="Unknown channel")
channel = get_or_404(db, Channel, channel_id, what="channel")
existing = db.execute(
select(Subscription).where(
Subscription.user_id == user.id, Subscription.channel_id == channel_id
@@ -663,8 +655,7 @@ def block_channel(
"""Block a channel for this user: its videos won't appear in their live search (and won't be
ingested for them) and are hidden from their feed / explore. Idempotent. The now-hidden
un-kept search/explore videos are reclaimed by the discovery-cleanup job in due course."""
if db.get(Channel, channel_id) is None:
raise HTTPException(status_code=404, detail="Unknown channel")
get_or_404(db, Channel, channel_id, what="channel")
if not _is_blocked(db, user, channel_id):
db.add(BlockedChannel(user_id=user.id, channel_id=channel_id))
# Stop any active exploration of it so it can be cleaned up.
@@ -706,7 +697,7 @@ def attach_tag(
_user_subscription(db, user, channel_id)
# Only the user's own tags may be attached as personal links (owned_or_404 → 404 if the tag
# is missing OR belongs to someone else, without leaking which).
owned_or_404(db, Tag, body.tag_id, user, detail="Unknown tag")
owned_or_404(db, Tag, body.tag_id, user, what="tag")
exists = db.execute(
select(ChannelTag).where(
ChannelTag.channel_id == channel_id,
+197
View File
@@ -0,0 +1,197 @@
"""E24 · the Home surface's HTTP shell.
Thin by construction: every decision — what counts as in progress, what fits a time budget, which
tier the hero comes from — lives in `app/discover/`, so the scheduler, a future CLI or a test can ask
the same questions without a request. This file validates, gates and serializes.
The consolidated `GET /api/discover` mirrors the Plex Discover route's one-payload shape (E5 S4): a
landing page that fires four requests to answer one question would spend its whole budget on the
thing it exists to remove.
"""
from typing import Literal
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy import select, update
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from app.auth import current_user, require_human
from app.db import get_db
from app.discover import hero as hero_mod
from app.discover import pins, rails
from app.discover.budget import Budget, DEFAULT_BUDGET
from app.models import ComfortPin, User
from app.routes._common import owned_or_404
router = APIRouter(prefix="/api/discover", tags=["discover"])
class PinIn(BaseModel):
# From `pins.PIN_KINDS`, not a second hand-written list: the vocabulary, the schema columns and
# this request contract have to move together, and re-typing them here is how they would not.
kind: Literal[*pins.PIN_KINDS]
# The id the rest of the API speaks for that kind: a video id, a channel id, a Plex rating_key,
# or a playlist id. `pins.resolve_ref` does the translation to the row's internal key.
ref: str
class ReorderIn(BaseModel):
ids: list[int]
@router.get("")
def get_discover(
budget: Budget = DEFAULT_BUDGET,
seed: int = 0,
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
"""Everything the Home page renders, in one pass.
`resume` and `shelf` are built once and handed to the hero, so the two rails the hero draws from
are not queried twice for a single page load."""
resume = rails.continue_watching(db, user)
shelf = pins.shelf(db, user)
return {
"budget": budget,
"hero": hero_mod.pick(db, user, budget, seed, resume=resume, shelf=shelf),
"continue_watching": resume,
"comfort": shelf,
}
# How many already-shown cards a reroll may carry, matched to the client's own rolling window
# (`SEEN_WINDOW` in `Home.tsx`). A cheap sanity clamp, and deliberately no more than that: `exclude`
# never reaches SQL — `hero.pick` uses it only as set membership over an already-bounded candidate
# pool (the resume rail's `RAIL_LIMIT` rows, the shelf, `SURPRISE_DRAW` surprise draws) — so there is
# no scan for a crafted URL to unbound, and the set is built in full before this slice anyway. The
# comment that claimed otherwise was corrected in round 2; the request line's own length is what
# actually bounds a hostile `exclude`.
MAX_EXCLUDE = 32
@router.get("/pick")
def get_pick(
budget: Budget = DEFAULT_BUDGET,
seed: int = 0,
exclude: str = "",
start: int = 0,
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
"""The "Mást kérek" reroll: a hero on its own, for a new seed and/or a new budget.
One seed drives BOTH sources (see `discover/queries.py`) — otherwise a reroll would re-draw the
Plex half while the YouTube half stood still, and the button would look broken half the time.
`exclude` is a comma-separated list of `<source>:<id>` keys the user has already been shown. It
is what makes the button honest: a seed alone only moves the surprise tier, so a user with
something half-finished got the same card back every press.
`start` is the press count: it rotates which tier answers first, so consecutive presses offer a
different KIND of answer instead of walking down the resume rail. Taken modulo the tier count in
`hero.pick`, so any integer is valid and no value can 422 a button."""
skip = {p for p in exclude.split(",") if p}
return {
"budget": budget,
"hero": hero_mod.pick(
db, user, budget, seed, exclude=set(list(skip)[:MAX_EXCLUDE]), start=start
),
}
@router.get("/pins")
def list_pins(
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
"""The shelf on its own — for the shelf editor, which needs it without the rest of the page."""
return {"items": pins.shelf(db, user)}
@router.post("/pins")
def add_pin(
body: PinIn,
user: User = Depends(require_human),
db: Session = Depends(get_db),
) -> dict:
"""Pin something to the comfort shelf.
`require_human`, like every other personal-state write: the shared demo account must not be able
to curate a communal shelf. Re-pinning something already on the shelf is a no-op that returns the
existing pin rather than a 409 — the gesture is "make sure this is on my shelf", and the unique
constraint is the mechanism, not the message."""
resolved = pins.resolve_ref(db, user, body.kind, body.ref)
if resolved is None:
raise HTTPException(status_code=404, detail="Unknown item")
column, value = resolved
existing = db.execute(
select(ComfortPin).where(
ComfortPin.user_id == user.id,
getattr(ComfortPin, column) == value,
)
).scalar_one_or_none()
if existing is not None:
return {"ok": True, "id": existing.id, "created": False}
pin = ComfortPin(user_id=user.id, position=pins.next_position(db, user), **{column: value})
db.add(pin)
try:
db.commit()
except IntegrityError:
# A concurrent pin of the same thing from a second tab. The unique constraint did its job;
# the user's intent ("be on my shelf") is satisfied either way, so report the winner rather
# than a 500 the user can do nothing with.
db.rollback()
winner = db.execute(
select(ComfortPin).where(
ComfortPin.user_id == user.id,
getattr(ComfortPin, column) == value,
)
).scalar_one_or_none()
if winner is None:
# Not the unique constraint, then: the only other way in is the target row being pruned
# between `resolve_ref` confirming it and this INSERT (a Plex sync dropping a removed
# item), which trips the FK. Semantically that is the same "no such thing" the pre-check
# returns above — re-raising handed the user a 500 they could do nothing with.
raise HTTPException(status_code=404, detail="Unknown item")
return {"ok": True, "id": winner.id, "created": False}
return {"ok": True, "id": pin.id, "created": True}
@router.delete("/pins/{pin_id}")
def remove_pin(
pin_id: int,
user: User = Depends(require_human),
db: Session = Depends(get_db),
) -> dict:
pin = owned_or_404(db, ComfortPin, pin_id, user, what="pin")
db.delete(pin)
db.commit()
return {"deleted": pin_id}
@router.post("/pins/reorder")
def reorder_pins(
body: ReorderIn,
user: User = Depends(require_human),
db: Session = Depends(get_db),
) -> dict:
"""Rewrite the shelf order from a full list of pin ids.
Whole-list rather than a move-one delta: the shelf is short, and a full rewrite cannot leave two
tiles claiming the same slot the way an interrupted sequence of single moves can. Ids that are not
this user's are ignored rather than 404'd — a stale tab replaying an order that includes a pin
deleted meanwhile should reorder what still exists, not fail wholesale."""
owned = {p.id for p in pins.list_pins(db, user)}
for position, pin_id in enumerate(i for i in body.ids if i in owned):
db.execute(
update(ComfortPin)
.where(ComfortPin.id == pin_id, ComfortPin.user_id == user.id)
.values(position=position)
)
db.commit()
return {"ok": True}
+38 -10
View File
@@ -38,6 +38,7 @@ from app.models import (
MediaAsset,
User,
)
from app.routes._common import get_or_404, not_found
from app.security import hash_password
from app.userscope import is_messageable_user, messageable_clauses
from app.utils import iso
@@ -106,6 +107,7 @@ def _serialize(job: DownloadJob, asset: MediaAsset | None) -> dict:
"phase": job.phase,
"error": job.error,
"error_code": job.error_code,
"error_detail": job.error_detail,
"queue_pos": job.queue_pos,
"created_at": iso(job.created_at),
"source_kind": job.source_kind,
@@ -171,6 +173,32 @@ def _own_job(db: Session, user: User, job_id: int) -> DownloadJob:
return job
def _own_job_for_update(db: Session, user: User, job_id: int) -> DownloadJob:
"""`_own_job` for a transition that moves its asset's ref count (resume, cancel, delete, redownload)
or writes the status those decide on (pause). The job is re-read under a row lock, so two
overlapping requests for one job run one after the other and the second decides on the first's
committed result — never both releasing the same hold, and never a pause turning a job a cancel
just released back into a holder; either can free a file another job still holds.
The asset is locked BEFORE the job: the order a finishing worker child takes them in
(`service.finalize_if_running`), so the two can never deadlock."""
while True:
asset_id = _own_job(db, user, job_id).asset_id
if asset_id is not None:
db.execute(select(MediaAsset.id).where(MediaAsset.id == asset_id).with_for_update())
job = db.execute(
select(DownloadJob)
.where(DownloadJob.id == job_id)
.with_for_update()
.execution_options(populate_existing=True)
).scalar_one_or_none()
if job is None:
raise HTTPException(status_code=404, detail="Unknown download")
if job.asset_id == asset_id:
return job
db.rollback() # another request moved the job to another asset meanwhile: lock that one
# --- profiles ------------------------------------------------------------------------------
@@ -604,7 +632,7 @@ def update_download_meta(
def pause_download(
job_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
job = _own_job(db, user, job_id)
job = _own_job_for_update(db, user, job_id)
service.pause(db, job)
return _serialize_job(db, job)
@@ -613,7 +641,7 @@ def pause_download(
def resume_download(
job_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
job = _own_job(db, user, job_id)
job = _own_job_for_update(db, user, job_id)
service.resume(db, job)
return _serialize_job(db, job)
@@ -628,12 +656,13 @@ def redownload_download(
resolves to the newly-downloaded file — a re-download never breaks a link you already sent out
(only a *failed* re-download leaves the job in `error`, which is the intended broken-link case).
The rebuild always re-fetches into a FRESH, private asset (the cache sig is salted with the job
id + current asset id) rather than the dedup pool. That means it (a) always genuinely rebuilds
The rebuild always re-fetches into a FRESH, private asset (the cache sig is salted with the new
asset's own id, and the user's cookie salt when one applies — `credentials.rebuild_salt`) rather
than the dedup pool. That means it (a) always genuinely rebuilds
under the current format/naming rules — never silently reuses a shared cached file — and (b)
never deletes a file another job/user still shares: the old asset is only removed if THIS job
was its last holder."""
job = _own_job(db, user, job_id)
job = _own_job_for_update(db, user, job_id)
if job.job_kind in ("edit", "upload"):
# Neither has a re-fetchable source: an editor clip is derived locally, an upload IS the
# source. Re-running yt-dlp against source_kind='upload'/'edit' would just fail.
@@ -648,7 +677,7 @@ def redownload_download(
def cancel_download(
job_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
job = _own_job(db, user, job_id)
job = _own_job_for_update(db, user, job_id)
service.cancel(db, job)
return _serialize_job(db, job)
@@ -657,7 +686,7 @@ def cancel_download(
def delete_download(
job_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
job = _own_job(db, user, job_id)
job = _own_job_for_update(db, user, job_id)
service.delete(db, job)
return {"deleted": job_id}
@@ -801,7 +830,7 @@ def share_download(
# Uniform "no user" for a missing OR non-messageable (demo/suspended/deactivated) target, so
# sharing can't be used to probe which addresses exist or to reach a suspended account.
if not is_messageable_user(recipient):
raise HTTPException(status_code=404, detail="No user with that email.")
raise not_found("recipient")
if recipient.id == user.id:
raise HTTPException(status_code=400, detail="That's already your download.")
exists = db.execute(
@@ -1091,8 +1120,7 @@ def admin_set_quota(
admin: User = Depends(admin_user),
db: Session = Depends(get_db),
) -> dict:
if db.get(User, user_id) is None:
raise HTTPException(status_code=404, detail="Unknown user")
get_or_404(db, User, user_id, what="user")
cur = quota.resolve(db, user_id)
row = db.get(DownloadQuota, user_id)
if row is None:
+12 -7
View File
@@ -1,7 +1,7 @@
from datetime import date, datetime, timezone
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
from pydantic import BaseModel, ConfigDict, Field
from sqlalchemy import and_, func, or_, select
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.orm import Session
@@ -22,12 +22,14 @@ from app.feedquery import (
sort_keys,
)
from app.models import (
POSITION_MAX_S,
ChannelTag,
Tag,
User,
Video,
VideoState,
)
from app.routes._common import get_or_404
from app.sync.videos import parse_iso8601_duration
from app.utils import iso
from app.youtube.client import YouTubeClient, YouTubeError
@@ -234,8 +236,7 @@ def set_video_state(
status = body.status
if status not in VALID_STATES:
raise HTTPException(status_code=400, detail=f"status must be one of {VALID_STATES}")
if db.get(Video, video_id) is None:
raise HTTPException(status_code=404, detail="Unknown video")
get_or_404(db, Video, video_id, what="video")
now = datetime.now(timezone.utc)
@@ -303,11 +304,16 @@ def clear_video_state(
class VideoProgressIn(BaseModel):
# Finite only: JSON parsing here accepts `Infinity`/`NaN`, and the handler's `int()` turned
# one into a 500.
model_config = ConfigDict(allow_inf_nan=False)
# Accept a float and truncate (below) — the player may post a raw currentTime (a float), which
# the old `int(payload.get(...))` silently truncated. A plain `int` field would 422 that on the
# 10s progress hot path, losing the resume position; `float` keeps the old tolerance.
position_seconds: float = 0
duration_seconds: float = 0
# Bounded above by the int4 column (see POSITION_MAX_S): past it the flush was a 500.
position_seconds: float = Field(0, le=POSITION_MAX_S)
duration_seconds: float = Field(0, le=POSITION_MAX_S)
@router.post("/videos/{video_id}/progress")
@@ -326,8 +332,7 @@ def set_video_progress(
if position < 0:
position = 0
if db.get(Video, video_id) is None:
raise HTTPException(status_code=404, detail="Unknown video")
get_or_404(db, Video, video_id, what="video")
# Decide whether this position is worth keeping (else clear it).
near_end = duration > 0 and position > duration - FINISH_MARGIN_SECONDS
+6 -1
View File
@@ -159,8 +159,13 @@ def delete_my_account(
which all cascade on the users row. Quota-audit events are kept but anonymised (FK SET NULL).
The shared demo account can't be deleted, and the last remaining admin can't delete itself
(that would lock everyone out of the admin surfaces)."""
# Structured, like the other three demo refusals: `req` leaves 403 to the caller and
# `notifyActionError` discards a plain-string detail in favour of the caller's translated
# fallback, so a hand-written English sentence here reaches a Hungarian UI as a generic
# "couldn't do that" — or, on this affordance, as nothing at all. `reason` distinguishes it
# from the other sites' bare `{code: "demo"}` so the UI can say what specifically is refused.
if user.is_demo:
raise HTTPException(status_code=403, detail="The shared demo account can't be deleted.")
raise HTTPException(status_code=403, detail={"code": "demo", "reason": "delete_account"})
if user.role == "admin" and count_admins(db) <= 1:
raise HTTPException(
status_code=400,
+1 -1
View File
@@ -46,7 +46,7 @@ WELCOME = {
"This is your private message inbox. Conversations with other members are end-to-end "
"encrypted — only you and the person you're chatting with can read them, not even an "
"admin. To start messaging someone, set up secure messaging with a passphrase first.\n\n"
"Enjoy your feed!"
"Enjoy your Feed!"
),
"hu": (
"Üdv a Siftlode-on! 👋\n\n"
+27 -33
View File
@@ -23,6 +23,7 @@ from app.sync.playlists import (
sync_user_playlists,
)
from app.ratelimit import RateLimiter
from app.routes._common import get_or_404, owned_or_404
from app.youtube.client import YouTubeClient, YouTubeError
router = APIRouter(prefix="/api/playlists", tags=["playlists"])
@@ -48,10 +49,13 @@ def _check_pull_cooldown(user: User) -> None:
def _own_playlist(db: Session, user: User, playlist_id: int) -> Playlist:
pl = db.get(Playlist, playlist_id)
if pl is None or pl.user_id != user.id:
raise HTTPException(status_code=404, detail="Unknown playlist")
return pl
return owned_or_404(db, Playlist, playlist_id, user, what="playlist")
def _refused(status: int, reason: str) -> HTTPException:
"""A playlist action the server refuses, as a reason the UI translates
(`playlists.errors.<reason>`) — a bare English detail reached the Hungarian UI verbatim."""
return HTTPException(status_code=status, detail={"code": "playlist", "reason": reason})
def _touch(pl: Playlist) -> None:
@@ -256,7 +260,7 @@ def create_playlist(
) -> dict:
name = body.name.strip()
if not name:
raise HTTPException(status_code=400, detail="name is required")
raise _refused(400, "name_required")
pl = Playlist(user_id=user.id, name=name, position=_next_playlist_position(db, user.id))
db.add(pl)
db.commit()
@@ -288,8 +292,7 @@ def add_watch_later(
) -> dict:
"""Bookmark shortcut: add a video to the built-in Watch later playlist."""
video_id = body.video_id
if db.get(Video, video_id) is None:
raise HTTPException(status_code=404, detail="Unknown video")
get_or_404(db, Video, video_id, what="video")
pl = _watch_later(db, user)
_add_item(db, pl, video_id, mark_dirty=False)
return {"saved": True, "playlist_id": pl.id}
@@ -317,10 +320,7 @@ def sync_youtube(
"""Pull the user's YouTube playlists into local mirrors now (read-only direction).
Also runs automatically on the scheduler; this is the manual 'sync now'."""
if not has_read_scope(user):
raise HTTPException(
status_code=403,
detail="Connect YouTube (read access) to sync your playlists.",
)
raise _refused(403, "youtube_read_scope")
_check_pull_cooldown(user)
with quota.attribute(user.id, quota.QuotaAction.PLAYLISTS_PULL):
return sync_user_playlists(db, user)
@@ -337,18 +337,16 @@ def revert_youtube(
once the in-session undo history is gone."""
pl = _own_playlist(db, user, playlist_id)
if not pl.yt_playlist_id:
raise HTTPException(status_code=400, detail="This playlist isn't linked to YouTube.")
raise _refused(400, "not_on_youtube")
if not has_read_scope(user):
raise HTTPException(
status_code=403, detail="Connect YouTube (read access) to sync your playlists."
)
raise _refused(403, "youtube_read_scope")
_check_pull_cooldown(user)
try:
with quota.attribute(user.id, quota.QuotaAction.PLAYLISTS_PULL):
return repull_playlist(db, user, pl)
except YouTubeError:
db.rollback()
raise HTTPException(status_code=422, detail="Couldn't refresh from YouTube.")
raise _refused(422, "youtube_failed")
def _pushable(db: Session, user: User, playlist_id: int) -> Playlist:
@@ -356,9 +354,7 @@ def _pushable(db: Session, user: User, playlist_id: int) -> Playlist:
Watch later and read-only YouTube mirrors are never pushable."""
pl = _own_playlist(db, user, playlist_id)
if pl.kind == "watch_later":
raise HTTPException(
status_code=400, detail="The Watch later list can't be sent to YouTube."
)
raise _refused(400, "watch_later_not_pushable")
return pl
@@ -376,7 +372,7 @@ def push_plan(
with quota.attribute(user.id, quota.QuotaAction.PLAYLISTS_PUSH):
plan = plan_merge(db, user, pl)
except YouTubeError:
raise HTTPException(status_code=422, detail="Couldn't read the playlist on YouTube.")
raise _refused(422, "youtube_failed")
plan["remaining_today"] = quota.remaining_today(db)
plan["affordable"] = plan["units_estimate"] <= plan["remaining_today"]
return plan
@@ -406,14 +402,11 @@ def push(
)
plan = plan_merge(db, user, pl, live=live)
if plan["units_estimate"] > quota.remaining_today(db):
raise HTTPException(
status_code=429,
detail="Not enough YouTube quota left today for this sync.",
)
raise _refused(429, "quota_exceeded")
return merge_playlist(db, user, pl, live=live)
except YouTubeError:
db.rollback()
raise HTTPException(status_code=422, detail="YouTube playlist sync failed.")
raise _refused(422, "youtube_failed")
@router.get("/{playlist_id}")
@@ -458,7 +451,7 @@ def rename_playlist(
if "name" in body.model_fields_set:
name = (body.name or "").strip()
if not name:
raise HTTPException(status_code=400, detail="name cannot be empty")
raise _refused(400, "name_required")
if name != pl.name:
pl.name = name
recompute_dirty(db, pl)
@@ -493,18 +486,18 @@ def delete_playlist(
) -> dict:
pl = _own_playlist(db, user, playlist_id)
if pl.kind == "watch_later":
raise HTTPException(status_code=400, detail="The Watch later list can't be deleted.")
raise _refused(400, "watch_later_undeletable")
if on_youtube:
if not pl.yt_playlist_id:
raise HTTPException(status_code=400, detail="This playlist isn't on YouTube.")
raise _refused(400, "not_on_youtube")
if not has_write_scope(user):
raise HTTPException(status_code=403, detail="Enable YouTube editing in Settings first.")
raise _refused(403, "youtube_editing_off")
try:
with quota.attribute(user.id, quota.QuotaAction.PLAYLISTS_DELETE):
with YouTubeClient(db, user) as yt:
yt.delete_playlist(pl.yt_playlist_id)
except YouTubeError:
raise HTTPException(status_code=422, detail="YouTube delete failed.")
raise _refused(422, "youtube_failed")
db.delete(pl) # items cascade
db.commit()
return {"deleted": playlist_id}
@@ -519,8 +512,7 @@ def add_item(
) -> dict:
pl = _own_playlist(db, user, playlist_id)
video_id = body.video_id
if db.get(Video, video_id) is None:
raise HTTPException(status_code=404, detail="Unknown video")
get_or_404(db, Video, video_id, what="video")
_add_item(db, pl, video_id, mark_dirty=True)
return {"playlist_id": pl.id, "video_id": video_id}
@@ -557,7 +549,9 @@ def move_item(
case. Idempotent: if the video is already in the target, the source row is simply dropped
(moving into a list that already has it is a removal, not a duplicate)."""
src = _own_playlist(db, user, playlist_id)
dst = _own_playlist(db, user, body.target_playlist_id)
# Its own reason: "this playlist doesn't exist any more" about a TARGET that is gone was read as
# the list on screen — which is still there.
dst = owned_or_404(db, Playlist, body.target_playlist_id, user, what="move_target")
if src.id == dst.id:
raise HTTPException(
status_code=400, detail={"code": "playlist_move", "reason": "same_playlist"}
+1 -23
View File
@@ -3,13 +3,9 @@
import logging
from fastapi import HTTPException
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.models import (
PlexItem,
PlexLibrary,
)
from app.models import PlexItem
from app.plex.lookups import by_rating_key
log = logging.getLogger("siftlode.plex")
@@ -25,21 +21,3 @@ def item_or_404(db: Session, rating_key: str) -> PlexItem:
def _norm_lang(lang: str | None) -> str:
"""The overlay has an EN base + an optional HU twin; anything other than `hu` serves EN."""
return "hu" if lang == "hu" else "en"
def _content_libs(db: Session) -> tuple[list[int], list[int]]:
"""The BROWSABLE content libraries, partitioned into (movie_lib_ids, show_lib_ids) — the single
source of the browse scope. Excludes HU overlay libraries (`overlay_of` set): those are never
browsed on their own, they only lend their Hungarian metadata to the EN twin's rows (E5 S5)."""
libs = (
db.execute(
select(PlexLibrary).where(
PlexLibrary.enabled.is_(True), PlexLibrary.overlay_of.is_(None)
)
)
.scalars()
.all()
)
movie_lib_ids = [lb.id for lb in libs if lb.kind == "movie"]
show_lib_ids = [lb.id for lb in libs if lb.kind == "show"]
return movie_lib_ids, show_lib_ids
+21 -11
View File
@@ -34,6 +34,7 @@ from app.plex.query import (
apply_meta_filters,
apply_movie_state_filter,
apply_show_state_filter,
content_libs,
field_recheck,
genres_col,
meta_filter_conds,
@@ -47,12 +48,28 @@ from app.plex.query import (
vector_col,
vector_match,
)
from app.routes.plex._shared import _content_libs, _norm_lang
from app.routes.plex._shared import _norm_lang
from app.utils import to_tsquery_str
router = APIRouter(prefix="/api/plex", tags=["plex"])
def sort_column(u, sort: str):
"""The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline
in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control
offers but this cannot order would quietly fall back to `added_at` and look like a broken control.
An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback."""
return {
"added": u.c.added_at,
"title": func.lower(u.c.title),
"year": u.c.year,
"rating": u.c.rating,
"duration": u.c.duration_s,
"release": u.c.rel,
"lastwatched": u.c.last_watched,
}.get(sort, u.c.added_at)
@router.get("/libraries")
def list_libraries(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
"""The mirrored libraries the user can browse (drives the Plex scope selector). HU overlay
@@ -109,7 +126,7 @@ def unified_library(
offset = max(0, offset)
lang = _norm_lang(lang)
p = f.as_dict()
movie_lib_ids, show_lib_ids = _content_libs(db)
movie_lib_ids, show_lib_ids = content_libs(db)
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
want_shows = scope in ("show", "both") and bool(show_lib_ids)
@@ -220,14 +237,7 @@ def unified_library(
if tsq is not None:
order = [u.c.rank.desc()]
else:
col = {
"title": func.lower(u.c.title),
"year": u.c.year,
"rating": u.c.rating,
"duration": u.c.duration_s,
"release": u.c.rel,
"lastwatched": u.c.last_watched,
}.get(sort, u.c.added_at)
col = sort_column(u, sort)
order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()]
order.append(u.c.rk.desc())
rows = db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all()
@@ -328,7 +338,7 @@ def facets(
"duration_min": None,
"duration_max": None,
}
all_movie_ids, all_show_ids = _content_libs(db)
all_movie_ids, all_show_ids = content_libs(db)
movie_ids = all_movie_ids if scope in ("movie", "both") else []
show_ids = all_show_ids if scope in ("show", "both") else []
if not movie_ids and not show_ids:
+136 -17
View File
@@ -1,5 +1,9 @@
"""Plex collections: browse mirrored collections + the admin manual-collection editor."""
from collections.abc import Callable
from datetime import date, datetime
from typing import Any, NamedTuple
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from sqlalchemy import func, select
@@ -12,6 +16,7 @@ from app.models import (
PlexItem,
PlexLibrary,
PlexShow,
PlexState,
User,
)
from app.plex import rollup
@@ -19,8 +24,7 @@ from app.plex import sync as plex_sync
from app.plex.cards import movie_card, show_card
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
from app.plex.lookups import by_rating_key
from app.plex.query import like_escape, runtime_by_key, top_art_keys
from app.routes.plex._shared import _content_libs
from app.plex.query import content_libs, like_escape, runtime_by_key, top_art_keys
# The most-restrictive → least ordering used to pick a collection's effective content rating from its
# members (Plex shows one cert on the collection; we don't store it, so derive it).
@@ -84,13 +88,20 @@ def _collection_card(c: PlexCollection) -> dict:
def collections(
library: str | None = None,
q: str | None = None,
keys: str | None = None,
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
"""Mirrored collections as cards (most-populated first), optionally name-filtered by `q`. With a
`library` plex_key → that one library's collections (used by the collection editor). WITHOUT one →
the UNION across all enabled libraries (the unified-scope sidebar picker, which has no single
library selected)."""
library selected).
`keys` (CSV of rating_keys) narrows to just those. It exists because a Plex link carries the
collection FILTER as bare rating_keys — nothing in the URL says what "17045" is called — so the
sidebar token and the applied-filter chip had nothing to render but the raw id. Resolving them
through this endpoint costs one small row each; pulling the whole list to find two titles is
~200 KB."""
if library is not None:
lib = (
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == str(library)))
@@ -114,6 +125,13 @@ def collections(
term = (q or "").strip()
if term:
query = query.where(PlexCollection.title.ilike(f"%{like_escape(term)}%"))
if keys is not None:
wanted = [k for k in (keys or "").split(",") if k]
# An explicit but empty `keys=` asks for nothing — answer nothing, rather than falling through
# to "every collection" (which is what an unguarded empty IN () would have to become).
if not wanted:
return {"collections": []}
query = query.where(PlexCollection.rating_key.in_(wanted))
rows = (
db.execute(
query.order_by(
@@ -128,7 +146,7 @@ def collections(
# members (a collection lives in one library, so exactly one side has art) — a show collection would
# otherwise render blank; the collection's own poster is the next fallback, then the colour hash.
# Runtime is movie-only (shows have no single duration).
movie_ids, show_ids = _content_libs(db)
movie_ids, show_ids = content_libs(db)
keys = [c.rating_key for c in rows]
art_m = top_art_keys(db, PlexItem, movie_ids, PlexItem.collection_keys, keys, per=2)
art_s = top_art_keys(db, PlexShow, show_ids, PlexShow.collection_keys, keys, per=2)
@@ -150,14 +168,69 @@ def collections(
}
class Member(NamedTuple):
"""One collection member flattened to everything the Sort control can order by. The page's two
legs are different tables (movies and shows), so there is no single query to ORDER BY — the
ordering is a pure list sort over these, which is also what makes it unit-testable."""
kind: str # "movie" | "show"
obj: Any # PlexItem | PlexShow, carried through to the card serializer
title: str # lower-cased; never NULL (the column is not nullable)
year: int | None
rating: float | None
added: datetime | None
release: date | None
duration: int | None # movies only — a show has no single runtime
last_watched: datetime | None
# Sort key -> what to read off a Member. Pinned to `PLEX_SORT_KEYS` by a test: the collection page
# shows the SAME Sort control as the browse grid, so a key this map is missing would be an option
# that silently does nothing here.
MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = {
"added": lambda m: m.added,
"lastwatched": lambda m: m.last_watched,
"release": lambda m: m.release,
"year": lambda m: m.year,
"rating": lambda m: m.rating,
"duration": lambda m: m.duration,
"title": lambda m: m.title,
}
# What the page ordered by before it had a control, and still its default: a film series wants to be
# read in release order, not by whenever Plex happened to ingest it.
DEFAULT_MEMBER_SORT = "year"
def sort_members(members: list[Member], sort: str, sort_dir: str) -> list[Member]:
"""Order a collection's members by one of `PLEX_SORT_KEYS`.
Two properties worth stating because they are easy to get wrong with a plain `reverse=True`:
members with no value for the key sink to the END in BOTH directions (matching the browse grid's
`nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve
title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside
it. An unknown key falls back to the page's default instead of returning an arbitrary order."""
key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT])
ordered = sorted(members, key=lambda m: m.title) # stable secondary, applied first
have = [m for m in ordered if key(m) is not None]
missing = [m for m in ordered if key(m) is None]
have.sort(key=key, reverse=sort_dir != "asc")
return have + missing
@router.get("/collections/{rating_key}")
def collection_detail(
rating_key: str,
sort: str = DEFAULT_MEMBER_SORT,
sort_dir: str = "asc",
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
"""One collection's hero data + its members (movies + shows), ordered by release year then title —
the Collection info page. Read-only here; edits go through the admin mutation endpoints below."""
"""One collection's hero data + its members (movies + shows) in the requested order — the
Collection info page. The sort vocabulary is the browse grid's (`PLEX_SORT_KEYS`) so the same Sort
control means the same thing on both surfaces; the default is release year ascending, which is
what this page did before it had a control. Read-only here; edits go through the admin mutation
endpoints below."""
col = by_rating_key(db, PlexCollection, rating_key)
if col is None:
raise HTTPException(status_code=404, detail="Unknown collection")
@@ -180,19 +253,63 @@ def collection_detail(
.all()
)
mstate = rollup.item_state_map(db, user.id, [m.id for m in movies])
sstate = rollup.state_map(db, user.id, [s.id for s in shows])
# Merge the two type-homogeneous lists into ONE member list ordered by (year, title).
tagged = [(m.year, (m.title or "").lower(), "movie", m) for m in movies] + [
(s.year, (s.title or "").lower(), "show", s) for s in shows
# ONE rollup pass over the show members, for both the aggregate badge status and `resumed_at` —
# the browse grid's "last watched" for a show (the newest resume/watched across its episodes). The
# cheaper `rollup.state_map` gives only the status, which would leave `lastwatched` unorderable on
# a show collection while the control still offered it.
show_rows = (
db.execute(
rollup.agg_subquery(db, user.id, include_resumed=True).where(
PlexItem.show_id.in_([s.id for s in shows])
)
).all()
if shows
else []
)
sstate = {r.show_id: rollup.show_status(r.total, r.watched, r.inprog) for r in show_rows}
sresumed = {r.show_id: r.resumed_at for r in show_rows}
def _movie_last_watched(st: PlexState | None) -> datetime | None:
# Same definition as the browse grid's: when it was last resumed, else when it was marked
# watched (coalesce(progress_updated_at, watched_at)).
return (st.progress_updated_at or st.watched_at) if st else None
# Merge the two type-homogeneous lists into ONE member list, then order it.
members = [
Member(
kind="movie",
obj=m,
title=(m.title or "").lower(),
year=m.year,
rating=m.rating,
added=m.added_at,
release=m.originally_available_at,
duration=m.duration_s,
last_watched=_movie_last_watched(mstate.get(m.id)),
)
for m in movies
] + [
Member(
kind="show",
obj=sh,
title=(sh.title or "").lower(),
year=sh.year,
rating=sh.rating,
added=sh.added_at,
release=sh.originally_available_at,
duration=None,
last_watched=sresumed.get(sh.id),
)
for sh in shows
]
tagged.sort(key=lambda t: (t[0] is None, t[0] or 0, t[1]))
members = sort_members(members, sort, sort_dir)
items = [
movie_card(o, mstate.get(o.id))
if kind == "movie"
else show_card(o, sstate.get(o.id, "new"))
for _, _, kind, o in tagged
movie_card(m.obj, mstate.get(m.obj.id))
if m.kind == "movie"
else show_card(m.obj, sstate.get(m.obj.id, "new"))
for m in members
]
years = [y for y, _, _, _ in tagged if y is not None]
years = [m.year for m in members if m.year is not None]
certs = [m.content_rating for m in movies] + [s.content_rating for s in shows]
return {
**_collection_card(col),
@@ -230,7 +347,9 @@ def _reject_duplicate_name(
.first()
)
if dup is not None and dup.rating_key != exclude_rk:
raise HTTPException(status_code=409, detail="A collection with this name already exists")
# Structured, so the bilingual UI says it in the reader's language (it was toasted in
# Hungarian beside an English error dialog saying the same thing).
raise HTTPException(status_code=409, detail={"code": "collection", "reason": "name_taken"})
def _write_or_pending_error(wrote: bool, e: Exception) -> HTTPException:
+9 -7
View File
@@ -27,12 +27,13 @@ from app.plex.query import (
LibraryFilters,
apply_meta_filters,
apply_show_state_filter,
content_libs,
genres_col,
meta_filter_conds,
top_art_keys,
)
from app.plex.tmdb import resolve_recs
from app.routes.plex._shared import _content_libs, _norm_lang
from app.routes.plex._shared import _norm_lang
router = APIRouter(prefix="/api/plex", tags=["plex"])
@@ -43,7 +44,7 @@ SURPRISE_PER_KIND = 3 # Surprise-me: at most this many movies + this many shows
def _scope_libs(db: Session, scope: str) -> tuple[list[int], list[int]]:
"""The movie/show library ids visible for this scope (empty list = out of scope)."""
movie_ids, show_ids = _content_libs(db)
movie_ids, show_ids = content_libs(db)
return (
movie_ids if scope in ("movie", "both") else [],
show_ids if scope in ("show", "both") else [],
@@ -112,10 +113,11 @@ def _recently_added(
else []
)
# Merge the two type-homogeneous lists by added_at, then serialize each side in one batch so the
# per-card state lookups stay batched (no N+1). The `(is not None, value)` key sorts a missing
# timestamp last under reverse.
# per-card state lookups stay batched (no N+1). Same `or NO_TS` key as the two rails below — the
# E24 review found this one still on the older `(is not None, value)` form, i.e. one rule in two
# spellings, with the two comments explaining it differently.
tagged = [(m.added_at, "movie", m) for m in movies] + [(s.added_at, "show", s) for s in shows]
tagged.sort(key=lambda t: (t[0] is not None, t[0]), reverse=True)
tagged.sort(key=lambda t: t[0] or rollup.NO_TS, reverse=True)
top = tagged[:RAIL_LIMIT]
# Key the serialized cards back by rating_key (globally unique in Plex — a movie and a show never
# share one), so the merge doesn't depend on object identity staying alive.
@@ -170,7 +172,7 @@ def _recently_watched(
)
for ep, st, sh_title, sh_rk in db.execute(eq).all():
rows.append((st.watched_at, leaf_card(ep, st, show_title=sh_title, show_rk=sh_rk)))
rows.sort(key=lambda r: (r[0] is not None, r[0]), reverse=True)
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
return [card for _, card in rows[:RAIL_LIMIT]]
@@ -234,7 +236,7 @@ def _continue_watching(
for sh, resumed_at in srows:
rows.append((resumed_at, show_card(sh, "in_progress")))
rows.sort(key=lambda r: (r[0] is not None, r[0]), reverse=True)
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
return [card for _, card in rows[:RAIL_LIMIT]]
+3 -4
View File
@@ -19,8 +19,7 @@ from app.models import (
from app.plex import rollup
from app.plex import tmdb as plex_tmdb
from app.plex.cards import movie_card, show_card
from app.plex.query import split_csv
from app.routes.plex._shared import _content_libs
from app.plex.query import content_libs, split_csv
router = APIRouter(prefix="/api/plex", tags=["plex"])
log = logging.getLogger("siftlode.plex")
@@ -71,7 +70,7 @@ def people_cards(
wanted = split_csv(names)[:16]
if not wanted:
return {"people": []}
movie_lib_ids, show_lib_ids = _content_libs(db)
movie_lib_ids, show_lib_ids = content_libs(db)
want_movies = scope in ("movie", "both") and bool(movie_lib_ids)
want_shows = scope in ("show", "both") and bool(show_lib_ids)
photos = {
@@ -185,7 +184,7 @@ def person_detail(
and _should_schedule_enrich(person.id)
):
background_tasks.add_task(_enrich_person_bg, person.id)
movie_lib_ids, show_lib_ids = _content_libs(db)
movie_lib_ids, show_lib_ids = content_libs(db)
movies: list[dict] = []
if movie_lib_ids:
+154 -83
View File
@@ -1,8 +1,6 @@
"""Plex playback: image/subtitle proxies, HLS streaming, and watch-state writes."""
import hashlib
import os
import tempfile
import threading
from datetime import datetime, timedelta, timezone
@@ -12,17 +10,20 @@ from fastapi import (
BackgroundTasks,
Depends,
HTTPException,
Path,
Query,
Request,
Response,
)
from fastapi.responses import FileResponse
from pydantic import BaseModel
from pydantic import BaseModel, ConfigDict, Field
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.auth import current_user, resolved_user_id
from app.db import SessionLocal, get_db
from app.models import (
POSITION_MAX_S,
PlexItem,
PlexSeason,
PlexShow,
@@ -89,62 +90,70 @@ def image(rating_key: str, request: Request, variant: str = "thumb") -> Response
return imgcache.response(data, ct)
# Both subtitle routes authenticate with the full `current_user`, not the cookie-only
# `_require_session` the image proxies use. That lighter gate exists for bursts of low-sensitivity
# poster fetches and skips the is_active / is_suspended / session-epoch checks — acceptable for a
# poster, not for a route that starts reads of the media library: a suspended or "log out
# everywhere"-revoked cookie kept scheduling them. They are not bursty (one poll a second, one fetch
# per track), so the DB user-load costs nothing that matters.
#
# Both path parameters are bounded, the same way on both routes: they become a cache FILE name
# (`sub_<rating_key>_<ord>`), and an absurd one — a 250-digit key or ordinal — outgrew the OS's name
# limit and turned the status poll into a 500. A Plex rating_key is a short number; the bound only
# has to be generous.
_RatingKey = Path(max_length=32)
_SubOrd = Path(ge=0, le=9999)
@router.get("/subtitle/{rating_key}/{sub_ord}/status")
def subtitle_status(
rating_key: str = _RatingKey,
sub_ord: int = _SubOrd,
user: User = Depends(current_user),
# Bounded both ways: a non-finite position used to reach the scheduler's arithmetic and 500, and
# an absurd one reached ffmpeg's `-ss`, which refused it.
at: float = Query(0.0, ge=0.0, le=subtitles.MAX_POSITION_S, allow_inf_nan=False),
) -> dict:
"""Is this subtitle track showable yet — and if not, get it moving. Extraction runs in the
background (see plex/subtitles.py for the measurements that forced it there), so the player polls
this while it waits instead of hanging on a <track> that will not load for a minute or more.
`at` is the viewer's absolute position: the extractor windows around it, so what gets read first
is the part actually about to be watched."""
return subtitles.ensure(rating_key, sub_ord, at)
@router.get("/subtitle/{rating_key}/{sub_ord}")
def subtitle(rating_key: str, sub_ord: int, request: Request, offset: float = 0.0) -> Response:
def subtitle(
rating_key: str = _RatingKey,
sub_ord: int = _SubOrd,
user: User = Depends(current_user),
# Finite, like every float this router takes: `-inf` crashed the shift (a 500), `nan` stacked
# every cue at 0. It can legitimately be negative (a subtitle nudged later than the session start).
offset: float = Query(0.0, allow_inf_nan=False),
) -> Response:
"""Serve the ord-th subtitle track as WebVTT for the browser's <track> — so choosing a subtitle
overlays it directly (no HLS re-mux / session restart). Handles EXTERNAL sidecar subs (fetched
from Plex via the stream `key` — the old `-map 0:s` broke on these since they're not in the file)
AND embedded text subs (extracted from the local file). Image subs (PGS/VobSub) → 415. Disk-cached."""
_require_session(request)
# Cache the ABSOLUTE-time VTT once (per rk+ord); the per-session cue-shift is applied on the way
# out, so a resume/seek to any offset reuses the same cached track (no cache-per-offset explosion).
cache_key = f"sub_{rating_key}_{sub_ord}"
hit = subtitles.cache_read(cache_key)
if hit is not None:
return Response(subtitles.shift_vtt(hit, offset), media_type="text/vtt")
try:
with SessionLocal() as db:
it = item_or_404(db, rating_key)
with PlexClient(db) as plex:
part = ((plex.metadata(it.rating_key) or {}).get("Media") or [{}])[0].get(
"Part"
) or [{}]
subs = [s for s in (part[0].get("Stream") or []) if s.get("streamType") == 3]
if sub_ord < 0 or sub_ord >= len(subs):
raise HTTPException(status_code=404, detail="No such subtitle")
s = subs[sub_ord]
codec = (s.get("codec") or s.get("format") or "").lower()
if codec in subtitles.BITMAP_SUBS:
raise HTTPException(
status_code=415,
detail="Image-based subtitle can't be shown as text",
)
if s.get("key"): # external sidecar → fetch from Plex, convert to VTT
raw = plex.raw_get(s["key"])
if codec in ("srt", "subrip", ""):
vtt = subtitles.srt_to_vtt(raw.decode("utf-8", "replace"))
else:
fd, tmp = tempfile.mkstemp(suffix="." + (codec or "sub"))
try:
os.write(fd, raw)
os.close(fd)
vtt = subtitles.ffmpeg_to_vtt(tmp, None)
finally:
os.unlink(tmp)
elif s.get("index") is not None: # embedded → extract from the local file
src = plex_paths.local_media_path(db, it.file_path)
if src is None:
raise HTTPException(status_code=404, detail="Media file not found on disk")
emb_ord = sum(1 for x in subs[:sub_ord] if x.get("index") is not None)
vtt = subtitles.ffmpeg_to_vtt(str(src), emb_ord)
else:
raise HTTPException(status_code=404, detail="Subtitle not fetchable")
except PlexNotConfigured as e:
raise HTTPException(status_code=400, detail=str(e))
except PlexError as e:
raise HTTPException(status_code=502, detail=f"Subtitle fetch failed: {e}")
subtitles.cache_write(cache_key, vtt)
return Response(subtitles.shift_vtt(vtt, offset), media_type="text/vtt")
AND embedded text subs (extracted from the local file). Image subs (PGS/VobSub) → 415. Disk-cached.
A pure READ: it serves whatever is cached and schedules nothing — the status poll owns that,
because only it knows where the viewer is (see `subtitles.cached_track`). The player mounts the
<track> only once /status says there is something to show, so in practice this is only ever hit
warm; on a cache eviction race the 202 sends the player back to polling."""
# The cached VTT holds ABSOLUTE cue times (per rk+ord); the per-session cue-shift is applied on
# the way out, so a resume/seek to any offset reuses the same track (no cache-per-offset explosion).
hit, code = subtitles.cached_track(rating_key, sub_ord)
if hit is None and code:
raise HTTPException(
status_code=subtitles.UNAVAILABLE_STATUS.get(code, 422),
detail={"code": "subtitle", "reason": code},
)
if hit is None:
return Response(
status_code=202, media_type="application/json", content='{"state":"preparing"}'
)
return Response(subtitles.shift_vtt(hit, offset), media_type="text/vtt")
# Resume/watched thresholds. _PROGRESS_MIN_S matches the feed's PROGRESS_MIN_SECONDS (5), but
@@ -287,10 +296,15 @@ def _push_watch(
class ItemProgressIn(BaseModel):
# Finite only: JSON parsing here accepts `Infinity`/`NaN`, and the handler's `int()` turned
# one into a 500.
model_config = ConfigDict(allow_inf_nan=False)
# float, truncated in the handler — the player may post a raw currentTime (a float); a plain
# `int` field would 422 it on the periodic progress checkpoint (the old `int(...)` truncated).
position_seconds: float = 0
duration_seconds: float = 0
# Bounded above by the int4 column (see POSITION_MAX_S): past it the flush was a 500.
position_seconds: float = Field(0, le=POSITION_MAX_S)
duration_seconds: float = Field(0, le=POSITION_MAX_S)
final: bool = False
@@ -309,6 +323,14 @@ def item_progress(
`_should_push_resume`). `final` (sent on pause / seek / pagehide / unmount) pushes immediately and
resets the throttle window, so Plex sees a settled position the instant the user pauses or seeks."""
it = item_or_404(db, rating_key)
# A player parked mid-item — playing or PAUSED — checkpoints every 10 s, so this is the signal
# that such a viewer still wants their HLS session: the one thing `last_access` could not see,
# since it moved only on a file fetch and a paused player fetches nothing (see
# stream.reap_idle). It is NOT every open player: `usePlexProgress.flush` sends nothing once the
# finish latch is up, or before the first position is known. Those are the states in which
# reaping is the right answer anyway, and a viewer who scrubs back into a reaped session gets
# the same 404 → reclaim the player already handles.
plex_stream.touch_session(str(rating_key))
position = max(0, int(body.position_seconds))
duration = int(body.duration_seconds)
final = body.final
@@ -406,10 +428,20 @@ def item_state(
)
prev_status = st.status if st is not None else "new"
if status == "new":
# Does Plex hold anything for this item that the un-mark has to clear? Two things qualify: a
# view count, and a resume point. An IN-PROGRESS item carries status "new" WITH a position —
# that is how the importer maps a bare `viewOffset` (`_plex_watch_to_state`) — so keying the
# push on the status alone silently skipped the single case "Clear resume point" exists for:
# the un-mark never reached Plex, the offset stayed, and the next pull imported it straight
# back (Protector, cleared twice by the user and back at 92s both times). `hidden` is
# Siftlode-only and still pushes nothing, exactly as this endpoint's docstring says.
mirrored_on_plex = st is not None and (
st.status == "watched" or (st.position_seconds or 0) > 0
)
if st is not None:
db.delete(st)
db.commit()
if prev_status != "new": # something → unwatched: mirror the un-mark to Plex
if mirrored_on_plex:
_push_watch(background, db, user.id, it.id, rating_key, "unwatched")
return {"status": "new"}
if st is None:
@@ -524,9 +556,10 @@ def season_state(
@router.post("/stream/{rating_key}/session")
def stream_session(
rating_key: str,
start: float = 0.0,
# Finite: `start=inf` reached `int()` in the session builder and came back as a 500.
start: float = Query(0.0, allow_inf_nan=False),
audio: int | None = None,
aoff: float = 0.0,
aoff: float = Query(0.0, allow_inf_nan=False),
multi: bool = False,
user: User = Depends(current_user),
db: Session = Depends(get_db),
@@ -536,7 +569,6 @@ def stream_session(
HEVC/VP9/AV1/…) take the same HLS path but with the video re-encoded to H.264 (VAAPI or libx264).
Selecting a non-default audio track (`audio` = stream ordinal) forces the HLS path so the chosen audio
can be muxed in. Subtitles are independent WebVTT tracks (GET /subtitle) — no session restart."""
plex_stream.reap_idle() # opportunistic cleanup of idle sessions on each new playback
it = item_or_404(db, rating_key)
transcode = it.playable == "transcode"
# `multi` (item has ≥2 audio tracks) forces the HLS path so ALL audio tracks ship as renditions and
@@ -560,7 +592,9 @@ def stream_session(
return {
"mode": "hls",
"start_s": s.media_start_s,
"url": f"/api/plex/stream/{it.rating_key}/hls/{s.entry}",
# The session id is part of the path (not a query param) so the playlist's RELATIVE segment
# names resolve back into the same session — see `stream_hls_file` for why it has to.
"url": f"/api/plex/stream/{it.rating_key}/hls/{s.id}/{s.entry}",
"transcode": s.transcode,
"hw": s.hw,
"tonemap": s.tonemap,
@@ -592,45 +626,82 @@ def stream_file(
return FileResponse(str(src)) # Starlette honours the Range header (206)
# What a session directory can contain, and nothing else. No `.vtt`: ffmpeg runs with `-sn` and
# subtitles are independent tracks served by GET /subtitle, so a WebVTT name here could only ever be
# a name that will not arrive — and accepting it cost a 20 s parked worker per request.
_HLS_CT = {
".m3u8": "application/vnd.apple.mpegurl",
".ts": "video/mp2t",
".vtt": "text/vtt",
}
# A real artifact name is ~20 characters. The bound is what keeps an absurd one out of the parser:
# `int()` on an unbounded digit run raises ValueError past 4300 digits (CPython's
# `sys.get_int_max_str_digits`), which reached the client as a 500 — the same shape of bug the
# subtitle routes were bounded for (`_RatingKey`, above).
_HLS_NAME_MAX = 128
@router.get("/stream/{rating_key}/hls/{filename}")
@router.get("/stream/{rating_key}/hls/{session_id}/{filename}")
def stream_hls_file(
rating_key: str,
session_id: str,
filename: str,
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> Response:
"""Serve a file (playlist / segment / subtitle) from the current HLS session directory. The
playlist references its segments/subtitles by relative name, all resolving back here."""
"""Serve a file (playlist / segment / subtitle) from ONE HLS session's directory. The playlist
references its segments/subtitles by relative name, all resolving back here.
The session id in the path is what makes "back here" mean the same session. Sessions are keyed by
rating_key alone and a seek-restart or a second viewer of the same item replaces the registered
one, while the DIRECTORY is per (key, offset, tag) — so resolving by rating_key handed the old
viewer the new session's `seg_12.ts`, produced from a different offset under `-copyts`: wrong
bytes with a 200, which the player has no way to notice (deep-review round 5). A request for a
session that is no longer the registered one is a 404, and the player reclaims at the playhead.
That reclaim is also why a request with no live session is no longer auto-started here: the
auto-start could only ever begin at 0, silently throwing away the viewer's position."""
# Filename must be a plain hls artifact — no path separators, only the expected extensions.
ext = "." + filename.rsplit(".", 1)[-1] if "." in filename else ""
if ext not in _HLS_CT or "/" in filename or "\\" in filename or ".." in filename:
if (
len(filename) > _HLS_NAME_MAX
or ext not in _HLS_CT
or "/" in filename
or "\\" in filename
or ".." in filename
):
raise HTTPException(status_code=404, detail="Not found")
s = plex_stream.current_session(str(rating_key))
if s is None:
# No session yet (e.g. reaped): the ENTRY playlist may auto-start one at 0 — master.m3u8 for a
# multi-audio item, index.m3u8 for a single-audio remux item.
it = item_or_404(db, rating_key)
if filename == "master.m3u8":
s = plex_stream.start_session(db, it, 0.0, multi=True)
elif filename == "index.m3u8" and it.playable == "remux":
s = plex_stream.start_session(db, it, 0.0)
else:
raise HTTPException(status_code=404, detail="No active playback session")
if s is None:
raise HTTPException(status_code=404, detail="Media file not found on disk")
if s is None or s.id != session_id:
raise HTTPException(status_code=404, detail="No active playback session")
path = s.dir / filename
timeout = 30 if ext == ".m3u8" else 20
if not plex_stream.wait_for(path, timeout):
# Only a name THIS session can actually produce is worth waiting for: its own playlists, and a
# segment of one of its own renditions whose predecessor is already on disk (ffmpeg writes them
# in order, so if it is not there this one is not next). The session is passed, not just its
# directory, because the single- and multi-audio argv branches spell their names differently and
# neither can produce the other's. Answering now instead of waiting it out is what keeps a seek
# past the produced region — or a deliberate flood of hopeless names — from parking threadpool
# workers and their DB connections (see `artifact_may_arrive`).
if not plex_stream.artifact_may_arrive(s, filename):
raise HTTPException(status_code=404, detail="Not available yet")
timeout = 30 if ext == ".m3u8" else 20
# `proc` so a session whose ffmpeg has already exited answers at once instead of polling out the
# whole timeout: this is a SYNC route on the ~40-slot threadpool, holding a pooled DB connection
# with it, so every such wait is a worker (and a connection) parked for 20 s — an ordinary seek
# past the produced region was enough to do it (round 5).
if not plex_stream.wait_for(path, timeout, proc=s.proc):
raise HTTPException(status_code=404, detail="Not available yet")
try:
content = path.read_bytes()
except OSError:
# The session was torn down between the wait and the read — `_kill` rmtrees the whole
# directory, and nothing holds `_lock` across those two steps. It has to present as the same
# 404 a gone session does: the player's recovery is keyed on the STATUS (`code === 404`), so
# a 500 here skipped the reclaim and ended playback with "couldn't play" instead of
# restarting at the playhead. Ordinary trigger: `loadSession` awaits the new session — which
# kills and rmtrees the old one — before destroying the old hls instance, so the outgoing
# instance's in-flight segment request races the rmtree on every seek-restart.
raise HTTPException(status_code=404, detail="No active playback session") from None
return Response(
content=path.read_bytes(),
content=content,
media_type=_HLS_CT[ext],
headers={"Cache-Control": "no-store"},
)
+1 -1
View File
@@ -25,7 +25,7 @@ router = APIRouter(prefix="/api/plex", tags=["plex"])
def _own_playlist_or_404(db: Session, pid: int, user: User) -> PlexPlaylist:
return owned_or_404(db, PlexPlaylist, pid, user, detail="Playlist not found")
return owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
def _playlist_card(pl: PlexPlaylist, count: int, thumb_rk: str | None) -> dict:
+1 -1
View File
@@ -91,7 +91,7 @@ def create_view(
def _own_view(db: Session, user: User, view_id: int) -> SavedView:
return owned_or_404(db, SavedView, view_id, user, detail="Unknown view")
return owned_or_404(db, SavedView, view_id, user, what="view")
class UpdateViewIn(BaseModel):
+3 -2
View File
@@ -23,6 +23,7 @@ from app.scheduler import (
trigger_job,
)
from app.sync.runner import estimate_deep_backfill
from app.routes._common import not_found
router = APIRouter(prefix="/api/admin/scheduler", tags=["admin"])
@@ -41,7 +42,7 @@ def set_job_interval(
"""Change how often a scheduler job runs (minutes). Persisted as an override and applied
to the live scheduler immediately."""
if job_id not in JOB_INTERVALS:
raise HTTPException(status_code=404, detail="Unknown job")
raise not_found("job")
minutes = body.interval_minutes
if not (MIN_INTERVAL <= minutes <= MAX_INTERVAL):
raise HTTPException(
@@ -73,7 +74,7 @@ def run_job_now(
try:
result = trigger_job(job_id, actor_id=user.id)
except KeyError:
raise HTTPException(status_code=404, detail="Unknown job")
raise not_found("job")
if result == "already_running":
raise HTTPException(status_code=409, detail="Job is already running")
return {"id": job_id, "started": True}
+1 -1
View File
@@ -80,7 +80,7 @@ def create_tag(
def _own_tag(db: Session, user: User, tag_id: int) -> Tag:
# System tags (user_id NULL) are not user-editable → owned_or_404 404s them too.
return owned_or_404(db, Tag, tag_id, user, detail="Unknown tag")
return owned_or_404(db, Tag, tag_id, user, what="tag")
class TagUpdateIn(BaseModel):
+2
View File
@@ -167,6 +167,8 @@ SPECS: tuple[ConfigSpec, ...] = (
ConfigSpec(
"plex_transcode_max_height", "int", "plex", "plex_transcode_max_height", min=0, max=4320
),
# Audio loudness normalisation on the HLS path (remux + transcode alike, not direct play).
ConfigSpec("plex_audio_loudnorm", "bool", "plex", "plex_audio_loudnorm"),
# NOTE: plex_watch_sync_interval_min is a config.py default read by the watch-sync scheduler job
# (scheduler.py); it is intentionally NOT a runtime admin knob, so it stays out of SPECS.
# --- TMDB enrichment (E5 S2; optional) — the API key is the only secret ---
+44 -10
View File
@@ -32,12 +32,12 @@ from app import sysconfig
from app.config import settings
from app.db import SessionLocal
from app.downloads import edit as editmod
from app.downloads import quota, service
from app.downloads import credentials, formats, quota, service
from app.downloads import errors as dlerrors
from app.downloads.jobdb import (
_edit_staging_dir,
_requeue_if_running,
_reset_asset_pending,
_set_job,
_staging_dir,
)
from app.downloads.status import STOPPED_BY_ROUTE, JobStatus
@@ -253,7 +253,27 @@ def _fail_after_kill(
"""`_stop_and_free` + mark the job errored — for the watchdog's stall/timeout kills and the
supervisor-error fallback."""
_stop_and_free(job_id, asset_id, proc, grace_s)
_set_job(job_id, status="error", error=error, error_code=error_code, phase=None)
_fail_running(job_id, error=error, error_code=error_code)
def _fail_running(
job_id: int, *, error: str, error_code: str | None, error_detail: str | None = None
) -> None:
"""Fail a claimed job — only while it is still `running` (`service.finalize_if_running`), so a
cancel/pause a route landed first is never turned back into `error`, which holds the asset the
route already released. `error_detail` is written every time: a failure never shows an earlier
failure's explanation."""
with SessionLocal() as db:
service.finalize_if_running(
db,
job_id,
status=JobStatus.ERROR,
error=error,
error_code=error_code,
error_detail=error_detail,
phase=None,
)
db.commit()
def _run_supervised(job_id: int, asset_id: int) -> None:
@@ -329,12 +349,10 @@ def _run_supervised(job_id: int, asset_id: int) -> None:
status, _ = _job_liveness(job_id)
if status == "running":
_cleanup_staging(asset_id)
_set_job(
_fail_running(
job_id,
status="error",
error_code="unknown",
phase=None,
error=f"The download process exited unexpectedly (code {proc.returncode}).",
error_code="unknown",
)
log.warning(
"job %s: child exited %s without finishing", job_id, proc.returncode
@@ -399,16 +417,32 @@ def _process_job(job_id: int) -> None:
asset_id = asset.id if asset else None
asset_status = asset.status if asset else None
asset_error = asset.error if asset else None
if asset_status == "error":
# A job that meets an already-failed asset gets that failure's explanation — not a bare
# "Download failed". Only the text is on the asset, so it is diagnosed again, for this
# link, with whether a cookie goes with it asked of the same gate the download child
# used (`credentials.usable_cookie`) — as it stands now, so the advice fits this job's Retry.
url = service.source_url(job.source_kind, job.source_ref)
cookie = credentials.usable_cookie(
db, job.user_id, url, asset, formats.normalize(job.profile_snapshot)
)
diagnosis = dlerrors.diagnose(asset_error, url=url, cookie_attached=cookie is not None)
del cookie # only whether there is one matters here
if asset is None:
_set_job(job_id, status="error", error="No media asset", phase=None)
_fail_running(job_id, error="No media asset", error_code=None)
return
if asset_status == "ready":
_finish_from_cache(job_id, asset_id)
return
if asset_status == "error":
_set_job(job_id, status="error", error=asset_error or "Download failed", phase=None)
_fail_running(
job_id,
error=asset_error or "Download failed",
error_code=diagnosis.code,
error_detail=diagnosis.detail,
)
return
if asset_status == "downloading":
# Another worker won this asset between our claim and here (a narrow race — the claim SQL
@@ -461,7 +495,7 @@ def _worker_loop(idx: int) -> None:
_process_job(job_id)
except Exception: # noqa: BLE001
log.exception("processing job %s crashed", job_id)
_set_job(job_id, status="error", error="Worker error", phase=None)
_fail_running(job_id, error="Worker error", error_code=None)
def main() -> None:
+5
View File
@@ -7,6 +7,7 @@ Authlib==1.7.2
Mako==1.4.1
MarkupSafe==3.0.3
PyYAML==6.0.3
Pygments==2.21.0
SQLAlchemy==2.0.52
alembic==1.19.1
annotated-doc==0.0.5
@@ -21,6 +22,7 @@ cffi==2.1.1
charset-normalizer==3.5.1
click==8.4.2
cryptography==50.0.0
curl_cffi==0.15.0
fastapi==0.141.1
feedparser-sgmllib==2.1.0
feedparser==6.0.14
@@ -32,6 +34,8 @@ httpx==0.28.1
idna==3.18
itsdangerous==2.2.0
joserfc==1.7.4
markdown-it-py==4.2.0
mdurl==0.1.2
mutagen==1.48.1
numpy==2.5.2
psycopg-binary==3.3.4
@@ -45,6 +49,7 @@ pydantic_core==2.46.4
python-dotenv==1.2.3
python-multipart==0.0.32
requests==2.34.2
rich==15.0.0
starlette==1.6.0
typing-inspection==0.4.4
typing_extensions==4.16.0
+7
View File
@@ -21,6 +21,13 @@ argon2-cffi>=23.1,<26
# scripts Deno runs for YouTube's n-signature). yt-dlp is release-often (YouTube changes), so
# keep the floor loose and bump it when extraction breaks.
yt-dlp[default]>=2025.5.22
# Browser impersonation (`impersonate`) for Facebook downloads made with the user's cookie: some
# links answer a signed-in plain request with "Cannot parse data" and only parse as Chrome (E11.5,
# measured). The range is YT-DLP's, not ours — yt-dlp 2026.07.04 imports curl_cffi only in
# 0.10.x–0.15.x and silently reports every target "unavailable" outside it. Move it when bumping
# yt-dlp; `tests/test_facebook_impersonation.py::test_the_locked_curl_cffi_can_impersonate_chrome`
# fails the gate if the locked pair can no longer impersonate.
curl_cffi>=0.10,<0.16
# PO Token provider plugin: talks to the bgutil sidecar (brainicism/bgutil-ytdlp-pot-provider)
# to mint YouTube Proof-of-Origin tokens on demand, bypassing "confirm you're not a bot" and
# unlocking high-quality formats. Pin to match the sidecar image tag (versions must align).
+10 -1
View File
@@ -103,12 +103,21 @@ def test_generic_source_url_is_passthrough_youtube_reconstructs():
)
def test_client_sets_and_retriable_are_youtube_only():
def test_client_sets_and_retriable_are_per_source():
# Generic sources make a single attempt and never retry on the YouTube per-client markers.
assert SourceAdapter().client_sets() == [None]
assert SourceAdapter().client_sets(authed=True) == [None]
assert SourceAdapter().retriable("Sign in to confirm you're not a bot") is False
assert YouTubeAdapter().retriable("Sign in to confirm you're not a bot") is True
assert YouTubeAdapter().retriable("some unrelated network blip") is False
# Facebook: a second, Chrome-impersonating attempt only WITH the user's cookie, and only behind
# the parse wall (without a cookie that wall is the login wall — no disguise gets past it).
fb = FacebookAdapter()
assert fb.client_sets() == [None]
assert len(fb.client_sets(authed=True)) == 2
assert fb.ydl_opts_extra(fb.client_sets(authed=True)[0]) == {}
assert fb.retriable("ERROR: [facebook] 1: Cannot parse data; please report this issue") is True
assert fb.retriable("ERROR: [facebook] 1: No video formats found!") is False
def test_youtube_ydl_opts_extra_carries_player_client_and_pot():
+90
View File
@@ -0,0 +1,90 @@
"""The shared demo account's refusal says WHY, in the reader's language (deep-review round 6).
`require_human` answered `403 "Not available in the demo account."` — a plain string. `req` leaves
403 to the caller, and round 5 taught `notifyActionError` to discard a plain-string detail in favour
of the caller's already-translated fallback, so from that commit on the demo user got a generic
"couldn't do that" and no hint that the demo account was the reason. Not every affordance behind
this gate is UI-gated on `is_demo` either — the comfort shelf is not, so the pin button is live.
A structured `{code: "demo"}` is what `localizeDetail` can turn into `errors.demo` in both locales.
Pinned at all FOUR sites: the shared dependency, the two auth routes that hand-roll the same check,
and the account-delete route in `routes/me.py` — which round 6 missed while its docstring claimed
all the sites were covered, so the next reader had no reason to look (round 7). An asymmetric fix is
the smell this repo bans, and a docstring asserting symmetry it does not have is worse than none.
The list is derived from the source, not typed out: `test_every_is_demo_refusal_is_on_the_list`
below fails when a fifth `is_demo` 403 appears anywhere in `app/`, so this file cannot fall behind
the code the way round 6's did. Needs the DB.
"""
import re
from pathlib import Path
from app.models import User
def test_every_demo_refusal_carries_the_same_structured_reason(db, make_user, as_user):
demo = make_user(email="demo@example.com", is_demo=True)
client = as_user(demo)
for method, url, body in [
# `require_human` itself, through an affordance the UI does NOT gate on is_demo.
("post", "/api/discover/pins", {"kind": "video", "ref": "v1"}),
# The three routes that repeat the check inline rather than depending on the gate.
("post", "/auth/logout-others", None),
("post", "/auth/set-password", {"password": "a long enough password"}),
("delete", "/api/me/account", None),
]:
r = getattr(client, method)(url, **({"json": body} if body is not None else {}))
assert r.status_code == 403, (url, r.text)
detail = r.json()["detail"]
# Every one of them is a dict whose `code` the frontend can localize; the delete route adds
# a `reason` so its refusal can be worded specifically.
assert isinstance(detail, dict), (url, detail)
assert detail["code"] == "demo", (url, detail)
def test_every_is_demo_refusal_is_on_the_list():
"""A fifth site must not be able to appear unnoticed.
Round 6 converted three of four `is_demo` 403s and wrote a docstring saying it had converted
them all; the fourth kept answering a bare English string for another round because nothing
counted them. So count them here: every `is_demo`-guarded 403 in `app/` has to raise a
structured detail, and the number of them has to match what this file exercises."""
app_dir = Path(__file__).resolve().parents[1] / "app"
sites = []
for path in sorted(app_dir.rglob("*.py")):
lines = path.read_text(encoding="utf-8").splitlines()
for i, line in enumerate(lines):
# `if user.is_demo:` / `if u.is_demo:` — the hand-rolled gate, not the ORM filters
# (`User.is_demo.is_(False)`) and not the column definition.
if not re.search(r"^\s*if\s+\w+\.is_demo\s*:", line):
continue
block = "\n".join(lines[i : i + 6])
if "403" not in block:
continue
sites.append(f"{path.relative_to(app_dir)}:{i + 1}")
assert 'detail={"code": "demo"' in block, (
f"{path.relative_to(app_dir)}:{i + 1} refuses the demo account with an "
f"unstructured detail — the frontend cannot localize it:\n{block}"
)
# auth.py's shared gate + routes/auth.py's two + routes/me.py's delete. A new one is welcome,
# but it has to be added to the loop above in the same commit.
assert len(sites) == 4, sites
def test_a_real_account_is_not_refused_by_the_gate(db, make_user, as_user):
"""The other half: the structured detail must not come from having broken the gate open the
other way. A non-demo account gets past `require_human` and fails (or succeeds) on its own
merits — anything but a 403 demo refusal."""
client = as_user(make_user(email="real@example.com"))
r = client.post("/auth/logout-others")
assert r.status_code == 200, r.text
assert not isinstance(r.json().get("detail"), dict)
def test_the_gate_reads_the_flag_and_not_the_email(db):
"""`is_demo` is a column, not a convention about the address — a real account that happens to
be called demo@ must not be refused."""
from app.auth import require_human
assert require_human(User(email="demo@example.com", is_demo=False)) is not None
+942
View File
@@ -0,0 +1,942 @@
"""E24 S1 — the Home surface's read-model.
Covers the four things that can silently rot here: the time-budget bands (pure), the cross-source
resume rail's merge order, the hero's tier ladder, and the comfort shelf (its schema-level one-of
invariant, its cascade, and the resolved `next` that makes one click enough).
DB lane (`db` fixture) except for the budget block — skipped without Postgres.
"""
from datetime import datetime, timedelta, timezone
import pytest
from sqlalchemy import delete, func, select, text
from sqlalchemy.exc import IntegrityError
from app.discover import hero, pins, queries, rails
from app.discover.budget import BANDS, fits, remaining_seconds
from app.plex import rollup
from app.plex.cards import leaf_card
from app.models import (
BlockedChannel,
Channel,
ComfortPin,
PlexItem,
PlexLibrary,
PlexShow,
PlexState,
Playlist,
PlaylistItem,
Subscription,
User,
Video,
VideoState,
)
NOW = datetime(2026, 6, 1, tzinfo=timezone.utc)
# ── budget (pure) ─────────────────────────────────────────────────────────────────────────────────
def test_bands_cover_every_runtime_and_overlap_deliberately():
"""Every runtime has a home in `any`, and the bands overlap so "30 perc" is not answered with a
two-minute Short just because it fits under the cap."""
assert fits(3, "any") and fits(10_000, "any")
assert fits(2 * 60, "short") and not fits(2 * 60, "medium") # a Short is not a 30-minute answer
assert fits(30 * 60, "medium") and not fits(30 * 60, "short")
assert fits(110 * 60, "long") and not fits(110 * 60, "medium")
def test_unknown_duration_only_fits_the_unbounded_band():
"""We cannot promise an unknown-length item finishes inside ten minutes, so it is not offered as
an answer to a bounded question — the one control on the page has to mean what it says."""
for budget in BANDS:
assert fits(None, budget) is (budget == "any")
def test_budget_is_matched_against_what_is_LEFT():
"""A 90-minute film with 8 minutes to go is a ten-minute answer."""
assert remaining_seconds(90 * 60, 82 * 60) == 8 * 60
assert fits(remaining_seconds(90 * 60, 82 * 60), "short")
assert not fits(remaining_seconds(90 * 60, 0), "short")
assert remaining_seconds(None, 30) is None
assert remaining_seconds(600, 900) == 0 # a position past the end never goes negative
# ── fixtures ──────────────────────────────────────────────────────────────────────────────────────
def _seed_yt(db) -> tuple[User, Channel]:
"""A subscribed channel with three videos of distinct lengths, so a budget can pick between them."""
user = User(email="discover-e24@example.com")
ch = Channel(id="UCe24", title="Comfort Channel", handle="@comfort")
db.add_all([user, ch])
db.flush()
db.add(Subscription(user_id=user.id, channel_id=ch.id))
db.add_all(
[
Video(
id="vshort",
channel_id=ch.id,
title="Ten minute video",
duration_seconds=9 * 60,
published_at=NOW - timedelta(days=1),
),
Video(
id="vmed",
channel_id=ch.id,
title="Half hour video",
duration_seconds=30 * 60,
published_at=NOW - timedelta(days=2),
),
Video(
id="vlong",
channel_id=ch.id,
title="Feature length video",
duration_seconds=95 * 60,
published_at=NOW - timedelta(days=3),
),
]
)
db.commit()
return user, ch
def _seed_plex(db, user: User) -> tuple[PlexItem, PlexShow, PlexItem]:
lib_m = PlexLibrary(plex_key="90", title="Movies", kind="movie", enabled=True)
lib_s = PlexLibrary(plex_key="91", title="Shows", kind="show", enabled=True)
db.add_all([lib_m, lib_s])
db.flush()
movie = PlexItem(
rating_key="pm1",
library_id=lib_m.id,
kind="movie",
title="A Movie",
duration_s=100 * 60,
playable="direct",
)
show = PlexShow(rating_key="ps1", library_id=lib_s.id, title="A Show", child_count=1)
db.add_all([movie, show])
db.flush()
ep1 = PlexItem(
rating_key="ps1e1",
library_id=lib_s.id,
kind="episode",
title="Ep 1",
show_id=show.id,
season_number=1,
episode_number=1,
duration_s=45 * 60,
)
ep2 = PlexItem(
rating_key="ps1e2",
library_id=lib_s.id,
kind="episode",
title="Ep 2",
show_id=show.id,
season_number=1,
episode_number=2,
duration_s=45 * 60,
)
db.add_all([ep1, ep2])
db.commit()
return movie, show, ep1
def _enable_plex(db):
from app import sysconfig
sysconfig.set_value(db, "plex_enabled", True)
db.commit()
# ── continue watching ─────────────────────────────────────────────────────────────────────────────
def test_continue_watching_merges_both_sources_by_recency(db):
"""One rail, not two lists side by side — and ordered by when the user last touched each thing,
across sources. Two side-by-side rails would just move the decision up a level."""
user, _ = _seed_yt(db)
movie, _, _ = _seed_plex(db, user)
_enable_plex(db)
db.add_all(
[
VideoState(
user_id=user.id,
video_id="vmed",
status="new",
position_seconds=300,
progress_updated_at=NOW - timedelta(hours=5),
),
PlexState(
user_id=user.id,
item_id=movie.id,
status="new",
position_seconds=600,
progress_updated_at=NOW - timedelta(hours=1),
),
]
)
db.commit()
got = rails.continue_watching(db, user)
assert [(c["source"], c["id"]) for c in got] == [("plex", "pm1"), ("youtube", "vmed")]
assert got[0]["remaining_seconds"] == 100 * 60 - 600
def test_continue_watching_is_youtube_only_when_plex_is_off(db):
"""Plex is a deployment toggle; with it off the surface says nothing about Plex at all."""
user, _ = _seed_yt(db)
movie, _, _ = _seed_plex(db, user)
db.add_all(
[
VideoState(
user_id=user.id,
video_id="vmed",
status="new",
position_seconds=300,
progress_updated_at=NOW,
),
PlexState(
user_id=user.id,
item_id=movie.id,
status="new",
position_seconds=600,
progress_updated_at=NOW,
),
]
)
db.commit()
assert {c["source"] for c in rails.continue_watching(db, user)} == {"youtube"}
def test_the_resume_rail_obeys_the_channel_blocklist(db):
"""Blocking a channel does not touch `video_states`, so a half-watched video from it survives
with its resume position — and this rail builds its own SELECT instead of going through
`filtered_query`. "Never show me this channel" has to outrank "you started it", or the landing
page answers "what should I watch" with the thing the user has just banned."""
user, ch = _seed_yt(db)
db.add(
VideoState(
user_id=user.id,
video_id="vmed",
status="new",
position_seconds=300,
progress_updated_at=NOW,
)
)
db.commit()
assert [c["id"] for c in rails.continue_watching(db, user)] == ["vmed"]
db.add(BlockedChannel(user_id=user.id, channel_id=ch.id))
db.commit()
assert rails.continue_watching(db, user) == []
# And so the hero, whose resume tier draws from that rail, has nothing to offer either.
assert hero.pick(db, user, "any", 0) is None
def test_a_null_timestamp_sorts_last(db):
"""A missing progress timestamp sorts last rather than 500ing the landing page.
Narrower than it was written to be, and the epic-end review corrected it: the previous key,
`(ts is not None, ts)`, does NOT raise on two nulls — tuple comparison finds the first UNEQUAL
element via `==`, so `None < None` is never reached, and this test passes under that key too.
What it does discriminate is the naive `key=lambda r: r[0]`, which really does raise. See
`plex.rollup.NO_TS` for the corrected rationale."""
user, _ = _seed_yt(db)
db.add_all(
[
VideoState(
user_id=user.id, video_id="vmed", status="new", position_seconds=1
), # no progress_updated_at
VideoState(
user_id=user.id, video_id="vlong", status="new", position_seconds=1
), # nor here
VideoState(
user_id=user.id,
video_id="vshort",
status="new",
position_seconds=1,
progress_updated_at=NOW,
),
]
)
db.commit()
got = rails.continue_watching(db, user)
assert got[0]["id"] == "vshort"
assert len(got) == 3
# ── hero ──────────────────────────────────────────────────────────────────────────────────────────
def test_hero_prefers_resume_and_labels_the_tier(db):
user, _ = _seed_yt(db)
db.add(
VideoState(
user_id=user.id,
video_id="vmed",
status="new",
position_seconds=300,
progress_updated_at=NOW,
)
)
db.commit()
got = hero.pick(db, user, "any", seed=1)
assert got["tier"] == "resume"
assert got["card"]["id"] == "vmed"
def test_hero_skips_a_resume_that_does_not_fit_the_budget(db):
""" "I have ten minutes" must not be answered with the 80 minutes left on a film, even though
that film is the most recent thing the user touched."""
user, _ = _seed_yt(db)
db.add(
VideoState(
user_id=user.id,
video_id="vlong",
status="new",
position_seconds=60,
progress_updated_at=NOW,
)
)
db.commit()
got = hero.pick(db, user, "short", seed=1)
assert got["tier"] == "surprise"
assert got["card"]["duration_seconds"] <= BANDS["short"][1]
def test_hero_falls_back_to_the_shelf_before_guessing(db):
"""The user's own declared fallback beats anything we could infer."""
user, ch = _seed_yt(db)
db.add(ComfortPin(user_id=user.id, video_id="vmed", position=0))
db.commit()
got = hero.pick(db, user, "medium", seed=1)
assert got["tier"] == "comfort"
assert got["card"]["id"] == "vmed"
def test_hero_resolves_a_show_to_the_episode_that_would_play(db):
"""A show has no Play button. The hero of a partially-watched show is its next episode."""
user, _ = _seed_yt(db)
_, show, ep1 = _seed_plex(db, user)
_enable_plex(db)
db.add(
PlexState(
user_id=user.id,
item_id=ep1.id,
status="watched",
watched_at=NOW,
progress_updated_at=NOW,
)
)
db.commit()
got = hero.pick(db, user, "any", seed=1)
assert got["tier"] == "resume"
assert got["card"]["kind"] == "episode"
assert got["card"]["id"] == "ps1e2" # ep1 is finished → ep2 is what plays
def test_hero_is_none_when_nothing_fits(db):
"""An empty answer is an honest answer: the page renders its own empty state rather than being
handed a card that does not answer what was asked. Here the whole library is under 40 minutes,
so "egy este" has nothing to offer."""
user, ch = _seed_yt(db)
db.execute(text("DELETE FROM videos WHERE id = 'vlong'"))
db.commit()
assert hero.pick(db, user, "long", seed=1) is None
def test_surprise_is_reproducible_for_a_seed(db):
"""The reroll button depends on this: one seed drives both sources, so the same seed must give
the same answer and a different seed must be free to give another."""
user, _ = _seed_yt(db)
a = rails.surprise(db, user, "any", seed=7, limit=3)
b = rails.surprise(db, user, "any", seed=7, limit=3)
assert [c["id"] for c in a] == [c["id"] for c in b]
seeds = {tuple(c["id"] for c in rails.surprise(db, user, "any", s, 3)) for s in range(12)}
assert len(seeds) > 1 # the seed actually reaches the ordering
def test_surprise_never_offers_something_already_started(db):
"""The resume tier may have skipped an item because its REMAINING time did not fit; handing the
same item back as "Meglepetés" — where the full runtime does fit — would contradict the question
the user just answered."""
user, _ = _seed_yt(db)
db.add(
VideoState(
user_id=user.id,
video_id="vlong",
status="new",
position_seconds=90 * 60,
progress_updated_at=NOW,
)
)
db.commit()
assert "vlong" not in {c["id"] for c in rails.surprise(db, user, "long", seed=3, limit=5)}
def test_the_seed_decides_which_source_leads(db):
"""With a pool of one, a fixed source order would mean the Plex library could never answer "egy
este" whenever a YouTube video also fit — the cross-source promise would quietly be a YouTube
promise."""
user, _ = _seed_yt(db)
_seed_plex(db, user)
_enable_plex(db)
leaders = {rails.surprise(db, user, "any", s, limit=1)[0]["source"] for s in range(6)}
assert leaders == {"youtube", "plex"}
def test_the_reroll_must_be_able_to_move_past_the_resume_tier(db):
"""A new seed alone only re-draws the SURPRISE tier — the resume tier deterministically returns
the most recently progressed thing that fits, so without an exclusion the reroll handed the same
card straight back and the button looked dead (found in the browser, 2026-09-06)."""
user, _ = _seed_yt(db)
db.add_all(
[
VideoState(
user_id=user.id,
video_id="vmed",
status="new",
position_seconds=300,
progress_updated_at=NOW,
),
VideoState(
user_id=user.id,
video_id="vshort",
status="new",
position_seconds=60,
progress_updated_at=NOW - timedelta(hours=1),
),
]
)
db.commit()
first = hero.pick(db, user, "any", seed=1)
assert first["card"]["id"] == "vmed"
# Same seed, same everything — only the exclusion differs.
second = hero.pick(db, user, "any", seed=1, exclude={"youtube:vmed"})
assert second["card"]["id"] == "vshort"
def test_the_reroll_rotates_which_tier_answers(db):
"""Measured 2026-09-06: exclusion alone left the button walking down the resume rail (20 rows)
and cycling nine cards, every one of them `resume` — the shelf and the surprise tier were
unreachable by pressing it. `start` is the press count, so consecutive presses offer a different
KIND of answer."""
user, _ = _seed_yt(db)
# One of each: something half-finished, something pinned, and a third video for the draw.
db.add(
VideoState(
user_id=user.id,
video_id="vmed",
status="new",
position_seconds=300,
progress_updated_at=NOW,
)
)
db.add(ComfortPin(user_id=user.id, video_id="vlong", position=0))
db.commit()
tiers = [hero.pick(db, user, "any", seed=1, start=n)["tier"] for n in range(4)]
assert tiers == ["resume", "comfort", "surprise", "resume"]
def test_the_rotation_still_falls_through_an_empty_tier(db):
"""A press must not come back empty just because the tier it happened to lead with has nothing.
Here there is no shelf at all, so the comfort-led press has to answer from somewhere else."""
user, _ = _seed_yt(db)
assert db.execute(select(func.count()).select_from(ComfortPin)).scalar_one() == 0
got = hero.pick(db, user, "any", seed=1, start=1)
assert got is not None and got["tier"] != "comfort"
def test_excluding_a_show_also_excludes_the_episode_it_resolves_to(db):
"""The user rerolled past the SHOW they were shown; handing back its next episode under a
different title would be the same suggestion wearing a hat."""
user, _ = _seed_yt(db)
_, show, ep1 = _seed_plex(db, user)
_enable_plex(db)
db.add(
PlexState(
user_id=user.id,
item_id=ep1.id,
status="new",
position_seconds=120,
progress_updated_at=NOW,
)
)
db.commit()
shown = hero.pick(db, user, "any", seed=1)
assert shown["card"]["id"] == "ps1e1" # the show resolved to its in-progress episode
# Excluding what the USER saw — the show — must move the tier on, not return the episode again.
after = hero.pick(db, user, "any", seed=1, exclude={"plex:ps1"})
assert after is None or after["card"]["id"] != "ps1e1"
# ── comfort shelf ─────────────────────────────────────────────────────────────────────────────────
def test_a_pin_must_reference_exactly_one_thing(db):
"""The union invariant lives in the schema, not in a comment at every insert site."""
user, ch = _seed_yt(db)
for kwargs in ({}, {"video_id": "vmed", "channel_id": ch.id}):
db.add(ComfortPin(user_id=user.id, position=0, **kwargs))
with pytest.raises(IntegrityError):
db.flush()
db.rollback()
def test_the_same_thing_cannot_be_pinned_twice(db):
"""The per-kind unique constraints are what make the route's concurrent-tab path safe (it catches
`IntegrityError` precisely because of them) — and the migration drift check filters constraint
diffs out as noise, so nothing else would notice them going missing. The route's own dedupe
returns early without ever reaching the DB, so only this test exercises them."""
user, _ = _seed_yt(db)
db.add(ComfortPin(user_id=user.id, video_id="vmed", position=0))
db.commit()
db.add(ComfortPin(user_id=user.id, video_id="vmed", position=1))
with pytest.raises(IntegrityError):
db.flush()
db.rollback()
def test_deleting_the_pinned_row_removes_the_pin(db):
"""A film dropped from the library must not leave a dead tile on the one surface whose whole job
is to be reliable — which is why the shelf holds foreign keys, not strings."""
user, _ = _seed_yt(db)
db.add(ComfortPin(user_id=user.id, video_id="vmed", position=0))
db.commit()
db.execute(text("DELETE FROM videos WHERE id = 'vmed'"))
db.commit()
assert db.execute(select(func.count()).select_from(ComfortPin)).scalar() == 0
def test_shelf_resolves_a_channel_to_its_newest_unwatched_video(db):
""" "Put that channel on" has to be one click, so the tile carries what would play."""
user, ch = _seed_yt(db)
db.add(ComfortPin(user_id=user.id, channel_id=ch.id, position=0))
db.commit()
(card,) = pins.shelf(db, user)
assert card["kind"] == "channel"
assert card["unwatched_count"] == 3
assert card["next"]["id"] == "vshort" # newest
def test_shelf_resolves_a_playlist_to_its_first_unwatched_item(db):
user, _ = _seed_yt(db)
pl = Playlist(user_id=user.id, name="Elalvós")
db.add(pl)
db.flush()
db.add_all(
[
PlaylistItem(playlist_id=pl.id, video_id="vshort", position=0),
PlaylistItem(playlist_id=pl.id, video_id="vmed", position=1),
]
)
db.add(VideoState(user_id=user.id, video_id="vshort", status="watched", watched_at=NOW))
db.add(ComfortPin(user_id=user.id, playlist_id=pl.id, position=0))
db.commit()
(card,) = pins.shelf(db, user)
assert card["kind"] == "playlist"
assert card["item_count"] == 2
assert card["next"]["id"] == "vmed"
def test_a_pinned_video_survives_leaving_the_feed_scope(db):
"""The shelf is the fallback surface. An entry silently vanishing because the user unsubscribed
from the channel is exactly the failure it must not have."""
user, ch = _seed_yt(db)
db.add(ComfortPin(user_id=user.id, video_id="vmed", position=0))
db.commit()
db.execute(delete(Subscription))
db.commit()
(card,) = pins.shelf(db, user)
assert card["id"] == "vmed"
assert card["next"]["id"] == "vmed"
def test_the_plex_surprise_obeys_the_time_budget(db):
"""The one control on the page has to mean what it says: a 100-minute film is not an answer to
"10 perc". The YouTube half of this rule has had its own test since S1; the Plex half's duration
bounds could be deleted with the whole suite still green, because every test that reached
`plex_surprise` passed budget="any"."""
user, _ = _seed_yt(db)
_seed_plex(db, user)
_enable_plex(db)
drawn = lambda budget: { # noqa: E731
(c["source"], c["id"]) for s in range(6) for c in rails.surprise(db, user, budget, s, 5)
}
assert ("plex", "pm1") not in drawn("short")
# …and it IS offered where it fits, so this cannot pass by the rail being empty.
assert ("plex", "pm1") in drawn("long")
def test_the_plex_surprise_never_offers_something_already_started(db):
"""A half-watched film is not a surprise, and the resume tier has already had its chance at it.
The mirror of `test_surprise_never_offers_something_already_started` for the other source — the
asymmetric half of one principle, which the house rules say to fix everywhere."""
user, _ = _seed_yt(db)
movie, _, _ = _seed_plex(db, user)
_enable_plex(db)
db.add(
PlexState(
user_id=user.id,
item_id=movie.id,
status="new",
position_seconds=600,
progress_updated_at=NOW,
)
)
db.commit()
drawn = {
(c["source"], c["id"]) for s in range(6) for c in rails.surprise(db, user, "long", s, 5)
}
assert ("plex", "pm1") not in drawn
def test_a_new_pin_lands_at_the_end_even_after_a_hole(db):
"""Positions are never renumbered on removal — `remove_pin` deletes the row, and every FK cascade
silently removes a pin when its target disappears — so the append position has to come from
`max(position)`, not from the row count. With a count, the first hole drops the next pin into the
MIDDLE of the shelf and the second makes two pins claim one slot."""
user, ch = _seed_yt(db)
db.add_all(
[
ComfortPin(user_id=user.id, video_id="vshort", position=0),
ComfortPin(user_id=user.id, video_id="vmed", position=1),
ComfortPin(user_id=user.id, video_id="vlong", position=2),
]
)
db.commit()
db.execute(delete(ComfortPin).where(ComfortPin.video_id.in_(("vshort", "vmed"))))
db.commit()
db.add(ComfortPin(user_id=user.id, channel_id=ch.id, position=pins.next_position(db, user)))
db.commit()
assert [c["kind"] for c in pins.shelf(db, user)] == ["video", "channel"]
positions = (
db.execute(select(ComfortPin.position).order_by(ComfortPin.position)).scalars().all()
)
assert len(set(positions)) == len(positions) # and no two tiles claim the same slot
def test_a_self_playing_pin_does_not_reference_itself(db):
"""`next` must be a distinct dict even when the pin plays itself. Aliasing the card into its own
`next` is a cycle, and the serializer turns the whole shelf into a 500 over it."""
user, _ = _seed_yt(db)
db.add(ComfortPin(user_id=user.id, video_id="vmed", position=0))
db.commit()
(card,) = pins.shelf(db, user)
assert card["next"] is not card
assert "next" not in card["next"]
def test_the_resume_rail_says_what_would_actually_play(db):
"""A show tile that only opens the show page moves the decision instead of removing it — the
click lands on six seasons to choose between, which is the paralysis this whole surface exists
for. The shelf and the hero already resolved; this rail did not (reported 2026-09-06). The tile
stays the show (its artwork is what the user recognises); `next` is what starts."""
user, _ = _seed_yt(db)
_, show, ep1 = _seed_plex(db, user)
_enable_plex(db)
db.add(
PlexState(
user_id=user.id,
item_id=ep1.id,
status="watched",
watched_at=NOW,
progress_updated_at=NOW,
)
)
db.commit()
rail = rails.continue_watching(db, user)
tile = next(c for c in rail if c["kind"] == "show")
assert tile["id"] == "ps1" # still the show
assert tile["next"]["kind"] == "episode"
assert tile["next"]["id"] == "ps1e2" # ep1 is finished → ep2 is what plays
def test_only_a_show_row_pays_for_a_resolution(db):
"""`next` is the answer to "a show has no Play button". Attaching it to a movie or a video would
both cost a query for nothing and re-open the self-reference cycle the shelf already hit.
The states are what make this test exist: without them every rail query inner-joins its way to
zero rows, the loop body never runs, and the assertion is vacuous — which is exactly how the
epic-end review found it."""
user, _ = _seed_yt(db)
movie, _, ep1 = _seed_plex(db, user)
_enable_plex(db)
db.add_all(
[
VideoState(
user_id=user.id,
video_id="vmed",
status="new",
position_seconds=300,
progress_updated_at=NOW,
),
PlexState(
user_id=user.id,
item_id=movie.id,
status="new",
position_seconds=600,
progress_updated_at=NOW,
),
PlexState(
user_id=user.id,
item_id=ep1.id,
status="new",
position_seconds=120,
progress_updated_at=NOW,
),
]
)
db.commit()
rail = rails.continue_watching(db, user)
# A video, a movie and a show — one row of each kind that can appear here, so the assertion below
# actually exercises both sides of the equality.
assert {c["kind"] for c in rail} == {"video", "movie", "show"}
for card in rail:
assert ("next" in card) == (card["kind"] == "show"), card["id"]
def test_next_episode_takes_an_episode_with_no_state_row_at_all(db):
"""The NULL trap in the SQL rewrite: `status NOT IN ('watched','hidden')` is UNKNOWN — not true —
for an episode the user has never touched, so a bare filter drops exactly the common case and the
rail would offer episode 1 of a show already half-watched."""
user, _ = _seed_yt(db)
_, show, ep1 = _seed_plex(db, user)
db.add(PlexState(user_id=user.id, item_id=ep1.id, status="watched", watched_at=NOW))
db.commit() # ep2 has NO PlexState row
assert queries.next_episode(db, user.id, show.id)["id"] == "ps1e2"
def test_a_finished_show_starts_again_from_the_top(db):
"""Nothing unwatched left is not "nothing to play" — it is "watch it again"."""
user, _ = _seed_yt(db)
_, show, ep1 = _seed_plex(db, user)
ep2 = db.execute(select(PlexItem).where(PlexItem.rating_key == "ps1e2")).scalar_one()
db.add_all(
[
PlexState(user_id=user.id, item_id=ep1.id, status="watched", watched_at=NOW),
PlexState(user_id=user.id, item_id=ep2.id, status="watched", watched_at=NOW),
]
)
db.commit()
assert queries.next_episode(db, user.id, show.id)["id"] == "ps1e1"
def test_next_episode_agrees_with_the_show_page_when_two_are_in_progress(db):
"""One show, one answer. `plex.rollup.rollup` — which drives the show page's Resume button —
takes the LAST in-progress episode in broadcast order; taking the FIRST here gave the same show
two different "continue" answers, with the rail ranking it by its most recent touch and then
offering the episode abandoned weeks earlier. Two in-progress episodes are not exotic: a scrub to
the end leaves a stale `status='new', position>0` row until the daily reconcile."""
user, _ = _seed_yt(db)
_, show, ep1 = _seed_plex(db, user)
ep2 = db.execute(select(PlexItem).where(PlexItem.rating_key == "ps1e2")).scalar_one()
db.add_all(
[
PlexState(
user_id=user.id,
item_id=ep1.id,
status="new",
position_seconds=300,
progress_updated_at=NOW - timedelta(days=9),
),
PlexState(
user_id=user.id,
item_id=ep2.id,
status="new",
position_seconds=600,
progress_updated_at=NOW,
),
]
)
db.commit()
got = queries.next_episode(db, user.id, show.id)
assert got["id"] == "ps1e2"
# Asserted against the other implementation rather than against a literal, so the two cannot
# drift apart again without this failing.
ordered = [
leaf_card(ep, db.execute(select(PlexState).where(PlexState.item_id == ep.id)).scalar_one())
for ep in (ep1, ep2)
]
assert rollup.rollup(ordered)["resume"]["id"] == got["id"]
def test_next_episode_prefers_the_one_in_progress(db):
"""In progress beats "next unwatched": resuming is cheaper than starting."""
user, _ = _seed_yt(db)
_, show, ep1 = _seed_plex(db, user)
ep2 = db.execute(select(PlexItem).where(PlexItem.rating_key == "ps1e2")).scalar_one()
db.add_all(
[
PlexState(user_id=user.id, item_id=ep1.id, status="watched", watched_at=NOW),
PlexState(user_id=user.id, item_id=ep2.id, status="new", position_seconds=120),
]
)
db.commit()
assert queries.next_episode(db, user.id, show.id)["id"] == "ps1e2"
# ── HTTP surface ──────────────────────────────────────────────────────────────────────────────────
def test_discover_endpoint_answers_in_one_pass(db, as_user):
user, _ = _seed_yt(db)
db.add(
VideoState(
user_id=user.id,
video_id="vmed",
status="new",
position_seconds=300,
progress_updated_at=NOW,
)
)
db.commit()
r = as_user(user).get("/api/discover?budget=medium")
assert r.status_code == 200
body = r.json()
assert body["budget"] == "medium"
assert body["hero"]["tier"] == "resume"
assert [c["id"] for c in body["continue_watching"]] == ["vmed"]
assert body["comfort"] == []
def test_an_unknown_budget_is_refused_rather_than_silently_widened(db, as_user):
user, _ = _seed_yt(db)
assert as_user(user).get("/api/discover?budget=whenever").status_code == 422
def test_pinning_twice_is_the_same_shelf(db, as_user):
"""The gesture is "make sure this is on my shelf", so a re-pin is a no-op, not a 409."""
user, _ = _seed_yt(db)
c = as_user(user)
first = c.post("/api/discover/pins", json={"kind": "video", "ref": "vmed"}).json()
second = c.post("/api/discover/pins", json={"kind": "video", "ref": "vmed"}).json()
assert first["created"] is True and second["created"] is False
assert first["id"] == second["id"]
assert len(c.get("/api/discover/pins").json()["items"]) == 1
def test_pinning_a_plex_show_resolves_to_the_episode_that_would_play(db, as_user):
"""Two of the five pinnable kinds are Plex, and the rating_key → internal-id translation is
theirs alone — a wrong column or a wrong model in either arm pins the wrong row, or 404s every
Plex pin. Neither arm had a test before the epic-end review."""
user, _ = _seed_yt(db)
_, _, ep1 = _seed_plex(db, user)
_enable_plex(db)
db.add(PlexState(user_id=user.id, item_id=ep1.id, status="watched", watched_at=NOW))
db.commit()
c = as_user(user)
assert c.post("/api/discover/pins", json={"kind": "plex_show", "ref": "ps1"}).status_code == 200
(tile,) = c.get("/api/discover/pins").json()["items"]
assert (tile["kind"], tile["id"]) == ("show", "ps1")
assert tile["next"]["id"] == "ps1e2" # ep1 is finished
def test_pinning_a_plex_movie_plays_itself_without_a_cycle(db, as_user):
"""The self-reference that once 500'd the whole shelf, on the other leaf kind: `next` has to be a
distinct dict, not the tile aliased into itself. Covered for a video since S1, never for a movie."""
user, _ = _seed_yt(db)
_seed_plex(db, user)
_enable_plex(db)
c = as_user(user)
assert c.post("/api/discover/pins", json={"kind": "plex_item", "ref": "pm1"}).status_code == 200
(tile,) = c.get("/api/discover/pins").json()["items"]
assert tile["id"] == "pm1" and tile["next"]["id"] == "pm1"
def test_the_reroll_endpoint_parses_its_whole_exclusion_list_and_its_press_count(db, as_user):
"""The reroll's behaviour is covered at the domain level; the ROUTE's query-string translation —
the comma split and the `start` passthrough — was not covered at all, so a change there could
hand back the same card on every press with the suite green. Excluding all three candidates is
what proves the whole list parsed: a delimiter change leaves one unsplit string in the set, and a
hero comes back.
NOT the `MAX_EXCLUDE` clamp, which this sends three keys against a cap of 32 and cannot see. The
docstring used to claim it did (round 2). The clamp is deliberately left unpinned: it guards
nothing the route would otherwise do — see the constant's own comment — and a test that sent 33
keys could only assert that the request still answers, which the comma split already proves."""
user, _ = _seed_yt(db)
db.add(
VideoState(
user_id=user.id,
video_id="vmed",
status="new",
position_seconds=300,
progress_updated_at=NOW,
)
)
db.add(ComfortPin(user_id=user.id, video_id="vlong", position=0))
db.commit()
c = as_user(user)
# `start` rotates which tier answers first: 0 leads with resume, 1 with the shelf.
assert c.get("/api/discover/pick?seed=1&start=0").json()["hero"]["tier"] == "resume"
assert c.get("/api/discover/pick?seed=1&start=1").json()["hero"]["tier"] == "comfort"
skip = ",".join(f"youtube:{v}" for v in ("vshort", "vmed", "vlong"))
assert c.get(f"/api/discover/pick?seed=2&exclude={skip}").json()["hero"] is None
assert c.get("/api/discover/pick?budget=whenever").status_code == 422
def test_pinning_something_that_does_not_exist_is_a_404(db, as_user):
user, _ = _seed_yt(db)
r = as_user(user).post("/api/discover/pins", json={"kind": "video", "ref": "nope"})
assert r.status_code == 404
def test_a_pin_whose_target_vanishes_mid_request_is_a_404_not_a_500(db, as_user, monkeypatch):
"""`add_pin`'s `except IntegrityError` arm was reached by no test at all, so the 500→404
narrowing could be reverted with the whole suite green (round 2). The 404 above comes from the
`resolve_ref` PRE-CHECK, a different branch that returns before the INSERT.
The race the arm exists for — a Plex sync dropping the row between `resolve_ref` confirming it
and the INSERT — needs two transactions, but the BRANCH does not: a `resolve_ref` pointing at an
id no row carries lets the pre-check pass and trips the same FK on commit, with the re-query
finding no winner. The sentinel has to FIT `comfort_pins.video_id` (varchar(16)) — a longer one
raises `DataError` on the way in, which this arm does not catch and never meant to. Its sibling arm (a concurrent duplicate, where the re-query DOES find a winner
and answers 200) genuinely needs a second connection and stays uncovered."""
user, _ = _seed_yt(db)
monkeypatch.setattr(pins, "resolve_ref", lambda *a, **k: ("video_id", "gone-mid-req"))
r = as_user(user).post("/api/discover/pins", json={"kind": "video", "ref": "vshort"})
assert r.status_code == 404
def test_another_users_playlist_is_not_pinnable(db, as_user):
"""404, not 403 — a 403 would confirm the id exists to a stranger."""
user, _ = _seed_yt(db)
other = User(email="other-e24@example.com")
db.add(other)
db.flush()
pl = Playlist(user_id=other.id, name="Theirs")
db.add(pl)
db.commit()
r = as_user(user).post("/api/discover/pins", json={"kind": "playlist", "ref": str(pl.id)})
assert r.status_code == 404
def test_reorder_ignores_ids_that_are_not_yours(db, as_user):
"""A stale tab replaying an order that includes a since-deleted pin should reorder what still
exists rather than fail wholesale."""
user, ch = _seed_yt(db)
c = as_user(user)
a = c.post("/api/discover/pins", json={"kind": "video", "ref": "vmed"}).json()["id"]
b = c.post("/api/discover/pins", json={"kind": "channel", "ref": ch.id}).json()["id"]
assert c.post("/api/discover/pins/reorder", json={"ids": [b, 9999, a]}).status_code == 200
assert [p["pin_id"] for p in c.get("/api/discover/pins").json()["items"]] == [b, a]
def test_deleting_a_pin_reports_what_went(db, as_user):
user, _ = _seed_yt(db)
c = as_user(user)
pin_id = c.post("/api/discover/pins", json={"kind": "video", "ref": "vmed"}).json()["id"]
r = c.delete(f"/api/discover/pins/{pin_id}")
assert r.status_code == 200 and r.json() == {"deleted": pin_id}
assert c.get("/api/discover/pins").json()["items"] == []
+153 -5
View File
@@ -18,7 +18,7 @@ from cryptography.fernet import Fernet
from app import security
from app.downloads import credentials, formats, service
from app.models import DownloadCredential, MediaAsset, User
from app.models import DownloadCredential, DownloadJob, MediaAsset, User
@pytest.fixture(autouse=True)
@@ -189,7 +189,9 @@ class TestResolveCookiefile:
db.commit()
spec = formats.normalize({})
salted = credentials.authed_format_sig(spec, u.id, "facebook")
path = credentials.resolve_cookiefile(db, u.id, FB_URL, salted, spec, tmp_path)
path = credentials.resolve_cookiefile(
db, u.id, FB_URL, self._asset_sig(salted), spec, tmp_path
)
assert path is not None and Path(path).read_text() == COOKIES
def test_never_attaches_to_a_shared_asset(self, db, tmp_path):
@@ -200,7 +202,12 @@ class TestResolveCookiefile:
db.commit()
spec = formats.normalize({})
shared = formats.format_sig(spec)
assert credentials.resolve_cookiefile(db, u.id, FB_URL, shared, spec, tmp_path) is None
assert (
credentials.resolve_cookiefile(
db, u.id, FB_URL, self._asset_sig(shared), spec, tmp_path
)
is None
)
def test_non_matching_host_gets_no_cookie(self, db, tmp_path):
u = _user(db)
@@ -210,13 +217,154 @@ class TestResolveCookiefile:
# A YouTube URL has no supported host_key -> no cookie regardless of what's stored.
yt = "https://www.youtube.com/watch?v=abc"
salted = credentials.authed_format_sig(spec, u.id, "facebook")
assert credentials.resolve_cookiefile(db, u.id, yt, salted, spec, tmp_path) is None
assert (
credentials.resolve_cookiefile(db, u.id, yt, self._asset_sig(salted), spec, tmp_path)
is None
)
def test_no_credential_gets_no_cookie(self, db, tmp_path):
u = _user(db)
spec = formats.normalize({})
salted = credentials.authed_format_sig(spec, u.id, "facebook")
assert credentials.resolve_cookiefile(db, u.id, FB_URL, salted, spec, tmp_path) is None
assert (
credentials.resolve_cookiefile(
db, u.id, FB_URL, self._asset_sig(salted), spec, tmp_path
)
is None
)
# --- Retry / Re-download take a cookie added later (DB) -----------------------------------------
# Measured on dev (E11.5): a Facebook reel failed on the login wall, the user added their cookie as
# the failure copy tells them to, pressed Retry — and it failed again, because the job still held the
# shared asset the worker never attaches a cookie to. Re-download forked a salt the gate could not
# recognise, so even a job downloaded WITH a cookie rebuilt without it.
def _held_job(db, u, *, sig, asset_status="error", job_status="error"):
a = MediaAsset(
source_kind="url", source_ref=FB_URL, format_sig=sig, status=asset_status, ref_count=1
)
db.add(a)
db.flush()
j = DownloadJob(
user_id=u.id,
asset_id=a.id,
source_kind="url",
source_ref=FB_URL,
status=job_status,
queue_pos=0,
profile_snapshot={},
)
db.add(j)
db.commit()
return a, j
class TestRequeueTakesTheCookie:
spec = formats.normalize({})
@pytest.mark.parametrize(
("asset_status", "job_status"),
# A failed download; and a job paused before it ever started (its asset never downloaded).
[("error", "error"), ("pending", "paused")],
)
def test_retry_after_adding_a_cookie_moves_the_job_to_the_private_asset(
self, db, tmp_path, asset_status, job_status
):
u = _user(db)
shared, j = _held_job(
db,
u,
sig=formats.format_sig(self.spec),
asset_status=asset_status,
job_status=job_status,
)
credentials.upsert_credential(db, u.id, "facebook", COOKIES)
db.commit()
service.resume(db, j)
db.expire_all()
job = db.get(DownloadJob, j.id)
target = db.get(MediaAsset, job.asset_id)
assert job.status == "queued" and target.id != shared.id
assert target.format_sig == credentials.authed_format_sig(self.spec, u.id, "facebook")
assert (target.status, target.ref_count) == ("pending", 1)
assert db.get(MediaAsset, shared.id).ref_count == 0 # the old hold went with the move
assert credentials.resolve_cookiefile(db, u.id, FB_URL, target, self.spec, tmp_path)
def test_retry_without_a_cookie_keeps_the_shared_asset(self, db):
u = _user(db)
shared, j = _held_job(db, u, sig=formats.format_sig(self.spec))
service.resume(db, j)
db.expire_all()
assert db.get(DownloadJob, j.id).asset_id == shared.id
assert db.get(MediaAsset, shared.id).ref_count == 1
@pytest.mark.parametrize(
("asset_status", "job_status"), [("ready", "error"), ("downloading", "paused")]
)
def test_retry_never_moves_off_a_file_or_a_download_in_flight(
self, db, asset_status, job_status
):
u = _user(db)
shared, j = _held_job(
db,
u,
sig=formats.format_sig(self.spec),
asset_status=asset_status,
job_status=job_status,
)
credentials.upsert_credential(db, u.id, "facebook", COOKIES)
db.commit()
service.resume(db, j)
db.expire_all()
assert db.get(DownloadJob, j.id).asset_id == shared.id
def test_redownload_with_a_cookie_rebuilds_into_an_asset_the_cookie_goes_with(
self, db, tmp_path
):
u = _user(db)
other = _user(db, "other@example.com")
credentials.upsert_credential(db, u.id, "facebook", COOKIES)
credentials.upsert_credential(db, other.id, "facebook", COOKIES)
authed = credentials.authed_format_sig(self.spec, u.id, "facebook")
first, j = _held_job(db, u, sig=authed, asset_status="ready", job_status="done")
seen = {first.id}
for _ in range(2): # a rebuild of a rebuild is still recognised
service.redownload(db, j)
db.expire_all()
job = db.get(DownloadJob, j.id)
target = db.get(MediaAsset, job.asset_id)
assert target.id not in seen and target.format_sig != authed
seen.add(target.id)
assert credentials.resolve_cookiefile(db, u.id, FB_URL, target, self.spec, tmp_path)
# Private: another user's cookie is never attached to it.
assert not credentials.is_authed_asset(target, self.spec, other.id, FB_URL)
# A Retry of a failed rebuild keeps the rebuild — it is already the cookie's identity.
target.status, job.status = "error", "error"
db.commit()
service.resume(db, job)
db.expire_all()
assert db.get(DownloadJob, j.id).asset_id == target.id
def test_a_rebuild_made_without_a_cookie_is_no_cookie_identity(self, db, tmp_path):
u = _user(db)
_, j = _held_job(db, u, sig=formats.format_sig(self.spec))
service.redownload(db, j)
credentials.upsert_credential(db, u.id, "facebook", COOKIES)
db.commit()
db.expire_all()
target = db.get(MediaAsset, db.get(DownloadJob, j.id).asset_id)
assert not credentials.is_authed_asset(target, self.spec, u.id, FB_URL)
# --- API: write-only, never echoes the blob (DB) ----------------------------------------------
+93
View File
@@ -1,5 +1,6 @@
"""Category classification of raw yt-dlp / ffmpeg download-failure strings (S2)."""
from app.downloads import adapters
from app.downloads import errors as dlerrors
from app.downloads.errors import classify
@@ -64,6 +65,98 @@ def test_our_own_worker_refusals_classify_as_themselves():
assert classify(dlerrors.QUOTA_FULL_MESSAGE) == "quota_exceeded"
def test_the_redacted_network_text_classifies_back_to_network():
# The redacted text is also what a failed asset keeps, and a second job that meets that asset is
# diagnosed from it — without its own rule it would read "Download failed".
assert classify(dlerrors.NETWORK_REDACTED_MESSAGE) == "network"
# Verbatim yt-dlp 2026.07.04 output from prod jobs, measured on dev 2026-09-13 (E11.5).
_FB_PARSE = (
"ERROR: [facebook] 1736360027626346: Cannot parse data; please report this issue on "
"https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template."
)
_FB_NO_FORMATS = (
"ERROR: [facebook] 1755988225525547: No video formats found!; please report this issue on "
"https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template."
)
_REEL = "https://www.facebook.com/reel/1736360027626346/?fs=e&fs=e"
_GROUP_POST = "https://www.facebook.com/groups/983554479435596/permalink/1755988225525547/?"
def test_facebook_cannot_parse_data_is_the_login_wall_without_a_cookie():
for url in (
_REEL,
_GROUP_POST,
"https://fb.watch/Ia_iJYI2Vw/?fs=e",
"https://m.facebook.com/x",
):
assert dlerrors.diagnose(_FB_PARSE, url=url) == ("login_required", "facebook.login_wall")
def test_facebook_cannot_parse_data_with_a_cookie_is_the_cookie_refused():
# Telling a user who already gave a cookie to add one is the failure this entry exists to avoid.
assert dlerrors.diagnose(_FB_PARSE, url=_REEL, cookie_attached=True) == (
"login_required",
"facebook.cookie_rejected",
)
def test_a_facebook_group_post_without_formats_names_the_link():
# Without a cookie yt-dlp is given the post link itself (measured, prod jobs 66 and 145).
assert dlerrors.diagnose(_FB_NO_FORMATS, url=_GROUP_POST) == (
"format_unavailable",
"facebook.group_post_video",
)
# With one it is only ever given the post's VIDEO link, so the text is about that video: the adapter
# found it, and "this post has no video" would be false.
assert dlerrors.diagnose(_FB_NO_FORMATS, url=_GROUP_POST, cookie_attached=True) == (
"format_unavailable",
None,
)
# A reel, or a group link that is not a post, is another failure: only the category.
for url in (_REEL, "https://www.facebook.com/groups/983554479435596/"):
assert dlerrors.diagnose(_FB_NO_FORMATS, url=url) == ("format_unavailable", None), url
def test_our_post_not_shown_verdict_is_the_refused_cookie():
# The adapter's text when a signed-in read of the post got a page without the post (the login page).
msg = adapters.POST_NOT_SHOWN_MESSAGE
assert classify(msg) == "login_required"
assert dlerrors.diagnose(msg, url=_GROUP_POST, cookie_attached=True) == (
"login_required",
"facebook.cookie_rejected",
)
def test_our_post_without_a_video_verdict_names_the_group_link():
# The adapter's own text when the post page holds no single video (S4): the message and its rule and
# catalog entry are pinned together, or an edit to the wording orphans the copy.
msg = adapters.POST_WITHOUT_VIDEO_MESSAGE
assert classify(msg) == "format_unavailable"
assert dlerrors.diagnose(msg, url=_GROUP_POST, cookie_attached=True) == (
"format_unavailable",
"facebook.group_post_video",
)
assert dlerrors.diagnose(msg, url=_REEL, cookie_attached=True) == ("format_unavailable", None)
def test_a_catalog_entry_never_leaves_its_host():
# "Cannot parse data" is yt-dlp's wording for a broken extractor on every site.
other = _FB_PARSE.replace("[facebook]", "[vimeo]")
assert dlerrors.diagnose(other, url="https://vimeo.com/1") == ("unknown", None)
# The Facebook text for a link that is not Facebook's (a look-alike host) is not the Facebook wall.
assert dlerrors.diagnose(_FB_PARSE, url="https://facebook.com.evil/reel/1") == ("unknown", None)
# No link known: the category only.
assert dlerrors.diagnose(_FB_PARSE) == ("unknown", None)
def test_diagnose_falls_back_to_the_category():
msg = "ERROR: [facebook] 123: You must be logged in to see this post"
assert dlerrors.diagnose(msg, url=_REEL) == ("login_required", None)
assert dlerrors.diagnose(None, url=_REEL) == ("unknown", None)
def test_a_url_jobs_network_failure_is_redacted_but_youtube_keeps_its_text():
# SEC-12: the raw transport text for a user-chosen host:port is a port-scanner readout, so it is
# replaced for `url` jobs. Everything else must survive verbatim, or a real diagnosis is lost.
@@ -0,0 +1,316 @@
"""DB-backed tests for the download failure catalog (E11.5 S1) at the places a failure is WRITTEN.
`test_download_errors` pins which entry a message matches; these pin that every writer hands the
matcher what it needs — the link and whether the user's cookie went with it — and that a stored entry
never outlives the failure it describes:
- the download child's failure branch (`pipeline._download`);
- a second job meeting an already-failed asset (`worker._process_job`), which only has the text;
- Retry / Re-download, which requeue the job;
- the `0084` backfill of the rows that failed before the catalog existed;
- the downloads list, which is how the entry reaches the UI.
It also holds the one success-path check of the same child: the duration it stores (E11.5 S3).
"""
import shutil
import subprocess
import pytest
from alembic import command
from alembic.config import Config
from cryptography.fernet import Fernet
from sqlalchemy import text
from app import security
from app.config import settings
from app.db import engine
from app.downloads import credentials, formats, pipeline, service
from app.models import DownloadJob, DownloadQuota, MediaAsset, User
from app.worker import _process_job
_REEL = "https://www.facebook.com/reel/1736360027626346/?fs=e&fs=e"
# Verbatim yt-dlp 2026.07.04, prod job 256.
_FB_PARSE = (
"ERROR: [facebook] 1736360027626346: Cannot parse data; please report this issue on "
"https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template."
)
_COOKIES = "# Netscape HTTP Cookie File\n.facebook.com\tTRUE\t/\tTRUE\t0\tc_user\t123\n"
@pytest.fixture
def _fernet(monkeypatch):
# A throwaway key, so a stored credential encrypts and decrypts (mirrors test_download_credentials).
monkeypatch.setattr(security, "_fernet", Fernet(Fernet.generate_key()))
def _user(db, email):
u = User(email=email)
db.add(u)
db.flush()
db.add(
DownloadQuota(
user_id=u.id, max_bytes=10**12, max_concurrent=5, max_jobs=10**6, unlimited=True
)
)
return u
def _asset_and_job(db, user, *, sig, asset_status, job_status, error=None):
a = MediaAsset(
source_kind="url",
source_ref=_REEL,
format_sig=sig,
status=asset_status,
error=error,
ref_count=1,
)
db.add(a)
db.flush()
j = DownloadJob(
user_id=user.id,
asset_id=a.id,
source_kind="url",
source_ref=_REEL,
status=job_status,
queue_pos=0,
profile_snapshot={},
)
db.add(j)
db.commit()
return a, j
def _fail_the_child(monkeypatch, tmp_path, *, cookie: bool):
monkeypatch.setattr(settings, "download_root", str(tmp_path))
monkeypatch.setattr(pipeline.urlguard, "check_url", lambda url: None)
cookiefile = tmp_path / "cookies.txt"
cookiefile.write_text("# Netscape HTTP Cookie File\n")
monkeypatch.setattr(
pipeline.credentials,
"resolve_cookiefile",
lambda *a, **k: cookiefile if cookie else None,
)
def extract(*a, **k):
raise RuntimeError(_FB_PARSE)
monkeypatch.setattr(pipeline, "_extract_with_fallback", extract)
def test_the_download_child_diagnoses_with_the_link_and_no_cookie(db, monkeypatch, tmp_path):
u = _user(db, "fb-nocookie@example.com")
a, j = _asset_and_job(db, u, sig="shared", asset_status="downloading", job_status="running")
_fail_the_child(monkeypatch, tmp_path, cookie=False)
pipeline._download(j.id, a.id, "url", _REEL, {}, u.id)
db.expire_all()
job = db.get(DownloadJob, j.id)
assert (job.status, job.error_code, job.error_detail) == (
"error",
"login_required",
"facebook.login_wall",
)
def test_the_download_child_knows_the_cookie_went_with_it(db, monkeypatch, tmp_path):
# The same text with the user's cookie attached: telling them to add cookies would be wrong.
u = _user(db, "fb-cookie@example.com")
a, j = _asset_and_job(db, u, sig="private", asset_status="downloading", job_status="running")
_fail_the_child(monkeypatch, tmp_path, cookie=True)
pipeline._download(j.id, a.id, "url", _REEL, {}, u.id)
db.expire_all()
job = db.get(DownloadJob, j.id)
assert (job.error_code, job.error_detail) == ("login_required", "facebook.cookie_rejected")
def test_a_job_meeting_a_failed_shared_asset_gets_the_same_explanation(db):
# Before the catalog this path wrote the text and NO code: "Download failed" for the second
# job, "Facebook sign-in required" for the first.
u = _user(db, "fb-second@example.com")
_, j = _asset_and_job(
db, u, sig="shared", asset_status="error", job_status="running", error=_FB_PARSE
)
_process_job(j.id)
db.expire_all()
job = db.get(DownloadJob, j.id)
assert (job.status, job.error_code, job.error_detail) == (
"error",
"login_required",
"facebook.login_wall",
)
def test_a_job_meeting_its_users_failed_private_asset_knows_the_cookie_was_used(db, _fernet):
u = _user(db, "fb-private@example.com")
credentials.upsert_credential(db, u.id, "facebook", _COOKIES)
sig = credentials.authed_format_sig(formats.normalize({}), u.id, "facebook")
_, j = _asset_and_job(
db, u, sig=sig, asset_status="error", job_status="running", error=_FB_PARSE
)
_process_job(j.id)
db.expire_all()
job = db.get(DownloadJob, j.id)
assert (job.error_code, job.error_detail) == ("login_required", "facebook.cookie_rejected")
def test_a_private_asset_the_cookie_could_not_go_with_is_explained_as_the_download_had_it(db):
# The cookie was deleted (or can no longer be decrypted) after the jobs were queued: the download
# child had nothing to attach and said "add your cookies". The job meeting that failed asset must say
# the same — the asset's salt alone would have told it "your cookies were refused".
u = _user(db, "fb-gone@example.com")
sig = credentials.authed_format_sig(formats.normalize({}), u.id, "facebook")
_, j = _asset_and_job(
db, u, sig=sig, asset_status="error", job_status="running", error=_FB_PARSE
)
_process_job(j.id)
db.expire_all()
job = db.get(DownloadJob, j.id)
assert (job.error_code, job.error_detail) == ("login_required", "facebook.login_wall")
def test_a_job_canceled_before_it_meets_a_failed_asset_stays_canceled(db):
# The cancel landed between the claim and this write. `error` would hold the asset again after the
# route already released it — a later delete releases twice.
u = _user(db, "fb-canceled@example.com")
_, j = _asset_and_job(
db, u, sig="shared", asset_status="error", job_status="canceled", error=_FB_PARSE
)
_process_job(j.id)
db.expire_all()
job = db.get(DownloadJob, j.id)
assert (job.status, job.error_code, job.error_detail) == ("canceled", None, None)
def test_the_downloads_list_carries_the_catalog_entry(db, as_user):
# The whole explanation rides on this one field: without it every row falls back to the category.
u = _user(db, "fb-list@example.com")
_, j = _asset_and_job(
db, u, sig="shared", asset_status="error", job_status="error", error=_FB_PARSE
)
j.error_code, j.error_detail = "login_required", "facebook.login_wall"
db.commit()
[row] = as_user(u).get("/api/downloads").json()
assert (row["error_code"], row["error_detail"]) == ("login_required", "facebook.login_wall")
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not available")
def test_a_download_yt_dlp_gave_no_duration_stores_the_files_duration(db, monkeypatch, tmp_path):
# Facebook reel 257 (E11.5): yt-dlp's info had no duration, the asset stored NULL, and the Library
# row lost its trim action. Through the real finalize, with a real 3-second file.
u = _user(db, "fb-duration@example.com")
a, j = _asset_and_job(db, u, sig="shared", asset_status="downloading", job_status="running")
monkeypatch.setattr(settings, "download_root", str(tmp_path))
monkeypatch.setattr(pipeline.urlguard, "check_url", lambda url: None)
def extract(job_id, asset_id, url, spec, staging, adapter, **kwargs):
media = staging / "1736360027626346.mp4"
subprocess.run(
[
"ffmpeg",
"-y",
"-loglevel",
"error",
"-f",
"lavfi",
"-i",
"testsrc=size=64x64:rate=10",
"-f",
"lavfi",
"-i",
"sine=frequency=440",
"-t",
"3",
"-c:v",
"libx264",
"-pix_fmt",
"yuv420p",
"-c:a",
"aac",
str(media),
],
check=True,
)
return {"id": "1736360027626346", "ext": "mp4", "title": "Reel", "filepath": str(media)}
monkeypatch.setattr(pipeline, "_extract_with_fallback", extract)
pipeline._download(j.id, a.id, "url", _REEL, {}, u.id)
db.expire_all()
assert db.get(DownloadJob, j.id).status == "done"
assert db.get(MediaAsset, a.id).duration_s == 3
def test_retry_and_redownload_drop_the_old_explanation(db):
u = _user(db, "fb-requeue@example.com")
for requeue in (service.resume, service.redownload):
_, j = _asset_and_job(
db, u, sig=f"s-{requeue.__name__}", asset_status="error", job_status="error"
)
j.error, j.error_code, j.error_detail = _FB_PARSE, "login_required", "facebook.login_wall"
db.commit()
requeue(db, j)
db.refresh(j)
assert (j.status, j.error_code, j.error_detail) == ("queued", None, None), requeue.__name__
def test_0084_backfills_only_the_facebook_parse_failures_it_measured(db):
cfg = Config("alembic.ini")
command.downgrade(cfg, "0083_comfort_pins")
upgraded = False
try:
with engine.begin() as conn:
uid = conn.execute(
text("INSERT INTO users (email) VALUES ('backfill@example.com') RETURNING id")
).scalar_one()
rows = {
"fb_unknown": ("error", "unknown", _FB_PARSE),
"fb_null": ("canceled", None, _FB_PARSE),
"other_site": ("error", "unknown", _FB_PARSE.replace("[facebook]", "[vimeo]")),
"fb_other_code": ("error", "network", _FB_PARSE),
}
ids = {}
for name, (status, code, error) in rows.items():
ids[name] = conn.execute(
text(
"INSERT INTO download_jobs (user_id, source_kind, source_ref, "
"profile_snapshot, status, error_code, error) "
"VALUES (:u, 'url', :ref, '{}', :s, :c, :e) RETURNING id"
),
{"u": uid, "ref": _REEL, "s": status, "c": code, "e": error},
).scalar_one()
command.upgrade(cfg, "head")
upgraded = True
with engine.connect() as conn:
got = {
name: tuple(
conn.execute(
text("SELECT error_code, error_detail FROM download_jobs WHERE id = :i"),
{"i": i},
).one()
)
for name, i in ids.items()
}
assert got == {
"fb_unknown": ("login_required", "facebook.login_wall"),
"fb_null": ("login_required", "facebook.login_wall"),
"other_site": ("unknown", None),
"fb_other_code": ("network", None),
}
finally:
if not upgraded:
command.upgrade(cfg, "head")
+139
View File
@@ -0,0 +1,139 @@
"""Overlapping requests on ONE job (E11.5 review) — DB-backed, two real sessions.
The routes that move a job's asset ref count (resume, cancel, delete, redownload) used to decide on an
unlocked read: two overlapping requests for one job (a double-click, a second tab, a replay after a
gateway timeout) both saw it holding its asset and both released the hold. The count then reaches 0
while another job still holds the asset, and GC or the ready-file delete removes it under that job.
A pause racing a cancel did the same one step later: its unlocked write turned the released job back
into a holder, which a later delete releases again. `_own_job_for_update` makes the second request
wait and decide on the first one's committed result.
"""
import threading
import pytest
from fastapi import HTTPException
from sqlalchemy import select
from app.db import SessionLocal
from app.downloads import service
from app.models import DownloadJob, MediaAsset, User
from app.routes.downloads import (
_own_job_for_update,
cancel_download,
delete_download,
pause_download,
)
_REF = "https://example.com/video"
def _asset_held_by_two_jobs(db, status="error"):
mine, theirs = User(email="lock-mine@example.com"), User(email="lock-theirs@example.com")
db.add_all([mine, theirs])
db.flush()
asset = MediaAsset(
source_kind="url", source_ref=_REF, format_sig="shared", status="error", ref_count=2
)
db.add(asset)
db.flush()
jobs = [
DownloadJob(
user_id=owner.id,
asset_id=asset.id,
source_kind="url",
source_ref=_REF,
status=status if owner is mine else "error",
queue_pos=0,
profile_snapshot={},
)
for owner in (mine, theirs)
]
db.add_all(jobs)
db.commit()
return mine, asset, jobs[0]
@pytest.mark.parametrize(
("status", "transition", "route", "left"),
[
("error", service.cancel, cancel_download, "canceled"),
("error", service.delete, delete_download, None),
("queued", service.cancel, pause_download, "canceled"),
],
)
def test_overlapping_requests_for_one_job_release_its_hold_once(
db, status, transition, route, left
):
user, asset, job = _asset_held_by_two_jobs(db, status)
job_id = job.id # read before the delete case removes the row under the expired instance
first, second = SessionLocal(), SessionLocal()
finished = threading.Event()
def the_other_request():
try:
route(job_id, user=user, db=second)
except HTTPException:
pass # a request after the delete: the job is gone, a 404
finally:
second.close()
finished.set()
try:
locked = _own_job_for_update(first, user, job_id)
other = threading.Thread(target=the_other_request)
other.start()
# The second request waits for the first one's locks instead of deciding on the same row.
assert not finished.wait(0.5), "the second request did not wait for the first"
transition(first, locked)
other.join(10)
assert finished.is_set()
finally:
first.close()
db.expire_all()
mine = select(DownloadJob.status).where(DownloadJob.id == job_id)
assert db.execute(mine).scalar_one_or_none() == left
assert db.get(MediaAsset, asset.id).ref_count == 1 # the other user's job still holds it
def test_losing_the_asset_insert_race_keeps_the_callers_own_writes(db, monkeypatch):
# Retry clears the job's error, THEN may create the cookie asset. When another request inserted that
# asset first, the loser used to roll the whole session back — and requeue the job with its stale
# error, which a stall or crash later leaves showing under the new failure.
user = User(email="race@example.com")
db.add(user)
db.flush()
job = DownloadJob(
user_id=user.id,
source_kind="url",
source_ref=_REF,
status="error",
error="the old failure",
queue_pos=0,
profile_snapshot={},
)
db.add(job)
db.commit()
job.error = None # the caller's pending write
real_flush, raced = db.flush, []
def flush_after_the_other_request_inserted(*args, **kwargs):
if not raced: # the other request commits the same asset between our lookup and our insert
raced.append(True)
with SessionLocal() as other:
other.add(
MediaAsset(source_kind="url", source_ref=_REF, format_sig="raced", ref_count=0)
)
other.commit()
return real_flush(*args, **kwargs)
monkeypatch.setattr(db, "flush", flush_after_the_other_request_inserted)
asset = service.get_or_create_asset(db, "url", _REF, "raced")
db.commit()
assert raced and asset.format_sig == "raced"
db.expire_all()
assert db.get(DownloadJob, job.id).error is None
@@ -0,0 +1,243 @@
"""E11.5 S3 + S4 — the Facebook download path with the user's cookie.
S3, measured on dev (yt-dlp 2026.07.04, the owner's cookie): reel 806379462330494 fails with the cookie
alone and extracts 2560p with `--impersonate chrome`; the reels that already download with the cookie
keep their plain first attempt. These pin the attempt order through the real download loop
(`pipeline._extract_with_fallback`, with yt-dlp faked at the `YoutubeDL` boundary), and that the
LOCKED curl_cffi can actually impersonate — the pair only works inside yt-dlp's own version range, and
outside it every target silently reports "unavailable".
S4, measured on dev and in the owner's Chrome: given a group post's link, yt-dlp finds no formats or
downloads another video from the page's feed. The post's own video is the single Video node in the
page scripts that carry the post id (prod posts 66, 145, 140, 143, 221; text posts have none while the
page holds ~10 other videos), and its watch link downloads.
"""
import pytest
import yt_dlp
from yt_dlp.networking.exceptions import TransportError
from yt_dlp.networking.impersonate import ImpersonateTarget
from app.downloads import adapters, pipeline
from app.downloads import errors as dlerrors
_URL = "https://www.facebook.com/reel/806379462330494/?fs=e&fs=e"
_POST = "1755988225525547"
_GROUP_LINK = f"https://www.facebook.com/groups/983554479435596/permalink/{_POST}/?"
_POST_VIDEO = "1408015447857753"
_FEED_VIDEO = "1056963063816966" # what yt-dlp downloaded from this post's link: a feed video
_CHROME = ImpersonateTarget.from_str("chrome")
def _video_node(video_id: str) -> str:
# The measured shape of a post's attachment in the page JSON.
return '{"media":{"__typename":"Video","id":"%s","__isNode":"Video"},"styles":{}}' % video_id
def _page(*post_videos: str, post: str = _POST) -> str:
attachments = ",".join(_video_node(v) for v in post_videos)
return (
'<html><script type="application/json" data-sjs>'
f'{{"post_id":"{post}","creation_time":1785084554,"attachments":[{attachments}]}}</script>'
'<script type="application/json" data-sjs>'
f'{{"feed":[{_video_node(_FEED_VIDEO)}]}}</script></html>'
)
_WALL = (
"ERROR: [facebook] 806379462330494: Cannot parse data; please report this issue on "
"https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template."
)
_NO_FORMATS = "ERROR: [facebook] 2541212983053999: No video formats found!"
def test_the_locked_curl_cffi_can_impersonate_chrome():
# YoutubeDL refuses to start when the target is unavailable — the same check a download makes.
with yt_dlp.YoutubeDL({"impersonate": ImpersonateTarget.from_str("chrome"), "quiet": True}):
pass
@pytest.fixture
def ydl(monkeypatch):
"""Fake `yt_dlp.YoutubeDL`: records each attempt's options and plays back `outcomes` in order;
`web["page"]` is what a page read returns (`web["fail"]`, when set, is raised instead), and `web`
records the pages opened, the byte cap each read asked for, and the links extracted."""
seen: list[dict] = []
outcomes: list = []
web: dict = {"page": "", "fail": None, "opened": [], "reads": [], "extracted": []}
class FakeYDL:
def __init__(self, opts):
seen.append(opts)
def __enter__(self):
return self
def __exit__(self, *exc):
return False
def urlopen(self, url):
web["opened"].append(url)
if web["fail"] is not None:
raise web["fail"]
body = web["page"].encode()
class Response:
def read(self, amt=None):
web["reads"].append(amt)
return body
return Response()
def extract_info(self, url, download):
web["extracted"].append(url)
outcome = outcomes.pop(0)
if isinstance(outcome, str):
raise yt_dlp.utils.DownloadError(outcome)
return outcome
monkeypatch.setattr(pipeline.yt_dlp, "YoutubeDL", FakeYDL)
monkeypatch.setattr(pipeline, "_touch_job", lambda job_id: None)
monkeypatch.setattr(pipeline, "_set_job", lambda job_id, **fields: None)
return seen, outcomes, web
def _run(tmp_path, cookiefile, url=_URL):
return pipeline._extract_with_fallback(
1, 1, url, {}, tmp_path, adapters.FacebookAdapter(), cookiefile=cookiefile
)
def test_a_cookie_download_at_the_wall_is_retried_as_chrome(ydl, tmp_path):
seen, outcomes, web = ydl
outcomes += [_WALL, {"id": "806379462330494"}]
assert _run(tmp_path, "/cookies.txt") == {"id": "806379462330494"}
assert [o.get("impersonate") for o in seen] == [None, _CHROME]
assert all(o["cookiefile"] == "/cookies.txt" for o in seen) # the retry stays signed in
assert web["opened"] == [] # a reel is not a group post: no page read
def test_without_a_cookie_the_wall_is_the_login_wall_and_final(ydl, tmp_path):
seen, outcomes, _ = ydl
outcomes += [_WALL]
with pytest.raises(yt_dlp.utils.DownloadError):
_run(tmp_path, None)
assert len(seen) == 1 and "impersonate" not in seen[0]
def test_other_facebook_failures_are_not_retried_as_chrome(ydl, tmp_path):
seen, outcomes, _ = ydl
outcomes += [_NO_FORMATS]
with pytest.raises(yt_dlp.utils.DownloadError):
_run(tmp_path, "/cookies.txt")
assert len(seen) == 1 and "impersonate" not in seen[0]
# --- S4: a group post downloads ITS video, or fails saying it has none ----------------------------
def test_a_cookie_group_post_downloads_the_posts_own_video(ydl, tmp_path):
seen, outcomes, web = ydl
web["page"] = _page(_POST_VIDEO)
outcomes += [_WALL, {"id": _POST_VIDEO}] # the watch link needs Chrome too (measured)
assert _run(tmp_path, "/cookies.txt", _GROUP_LINK) == {"id": _POST_VIDEO}
assert web["opened"] == [_GROUP_LINK]
# Never an unbounded read.
assert web["reads"] == [adapters.FacebookAdapter._POST_PAGE_MAX_BYTES]
assert seen[0]["cookiefile"] == "/cookies.txt" # the post page is read signed in
watch = f"https://www.facebook.com/watch/?v={_POST_VIDEO}"
# Never the post link: that is where the feed video comes from.
assert web["extracted"] == [watch, watch]
assert [o.get("impersonate") for o in seen[1:]] == [None, _CHROME]
def test_a_group_post_without_a_single_video_fails_saying_so(ydl, tmp_path):
_, _, web = ydl
web["page"] = _page() # a text post: the only video on the page is the feed's
with pytest.raises(RuntimeError) as exc:
_run(tmp_path, "/cookies.txt", _GROUP_LINK)
assert web["extracted"] == [] # yt-dlp is not left to guess from the post link
assert dlerrors.diagnose(str(exc.value), url=_GROUP_LINK, cookie_attached=True) == (
"format_unavailable",
"facebook.group_post_video",
)
def test_a_group_post_page_without_the_post_is_the_refused_cookie(ydl, tmp_path):
# Measured with a refused cookie (E11.5 review): Facebook answers the post link with its login page —
# a login form, no post id anywhere. Saying "this post has no video" would blame the post.
_, _, web = ydl
web["page"] = '<html><title>ChatGPT magyarul</title><form id="login_form"></form></html>'
with pytest.raises(RuntimeError) as exc:
_run(tmp_path, "/cookies.txt", _GROUP_LINK)
assert web["extracted"] == []
assert dlerrors.diagnose(str(exc.value), url=_GROUP_LINK, cookie_attached=True) == (
"login_required",
"facebook.cookie_rejected",
)
def test_a_failed_post_page_read_fails_the_download_as_itself(ydl, tmp_path):
# A read that failed says nothing about the post: it must not turn into "no video in this post".
_, _, web = ydl
web["fail"] = TransportError("Connection reset by peer")
with pytest.raises(TransportError) as exc:
_run(tmp_path, "/cookies.txt", _GROUP_LINK)
assert web["extracted"] == []
assert dlerrors.diagnose(str(exc.value), url=_GROUP_LINK, cookie_attached=True) == (
"network",
None,
)
def test_without_a_cookie_the_group_post_link_goes_to_ytdlp_unchanged(ydl, tmp_path):
_, outcomes, web = ydl
outcomes += [_NO_FORMATS]
with pytest.raises(yt_dlp.utils.DownloadError):
_run(tmp_path, None, _GROUP_LINK)
assert web["opened"] == [] # signed out the page is the login wall: nothing to read
assert web["extracted"] == [_GROUP_LINK]
def test_the_group_post_link_forms_are_recognised():
for url in (
_GROUP_LINK,
f"https://www.facebook.com/groups/983554479435596/posts/{_POST}/",
f"https://m.facebook.com/groups/983554479435596/permalink/{_POST}/?",
f"https://www.facebook.com/groups/chatgptmagyarul/posts/{_POST}",
):
assert adapters.facebook_group_post_id(url) == _POST, url
for url in (
_URL,
f"https://www.facebook.com/watch/?v={_POST_VIDEO}",
f"https://facebook.com.evil/groups/x/permalink/{_POST}/",
"https://www.facebook.com/groups/983554479435596/",
):
assert adapters.facebook_group_post_id(url) is None, url
assert adapters.SourceAdapter().resolve_url(_GROUP_LINK, {"cookiefile": "/c"}) == _GROUP_LINK
def test_only_the_single_video_of_the_posts_own_blocks_counts():
assert adapters.facebook_post_video_id(_page(_POST_VIDEO), _POST) == _POST_VIDEO
# No video in the post, although the page has one elsewhere.
assert adapters.facebook_post_video_id(_page(), _POST) is None
# Two videos: guessing between them would be the wrong-video bug again.
assert adapters.facebook_post_video_id(_page(_POST_VIDEO, "2063302677911049"), _POST) is None
# A longer number that merely STARTS with the post id is another object.
assert adapters.facebook_post_video_id(_page(_POST_VIDEO, post=_POST + "0"), _POST) is None
+204
View File
@@ -0,0 +1,204 @@
"""Idle HLS sessions are reaped on a timer, and a viewer who is merely PAUSED keeps theirs.
plex/stream.py promises "a periodic reaper kills idle ones", but `reap_idle` ran only inside
POST /session: after the last playback, an abandoned session (its ffmpeg still working through the
film, its directory a whole segmented copy of it) lived until somebody pressed play again
(deep-review round 4).
And it must not take a PAUSED viewer's session: a remux finishes long before the film does, and a
paused player with a full buffer fetches nothing, so "finished and unfetched for 30 s" does not mean
"abandoned". Round 4 bought that safety by keeping EVERY finished session for the full ten-minute
idle window — one segmented copy of a film per item, on the volume the download center shares. Round
5 replaced the blanket retention with the signal that already existed: an open player checkpoints its
position every 10 s whether or not it is playing, and that route says "still here".
Both call sites are pinned here, because the bug each time was that only one of them moved: the
periodic pass, and the fact that starting a session no longer sweeps OTHER viewers' sessions at all.
"""
import asyncio
import time
from types import SimpleNamespace
import pytest
from fastapi.testclient import TestClient
import app.main as app_main
import app.state as state
from app.auth import current_user
from app.db import get_db
from app.main import app
from app.models import PlexItem, PlexLibrary, User
from app.plex import paths as plex_paths
from app.plex import stream
class _Configured:
def __enter__(self):
return None
def __exit__(self, *exc):
return False
def test_the_periodic_pass_is_the_one_that_runs_and_it_uses_the_short_rule(monkeypatch):
async def _no_sweep():
return None
reaped: list[int] = []
monkeypatch.setattr(app_main, "SessionLocal", _Configured)
monkeypatch.setattr(app_main.state, "is_configured", lambda db: True)
monkeypatch.setattr(app_main, "_sweep_hls_orphans", _no_sweep)
monkeypatch.setattr(app_main, "start_scheduler", lambda: None)
monkeypatch.setattr(app_main, "shutdown_scheduler", lambda: None)
monkeypatch.setattr(app_main.plex_subtitles, "shutdown", lambda: None)
monkeypatch.setattr(app_main, "_HLS_REAP_EVERY_S", 0.01)
# `reap_idle`, i.e. the SHORT finished rule: a finished session nobody has said they still want
# is a whole copy of a film sitting on the download volume.
monkeypatch.setattr(app_main.plex_stream, "reap_idle", lambda: reaped.append(1) or 0)
async def _cycle():
async with app_main.lifespan(app):
await asyncio.sleep(0.2)
asyncio.run(_cycle())
assert len(reaped) >= 2, "no periodic reap ran while the app was up"
class _Proc:
def __init__(self, finished: bool):
self.finished = finished
self.killed = False
def poll(self):
return 0 if self.finished else None
def terminate(self):
self.killed = True
def wait(self, timeout=None):
return 0
class _Session:
def __init__(self, tmp_path, name: str, finished: bool, idle_s: float):
self.dir = tmp_path / name
self.dir.mkdir()
self.proc = _Proc(finished)
self.last_access = time.time() - idle_s
@pytest.fixture
def sessions(monkeypatch):
live: dict = {}
monkeypatch.setattr(stream, "_sessions", live)
return live
def test_the_periodic_pass_reaps_a_finished_session_nobody_has_claimed(tmp_path, sessions):
sessions["12345"] = _Session(tmp_path, "done", finished=True, idle_s=45)
assert stream.reap_idle() == 1
assert sessions == {}
def test_the_periodic_pass_still_reaps_what_nobody_has_touched_for_the_idle_limit(
tmp_path, sessions
):
sessions["1"] = _Session(
tmp_path, "gone_done", finished=True, idle_s=stream._SESSION_IDLE_S + 5
)
sessions["2"] = _Session(
tmp_path, "gone_live", finished=False, idle_s=stream._SESSION_IDLE_S + 5
)
assert stream.reap_idle() == 2
assert sessions == {}
def test_touching_a_session_keeps_it_through_the_short_rule(tmp_path, sessions):
s = _Session(tmp_path, "paused", finished=True, idle_s=90)
sessions["12345"] = s
stream.touch_session("12345")
assert stream.reap_idle() == 0
assert "12345" in sessions and s.dir.exists() and not s.proc.killed
# --- the two call sites, driven through the real routes (DB lane) -------------------------------
def _library_item(db, rating_key: str, *, playable: str = "remux"):
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
it = PlexItem(
rating_key=rating_key,
library_id=lib.id,
kind="movie",
title="A Movie",
file_path="/media/a.mkv",
playable=playable,
)
db.add(it)
db.commit()
return it
def _client(db, user):
app.dependency_overrides[get_db] = lambda: db
app.dependency_overrides[current_user] = lambda: user
return TestClient(app)
def test_a_paused_viewers_checkpoint_keeps_their_session_alive(db, monkeypatch, tmp_path, sessions):
"""The whole reason the short rule is safe again: a paused player fetches nothing, so
`last_access` never moved on its own — but it does keep checkpointing its position."""
monkeypatch.setattr(state, "_configured_cache", True)
it = _library_item(db, "paused1")
user = User(email="paused@example.com")
db.add(user)
db.commit()
s = _Session(tmp_path, "paused", finished=True, idle_s=90)
sessions["paused1"] = s
try:
resp = _client(db, user).post(
f"/api/plex/item/{it.rating_key}/progress",
json={"position_seconds": 600, "duration_seconds": 3000, "final": False},
)
finally:
app.dependency_overrides.clear()
assert resp.status_code == 200, resp.text
assert stream.reap_idle() == 0, "the checkpoint did not say the viewer is still there"
assert "paused1" in sessions and not s.proc.killed
def test_starting_a_session_leaves_another_viewers_paused_one_alone(
db, monkeypatch, tmp_path, sessions
):
"""The opportunistic sweep this route used to run was not scoped to the key being started: it
walked the WHOLE registry with the 30 s finished rule, so pressing play on any item killed a
paused viewer's session — the very case round 4 set out to protect (found in round 5)."""
monkeypatch.setattr(state, "_configured_cache", True)
monkeypatch.setattr(plex_paths, "local_media_path", lambda db_, fp: "/local/a.mkv")
started = _library_item(db, "starting")
user = User(email="starter@example.com")
db.add(user)
db.commit()
paused = _Session(tmp_path, "paused", finished=True, idle_s=90)
sessions["someone-else"] = paused
def fake_start(db_, item, start, audio=None, aoff=0.0, multi=False, transcode=False):
return SimpleNamespace(
id="sid",
media_start_s=start,
entry="index.m3u8",
transcode=transcode,
hw=False,
tonemap=False,
)
monkeypatch.setattr(stream, "start_session", fake_start)
try:
resp = _client(db, user).post(f"/api/plex/stream/{started.rating_key}/session")
finally:
app.dependency_overrides.clear()
assert resp.status_code == 200, resp.text
assert "someone-else" in sessions and not paused.proc.killed
+148
View File
@@ -0,0 +1,148 @@
"""A row that is gone answers with a reason the bilingual UI can translate (deep-review round 3).
The 404 a user actually meets is the one for something deleted in another tab or on another device.
`owned_or_404` / `get_or_404` sent it as a bare English string ("Unknown view", "Not found"), and the
Hungarian UI toasted it verbatim. The structured-404 fix had gone into one ad-hoc helper instead of
the shared one, so every other owner check kept the English. Needs the DB.
"""
from app.auth import require_write_scope
from app.models import DownloadJob, MediaAsset
def test_a_gone_view_tag_or_pin_says_what_is_gone(db, make_user, as_user):
client = as_user(make_user())
for method, url, what in [
("patch", "/api/saved-views/999999", "view"),
("delete", "/api/saved-views/999999", "view"),
("delete", "/api/tags/999999", "tag"),
("delete", "/api/discover/pins/999999", "pin"),
]:
r = getattr(client, method)(url, **({"json": {"name": "x"}} if method == "patch" else {}))
assert r.status_code == 404, (url, r.text)
assert r.json()["detail"] == {"code": "not_found", "reason": what}, url
def test_a_gone_channel_video_or_user_says_what_is_gone(db, make_user, as_user):
"""The siblings that still hand-wrote `db.get(...) is None → 404 "Unknown …"` (deep-review
round 4): the same gone row, but the Hungarian UI showed the English."""
admin = make_user(role="admin")
client = as_user(admin)
for method, url, body, what in [
("post", "/api/channels/UCgone000000000000000000/block", None, "channel"),
("post", "/api/videos/gone0000000/state", {"status": "watched"}, "video"),
(
"post",
"/api/videos/gone0000000/progress",
{"position_seconds": 60, "duration_seconds": 600},
"video",
),
("put", "/api/admin/downloads/quota/999999", {"max_bytes": 1}, "user"),
]:
r = getattr(client, method)(url, **({"json": body} if body is not None else {}))
assert r.status_code == 404, (url, r.text)
assert r.json()["detail"] == {"code": "not_found", "reason": what}, url
def test_every_gone_channel_route_says_it_the_same_way(db, make_user, as_user, override_dep):
"""The round-4 sweep converted ONE of the five `db.get(Channel, …) is None -> 404 "Unknown
channel"` sites (block), and shipped the HU/EN string with it — so the Hungarian UI said it in
Hungarian on the Block button and in English on the four beside it (deep-review round 5). The
asymmetric fix is the smell this repo bans, so all five are pinned together.
Literally all five: round 6 found this loop issuing three requests under a docstring claiming
five, with explore silently absent and subscribe excused as "covered by the tags route" — which
reaches `_user_subscription`, a different site answering `reason: subscription`. Reverting
either of those two to its bare English string left the whole lane green."""
admin = make_user(role="admin")
client = as_user(admin)
# Subscribe's own gate (the optional YouTube write scope) sits in FRONT of the lookup, so a
# plain account 403s before reaching it. The gate is not what is under test here.
override_dep(require_write_scope, lambda: admin)
gone = "UCgone000000000000000000"
for method, url in [
("get", f"/api/channels/{gone}"),
("post", f"/api/channels/{gone}/explore"),
("post", f"/api/channels/{gone}/reset-backfill"),
("post", f"/api/channels/{gone}/subscribe"),
("post", f"/api/channels/{gone}/block"),
]:
r = getattr(client, method)(url)
assert r.status_code == 404, (url, r.text)
assert r.json()["detail"] == {"code": "not_found", "reason": "channel"}, url
def test_a_gone_subscription_recipient_or_job_says_what_is_gone(db, make_user, as_user):
"""The three bare-English 404s the same sweep left behind, which `notifyActionError` printed
verbatim into an otherwise Hungarian UI: "Not subscribed to this channel", "No user with that
email.", "Unknown job".
All three, with a request each. Round 6 found this test naming three and issuing two: the
recipient site had no test in the whole suite (nor did the `errors.notFound.recipient` keys
added beside it), and `run_job_now` rode on its PATCH sibling's coverage."""
admin = make_user(role="admin")
client = as_user(admin)
r = client.post("/api/channels/UCgone000000000000000000/tags", json={"tag_id": 1})
assert r.status_code == 404, r.text
assert r.json()["detail"] == {"code": "not_found", "reason": "subscription"}
for method, url in [
("patch", "/api/admin/scheduler/jobs/no_such_job"),
("post", "/api/admin/scheduler/jobs/no_such_job/run"),
]:
kw = {"json": {"interval_minutes": 30}} if method == "patch" else {}
r = getattr(client, method)(url, **kw)
assert r.status_code == 404, (url, r.text)
assert r.json()["detail"] == {"code": "not_found", "reason": "job"}, url
# Sharing needs a finished download of one's own to share; the recipient lookup is the next check.
asset = MediaAsset(
source_kind="url", source_ref="share-404", format_sig="sig", status="ready", ref_count=1
)
db.add(asset)
db.flush()
job = DownloadJob(
user_id=admin.id,
asset_id=asset.id,
source_kind="url",
source_ref="share-404",
status="done",
queue_pos=1,
profile_snapshot={},
)
db.add(job)
db.commit()
r = client.post(f"/api/downloads/{job.id}/share", json={"email": "nobody@example.com"})
assert r.status_code == 404, r.text
assert r.json()["detail"] == {"code": "not_found", "reason": "recipient"}
def test_a_nul_in_a_request_string_is_a_422_not_a_500(db, make_user, as_user):
"""Postgres cannot hold NUL in text, so the driver refuses the query outright: a DataError, i.e.
a 500 with a traceback, wherever a `%00` or an escaped NUL reached a comparison (deep-review
round 4 — round 3 had guarded only the by-rating_key lookup). One app-wide handler now answers
all of them."""
client = as_user(make_user())
r = client.get("/api/plex/playlists?contains=%00")
assert r.status_code == 422, r.text
assert r.json()["detail"] == {"code": "invalid_input", "reason": "nul"}
assert client.get("/api/plex/playlists?contains_group=1,%00").status_code == 422
def test_any_other_data_error_is_still_a_500():
"""The other half of the app-wide handler, and the half a refactor can delete in silence: it is
registered for EVERY DataError on every route and decides on a substring of the driver's message,
so only a NUL may become a 422. Without this, simplifying it to an unconditional 422 — it reads
like dead defensive code — would answer a numeric overflow or an over-long ingested title with
"your input was bad", losing the traceback that names the real fault (deep-review round 5)."""
import asyncio
import pytest
from sqlalchemy.exc import DataError
from app.main import _nul_in_text
overflow = DataError("SELECT 1", {}, Exception("numeric field overflow"))
with pytest.raises(DataError):
asyncio.run(_nul_in_text(None, overflow)) # re-raised: it stays the 500 it always was
nul = DataError("SELECT 1", {}, Exception("invalid byte sequence: NUL (0x00)"))
assert asyncio.run(_nul_in_text(None, nul)).status_code == 422
+54
View File
@@ -128,6 +128,60 @@ def test_move_into_someone_elses_list_404s_and_changes_nothing(db, make_user, as
assert _ids(db, theirs.id) == []
def test_a_list_that_is_not_yours_answers_with_a_reason_the_ui_can_translate(
db, make_user, as_user
):
# The one 404 a user meets in practice (the list went away in another tab). It used to be a bare
# English "Unknown playlist", which the Hungarian UI then showed verbatim.
user, other = make_user(), make_user()
_seed_videos(db, ["v1"])
theirs = _playlist(db, other, "Theirs", ["v1"])
r = as_user(user).delete(f"/api/playlists/{theirs.id}/items/v1")
assert r.status_code == 404
assert r.json()["detail"] == {"code": "not_found", "reason": "playlist"}
db.expire_all()
assert _ids(db, theirs.id) == ["v1"]
def test_a_move_into_a_list_that_is_gone_names_the_target_not_the_list_on_screen(
db, make_user, as_user
):
# Both lists 404'd with the same reason, which the UI reads as "THIS playlist doesn't exist any
# more" — said about the source the reader is looking at, which is still there.
user = make_user()
_seed_videos(db, ["v1"])
src = _playlist(db, user, "Mine", ["v1"])
gone = _playlist(db, user, "Gone", [])
gone_id = gone.id
db.delete(gone)
db.commit()
r = _move(as_user(user), src.id, "v1", gone_id)
assert r.status_code == 404
assert r.json()["detail"] == {"code": "not_found", "reason": "move_target"}
db.expire_all()
assert _ids(db, src.id) == ["v1"]
def test_a_refused_playlist_action_answers_with_a_reason_the_ui_can_translate(
db, make_user, as_user
):
# The rest of the router's refusals were bare English too ("The Watch later list can't be
# deleted.", "YouTube delete failed.", ...), shown verbatim under the Hungarian UI.
user = make_user()
client = as_user(user)
wl = client.post("/api/playlists/watch-later", json={"video_id": "nope"})
assert wl.status_code == 404
assert wl.json()["detail"] == {"code": "not_found", "reason": "video"}
_seed_videos(db, ["v1"])
saved = client.post("/api/playlists/watch-later", json={"video_id": "v1"})
wl_id = saved.json()["playlist_id"]
r = client.delete(f"/api/playlists/{wl_id}")
assert r.status_code == 400
assert r.json()["detail"] == {"code": "playlist", "reason": "watch_later_undeletable"}
def test_move_marks_both_linked_sides_dirty(db, make_user, as_user):
"""A move edits two YouTube mirrors at once — the source lost an item, the target gained one.
Marking only the source is the asymmetric fix this repo keeps re-learning."""
@@ -0,0 +1,83 @@
""" "Clear resume point" has to reach Plex — the case the status-only guard skipped.
`POST /api/plex/item/{rk}/state {"status": "new"}` backs the *Clear resume point* / *Mark unwatched*
buttons. It deletes the local row and mirrors the un-mark to Plex — but it used to decide whether to
push from the PREVIOUS STATUS alone (`prev_status != "new"`), and an in-progress item's row is
`status="new"` carrying a position: that is how the importer maps a bare `viewOffset`
(`_plex_watch_to_state`). So for the one case the button exists for, nothing was ever sent to Plex,
Plex kept the offset, and the next pull imported it straight back — the user cleared *Protector*
twice and it returned to Continue watching at 92s both times (2026-09-09).
One `unscrobble` clears the view count AND the offset (measured against the live server), so the
push itself needed no change; what needed fixing is WHEN it is scheduled.
DB lane (`db` fixture) — skipped when no Postgres is reachable.
"""
from types import SimpleNamespace
from fastapi.testclient import TestClient
import app.state as state
from app.auth import current_user
from app.db import get_db
from app.main import app
from app.models import PlexItem, PlexLibrary, PlexState, User
from app.plex import watch_sync as plex_watch
def _seed(db, *, status: str, position: int):
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
it = PlexItem(
rating_key="98614", library_id=lib.id, kind="movie", title="Protector", duration_s=5495
)
db.add(it)
user = User(email="clear@example.com")
db.add(user)
db.flush()
db.add(PlexState(user_id=user.id, item_id=it.id, status=status, position_seconds=position))
db.commit()
return it, user
def _clear(db, user, it, monkeypatch) -> list[tuple]:
"""Drive the endpoint with the Plex link satisfied and the background push recorded."""
pushes: list[tuple] = []
monkeypatch.setattr(state, "_configured_cache", True) # pass the instance-setup request gate
monkeypatch.setattr(plex_watch, "link_for_push", lambda db_, uid: SimpleNamespace(id=1))
monkeypatch.setattr(
plex_watch,
"push_state_to_plex",
lambda user_id, item_id, rating_key, action, *rest: pushes.append((rating_key, action)),
)
app.dependency_overrides[get_db] = lambda: db
app.dependency_overrides[current_user] = lambda: user
try:
resp = TestClient(app).post(f"/api/plex/item/{it.rating_key}/state", json={"status": "new"})
assert resp.status_code == 200, resp.text
finally:
app.dependency_overrides.clear()
return pushes
def test_clearing_an_in_progress_item_tells_plex(db, monkeypatch):
# The regression: status "new" + a position IS the in-progress shape, and it must push.
it, user = _seed(db, status="new", position=92)
assert _clear(db, user, it, monkeypatch) == [("98614", "unwatched")]
assert db.query(PlexState).count() == 0
def test_clearing_a_watched_item_still_tells_plex(db, monkeypatch):
# The case the old guard did cover — kept, so the fix cannot narrow into a new hole.
it, user = _seed(db, status="watched", position=0)
assert _clear(db, user, it, monkeypatch) == [("98614", "unwatched")]
def test_unhiding_touches_plex_not_at_all(db, monkeypatch):
# `hidden` is a Siftlode-only concept "with no Plex analogue, so it never touches Plex" — the
# endpoint's own docstring. The old status-only guard pushed here anyway, un-marking an item on
# Plex that the user had merely hidden.
it, user = _seed(db, status="hidden", position=0)
assert _clear(db, user, it, monkeypatch) == []
+291
View File
@@ -0,0 +1,291 @@
"""The Collection info page's member ordering (`GET /api/plex/collections/{rk}?sort=&sort_dir=`) and
the `keys=` lookup that gives the sidebar token / filter chip a title instead of a raw rating_key.
Reported from daily use: "Collectionben nem működik a rendezés". The browse grid sorts fine — the
collection page had no control at all and a hardcoded year-ascending order. Adding the control means
the SAME vocabulary must actually work here, hence the pin against `PLEX_SORT_KEYS`.
Pure-logic tests run everywhere; the endpoint tests use the `db` fixture (DB lane).
"""
from datetime import date, datetime, timedelta, timezone
from fastapi.testclient import TestClient
from sqlalchemy import literal_column
import app.state as state
from app.auth import current_user
from app.db import get_db
from app.main import app
from app.models import PlexCollection, PlexItem, PlexLibrary, PlexShow, PlexState, User
from app.plex.query import PLEX_SORT_KEYS
from app.routes.plex.browse import sort_column
from app.routes.plex.collections import (
DEFAULT_MEMBER_SORT,
MEMBER_SORT_KEYS,
Member,
sort_members,
)
NOW = datetime(2024, 6, 1, tzinfo=timezone.utc)
def _m(title: str, **over) -> Member:
base = dict(
kind="movie",
obj=None,
title=title,
year=None,
rating=None,
added=None,
release=None,
duration=None,
last_watched=None,
)
base.update(over)
return Member(**base) # type: ignore[arg-type]
def _titles(members: list[Member]) -> list[str]:
return [m.title for m in members]
# --- the vocabulary is ONE vocabulary ---------------------------------------------------------
def test_every_sort_key_is_orderable_on_a_collection():
"""The collection page shows the same Sort control as the browse grid, so a key it cannot order
would be an option that silently does nothing."""
assert set(MEMBER_SORT_KEYS) == set(PLEX_SORT_KEYS)
def test_every_sort_key_is_orderable_on_the_browse_grid():
"""The other half of the same pin: browse falls back to `added_at` for an unknown key, so a key
missing from its map would order by date while the control claimed otherwise."""
class _Cols:
title = literal_column("title")
year = literal_column("year")
rating = literal_column("rating")
duration_s = literal_column("duration_s")
rel = literal_column("rel")
last_watched = literal_column("last_watched")
added_at = literal_column("added_at")
class _U:
c = _Cols()
u = _U()
fallback = u.c.added_at
unhandled = [k for k in PLEX_SORT_KEYS if k != "added" and sort_column(u, k) is fallback]
assert unhandled == []
# …and the fallback itself is still what an unknown key gets.
assert sort_column(u, "no-such-key") is fallback
# --- ordering behaviour -----------------------------------------------------------------------
def test_orders_by_the_requested_key_in_both_directions():
members = [_m("b", year=2006), _m("c", year=2017), _m("a", year=2003)]
assert _titles(sort_members(members, "year", "asc")) == ["a", "b", "c"]
assert _titles(sort_members(members, "year", "desc")) == ["c", "b", "a"]
def test_members_without_a_value_sink_to_the_end_in_both_directions():
"""A movie with no rating is not "the worst rated" — nullslast, like the browse grid."""
members = [_m("rated-low", rating=6.5), _m("unrated"), _m("rated-high", rating=8.1)]
assert _titles(sort_members(members, "rating", "desc")) == [
"rated-high",
"rated-low",
"unrated",
]
assert _titles(sort_members(members, "rating", "asc")) == ["rated-low", "rated-high", "unrated"]
def test_ties_stay_title_ascending_when_the_direction_flips():
"""Flipping the direction must reverse the RANKING, not the alphabet inside a tie."""
members = [_m("zulu", year=2003), _m("alpha", year=2003), _m("mike", year=2017)]
assert _titles(sort_members(members, "year", "asc")) == ["alpha", "zulu", "mike"]
assert _titles(sort_members(members, "year", "desc")) == ["mike", "alpha", "zulu"]
def test_an_unknown_key_falls_back_to_the_pages_default():
members = [_m("b", year=2006), _m("a", year=2017)]
assert _titles(sort_members(members, "no-such-key", "asc")) == _titles(
sort_members(members, DEFAULT_MEMBER_SORT, "asc")
)
def test_a_show_has_no_duration_and_therefore_sinks_on_that_key():
members = [
_m("short movie", duration=3600),
_m("a show", kind="show", duration=None),
_m("long movie", duration=9000),
]
assert _titles(sort_members(members, "duration", "desc")) == [
"long movie",
"short movie",
"a show",
]
# --- the endpoint (DB lane) -------------------------------------------------------------------
def _seed(db):
lib_m = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
lib_s = PlexLibrary(plex_key="2", title="Shows", kind="show", enabled=True)
db.add_all([lib_m, lib_s])
db.flush()
col = PlexCollection(rating_key="col1", library_id=lib_m.id, title="Franchise", child_count=3)
other = PlexCollection(rating_key="col2", library_id=lib_m.id, title="Elsewhere", child_count=1)
db.add_all([col, other])
db.flush()
# Titles deliberately alphabetical-opposite to the years, so a year sort can't accidentally pass
# by falling through to the title tiebreak.
old = PlexItem(
rating_key="mo_old",
library_id=lib_m.id,
kind="movie",
title="Zulu",
year=2003,
rating=8.1,
duration_s=8000,
added_at=NOW - timedelta(days=1),
originally_available_at=date(2003, 7, 9),
collection_keys=["col1"],
)
mid = PlexItem(
rating_key="mo_mid",
library_id=lib_m.id,
kind="movie",
title="Mike",
year=2006,
rating=7.4,
duration_s=9000,
added_at=NOW - timedelta(days=3),
originally_available_at=date(2006, 7, 7),
collection_keys=["col1"],
)
new = PlexItem(
rating_key="mo_new",
library_id=lib_m.id,
kind="movie",
title="Alpha",
year=2017,
rating=6.5,
duration_s=7000,
added_at=NOW,
originally_available_at=date(2017, 5, 26),
collection_keys=["col1"],
)
# A show member with a resumed episode: `lastwatched` must work on this leg too, not just movies.
show = PlexShow(
rating_key="sh1",
library_id=lib_s.id,
title="Series",
year=2010,
child_count=1,
collection_keys=["col1"],
)
db.add_all([old, mid, new, show])
db.flush()
ep = PlexItem(
rating_key="sh1_e1",
library_id=lib_s.id,
kind="episode",
title="E1",
show_id=show.id,
season_number=1,
episode_number=1,
)
db.add(ep)
db.flush()
user = User(email="colsort@example.com")
db.add(user)
db.flush()
db.add_all(
[
# The show's episode is the most recently touched thing in the collection…
PlexState(
user_id=user.id,
item_id=ep.id,
status="new",
position_seconds=60,
progress_updated_at=NOW,
),
# …and one movie was watched earlier.
PlexState(
user_id=user.id,
item_id=old.id,
status="watched",
watched_at=NOW - timedelta(days=2),
),
]
)
db.commit()
return user
def _get(db, user, monkeypatch, path):
monkeypatch.setattr(state, "_configured_cache", True)
app.dependency_overrides[get_db] = lambda: db
app.dependency_overrides[current_user] = lambda: user
try:
return TestClient(app).get(path)
finally:
app.dependency_overrides.clear()
def _order(r) -> list[str]:
return [i["title"] for i in r.json()["items"]]
def test_collection_detail_defaults_to_release_year_ascending(db, monkeypatch):
"""The order the page had before it grew a control — a franchise reads in release order."""
user = _seed(db)
r = _get(db, user, monkeypatch, "/api/plex/collections/col1")
assert r.status_code == 200
assert _order(r) == ["Zulu", "Mike", "Series", "Alpha"]
def test_collection_detail_honours_sort_and_direction(db, monkeypatch):
user = _seed(db)
r = _get(db, user, monkeypatch, "/api/plex/collections/col1?sort=year&sort_dir=desc")
assert _order(r) == ["Alpha", "Series", "Mike", "Zulu"]
r = _get(db, user, monkeypatch, "/api/plex/collections/col1?sort=title&sort_dir=asc")
assert _order(r) == ["Alpha", "Mike", "Series", "Zulu"]
r = _get(db, user, monkeypatch, "/api/plex/collections/col1?sort=rating&sort_dir=desc")
# The show has no rating, so it sinks — it is not "rated 0".
assert _order(r) == ["Zulu", "Mike", "Alpha", "Series"]
def test_lastwatched_ranks_a_shows_episodes_alongside_the_movies(db, monkeypatch):
"""The show leg needs the rollup's `resumed_at`; with only a status map this key would order
every show as never-watched."""
user = _seed(db)
r = _get(db, user, monkeypatch, "/api/plex/collections/col1?sort=lastwatched&sort_dir=desc")
# Series (episode resumed NOW) > Zulu (watched 2 days ago) > the two never-touched, title-asc.
assert _order(r) == ["Series", "Zulu", "Alpha", "Mike"]
def test_collections_list_resolves_titles_by_key(db, monkeypatch):
"""What a shared Plex link needs: the URL carries bare rating_keys, so the sidebar token and the
filter chip have nothing to show but `17045` until these are resolved."""
user = _seed(db)
r = _get(db, user, monkeypatch, "/api/plex/collections?keys=col1")
assert r.status_code == 200
assert [c["title"] for c in r.json()["collections"]] == ["Franchise"]
def test_an_empty_keys_param_asks_for_nothing(db, monkeypatch):
"""`keys=` with nothing in it must not fall through to "every collection" — that would be a
200 KB answer to a question about zero ids."""
user = _seed(db)
r = _get(db, user, monkeypatch, "/api/plex/collections?keys=")
assert r.status_code == 200
assert r.json()["collections"] == []
@@ -155,6 +155,7 @@ def test_create_collection_rejects_duplicate_name(db, monkeypatch):
json={"library": "1", "title": "marvel", "item_rating_key": "m1"},
)
assert r.status_code == 409, r.text
assert r.json()["detail"] == {"code": "collection", "reason": "name_taken"}
finally:
app.dependency_overrides.clear()
+322
View File
@@ -0,0 +1,322 @@
"""An HLS artifact request names the session it belongs to, and gets nothing else.
Sessions are keyed by rating_key ALONE, while a session's directory is per (key, offset, tag). So a
seek-restart — or a second viewer of the same item — replaces the registered session, and the old
viewer's next request for `seg_12.ts`, resolved by rating_key, was answered 200 with the NEW
session's twelfth segment: bytes produced from a different `-ss` offset under `-copyts`, i.e. a
timeline the client is not on. Wrong content with a success status is the one failure a player cannot
notice, and the reclaim-on-404 added in round 4 could never fire on it (deep-review round 5).
No `db` fixture: the route reads nothing from the database (its own `Depends(get_db)` went with the
removed auto-start, and `current_user` is overridden wholesale), so these must not sit in the DB
lane — there they SKIP on a machine with no Postgres while pytest prints green (round 6).
"""
import time
import pytest
from fastapi.testclient import TestClient
import app.state as state
from app.auth import current_user
from app.main import app
from app.models import User
from app.plex import stream as plex_stream
class _Proc:
def __init__(self, running: bool = False):
self.running = running
def poll(self):
return None if self.running else 0
def terminate(self):
pass
def wait(self, timeout=None):
return 0
class _Session:
def __init__(self, session_id: str, directory, running: bool = False, audio_langs=None):
self.id = session_id
self.dir = directory
self.proc = _Proc(running)
self.last_access = time.time()
# The real session derives both of these from the argv it spawned; the guard reads
# `variants` to tell a name this session can produce from one only the OTHER argv branch
# writes, so a stand-in without it would test a shape that cannot exist.
self.variants = plex_stream.session_variants(audio_langs)
self.entry = "master.m3u8" if self.variants else "index.m3u8"
@pytest.fixture
def sessions(monkeypatch):
live: dict = {}
monkeypatch.setattr(plex_stream, "_sessions", live)
return live
@pytest.fixture
def client(monkeypatch):
monkeypatch.setattr(state, "_configured_cache", True)
user = User(email="hls@example.com")
app.dependency_overrides[current_user] = lambda: user
try:
yield TestClient(app)
finally:
app.dependency_overrides.pop(current_user, None)
def test_a_segment_is_served_from_the_session_the_url_names(client, tmp_path, sessions):
d = tmp_path / "m1_0_0_+0.00"
d.mkdir()
(d / "seg_11.ts").write_bytes(b"") # its predecessor, so seg_12 is the next one
(d / "seg_12.ts").write_bytes(b"mine")
sessions["m1"] = _Session("sid-a", d)
r = client.get("/api/plex/stream/m1/hls/sid-a/seg_12.ts")
assert r.status_code == 200, r.text
assert r.content == b"mine"
def test_a_segment_of_a_replaced_session_is_a_404_not_the_new_sessions_bytes(
client, tmp_path, sessions
):
"""Viewer A is at 0:00; viewer B seeks to 25:00 and their session takes the registry slot. A's
next segment must fail honestly — the player reclaims at the playhead — not arrive as B's."""
theirs = tmp_path / "m1_1500_0_+0.00"
theirs.mkdir()
(theirs / "seg_11.ts").write_bytes(b"")
(theirs / "seg_12.ts").write_bytes(b"theirs")
sessions["m1"] = _Session("sid-b", theirs)
r = client.get("/api/plex/stream/m1/hls/sid-a/seg_12.ts")
assert r.status_code == 404, r.text
assert b"theirs" not in r.content
def test_a_playlist_with_no_live_session_is_a_404_not_a_silent_restart_at_zero(
client, monkeypatch, sessions
):
"""The entry playlist used to auto-start a session when none was registered — always at 0.0,
silently throwing away the viewer's position. A 404 sends the player through its own reclaim,
which restarts at the playhead."""
started: list[float] = []
monkeypatch.setattr(
plex_stream,
"start_session",
lambda *a, **k: started.append(a[2] if len(a) > 2 else 0.0),
)
r = client.get("/api/plex/stream/m1/hls/sid-gone/index.m3u8")
assert r.status_code == 404, r.text
assert started == []
def test_a_segment_that_will_never_arrive_fails_at_once(client, tmp_path, sessions):
"""`stream_hls_file` is a SYNC route: it runs on the ~40-slot threadpool and holds a pooled DB
connection while it waits. It used to poll out the whole 20 s for a name that could never appear
— an ordinary seek past the produced region was enough to park a worker and a connection, and
forty hopeless names parked the API (deep-review round 5). ffmpeg writes segments in order, so a
segment whose predecessor is absent is not the next one, and that is knowable at once — even
while ffmpeg is still running, which is the case `proc` alone cannot shorten."""
d = tmp_path / "m1_0_0_+0.00"
d.mkdir()
(d / "seg_0.ts").write_bytes(b"x") # the session is producing; seg_99999 is simply not next
sessions["m1"] = _Session("sid-a", d, running=True)
started = time.monotonic()
r = client.get("/api/plex/stream/m1/hls/sid-a/seg_99999.ts")
elapsed = time.monotonic() - started
assert r.status_code == 404, r.text
# The wait is 20 s; anything in that neighbourhood means the producer's exit was not consulted.
assert elapsed < 2.0, f"waited {elapsed:.1f}s for a segment that can never arrive"
@pytest.mark.parametrize(
"name",
[
"seg_0_9000.ts", # the VIDEO rendition of a multi-audio session: variant 0, segment 9000
"seg_a0_9000.ts", # the first audio rendition: `name:a0` from -var_stream_map
],
)
def test_a_multi_audio_segment_beyond_the_region_fails_at_once_too(
client, tmp_path, sessions, name
):
"""The round-5 guard read `seg_<seq>_<variant>.ts`, but ffmpeg substitutes `%v` FIRST — so the
real names are `seg_<variant>_<seq>.ts` and every multi-audio request fell through to the full
20 s wait: the video rendition because its variant "0" parsed as segment 0, the audio renditions
(`a0`, `a1`) because they did not parse at all. Dual-audio is the common shape in this library,
so the guard was inert exactly where it was needed (round 6)."""
d = tmp_path / "m1_0_multi_+0.00"
d.mkdir()
(d / "seg_0_0.ts").write_bytes(b"x")
(d / "seg_a0_0.ts").write_bytes(b"x")
sessions["m1"] = _Session("sid-a", d, running=True, audio_langs=["hu", "en"])
started = time.monotonic()
r = client.get(f"/api/plex/stream/m1/hls/sid-a/{name}")
elapsed = time.monotonic() - started
assert r.status_code == 404, r.text
assert elapsed < 2.0, f"waited {elapsed:.1f}s for {name}"
def test_a_multi_audio_segment_whose_predecessor_exists_is_served(client, tmp_path, sessions):
"""The other half of the same rule: reading the names correctly must not start refusing real
ones. `seg_a0_1.ts` follows `seg_a0_0.ts` — the previous SEGMENT of that rendition, not the
previous variant."""
d = tmp_path / "m1_0_multi_+0.00"
d.mkdir()
(d / "seg_a0_0.ts").write_bytes(b"x")
(d / "seg_a0_1.ts").write_bytes(b"audio")
sessions["m1"] = _Session("sid-a", d, running=True, audio_langs=["hu", "en"])
r = client.get("/api/plex/stream/m1/hls/sid-a/seg_a0_1.ts")
assert r.status_code == 200, r.text
assert r.content == b"audio"
def test_a_name_the_session_cannot_produce_fails_at_once(client, tmp_path, sessions):
"""The guard was a blocklist: anything that was not a single-audio segment name was waved
through to the timeout, so `GET …/anything.m3u8` still parked a worker for the full 30 s — and
the route comment claimed a flood of hopeless names was what it prevented. A session produces
its playlists and its segments and nothing else, so any other name is answerable now."""
d = tmp_path / "m1_0_0_+0.00"
d.mkdir()
sessions["m1"] = _Session("sid-a", d, running=True)
started = time.monotonic()
r = client.get("/api/plex/stream/m1/hls/sid-a/nope.m3u8")
elapsed = time.monotonic() - started
assert r.status_code == 404, r.text
assert elapsed < 2.0, f"waited {elapsed:.1f}s for a playlist this session never writes"
@pytest.mark.parametrize("digits", [100, 5000])
def test_an_absurd_segment_index_is_a_404_not_a_500(client, tmp_path, sessions, digits):
"""`int()` on an unbounded digit run raises ValueError past 4300 digits, and only OSError was
caught — so a long enough name reached the client as a 500 with a traceback. Same bug class the
subtitle routes were bounded for; the answer is the same 404 any other unproducible name gets."""
d = tmp_path / "m1_0_0_+0.00"
d.mkdir()
sessions["m1"] = _Session("sid-a", d, running=True)
r = client.get(f"/api/plex/stream/m1/hls/sid-a/seg_{'9' * digits}.ts")
assert r.status_code == 404, r.text
def test_a_session_torn_down_during_the_read_is_a_404_not_a_500(
client, monkeypatch, tmp_path, sessions
):
"""`_kill` rmtrees the whole directory and nothing holds `_lock` across the wait and the read,
so the file can vanish in between — on every seek-restart, since `loadSession` awaits the new
session before destroying the old hls instance. The player's recovery is keyed on the 404
STATUS, so a 500 here skipped the reclaim and ended playback with "couldn't play"."""
d = tmp_path / "m1_0_0_+0.00"
d.mkdir()
(d / "seg_11.ts").write_bytes(b"")
sessions["m1"] = _Session("sid-a", d)
# The file existed when the wait checked, and is gone by the time the route reads it.
monkeypatch.setattr(plex_stream, "wait_for", lambda *a, **k: True)
r = client.get("/api/plex/stream/m1/hls/sid-a/seg_12.ts")
assert r.status_code == 404, r.text
def test_a_dead_producer_is_not_waited_out(client, tmp_path, sessions):
"""The route passes `proc` to `wait_for`. Without it a session whose ffmpeg has already exited
— a finished remux, a VAAPI init refusal — polls the whole 20 s for a segment that will never
be written, holding a threadpool worker and its pooled DB connection. The predecessor IS on
disk here, so `artifact_may_arrive` says yes and only `proc` can end the wait."""
d = tmp_path / "m1_0_0_+0.00"
d.mkdir()
(d / "seg_5.ts").write_bytes(b"x")
sessions["m1"] = _Session("sid-a", d, running=False)
started = time.monotonic()
r = client.get("/api/plex/stream/m1/hls/sid-a/seg_6.ts")
elapsed = time.monotonic() - started
assert r.status_code == 404, r.text
assert elapsed < 2.0, f"waited {elapsed:.1f}s on a producer that had already exited"
@pytest.mark.parametrize("audio_langs", [None, ["hu", "en"]])
def test_the_name_ffmpeg_is_asked_for_is_a_name_the_guard_understands(tmp_path, audio_langs):
"""One vocabulary, enforced. `segment_name` is what the warm-up and the PTS probe wait for, what
the argv's own `-hls_segment_filename` pattern is built from, and what `artifact_may_arrive`
parses back — they were three separate statements of the layout and two of them were wrong. Pin
them to each other instead."""
sess = _Session("sid", tmp_path, audio_langs=audio_langs)
variant = sess.variants[0] if sess.variants else None
name = plex_stream.segment_name(3, variant)
prev = plex_stream.segment_name(2, variant)
assert not plex_stream.artifact_may_arrive(sess, name)
(tmp_path / prev).write_bytes(b"x")
assert plex_stream.artifact_may_arrive(sess, name)
def test_the_argv_asks_for_exactly_the_names_the_guard_accepts(tmp_path):
"""The producer and the parser, pinned to each other through the argv itself.
Round 6's HIGH was a guard that read `seg_%v_%d.ts` sequence-first, so it was inert on every
multi-audio item, and a warm-up that waited 8 s for a name ffmpeg never writes. The fix was to
give the layout one spelling — but a comment saying so is what had just failed, so assert that
the `-hls_segment_filename` pattern in the real argv is literally what `segment_name` produces,
and that every playlist the argv names is one `playlist_names` accepts."""
for audio_langs in (None, ["hu", "en", "de"]):
args = plex_stream._ffmpeg_cmd(
tmp_path / "in.mkv", 0.0, tmp_path, 0, 0.0, audio_langs, loudnorm=False
)
variants = plex_stream.session_variants(audio_langs)
sess = _Session("sid", tmp_path, audio_langs=audio_langs)
# The two patterns in the argv are the vocabulary's own spelling, `%d`/`%v` and all.
seg_pattern = args[args.index("-hls_segment_filename") + 1]
pl_pattern = args[-1].rsplit("/", 1)[-1].rsplit("\\", 1)[-1]
if variants:
assert seg_pattern.endswith(plex_stream.segment_name("%d", "%v")), seg_pattern
assert pl_pattern == plex_stream.variant_playlist_name("%v"), pl_pattern
assert args[args.index("-master_pl_name") + 1] == plex_stream._MASTER_PL
else:
assert seg_pattern.endswith(plex_stream.segment_name("%d")), seg_pattern
assert pl_pattern == plex_stream._INDEX_PL, pl_pattern
assert "-master_pl_name" not in args
# And every name those patterns can actually expand to is one the guard lets through — the
# entry playlist hls.js is handed included, since a session whose own manifest is refused
# cannot start at all.
for v in variants or (None,):
assert plex_stream.artifact_may_arrive(sess, plex_stream.segment_name(0, v))
if v is not None:
assert plex_stream.artifact_may_arrive(sess, plex_stream.variant_playlist_name(v))
assert plex_stream.artifact_may_arrive(sess, sess.entry)
def test_a_name_only_the_other_argv_branch_writes_is_refused_at_once(tmp_path, client, sessions):
"""The guard is keyed to THIS session's vocabulary, not to HLS in general.
The two argv branches are disjoint: a single-audio session is given `seg_%d.ts` + `index.m3u8`
and no `%v` at all, so `master.m3u8`, `stream_*.m3u8` and every `seg_<variant>_<n>.ts` are names
it can never write — and vice versa. Accepting the union read them as "an HLS name, so wait and
see", and each one then held a threadpool worker and its pooled DB connection for the full
timeout: measured through this route against a live single-audio session, `master.m3u8` took
30.0 s and `seg_a0_0.ts` 20.1 s to answer 404 (deep-review round 7)."""
single = tmp_path / "m1_0_0_+0.00"
single.mkdir()
sessions["m1"] = _Session("sid-a", single, running=True)
multi = tmp_path / "m2_0_multi_+0.00"
multi.mkdir()
sessions["m2"] = _Session("sid-b", multi, running=True, audio_langs=["hu", "en"])
# Each of these belongs to the other shape (or to no shape at all) and must be refused without
# a wait — the assertion is the ELAPSED time as much as the status.
for key, sid, names in (
("m1", "sid-a", ["master.m3u8", "stream_0.m3u8", "seg_0_0.ts", "seg_zzzz_0.ts"]),
("m2", "sid-b", ["index.m3u8", "seg_0.ts", "seg_a9_0.ts", "stream_zz.m3u8"]),
):
for name in names:
started = time.monotonic()
r = client.get(f"/api/plex/stream/{key}/hls/{sid}/{name}")
elapsed = time.monotonic() - started
assert r.status_code == 404, f"{name}: {r.text}"
assert elapsed < 2.0, (
f"{name}: waited {elapsed:.1f}s for a name this session cannot write"
)
# And the control: the names each session CAN write are still waited for, not refused.
assert plex_stream.artifact_may_arrive(sessions["m1"], "index.m3u8")
assert plex_stream.artifact_may_arrive(sessions["m1"], "seg_0.ts")
assert plex_stream.artifact_may_arrive(sessions["m2"], "master.m3u8")
assert plex_stream.artifact_may_arrive(sessions["m2"], "stream_a1.m3u8")
assert plex_stream.artifact_may_arrive(sessions["m2"], "seg_a1_0.ts")
+3 -3
View File
@@ -12,7 +12,7 @@ import app.plex.sync as syncmod
from app import sysconfig
from app.models import PlexItem, PlexLibrary
from app.plex.sync import _apply_item, _canonical_guid, _sync_overlay
from app.routes.plex._shared import _content_libs
from app.plex.query import content_libs
class _StubPlex:
@@ -155,7 +155,7 @@ def test_sync_designates_overlay_and_writes_hu(db, monkeypatch):
assert m.title_hu == "Öldöklés"
assert m.genres_hu == ["Akció"]
# The HU section is not browsable and mirrored no rows of its own.
movie_ids, _ = _content_libs(db)
movie_ids, _ = content_libs(db)
assert en.id in movie_ids and hu.id not in movie_ids
assert db.scalar(select(func.count(PlexItem.id)).where(PlexItem.library_id == hu.id)) == 0
assert stats["overlaid"] == 1
@@ -240,7 +240,7 @@ def test_content_libs_excludes_overlay_libraries(db):
db.add(overlay)
db.commit()
movie_ids, show_ids = _content_libs(db)
movie_ids, show_ids = content_libs(db)
assert en_lib.id in movie_ids
assert overlay.id not in movie_ids # the HU overlay is never browsable
assert show_ids == [en_shows.id]
+3 -3
View File
@@ -1,6 +1,6 @@
"""DB-backed regression (E5 S4): GET /api/plex/person/{name} returns a cast/crew member's
filmography — movies + shows in the ENABLED libraries whose cast OR directors include the name,
each carrying the caller's watch-state, plus the cached headshot. Also covers `_content_libs`
each carrying the caller's watch-state, plus the cached headshot. Also covers `content_libs`
scoping (disabled libraries excluded) and the kind filter (episodes are not movies).
Uses the DB lane (`db` fixture) — skipped when no Postgres is reachable.
@@ -20,7 +20,7 @@ from app.models import (
PlexState,
User,
)
from app.routes.plex._shared import _content_libs
from app.plex.query import content_libs
NAME = "Robert De Niro"
@@ -151,5 +151,5 @@ def test_content_libs_partitions_only_enabled(db):
]
)
db.commit()
movie_ids, show_ids = _content_libs(db)
movie_ids, show_ids = content_libs(db)
assert len(movie_ids) == 1 and len(show_ids) == 1 # the disabled movie lib is excluded
+6 -1
View File
@@ -58,7 +58,12 @@ def _post(db, user, monkeypatch, rating_key, *, captured):
def fake_start(db_, it, start, audio, aoff, multi, transcode=False):
captured["transcode"] = transcode
return SimpleNamespace(
media_start_s=start, entry="index.m3u8", transcode=transcode, hw=False, tonemap=False
id="sid",
media_start_s=start,
entry="index.m3u8",
transcode=transcode,
hw=False,
tonemap=False,
)
monkeypatch.setattr(plex_stream, "start_session", fake_start)
+77
View File
@@ -483,6 +483,7 @@ class TestColdStartWarmup:
src.write_bytes(b"x")
monkeypatch.setattr(stream.paths, "local_media_path", lambda db, fp: src)
monkeypatch.setattr(stream.sysconfig, "get_int", lambda db, k: 2)
monkeypatch.setattr(stream.sysconfig, "get_bool", lambda db, k: False)
monkeypatch.setattr(stream.subprocess, "Popen", lambda *a, **k: _FakeProc())
return tmp_path
@@ -554,6 +555,12 @@ class TestTranscodeStartSession:
"get_str",
lambda db, k: {"plex_transcode_preset": "slow"}.get(k, "auto"),
)
# Loudness normalisation (item 14) is off unless a test turns it on via `stream._loudnorm_on`.
monkeypatch.setattr(
stream.sysconfig,
"get_bool",
lambda db, k: {"plex_audio_loudnorm": getattr(stream, "_loudnorm_on", False)}[k],
)
monkeypatch.setattr(stream, "wait_for", lambda path, timeout=20.0, proc=None: True)
_CapturingProc.calls = []
monkeypatch.setattr(stream.subprocess, "Popen", _CapturingProc)
@@ -868,3 +875,73 @@ class TestWaitForProcFastFail:
seg = tmp_path / "seg_0.ts"
seg.write_bytes(b"")
assert stream.wait_for(seg, timeout=30.0, proc=self._DeadProc()) is False
class TestAudioLoudnorm:
"""`plex_audio_loudnorm` (bug hunt 2026-09, item 14): the HLS path always downmixes 5.1 to stereo
AAC and never normalised it, so a cinema mix arrives with whispered dialogue and deafening action.
The knob is OFF by default — it makes a film ~11 dB louder and narrows its dynamic range, which is
a taste call. Measured on a real 5.1 AC3 film: I -27.6 -> -16.6 LUFS, LRA 13.3 -> 9.8 LU.
Reuses TestTranscodeStartSession's capturing fixture: what matters is the argv ffmpeg is actually
spawned with, not `_ffmpeg_cmd` called with literals."""
capturing_env = TestTranscodeStartSession.capturing_env
@staticmethod
def _spawn(monkeypatch, *, on: bool, multi: bool = False):
monkeypatch.setattr(stream, "_loudnorm_on", on, raising=False)
monkeypatch.setattr(
stream, "_select_encode", lambda db, item, transcode, src: (False, False)
)
if multi:
monkeypatch.setattr(stream, "_probe_audio_langs", lambda src: ["hun", "eng"])
s = stream.start_session(db=None, item=_FakeItem(), start_s=0.0, multi=multi)
assert s is not None
(cmd, *_), _ = _CapturingProc.calls[0]
return s, cmd
def test_the_knob_off_leaves_the_audio_exactly_as_it_was(self, capturing_env, monkeypatch):
_, cmd = self._spawn(monkeypatch, on=False)
assert "-af" not in cmd
assert cmd[cmd.index("-c:a") + 1] == "aac" and cmd[cmd.index("-ac") + 1] == "2"
def test_the_knob_on_normalises_AFTER_the_downmix(self, capturing_env, monkeypatch):
# Order is the whole point: measuring 5.1 and folding it down afterwards would normalise
# something other than what the viewer hears.
_, cmd = self._spawn(monkeypatch, on=True)
af = cmd[cmd.index("-af") + 1]
stages = af.split(",")
assert stages[0] == "aformat=channel_layouts=stereo"
assert stages[1].startswith("loudnorm=")
# loudnorm runs its own 192 kHz resampler internally; the AAC encoder needs 48 kHz back.
assert stages[-1] == "aresample=48000"
def test_every_audio_rendition_of_a_multi_audio_file_is_normalised(
self, capturing_env, monkeypatch
):
# A multi-audio session emits one AAC output per language. `-af` without a stream index
# covers all of them — a per-stream filter would have normalised only the first.
_, cmd = self._spawn(monkeypatch, on=True, multi=True)
assert cmd.count("-map") == 3 # video + two audio tracks
assert cmd.count("-af") == 1
assert "loudnorm=" in cmd[cmd.index("-af") + 1]
def test_the_session_directory_records_the_setting(self, capturing_env, monkeypatch):
# The knob changes the AUDIO THE SEGMENTS CONTAIN, so it belongs in the directory name with
# the other parameters that do. Otherwise a session left from before the flip is reused.
off, _ = self._spawn(monkeypatch, on=False)
_CapturingProc.calls = []
on, _ = self._spawn(monkeypatch, on=True)
assert off.dir.name != on.dir.name
assert on.dir.name.endswith("-ln_+0.00")
@pytest.mark.parametrize("on", [False, True])
def test_the_orphan_sweep_still_recognises_its_own_directory(
self, capturing_env, monkeypatch, on
):
# The tag is appended with a DASH, not an underscore, precisely so `_SESSION_DIR_RE` (the only
# thing keeping the sweep off a shared scratch path) still matches. A session directory the
# sweep no longer recognises is one that never gets cleaned up.
s, _ = self._spawn(monkeypatch, on=on)
assert _SESSION_DIR_RE.match(s.dir.name)
+45
View File
@@ -0,0 +1,45 @@
"""The subtitle routes must refuse a session the rest of the app refuses.
They used to sit behind `_require_session`, the cookie-only gate the poster proxies use — a signed
cookie was enough, with none of `current_user`'s checks (is_active, is_suspended, the session epoch
that "log out everywhere" bumps). Fine for a poster; not for a route that starts reads of the media
library. Found by the 2026-09 bug-hunt deep-review; the gate itself pre-dates that round.
"""
from fastapi.testclient import TestClient
import app.auth as auth
import app.state as app_state
from app.db import get_db
from app.main import app
from app.plex import subtitles
from app.routes.plex import playback
class _NoPool:
def __init__(self) -> None:
self.jobs: list = []
def submit(self, fn, *args) -> None:
self.jobs.append((fn, args))
def test_a_suspended_account_cannot_drive_the_subtitle_extractor(
db, make_user, override_dep, monkeypatch, tmp_path
):
user = make_user(is_suspended=True)
override_dep(get_db, lambda: db)
monkeypatch.setattr(app_state, "_configured_cache", True)
# A validly SIGNED cookie for that account — which is all the old gate looked at.
for mod in (auth, playback):
monkeypatch.setattr(mod, "resolved_user_id", lambda request: (user.id, True))
monkeypatch.setattr(subtitles, "CACHE_DIR", tmp_path)
pool = _NoPool()
monkeypatch.setattr(subtitles, "_WIN_POOL", pool)
monkeypatch.setattr(subtitles, "_FULL_POOL", pool)
subtitles.reset_state()
client = TestClient(app)
assert client.get("/api/plex/subtitle/98614/0/status?at=600").status_code == 401
assert client.get("/api/plex/subtitle/98614/0").status_code == 401
assert pool.jobs == [], "a refused session still scheduled an extraction"
File diff suppressed because it is too large Load Diff
+63
View File
@@ -0,0 +1,63 @@
"""A progress checkpoint takes a FINITE position.
The JSON parser under FastAPI accepts `Infinity` and `NaN`, and both progress handlers truncate the
position with `int()` — so one hand-written checkpoint came back as a 500 with a traceback. Found in
the 2026-09 bug hunt's second review round, as the sibling of the subtitle routes' non-finite
`at`/`offset`. Pure: the body is refused before the handler touches the database.
"""
from types import SimpleNamespace
import pytest
from fastapi.testclient import TestClient
import app.state as app_state
from app.auth import current_user
from app.main import app
@pytest.fixture
def client(monkeypatch, override_dep) -> TestClient:
override_dep(current_user, lambda: SimpleNamespace(id=1))
monkeypatch.setattr(app_state, "_configured_cache", True) # pass the instance-setup gate, no DB
return TestClient(app)
@pytest.mark.parametrize("path", ["/api/videos/v1/progress", "/api/plex/item/98614/progress"])
@pytest.mark.parametrize("raw", ["Infinity", "-Infinity", "NaN"])
def test_a_non_finite_position_is_refused(client, path, raw):
r = client.post(
path,
content=f'{{"position_seconds": {raw}, "duration_seconds": 3000}}',
headers={"content-type": "application/json"},
)
assert r.status_code == 422, r.text
@pytest.mark.parametrize("path", ["/api/videos/v1/progress", "/api/plex/item/98614/progress"])
@pytest.mark.parametrize("field", ["position_seconds", "duration_seconds"])
def test_a_position_past_the_int4_column_is_refused_not_a_500(client, path, field):
"""Finite but past the int4 column: the flush was a DataError, i.e. a 500 with a traceback."""
body = {"position_seconds": 10, "duration_seconds": 3000, field: 2**31}
assert client.post(path, json=body).status_code == 422
def test_the_refusal_keeps_the_shape_the_ui_reads(client):
"""The handler replaces FastAPI's own for every validation error in the app, and the UI turns
exactly this shape — a LIST of errors — into its localized "check your input" sentence. The
refused value is echoed by its name, because JSON cannot carry NaN."""
r = client.post(
"/api/videos/v1/progress",
content='{"position_seconds": NaN, "duration_seconds": 3000}',
headers={"content-type": "application/json"},
)
detail = r.json()["detail"]
assert isinstance(detail, list) and detail[0]["loc"] == ["body", "position_seconds"]
assert detail[0]["input"] == "nan"
@pytest.mark.parametrize("q", ["start=inf", "start=nan", "aoff=inf", "aoff=-inf", "aoff=nan"])
def test_a_session_at_an_impossible_offset_is_refused(client, q):
"""`start=inf` reached `int()` in the session builder (a 500); `aoff=inf` spawned an ffmpeg
with a non-finite offset."""
assert client.post(f"/api/plex/stream/98614/session?{q}").status_code == 422
+34 -6
View File
@@ -424,10 +424,36 @@ class TestResolveMediaFacts:
],
},
)
v, a, w, h = probemod.resolve_media_facts(Path("x"), {"width": None, "height": None})
v, a, w, h, _d = probemod.resolve_media_facts(Path("x"), {"width": None, "height": None})
assert (v, a, w, h) == ("h264", "aac", 720, 1280)
assert mediakind.derive(v, a) == mediakind.VIDEO # NOT 'file'
def test_the_duration_comes_from_the_file_when_info_has_none(self, monkeypatch):
# Facebook reel 257 (E11.5): yt-dlp gave no duration, the file is 128.4 s. A NULL duration hid
# the Library row's trim action.
self._ffprobe(
monkeypatch,
{
"format": {"format_name": "mov,mp4,m4a", "duration": "128.400000"},
"streams": [{"codec_type": "video", "codec_name": "h264"}],
},
)
assert probemod.resolve_media_facts(Path("x"), {"duration": None})[4] == 128
# The file wins over a differing info duration, like its codecs do.
assert probemod.resolve_media_facts(Path("x"), {"duration": 300})[4] == 128
def test_info_fills_the_duration_the_probe_could_not_read(self, monkeypatch):
self._ffprobe(
monkeypatch,
{
"format": {"format_name": "mov,mp4,m4a"},
"streams": [{"codec_type": "video", "codec_name": "h264"}],
},
)
assert probemod.resolve_media_facts(Path("x"), {"duration": 61.7})[4] == 61
self._ffprobe(monkeypatch, None)
assert probemod.resolve_media_facts(Path("x"), {"duration": 61.7})[4] == 61
def test_probe_wins_over_a_DIFFERING_populated_info(self, monkeypatch):
# Both the probe AND info are populated but DISAGREE (info carries a pre-remux/pre-merge codec).
# The finished file on disk is ground truth, so the probe wins outright — pins the precedence a
@@ -442,7 +468,7 @@ class TestResolveMediaFacts:
],
},
)
v, a, w, h = probemod.resolve_media_facts(
v, a, w, h, _d = probemod.resolve_media_facts(
Path("x"), {"vcodec": "vp9", "acodec": "opus", "width": 640, "height": 480}
)
assert (v, a, w, h) == ("h264", "aac", 1920, 1080) # probe, not info
@@ -466,7 +492,7 @@ class TestResolveMediaFacts:
],
},
)
v, a, w, h = probemod.resolve_media_facts(
v, a, w, h, _d = probemod.resolve_media_facts(
Path("x"), {"vcodec": "avc1", "acodec": "mp4a", "width": 1280, "height": 720}
)
assert v is None and a == "mp3" # info's avc1 did NOT win
@@ -482,7 +508,7 @@ class TestResolveMediaFacts:
"streams": [{"codec_type": "video", "codec_name": "png", "width": 8, "height": 8}],
},
)
v, a, w, h = probemod.resolve_media_facts(
v, a, w, h, _d = probemod.resolve_media_facts(
Path("x"), {"vcodec": "png", "width": 8, "height": 8}
)
assert v is None and mediakind.derive(v, a) == mediakind.FILE
@@ -490,7 +516,7 @@ class TestResolveMediaFacts:
def test_info_fills_in_only_when_ffprobe_could_not_read_the_file(self, monkeypatch):
# ffprobe failed entirely (None) → fall back to info so we never regress the working path.
self._ffprobe(monkeypatch, None)
v, a, w, h = probemod.resolve_media_facts(
v, a, w, h, _d = probemod.resolve_media_facts(
Path("x"), {"vcodec": "vp9", "acodec": "opus", "width": 1920, "height": 1080}
)
assert (v, a, w, h) == ("vp9", "opus", 1920, 1080)
@@ -498,6 +524,8 @@ class TestResolveMediaFacts:
def test_info_none_codec_string_is_treated_as_absent(self, monkeypatch):
# yt-dlp writes the literal "none" for a missing stream — it must not become a codec name.
self._ffprobe(monkeypatch, None) # probe failed → info path, where the "none" scrub applies
v, a, w, h = probemod.resolve_media_facts(Path("x"), {"vcodec": "none", "acodec": "aac"})
v, a, w, h, _d = probemod.resolve_media_facts(
Path("x"), {"vcodec": "none", "acodec": "aac"}
)
assert (v, a) == (None, "aac")
assert mediakind.derive(v, a) == mediakind.AUDIO
+14 -15
View File
@@ -51,7 +51,7 @@ def rec(monkeypatch):
monkeypatch.setattr(worker, "_reset_asset_pending", record("reset_asset"))
monkeypatch.setattr(worker, "_requeue_if_running", record("requeue"))
monkeypatch.setattr(worker, "_set_job", record("set_job"))
monkeypatch.setattr(worker, "_fail_running", record("fail"))
monkeypatch.setattr(worker, "_cleanup_staging", record("cleanup"))
monkeypatch.setattr(worker, "_stop_child", record("stop_child"))
monkeypatch.setattr(worker._stop, "is_set", lambda: False)
@@ -84,9 +84,9 @@ def test_no_progress_stall_kills_and_fails_the_job(monkeypatch, rec):
worker._run_supervised(1, 99)
names = _names(rec)
assert "stop_child" in names and "reset_asset" in names
set_job = next(c for c in rec if c[0] == "set_job")
assert set_job[2]["status"] == "error" and set_job[2]["error_code"] == "network"
assert "stalled" in set_job[2]["error"].lower()
fail = next(c for c in rec if c[0] == "fail")
assert fail[2]["error_code"] == "network"
assert "stalled" in fail[2]["error"].lower()
def test_route_pause_kills_but_keeps_the_route_status(monkeypatch, rec):
@@ -97,7 +97,7 @@ def test_route_pause_kills_but_keeps_the_route_status(monkeypatch, rec):
worker._run_supervised(1, 99)
names = _names(rec)
assert "stop_child" in names and "reset_asset" in names
assert "set_job" not in names and "requeue" not in names
assert "fail" not in names and "requeue" not in names
def test_route_cancel_kills_but_keeps_the_route_status(monkeypatch, rec):
@@ -110,7 +110,7 @@ def test_route_cancel_kills_but_keeps_the_route_status(monkeypatch, rec):
worker._run_supervised(1, 99)
names = _names(rec)
assert "stop_child" in names and "reset_asset" in names
assert "set_job" not in names and "requeue" not in names
assert "fail" not in names and "requeue" not in names
def test_wall_clock_cap_kills_even_while_progressing(monkeypatch, rec):
@@ -118,9 +118,8 @@ def test_wall_clock_cap_kills_even_while_progressing(monkeypatch, rec):
_spawn(monkeypatch, _FakeProc(["timeout"]))
_liveness(monkeypatch, ("running", 1.0)) # fresh progress, but over the absolute cap
worker._run_supervised(1, 99)
set_job = next(c for c in rec if c[0] == "set_job")
assert set_job[2]["status"] == "error"
assert "time limit" in set_job[2]["error"].lower()
fail = next(c for c in rec if c[0] == "fail")
assert "time limit" in fail[2]["error"].lower()
def test_child_crash_without_finishing_is_failed(monkeypatch, rec):
@@ -130,8 +129,8 @@ def test_child_crash_without_finishing_is_failed(monkeypatch, rec):
worker._run_supervised(1, 99)
names = _names(rec)
assert "reset_asset" in names
set_job = next(c for c in rec if c[0] == "set_job")
assert set_job[2]["status"] == "error" and "unexpectedly" in set_job[2]["error"].lower()
fail = next(c for c in rec if c[0] == "fail")
assert "unexpectedly" in fail[2]["error"].lower()
def test_child_nonzero_exit_that_already_set_state_is_left_alone(monkeypatch, rec):
@@ -139,7 +138,7 @@ def test_child_nonzero_exit_that_already_set_state_is_left_alone(monkeypatch, re
_spawn(monkeypatch, _FakeProc([("exit", 1)]))
_liveness(monkeypatch, ("error", None))
worker._run_supervised(1, 99)
assert "set_job" not in _names(rec)
assert "fail" not in _names(rec)
def test_spawn_failure_frees_the_asset_and_requeues(monkeypatch, rec):
@@ -158,7 +157,7 @@ def test_shutdown_kills_and_requeues_for_next_boot(monkeypatch, rec):
_liveness(monkeypatch, ("running", 1.0))
worker._run_supervised(1, 99)
names = _names(rec)
assert "stop_child" in names and "requeue" in names and "set_job" not in names
assert "stop_child" in names and "requeue" in names and "fail" not in names
def test_a_long_download_survives_while_progress_stays_fresh(monkeypatch, rec):
@@ -191,7 +190,7 @@ def test_a_wedged_finished_child_is_killed_by_the_wall_clock(monkeypatch, rec):
_liveness(monkeypatch, ("done", 1.0)) # finished, but the process lingers
worker._run_supervised(1, 99)
names = _names(rec)
assert "stop_child" in names and "set_job" not in names # killed to free the slot, status kept
assert "stop_child" in names and "fail" not in names # killed to free the slot, status kept
def test_a_watchdog_error_still_stops_the_child(monkeypatch, rec):
@@ -206,7 +205,7 @@ def test_a_watchdog_error_still_stops_the_child(monkeypatch, rec):
worker._run_supervised(1, 99)
names = _names(rec)
assert "stop_child" in names and "reset_asset" in names
assert next(c for c in rec if c[0] == "set_job")[2]["status"] == "error"
assert "fail" in _names(rec)
# --- _stop_child escalation -----------------------------------------------------------------
+22
View File
@@ -109,5 +109,27 @@ export default tseslint.config(
],
},
},
{
// E11 S3: the same one-way rule one layer down. `lib/` is what components are built ON (hooks,
// the API slices, pure helpers), so it may not reach back UP into `components/` — not even for a
// type. That is how the `Column` contract came to be owned by a ui component that three lib
// helpers had to import from. A piece both layers need moves down into `lib/`.
files: ["src/lib/**/*.{ts,tsx}"],
rules: {
"no-restricted-imports": [
"error",
{
patterns: [
{
// Any depth (`lib/api/*` is one level deeper), and `(/|$)` so a bare barrel counts too.
regex: String.raw`^(\.\./)+components(/|$)`,
message:
"lib/ sits below components/: it may not import from components/, not even a type. Move the shared piece down into lib/.",
},
],
},
],
},
},
prettier,
);
+25 -7
View File
@@ -3,6 +3,7 @@ import { qk } from "./lib/queryKeys";
import { useTranslation } from "react-i18next";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { api, getActiveAccount, setActiveAccount, setUnauthorizedHandler } from "./lib/api";
import { notifyActionError } from "./lib/actionErrors";
import { setLanguage, isSupported, type LangCode } from "./i18n";
import { clearParams } from "./lib/urlState";
import type { Page } from "./lib/urlState";
@@ -22,7 +23,7 @@ import { useNavigation, useNavigationActions } from "./components/NavigationProv
import { useFeedFilters, useFeedFiltersActions } from "./components/FeedFiltersProvider";
import { useChannelLayout } from "./components/ChannelsProvider";
import { useFeedView, useFeedViewActions } from "./components/FeedViewProvider";
import { moduleDef } from "./lib/modules";
import { moduleDef, moduleOrder } from "./lib/modules";
import { PAGE_CONTENT, PAGE_RAIL } from "./components/moduleRegistry";
// Persistent shell — always mounted, so eagerly imported.
import Welcome from "./components/Welcome";
@@ -97,7 +98,9 @@ export default function App() {
function setPanelLayout(panel: PanelId, next: PanelLayout) {
setPanelLayouts((prev) => ({ ...prev, [panel]: next }));
saveLayoutLocal(panel, next);
api.savePrefs({ [PREF_KEY[panel]]: next }).catch(() => {});
api
.savePrefs({ [PREF_KEY[panel]]: next })
.catch((e: unknown) => notifyActionError(e, t("errors.writeFailed.prefs")));
}
// Collapse state for the two full-height panels (left nav + filter sidebar). Persisted to the
@@ -137,10 +140,20 @@ export default function App() {
refetchInterval: (query) => (query.state.data ? 60_000 : false),
});
// The `global` key scope: `h` opens the cheat sheet, Mod+K the jump-to palette. Bound only once
// signed in, because that is where both overlays are mounted (bottom of this tree) — see
// useGlobalKeymap on why binding it above the signed-out branch is worse than not binding it.
useGlobalKeymap(!!meQuery.data);
// The `global` key scope: `h` opens the cheat sheet, Mod+K the jump-to palette, Shift+W jumps to
// Home ("mit nézzek?"). Bound only once signed in, because that is where the overlays are mounted
// (bottom of this tree) and where the modules exist at all — see useGlobalKeymap on why binding it
// above the signed-out branch is worse than not binding it.
// A global key can fire while an overlay is open; `setPage` closes every overlay before it pushes,
// so Shift+W over the player lands on Home with the player closed — also when Home is already the
// page underneath (owner decision 2026-09-09: the key always ends with the answer on screen).
// The `page` scope rides the same binding: the `g` chords jump to a module, but only to one this
// account can reach (the nav rail's own list, plus Settings, which is not in the rail's order).
useGlobalKeymap(
!!meQuery.data,
setPage,
meQuery.data ? [...moduleOrder(meQuery.data), "settings"] : [],
);
// Pin this tab to whatever account it first loaded as. Without this, a tab that never picked
// an account (it was riding the session's default) would silently swap identity when ANOTHER
@@ -271,6 +284,7 @@ export default function App() {
label: t("common.accessRequestsReview"),
onClick: () => {
focusAccessRequestsTab();
// Clicked inside whatever overlay hosts the toast; setPage closes it before pushing.
setPage("users");
},
},
@@ -282,7 +296,11 @@ export default function App() {
// Language is edited outside the Settings page (sidebar/login), so it stays instant.
function changeLanguage(code: LangCode) {
setLanguage(code);
api.savePrefs({ language: code }).catch(() => {});
// Also from the signed-out login page, where the save 401s — `req` marks that as handled, so
// the reporter stays quiet there.
api
.savePrefs({ language: code })
.catch((e: unknown) => notifyActionError(e, t("errors.writeFailed.prefs")));
}
// First-run: a fresh instance is in setup mode (all other routes are locked) → show the wizard.
+1 -1
View File
@@ -49,7 +49,7 @@ export default function AboutCell({ text, channelId }: { text: string | null; ch
ref={pop.refs.setReference}
{...pop.getReferenceProps()}
tabIndex={0}
className="block max-w-88 truncate text-muted cursor-help rounded-sm outline-hidden focus-visible:outline-2 focus-visible:outline-accent/60 focus-visible:outline-offset-2"
className="block max-w-88 truncate text-muted cursor-help rounded-sm outline-hidden focus-visible:outline-2 focus-visible:outline-accent focus-visible:outline-offset-2"
>
{preview}
</span>
+8 -11
View File
@@ -7,8 +7,9 @@ import { LoadingState, StateMessage } from "./ui/QueryState";
import { api } from "../lib/api";
import { qk } from "../lib/queryKeys";
import { notify } from "../lib/notifications";
import { notifyActionError } from "../lib/actionErrors";
import { copyText } from "../lib/clipboard";
import { invalidateUserAccess, isEmailOff } from "../lib/adminUsers";
import { invalidateUserAccess } from "../lib/adminUsers";
import type { AdminUserAccess, AdminUserRow } from "../lib/api/admin";
// The admin's answer to "why can this person not get in?", and the place every recovery action for
@@ -66,25 +67,19 @@ export default function AccountPanel({
const issue = useMutation({
mutationFn: () => api.createSetPasswordLink(user.id),
onSuccess: (r) => setLink({ url: r.url, expires_at: r.expires_at }),
onError: () => notify({ level: "error", message: t("users.account.linkFailed") }),
onError: (err) => notifyActionError(err, t("users.account.linkFailed")),
});
const resend = useMutation({
mutationFn: () => api.resendUserVerification(user.id),
onSuccess: () => notify({ level: "success", message: t("users.account.resendSent") }),
onError: (err) =>
notify({
level: "error",
message: isEmailOff(err)
? t("users.pendingAccess.resendEmailOff")
: t("users.pendingAccess.resendFailed"),
}),
onError: (err) => notifyActionError(err, t("users.pendingAccess.resendFailed")),
});
const activate = useMutation({
mutationFn: () => api.activateUser(user.id),
onSuccess: () => invalidateUserAccess(qc, user.id),
onError: () => notify({ level: "error", message: t("users.pendingAccess.activateFailed") }),
onError: (err) => notifyActionError(err, t("users.pendingAccess.activateFailed")),
});
const copy = async () => {
@@ -201,7 +196,9 @@ export default function AccountPanel({
<button
onClick={() => resend.mutate()}
disabled={!a.email_configured || a.email_verified || resend.isPending}
// The demo account's address is never confirmed, and the server refuses to mail it — as
// it refuses it a set-password link below (deep-review E11 round 3).
disabled={user.is_demo || !a.email_configured || a.email_verified || resend.isPending}
className="w-full glass-card glass-hover flex items-center gap-2 px-3 py-2.5 rounded-lg text-xs disabled:opacity-45 transition"
>
<Send className="w-4 h-4 shrink-0" />
+10 -2
View File
@@ -4,6 +4,7 @@ import { useQuery, useQueryClient } from "@tanstack/react-query";
import { qk } from "../lib/queryKeys";
import { Check, ListPlus, Plus, Youtube } from "lucide-react";
import { api, type Playlist } from "../lib/api";
import { notifyActionError } from "../lib/actionErrors";
import { useScrollFade } from "../lib/useScrollFade";
import { playlistName } from "../lib/playlistName";
import { PopoverItem, PopoverSurface, useAnchoredPopover } from "./ui/AnchoredPopover";
@@ -49,14 +50,19 @@ export default function AddToPlaylist({
qc.invalidateQueries({ queryKey: qk.playlist() });
}
// A failure is said, and the list refreshed either way. With no catch, a playlist deleted on
// another device (a 404 `not_found/playlist` — still offered, from a cache up to 30 s old) was an
// unhandled rejection: the tick did not change, nothing was said, the gone row stayed.
async function toggle(pl: Playlist) {
if (busy) return;
setBusy(true);
try {
if (pl.has_video) await api.removeFromPlaylist(pl.id, videoId);
else await api.addToPlaylist(pl.id, videoId);
refresh();
} catch (e) {
notifyActionError(e, t(pl.has_video ? "playlists.removeFailed" : "playlists.addFailed"));
} finally {
refresh();
setBusy(false);
}
}
@@ -69,8 +75,10 @@ export default function AddToPlaylist({
const pl = await api.createPlaylist(name);
await api.addToPlaylist(pl.id, videoId);
setNewName("");
refresh();
} catch (e) {
notifyActionError(e, t("playlists.addFailed"));
} finally {
refresh();
setBusy(false);
}
}
@@ -0,0 +1,104 @@
// @vitest-environment jsdom
//
// A failed user action is said once, and a user who is gone is not left listed (deep-review round 4).
// setRole / suspend / delete had no onError at all — "the global error dialog says it" — but a 404
// (the user was deleted in another tab) gets no dialog, so the click did nothing and the row stayed.
// And the invite approve/deny toasted unconditionally on top of the dialog `req` opens for a 409.
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { render, cleanup, screen, fireEvent, waitFor } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
vi.mock("react-i18next", async (importOriginal) => {
const actual = await importOriginal<typeof import("react-i18next")>();
return {
...actual,
useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }),
};
});
vi.mock("../lib/useMe", () => ({
useMe: () => ({
id: 1,
email: "admin@x",
role: "admin",
pending_invites: 0,
pending_activations: 0,
}),
}));
vi.mock("./ConfirmProvider", () => ({ useConfirm: () => async () => true }));
const { notify } = vi.hoisted(() => ({
notify: vi.fn((_n: { level: string; message: string }) => 0),
}));
vi.mock("../lib/notifications", async (importOriginal) => ({
...(await importOriginal<typeof import("../lib/notifications")>()),
notify,
}));
vi.mock("../lib/errorDialog", () => ({ reportError: vi.fn() }));
import AdminUsers from "./AdminUsers";
import { api, type AdminUserRow } from "../lib/api";
import i18n from "../i18n";
const row = {
id: 2,
email: "gone@x",
role: "user",
is_suspended: false,
is_demo: false,
email_verified: true,
is_active: true,
} as unknown as AdminUserRow;
function answer(status: number, detail: unknown) {
return new Response(JSON.stringify({ detail }), {
status,
headers: { "content-type": "application/json" },
});
}
beforeEach(() => {
notify.mockClear();
localStorage.clear();
});
afterEach(() => {
cleanup();
vi.restoreAllMocks();
});
function mount() {
const users = vi.spyOn(api, "adminUsers").mockResolvedValue([row] as never);
render(
<QueryClientProvider
client={new QueryClient({ defaultOptions: { queries: { retry: false } } })}
>
<AdminUsers />
</QueryClientProvider>,
);
return users;
}
describe("a user action that fails", () => {
it("says a user who is gone, and refetches the list", async () => {
const users = mount();
vi.spyOn(globalThis, "fetch").mockResolvedValue(
answer(404, { code: "not_found", reason: "user" }),
);
fireEvent.click(await screen.findByText("users.roles.makeAdmin"));
await waitFor(() =>
expect(notify).toHaveBeenCalledWith({
level: "error",
title: "users.roles.failed", // what failed…
message: i18n.t("errors.notFound.user"), // …and why: the request layer localizes a structured detail
}),
);
await waitFor(() => expect(users).toHaveBeenCalledTimes(2));
});
it("is not toasted on top of the error dialog the request already opened", async () => {
mount();
vi.spyOn(globalThis, "fetch").mockResolvedValue(answer(409, "Cannot demote the last admin"));
fireEvent.click(await screen.findByText("users.roles.makeAdmin"));
const { reportError } = await import("../lib/errorDialog");
await waitFor(() => expect(reportError).toHaveBeenCalled());
expect(notify.mock.calls.filter(([n]) => n.level === "error")).toEqual([]);
});
});
+28 -39
View File
@@ -24,7 +24,7 @@ import {
type PendingAccessRow,
} from "../lib/api";
import { notify } from "../lib/notifications";
import { invalidateUserAccess, isEmailOff } from "../lib/adminUsers";
import { invalidateUserAccess } from "../lib/adminUsers";
import { ADMIN_USERS_TAB_KEY } from "../lib/adminUsersTab";
import { Button } from "./ui/Button";
import Tooltip from "./ui/Tooltip";
@@ -38,6 +38,7 @@ import { useConfirm } from "./ConfirmProvider";
import Tabs, { usePersistedTab } from "./ui/Tabs";
import { PageToolbar } from "./ui/PageShell";
import { useMe } from "../lib/useMe";
import { notifyActionError, notifyKnownFailure } from "../lib/actionErrors";
// Admin-only user & access management: roles, access requests (the Invite whitelist), and the
// demo whitelist + reset. Each section is its own tab (persisted across reloads); the Access tab
@@ -120,6 +121,13 @@ function UsersRoles({ me }: { me: Me }) {
n.delete(id);
return n;
});
// A failed user action: said once (notifyActionError skips what `req` already put in its dialog),
// and the list refetched — a user deleted in another tab would otherwise stay listed, every click
// on the row doing nothing.
const onGoneOrFailed = (e: unknown, fallback: string) => {
notifyActionError(e, fallback);
qc.invalidateQueries({ queryKey: qk.adminUsers() });
};
const setRole = useMutation({
mutationFn: ({ id, role }: { id: number; role: "user" | "admin"; email: string }) =>
api.setUserRole(id, role),
@@ -129,7 +137,9 @@ function UsersRoles({ me }: { me: Me }) {
notify({ level: "success", message: t("users.roles.updated", { email: vars.email }) });
},
onSettled: (_d, _e, vars) => endPending(vars.id),
// Errors (e.g. last-admin / self) surface via the global error dialog with the server's detail.
// Guards (last admin / self) surface via the global error dialog with the server's detail; what
// it does not show (a user deleted meanwhile, a 404) is said here — see onGoneOrFailed.
onError: (e) => onGoneOrFailed(e, t("users.roles.failed")),
});
const suspend = useMutation({
@@ -146,7 +156,8 @@ function UsersRoles({ me }: { me: Me }) {
});
},
onSettled: (_d, _e, vars) => endPending(vars.id),
// Guards (demo / self / last admin) surface via the global error dialog.
// Guards (demo / self / last admin) surface via the global error dialog; the rest, as above.
onError: (e) => onGoneOrFailed(e, t("users.suspend.failed")),
});
const del = useMutation({
mutationFn: ({ id }: { id: number; email: string }) => api.adminDeleteUser(id),
@@ -156,6 +167,7 @@ function UsersRoles({ me }: { me: Me }) {
notify({ level: "success", message: t("users.delete.done", { email: vars.email }) });
},
onSettled: (_d, _e, vars) => endPending(vars.id),
onError: (e) => onGoneOrFailed(e, t("users.delete.failed")),
});
const onToggle = async (u: AdminUserRow) => {
@@ -307,12 +319,6 @@ function UsersRoles({ me }: { me: Me }) {
);
}
/** Did the invitation fail because this instance has no working SMTP? */
function inviteEmailOff(err: HttpError): boolean {
const d = err.detailData as { code?: string; reason?: string } | null | undefined;
return d?.code === "invite_send" && d.reason === "email_off";
}
// --- Sign-ups that are still outside (roadmap UX-25) --------------------------------------------
//
// The gap this closes: approve an address BEFORE the person registers and their sign-up produces no
@@ -339,7 +345,7 @@ function PendingAccess() {
refresh();
notify({ level: "success", message: t("users.pendingAccess.activated", { email: u.email }) });
},
onError: () => notify({ level: "error", message: t("users.pendingAccess.activateFailed") }),
onError: (err) => notifyActionError(err, t("users.pendingAccess.activateFailed")),
});
const resend = useMutation({
@@ -348,15 +354,9 @@ function PendingAccess() {
refresh();
notify({ level: "success", message: t("users.pendingAccess.resent", { email: u.email }) });
},
// The public resend is deliberately silent; this one is not, so say WHY it failed — an instance
// with no working SMTP would otherwise show a success toast for a mail that was never sent.
onError: (err) =>
notify({
level: "error",
message: isEmailOff(err)
? t("users.pendingAccess.resendEmailOff")
: t("users.pendingAccess.resendFailed"),
}),
// The public resend is deliberately silent; this one is not, and the server's reason says WHY
// (no working SMTP, a refused send) — a success toast for a mail never sent would be the lie.
onError: (err) => notifyActionError(err, t("users.pendingAccess.resendFailed")),
});
const list = rows.data ?? [];
@@ -518,7 +518,7 @@ function AdminInvites() {
refresh();
notify({ level: "success", message: t("settings.invites.approved", { email: vars.email }) });
},
onError: () => notify({ level: "error", message: t("settings.invites.approveFailed") }),
onError: (e) => notifyActionError(e, t("settings.invites.approveFailed")),
});
const deny = useMutation({
mutationFn: (id: number) => api.denyInvite(id),
@@ -526,7 +526,7 @@ function AdminInvites() {
refresh();
notify({ level: "success", message: t("settings.invites.denied") });
},
onError: () => notify({ level: "error", message: t("settings.invites.denyFailed") }),
onError: (e) => notifyActionError(e, t("settings.invites.denyFailed")),
});
const onDeny = async (i: { id: number; email: string }) => {
const ok = await confirm({
@@ -551,12 +551,8 @@ function AdminInvites() {
// A 409 means the list was stale — someone registered on that address since it loaded. Say
// what to do instead of a generic failure, and refetch so the row re-renders as read-only.
const conflict = e instanceof HttpError && e.status === 409;
notify({
level: "error",
message: t(
conflict ? "settings.invites.deleteHasAccount" : "settings.invites.deleteFailed",
),
});
if (conflict) notifyKnownFailure(e, t("settings.invites.deleteHasAccount"));
else notifyActionError(e, t("settings.invites.deleteFailed"));
if (conflict) refresh();
},
});
@@ -586,14 +582,7 @@ function AdminInvites() {
refresh();
notify({ level: "success", message: t("settings.invites.invited", { email: v.inv.email }) });
},
onError: (err) =>
notify({
level: "error",
message:
err instanceof HttpError && inviteEmailOff(err)
? t("settings.invites.inviteEmailOff")
: t("settings.invites.inviteFailed"),
}),
onError: (err) => notifyActionError(err, t("settings.invites.inviteFailed")),
});
// The one recipient the app has no language signal about: the admin typed this address in, so
// nobody ever told us how to write to its owner. The dialog is where that gets decided — and it
@@ -630,7 +619,7 @@ function AdminInvites() {
refresh();
notify({ level: "success", message: t("settings.invites.addedToWhitelist", { email }) });
},
onError: () => notify({ level: "error", message: t("settings.invites.addFailed") }),
onError: (e) => notifyActionError(e, t("settings.invites.addFailed")),
});
const list = invites.data ?? [];
@@ -780,7 +769,7 @@ function AdminDemo() {
qc.invalidateQueries({ queryKey: qk.demoWhitelist() });
notify({ level: "success", message: t("settings.demo.added", { email }) });
},
onError: () => notify({ level: "error", message: t("settings.demo.addFailed") }),
onError: (e) => notifyActionError(e, t("settings.demo.addFailed")),
});
const remove = useMutation({
mutationFn: (id: number) => api.removeDemoWhitelist(id),
@@ -788,7 +777,7 @@ function AdminDemo() {
qc.invalidateQueries({ queryKey: qk.demoWhitelist() });
notify({ level: "success", message: t("settings.demo.removed") });
},
onError: () => notify({ level: "error", message: t("settings.demo.removeFailed") }),
onError: (e) => notifyActionError(e, t("settings.demo.removeFailed")),
});
const onRemove = async (r: { id: number; email: string }) => {
const ok = await confirm({
@@ -806,7 +795,7 @@ function AdminDemo() {
level: "success",
message: t("settings.demo.resetDone", { count: r.playlists_seeded }),
}),
onError: () => notify({ level: "error", message: t("settings.demo.resetFailed") }),
onError: (e) => notifyActionError(e, t("settings.demo.resetFailed")),
});
const rows = list.data ?? [];
+17 -1
View File
@@ -1,7 +1,7 @@
import { useMemo } from "react";
import { useTranslation } from "react-i18next";
import { useQuery } from "@tanstack/react-query";
import { Keyboard, ListVideo, Palette, Tv } from "lucide-react";
import { Keyboard, ListVideo, Palette, Sparkles, Tv } from "lucide-react";
import CommandPalette, { type CommandGroup } from "./ui/CommandPalette";
import { api } from "../lib/api";
import { qk } from "../lib/queryKeys";
@@ -9,6 +9,7 @@ import { useMe } from "../lib/useMe";
import { moduleDef, moduleOrder } from "../lib/modules";
import { SCHEMES, type Scheme } from "../lib/theme";
import { closePalette, toggleKeyboardHelp, usePaletteOpen } from "../lib/keymap";
import { requestSurprise } from "../lib/homeIntent";
import { useNavigationActions } from "./NavigationProvider";
import { usePlaylistsActions } from "./PlaylistsProvider";
import { usePrefs } from "./PrefsProvider";
@@ -124,6 +125,21 @@ export default function AppCommandPalette() {
id: "actions",
label: t("palette.groups.actions"),
items: [
// "Meglepetés" — Home's reroll, reachable from anywhere (E24). There is deliberately no
// second "Mit nézzek?" row beside it: the Home module row above ALREADY matches that phrase,
// because the palette searches a module's descriptive `titleKey` too and Home's is literally
// "Mit nézzek most?". A row that navigates where an existing row already navigates would be
// duplication in the one list whose whole job is to be scanned quickly.
{
id: "action:surprise",
label: t("palette.actions.surprise"),
icon: <Sparkles className="w-4 h-4" />,
keywords: t("home.title"),
run: () => {
setPage("home");
requestSurprise();
},
},
{
id: "action:shortcuts",
label: t("keys.title"),
+2 -1
View File
@@ -12,6 +12,7 @@ import { useConfirm } from "./ConfirmProvider";
import DataTable from "./ui/DataTable";
import { LoadingState, StateMessage } from "./ui/QueryState";
import { auditColumns } from "./auditColumns";
import { notifyActionError } from "../lib/actionErrors";
// Admin-only audit trail: an append-only who/what/when log of admin actions + scheduler run
// summaries (never per-row). Read-only apart from Clear. Client-side sort/filter/paginate via
@@ -36,7 +37,7 @@ export default function AuditLog() {
qc.invalidateQueries({ queryKey: qk.adminAudit() });
notify({ level: "success", message: t("audit.cleared", { count: r.cleared }) });
},
onError: () => notify({ level: "error", message: t("audit.clearFailed") }),
onError: (e) => notifyActionError(e, t("audit.clearFailed")),
});
const onClear = async () => {
if (
+8 -3
View File
@@ -108,9 +108,13 @@ export default function ChannelDiscovery({
}
>
<button
onClick={() => onSubscribe(c)}
disabled={!canWrite || subscribe.isPending}
className="glass-card glass-hover inline-flex items-center gap-1.5 px-2.5 py-1.5 rounded-lg text-xs disabled:opacity-50 transition"
// Busy blocks the click, not the focus: a focused button that turns natively `disabled`
// drops the keyboard to <body> (the rule and its measurement: ui/Button).
onClick={() => !subscribe.isPending && onSubscribe(c)}
disabled={!canWrite}
aria-disabled={subscribe.isPending || undefined}
aria-busy={subscribe.isPending || undefined}
className="glass-card glass-hover inline-flex items-center gap-1.5 px-2.5 py-1.5 rounded-lg text-xs disabled:opacity-50 aria-disabled:opacity-50 transition"
>
<UserPlus className="w-3.5 h-3.5" />
{t("channels.discovery.subscribe")}
@@ -238,6 +242,7 @@ export default function ChannelDiscovery({
sort={sort}
onSortChange={setSort}
emptyText={t("channels.discovery.empty")}
keyboardNav
/>
) : (
<ChannelLayoutGrid
@@ -0,0 +1,87 @@
// @vitest-environment jsdom
//
// The channel manager's card layout as a keyboard list (E11 S4c): each card is an item, and Enter opens
// the channel through the name ChannelLink renders. VirtualGrid is replaced by a stand-in that mounts
// every item — jsdom has no layout for the virtualizer to measure — and registers with listNav exactly
// when the real one would: when it is asked to (`keyboardNav`).
import { Fragment, useRef } from "react";
import { afterEach, describe, expect, it, vi } from "vitest";
import { act, cleanup, render } from "@testing-library/react";
import type { VirtualGridProps } from "./ui/VirtualGrid";
vi.mock("react-i18next", async (importOriginal) => {
const actual = await importOriginal<typeof import("react-i18next")>();
return { ...actual, useTranslation: () => ({ t: (key: string) => key }) };
});
vi.mock("./ui/VirtualGrid", async () => {
const { useListNav } = await import("../lib/listNav");
return {
default: function GridStandIn({
items,
getKey,
renderItem,
keyboardNav,
}: VirtualGridProps<unknown>) {
const ref = useRef<HTMLDivElement>(null);
useListNav(ref, { enabled: !!keyboardNav });
return (
<div ref={ref}>
{items.map((item, i) => (
<Fragment key={getKey(item)}>{renderItem(item, 0, i)}</Fragment>
))}
</div>
);
},
};
});
import ChannelLayoutGrid from "./ChannelLayoutGrid";
import ChannelLink from "./ChannelLink";
import { type Column } from "./ui/DataTable";
afterEach(cleanup);
type Ch = { id: string; title: string };
const rows: Ch[] = [
{ id: "UCa", title: "Alpha" },
{ id: "UCb", title: "Beta" },
];
function press(key: string) {
act(() => {
(document.activeElement ?? document.body).dispatchEvent(
new KeyboardEvent("keydown", { key, bubbles: true, cancelable: true }),
);
});
}
const focusedTitle = () =>
(document.activeElement as HTMLElement).querySelector("[data-nav-open]")?.textContent;
describe("ChannelLayoutGrid · keyboard", () => {
it("walks the cards with j/k and opens the channel on Enter", () => {
const opened: string[] = [];
const columns: Column<Ch>[] = [
{
key: "channel",
header: "Channel",
cardPrimary: true,
render: (c) => (
<ChannelLink
id={c.id}
title={c.title}
thumbnailUrl={null}
onView={() => opened.push(c.id)}
/>
),
},
];
render(<ChannelLayoutGrid rows={rows} columns={columns} rowKey={(c) => c.id} />);
press("j");
expect(focusedTitle()).toBe("Alpha");
press("j");
expect(focusedTitle()).toBe("Beta");
press("Enter");
expect(opened).toEqual(["UCb"]);
});
});
+13 -2
View File
@@ -3,6 +3,7 @@ import { useTranslation } from "react-i18next";
import { type Column } from "./ui/DataTable";
import VirtualGrid from "./ui/VirtualGrid";
import { CHANNEL_LAYOUT_SPEC } from "../lib/channelLayout";
import { navItemAttrs } from "../lib/listNav";
// The channel manager's `cards` layout (E4 S3), built by reusing the SAME `Column<T>` render closures
// the DataTable draws — so the subscribed and discovery tables (which have different columns) both get
@@ -12,6 +13,9 @@ import { CHANNEL_LAYOUT_SPEC } from "../lib/channelLayout";
// Named slots place the columns that need bespoke positions; everything else flows in the meta line.
// Unlike DataTable's cramped narrow-screen card fallback, the desktop card is roomy, so it shows the
// `hideInCard` columns too (About, priority) — the user wants those affordances in every layout.
//
// Each card is a keyboard item (lib/listNav): `j`/`k` and the arrows walk the cards, and Enter opens
// the channel — the name ChannelLink renders is the card's open element.
interface ChannelLayoutGridProps<T> {
rows: T[];
@@ -24,6 +28,9 @@ interface ChannelLayoutGridProps<T> {
/** Column key rendered as the action cluster, pinned to the card's bottom edge. */
actionsKey?: string;
rowClassName?: (row: T) => string;
/** Extra DOM props per card — same escape hatch, same single consumer, as `DataTable.rowProps`, so
* a channel answers the right-click in the card layout exactly as it does in the table. */
rowProps?: (row: T) => Record<string, unknown>;
emptyText?: string;
}
@@ -35,6 +42,7 @@ export default function ChannelLayoutGrid<T>({
aboutKey,
actionsKey,
rowClassName,
rowProps,
emptyText,
}: ChannelLayoutGridProps<T>) {
const { t } = useTranslation();
@@ -48,8 +56,10 @@ export default function ChannelLayoutGrid<T>({
// Everything else flows in the meta line: not the heading, not a slotted column.
const meta = columns.filter((c) => !c.cardPrimary && !slotted.has(c.key));
const renderCard = (row: T) => (
const renderCard = (row: T, index: number) => (
<div
{...rowProps?.(row)}
{...navItemAttrs(index)}
className={`glass-card rounded-xl p-3 h-full flex flex-col gap-2 ${rowClassName?.(row) ?? ""}`}
>
<div className="flex items-start justify-between gap-2">
@@ -86,7 +96,8 @@ export default function ChannelLayoutGrid<T>({
minCol={spec.minCol}
colMaxWidth={null}
rowEst={spec.rowEst}
renderItem={renderCard}
keyboardNav
renderItem={(row, _itemWidth, index) => renderCard(row, index)}
/>
);
}
+3
View File
@@ -30,6 +30,9 @@ export default function ChannelLink({
{onView ? (
<button
onClick={onView}
// The channel's "open" element: in a keyboard-walkable list (lib/listNav) Enter on the row or
// card clicks this.
data-nav-open=""
onAuxClick={(e) => {
// Middle-click opens the channel's YouTube page in a new tab; left-click keeps
// opening the in-app channel detail.
@@ -0,0 +1,109 @@
// @vitest-environment jsdom
//
// Reported from daily use: middle-clicking the channel name in the YT player did nothing. It looked
// like a link — the YouTube icon beside it IS one — but it was a `<button>`, so the browser had no
// href to open a tab with. These pin the two halves the fix has to keep true at once: there is a real
// href pointing where the in-app navigation would go, AND a plain left click still takes the in-app
// path (which does more than navigate — the player has to close and reclaim its history entry first).
import { describe, it, expect, vi, beforeEach } from "vitest";
import { render, screen, cleanup } from "@testing-library/react";
vi.mock("react-i18next", async (importOriginal) => ({
...(await importOriginal<typeof import("react-i18next")>()),
useTranslation: () => ({ t: (key: string) => key }),
}));
import ChannelNameLink from "./ChannelNameLink";
const CH = "UCsXVk37bltHxD1rDPwtNM8Q";
beforeEach(() => {
// No global auto-cleanup in this project's vitest setup, and several cases render more than once.
cleanup();
window.history.replaceState({}, "", "/?page=feed&show=unwatched");
});
const renderLink = () => {
const onOpenInApp = vi.fn();
render(<ChannelNameLink channelId={CH} title="LEMMiNO" onOpenInApp={onOpenInApp} />);
return { link: screen.getByText("LEMMiNO") as HTMLAnchorElement, onOpenInApp };
};
/** Dispatch a click and report whether the component prevented it (i.e. handled it in-app).
* The document-level guard runs AFTER React's handler and swallows the default, so an
* un-prevented click on a real href does not make jsdom log "Not implemented: navigation". */
const clickWasHandled = (el: HTMLElement, init: MouseEventInit = {}): boolean => {
let prevented = false;
const guard = (e: Event) => {
prevented = e.defaultPrevented;
e.preventDefault();
};
document.addEventListener("click", guard);
el.dispatchEvent(new MouseEvent("click", { bubbles: true, cancelable: true, ...init }));
document.removeEventListener("click", guard);
return prevented;
};
describe("ChannelNameLink", () => {
it("is an anchor whose href opens this channel on the current page", () => {
const { link } = renderLink();
expect(link.tagName).toBe("A");
// The same URL openChannel pushes: the current page's query plus ?chan=<id>.
expect(link.getAttribute("href")).toBe(`/?page=feed&show=unwatched&chan=${CH}`);
});
it("takes the in-app path on a plain left click, and suppresses the browser navigation", () => {
const { link, onOpenInApp } = renderLink();
expect(clickWasHandled(link, { button: 0 })).toBe(true);
expect(onOpenInApp).toHaveBeenCalledTimes(1);
});
it.each([
["middle-click", { button: 1 }], // the reported gesture
["ctrl-click", { ctrlKey: true }],
["cmd-click", { metaKey: true }],
["shift-click", { shiftKey: true }],
// Alt/Option-click on a link is "download the target" — the one arm no test dispatched, so
// dropping it from the shared predicate hijacked it on every card thumbnail too.
["alt-click", { altKey: true }],
])("leaves %s to the browser", (_name, mod) => {
const { link, onOpenInApp } = renderLink();
// Neither hijacked in-app nor prevented — the href is left to do its job.
expect(clickWasHandled(link, mod)).toBe(false);
expect(onOpenInApp).not.toHaveBeenCalled();
});
it("does not lose the rest of the address bar", () => {
window.history.replaceState({}, "", "/?page=channels&q=abc");
const { link } = renderLink();
expect(link.getAttribute("href")).toContain("page=channels");
expect(link.getAttribute("href")).toContain("q=abc");
});
it("gives a channel with no title yet an accessible name", () => {
// A channel first seen through a live search can be a stub with no title. The anchor used to
// render empty — still in the tab order, announced by nothing but its href.
render(<ChannelNameLink channelId={CH} title={null} onOpenInApp={() => {}} />);
expect(screen.getByRole("link", { name: "player.channel" })).toBeTruthy();
});
it("keeps every click from a clickable ancestor when asked to — the browser's gestures too", () => {
// The video card's row reacts to clicks; its channel name must not trigger it, whichever way it
// is clicked.
const row = vi.fn();
render(
<div onClick={row}>
<ChannelNameLink channelId={CH} title="LEMMiNO" onOpenInApp={() => {}} stopPropagation />
</div>,
);
const link = screen.getByText("LEMMiNO");
// The bubbling guard in clickWasHandled never sees these (that is the point), so stop the
// browser's navigation in the capture phase instead.
const noNav = (e: Event) => e.preventDefault();
document.addEventListener("click", noNav, true);
for (const init of [{ button: 0 }, { ctrlKey: true }, { button: 1 }])
link.dispatchEvent(new MouseEvent("click", { bubbles: true, cancelable: true, ...init }));
document.removeEventListener("click", noNav, true);
expect(row).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,52 @@
import { forwardRef, type AnchorHTMLAttributes } from "react";
import { useTranslation } from "react-i18next";
import { isPlainLeftClick } from "../lib/nav";
import { urlWithParams } from "../lib/urlState";
// A channel name that is a REAL link. The player showed it as a `<button>`, so the browser's own
// gestures did nothing on it: middle-click and Ctrl/Cmd-click opened no tab, on a thing that looks
// exactly like a link (the YouTube icon right next to it IS an anchor and does open one). The video
// card's channel name had the same `<button>`, so it renders through here too.
//
// The href is the same URL `openChannel` pushes — `?chan=<id>` on the current page — so a new tab
// lands where the in-app navigation would have. A plain left click is still handled in-app, because
// the SPA path does more than navigate (the player has to close first and reclaim its own
// Back-to-close history entry; see lib/history.closeOverlaysThenNavigate).
type Props = Omit<AnchorHTMLAttributes<HTMLAnchorElement>, "href" | "onClick" | "title"> & {
channelId: string;
title: string | null | undefined;
/** The in-app navigation, run only for a plain left click. */
onOpenInApp: () => void;
/** The native tooltip (`title` is taken by the channel's name). */
tooltip?: string;
/** Keep EVERY click on the link — the browser's own gestures too — from reaching a clickable
* ancestor (the video card's row). */
stopPropagation?: boolean;
};
const ChannelNameLink = forwardRef<HTMLAnchorElement, Props>(function ChannelNameLink(
{ channelId, title, onOpenInApp, tooltip, stopPropagation, ...rest },
ref,
) {
const { t } = useTranslation();
return (
<a
{...rest}
ref={ref}
href={urlWithParams({ chan: channelId })}
title={tooltip}
onClick={(e) => {
if (stopPropagation) e.stopPropagation();
if (!isPlainLeftClick(e)) return; // let the browser open its tab/window
e.preventDefault();
onOpenInApp();
}}
>
{/* A channel first seen through a live search can have no title yet. An empty anchor would
still be in the tab order, announced by nothing but its href. */}
{title || t("player.channel")}
</a>
);
});
export default ChannelNameLink;
@@ -0,0 +1,89 @@
// @vitest-environment jsdom
//
// The channel page's video grid follows a running full-history backfill (bug hunt 2026-09, item 6).
// The rule itself — when a poll reading should refresh the grid — is pinned on the hook in
// lib/channelDetailPoll.backfill.dom.test.tsx. This pins that the PAGE runs it: deleting the hook's
// one call site left every lane green, knip included (the hook test's import kept the export "used").
//
// The page's own poll is what feeds the hook; a newer reading is handed to it here through the same
// query key the poll writes, rather than waiting out the 4 s interval. The Feed itself is stubbed —
// what matters is whether its queries are told they are stale.
import { describe, it, expect, vi, beforeAll, afterEach } from "vitest";
import { render, waitFor, cleanup, act } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
vi.mock("react-i18next", async (importOriginal) => ({
...(await importOriginal<typeof import("react-i18next")>()),
useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }),
}));
vi.mock("./NavigationProvider", () => ({
useNavigationActions: () => ({ closeChannel: vi.fn(), openChannel: vi.fn() }),
}));
vi.mock("./ConfirmProvider", () => ({ useConfirm: () => async () => true }));
vi.mock("./FeedFiltersProvider", () => ({
useFeedFilters: () => ({ includeNormal: true, includeShorts: false, includeLive: false }),
}));
vi.mock("./Feed", () => ({ default: () => null }));
vi.mock("./ChannelAbout", () => ({ default: () => null }));
import ChannelPage from "./ChannelPage";
import { api, type ChannelDetail, type FeedFilters, type Me } from "../lib/api";
import { qk } from "../lib/queryKeys";
beforeAll(() => {
class Stub {
observe() {}
unobserve() {}
disconnect() {}
takeRecords() {
return [];
}
}
globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver;
globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver;
});
afterEach(() => {
cleanup();
vi.restoreAllMocks();
});
const CH = "UCsXVk37bltHxD1rDPwtNM8Q";
const reading = (stored_videos: number): ChannelDetail =>
({
id: CH,
title: "LEMMiNO",
subscribed: true,
deep_requested: true,
backfill_done: false,
stored_videos,
}) as unknown as ChannelDetail;
const feedKey = qk.feed({ q: "", channel: CH } as unknown as FeedFilters);
describe("ChannelPage — the grid follows a running backfill", () => {
it("marks the channel's video grid stale when the poll sees more stored videos", async () => {
vi.spyOn(api, "channelDetail").mockResolvedValue(reading(19));
vi.spyOn(api, "tags").mockResolvedValue([] as never);
const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
qc.setQueryData(feedKey, { pages: [{ items: [], next_cursor: null }], pageParams: [null] });
render(
<QueryClientProvider client={qc}>
<ChannelPage
channelId={CH}
initialName="LEMMiNO"
me={{ id: 1, can_write: true } as unknown as Me}
view="cards"
setView={() => {}}
onShowInFeed={() => {}}
/>
</QueryClientProvider>,
);
await waitFor(() => expect(api.channelDetail).toHaveBeenCalled());
await waitFor(() => expect(qc.getQueryData(qk.channel(CH))).toBeTruthy());
expect(qc.getQueryState(feedKey)?.isInvalidated).toBe(false);
// The next poll tick: the background job has stored 60 more.
act(() => qc.setQueryData(qk.channel(CH), reading(79)));
await waitFor(() => expect(qc.getQueryState(feedKey)?.isInvalidated).toBe(true));
});
});
@@ -0,0 +1,99 @@
// @vitest-environment jsdom
//
// The channel page's Block and Hide toggles (deep-review E11 round 2) and its Subscribe / Unsubscribe
// button (round 3: the same row, missed) while their request runs. A
// focused button that turns natively `disabled` drops the keyboard to <body> in Chrome (measured for
// ui/Button in E11 S2), so a busy toggle blocks the click with aria-disabled instead — focusable, and
// still pressed only once.
import type { ReactNode } from "react";
import { describe, it, expect, vi, beforeAll, afterEach } from "vitest";
import { render, screen, fireEvent, waitFor, cleanup, act } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
vi.mock("react-i18next", async (importOriginal) => ({
...(await importOriginal<typeof import("react-i18next")>()),
useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }),
}));
vi.mock("./NavigationProvider", () => ({
useNavigationActions: () => ({ closeChannel: vi.fn(), openChannel: vi.fn() }),
}));
vi.mock("./ConfirmProvider", () => ({ useConfirm: () => async () => true }));
vi.mock("./FeedFiltersProvider", () => ({
useFeedFilters: () => ({ includeNormal: true, includeShorts: false, includeLive: false }),
}));
vi.mock("./Feed", () => ({ default: () => null }));
vi.mock("./ChannelAbout", () => ({ default: () => null }));
// The page's chrome portals into the shell's fixed toolbar band, which this test has no shell for.
vi.mock("./ui/PageShell", async (importOriginal) => ({
...(await importOriginal<typeof import("./ui/PageShell")>()),
PageToolbar: ({ children }: { children?: ReactNode }) => <>{children}</>,
}));
import ChannelPage from "./ChannelPage";
import { api, type ChannelDetail, type Me } from "../lib/api";
beforeAll(() => {
class Stub {
observe() {}
unobserve() {}
disconnect() {}
takeRecords() {
return [];
}
}
globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver;
globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver;
});
afterEach(() => {
cleanup();
vi.restoreAllMocks();
});
const CH = "UCsXVk37bltHxD1rDPwtNM8Q";
describe("ChannelPage — a busy toggle keeps the keyboard", () => {
it.each([
["channel.block", "blockChannel", true],
["channel.hide", "updateChannel", true],
["channel.subscribe", "subscribeChannel", false],
["channel.subscribedState", "unsubscribeChannel", true],
] as const)(
"%s stays focusable and fires once while its request runs",
async (label, call, subscribed) => {
vi.spyOn(api, "channelDetail").mockResolvedValue({
id: CH,
title: "LEMMiNO",
subscribed,
blocked: false,
hidden: false,
deep_requested: false,
backfill_done: true,
stored_videos: 10,
} as unknown as ChannelDetail);
vi.spyOn(api, "tags").mockResolvedValue([] as never);
const pending = vi.spyOn(api, call).mockReturnValue(new Promise(() => {}) as never);
render(
<QueryClientProvider
client={new QueryClient({ defaultOptions: { queries: { retry: false } } })}
>
<ChannelPage
channelId={CH}
initialName="LEMMiNO"
me={{ id: 1, can_write: true } as unknown as Me}
view="cards"
setView={() => {}}
onShowInFeed={() => {}}
/>
</QueryClientProvider>,
);
const button = await screen.findByRole("button", { name: label });
act(() => button.focus());
fireEvent.click(button);
await waitFor(() => expect(button.getAttribute("aria-disabled")).toBe("true"));
expect(button.hasAttribute("disabled")).toBe(false);
expect(document.activeElement).toBe(button);
fireEvent.click(button);
expect(pending).toHaveBeenCalledTimes(1);
},
);
});
+47 -29
View File
@@ -2,7 +2,7 @@ import { useCallback, useEffect, useRef, useState } from "react";
import { useTranslation } from "react-i18next";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { useLiveQuery } from "../lib/useLiveQuery";
import { channelDetailPollMs } from "../lib/channelDetailPoll";
import { channelDetailPollMs, useBackfillFeedRefresh } from "../lib/channelDetailPoll";
import { qk } from "../lib/queryKeys";
import { useNavigationActions } from "./NavigationProvider";
import { PriorityStepper } from "./ui/PriorityStepper";
@@ -29,6 +29,7 @@ import { useFeedFilters } from "./FeedFiltersProvider";
import { useConfirm } from "./ConfirmProvider";
import { notify } from "../lib/notifications";
import { notifyYouTubeActionError } from "../lib/youtubeErrors";
import { notifyActionError } from "../lib/actionErrors";
import { api, DEFAULT_TAG_MODES, type FeedFilters, type Me, type ChannelDetail } from "../lib/api";
import type { FeedView } from "../lib/feedView";
import { channelYouTubeUrl, formatViews } from "../lib/format";
@@ -93,6 +94,10 @@ export default function ChannelPage({
});
const ch = detail.data;
useEffect(() => setBannerAttempt(0), [ch?.banner_url]); // fresh channel/banner → try again
// The poll above is also the ONLY signal that the running backfill has written videos, so it
// drives the video grid's refresh too (the "Get full history" mutation below can't: it fires on
// the PATCH response, before the background job has stored anything).
useBackfillFeedRefresh(ch, qc);
// Background auto-ingest ("explore") of an un-subscribed channel's uploads.
const [nextToken, setNextToken] = useState<string | null | undefined>(undefined);
@@ -102,18 +107,23 @@ export default function ChannelPage({
const runExplore = useCallback(
(token?: string | null) => {
setExploring(true);
return api
.exploreChannel(channelId, token ?? undefined)
.then((r) => {
setNextToken(r.next_page_token);
qc.invalidateQueries({ queryKey: qk.feed() });
// Refetch the detail so `explored` flips true → the "Exploring" badge shows on the
// first visit (the GET that drove this render predated the explore that just ran).
qc.invalidateQueries({ queryKey: qk.channel(channelId) });
})
.finally(() => setExploring(false));
return (
api
.exploreChannel(channelId, token ?? undefined)
.then((r) => {
setNextToken(r.next_page_token);
qc.invalidateQueries({ queryKey: qk.feed() });
// Refetch the detail so `explored` flips true → the "Exploring" badge shows on the
// first visit (the GET that drove this render predated the explore that just ran).
qc.invalidateQueries({ queryKey: qk.channel(channelId) });
})
// Both the automatic first-visit explore and "load more" used to fail in silence: an empty
// channel page that looked like a channel with no videos (E11 S2b).
.catch((err: unknown) => notifyActionError(err, t("errors.writeFailed.exploreChannel")))
.finally(() => setExploring(false))
);
},
[channelId, qc],
[channelId, qc, t],
);
// First visit of an un-subscribed channel → ingest its recent uploads. Skipped for the demo
@@ -125,7 +135,7 @@ export default function ChannelPage({
if (ch.subscribed || ch.blocked) return;
if (ch.explored && ch.stored_videos > 0) return;
autoTried.current = channelId;
runExplore().catch(() => {});
void runExplore();
}, [ch, channelId, me.is_demo, runExplore]);
const block = useMutation({
@@ -135,15 +145,16 @@ export default function ChannelPage({
qc.invalidateQueries({ queryKey: qk.feed() });
qc.invalidateQueries({ queryKey: qk.blockedChannels() });
},
// No onError: block/unblock is a local DB op (not a YouTube write, so no 403 "connect" case),
// and its 400/409/422/500 are already surfaced by the global error modal (see api.ts req) —
// a caller toast here would just double it.
// No onError: a local DB op (not a YouTube write, so no 403 "connect" case). The app-wide
// reporter names the failure the error dialog does not cover (a 404 for a channel gone).
meta: { failTitle: "errors.writeFailed.blockChannel" },
});
// "Hide from feed": the same per-subscription flag the manager toggles (updateChannel {hidden}).
// Local DB op — keeps the subscription, only drops the channel's videos from the feed/explore.
const toggleHidden = useMutation({
mutationFn: () => api.updateChannel(channelId, { hidden: !ch?.hidden }),
meta: { failTitle: "errors.writeFailed.hideChannel" },
onSuccess: () => {
qc.invalidateQueries({ queryKey: qk.channel(channelId) });
qc.invalidateQueries({ queryKey: qk.feed() });
@@ -468,14 +479,18 @@ export default function ChannelPage({
)}
{!me.is_demo && (
<button
onClick={() => block.mutate()}
disabled={block.isPending}
// Busy blocks the click, not the focus: a focused button that turns natively
// `disabled` drops the keyboard to <body> (the rule and its measurement: ui/Button).
// Nor does a busy toggle light up on hover, as if a click would do something.
onClick={() => !block.isPending && block.mutate()}
aria-disabled={block.isPending || undefined}
aria-busy={block.isPending || undefined}
title={ch?.blocked ? t("channel.unblock") : t("channel.block")}
aria-label={ch?.blocked ? t("channel.unblock") : t("channel.block")}
className={`inline-flex items-center justify-center w-9 h-9 rounded-lg border disabled:opacity-50 transition ${
className={`inline-flex items-center justify-center w-9 h-9 rounded-lg border aria-disabled:opacity-50 transition ${
ch?.blocked
? "border-danger text-danger"
: "border-border text-muted hover:text-danger hover:border-danger"
: "border-border text-muted not-aria-disabled:hover:text-danger not-aria-disabled:hover:border-danger"
}`}
>
<Ban className="w-4 h-4" />
@@ -486,14 +501,15 @@ export default function ChannelPage({
from the feed. EyeOff = currently hidden (click to unhide). */}
{ch?.subscribed && !me.is_demo && (
<button
onClick={() => toggleHidden.mutate()}
disabled={toggleHidden.isPending}
onClick={() => !toggleHidden.isPending && toggleHidden.mutate()}
aria-disabled={toggleHidden.isPending || undefined}
aria-busy={toggleHidden.isPending || undefined}
title={ch.hidden ? t("channel.hiddenHint") : t("channel.hideHint")}
aria-label={ch.hidden ? t("channel.unhide") : t("channel.hide")}
className={`inline-flex items-center justify-center w-9 h-9 rounded-lg border disabled:opacity-50 transition ${
className={`inline-flex items-center justify-center w-9 h-9 rounded-lg border aria-disabled:opacity-50 transition ${
ch.hidden
? "border-accent text-accent"
: "border-border text-muted hover:text-fg hover:border-fg/40"
: "border-border text-muted not-aria-disabled:hover:text-fg not-aria-disabled:hover:border-fg/40"
}`}
>
{ch.hidden ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
@@ -505,7 +521,7 @@ export default function ChannelPage({
<Button
variant="outline"
onClick={onUnsubscribe}
disabled={unsubscribe.isPending}
loading={unsubscribe.isPending}
title={t("channel.unsubscribe")}
icon={<Check className="w-4 h-4" />}
>
@@ -515,7 +531,7 @@ export default function ChannelPage({
<Button
variant="outline"
onClick={() => subscribe.mutate()}
disabled={subscribe.isPending}
loading={subscribe.isPending}
icon={<Plus className="w-4 h-4" />}
>
{t("channel.subscribe")}
@@ -562,9 +578,11 @@ export default function ChannelPage({
{!ch?.subscribed && nextToken && (
<div className="flex justify-center pb-6">
<button
onClick={() => runExplore(nextToken)}
disabled={exploring}
className="inline-flex items-center gap-2 px-4 py-2 rounded-lg text-sm border border-border hover:bg-surface disabled:opacity-50"
// Busy blocks the click, not the focus (see Block above).
onClick={() => !exploring && runExplore(nextToken)}
aria-disabled={exploring || undefined}
aria-busy={exploring || undefined}
className="inline-flex items-center gap-2 px-4 py-2 rounded-lg text-sm border border-border not-aria-disabled:hover:bg-surface aria-disabled:opacity-50"
>
{exploring ? (
<Loader2 className="w-4 h-4 animate-spin" />
+20 -3
View File
@@ -15,6 +15,9 @@ import Tooltip from "./ui/Tooltip";
import DataTable from "./ui/DataTable";
import Tabs, { type TabDef } from "./ui/Tabs";
import ChannelLayoutGrid from "./ChannelLayoutGrid";
import { ComfortPinMenu } from "./ComfortPinMenu";
import { pinChannel } from "./comfortPin";
import { useContextMenu } from "./ui/useContextMenu";
import { LoadingState, StateMessage } from "./ui/QueryState";
import { useChannelTagActions } from "../lib/useChannelTagActions";
import { useTagSuggestions } from "./useTagSuggestions";
@@ -28,7 +31,7 @@ import ChannelStatsPopover from "./ChannelStatsPopover";
import Pager from "./ui/Pager";
import { CHANNEL_LAYOUTS, type ChannelLayout } from "../lib/channelLayout";
import { sortRows } from "../lib/columnSort";
import { filterRows } from "./ui/tableHeader";
import { filterRows } from "../lib/tableColumns";
import { useCardPager } from "../lib/useCardPager";
import { PageToolbar } from "./ui/PageShell";
import ChannelDiscovery from "./ChannelDiscovery";
@@ -106,6 +109,7 @@ export default function Channels() {
const [tagManagerOpen, setTagManagerOpen] = useState(false);
const unblock = useMutation({
mutationFn: (id: string) => api.unblockChannel(id),
meta: { failTitle: "errors.writeFailed.unblockChannel" },
onSuccess: () => {
qc.invalidateQueries({ queryKey: qk.blockedChannels() });
qc.invalidateQueries({ queryKey: qk.feed() });
@@ -129,6 +133,7 @@ export default function Channels() {
id: string;
body: { priority?: number; hidden?: boolean; deep_requested?: boolean };
}) => api.updateChannel(v.id, v.body),
meta: { failTitle: "errors.writeFailed.channelSettings" },
// Requesting full history may have just pulled in a channel's recent uploads, so
// refresh the per-user status and feed too — not only the channel list.
onSuccess: () => {
@@ -178,11 +183,10 @@ export default function Channels() {
// A failed attach/detach reverts the optimistic change by refetching the list. Also refetch
// suggestions: acceptSuggestion optimistically dropped the channel from that cache, and on
// failure the tag never actually attached — so the server still suggests it and the ghost chips
// must come back rather than leaving an empty cell (honest states). Surface the write failure too.
// must come back rather than leaving an empty cell (honest states). The hook reports the failure.
onError: () => {
qc.invalidateQueries({ queryKey: qk.channels() });
qc.invalidateQueries({ queryKey: qk.tagSuggestions() });
notify({ level: "error", message: t("channels.tags.updateFailed") });
},
});
// --- Suggested tags (E8·S4): learned from the user's own manual-tag pattern, shown as accept/dismiss
@@ -412,6 +416,11 @@ export default function Channels() {
/>
);
const hasLayout = view === "subscribed" || view === "discovery";
// The comfort-shelf pin on a channel row (E24 S3). ONE menu for the list, shared by the table and
// the card layout — they are two paints of the same rows, so a right-click must mean the same
// thing in both. Only mounted under the SUBSCRIBED branch below: a discovery result is not a
// `Channel` row yet, so pinning one could only 404.
const pinMenu = useContextMenu<ManagedChannel>();
// In cards there are no column headers to sort by, so a dropdown drives the SAME shared sort state
// the table header uses (table ↔ cards stay in sync). Table mode keeps the header clicks.
const sortControl = (
@@ -613,6 +622,8 @@ export default function Channels() {
onFiltersChange={setFilters}
fixedLayout
emptyText={t("channels.empty")}
rowProps={pinMenu.triggerProps}
keyboardNav
/>
) : (
<ChannelLayoutGrid
@@ -623,8 +634,14 @@ export default function Channels() {
aboutKey="about"
actionsKey="actions"
emptyText={t("channels.empty")}
rowProps={pinMenu.triggerProps}
/>
)}
<ComfortPinMenu
menu={pinMenu}
target={(c) => pinChannel(c.id)}
title={(c) => c.title ?? t("channels.row.thisChannel")}
/>
</div>
)}
</>
+3 -1
View File
@@ -16,6 +16,7 @@ import { useSortState } from "../lib/useSortState";
import { useFilterState } from "../lib/useFilterState";
import { useActiveAccountId } from "../lib/activeAccount";
import { useUrlMirror } from "../lib/useUrlMirror";
import { rememberNavPosition } from "../lib/history";
import {
channelsParamsToState,
channelsParamUpdates,
@@ -23,7 +24,7 @@ import {
type ChannelsUrlState,
} from "../lib/channelsUrlState";
import { usePersistedTab } from "./ui/Tabs";
import type { FilterMap } from "./ui/tableHeader";
import type { FilterMap } from "../lib/tableColumns";
import { useNavigation, useNavigationActions } from "./NavigationProvider";
import type { ChannelsView } from "./Channels";
@@ -243,6 +244,7 @@ export function ChannelsProvider({ children }: { children: ReactNode }) {
if (window.history.state?.sfPage === "channels") {
// A FRESH state, like setPage's: carrying the old entry's `_chan`/`_yt` markers over would make
// Back re-open the channel overlay or a spent search on top of the tab.
rememberNavPosition();
window.history.pushState({ sfPage: "channels", _chTab: v }, "");
}
},
+10 -5
View File
@@ -3,13 +3,13 @@ import { qk } from "../lib/queryKeys";
import { useTranslation } from "react-i18next";
import { useMutation, useQueryClient } from "@tanstack/react-query";
import { Send } from "lucide-react";
import { api, HttpError, type Message, type MessageUser } from "../lib/api";
import { api, type Message, type MessageUser } from "../lib/api";
import { useLiveQuery } from "../lib/useLiveQuery";
import { useScrollFade } from "../lib/useScrollFade";
import { Button } from "./ui/Button";
import { LoadingState, StateMessage } from "./ui/QueryState";
import { relativeTime } from "../lib/format";
import { notify } from "../lib/notifications";
import { notifyActionError } from "../lib/actionErrors";
import { partnerPub, POLL_MS, refreshPartnerPub, useKeyState } from "../lib/messaging";
import * as e2ee from "../lib/e2ee";
import KeyGate from "./KeyGate";
@@ -132,9 +132,14 @@ export default function ChatThread({
// The draft is kept (only cleared on success) so the user can retry. 400/409/422/500 already
// raised the global error dialog; 404 (recipient gone) and 429 (rate-limited) are "caller-
// handled" in api.req (no modal), so surface those here or the send fails silently.
if (e instanceof HttpError && (e.status === 404 || e.status === 429)) {
notify({ level: "error", message: e.detail || t("messages.sendFailed") });
}
//
// Through the shared helper, which knows both of those rules: it no-ops on anything already
// reported, and it prefers the server's reason only when that reason is a STRUCTURED detail.
// The hand-written copy that used to live here did `e.detail || …`, so a Hungarian reader got
// the route's English sentence — "That member hasn't set up messaging yet." — which is
// exactly what round 5 changed `notifyActionError` to stop doing. A second copy of a contract
// is how the fix misses a caller (round 6).
notifyActionError(e, t("messages.sendFailed"));
},
});

Some files were not shown because too many files have changed in this diff Show More