Compare commits

..
56 Commits
Author SHA1 Message Date
peter 6edb22988c release: 0.98.0 (feature/e31-design-review-2)
gate / frontend (push) Successful in 8m29s
gate / backend (push) Successful in 9m54s
2026-10-03 02:58:46 +02:00
peter f9c3714025 release: 0.98.0 notes — design review 2 2026-10-03 02:58:07 +02:00
peter a4775e1cd0 fix(ui): the filter rail shows no tag counts until the facets land
It printed the tag's whole-library count as the current filter's while the
facets loaded, and for good when they failed: live, "Music 238" beside a
search that matched 3 of its channels. Ordering still falls back to it.
2026-10-03 02:43:52 +02:00
peter b40440b825 fix(ui): review round 2 — reveal floor, failed next page, count states, row Play
- DataTable: the reveal floor holds only in the view it was made for (Channels
  never clears its follow, so every later sort mounted rows up to it)
- Feed + Plex browse: a failed next page stops the read-ahead and shows a row
  that retries, instead of re-requesting it once a round-trip
- the YouTube Load more click uses nextPageOnClick; a contract pins NEXT_PAGE
  to the read-aheads and the click sites to nextPageOnClick
- ItemCount: a failed count says so; a stale one says updating to a screen reader
- the narrow-picture Play lift applies only where a hover row is there to clear
- PlayerModal: one description ladder for the popover and the panel, both tested
- plexImage contract reads every img src, any attribute order
- Tabs comment names who persists the sub-tab
2026-10-03 02:25:15 +02:00
peter c4c442bc5b fix(feed): a hand-made relevance cursor is refused, not a crash
The page halves compare the cursor's timestamp with the window's in Python;
a null or naive one raised there, past the route's 400. decode_cursor now
refuses a null non-nullable key and a naive timestamp.
2026-10-03 02:25:14 +02:00
peter c74ceb73e3 fix(feed): the count pill dims while a new search's count loads
keepPreviousData held the last filter's count, and a broad word's count
takes seconds: the pill showed the old number as the new search's. It now
dims and is aria-busy until the new count lands, as AdminLogs does.
2026-10-03 01:01:46 +02:00
peter e0e28c006c perf(feed): relevance ranks the newest 2000 matches, the rest follow newest first
A broad word matched most of the library and ts_rank read every match:
"a" took 4.4 s a page on prod. The window's start is read once (0.1 s) and
rides the cursor, so a video ingested mid-scroll cannot repeat a row; the
page is read as its two halves, each on an index (80 ms + 5 ms).
2026-10-03 00:28:29 +02:00
peter f8f22fc985 test(i18n): a language test unmounts its reader before switching the language back
useTags reset the language in a finally while its hook was still mounted;
the switch scheduled a render, and in a loaded gate run it landed after
the jsdom environment was torn down ('window is not defined', two
unhandled errors). auditColumns and comfortPin had the same shape.
2026-10-02 23:48:37 +02:00
peter 90e0baa870 style(db): ruff format the re-runnable index migrations 2026-10-02 23:43:21 +02:00
peter dfd43ae89c docs: the folding channel header in its e2e, and three comments brought up to date
The e2e asserted the channel chrome never moves; it folds once now. The
ui catalog lists scrollFold and progressiveRows, Tabs names the
per-tab memory its callers use, and the welcome note says a removed
language orphans its stored copies too.
2026-10-02 23:42:32 +02:00
peter 608fde2d55 fix(db): re-running 0095/0096 rebuilds an invalid leftover index
A concurrent build that failed part-way leaves an INVALID index under its
name; IF NOT EXISTS kept it on the re-run, and 0095 then dropped the old
channel_id index, leaving none usable. The leftover is now dropped and
built again, and the old index goes only once the new one is valid.
2026-10-02 23:42:32 +02:00
peter 0302c466e8 fix(i18n): a failed RSS poll over one channel reads as one, and keeps its stops
rss_all_failed is a plural pair counted on the raw total (the values are
formatted strings, which pick no plural form); both sentences end with a
stop. The Hungarian 'show N more' uses the house's megjelenítése.
2026-10-02 23:42:32 +02:00
peter a50055577a fix(plex): every small Plex picture asks for its card-size cut
The playlist rows, the search box and the collection picker still
fetched the 400px poster for a 32-80px picture. A contract test pins the
set of bare Plex pictures, and the srcset widths to imgcache.SIZES.
2026-10-02 23:42:32 +02:00
peter e16cdbd16f fix(ui): a card's Play clears its hover row at a large text size; Downloads links and the player description
At text size 1.3 a 242px feed thumbnail put the 63px Play 13px into the
hover row (the Play grows with the root font, the picture's height does
not). A picture narrower than 14rem centres the Play and the resume pair
above the row; the picture box is a size container for it.

The Downloads title and channel links were 21px tall inline boxes; they
stand on a 24px line now (B1). The player's description says a failed
load, with a retry, instead of 'No description' (both the side panel
and the popover).
2026-10-02 23:42:31 +02:00
peter 15ec67b029 fix(ui): a read-ahead waits out a background refetch, and a revealed row is mounted before it is focused
NEXT_PAGE joins ANY fetch in flight: a read-ahead during an invalidation
or a poll joined that refetch and appended no page, and nothing asked
again until the reader moved the range (the channel page during a
backfill). The read-ahead now waits while the list fetches anything
(mayReadAhead, re-run on isFetching); a Load older click cancels a poll
instead (nextPageOnClick).

A page turn to a revealed row restarted the progressive mount at 30
rows; Channels focuses that row in the next commit, and at index 40 of
a 50-row page it was not there, so focus fell to <body>. The mount now
reaches through the revealed row.
2026-10-02 23:42:31 +02:00
peter ac53a3b757 perf(search): a one- or two-letter last word is matched whole, not as a prefix
A 1-2 letter prefix matched most of the library and ts_rank scored every
match: "m" took ~4-8 s on prod against ~150 ms now. The prefix match
starts at three characters (MIN_PREFIX_LEN); shared by the YouTube feed
and the Plex library search.
2026-10-02 21:52:47 +02:00
peter e6de18b8c0 perf(plex): the facets ask the episode rollup once per call, not once per show group
A watch-state scope (watched / in progress / unwatched) joined the per-user episode rollup into
each show facet query: five passes over the user's episodes per call, half of a filtered facets
request. The shows in scope are now resolved once and every group filters on their ids; the
grid-parity test holds the counts. Localdev: 99 -> 61 ms.
2026-10-02 21:32:49 +02:00
peter fb2e3d8490 perf(plex): a card fetches its picture at a card's size, not the detail page's
Cards fetched the full cuts: a 400x600 poster (~38 KB) for a ~144px picture and the 1280x720
backdrop (~162 KB) for a ~212px shelf picture. The image proxy now also cuts thumb_sm (240x360)
and art_sm (480x270), cached apart; card pictures (grid, shelves, collection tiles, Up next) carry
both in srcset with sizes=auto, so the browser picks by laid-out width and DPR. Localdev:
17 KB a poster, 56 KB a backdrop. The detail page keeps the full cuts.
2026-10-02 21:27:13 +02:00
peter d928d50686 perf(db): each feed sort rides an index, and an ICU upgrade rebuilds the title ones
The views, duration and title sorts had no index: every page read the whole videos heap and kept a
top-N (prod 350 ms a page, 550-780 for title). Migration 0096 adds one per direction and, for title,
per reader collation, in the exact ORDER BY feedquery builds. Localdev prod copy: plain sorts
180-310 ms -> ~1 ms, page 20 3-5 ms, narrow filters unchanged. view_count/duration are only written
by the enrich, which already rewrites indexed columns, so no HOT update is lost.

The title indexes are the first on an ICU collation, and postgres:16-alpine floats: collation.heal_drift
rebuilds them (CONCURRENTLY) at startup when Postgres reports the ICU version moved.
test_feed_sort_indexes holds every videos sort to an index and checks the rebuild.
2026-10-02 21:17:41 +02:00
peter 540822d2b0 perf(ui): a next-page fetch joins the one in flight instead of sending the cursor again
react-query's fetchNextPage cancels a next page still loading and sends it again by default.
Our requests ignore the abort signal, so the server ran both: the feed's read-ahead effect
re-runs on scroll frames before isFetchingNextPage reaches it, and on prod every feed page
went out 2-3 times (a 0.8 s title sort queued to 2.3 s). Every call now passes NEXT_PAGE
({cancelRefetch: false}), lint-enforced, with a contract test.
2026-10-02 20:46:30 +02:00
peter b4ef07d28e perf(ui): a long table page paints its first rows at once and the rest in steps
The channel manager's All page (~400 rows) blocked one frame for 0.7-0.9 s on prod to mount,
and every sort and search key re-rendered every row (300-400 ms). No single cell is to blame,
measured column by column. DataTable now mounts the first 30 rows at once and 40 more per
background transition; a new page, size, sort, filter or caller-side search (narrowedBy)
starts over from the first rows. A row is its own component, so the compiler keeps the rows a
step already holds.
2026-10-02 03:16:37 +02:00
peter e995c664c6 perf(db): the channel manager and the feed sidebar read an index, not the video heap
GET /api/channels summed every subscribed channel's videos by reading each video's heap page
(681 MB on prod): 51-61 ms warm, but 2.25 s on a first open and 6.4 s under EXPLAIN once the
table had left memory. /feed/summary and /feed/years did the same (~140 ms warm, ~800 ms cold).
Migration 0095 replaces ix_videos_channel_pub_desc with the same key INCLUDING the columns
those queries read, so they are Index Only Scans over a 25 MB index. A test holds the queries
and the index's column list together.

The admin Configuration page did a db.get per setting (two for an unset one): it now reads
every override in one query (~40 ms of ~55 on prod).
2026-10-02 03:16:37 +02:00
peter 20010178b0 fix(ui): the Plex detail poster keeps its 2:3, and a long crew shows its first names
The hero's grid row is as tall as the crew column beside the text, and the poster, a flex child,
stretched to it: a 2:3 picture drawn ~175x590 and cropped. It now aligns to the top, and a wide hero
gives the movie and show poster a bigger rung (w-56). The crew, the hero's tallest part, shows its
first 6 names and a 'Show N more' toggle; a list of 8 or fewer shows whole.
2026-10-02 02:37:20 +02:00
peter 78fdcda6bb feat(ui): E31 S4c — the channel header folds, the Plex pages fill their hero, the player gets a side panel
- Channel page: the banner and identity block fold away once the grid scrolls
  (ui/scrollFold: folds past 32px only if the page can stay scrolled, unfolds
  at the very top) and the name rides in the tab row. One Back button in both
  states keeps its focus; folded at a tablet width the actions wrap instead of
  pushing the page sideways.
- Plex movie and show pages: the TMDB facts over the key crew are the hero's
  second column (plexDetailAside), the keywords sit under the text; the show
  page shows them for the first time, with the same three toggles. A
  collection's hero is sized to its text.
- YouTube player, 1280px and wider: the title, channel, actions, description
  and queue sit in a side panel beside the video, which takes the window's
  height (1438x809 at 1920x911, was ~1008 wide). The wheel scrolls that panel
  while it can and turns the volume elsewhere.
- DESIGN.md decision log: the keywords' place under the text.
2026-10-02 02:13:17 +02:00
peter 98d0ca6eeb fix(i18n): every counted string is a plural pair, so one reads singular
"1 seasons", "Synced 1 subscriptions", "1 video(s) were skipped": 21 English
strings carried {{count}} in one plain key and read plural at one. Each is a
_one/_other pair now (Hungarian keeps one form for both), and so is every other
counted string, so the next wording change is a plural too. A contract test
holds the rule for every locale file.
2026-10-02 02:13:17 +02:00
peter e4c80fb7bb fix(ui): a page's sub-tab is remembered per browser tab, not forever
Settings, Configuration, Statistics and Users kept the last sub-tab in localStorage, so every
fresh tab opened Configuration on "Plex" as if that were its home. useTabPersistedState keeps it
in sessionStorage (account-scoped): a reload and a trip to another page keep it, a new tab starts
on the first sub-tab. The pre-selects (a Review link, the Google-link return) write the same store;
a value an older build left in localStorage is cleared on first read.
2026-10-02 00:49:46 +02:00
peter be156fc52b feat(ui): E31 S4b — the admin pages behind one rail row, Configuration as a sectioned list, Statistics as tiles and bars
- NavRailGroup: one "Admin" rail row opening a panel of the admin pages (links, so a
  middle-click still opens a tab); the rail fits 133% zoom without a hidden scroll
- sysconfig.SECTIONS orders the config groups under four headings (every group in exactly
  one, test-held); the page shows them as Settings' side list (Tabs `group` headings);
  the lone registration group reads "Registration" under "Access"
- StatTiles / StatTile / StatBars: the Statistics overview's figures as tiles, the
  per-operation quota use as horizontal bars (a zero draws none)
2026-10-02 00:07:00 +02:00
peter 25905aa786 fix(audit): fold a job's successful runs whatever ran between them
The owner read a fold that broke at every other job as the same row five
times. A job's successful runs now fold into one row at the newest run's
place across other jobs and admin actions; only that job's own failed or
skipped run breaks the fold (it stays its own row, and older successes
start a new fold below it).
2026-10-01 23:05:38 +02:00
peter 87d71e3762 feat(ui): E31 S4a — rail rows are links, the channel page opens on unwatched, users and audit tidied
- Nav rail page rows are real links (/?page=<id>): a middle-click or
  Ctrl-click opens the page in a new tab; a plain click stays in-app.
- A fresh channel page shows the unwatched videos; a picked watch state
  is a remembered preference (channelShow).
- Users: the role, suspend and delete move into the row's one menu
  (the "…", right-click, Shift+F10); locked ones keep their reason.
- Audit log: back-to-back successful runs of one job fold into one row
  that opens to the earlier ones; a failed run reads in the danger colour.
- DESIGN.md decision log: the E31 picks.
2026-10-01 22:51:38 +02:00
peter ab146211fa fix(scheduler,i18n): a failure a job names reads in the reader's language
The audit log showed a failed RSS poll as the server's English sentence
under a Hungarian UI. A job can now raise progress.JobFailed(reason,
message, **params): the scheduler stores the reason and params as the
run's data (dashboard, manual-run notice, audit row) beside the English
line, and the client renders scheduler.failure.<reason>. The RSS outage
names its first channel and the reason code (HTTP 404 vs unreachable).
An upstream's own error text stays as raised (diagnostic).
2026-10-01 22:51:37 +02:00
peter ec22eefd31 fix(ui): E31 S2 — the tab names the object, the channel header wraps, the player fits a short window
- The browser tab names the place and the object: PageHeading reports the
  view's own title (lib/documentTitle), so a channel, playlist or any Plex
  view reads "<object> · <module> · Siftlode" however it was opened; the
  Feed is named too; a season adds its show (tabTitle).
- The channel page's tab + action row wraps and lets the tag lane fold into
  "+N": at 1024 it ran past the edge and the page scrolled sideways.
- The YouTube player card's width is capped from the window height less the
  chrome measured under the video: at 133% zoom (685px) it cut its channel row.
- Plex certificate words (Not Rated, Approved, Passed) are named through the
  vocabulary on the info badge, show page, collection, filter and chips.
- DESIGN.md: the zoom target is 125% and 133%, not 200% (decision log).
2026-10-01 21:17:20 +02:00
peter 2543d44860 fix(messages): the welcome reads in the reader's language, not the one it was created in
E31 S1: the welcome is rendered once, at account creation, so an account that
switched language kept it in the old one (the owner's, English under a
Hungarian UI). A stored copy is recognised by its opening line (derived from
WELCOME) and served in the reader's language; other system messages stay as
stored.
2026-10-01 21:17:09 +02:00
peter 29c0715237 release: 0.97.0 (feature/e30-app-in-hungarian)
gate / frontend (push) Successful in 6m16s
gate / backend (push) Successful in 7m19s
2026-10-01 10:44:40 +02:00
peter 6b86f99249 release: 0.97.0 notes — the app in Hungarian 2026-10-01 10:44:39 +02:00
peter 3563858fd1 fix(ui,i18n): E30 review round 3 — the stored language follows the screen, a fresh text starts collapsed, Retry waits again 2026-10-01 10:43:59 +02:00
peter 08e62bbb16 fix(plex): E30 review round 3 — a dead headshot breaks the Plex-failing streak, 410 held by a test 2026-10-01 10:43:59 +02:00
peter 9454717f47 fix(ui,i18n): E30 review round 2 — honest bio failure, a full show summary, one HU name per thing
- Person page: a bio ask that fails says so with a retry instead of reading as 'no biography'.
- Show page: the summary is a ClampedText (5 lines + More), like the person and collection pages;
  one More/Less pair (plex.textMore/textLess) for all three.
- Channels: a re-slotted row is placed in the language the server orders by (the stored
  preference, else English), not the UI's guess.
- A skipped run's body is only the reason (its title already says skipped).
- HU: Behúzási kvótatartalék; Eseménynapló ürítve / az eseménynaplót; indok; közvetlenül
  játszható / közvetlen közvetítés in the Plex hints.
- Tests: ClampedText re-measures on a resize, the show summary clamp, serverOrderLang.
2026-10-01 08:31:04 +02:00
peter 6ac9ffd45e fix(plex,audit): E30 review round 2 — only a missing file is a dead headshot, one spelling per portrait
- warm_portraits: only 404/410 mark a headshot dead (measured: Plex answers a dead CDN file 404);
  a refused token or a 5xx is Plex failing, ends the run after three in a row and marks nothing.
  'failed' counts every failure, not only the current streak.
- is_portrait_url: plain path segments only — no dot or empty segment an HTTP client folds onto
  one file; person_thumb_url uses the same predicate (27 852 stored headshots all pass).
- Tests: Plex refusing (401/403/429/5xx), failures between successes (portraits and bios), the kept
  original served from cache, the audit_clear / demo_reset / discovery_purge rows' counts as data,
  and the 0094 backfill on real rows.
2026-10-01 08:31:03 +02:00
peter fa55e01ef0 fix(ui,i18n): E30 review round 1 — audit rows in the reader's language, one HU name per thing
- audit log: a job run named from its result data (nested phases too), a skipped run says so, a
  secret's reset reads as a reset, choices named as Configuration names them, interval / batch /
  counts localized; the inbox shares the same jobRunText
- HU: remux is "Közvetlen közvetítés" (it shared "Közvetlen lejátszás" with direct play); backfill is
  behúzás, enrich is dúsítás, collection is kollekció, the allow-list is fehérlista; the last
  "jobnak" / "transzkódolás"; a test holds the options of one choice to distinct names
- contract: every skipped / reason word a job returns is named in both languages
- person page: the bio wait is timed from the open and a failing ask stops the polling; ClampedText
  (ui) shared with the collection page, whose short summary lost its dead More
- a playlist names season 0 the Specials; the channel re-slot compares in the reader's alphabet; an
  invite's date is in the admin's language; the language-ordered lists are asked again once a
  language switch is saved (saveLanguage)
- poster grids' column minimum in rem; lint refuses minmax(Npx,…)
2026-09-30 03:26:58 +02:00
peter ab963c81d2 fix(plex,audit): E30 review round 1 — the people job keeps going, HU genres everywhere, results as data
- people_enrich: a failed TMDB lookup or a dead headshot is skipped, not the end of the run (a dead
  file is marked and left for a week; three failures in a row end it); portraits wait for the Plex
  module switch; a no-Plex original keeps its own cache key, so the resize still replaces it; the job
  entry lives in the plex domain (run_people_enrich)
- person-image takes only a headshot-shaped CDN url (no query variants of one file)
- every synced row gets Hungarian genres, and a HU genre filter matches the word as picked too;
  migration 0094 fills the rows no overlay reached
- a job run's audit row carries its result as data; audit_clear / demo_reset store their counts
- one credit-column list for the prune and the enrichment order; _overlay_pass typed
- tests: batch skip + stop, HU bio stored, dead headshots, module switch, the person-image route,
  the job body, genre filter + facet write + twin fallback, tagline/summary fallbacks, audit data,
  the reader's alphabet at every remaining collated() call site
2026-09-30 03:26:57 +02:00
peter 59e6cc027d fix(plex): a short "Known for" rail sits right beside the text (E30 S6)
With one to three titles (about one person in five) the rail sat at the far edge of the hero with a gap
between it and the bio. The person's column is now capped at the bio's reading measure and a short rail
follows it directly, so the leftover is the panel's end (owner pick "b"). Four or more keep half the hero.
2026-09-30 02:27:28 +02:00
peter ca94a716d8 docs(design): composition and the reader — a chapter and a Conformance step (E30 S6)
The rules covered each piece; nothing asked whether a screen works as a whole, and the person page passed
every rule with half its panel empty. New chapter: the space is used, one focal point, worst real data,
where am I, a click answers at once (no outside service on open), zoom and text size first-class (the
owner reads zoomed in). Conformance step 5 writes one composition line per screenshot and checks 200%
zoom and the largest text size. Anti-patterns and a decision-log row to match.

HU: the download worker setting reads "Párhuzamos letöltések" (a leftover from the S5 sweep).
2026-09-30 02:19:54 +02:00
peter 63d01babe5 feat(plex): the person page in two columns; a bio on its way holds its place (E30 S6)
The biography sat at reading width in the left half of a full-width panel with nothing on the right.
Now the person (headshot, facts, the bio clamped to eight lines with More / Less when it is cut) is on the
left and "Known for" on the right; with fewer than four titles the rail takes only its posters' width.
One column where the panel is narrow (200% zoom, the largest text size).

While the server fetches a bio (`enriching`), the page shows a paragraph skeleton (a new "text" shape of
LoadingState) and asks again every 1.5 s for up to 20 s. The view is keyed by the person's name, so
another person starts fresh.
2026-09-30 02:19:54 +02:00
peter 705377c797 feat(plex): the people are local — bios and portraits fetched ahead, not on the click (E30 S6)
A person's TMDB bio was fetched only when their page was first opened, off the request path, so the first
open showed no bio and the page never asked again (2 of 18,202 linked people were ever enriched).

- people_enrich scheduler job (every 10 min, 300 per run): linked people's bios from TMDB and their
  portraits into the image cache, the most-credited people first. A failed TMDB lookup ends the batch.
- Portraits are resized through Plex's photo transcoder to a 320 box (~15 KB instead of ~170 KB; the
  person-image proxy uses the same cache key, and a full-size original is removed as its small twin lands).
- The person response says `enriching` while a bio is on its way (a person the job hasn't reached yet).
- "Known for" is ranked by vote count, without guest visits as themselves (talk shows had crowded out the
  films: Tom Cruise had 14 of 15), with directed works, from a 40-deep pool; the page shows up to 15.
  Migration 0093 un-stamps enriched people so the job refetches them.
- A phased job result (backfill, people_enrich) now counts as a change for the audit trail; before, an
  automatic backfill run that added videos was never audited.
- HU: the watch reconcile job reads "egyeztetés" (a leftover from the S5 sweep).
2026-09-30 02:19:54 +02:00
peter 2819049133 feat(audit): job runs and config changes read in the reader's language (E30 S5)
The audit log showed the server's fixed English lines ("rss_poll: ok — new=5, failed=0",
"log_level = DEBUG", "Background sync paused") to a Hungarian reader. The details are rebuilt from the
row's data on the client: a job run names the job and its result as the inbox does, a config change
names the field and its values (on/off, grouped counts, "default"), a pause/resume adds nothing the
action does not say. A failed run keeps its error text (diagnostic). parseSummary reads enrich's bare
count ("50") back as `enriched`, as the server's `_result_data` names it — the audit and old notices
keep only the one-liner. No backend change; old rows render too.
2026-09-30 01:08:29 +02:00
peter 95b94a1d72 fix(ui): a longer label fits its control — pill value truncates, headers get their room (E30 S5)
Found in the Hungarian UAT: a toolbar SelectTrigger's text value wrapped out of the pill in a narrow
place (the playlists rail) — it now truncates in its own span; the Channels "Prioritás" and "Utolsó
feltöltés" headers were clipped by their fixed columns — sized for the label, funnel and sort arrow;
a sortable header is at least 24px wide (Hungarian "Ki" was a 12px target). PRE-EXISTING, any
language: the messages unlock button and "Forgot passphrase" sat flush — one gapped row.
2026-09-30 01:08:29 +02:00
peter 12077fbebc feat(i18n): Hungarian words for the English loanwords, one name per thing (E30 S5)
The D3 list: Feed, Shorts, Admin, Token, Port stay; stream, Prio, Job, Audit, Demo, Auto, Info,
Frontend/Backend, Backfill and transcode get the Hungarian word (közvetítés, Prioritás, Feladat,
Eseménynapló, Demó, Automatikus, Infó, Felület/Szerver, videók behúzása, átkódolás). The final sweep
also took out reset/sync/reconcile/setup/logs/whitelist/email, and three features that went by two
names: access requests (kérés/kérelem), backfill (betöltés/behúzás), and the connection test
("Check the connection" beside "Test connection", in both languages). Two quota actions shared one
Hungarian name. The playlists rail title and two of its sort labels are shorter so they fit.
DESIGN.md Copy: the rule and its decision-log row.
2026-09-30 01:08:29 +02:00
peter 519fbe2a18 feat(i18n): the person page's biography in Hungarian, from TMDB's translation (E30 S4)
The /person fetch appends translations and keeps TMDB's Hungarian biography beside the instance-language
one (new plex_people.biography_hu, migration 0092). A Hungarian reader is served it, the English one when
TMDB has none. Already-enriched people are un-stamped, so their next open refetches them once.
2026-09-29 23:44:48 +02:00
peter a39504e617 fix(stats): the full-history ratio counts the requested channels on both sides
"Full history: done / requested" counted every subscribed channel whose
back-catalog was complete in the numerator - also the ones another user asked
for, or short enough to finish on their own - against only the channels this
user requested, so the Stats page read "342/329". The numerator now counts the
requested channels that are done; "channels without full history yet" still
counts every subscribed one.
2026-09-29 22:46:04 +02:00
peter aac79ffa53 fix(i18n): numbers, units, dates and order in the UI language, and after a switch (E30 S3)
Every locale formatter takes the language as an argument. Under the React
Compiler a formatter that read the i18n singleton was memoized on its other
arguments, so a card kept "2 hr 4 min" after a switch to Hungarian; plexLang()
had the same flaw and left Plex queries in the old language. Components now
take the language from useTranslation, and the signature makes forgetting it
a type error.

- ratings, file sizes, stream stats: the language's decimal mark ("8,3")
- view and subscriber counts compact in the language ("1,2 E")
- seconds as the language's unit ("6 mp") in the sliders and the stats
- a person's birth and death dates as dates, not ISO strings
- A-Z lists by the shown name in the language's alphabet (compareText),
  the tag catalog re-sorted by its translated names
- lint local/locale-format (was date-format) also refuses Intl.NumberFormat
  outside lib/format.ts, localeCompare, and a component reading the language
  off the i18n singleton
2026-09-29 22:39:57 +02:00
peter 67c9f10153 feat(i18n): every server A→Z list in the reader's alphabet (E30 S3)
The Postgres image is Alpine, whose en_US.utf8 is byte order: "Életrajzi" and
"Ötödik" sorted after "Zorró" in the feed, the channel lists, the Plex grid,
its genres and collections. `app/collation.py` names the ICU collation of the
reader's language (hu-x-icu puts "Ö" after "O" and "Cs" after "Cu"), wraps
every text ORDER BY with it - the feed's keyset cursor compares with the same
expression, so paging walks the order it sorted in - and orders the lists built
in Python (genre and studio facets, collection members, the discovery re-sort)
through the same collation. Plex title tiebreaks use the shown (HU) title.
2026-09-29 22:39:57 +02:00
peter 584d082b6b feat(inbox): a run notice from before the result rode along reads its one-liner back into one (E30 S2) 2026-09-29 21:32:29 +02:00
peter f3a85916ec feat(i18n): the server's closed English vocabularies named in the reader's language (E30 S2)
- lib/vocab + the `vocab` locale area: TMDB crew jobs and departments, auto-tags (D1 A: the data
  stays English, the catalog is named at its one source, useTags), built-in download profiles,
  YouTube topics; held key-for-key to the backend's own lists by backendIds.contract.
- Countries by ISO code: TMDB parse stores {code, name}; rows stored before are coded from TMDB's
  own country list in one request (run_enrich); Intl names them, historic codes (YU, SU, XI…)
  from the catalog.
- Player tracks named from the language tag, forced/SDH flags and codec tail, not Plex's
  server-language displayTitle; no invented "Audio N" on the server.
- Download lifecycle notices (shared / expiring / expired / evicted) named from data; the share
  notice carries the title.
- Scheduler: the job result rides along as data and is named per key (dashboard + run notice).
- Invite status named; a render crash's notice no longer shows the engine's English message.
- Fix: YouTube links …/wiki/Humour, the tagger looked for "Humor" — Comedy never came from a topic.
2026-09-29 21:21:19 +02:00
peter f6298d29af fix(plex): the recs rail's anchor and a held recommendation are named in the reader's language (E30 S1b)
Found in the S1b UAT: 'Mert megnézted: Batman Forever' kept the English title, and
the info page's TMDB recommendations kept TMDB's English titles even for titles we
hold. A held recommendation is now named as the library names it, in the reader's
language; one we lack keeps TMDB's title.
2026-09-29 02:25:36 +02:00
peter 632e39d458 feat(plex): every card rail in the reader's language; an episode in a poster rail stands as its show (E30 S1b)
The grid served Hungarian titles, but every other card surface (Discover rails, the
show page's episodes, search, collections, the person page, playlists, Home's resume
rail, hero and comfort shelf) served English. The card serializers now take a
required keyword-only lang (a forgotten call site fails instead of going English), and
each of those reads takes ?lang=hu, with the language in its query key. A Hungarian
search finds an episode by its Hungarian title too.

An episode on a poster rail showed its 16:9 still cropped into a poster, titled only
with the episode name; it now shows the show's poster and name, with S·E · title under
it (option A). Plex's generic season names (Season 1, Specials) are rebuilt from the
number in the reader's language; a season with a name of its own keeps it.

Home's comfort-shelf move wrote its optimistic order to the key without the language,
which nothing read any more; it writes the query's own key.
2026-09-29 02:12:32 +02:00
peter 13d95ef70e feat(plex): episodes, taglines and genres in Hungarian from the HU overlay (E30 S1a)
The HU TV section lists every episode with its Hungarian title and summary, but the
overlay read only movies and shows, so every episode stayed English. It now overlays
the episodes too, and the HU tagline (new tagline_hu column, served on ?lang=hu).

Plex's HU agent returns a mixed genre vocabulary (Action beside Dráma), and a title
with no HU twin kept its English list, so the HU genre list showed Drama and Dráma.
plex/genres.py maps every genre word to one Hungarian word; the overlay writes it
into genres_hu for every row, and a HU genre filter is read through the same map,
so a saved filter still holding Action keeps matching.
2026-09-29 01:52:02 +02:00
329 changed files with 12611 additions and 2732 deletions
+25
View File
@@ -7,6 +7,31 @@ edit the release notes and re-run the generator.
Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading).
## 0.98.0 — 2026-10-03
### New
- Search is much faster: a short or common word now answers in a fraction of a second, with the newest matches ranked by relevance.
- The feed, the channel list and the Plex pages load faster, and Plex cards fetch a picture sized to the card.
- While a new search counts its results, the count dims instead of showing the old number.
### Fixed
- A list whose next page failed to load says so and offers Retry, instead of asking again and again.
- The filter rail's tag counts follow the current search.
- Smaller fixes to layout, keyboard focus and wording.
## 0.97.0 — 2026-10-01
### New
- The app speaks Hungarian throughout: Plex genres and statuses, numbers, units and A→Z order follow your language.
- Person pages show a Hungarian biography where TMDB has one, and open faster, with headshots kept on the server.
### Fixed
- Smaller fixes to wording and the Plex pages.
## 0.96.0 — 2026-09-27
### New
+1 -1
View File
@@ -60,7 +60,7 @@ how it gets built. Concretely:
`routes/*` (HTTP only: validation, auth-dependency, serialization) → domain packages `sync/`, `plex/`,
`downloads/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `quota`,
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `realtime`, `ratelimit`, `bodylimit`, `progress`,
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `realtime`, `ratelimit`, `bodylimit`, `progress`, `collation`,
`security`, `logs`, `logsink`, `diagnostics`, `debugsession`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
Frontend: `main.tsx` provider tree → `App.tsx` shell → `moduleRegistry` route-chunks → module
components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
+1 -1
View File
@@ -1 +1 @@
0.96.0
0.98.0
-8
View File
@@ -639,14 +639,6 @@
"lineCount": 1
}
},
{
"code": "reportArgumentType",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportArgumentType",
"range": {
@@ -0,0 +1,27 @@
"""add tagline_hu to plex_items + plex_shows — the HU overlay's tagline (E30 S1)
Revision ID: 0091_plex_tagline_hu
Revises: 0090_debug_session_draft
Create Date: 2026-09-29
The Hungarian overlay section lists a tagline beside the title and summary; the info page showed the
English one to a Hungarian reader. NULL until the next overlay sync fills it (serving falls back to EN).
"""
import sqlalchemy as sa
from alembic import op
revision: str = "0091_plex_tagline_hu"
down_revision: str | None = "0090_debug_session_draft"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.add_column("plex_items", sa.Column("tagline_hu", sa.Text(), nullable=True))
op.add_column("plex_shows", sa.Column("tagline_hu", sa.Text(), nullable=True))
def downgrade() -> None:
op.drop_column("plex_shows", "tagline_hu")
op.drop_column("plex_items", "tagline_hu")
@@ -0,0 +1,31 @@
"""add biography_hu to plex_people — TMDB's Hungarian biography (E30 S4)
Revision ID: 0092_plex_person_biography_hu
Revises: 0091_plex_tagline_hu
Create Date: 2026-09-29
The person page showed a Hungarian reader the English biography although TMDB has a Hungarian one for
most people. The person fetch now appends `translations` and keeps the Hungarian biography beside the
English one. Already-enriched people are un-stamped, so their next page open refetches them once (the
same lazy path as a first open) and fills it; until then the English biography stands in.
"""
import sqlalchemy as sa
from alembic import op
revision: str = "0092_plex_person_biography_hu"
down_revision: str | None = "0091_plex_tagline_hu"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.add_column("plex_people", sa.Column("biography_hu", sa.Text(), nullable=True))
op.execute(
"UPDATE plex_people SET tmdb_enriched_at = NULL "
"WHERE tmdb_person_id IS NOT NULL AND tmdb_enriched_at IS NOT NULL"
)
def downgrade() -> None:
op.drop_column("plex_people", "biography_hu")
@@ -0,0 +1,29 @@
"""re-fetch linked people — "Known for" ranked by votes, without guest visits (E30 S6)
Revision ID: 0093_plex_person_known_for_votes
Revises: 0092_plex_person_biography_hu
Create Date: 2026-09-30
A person's "Known for" pool was their credits ranked by TMDB popularity, so talk-show visits as
themselves crowded out the films (Tom Cruise: 14 of 15 were talk shows, none in the library). The pool is
now ranked by vote count, without guest visits, with directed works, and deeper. Already-enriched people
are un-stamped so the people_enrich job fetches them again; their biography stays on the row meanwhile.
"""
from alembic import op
revision: str = "0093_plex_person_known_for_votes"
down_revision: str | None = "0092_plex_person_biography_hu"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.execute(
"UPDATE plex_people SET tmdb_enriched_at = NULL "
"WHERE tmdb_person_id IS NOT NULL AND tmdb_enriched_at IS NOT NULL"
)
def downgrade() -> None:
pass # the watermark only says "fetched"; a refetch under the old ranking is not worth undoing to
@@ -0,0 +1,45 @@
"""Hungarian genres on every Plex row, not only on overlaid ones (E30 review)
Revision ID: 0094_plex_genres_hu_everywhere
Revises: 0093_plex_person_known_for_votes
Create Date: 2026-09-30
The HU genre facets and filter read `genres_hu`, which only the overlay pass wrote — so a library with
no HU twin, or a guid-less row, offered English genres a Hungarian filter could never match. The sync
now writes `genres_hu` for every row (plex/sync.py `_apply_facet_fields`); this fills the rows it has
not reached yet, from their own English list through the same map.
"""
import json
import sqlalchemy as sa
from alembic import op
from app.plex.genres import hu_genres
revision: str = "0094_plex_genres_hu_everywhere"
down_revision: str | None = "0093_plex_person_known_for_votes"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
conn = op.get_bind()
for table in ("plex_items", "plex_shows"):
rows = conn.execute(
sa.text(
f"SELECT id, genres FROM {table} "
"WHERE genres_hu IS NULL AND jsonb_typeof(genres) = 'array'"
)
).fetchall()
updates = [
{"id": rid, "g": json.dumps(hu)} for rid, genres in rows if (hu := hu_genres(genres))
]
if updates:
conn.execute(
sa.text(f"UPDATE {table} SET genres_hu = CAST(:g AS jsonb) WHERE id = :id"), updates
)
def downgrade() -> None:
pass # derived from `genres`; the sync writes the same values again
@@ -0,0 +1,83 @@
"""videos: the per-channel index covers what the channel manager and the feed sidebar read (E31 S4d)
The channel manager (`GET /api/channels`) sums each subscribed channel's videos: count, last upload, total
duration. The feed sidebar (`/feed/summary`, `/feed/years`) counts the reader's videos per channel and per
year under the default filters: available, not via explore, not a short, not live. Every one of them found
its rows through `ix_videos_channel_pub_desc` (channel_id, published_at DESC NULLS LAST, id) and then read
each video's heap page for the rest: ~270k rows at ~2.5 KB each, 681 MB on prod. That stays in cache only
while nothing else needs the memory. Measured on prod (2026-10-02): `/api/channels` 51–61 ms warm, but
2.25 s and 1.96 s on a first open in the browser and 6.4 s under EXPLAIN with 30k blocks read from disk;
`/feed/summary` and `/feed/years` ~140 ms warm, ~800 ms cold.
The same key with those columns INCLUDEd makes them Index Only Scans. On a prod-copy localdev, after a
VACUUM, the manager's sums read 0 heap pages and the summary reads the 25 MB index instead of the heap.
The visibility map is what lets them skip the heap; prod's autovacuum kept it complete (relallvisible ==
relpages). The key columns are unchanged, so every reader of the old index (the per-channel feed, the FK
cascade, the sync upsert, channel detail) rides this one. The old index is dropped.
tests/test_channel_list_index_only.py holds the queries and this column list together.
CONCURRENTLY (autocommit_block) builds it without blocking a live prod. IF (NOT) EXISTS keeps the drift
test's downgrade → re-upgrade round trip idempotent. A re-run after a build that failed part-way rebuilds
the INVALID leftover, and the old index is dropped only once the new one is valid: a re-run that kept the
leftover and dropped the old one left `videos` with no usable channel_id index (E31 S5 review).
"""
from alembic import op
from sqlalchemy import text
revision: str = "0095_videos_channel_cover_index"
down_revision: str | None = "0094_plex_genres_hu_everywhere"
branch_labels: str | None = None
depends_on: str | None = None
def _valid(name: str) -> bool | None:
"""The index's `indisvalid`, or None when there is no index of that name."""
row = (
op.get_bind()
.execute(
text(
"SELECT i.indisvalid FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid "
"WHERE c.relname = :name"
),
{"name": name},
)
.first()
)
return None if row is None else bool(row[0])
def _build(name: str, ddl: str) -> None:
"""CREATE INDEX CONCURRENTLY `name` (`ddl` is what follows the name), re-runnable. A build that
failed part-way leaves an INVALID index under the name, and `IF NOT EXISTS` would keep it: the
planner never uses it, and nothing says so (E31 S5 review). Such a leftover is dropped and built
again."""
if _valid(name) is False:
op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {name}")
op.execute(f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {name} {ddl}")
def _swap(build: str, ddl: str, drop: str) -> None:
_build(build, ddl)
if not _valid(build):
raise RuntimeError(f"{build} did not build valid; {drop} is kept")
op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {drop}")
def upgrade() -> None:
with op.get_context().autocommit_block():
_swap(
"ix_videos_channel_pub_cover",
"ON videos (channel_id, published_at DESC NULLS LAST, id) "
"INCLUDE (duration_seconds, unavailable_since, via_explore, is_short, live_status)",
"ix_videos_channel_pub_desc",
)
def downgrade() -> None:
with op.get_context().autocommit_block():
_swap(
"ix_videos_channel_pub_desc",
"ON videos (channel_id, published_at DESC NULLS LAST, id ASC)",
"ix_videos_channel_pub_cover",
)
@@ -0,0 +1,86 @@
"""videos: an index per feed sort, so a sorted page stops after its rows (E31 S4e)
The feed's views, duration and title sorts had no index. Every page (the first and each next cursor)
read the whole videos heap, filtered it and kept a top-N: on prod ~350 ms a page for views/duration,
550–780 ms for title (the ICU comparison on every row), 2.3 s for page 2 when the copies of one request
queued. With an index in the exact ORDER BY (`feedquery.order_by`: the key, NULLS LAST, then id) the
planner walks it and stops after the page. On a prod-copy localdev every plain sort fell from 180–310 ms
to ~1 ms, deep pages (20) to 3–5 ms, and narrow filters (a tag, a channel, a year, nothing found) stayed
at 0.3–25 ms.
One index per direction: a backward scan of `DESC NULLS LAST` gives `ASC NULLS FIRST`, not the
`ASC NULLS LAST` the asc sorts ask for. The title pair is per reader language (`app.collation.ICU`):
an index only serves an ORDER BY in its own collation.
Cost: ~130 MB (localdev: 10 + 10 + 8 + 8 + 4 × 23). Writes stay cheap: view_count and duration are
written by the enrich, which rewrites indexed columns already (the revalidation job only touches
last_checked_at), so no HOT update is lost. A search combined with one of these sorts can now walk the
index too: a term matching ~2 % of the videos took 130–200 ms instead of 7 (still under the plain sort it
replaces); rarer and broader terms stay fast. Fencing the index off under a search made broad terms
(`m`, `the`) take 2 s, so it is left to the planner.
The title indexes are the schema's first ones on an ICU collation: an ICU upgrade in the Postgres image
reorders them, and `app.collation.heal_drift` rebuilds them at startup when it sees one.
tests/test_feed_sort_indexes.py holds each sort and its index together.
CONCURRENTLY (autocommit_block) builds them without blocking a live prod; IF (NOT) EXISTS keeps the drift
test's downgrade → re-upgrade round trip idempotent.
"""
from alembic import op
from sqlalchemy import text
revision: str = "0096_videos_sort_indexes"
down_revision: str | None = "0095_videos_channel_cover_index"
branch_labels: str | None = None
depends_on: str | None = None
# name → the indexed ORDER BY (feedquery.sort_keys + order_by: key NULLS LAST, then id).
INDEXES = {
"ix_videos_views_desc": "view_count DESC NULLS LAST, id",
"ix_videos_views_asc": "view_count ASC NULLS LAST, id",
"ix_videos_duration_desc": "duration_seconds DESC NULLS LAST, id",
"ix_videos_duration_asc": "duration_seconds ASC NULLS LAST, id",
"ix_videos_title_en": '(lower(title) COLLATE "en-x-icu") ASC NULLS LAST, id',
"ix_videos_title_en_desc": '(lower(title) COLLATE "en-x-icu") DESC NULLS LAST, id',
"ix_videos_title_hu": '(lower(title) COLLATE "hu-x-icu") ASC NULLS LAST, id',
"ix_videos_title_hu_desc": '(lower(title) COLLATE "hu-x-icu") DESC NULLS LAST, id',
}
def _valid(name: str) -> bool | None:
"""The index's `indisvalid`, or None when there is no index of that name."""
row = (
op.get_bind()
.execute(
text(
"SELECT i.indisvalid FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid "
"WHERE c.relname = :name"
),
{"name": name},
)
.first()
)
return None if row is None else bool(row[0])
def _build(name: str, ddl: str) -> None:
"""CREATE INDEX CONCURRENTLY `name` (`ddl` is what follows the name), re-runnable. A build that
failed part-way leaves an INVALID index under the name, and `IF NOT EXISTS` would keep it: the
planner never uses it, and nothing says so (E31 S5 review). Such a leftover is dropped and built
again."""
if _valid(name) is False:
op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {name}")
op.execute(f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {name} {ddl}")
def upgrade() -> None:
with op.get_context().autocommit_block():
for name, columns in INDEXES.items():
_build(name, f"ON videos ({columns})")
def downgrade() -> None:
with op.get_context().autocommit_block():
for name in INDEXES:
op.execute(f"DROP INDEX CONCURRENTLY IF EXISTS {name}")
+2 -1
View File
@@ -23,7 +23,8 @@ Three layers, by *who* runs the code — not by feature:
3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth`, `sysconfig`, `quota`, `audit`,
`notifications`, `userscope`, `realtime`, `ratelimit`, `progress`, `logs` (the one logging
configuration), `logsink` (its `log_events` table), `diagnostics` (a user's consented log
level) and `debugsession` (a recording that tags its user's lines, E16), … are single-file primitives many layers import. They depend on nothing above them (a core
level), `debugsession` (a recording that tags its user's lines, E16) and `collation` (A→Z in the
reader's alphabet, E30), … are single-file primitives many layers import. They depend on nothing above them (a core
module must never import a `routes/…` module).
**Extraction trigger: the SECOND consumer.** Keep new logic inline in its first caller. The moment a
+83
View File
@@ -0,0 +1,83 @@
"""Alphabetical order in the reader's language (E30 S3).
Why not the database default: the Postgres image is Alpine, and musl's `en_US.utf8` collation is BYTE
order — every accented initial sorted after "Z" ("Zorró < Életrajzi < Ötödik"), in every A→Z list the
server orders. Postgres ships the ICU collations on the same image (dev, test, prod alike), so a text
sort names one: Hungarian puts "Ö" after "O" and "Cs" after "Cu", English folds the accents in.
`collated` wraps an ORDER BY expression — and because a keyset cursor compares with the SAME expression
(`feedquery.sort_keys`), the cursor pages in the same order it sorted in. `sorted_words` orders a Python
list the same way through the database, so a list built in Python matches one built in SQL exactly.
The feed's title sorts ride indexes built in these collations (migration 0096), and an index keeps the
order of the ICU that built it. `postgres:16-alpine` is a floating tag: an image update can bring a newer
ICU, after which those indexes disagree with the comparisons a query makes (a keyset page skips or
repeats a title). `heal_drift` rebuilds them at startup once Postgres reports the version moved."""
from typing import Any
from sqlalchemy import Engine, SQLColumnExpression, column, func, select, text, values
from sqlalchemy.orm import Session
from sqlalchemy.sql.elements import ColumnElement
from sqlalchemy.types import String
from app import logs
from app.mailtext import normalize
log = logs.get_logger(__name__)
ICU: dict[str, str] = {"en": "en-x-icu", "hu": "hu-x-icu"}
def collation(lang: str | None) -> str:
"""The ICU collation for a UI language; anything unknown sorts as English."""
return ICU[normalize(lang)]
def collated(expr: SQLColumnExpression[Any], lang: str | None) -> ColumnElement[Any]:
"""`expr` compared in the reader's alphabet. Case-folded, so "apple" and "Apple" sit together."""
return func.lower(expr).collate(collation(lang))
def sorted_words(db: Session, words: list[str], lang: str | None) -> list[str]:
"""`words` in the reader's alphabetical order, by the same ICU collation as `collated`."""
if not words:
return []
w = values(column("w", String), name="words").data([(x,) for x in words])
return list(db.scalars(select(w.c.w).order_by(collated(w.c.w, lang), w.c.w)))
def heal_drift(engine: Engine) -> list[str]:
"""Rebuild every index on an ICU collation whose library version moved, then record the new version.
Returns the rebuilt indexes. CONCURRENTLY: the feed keeps reading while it runs."""
with engine.connect().execution_options(isolation_level="AUTOCOMMIT") as conn:
drifted = conn.execute(
text(
"SELECT oid, collname FROM pg_collation WHERE collname = ANY(:names) "
"AND collversion IS DISTINCT FROM pg_collation_actual_version(oid)"
),
{"names": list(ICU.values())},
).all()
if not drifted:
return []
indexes = list(
conn.execute(
text(
"SELECT DISTINCT d.objid::regclass::text FROM pg_depend d "
"JOIN pg_class c ON c.oid = d.objid AND c.relkind = 'i' "
"WHERE d.classid = 'pg_class'::regclass "
"AND d.refclassid = 'pg_collation'::regclass AND d.refobjid = ANY(:oids)"
),
{"oids": [row.oid for row in drifted]},
).scalars()
)
log.warning(
"ICU collation version changed (%s): rebuilding %d indexes",
", ".join(row.collname for row in drifted),
len(indexes),
)
for name in indexes:
conn.execute(text(f"REINDEX INDEX CONCURRENTLY {name}"))
for row in drifted:
conn.execute(text(f'ALTER COLLATION "{row.collname}" REFRESH VERSION'))
return indexes
+5
View File
@@ -366,6 +366,11 @@ class Settings(BaseSettings):
tmdb_enrich_interval_min: int = 120
# Titles fetched per TMDB enrichment run (TMDB is fast + generously rate-limited).
tmdb_enrich_batch: int = 100
# How often the people job (E30 S6) fetches a batch of person bios from TMDB and portraits through Plex,
# in minutes, and how many of each per run: ~18k linked people at 300 per 10 min is about ten hours
# once, then only the newcomers.
people_enrich_interval_min: int = 10
people_enrich_batch: int = 300
@property
def allowed_email_set(self) -> set[str]:
+5 -4
View File
@@ -48,6 +48,7 @@ def pick(
shelf: list[dict] | None = None,
exclude: set[str] | None = None,
start: int = 0,
lang: str,
) -> dict | None:
"""`{"tier": ..., "card": ...}`, or None when nothing at all fits.
@@ -70,8 +71,8 @@ def pick(
skip = exclude or set()
def _resume() -> dict | None:
for card in rails.continue_watching(db, user) if resume is None else resume:
playable = rails.playable(db, user.id, card)
for card in rails.continue_watching(db, user, lang=lang) if resume is None else resume:
playable = rails.playable(db, user.id, card, lang=lang)
# Excluded on the card the USER saw (the show), not only on what it resolves to, so a
# rerolled-past show does not come straight back as its own next episode.
if cards.key(card) in skip or (playable is not None and cards.key(playable) in skip):
@@ -82,7 +83,7 @@ def pick(
def _comfort() -> dict | None:
if shelf is None:
own = pins.shelf(db, user, include_plex=rails.plex_available(db))
own = pins.shelf(db, user, include_plex=rails.plex_available(db), lang=lang)
else:
own = shelf
for card in pins.playables(own):
@@ -93,7 +94,7 @@ def pick(
def _surprise() -> dict | None:
# A wider draw than the one card we need: the shuffle is seeded, so without the headroom an
# excluded pick would leave the tier empty rather than move past it.
for card in rails.surprise(db, user, budget, seed, limit=SURPRISE_DRAW):
for card in rails.surprise(db, user, budget, seed, limit=SURPRISE_DRAW, lang=lang):
if cards.key(card) not in skip:
return card
return None
+6 -6
View File
@@ -170,7 +170,7 @@ class _Batch:
return b
def _shelf_card(db: Session, user: User, pin: ComfortPin, batch: _Batch) -> dict | None:
def _shelf_card(db: Session, user: User, pin: ComfortPin, batch: _Batch, lang: str) -> dict | None:
"""One pin as a unified card with its resolved `next`. None when the pinned row has gone (the FK
cascade removes the pin itself, so this is only the window between a delete and a refetch)."""
card: dict | None = None
@@ -196,7 +196,7 @@ def _shelf_card(db: Session, user: User, pin: ComfortPin, batch: _Batch) -> dict
if it is None:
return None
st = rollup.item_state_map(db, user.id, [it.id]).get(it.id)
card = cards.from_plex(leaf_card(it, st))
card = cards.from_plex(leaf_card(it, st, lang=lang))
nxt = dict(card) # a movie or a single episode plays itself
elif pin.plex_show_id is not None:
@@ -204,8 +204,8 @@ def _shelf_card(db: Session, user: User, pin: ComfortPin, batch: _Batch) -> dict
if sh is None:
return None
status = rollup.state_map(db, user.id, [sh.id]).get(sh.id, "new")
card = cards.from_plex(show_card(sh, status))
ep = queries.next_episode(db, user.id, sh.id)
card = cards.from_plex(show_card(sh, status, lang=lang))
ep = queries.next_episode(db, user.id, sh.id, lang=lang)
nxt = cards.from_plex(ep) if ep else None
elif pin.playlist_id is not None:
@@ -229,7 +229,7 @@ def _shelf_card(db: Session, user: User, pin: ComfortPin, batch: _Batch) -> dict
_PLEX_COLUMNS = ("plex_item_id", "plex_show_id")
def shelf(db: Session, user: User, *, include_plex: bool = True) -> list[dict]:
def shelf(db: Session, user: User, *, include_plex: bool = True, lang: str) -> list[dict]:
"""The whole shelf, in the user's own order.
`include_plex=False` is the Home surface with the deployment's Plex toggle off (the owner's call,
@@ -243,7 +243,7 @@ def shelf(db: Session, user: User, *, include_plex: bool = True) -> list[dict]:
pins = [p for p in pins if all(getattr(p, c) is None for c in _PLEX_COLUMNS)]
batch = _Batch.load(db, user, pins)
for pin in pins:
card = _shelf_card(db, user, pin, batch)
card = _shelf_card(db, user, pin, batch, lang)
if card is not None:
out.append(card)
return out
+16 -8
View File
@@ -40,7 +40,7 @@ from app.models import (
VideoState,
)
from app.plex import rollup
from app.plex.cards import leaf_card, movie_card, show_card
from app.plex.cards import display_title, leaf_card, movie_card, show_card
from app.plex.query import apply_show_state_filter, content_libs
# How many rows a single rail may carry — Home's rails and the Plex module's Discover rails alike
@@ -221,7 +221,9 @@ def _unseen_movie_conds(uid: int):
return ~seen
def plex_in_progress(db: Session, uid: int, limit: int = RAIL_LIMIT) -> list[tuple[object, dict]]:
def plex_in_progress(
db: Session, uid: int, limit: int = RAIL_LIMIT, *, lang: str
) -> list[tuple[object, dict]]:
"""In-progress movies + partially-watched shows as (progressed_at, card).
Mirrors `routes/plex/discover.py::_continue_watching` — same `PlexState` predicate for movies (a
@@ -249,7 +251,7 @@ def plex_in_progress(db: Session, uid: int, limit: int = RAIL_LIMIT) -> list[tup
.limit(limit)
)
for it, st in db.execute(mq).all():
out.append((st.progress_updated_at, movie_card(it, st)))
out.append((st.progress_updated_at, movie_card(it, st, lang=lang)))
if show_libs:
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
@@ -263,12 +265,14 @@ def plex_in_progress(db: Session, uid: int, limit: int = RAIL_LIMIT) -> list[tup
for sh, resumed_at in db.execute(
sq.order_by(prog.c.resumed_at.desc().nullslast()).limit(limit)
).all():
out.append((resumed_at, show_card(sh, "in_progress")))
out.append((resumed_at, show_card(sh, "in_progress", lang=lang)))
return out
def plex_surprise(db: Session, uid: int, budget: str, seed: int, limit: int) -> list[dict]:
def plex_surprise(
db: Session, uid: int, budget: str, seed: int, limit: int, *, lang: str
) -> list[dict]:
"""Unseen playable movies inside the budget, in seeded-shuffle order.
Movies only. A show is not a thing you can start playing, and resolving every candidate show to
@@ -300,14 +304,14 @@ def plex_surprise(db: Session, uid: int, budget: str, seed: int, limit: int) ->
q = q.where(PlexItem.duration_s <= hi)
items = db.execute(q).scalars().all()
states = rollup.item_state_map(db, uid, [it.id for it in items])
return [movie_card(it, states.get(it.id)) for it in items]
return [movie_card(it, states.get(it.id), lang=lang) for it in items]
# Watch states that take an episode out of the running for "what would play next".
_DONE = ("watched", "hidden")
def next_episode(db: Session, uid: int, show_id: int) -> dict | None:
def next_episode(db: Session, uid: int, show_id: int, *, lang: str) -> dict | None:
"""What "put that show on" means: the episode in progress, else the first unwatched one, else the
first — over the same `(season, episode)` broadcast ordering `detail.py::_episode_nav` uses.
@@ -357,5 +361,9 @@ def next_episode(db: Session, uid: int, show_id: int) -> dict | None:
ep, st = pick
show = db.get(PlexShow, show_id)
return leaf_card(
ep, st, show_title=show.title if show else None, show_rk=show.rating_key if show else None
ep,
st,
show_title=display_title(show, lang) if show else None,
show_rk=show.rating_key if show else None,
lang=lang,
)
+14 -7
View File
@@ -22,7 +22,7 @@ def plex_available(db: Session) -> bool:
return sysconfig.get_bool(db, "plex_enabled")
def playable(db: Session, uid: int, card: dict) -> dict | None:
def playable(db: Session, uid: int, card: dict, *, lang: str) -> dict | None:
"""The thing that would actually START from this card — itself for anything with a runtime, and
for a Plex SHOW the episode that is next (in progress, else the first unwatched, else the first).
@@ -40,11 +40,11 @@ def playable(db: Session, uid: int, card: dict) -> dict | None:
sh = by_rating_key(db, PlexShow, card["id"])
if sh is None:
return None
ep = queries.next_episode(db, uid, sh.id)
ep = queries.next_episode(db, uid, sh.id, lang=lang)
return cards.from_plex(ep) if ep else None
def continue_watching(db: Session, user: User, limit: int = RAIL_LIMIT) -> list[dict]:
def continue_watching(db: Session, user: User, limit: int = RAIL_LIMIT, *, lang: str) -> list[dict]:
"""One resume rail across both sources, most recently progressed first.
A Plex SHOW enters as a show CARD — that is what the Plex rail shows, and it is the artwork the
@@ -57,17 +57,22 @@ def continue_watching(db: Session, user: User, limit: int = RAIL_LIMIT) -> list[
(ts, cards.from_video(r)) for ts, r in queries.yt_in_progress(db, user, limit)
]
if plex_available(db):
rows += [(ts, cards.from_plex(c)) for ts, c in queries.plex_in_progress(db, user.id, limit)]
rows += [
(ts, cards.from_plex(c))
for ts, c in queries.plex_in_progress(db, user.id, limit, lang=lang)
]
rows.sort(key=lambda r: r[0] or NO_TS, reverse=True)
out = [card for _, card in rows[:limit]]
# Resolve AFTER the slice: only the rows that survive it are worth a query.
for card in out:
if card["kind"] == "show":
card["next"] = playable(db, user.id, card)
card["next"] = playable(db, user.id, card, lang=lang)
return out
def surprise(db: Session, user: User, budget: str, seed: int, limit: int = 3) -> list[dict]:
def surprise(
db: Session, user: User, budget: str, seed: int, limit: int = 3, *, lang: str
) -> list[dict]:
"""The hero's draw pool: unseen things that fit the budget, from both sources under ONE seed.
Interleaved rather than concatenated — the same alternation `routes/plex/discover.py` uses for its
@@ -83,7 +88,9 @@ def surprise(db: Session, user: User, budget: str, seed: int, limit: int = 3) ->
(cards.from_video, queries.yt_surprise(db, user, budget, seed, limit)),
(
cards.from_plex,
queries.plex_surprise(db, user.id, budget, seed, limit) if plex_available(db) else [],
queries.plex_surprise(db, user.id, budget, seed, limit, lang=lang)
if plex_available(db)
else [],
),
]
if seed % 2:
+84 -15
View File
@@ -12,9 +12,10 @@ import binascii
import json
from datetime import date, datetime, timedelta, timezone
from sqlalchemy import BigInteger, Select, and_, cast, false, func, or_, select
from sqlalchemy import BigInteger, Select, and_, case, cast, false, func, or_, select
from sqlalchemy.orm import Session, aliased
from app.collation import collated
from app.models import (
LIVE_OR_UPCOMING,
BlockedChannel,
@@ -264,7 +265,8 @@ def filtered_query(
# the queries that surfaced the video > description; see Video.search_vector). YouTube-like
# "fuzzy" matching: word-order-independent, multi-word AND, prefix on the word being typed,
# accent-insensitive (unaccent_simple). The channel name still matches as a substring so
# channel searches work. `rank_expr` (weight-aware ts_rank) drives the "relevance" sort.
# channel searches work. `rank_expr` (weight-aware ts_rank) drives the "relevance" sort, over the
# newest RANK_WINDOW matches only (see there).
rank_expr = None
if q:
# Resolve the channels whose NAME matches up front (there are only a few hundred channels, so
@@ -478,8 +480,10 @@ def feed_years(db: Session, user: User, **params) -> list[dict]:
# replaces OFFSET/LIMIT: deep scrolling no longer scans-and-discards skipped rows,
# and the page boundary is stable even as the scheduler ingests new videos mid-scroll
# (OFFSET would shift and duplicate/skip rows).
def sort_keys(sort: str, seed: int, rank_expr=None) -> list[dict]:
def sort_keys(sort: str, seed: int, rank_expr=None, lang: str | None = None) -> list[dict]:
"""`lang` orders the title sorts in the reader's alphabet (`collation`); the other keys ignore it."""
pub = {"expr": Video.published_at, "asc": False, "nullable": False, "kind": "dt"}
title = collated(Video.title, lang)
table: dict[str, list[dict]] = {
"newest": [pub],
"oldest": [{**pub, "asc": True}],
@@ -491,10 +495,8 @@ def sort_keys(sort: str, seed: int, rank_expr=None) -> list[dict]:
"duration_asc": [
{"expr": Video.duration_seconds, "asc": True, "nullable": True, "kind": "num"}
],
"title": [{"expr": func.lower(Video.title), "asc": True, "nullable": True, "kind": "str"}],
"title_desc": [
{"expr": func.lower(Video.title), "asc": False, "nullable": True, "kind": "str"}
],
"title": [{"expr": title, "asc": True, "nullable": True, "kind": "str"}],
"title_desc": [{"expr": title, "asc": False, "nullable": True, "kind": "str"}],
"subscribers": [
{"expr": Channel.subscriber_count, "asc": False, "nullable": True, "kind": "num"}
],
@@ -542,6 +544,54 @@ def sort_keys(sort: str, seed: int, rank_expr=None) -> list[dict]:
return table.get(sort) or table["newest"]
# Relevance ranks only the newest RANK_WINDOW matches (E31): ts_rank reads every match's tsvector, and
# a broad word matches most of the library ("a" ranked 193k rows in 6.8 s on prod, against 1.1 s to just
# find them). The rows past the window keep rank 0, so they follow the ranked ones newest first; a word
# that broad has no meaningful relevance order anyway, and a specific one fits the window whole.
RANK_WINDOW = 2000
def rank_window_start(db: Session, query: Select) -> datetime | None:
"""The published_at the ranked window starts at: the RANK_WINDOW-th newest match, None when fewer
match (rank them all). The newest-first walk stops after RANK_WINDOW matches (40 ms for "a")."""
return db.scalar(
query.with_only_columns(Video.published_at)
.order_by(Video.published_at.desc())
.offset(RANK_WINDOW - 1)
.limit(1)
)
def windowed_rank(rank_expr, since: datetime | None):
"""`rank_expr` inside the window, 0 past it. CASE evaluates ts_rank only for the rows inside."""
if since is None:
return rank_expr
return case((Video.published_at >= since, rank_expr), else_=0)
def windowed_relevance_rows(
db: Session, query: Select, keys: list[dict], window: datetime, cursor_id: str | None, n: int
) -> list:
"""Up to `n` rows of the windowed relevance order, read as its two halves: the window ranked, then
the rest newest first (every window row sorts before every row past it). As one ORDER BY the
planner computes the CASE for every match and sorts them all (2.7 s for "a" on prod); split, each
half rides an index (80 ms + 5 ms). With a cursor, `keys` carry its values (keyset_predicate)."""
pub_key = keys[1] # relevance: [rank, published_at]
rows: list = []
if cursor_id is None or pub_key["value"] >= window:
inside = query.where(Video.published_at >= window).order_by(*order_by(keys))
if cursor_id is not None:
inside = inside.where(keyset_predicate(keys, cursor_id))
rows = list(db.execute(inside.limit(n)).all())
if len(rows) < n:
newest = [pub_key] # past the window every rank is 0: the order is newest first
rest = query.where(Video.published_at < window).order_by(*order_by(newest))
if cursor_id is not None:
rest = rest.where(keyset_predicate(newest, cursor_id))
rows += db.execute(rest.limit(n - len(rows))).all()
return rows
def order_by(keys: list[dict]):
cols = []
for k in keys:
@@ -588,26 +638,45 @@ def keyset_predicate(keys: list[dict], cursor_id: str):
return rec(0)
def encode_cursor(keys: list[dict], row) -> str:
def encode_cursor(keys: list[dict], row, window: datetime | None = None) -> str:
"""`window` is the relevance window's start: the cursor carries it, so every page ranks against the
window page one did. Recomputed per page, a video ingested mid-scroll would push the window's oldest
row to rank 0, behind the cursor, and the reader would see it twice."""
vals = []
for i, k in enumerate(keys):
v = getattr(row, f"_k{i}")
vals.append(v.isoformat() if (v is not None and k["kind"] == "dt") else v)
payload = json.dumps({"k": vals, "id": row.id})
body: dict = {"k": vals, "id": row.id}
if window is not None:
body["w"] = window.isoformat()
payload = json.dumps(body)
return base64.urlsafe_b64encode(payload.encode()).decode()
def decode_cursor(keys: list[dict], cursor: str) -> tuple[list, str]:
def _cursor_dt(v):
"""A cursor's timestamp: aware or refused. The server writes only aware ones (timestamptz), and
windowed_relevance_rows compares them in Python, where a naive one beside an aware one raises — a
500 for a hand-made cursor instead of the 400 (E31 review 2)."""
if v is None:
return None
dt = datetime.fromisoformat(v)
if dt.tzinfo is None:
raise ValueError("naive cursor timestamp")
return dt
def decode_cursor(keys: list[dict], cursor: str) -> tuple[list, str, datetime | None]:
try:
payload = json.loads(base64.urlsafe_b64decode(cursor.encode()))
raw = payload["k"]
if len(raw) != len(keys):
raise ValueError("key arity mismatch")
vals = [
datetime.fromisoformat(v) if (v is not None and k["kind"] == "dt") else v
for k, v in zip(keys, raw)
]
return vals, payload["id"]
vals = [_cursor_dt(v) if k["kind"] == "dt" else v for k, v in zip(keys, raw)]
for k, v in zip(keys, vals):
if v is None and not k["nullable"]:
raise ValueError("null value for a non-nullable key")
window = payload.get("w")
return vals, payload["id"], _cursor_dt(window) if window else None
except (binascii.Error, json.JSONDecodeError, KeyError, ValueError, TypeError) as exc:
# Framework-free: the route handler translates this into an HTTP 400 so the
# read-model stays web-layer-agnostic (see routes/feed.get_feed).
+18 -2
View File
@@ -24,11 +24,12 @@ from sqlalchemy.exc import DataError
from starlette.middleware.sessions import SessionMiddleware
from app import bodylimit
from app import collation
from app import logsink
from app import state
from app import sysconfig
from app.config import settings
from app.db import SessionLocal
from app.db import SessionLocal, engine
from app.plex import stream as plex_stream
from app.plex import subtitles as plex_subtitles
from app.routes import (
@@ -97,6 +98,9 @@ async def lifespan(app: FastAPI):
# owns, so racing it against early playback is safe.
sweep = asyncio.create_task(_sweep_hls_orphans())
reaper = asyncio.create_task(_reap_hls_sessions())
# Fire and forget as well: a rebuild after an ICU upgrade takes seconds, and the title sorts serve
# meanwhile (CONCURRENTLY) in the old index's order, as they did before the restart.
heal = asyncio.create_task(_heal_collation_drift())
start_scheduler()
try:
yield
@@ -106,7 +110,7 @@ async def lifespan(app: FastAPI):
# pending ("Task was destroyed but it is pending!" — noise that reads like a bug during
# triage). Cancelling does NOT interrupt the rmtree thread itself; this only makes our own
# bookkeeping honest.
for task in (sweep, reaper):
for task in (sweep, reaper, heal):
task.cancel()
with suppress(asyncio.CancelledError):
await task
@@ -139,6 +143,18 @@ async def _reap_hls_sessions() -> None:
log.exception("HLS session reap failed")
async def _heal_collation_drift() -> None:
"""Background startup check of the ICU collation the title-sort indexes are built in (see
`collation.heal_drift`). A failure is logged, never fatal: the sorts still answer, only possibly in
the old order."""
try:
await asyncio.to_thread(collation.heal_drift, engine)
except asyncio.CancelledError:
raise
except Exception:
log.exception("ICU collation drift check failed")
async def _sweep_hls_orphans() -> None:
"""Background half of the startup HLS sweep (see lifespan). Owns its own errors: a failure here
must never take the app down, and a cancelled shutdown is not one."""
+19 -7
View File
@@ -330,10 +330,12 @@ class Video(Base, TimestampMixin):
__tablename__ = "videos"
id: Mapped[str] = mapped_column(String(16), primary_key=True) # video id
# No standalone channel_id index: the composite `ix_videos_channel_pub_desc`
# (channel_id, published_at DESC NULLS LAST, id) from migration 0067 leads with channel_id, so it
# serves every channel_id lookup (FK cascade, sync upsert, channel-detail) AND the per-channel feed
# sort. Like the search_vector GIN, this composite is migration-defined, not an inline `index=True`.
# No standalone channel_id index: the composite `ix_videos_channel_pub_cover`
# (channel_id, published_at DESC NULLS LAST, id) from migration 0095 leads with channel_id, so it
# serves every channel_id lookup (FK cascade, sync upsert, channel-detail) and the per-channel feed
# sort. Its INCLUDEd columns make the channel manager's sums and the feed sidebar's counts Index Only
# Scans (no heap): a column those queries start reading belongs in it (test_channel_list_index_only).
# Like the search_vector GIN, this composite is migration-defined, not an inline `index=True`.
channel_id: Mapped[str] = mapped_column(ForeignKey("channels.id", ondelete="CASCADE"))
title: Mapped[str | None] = mapped_column(Text) # normalized for display (see app.titles)
original_title: Mapped[str | None] = mapped_column(Text) # raw YouTube title, preserved
@@ -347,6 +349,9 @@ class Video(Base, TimestampMixin):
DateTime(timezone=True), index=True, nullable=False, server_default=func.now()
)
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
# The feed's views / duration / title sorts each ride a migration-defined index in their exact
# ORDER BY, one per direction (and per ICU collation for title): migration 0096, held to
# `feedquery.order_by` by test_feed_sort_indexes.
duration_seconds: Mapped[int | None] = mapped_column(Integer)
view_count: Mapped[int | None] = mapped_column(BigInteger)
like_count: Mapped[int | None] = mapped_column(BigInteger)
@@ -1224,6 +1229,7 @@ class PlexShow(Base, TimestampMixin, UpdatedAtMixin):
# Hungarian-metadata twin fields, written by the overlay sync (E5 S5). NULL = no HU twin matched,
# so serving falls back to the EN columns. `people_text` is language-neutral, reused for both vectors.
title_hu: Mapped[str | None] = mapped_column(String(512))
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline (E30)
summary_hu: Mapped[str | None] = mapped_column(Text)
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
search_vector: Mapped[object | None] = mapped_column(
@@ -1385,6 +1391,7 @@ class PlexItem(Base, TimestampMixin, UpdatedAtMixin):
# Hungarian-metadata twin fields, written by the overlay sync (E5 S5). NULL = no HU twin matched
# (serving falls back to the EN columns). `people_text` is language-neutral, reused for both vectors.
title_hu: Mapped[str | None] = mapped_column(String(512))
tagline_hu: Mapped[str | None] = mapped_column(Text) # the HU overlay's tagline (E30)
summary_hu: Mapped[str | None] = mapped_column(Text)
genres_hu: Mapped[list | None] = mapped_column(JSONB(none_as_null=True))
search_vector: Mapped[object | None] = mapped_column(
@@ -1415,7 +1422,8 @@ class PlexPerson(Base, TimestampMixin, UpdatedAtMixin):
top few cast. Rows whose name no longer appears in ANY item's cast_names/directors (a library
removal orphaned them) are pruned by the maintenance sweep, mirroring how Plex drops orphaned
metadata. The filterable cast lives in plex_items/plex_shows.cast_names (self-cleaning with the
row); this table is ONLY the headshot cache."""
row); this table is the people's own facts: the headshot link, the TMDB link and the TMDB bio
(the headshot FILE lives in the image cache, filled by the people_enrich job)."""
__tablename__ = "plex_people"
@@ -1425,11 +1433,15 @@ class PlexPerson(Base, TimestampMixin, UpdatedAtMixin):
# TMDB person id, linked by name from a title's TMDB credits (E5 S2 lays the foundation; the /person
# biography fetch + person-page render land in S6). NULL until a TMDB-enriched title names this person.
tmdb_person_id: Mapped[int | None] = mapped_column(Integer)
# --- TMDB person enrichment (E5 S6): fetched from /person/{tmdb_person_id} the first time the person
# page is opened, then SERVED FROM THE ROW. `tmdb_enriched_at` is the watermark (set even on a
# --- TMDB person enrichment (E5 S6): fetched from /person/{tmdb_person_id} by the people_enrich job
# (E30 S6), or by the first person-page open the job hasn't reached, then SERVED FROM THE ROW. `tmdb_enriched_at` is the watermark (set even on a
# no-op fetch so it isn't retried every open). `known_for` = top combined-credits [{tmdb_id,
# media_type, title}], resolved to library rows by guid at serve time (like recommendations). ---
biography: Mapped[str | None] = mapped_column(Text)
# TMDB's Hungarian biography (E30 S4), from the same request's `translations`; a Hungarian reader is
# served it, and the instance-language `biography` when TMDB has none. Birthplace, aliases and
# keywords have no Hungarian on TMDB, so they stay in the instance language.
biography_hu: Mapped[str | None] = mapped_column(Text)
birthday: Mapped[str | None] = mapped_column(String(10)) # "YYYY-MM-DD", served as-is
deathday: Mapped[str | None] = mapped_column(String(10))
place_of_birth: Mapped[str | None] = mapped_column(Text)
+32 -14
View File
@@ -15,20 +15,35 @@ def _art_url(rating_key: str, art_key: str | None) -> str | None:
# Card hover-overview: a short synopsis for the reveal panel, trimmed so a strip/related row of cards
# stays a few hundred bytes each rather than carrying full synopses. (These non-grid card paths serve
# the EN summary/rating; the main grid coalesces the HU twin in browse.unified_library.)
# stays a few hundred bytes each rather than carrying full synopses.
def _card_summary(summary: str | None) -> str | None:
return (summary or "")[:300] or None
def movie_card(it: PlexItem, st: PlexState | None) -> dict:
def norm_lang(lang: str | None) -> str:
"""The overlay has an EN base + an optional HU twin; anything other than `hu` serves EN."""
return "hu" if lang == "hu" else "en"
# Every card speaks the reader's language (E30): `lang="hu"` serves the HU overlay twin's title and
# summary, falling back to EN where a row has none — the same rule as browse.unified_library's
# `title_col`/`summary_col`. These rails once served English to a Hungarian reader.
def display_title(row: PlexItem | PlexShow, lang: str) -> str:
return (row.title_hu or row.title) if lang == "hu" else row.title
def display_summary(row: PlexItem | PlexShow, lang: str) -> str | None:
return (row.summary_hu or row.summary) if lang == "hu" else row.summary
def movie_card(it: PlexItem, st: PlexState | None, *, lang: str) -> dict:
return {
"id": it.rating_key,
"type": "movie",
"title": it.title,
"title": display_title(it, lang),
"year": it.year,
"rating": it.rating,
"summary": _card_summary(it.summary),
"summary": _card_summary(display_summary(it, lang)),
"duration_seconds": it.duration_s,
"thumb": f"/api/plex/image/{it.rating_key}",
"art": _art_url(it.rating_key, it.art_key),
@@ -39,14 +54,14 @@ def movie_card(it: PlexItem, st: PlexState | None) -> dict:
}
def show_card(sh: PlexShow, status: str = "new") -> dict:
def show_card(sh: PlexShow, status: str = "new", *, lang: str) -> dict:
return {
"id": sh.rating_key,
"type": "show",
"title": sh.title,
"title": display_title(sh, lang),
"year": sh.year,
"rating": sh.rating,
"summary": _card_summary(sh.summary),
"summary": _card_summary(display_summary(sh, lang)),
"thumb": f"/api/plex/image/{sh.rating_key}",
"art": _art_url(sh.rating_key, sh.art_key),
"season_count": sh.child_count,
@@ -55,12 +70,12 @@ def show_card(sh: PlexShow, status: str = "new") -> dict:
}
def episode_card(e: PlexItem, st: PlexState | None) -> dict:
def episode_card(e: PlexItem, st: PlexState | None, *, lang: str) -> dict:
return {
"id": e.rating_key,
"type": "episode",
"title": e.title,
"summary": e.summary,
"title": display_title(e, lang),
"summary": display_summary(e, lang),
"season_number": e.season_number,
"episode_number": e.episode_number,
"duration_seconds": e.duration_s,
@@ -77,12 +92,15 @@ def leaf_card(
st: PlexState | None,
show_title: str | None = None,
show_rk: str | None = None,
*,
lang: str,
) -> dict:
"""A playable-leaf card (movie or episode). Used by playlists, which can mix both kinds.
`show_rk` (the show's rating_key) lets the client group a playlist's episodes by show/season."""
`show_rk` (the show's rating_key) lets the client group a playlist's episodes by show/season.
`show_title` is the caller's, already in `lang` (see `display_title`)."""
if it.kind == "episode":
card = episode_card(it, st)
card = episode_card(it, st, lang=lang)
card["show_title"] = show_title
card["show_id"] = show_rk
return card
return movie_card(it, st)
return movie_card(it, st, lang=lang)
+100
View File
@@ -0,0 +1,100 @@
"""One Hungarian word per Plex genre (E30).
Plex's Hungarian agent returns a MIXED vocabulary: some genres come back translated ("Dráma",
"Vígjáték"), others stay English ("Action", "Adventure", "Biography"), and a title with no HU twin has
only its English genres. The HU genre list therefore offered "Drama" beside "Dráma" and a dozen English
words. This map sends every genre word we have met — English (Plex EN agent, TMDB, IMDb) or Plex's
Hungarian — to the ONE word a Hungarian reader sees. The overlay sync writes it into `genres_hu`, and a
genre filter is read through it too, so a saved filter still holding "Action" keeps matching "Akció".
A word not in the map passes through unchanged: an unknown genre stays visible (in whatever language
Plex gave it) rather than disappearing.
"""
from collections.abc import Iterable
# Keys are matched case-insensitively (see `_BY_KEY`). The Hungarian values double as keys mapping to
# themselves, so running a list through twice changes nothing.
_HU: dict[str, str] = {
"Action": "Akció",
"Action & Adventure": "Akció és kaland",
"Adventure": "Kaland",
"Animation": "Animációs",
"Anime": "Anime",
"Biography": "Életrajzi",
"Children": "Gyerek",
"Comedy": "Vígjáték",
"Crime": "Bűnügyi",
"Documentary": "Dokumentum",
"Drama": "Dráma",
"Family": "Családi",
"Fantasy": "Fantasy",
"Film-Noir": "Film noir",
"Food": "Gasztronómia",
"Game Show": "Vetélkedő",
"Game-Show": "Vetélkedő",
"History": "Történelmi",
"Home and Garden": "Otthon és kert",
"Horror": "Horror",
"Indie": "Független",
"Kids": "Gyerek",
"Martial Arts": "Harcművészet",
"Mini-Series": "Minisorozat",
"Music": "Zenei",
"Musical": "Musical",
"Mystery": "Rejtély",
"News": "Hírek",
"Reality": "Valóságshow",
"Reality-TV": "Valóságshow",
"Romance": "Romantikus",
"Sci-Fi": "Sci-Fi",
"Sci-Fi & Fantasy": "Sci-Fi és fantasy",
"Science Fiction": "Sci-Fi",
"Short": "Rövidfilm",
"Soap": "Szappanopera",
"Sport": "Sport",
"Suspense": "Feszültség",
"Talk": "Talkshow",
"Talk Show": "Talkshow",
"Thriller": "Thriller",
"Travel": "Utazás",
"TV Movie": "TV film",
"War": "Háborús",
"War & Politics": "Háború és politika",
"Western": "Western",
# Plex's own Hungarian words, pinned to themselves (and to the spelling above where they differ).
"Akció": "Akció",
"Animációs": "Animációs",
"Bűnügyi": "Bűnügyi",
"Családi": "Családi",
"Dokumentum": "Dokumentum",
"Dokumentumfilm": "Dokumentum",
"Dráma": "Dráma",
"Életrajzi": "Életrajzi",
"Háborús": "Háborús",
"Kaland": "Kaland",
"Rejtély": "Rejtély",
"Romantikus": "Romantikus",
"Történelmi": "Történelmi",
"TV film": "TV film",
"Vígjáték": "Vígjáték",
"Zene": "Zenei",
"Zenei": "Zenei",
}
_BY_KEY = {k.casefold(): v for k, v in _HU.items()}
def hu_genre(word: str) -> str:
"""The Hungarian word for one genre; an unknown word comes back unchanged."""
return _BY_KEY.get(word.strip().casefold(), word)
def hu_genres(words: Iterable[str] | None) -> list[str] | None:
"""A genre list in Hungarian, in its first-seen order, each word once ("Drama" + "Dráma" → one
"Dráma"). None or empty stays None, like the listing's own `_tags`."""
out: list[str] = []
for w in words or ():
hu = hu_genre(w)
if hu not in out:
out.append(hu)
return out or None
+63 -3
View File
@@ -8,6 +8,9 @@ few hundred MB at most for a big library — acceptable, matching the download d
The cache directory (``cache.CACHE_DIR``) is shared with the subtitle cache (see ``subtitles.py``).
"""
import time
from datetime import timedelta
from fastapi import Response
from sqlalchemy.orm import Session
@@ -20,9 +23,23 @@ log = logs.get_logger(__name__)
_EXT_CT = [("jpg", "image/jpeg"), ("png", "image/png"), ("webp", "image/webp")]
_CT_EXT = {ct: ext for ext, ct in _EXT_CT}
# Target box per variant — Plex resizes to these server-side. Posters fill ≤176px cells (grid/info/
# strips), so 400px covers 2× DPR; the faint backdrop art only needs ~1280px. Keeps images ~tens of KB.
SIZES = {"thumb": (400, 600), "art": (1280, 720)}
# Target box per variant — Plex resizes to these server-side. The full poster (400px) covers a detail
# page's ≤224px poster at 2× DPR, the full art (1280px) a hero backdrop. A card is much smaller: a grid
# poster is ~144px and a shelf's landscape art ~212px, so the `_sm` pair serves those at 1.33× DPR
# (E31 S4e: cards fetched the 400 poster, 42 KB, and the 1280 art for a 212px picture). The browser
# picks between a pair itself (`srcset`, built by the frontend's `lib/plexImage` cardImage, whose
# widths plexImage.contract.test holds to these).
SIZES = {
"thumb": (400, 600),
"thumb_sm": (240, 360),
"art": (1280, 720),
"art_sm": (480, 270),
}
def picture(variant: str) -> str:
"""Which stored picture a size variant is cut from: the poster (`thumb`) or the backdrop (`art`)."""
return "art" if variant.startswith("art") else "thumb"
def key_for(db: Session, rating_key: str, variant: str) -> str | None:
@@ -56,6 +73,49 @@ def read(key: str) -> tuple[bytes, str] | None:
return None
def has(key: str) -> bool:
"""A cached, non-empty file exists for `key` (a prefetch's cheap check — no read)."""
for ext, _ in _EXT_CT:
try:
if (CACHE_DIR / f"{key}.{ext}").stat().st_size > 0:
return True
# silent-ok: a miss, as in read().
except OSError:
pass
return False
def drop(key: str) -> None:
"""Remove `key`'s file, whatever its extension (a superseded variant). Best-effort."""
for ext, _ in _EXT_CT:
try:
(CACHE_DIR / f"{key}.{ext}").unlink(missing_ok=True)
# silent-ok: a leftover original only costs disk; the small twin is already written.
except OSError:
pass
def mark_miss(key: str) -> None:
"""Remember that `key`'s source could not be fetched (an empty `.miss` file; its mtime is when), so
a prefetch can leave a dead source alone for a while. Best-effort."""
try:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
(CACHE_DIR / f"{key}.miss").touch()
# silent-ok: without the mark the dead source is only asked for again on the next run.
except OSError:
pass
def missed_within(key: str, age: timedelta) -> bool:
"""`key` was marked as a miss less than `age` ago."""
try:
mtime = (CACHE_DIR / f"{key}.miss").stat().st_mtime
# silent-ok: no mark (the usual case).
except OSError:
return False
return time.time() - mtime < age.total_seconds()
def write(key: str, data: bytes, ct: str) -> None:
ext = _CT_EXT.get((ct or "").split(";")[0].strip(), "jpg")
try:
+176 -10
View File
@@ -3,13 +3,23 @@ or show (cast_names + the people_text search blob + the plex_people headshot cac
orphaned people. Used by BOTH the info-page lazy enrichment (routes.plex) and the background rich
re-sync (plex.sync), so the two paths never diverge."""
from urllib.parse import quote
import hashlib
import re
from datetime import timedelta
from typing import Literal
from urllib.parse import quote, unquote
import httpx
from sqlalchemy import select, text
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.orm import Session
from app import logs, sysconfig
from app.models import PlexPerson
from app.plex import imgcache
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
log = logs.get_logger(__name__)
# Cast/crew photos come from Plex's PUBLIC metadata CDN (no token). We proxy them (host-whitelisted)
# so the user's browser makes no third-party request (privacy + self-host CSP friendliness).
@@ -19,9 +29,7 @@ PERSON_IMG_HOST = "https://metadata-static.plex.tv/"
def person_thumb_url(thumb: str | None) -> str | None:
"""Proxied /person-image URL for a Plex CDN headshot, or None when it's missing / off-CDN."""
t = str(thumb or "")
return (
f"/api/plex/person-image?u={quote(t, safe='')}" if t.startswith(PERSON_IMG_HOST) else None
)
return f"/api/plex/person-image?u={quote(t, safe='')}" if is_portrait_url(t) else None
def cast_from_meta(meta: dict, limit: int = 24) -> list[dict]:
@@ -89,14 +97,23 @@ def enrich_row_people(db: Session, row, cast: list[dict]) -> bool:
return dirty
# Every credit reference: one person name per (movie/show, credit column). The ONE list of credit
# columns, so the orphan prune and the enrichment order never disagree about who is credited.
_CREDIT_COLUMNS = [
(table, col) for table in ("plex_items", "plex_shows") for col in ("cast_names", "directors")
]
def _credit_refs_sql(union: Literal["UNION", "UNION ALL"]) -> str:
return f" {union} ".join(
f"SELECT jsonb_array_elements_text({col}) FROM {table} WHERE jsonb_typeof({col}) = 'array'"
for table, col in _CREDIT_COLUMNS
)
# The person names still referenced by ANY movie/show (cast OR directors). plex_people rows outside
# this set are orphans left by a library removal.
_REFERENCED_PEOPLE_SQL = """
SELECT jsonb_array_elements_text(cast_names) FROM plex_items WHERE jsonb_typeof(cast_names) = 'array'
UNION SELECT jsonb_array_elements_text(directors) FROM plex_items WHERE jsonb_typeof(directors) = 'array'
UNION SELECT jsonb_array_elements_text(cast_names) FROM plex_shows WHERE jsonb_typeof(cast_names) = 'array'
UNION SELECT jsonb_array_elements_text(directors) FROM plex_shows WHERE jsonb_typeof(directors) = 'array'
"""
_REFERENCED_PEOPLE_SQL = _credit_refs_sql("UNION")
def prune_orphan_people(db: Session) -> int:
@@ -113,3 +130,152 @@ def prune_orphan_people(db: Session) -> int:
)
db.commit()
return res.rowcount or 0
# How often each person is credited across movies, shows and episodes (cast OR director) — the order the
# people_enrich job works in, so the faces met most often are local first.
_CREDIT_COUNTS_SQL = (
f"SELECT ref, count(*) AS n FROM ({_credit_refs_sql('UNION ALL')}) AS refs(ref) GROUP BY ref"
)
_NEEDS = {
# linked to TMDB, bio never fetched
"bio": "p.tmdb_person_id IS NOT NULL AND p.tmdb_enriched_at IS NULL",
# has a Plex headshot (whether it is cached is the disk's answer, not the row's)
"portrait": "p.photo_thumb IS NOT NULL",
}
def people_by_credits(
db: Session, need: Literal["bio", "portrait"], limit: int | None
) -> list[int]:
"""plex_people ids that `need` something, the most-credited first (ties by id)."""
rows = db.execute(
text(
f"SELECT p.id FROM plex_people p LEFT JOIN ({_CREDIT_COUNTS_SQL}) c ON c.ref = p.name "
f"WHERE {_NEEDS[need]} ORDER BY coalesce(c.n, 0) DESC, p.id"
+ (" LIMIT :limit" if limit is not None else "")
),
{"limit": limit} if limit is not None else {},
)
return [r[0] for r in rows]
# --- Portraits (E30 S6) ---------------------------------------------------------------------------
# A Plex CDN headshot is a full-size original (~170 KB on average, measured). Plex's photo transcoder
# fits it into this box (~13 KB), which covers the largest place a portrait is drawn (the person page's
# 6rem avatar) at 2x density and a raised text size. The box is part of the cache key, like a poster's:
# a new box re-caches instead of serving a stale size. Only a resized image is ever written under it —
# an original fetched without Plex keeps its own key, and goes once its small twin lands.
PORTRAIT_BOX = 320
# A headshot Plex answered "not found" for (a dead CDN file) is not asked for again this long, so a
# few dead faces never hold up the rest of the prefetch.
PORTRAIT_RETRY = timedelta(days=7)
# The one shape a Plex CDN headshot has (…/7/people/<hash>.jpg): plain path segments, no query, no
# fragment, no dot segment or empty segment (an HTTP client folds `a/../b` and the CDN `a//b` onto one
# file). The person-image proxy takes nothing else, so it cannot be made to cache endless spellings of
# one file (the image cache has no eviction). Used where a stored headshot is made, served and warmed.
_SEGMENT = r"[A-Za-z0-9_-]+(?:\.[A-Za-z0-9_-]+)*" # a name, dots only between its parts
_PORTRAIT_URL = re.compile(re.escape(PERSON_IMG_HOST) + _SEGMENT + rf"(?:/{_SEGMENT})*")
def is_portrait_url(url: str) -> bool:
return _PORTRAIT_URL.fullmatch(url) is not None
def _portrait_keys(url: str) -> tuple[str, str]:
"""(the resized portrait's key, the full-size original's key)."""
base = "person_" + hashlib.sha1(url.encode()).hexdigest()
return f"{base}_{PORTRAIT_BOX}", base
def portrait_cached(url: str) -> tuple[bytes, str] | None:
"""The cached portrait: the resized one, else an original kept from a time without Plex."""
small, original = _portrait_keys(url)
return imgcache.read(small) or imgcache.read(original)
def cache_portrait(plex: PlexClient | None, url: str) -> tuple[bytes, str]:
"""Fetch one CDN headshot and cache it. With a Plex client it is resized (the box key; the original
is removed); without one the original comes straight from the public CDN and is cached under its own
key, so the job still resizes it once Plex is there. Raises PlexError / httpx.HTTPError."""
small, original = _portrait_keys(url)
if plex is None:
with httpx.Client(timeout=10.0, follow_redirects=True) as c:
r = c.get(url)
r.raise_for_status()
data, ct = r.content, r.headers.get("content-type", "image/jpeg")
imgcache.write(original, data, ct)
return data, ct
data, ct = plex.image_bytes(url, PORTRAIT_BOX, PORTRAIT_BOX)
imgcache.write(small, data, ct)
imgcache.drop(original)
return data, ct
def cdn_url(photo_thumb: str) -> str | None:
"""The CDN url inside a stored proxied `/person-image?u=` value (see person_thumb_url)."""
_, _, q = photo_thumb.partition("?u=")
url = unquote(q)
return url if is_portrait_url(url) else None
# Plex failing this many headshots in a row ends a run: it is Plex that is failing, not the files.
_PLEX_DOWN_AFTER = 3
# The statuses that mean the one file is gone.
_DEAD = frozenset({404, 410})
def warm_portraits(db: Session, limit: int) -> dict:
"""Cache up to `limit` not-yet-cached headshots, the most-credited people first (E30 S6), so a
person's face comes from local disk. Needs Plex (the resize) and the Plex module on; without them
nothing is prefetched — full-size originals for every person would cost gigabytes. A headshot Plex
answers "not found" for is marked and left for PORTRAIT_RETRY; Plex failing in a row ends the run."""
if not sysconfig.get_bool(db, "plex_enabled"):
return {"cached": 0, "skipped": "disabled"}
try:
plex = PlexClient(db)
# silent-ok: no Plex server set up is a setting, not a failure; the run's result says "disabled".
except PlexNotConfigured:
return {"cached": 0, "skipped": "disabled"}
photos: dict[int, str] = {
pid: thumb
for pid, thumb in db.execute(
select(PlexPerson.id, PlexPerson.photo_thumb).where(PlexPerson.photo_thumb.is_not(None))
).tuples()
if thumb
}
# `down` counts Plex failing in a row (any answer from Plex breaks it), `plex_failed` in all
done = dead = plex_failed = down = 0
with plex:
for pid in people_by_credits(db, "portrait", None):
if done + dead >= limit or down >= _PLEX_DOWN_AFTER:
break
url = cdn_url(photos.get(pid) or "")
if url is None:
continue
key = _portrait_keys(url)[0]
if imgcache.has(key) or imgcache.missed_within(key, PORTRAIT_RETRY):
continue
try:
cache_portrait(plex, url)
done, down = done + 1, 0
except PlexError as exc:
# "Not found" is Plex's answer about this one file (a dead CDN headshot: measured 404,
# the CDN itself says 403). Anything else — no answer, a refused token, a 5xx — is
# Plex failing, which must neither mark the file nor keep the run going.
cause = exc.__cause__
if isinstance(cause, httpx.HTTPStatusError) and cause.response.status_code in _DEAD:
dead, down = dead + 1, 0 # Plex answered: it is up, the streak is broken
imgcache.mark_miss(key)
else:
plex_failed, down = plex_failed + 1, down + 1
log.warning(
"a portrait could not be cached",
extra=logs.event(
"plex.people.portrait_failed",
person_id=pid,
reason=type(exc.__cause__).__name__,
),
)
failed = dead + plex_failed
return {"cached": done, **({"failed": failed} if failed else {})}
+8 -1
View File
@@ -11,6 +11,7 @@ from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Session
from app.models import PlexItem, PlexLibrary
from app.plex.genres import hu_genre
TS_CONFIG = "public.unaccent_simple"
@@ -183,7 +184,13 @@ def meta_filter_conds(model, p: dict, lang: str = "en") -> list:
gsel = split_csv(p.get("genres"))
if gsel:
gcol = genres_col(model, lang)
gc = [gcol.contains([g]) for g in gsel]
if lang == "hu":
# The HU column holds one Hungarian word per genre (plex/genres.py), so a filter saved while
# it offered "Action" is read as "Akció" too. The word as picked still matches as well: a row
# the sync has not reached since keeps its English (or Plex's mixed) list until it does.
gc = [or_(*(gcol.contains([w]) for w in dict.fromkeys((hu_genre(g), g)))) for g in gsel]
else:
gc = [gcol.contains([g]) for g in gsel]
conds.append(and_(*gc) if p.get("genre_mode") == "all" else or_(*gc))
crs = split_csv(p.get("content_ratings"))
if crs:
+64 -19
View File
@@ -15,6 +15,7 @@ dead `enabled` column) is a separate follow-up. Cast + intro/credit markers are
item-detail/player path (P2), not here.
"""
from collections.abc import Iterable
from datetime import datetime, timezone
from sqlalchemy import func, select
@@ -24,6 +25,7 @@ from app import logs, sysconfig
from app.models import PlexCollection, PlexItem, PlexLibrary, PlexSeason, PlexShow
from app.plex import detail, people
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
from app.plex.genres import hu_genres
from app.plex.lookups import by_rating_key
log = logs.get_logger(__name__)
@@ -448,6 +450,10 @@ def _apply_facet_fields(row, meta: dict) -> None:
row.studio = (meta.get("studio") or None) and str(meta.get("studio"))[:255]
row.originally_available_at = _date(meta.get("originallyAvailableAt"))
row.genres = _tags(meta, "Genre")
# Every row has Hungarian genres, not only one with an HU overlay twin (the overlay pass, which runs
# after this, puts the twin's own list over it): the HU genre facets and filter read this column,
# and an English word there could never match a filter read in Hungarian.
row.genres_hu = hu_genres(row.genres)
row.directors = _tags(meta, "Director")
# The listing carries only the top ~20 cast, so re-applying it would DOWNGRADE an already-enriched
# full cast (up to 24, grown by the rich backfill/info-page enrich), silently breaking the
@@ -659,35 +665,72 @@ def _sync_overlay(
db: Session, plex: PlexClient, lib: PlexLibrary, en_lib_id: int, stats: dict
) -> None:
"""Write an overlay section's Hungarian metadata onto its EN twin's rows, matched by guid. Creates
NO browsable rows — an overlay library is never listed on its own. Only title/summary/genres differ
by language; the language-neutral people_text feeds both search vectors.
NO browsable rows — an overlay library is never listed on its own. Only title/summary/tagline/genres
differ by language; the language-neutral people_text feeds both search vectors. A show section's
episodes are overlaid too (E30): the HU section lists every episode with its Hungarian title and
summary, and the episode rows live in the EN show library.
This is a full RECONCILE: every guid-bearing EN row's HU fields are cleared first, then the HU
section repopulates the matches — so an EN row whose HU twin vanished (or was re-guid'd) falls back
to EN instead of keeping stale Hungarian metadata. HU items with no EN match are skipped."""
if lib.kind == "movie":
rows = db.execute(
select(PlexItem).where(
PlexItem.library_id == en_lib_id,
PlexItem.kind == "movie",
PlexItem.guid.is_not(None),
passes = [
(
_TYPE_MOVIE,
select(PlexItem).where(
PlexItem.library_id == en_lib_id,
PlexItem.kind == "movie",
PlexItem.guid.is_not(None),
),
)
).scalars()
item_type = _TYPE_MOVIE
]
else:
rows = db.execute(
select(PlexShow).where(PlexShow.library_id == en_lib_id, PlexShow.guid.is_not(None))
).scalars()
item_type = _TYPE_SHOW
passes = [
(
_TYPE_SHOW,
select(PlexShow).where(
PlexShow.library_id == en_lib_id, PlexShow.guid.is_not(None)
),
),
(
_TYPE_EPISODE,
select(PlexItem).where(
PlexItem.library_id == en_lib_id,
PlexItem.kind == "episode",
PlexItem.guid.is_not(None),
),
),
]
for item_type, rows_q in passes:
_overlay_pass(db, plex, lib, item_type, db.execute(rows_q).scalars(), stats)
db.flush()
def _overlay_pass(
db: Session,
plex: PlexClient,
lib: PlexLibrary,
item_type: int,
rows: Iterable[PlexItem] | Iterable[PlexShow],
stats: dict,
) -> None:
"""One kind of the overlay (movies, shows or episodes) — see `_sync_overlay`."""
# guid -> the EN rows carrying it (a list: two editions of one film can share a guid; both overlay).
by_guid: dict[str, list] = {}
by_guid: dict[str, list[PlexItem | PlexShow]] = {}
for r in rows:
by_guid.setdefault(r.guid, []).append(r)
if r.guid: # the passes select guid-bearing rows only
by_guid.setdefault(r.guid, []).append(r)
# Reconcile step: clear HU fields on all guid-bearing rows. Only rows that actually held HU data
# become dirty (SQLAlchemy skips a no-net-change UPDATE), so this is cheap; matches are re-set below.
# The genres are the exception: every row gets Hungarian ones, from its HU twin or, lacking one, from
# its own English list — otherwise the HU genre list mixed "Drama" (no twin) with "Dráma".
episodes = item_type == _TYPE_EPISODE
for group in by_guid.values():
for row in group:
row.title_hu = row.summary_hu = row.genres_hu = None
row.title_hu = row.summary_hu = None
if not episodes:
row.tagline_hu = None
row.genres_hu = hu_genres(row.genres)
seen: set[str] = (
set()
) # first HU entry for a guid wins; a dup HU item can't overwrite or double-count
@@ -699,13 +742,15 @@ def _sync_overlay(
seen.add(g)
title_hu = (meta.get("title") or "").strip() or None
summary_hu = meta.get("summary")
genres_hu = _tags(meta, "Genre")
tagline_hu = (meta.get("tagline") or "").strip() or None
genres_hu = hu_genres(_tags(meta, "Genre"))
for row in targets:
row.title_hu = title_hu
row.summary_hu = summary_hu
row.genres_hu = genres_hu
if not episodes:
row.tagline_hu = tagline_hu
row.genres_hu = genres_hu or row.genres_hu
stats["overlaid"] = stats.get("overlaid", 0) + 1
db.flush()
def _upsert_collection(
+180 -27
View File
@@ -10,8 +10,10 @@ One-parser rule (mirrors app/plex/detail.py): `parse_tmdb` is the pure parser, `
the row. The enrich batch + scheduler entry live here too so the whole TMDB integration is one module.
"""
import re
import time
from datetime import datetime, timezone
from typing import cast
import httpx
from sqlalchemy import func, select
@@ -21,6 +23,8 @@ from sqlalchemy.orm import Session
from app import logs, sysconfig
from app.config import settings
from app.models import PlexItem, PlexPerson, PlexShow
from app.plex.cards import display_title
from app.plex.people import people_by_credits, warm_portraits
log = logs.get_logger(__name__)
# Politeness throttle: TMDB tolerates ~50 req/s; ~33/s leaves comfortable headroom for a shared host.
@@ -124,9 +128,17 @@ class TmdbClient:
"""Resolve an external id (imdb_id / tvdb_id) to TMDB items via /find."""
return self._get(f"/find/{external_id}", {"external_source": source})
def countries(self) -> list[dict] | None:
"""TMDB's country list: `[{iso_3166_1, english_name, native_name}]` (a JSON array, which
`_get`'s dict annotation does not describe)."""
return cast("list[dict] | None", self._get("/configuration/countries"))
def fetch_person(self, person_id: int) -> dict | None:
"""One person's bio + combined credits (E5 S6), in a single append_to_response request."""
return self._get(f"/person/{person_id}", {"append_to_response": "combined_credits"})
"""One person's bio + combined credits (E5 S6) + the bio's translations (E30 S4), in a single
append_to_response request."""
return self._get(
f"/person/{person_id}", {"append_to_response": "combined_credits,translations"}
)
def guid_to_tmdb_id(guid: str | None, media_type: str, tmdb: TmdbClient) -> int | None:
@@ -254,8 +266,12 @@ def parse_tmdb(payload: dict, media_type: str) -> dict:
c.get("name") for c in (payload.get("production_companies") or []) if c.get("name")
][:8]
or None,
# The ISO code beside TMDB's English name (E30): the client names the country in the reader's
# language from the code; the name is the fallback for a code it cannot name.
"production_countries": [
c.get("name") for c in (payload.get("production_countries") or []) if c.get("name")
{"code": c.get("iso_3166_1") or None, "name": c["name"]}
for c in (payload.get("production_countries") or [])
if c.get("name")
]
or None,
"recommendations": recs or None,
@@ -301,8 +317,8 @@ def _link_people(db: Session, links: dict[str, int]) -> None:
def enrich_batch(db: Session, tmdb: TmdbClient, limit: int) -> int:
"""Enrich up to `limit` not-yet-TMDB-enriched titles (movies first, shows fill the remainder —
mirrors _enrich_people_batch). A transient TmdbError leaves the watermark NULL for retry; a
"""Enrich up to `limit` not-yet-TMDB-enriched titles (movies first, shows fill the
remainder). A transient TmdbError leaves the watermark NULL for retry; a
definitive no-match (unresolvable guid / TMDB 404) still stamps the watermark so it isn't re-fetched
every run. One commit at the end. Returns the number processed (matched OR definitively unmatched)."""
movies = (
@@ -356,13 +372,62 @@ def enrich_batch(db: Session, tmdb: TmdbClient, limit: int) -> int:
return processed
def _country(c: dict | str) -> dict:
"""One stored production country as served: `{code, name}`. A row enriched before E30 stored the
bare English name; until `backfill_country_codes` reaches it, it serves with no code (the client
shows the name)."""
return c if isinstance(c, dict) else {"code": None, "name": c}
def backfill_country_codes(db: Session, tmdb: TmdbClient) -> int:
"""Give the countries stored before E30 (bare English names) their ISO codes, from TMDB's own
country list — the names came from that list, so they match it exactly. One request, and only
while such a row is left: re-fetching each title instead would take the enrichment batch days.
A name the list does not hold keeps no code. Returns the number of rows rewritten."""
stale = [
row
for model in (PlexItem, PlexShow)
for row in db.execute(
select(model).where(func.jsonb_typeof(model.production_countries[0]) == "string")
).scalars()
]
if not stale:
return 0
codes = {
c["english_name"]: c["iso_3166_1"]
for c in tmdb.countries() or []
if c.get("english_name") and c.get("iso_3166_1")
}
for row in stale:
row.production_countries = [
{"code": codes.get(c), "name": c} if isinstance(c, str) else c
for c in row.production_countries or []
]
db.commit()
return len(stale)
def run_enrich(db: Session) -> dict:
"""Scheduler entry point. No-op (never an error) when the TMDB key isn't configured."""
if not sysconfig.get_str(db, "tmdb_api_key"):
return {"enriched": 0, "skipped": "tmdb_api_key not set"}
with TmdbClient(db) as tmdb:
coded = backfill_country_codes(db, tmdb)
n = enrich_batch(db, tmdb, settings.tmdb_enrich_batch)
return {"enriched": n}
return {"enriched": n, **({"countries_coded": coded} if coded else {})}
def run_people_enrich(db: Session) -> dict:
"""Scheduler entry point of people_enrich (E30 S6): a batch of person bios (the most-credited
first) and a batch of portraits. Each half is a no-op — never an error — without its TMDB key /
without Plex."""
batch = settings.people_enrich_batch
if sysconfig.get_str(db, "tmdb_api_key"):
with TmdbClient(db) as tmdb:
bios = enrich_people_batch(db, tmdb, batch)
else:
bios = {"skipped": "tmdb_api_key not set"}
return {"bios": bios, "portraits": warm_portraits(db, batch)}
def resolve_library_guids(
@@ -406,10 +471,11 @@ def resolve_library_guids(
return movies, shows
def resolve_recs(db: Session, recs: list[dict] | None) -> list[dict]:
def resolve_recs(db: Session, recs: list[dict] | None, *, lang: str) -> list[dict]:
"""Tag each recommendation with whether it's in OUR library (matched by guid `tmdb:<id>`, bucketed
by media_type — see resolve_library_guids) and, if so, its rating_key — so the client can deep-link
the ones we have and merely display the rest."""
the ones we have and merely display the rest. One we have is named as the library names it, in
`lang` (E30); one we lack keeps the title TMDB gave."""
if not recs:
return []
movies, shows = resolve_library_guids(db, recs)
@@ -417,13 +483,19 @@ def resolve_recs(db: Session, recs: list[dict] | None) -> list[dict]:
for r in recs:
g = f"tmdb:{r.get('tmdb_id')}"
row = shows.get(g) if r.get("media_type") == "tv" else movies.get(g)
named = {"title": display_title(row, lang)} if row is not None else {}
out.append(
{**r, "in_library": row is not None, "rating_key": row.rating_key if row else None}
{
**r,
**named,
"in_library": row is not None,
"rating_key": row.rating_key if row else None,
}
)
return out
def serve_block(db: Session, row) -> dict | None:
def serve_block(db: Session, row, *, lang: str) -> dict | None:
"""The TMDB extras for a movie/show detail response, served from the row (no live TMDB call).
None until the row has been TMDB-enriched (the client then shows nothing extra)."""
if row.tmdb_enriched_at is None:
@@ -437,30 +509,58 @@ def serve_block(db: Session, row) -> dict | None:
"keywords": row.keywords or [],
"crew": row.crew or [],
"studios": row.studios or [],
"production_countries": row.production_countries or [],
"recommendations": resolve_recs(db, row.recommendations),
"production_countries": [_country(c) for c in row.production_countries or []],
"recommendations": resolve_recs(db, row.recommendations, lang=lang),
"backdrops": row.backdrops or [],
}
# --- Person enrichment (E5 S6) -------------------------------------------------------------------
# A person's bio + birth facts + a "Known for" credit pool, fetched once (the first person-page open)
# and served from the plex_people row. `known_for` stores the top combined-credits ids; the person
# A person's bio + birth facts + a "Known for" credit pool, fetched once — by the people_enrich job, or
# by the first person-page open if the job has not reached them yet — and served from the plex_people row. `known_for` stores the top combined-credits ids; the person
# endpoint resolves them to library rows by guid at serve time (like recommendations).
_KNOWN_FOR_MAX = 15
# The candidate pool kept per person. Only the titles that are in the library are shown (at most
# routes.plex.people._KNOWN_FOR_SHOWN), so the pool is deeper than the rail: a star's 15 most-voted
# credits are often not the 15 a given library holds.
_KNOWN_FOR_MAX = 40
# TMDB genre ids of the shows a person visits as a guest (News, Reality, Talk). Ranked by popularity,
# talk-show visits crowded out the films (Tom Cruise: 14 of 15 were talk shows; E30 S6).
_GUEST_GENRES = {10763, 10764, 10767}
# A credit in which the person plays themselves ("Self", "Himself - Guest", "Herself (archive footage)").
_SELF_CREDIT = re.compile(r"^\s*(self|himself|herself|themselves)\b", re.IGNORECASE)
# The crew jobs that make a title the person's own work (a director is known for what they directed).
_KNOWN_FOR_CREW_JOBS = {"Director", "Creator"}
def _known_for_credit(c: dict, crew: bool) -> bool:
if crew:
return c.get("job") in _KNOWN_FOR_CREW_JOBS
if _SELF_CREDIT.match(c.get("character") or ""):
return False
return not _GUEST_GENRES.intersection(c.get("genre_ids") or [])
def parse_person(payload: dict) -> dict:
"""TMDB /person/{id} (+combined_credits) → the persisted bio fields. `known_for` = the person's
most-popular movie/tv credits (deduped by id), top `_KNOWN_FOR_MAX`, resolved to library later."""
credits = (payload.get("combined_credits") or {}).get("cast") or []
most-voted movie/tv credits — roles and directed works, not guest visits as themselves — deduped,
the top `_KNOWN_FOR_MAX`, resolved to library rows at serve time. Ranked by vote count, as TMDB's
own "Known for" is: popularity is this week's attention, which a talk-show visit wins."""
combined = payload.get("combined_credits") or {}
credits = [c for c in combined.get("cast") or [] if _known_for_credit(c, crew=False)] + [
c for c in combined.get("crew") or [] if _known_for_credit(c, crew=True)
]
# Dedup on (id, media_type), not the bare id: TMDB movie ids and tv ids are independent sequences
# (id 603 can be both a movie and a show — see resolve_library_guids), so a same-id movie+show pair
# is two distinct credits and both must survive.
seen: set[tuple[int, str]] = set()
known: list[dict] = []
for c in sorted(credits, key=lambda c: c.get("popularity") or 0, reverse=True):
ranked = sorted(
credits,
key=lambda c: (c.get("vote_count") or 0, c.get("popularity") or 0),
reverse=True,
)
for c in ranked:
tid, mt = c.get("id"), c.get("media_type")
if not tid or mt not in ("movie", "tv") or (tid, mt) in seen:
continue
@@ -470,6 +570,7 @@ def parse_person(payload: dict) -> dict:
break
return {
"biography": (payload.get("biography") or "").strip() or None,
"biography_hu": _translated_bio(payload, "hu"),
"birthday": payload.get("birthday") or None,
"deathday": payload.get("deathday") or None,
"place_of_birth": payload.get("place_of_birth") or None,
@@ -479,8 +580,17 @@ def parse_person(payload: dict) -> dict:
}
def _translated_bio(payload: dict, lang: str) -> str | None:
"""The `lang` biography from the appended `translations` (TMDB leaves an untranslated one blank)."""
for tr in (payload.get("translations") or {}).get("translations") or []:
if tr.get("iso_639_1") == lang:
return ((tr.get("data") or {}).get("biography") or "").strip() or None
return None
def apply_person(row: PlexPerson, parsed: dict) -> None:
row.biography = parsed["biography"]
row.biography_hu = parsed["biography_hu"]
row.birthday = parsed["birthday"]
row.deathday = parsed["deathday"]
row.place_of_birth = parsed["place_of_birth"]
@@ -490,11 +600,22 @@ def apply_person(row: PlexPerson, parsed: dict) -> None:
row.tmdb_enriched_at = datetime.now(timezone.utc)
def _fetch_person_into(tmdb: TmdbClient, person: PlexPerson) -> None:
"""Fetch one person from TMDB and write the row (not committed). A 404 (deleted person) stamps the
watermark with empty fields so it isn't fetched again. Raises TmdbError on a transient failure,
leaving the row untouched and `tmdb_enriched_at` NULL for a retry."""
payload = tmdb.fetch_person(cast(int, person.tmdb_person_id))
if payload is None:
person.tmdb_enriched_at = datetime.now(timezone.utc)
else:
apply_person(person, parse_person(payload))
def enrich_person(db: Session, person: PlexPerson) -> bool:
"""Lazily fetch + persist one person's TMDB bio (first person-page open), committing the write so it
survives regardless of the read path. No-op (False) without a key or a `tmdb_person_id`; a transient
error leaves `tmdb_enriched_at` NULL so the next open retries. A 404 (deleted person) stamps the
watermark with empty fields so we don't refetch it every open."""
"""Lazily fetch + persist one person's TMDB bio (a person page opened before the background job
reached them), committing the write so it survives regardless of the read path. No-op (False)
without a key or a `tmdb_person_id`; a transient error leaves `tmdb_enriched_at` NULL so the next
open retries."""
if not person.tmdb_person_id or not sysconfig.get_str(db, "tmdb_api_key"):
return False
try:
@@ -502,7 +623,7 @@ def enrich_person(db: Session, person: PlexPerson) -> bool:
# longer blocks a serve; still cap the outbound TMDB call with a short timeout so a hanging TMDB
# doesn't tie up a threadpool worker for the default 30s.
with TmdbClient(db, timeout=8.0) as tmdb:
payload = tmdb.fetch_person(person.tmdb_person_id)
_fetch_person_into(tmdb, person)
except TmdbError as exc:
# False leaves the person un-enriched and unstamped, so it is retried — but the page in
# front of the user right now shows an empty biography with nothing to say why.
@@ -515,9 +636,41 @@ def enrich_person(db: Session, person: PlexPerson) -> bool:
),
)
return False
if payload is None:
person.tmdb_enriched_at = datetime.now(timezone.utc)
else:
apply_person(person, parse_person(payload))
db.commit()
return True
# Lookups failing this many people in a row end a people batch: TMDB itself is degraded, and hammering
# it helps nobody. One person failing alone is skipped, so an id TMDB keeps refusing never stalls the rest.
_PEOPLE_FAILS_IN_A_ROW = 3
def enrich_people_batch(db: Session, tmdb: TmdbClient, limit: int) -> dict:
"""Fetch up to `limit` linked, not-yet-enriched people (E30 S6), the most-credited first, so a
person page reads its bio from the row instead of waiting on TMDB. Each person is committed as it
lands: a page open during the run already sees it. A failed lookup is skipped (left unstamped for the
next run); _PEOPLE_FAILS_IN_A_ROW of them in a row end the batch."""
ids = people_by_credits(db, "bio", limit)
done = failed = in_a_row = 0
for pid in ids:
if in_a_row >= _PEOPLE_FAILS_IN_A_ROW:
break
person = db.get(PlexPerson, pid)
# A page open may have enriched this person since the select: skip, don't fetch twice.
if person is None or person.tmdb_enriched_at is not None:
continue
try:
_fetch_person_into(tmdb, person)
except TmdbError as exc:
db.rollback()
failed, in_a_row = failed + 1, in_a_row + 1
log.warning(
"a TMDB person lookup in the batch failed",
extra=logs.event(
"plex.tmdb.people_batch_failed", person_id=pid, reason=type(exc).__name__
),
)
continue
db.commit()
done, in_a_row = done + 1, 0
return {"enriched": done, **({"failed": failed} if failed else {})}
+13
View File
@@ -24,6 +24,19 @@ def report(current: int, total: int | None = None, phase: str | None = None) ->
sink(current, total, phase)
class JobFailed(RuntimeError):
"""A job's own failure, said so the reader's language can name it (E31): the client reads
`scheduler.failure.<reason>` and interpolates `params` (keep them to values the reader may see —
numbers, ids, short codes). The message is the English line the log and `last_error` keep. An
upstream's own error (a timeout, a refused connection) is not one of these: it stays as raised,
being diagnostic."""
def __init__(self, reason: str, message: str, **params: object):
super().__init__(message)
self.reason = reason
self.params = params
class bind:
"""Context manager that binds a progress sink for the duration of a job run."""
+5 -1
View File
@@ -834,7 +834,11 @@ def reset_demo(admin: User = Depends(admin_user), db: Session = Depends(get_db))
demo = get_or_create_demo_user(db)
seeded = reset_demo_state(db, demo)
audit.record(
db, admin.id, AuditAction.DEMO_RESET, summary=f"{seeded} sample playlists re-seeded"
db,
admin.id,
AuditAction.DEMO_RESET,
summary=f"{seeded} sample playlists re-seeded",
after={"playlists_seeded": seeded}, # the count as data: named in the reader's language
)
db.commit()
return {"ok": True, "reset": True, "playlists_seeded": seeded}
+7 -1
View File
@@ -66,6 +66,12 @@ def list_audit(
def clear_audit(admin: User = Depends(admin_user), db: Session = Depends(get_db)) -> dict:
"""Clear the audit log. Leaves ONE row recording who cleared it + how many (tamper-evidence)."""
n = db.execute(delete(AuditLog)).rowcount or 0
audit.record(db, admin.id, AuditAction.AUDIT_CLEAR, summary=f"Cleared {n} audit entries")
audit.record(
db,
admin.id,
AuditAction.AUDIT_CLEAR,
summary=f"Cleared {n} audit entries",
after={"cleared": n}, # the count as data: the log names it in the reader's language
)
db.commit()
return {"ok": True, "cleared": n}
+33 -20
View File
@@ -6,9 +6,11 @@ from datetime import datetime, timezone
from fastapi import APIRouter, BackgroundTasks, Depends
from pydantic import BaseModel
from sqlalchemy import func, select, update
from sqlalchemy import Select, func, select, update
from sqlalchemy.orm import Session
from app.collation import collated, sorted_words
from app.mailtext import lang_for_user
from app import logs, quota
from app.auth import admin_user, current_user, require_human, require_write_scope
from app.db import get_db
@@ -38,33 +40,40 @@ router = APIRouter(prefix="/api/channels", tags=["channels"])
log = logs.get_logger(__name__)
def channel_aggregates(channel_ids: list[str]) -> Select:
"""Per-channel stored count, last upload and total duration, in one grouped pass. Every column it
reads is in `ix_videos_channel_pub_cover` (migration 0095), so Postgres answers it from the index
alone; a test holds that (test_channel_list_index_only)."""
return (
select(
Video.channel_id,
func.count(Video.id),
func.max(Video.published_at),
func.coalesce(func.sum(Video.duration_seconds), 0),
)
.where(Video.channel_id.in_(channel_ids))
.group_by(Video.channel_id)
)
@router.get("")
def list_channels(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
rows = db.execute(
select(Subscription, Channel)
.join(Channel, Channel.id == Subscription.channel_id)
.where(Subscription.user_id == user.id)
.order_by(Subscription.priority.desc(), func.lower(Channel.title))
.order_by(Subscription.priority.desc(), collated(Channel.title, lang_for_user(user)))
).all()
# Per-channel aggregates over the shared catalog in ONE grouped pass: stored count, last
# upload, total duration. Cheap — a single grouped scan, so computed on read rather than
# denormalised (measure-first: reads aren't frequent/hot enough to warrant cached columns).
# Per-channel aggregates over the shared catalog (channel_aggregates), computed on read rather than
# denormalised: an Index Only Scan over a ~20 MB index, where it used to read every video's heap page
# (681 MB on prod, a 2–6 s first open once something else had pushed it out of memory, E31 S4d).
# (The Normal/Shorts/Live breakdown moved to the channel-detail page in R14, so it is no longer
# computed here — the manager table dropped its N/S/L column.)
channel_ids = [c.id for _, c in rows]
agg: dict[str, dict] = {}
if channel_ids:
for cid, n, last_up, total_dur in db.execute(
select(
Video.channel_id,
func.count(Video.id),
func.max(Video.published_at),
func.coalesce(func.sum(Video.duration_seconds), 0),
)
.where(Video.channel_id.in_(channel_ids))
.group_by(Video.channel_id)
).all():
for cid, n, last_up, total_dur in db.execute(channel_aggregates(channel_ids)).all():
agg[cid] = {
"stored": n,
"last_video_at": iso(last_up),
@@ -139,7 +148,7 @@ def _discovery_rows(db: Session, user: User) -> list:
return db.execute(
q.group_by(Channel.id).order_by(
func.count(func.distinct(PlaylistItem.video_id)).desc(),
func.lower(Channel.title),
collated(Channel.title, lang_for_user(user)),
)
).all()
@@ -192,10 +201,14 @@ def discover_channels(
):
apply_channel_details(db, yt.get_channels(need), need)
db.commit()
# Match _discovery_rows' ORDER BY exactly: video-count DESC, then title (NULLs LAST,
# as Postgres orders them — coercing NULL→"" would wrongly float untitled channels up).
# Match _discovery_rows' ORDER BY exactly: video-count DESC, then title in the reader's
# alphabet (NULLs LAST, as Postgres orders them — coercing NULL→"" would wrongly float
# untitled channels up).
titles = [r[0].title for r in rows if r[0].title is not None]
alpha = {w: i for i, w in enumerate(sorted_words(db, titles, lang_for_user(user)))}
rows = sorted(
rows, key=lambda r: (-int(r[1]), r[0].title is None, (r[0].title or "").lower())
rows,
key=lambda r: (-int(r[1]), r[0].title is None, alpha.get(r[0].title or "", 0)),
)
except YouTubeError as exc:
log.warning(
@@ -659,7 +672,7 @@ def list_blocked(user: User = Depends(current_user), db: Session = Depends(get_d
select(Channel)
.join(BlockedChannel, BlockedChannel.channel_id == Channel.id)
.where(BlockedChannel.user_id == user.id)
.order_by(func.lower(Channel.title))
.order_by(collated(Channel.title, lang_for_user(user)))
)
.scalars()
.all()
+16 -5
View File
@@ -23,6 +23,7 @@ from app.discover import hero as hero_mod
from app.discover import pins, rails
from app.discover.budget import Budget, DEFAULT_BUDGET
from app.models import ComfortPin, User
from app.plex.cards import norm_lang
from app.routes._common import owned_or_404, not_found
router = APIRouter(prefix="/api/discover", tags=["discover"])
@@ -45,6 +46,7 @@ class ReorderIn(BaseModel):
def get_discover(
budget: Budget = DEFAULT_BUDGET,
seed: int = 0,
lang: str = "en",
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
@@ -52,13 +54,14 @@ def get_discover(
`resume` and `shelf` are built once and handed to the hero, so the two rails the hero draws from
are not queried twice for a single page load."""
resume = rails.continue_watching(db, user)
lang = norm_lang(lang)
resume = rails.continue_watching(db, user, lang=lang)
# Plex off → the shelf rail (and so the hero's comfort tier) goes YouTube-only like the rails
# beside it; the pins are kept and `GET /pins` still lists them (`pins.shelf`).
shelf = pins.shelf(db, user, include_plex=rails.plex_available(db))
shelf = pins.shelf(db, user, include_plex=rails.plex_available(db), lang=lang)
return {
"budget": budget,
"hero": hero_mod.pick(db, user, budget, seed, resume=resume, shelf=shelf),
"hero": hero_mod.pick(db, user, budget, seed, resume=resume, shelf=shelf, lang=lang),
"continue_watching": resume,
"comfort": shelf,
}
@@ -80,6 +83,7 @@ def get_pick(
seed: int = 0,
exclude: str = "",
start: int = 0,
lang: str = "en",
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
@@ -99,18 +103,25 @@ def get_pick(
return {
"budget": budget,
"hero": hero_mod.pick(
db, user, budget, seed, exclude=set(list(skip)[:MAX_EXCLUDE]), start=start
db,
user,
budget,
seed,
exclude=set(list(skip)[:MAX_EXCLUDE]),
start=start,
lang=norm_lang(lang),
),
}
@router.get("/pins")
def list_pins(
lang: str = "en",
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
"""The shelf on its own — for the shelf editor, which needs it without the rest of the page."""
return {"items": pins.shelf(db, user)}
return {"items": pins.shelf(db, user, lang=norm_lang(lang))}
@router.post("/pins")
+11 -2
View File
@@ -21,6 +21,8 @@ from pydantic import BaseModel
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.mailtext import lang_for_user
from app.collation import collated
from app import audit, sysconfig
from app.auth import admin_user, require_human
from app.config import settings
@@ -917,7 +919,14 @@ def share_download(
user_id=recipient.id,
type="download_shared",
title=(job.display_name or "A download") + " was shared with you",
data={"kind": "download_shared", "from": user.email, "job_id": job_id},
# The client names the notice in the reader's language from `data` (E30); the title is the
# English fallback only.
data={
"kind": "download_shared",
"from": user.email,
"job_id": job_id,
"title": job.display_name,
},
commit=False,
)
db.commit()
@@ -981,7 +990,7 @@ def share_recipients(
db.execute(
select(User)
.where(User.id != user.id, *messageable_clauses())
.order_by(func.lower(User.email))
.order_by(collated(User.email, lang_for_user(user)))
)
.scalars()
.all()
+29 -10
View File
@@ -10,6 +10,7 @@ from sqlalchemy.orm.exc import StaleDataError
from app import quota
from app.auth import current_user
from app.db import get_db
from app.mailtext import lang_for_user
from app.feedquery import (
decode_cursor,
encode_cursor,
@@ -18,8 +19,11 @@ from app.feedquery import (
filtered_query,
keyset_predicate,
order_by,
rank_window_start,
serialize,
sort_keys,
windowed_rank,
windowed_relevance_rows,
)
from app.models import (
POSITION_MAX_S,
@@ -108,30 +112,45 @@ def get_feed(
db: Session = Depends(get_db),
) -> dict:
query, rank_expr = filtered_query(db, user, **params)
keys = sort_keys(sort, seed, rank_expr)
# Expose each key column on the row so we can build the next cursor from the last item.
query = query.add_columns(*[k["expr"].label(f"_k{i}") for i, k in enumerate(keys)])
query = query.order_by(*order_by(keys))
lang = lang_for_user(user)
keys = sort_keys(sort, seed, rank_expr, lang)
vals, cursor_id, window = None, None, None
if cursor:
# The read-model raises a plain ValueError on a malformed cursor; translate it to the
# HTTP 400 here so feedquery stays web-framework-free.
try:
vals, cursor_id = decode_cursor(keys, cursor)
vals, cursor_id, window = decode_cursor(keys, cursor)
except ValueError:
# `from None`: the ValueError is an expected bad-input signal, not an error to chain.
raise refused(400, "invalid_cursor") from None
ranked = sort == "relevance" and rank_expr is not None
if ranked and not cursor:
window = rank_window_start(db, query)
if ranked and window is not None:
keys = sort_keys(sort, seed, windowed_rank(rank_expr, window), lang)
# Expose each key column on the row so we can build the next cursor from the last item.
query = query.add_columns(*[k["expr"].label(f"_k{i}") for i, k in enumerate(keys)])
if vals is not None:
for k, v in zip(keys, vals):
k["value"] = v
query = query.where(keyset_predicate(keys, cursor_id))
rows = db.execute(query.limit(limit + 1)).all()
if ranked and window is not None:
rows = windowed_relevance_rows(db, query, keys, window, cursor_id, limit + 1)
else:
query = query.order_by(*order_by(keys))
if cursor_id is not None:
query = query.where(keyset_predicate(keys, cursor_id))
rows = db.execute(query.limit(limit + 1)).all()
has_more = len(rows) > limit
page = rows[:limit]
return {
"items": [serialize(r) for r in page],
"next_cursor": encode_cursor(keys, page[-1]) if (has_more and page) else None,
"next_cursor": (
encode_cursor(keys, page[-1], window if ranked else None)
if (has_more and page)
else None
),
"has_more": has_more,
"limit": limit,
}
+26 -6
View File
@@ -21,6 +21,8 @@ from pydantic import BaseModel
from sqlalchemy import and_, case, delete, func, or_, select
from sqlalchemy.orm import Session
from app.mailtext import lang_for_user
from app.collation import collated
from app.auth import require_human, resolved_user_id
from app.db import SessionLocal, get_db
from app.models import Message, MessageKey, User
@@ -92,13 +94,31 @@ def _system_partner() -> dict:
return {"id": SYSTEM_PARTNER_ID, "name": SYSTEM_NAME, "avatar_url": None}
def _serialize_msg(m: Message) -> dict:
# The welcome's opening line in every language — how a stored copy is recognised. It was rendered
# once, in the language of the day it was created, so an account that later switched language kept
# reading it in the old one (E31 S1: the owner's, in English). Recognised here, it is served in the
# reader's language instead. Derived from WELCOME itself, so a change to an OPENING line, or a
# language taken out of WELCOME, orphans the copies already stored: they are served as stored. The
# one retired language (German, 2026-07) left none on prod (checked 2026-10-02).
_WELCOME_OPENINGS = {text.partition("\n")[0] for text in WELCOME.values()}
def _system_body(body: str | None, lang: str | None) -> str | None:
"""A system message's text for a reader of `lang`: the welcome in their language, any other as
stored."""
if body is None or lang is None or body.partition("\n")[0] not in _WELCOME_OPENINGS:
return body
return WELCOME.get(lang, WELCOME["en"])
def _serialize_msg(m: Message, lang: str | None = None) -> dict:
"""`lang`: the reader's language, for a system message's text (a user message is ciphertext)."""
return {
"id": m.id,
"kind": m.kind,
"sender_id": m.sender_id,
"recipient_id": m.recipient_id,
"body": m.body, # only set for system messages
"body": _system_body(m.body, lang) if m.kind == "system" else m.body,
"ciphertext": m.ciphertext,
"iv": m.iv,
"read_at": iso(m.read_at),
@@ -122,7 +142,7 @@ def ensure_welcome(db: Session, user: User) -> None:
db.add(m)
db.commit()
db.refresh(m)
manager.push(user.id, {"type": "message", "message": _serialize_msg(m)})
manager.push(user.id, {"type": "message", "message": _serialize_msg(m, lang_for_user(user))})
# --- E2EE key directory / blob store -----------------------------------------
@@ -246,7 +266,7 @@ def list_recipients(
select(User)
.join(MessageKey, MessageKey.user_id == User.id)
.where(User.id != user.id, *messageable_clauses())
.order_by(func.lower(func.coalesce(User.display_name, User.email)))
.order_by(collated(func.coalesce(User.display_name, User.email), lang_for_user(user)))
)
.scalars()
.all()
@@ -311,7 +331,7 @@ def list_conversations(user: User = Depends(require_human), db: Session = Depend
items.append(
{
"partner": partner,
"last_message": _serialize_msg(m),
"last_message": _serialize_msg(m, lang_for_user(user)),
"unread": int(unread.get(pid, 0)),
}
)
@@ -364,7 +384,7 @@ def get_thread(
# refreshes live instead of waiting for the next 20s poll. The client re-fetches the
# count on this signal, so no number is carried here.
manager.push(user.id, {"type": "unread"})
return {"partner": partner, "items": [_serialize_msg(m) for m in msgs]}
return {"partner": partner, "items": [_serialize_msg(m, lang_for_user(user)) for m in msgs]}
@router.post("")
+1 -5
View File
@@ -4,6 +4,7 @@ from sqlalchemy.orm import Session
from app import logs
from app.models import PlexItem
from app.plex.cards import norm_lang as _norm_lang # noqa: F401 — the package's historic name
from app.plex.lookups import by_rating_key
from app.routes._common import not_found
@@ -15,8 +16,3 @@ def item_or_404(db: Session, rating_key: str) -> PlexItem:
if it is None:
raise not_found("plex_item")
return it
def _norm_lang(lang: str | None) -> str:
"""The overlay has an EN base + an optional HU twin; anything other than `hu` serves EN."""
return "hu" if lang == "hu" else "en"
+51 -26
View File
@@ -10,10 +10,13 @@ from sqlalchemy import (
func,
literal,
null,
or_,
select,
)
from sqlalchemy.orm import Session, aliased
from app.mailtext import lang_for_user
from app.collation import collated, sorted_words
from app import sysconfig
from app.auth import current_user
from app.db import get_db
@@ -25,7 +28,7 @@ from app.models import (
User,
)
from app.plex import rollup
from app.plex.cards import episode_card
from app.plex.cards import display_title, episode_card
from app.plex.query import (
FIELD_COLUMNS,
PLAYBACK_CLASSES,
@@ -54,14 +57,15 @@ from app.utils import to_tsquery_str
router = APIRouter(prefix="/api/plex", tags=["plex"])
def sort_column(u, sort: str):
def sort_column(u, sort: str, lang: str = "en"):
"""The browse grid's sort key -> orderable column on the union subquery. Module-level (not inline
in the handler) so a test can pin its vocabulary to `PLEX_SORT_KEYS` — a key the Sort control
offers but this cannot order would quietly fall back to `added_at` and look like a broken control.
An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback."""
An unknown key (a stale bookmark, a hand-edited URL) deliberately keeps that fallback. The title sorts
in the reader's alphabet (`collation`)."""
return {
"added": u.c.added_at,
"title": func.lower(u.c.title),
"title": collated(u.c.title, lang),
"year": u.c.year,
"rating": u.c.rating,
"duration": u.c.duration_s,
@@ -78,7 +82,7 @@ def list_libraries(user: User = Depends(current_user), db: Session = Depends(get
db.execute(
select(PlexLibrary)
.where(PlexLibrary.overlay_of.is_(None))
.order_by(PlexLibrary.kind.desc(), PlexLibrary.title)
.order_by(PlexLibrary.kind.desc(), collated(PlexLibrary.title, lang_for_user(user)))
)
.scalars()
.all()
@@ -237,7 +241,7 @@ def unified_library(
if tsq is not None:
order = [u.c.rank.desc()]
else:
col = sort_column(u, sort)
col = sort_column(u, sort, lang)
order = [(col.asc() if sort_dir == "asc" else col.desc()).nullslast()]
order.append(u.c.rk.desc())
rows = db.execute(select(u).order_by(*order).offset(offset).limit(limit)).all()
@@ -281,26 +285,32 @@ def unified_library(
episodes = []
if tsq is not None and want_shows and field in ("all", "episode"):
ep_st = aliased(PlexState)
ep_vec = (
func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title, ""))
if field == "episode"
else PlexItem.search_vector
)
# In HU an episode is found by its Hungarian title too (E30: the overlay now fills it) — and
# still by its English one, like `vector_match` does for the grid.
if field == "episode":
ep_vec = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title, ""))
ep_vec_hu = func.to_tsvector(TS_CONFIG, func.coalesce(PlexItem.title_hu, ""))
else:
ep_vec, ep_vec_hu = PlexItem.search_vector, PlexItem.search_vector_hu
hit = ep_vec.op("@@")(tsq)
if lang == "hu":
hit = or_(ep_vec_hu.op("@@")(tsq), hit)
ep_vec = ep_vec_hu
eps = db.execute(
select(PlexItem, ep_st, PlexShow.title.label("show_title"))
select(PlexItem, ep_st, PlexShow)
.outerjoin(ep_st, and_(ep_st.item_id == PlexItem.id, ep_st.user_id == user.id))
.outerjoin(PlexShow, PlexShow.id == PlexItem.show_id)
.where(
PlexItem.kind == "episode",
PlexItem.library_id.in_(show_lib_ids),
ep_vec.op("@@")(tsq),
hit,
)
.order_by(func.ts_rank(ep_vec, tsq).desc())
.limit(24)
).all()
for e, s, show_title in eps:
card = episode_card(e, s)
card["show_title"] = show_title
for e, s, sh in eps:
card = episode_card(e, s, lang=lang)
card["show_title"] = display_title(sh, lang) if sh is not None else None
episodes.append(card)
return {
@@ -348,6 +358,24 @@ def facets(
uid = user.id
# The shows that pass the watch-state scope, asked ONCE. The per-user episode rollup behind it is a
# full pass over the user's episodes (~10 ms on a prod-sized library), and each of the show groups
# (genres, ratings, studios, years, rating max) asked it again: half of a filtered facets call
# (E31 S4e). The same filter on the same rows, so the counts do not change
# (test_plex_facets_grid_parity).
shows_in_scope: list[int] | None = None
if show_ids and show in ("watched", "in_progress", "unwatched"):
agg = rollup.agg_subquery(db, uid).subquery()
in_scope = select(PlexShow.id).outerjoin(agg, agg.c.show_id == PlexShow.id)
in_scope = apply_show_state_filter(
in_scope,
func.coalesce(agg.c.total, 0),
func.coalesce(agg.c.watched, 0),
func.coalesce(agg.c.inprog, 0),
show,
)
shows_in_scope = list(db.scalars(in_scope.where(PlexShow.library_id.in_(show_ids))))
def wsq(query, model, ids, pp, kind_movie, with_dur):
"""Apply the (self-excluded) metadata filters + library scope + the per-user watch-state to a
facet aggregation query, so the facets reflect exactly the visible result set."""
@@ -366,14 +394,8 @@ def facets(
st = aliased(PlexState)
query = query.outerjoin(st, and_(st.item_id == model.id, st.user_id == uid))
query = apply_movie_state_filter(query, st, show)
else:
if show in ("watched", "in_progress", "unwatched"):
agg = rollup.agg_subquery(db, uid).subquery()
query = query.outerjoin(agg, agg.c.show_id == model.id)
tot = func.coalesce(agg.c.total, 0)
wat = func.coalesce(agg.c.watched, 0)
inp = func.coalesce(agg.c.inprog, 0)
query = apply_show_state_filter(query, tot, wat, inp, show)
elif shows_in_scope is not None:
query = query.where(model.id.in_(shows_in_scope))
return query
genre_counts: dict[str, int] = {}
@@ -523,16 +545,19 @@ def facets(
for c in split_csv(f.playable):
if c in PLAYBACK_CLASSES:
playable_counts.setdefault(c, 0)
# Names in the reader's alphabet: the genres are Hungarian words for a Hungarian reader (E30 S1).
alpha = {w: i for i, w in enumerate(sorted_words(db, [*genre_counts, *studio_counts], lang))}
return {
"genres": [
{"value": g, "count": c} for g, c in sorted(genre_counts.items(), key=lambda kv: kv[0])
{"value": g, "count": c}
for g, c in sorted(genre_counts.items(), key=lambda kv: alpha[kv[0]])
],
"content_ratings": [
{"value": cr, "count": c} for cr, c in sorted(cr_counts.items(), key=lambda kv: -kv[1])
],
"studios": [
{"value": s, "count": c}
for s, c in sorted(studio_counts.items(), key=lambda kv: (-kv[1], kv[0]))
for s, c in sorted(studio_counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))
],
# Fixed good→heavy order (not by count), so the class list reads as a severity scale.
"playable": [
+26 -13
View File
@@ -9,6 +9,8 @@ from pydantic import BaseModel
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.mailtext import lang_for_user
from app.collation import collated, sorted_words
from app import logs
from app.auth import admin_user, current_user
from app.db import get_db
@@ -22,10 +24,10 @@ from app.models import (
)
from app.plex import rollup
from app.plex import sync as plex_sync
from app.plex.cards import movie_card, show_card
from app.plex.cards import display_title, movie_card, norm_lang, show_card
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
from app.plex.lookups import by_rating_key
from app.plex.query import content_libs, runtime_by_key, top_art_keys
from app.plex.query import content_libs, runtime_by_key, title_col, top_art_keys
from app.utils import like_escape
from app.refusal import refused
from app.routes._common import not_found
@@ -142,7 +144,7 @@ def collections(
db.execute(
query.order_by(
PlexCollection.child_count.desc().nullslast(),
func.lower(PlexCollection.title),
collated(PlexCollection.title, lang_for_user(user)),
)
)
.scalars()
@@ -208,16 +210,24 @@ MEMBER_SORT_KEYS: dict[str, Callable[[Member], Any]] = {
DEFAULT_MEMBER_SORT = "year"
def sort_members(members: list[Member], sort: str, sort_dir: str) -> list[Member]:
def sort_members(
members: list[Member], sort: str, sort_dir: str, alpha: dict[str, int] | None = None
) -> list[Member]:
"""Order a collection's members by one of `PLEX_SORT_KEYS`.
Two properties worth stating because they are easy to get wrong with a plain `reverse=True`:
members with no value for the key sink to the END in BOTH directions (matching the browse grid's
`nullslast()` — a movie with no rating is not "the worst rated"), and ties always resolve
title-ascending, so flipping the direction reverses the ranking rather than the alphabet inside
it. An unknown key falls back to the page's default instead of returning an arbitrary order."""
it. An unknown key falls back to the page's default instead of returning an arbitrary order.
`alpha` ranks each title in the reader's alphabet (`collation.sorted_words`); without it, titles
compare as Python strings, which puts every accented initial after "z"."""
by_title = (lambda m: alpha[m.title]) if alpha is not None else (lambda m: m.title)
key = MEMBER_SORT_KEYS.get(sort, MEMBER_SORT_KEYS[DEFAULT_MEMBER_SORT])
ordered = sorted(members, key=lambda m: m.title) # stable secondary, applied first
if key is MEMBER_SORT_KEYS["title"]:
key = by_title
ordered = sorted(members, key=by_title) # stable secondary, applied first
have = [m for m in ordered if key(m) is not None]
missing = [m for m in ordered if key(m) is None]
have.sort(key=key, reverse=sort_dir != "asc")
@@ -229,6 +239,7 @@ def collection_detail(
rating_key: str,
sort: str = DEFAULT_MEMBER_SORT,
sort_dir: str = "asc",
lang: str = "en",
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
@@ -237,6 +248,7 @@ def collection_detail(
control means the same thing on both surfaces; the default is release year ascending, which is
what this page did before it had a control. Read-only here; edits go through the admin mutation
endpoints below."""
lang = norm_lang(lang)
col = by_rating_key(db, PlexCollection, rating_key)
if col is None:
raise not_found("collection")
@@ -244,7 +256,7 @@ def collection_detail(
db.execute(
select(PlexItem)
.where(PlexItem.kind == "movie", PlexItem.collection_keys.contains([rating_key]))
.order_by(PlexItem.year.asc().nullslast(), func.lower(PlexItem.title))
.order_by(PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang))
)
.scalars()
.all()
@@ -253,7 +265,7 @@ def collection_detail(
db.execute(
select(PlexShow)
.where(PlexShow.collection_keys.contains([rating_key]))
.order_by(PlexShow.year.asc().nullslast(), func.lower(PlexShow.title))
.order_by(PlexShow.year.asc().nullslast(), collated(title_col(PlexShow, lang), lang))
)
.scalars()
.all()
@@ -285,7 +297,7 @@ def collection_detail(
Member(
kind="movie",
obj=m,
title=(m.title or "").lower(),
title=display_title(m, lang).lower(),
year=m.year,
rating=m.rating,
added=m.added_at,
@@ -298,7 +310,7 @@ def collection_detail(
Member(
kind="show",
obj=sh,
title=(sh.title or "").lower(),
title=display_title(sh, lang).lower(),
year=sh.year,
rating=sh.rating,
added=sh.added_at,
@@ -308,11 +320,12 @@ def collection_detail(
)
for sh in shows
]
members = sort_members(members, sort, sort_dir)
alpha = {w: i for i, w in enumerate(sorted_words(db, [m.title for m in members], lang))}
members = sort_members(members, sort, sort_dir, alpha)
items = [
movie_card(m.obj, mstate.get(m.obj.id))
movie_card(m.obj, mstate.get(m.obj.id), lang=lang)
if m.kind == "movie"
else show_card(m.obj, sstate.get(m.obj.id, "new"))
else show_card(m.obj, sstate.get(m.obj.id, "new"), lang=lang)
for m in members
]
years = [m.year for m in members if m.year is not None]
+45 -20
View File
@@ -1,14 +1,16 @@
"""Plex show/item detail: the show drill-down + the per-item player payload (markers, cast, nav)."""
import re
import threading
import time
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime, timezone
from fastapi import APIRouter, Depends
from sqlalchemy import func, select
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.collation import collated, sorted_words
from app import logs
from app.auth import current_user
from app.db import get_db
@@ -24,6 +26,7 @@ from app.models import (
from app.plex import detail as plex_detail
from app.plex import people as plex_people
from app.plex import rollup, subtitles
from app.plex.query import title_col
from app.plex import tmdb as plex_tmdb
from app.plex.cards import episode_card, movie_card, show_card
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
@@ -107,7 +110,8 @@ def show_detail(
db: Session = Depends(get_db),
) -> dict:
"""A show's seasons + episodes (the drill-down view) with per-user watch state."""
hu = _norm_lang(lang) == "hu"
lang = _norm_lang(lang)
hu = lang == "hu"
sh = by_rating_key(db, PlexShow, rating_key)
if sh is None:
raise not_found("show")
@@ -144,7 +148,7 @@ def show_detail(
by_season: dict[int | None, list] = {}
all_cards: list[dict] = [] # every episode, in season/episode order → show-level rollup
for e in eps:
card = episode_card(e, states.get(e.id))
card = episode_card(e, states.get(e.id), lang=lang)
by_season.setdefault(e.season_id, []).append(card)
all_cards.append(card)
show_roll = rollup.rollup(all_cards)
@@ -156,7 +160,7 @@ def show_detail(
related: list[dict] = []
try:
meta, related_raw = _show_enrichment(db, sh.rating_key)
related = _relatedshow_cards(db, user.id, related_raw, exclude=sh.id)
related = _relatedshow_cards(db, user.id, related_raw, exclude=sh.id, lang=lang)
if sh.people_enriched_at is None and meta:
plex_people.enrich_row_people(db, sh, plex_people.cast_from_meta(meta))
plex_detail.apply_rich_detail(sh, meta) # imdb/tagline/full_cast for future views
@@ -210,7 +214,7 @@ def show_detail(
"episode_count": show_roll["episode_count"],
"collection_keys": sh.collection_keys or [],
# TMDB enrichment (E5 S2), served from the row.
"tmdb": plex_tmdb.serve_block(db, sh),
"tmdb": plex_tmdb.serve_block(db, sh, lang=lang),
},
"seasons": [_season_block(se, by_season.get(se.id, [])) for se in seasons],
"related": related,
@@ -234,7 +238,9 @@ def _season_block(se: PlexSeason, cards: list[dict]) -> dict:
}
def _relatedshow_cards(db: Session, user_id: int, related: list[dict], exclude: int) -> list[dict]:
def _relatedshow_cards(
db: Session, user_id: int, related: list[dict], exclude: int, lang: str
) -> list[dict]:
"""Map Plex 'related' metadata to show cards for the shows we actually mirror (so they're openable),
excluding the current show. Order preserved; capped."""
rks = [str(m.get("ratingKey")) for m in related if m.get("ratingKey")]
@@ -254,13 +260,13 @@ def _relatedshow_cards(db: Session, user_id: int, related: list[dict], exclude:
if sh is None or rk in seen:
continue
seen.add(rk)
out.append(show_card(sh, statuses.get(sh.id, "new")))
out.append(show_card(sh, statuses.get(sh.id, "new"), lang=lang))
if len(out) >= 20:
break
return out
def _collection_strips(db: Session, it: PlexItem, user_id: int) -> list[dict]:
def _collection_strips(db: Session, it: PlexItem, user_id: int, lang: str) -> list[dict]:
"""The collections this movie belongs to, each with its member titles (playable cards) — the
'Avatar Collection' strip on the info page. Pure local lookup (no Plex call)."""
keys = it.collection_keys or []
@@ -273,7 +279,8 @@ def _collection_strips(db: Session, it: PlexItem, user_id: int) -> list[dict]:
).scalars()
}
# Smallest (most specific — e.g. a franchise) first; the big auto-lists (IMDb/TMDb Popular) last.
ordered = sorted((c for c in cols.values()), key=lambda c: (c.child_count or 0, c.title))
alpha = {w: i for i, w in enumerate(sorted_words(db, [c.title for c in cols.values()], lang))}
ordered = sorted(cols.values(), key=lambda c: (c.child_count or 0, alpha[c.title]))
out = []
for col in ordered:
rk = col.rating_key
@@ -285,7 +292,9 @@ def _collection_strips(db: Session, it: PlexItem, user_id: int) -> list[dict]:
PlexItem.kind == "movie",
PlexItem.collection_keys.contains([rk]),
)
.order_by(PlexItem.year.asc().nullslast(), func.lower(PlexItem.title))
.order_by(
PlexItem.year.asc().nullslast(), collated(title_col(PlexItem, lang), lang)
)
.limit(60)
)
.scalars()
@@ -305,7 +314,7 @@ def _collection_strips(db: Session, it: PlexItem, user_id: int) -> list[dict]:
"id": rk,
"title": col.title,
"source": _collection_source(col),
"items": [movie_card(m, states.get(m.id)) for m in members],
"items": [movie_card(m, states.get(m.id), lang=lang) for m in members],
}
)
return out
@@ -339,6 +348,26 @@ def _episode_nav(db: Session, it: PlexItem) -> tuple[str | None, str | None]:
return (keys[i - 1] if i > 0 else None), (keys[i + 1] if i < len(keys) - 1 else None)
# The technical tail of Plex's track title — "English (AC3 5.1)" → "AC3 5.1".
_TRACK_TAIL = re.compile(r"\(([^()]+)\)\s*$")
def _track_naming(s: dict) -> dict:
"""What the player names a track from (E30). Plex's `displayTitle` is in the SERVER's language
("English (AC3 5.1)", "magyar Forced", "中文 (EAC3 Stereo)"), so it is sent only as the fallback
`label`; the client builds the name in the reader's language from the language tag, the
forced / SDH flags and the codec tail. No title and no language → no label (the client numbers it)."""
title = s.get("displayTitle") or s.get("language") or None
tail = _TRACK_TAIL.search(title) if title else None
return {
"label": title,
"lang_tag": s.get("languageTag") or None,
"forced": bool(s.get("forced")),
"hearing_impaired": bool(s.get("hearingImpaired")),
"detail": tail.group(1).strip() if tail else None,
}
@router.get("/item/{rating_key}")
def item_detail(
rating_key: str,
@@ -384,9 +413,7 @@ def item_detail(
audio_streams.append(
{
"ord": len(audio_streams),
"label": s.get("displayTitle")
or s.get("language")
or f"Audio {len(audio_streams) + 1}",
**_track_naming(s),
"language": s.get("languageTag") or s.get("language"),
"default": bool(s.get("selected") or s.get("default")),
}
@@ -396,9 +423,7 @@ def item_detail(
subtitle_streams.append(
{
"ord": len(subtitle_streams),
"label": s.get("displayTitle")
or s.get("language")
or f"Subtitle {len(subtitle_streams) + 1}",
**_track_naming(s),
"language": s.get("languageTag") or s.get("language"),
"codec": codec,
# Text subs (srt/ass/…) can be served as WebVTT tracks; image subs (PGS/VobSub)
@@ -468,7 +493,7 @@ def item_detail(
"genres": (it.genres_hu or it.genres if hu else it.genres) or [],
"directors": it.directors or [],
"studio": it.studio,
"tagline": it.tagline,
"tagline": (it.tagline_hu or it.tagline) if hu else it.tagline,
}
else:
rich = plex_detail.rich_detail(meta)
@@ -512,7 +537,7 @@ def item_detail(
"episode_number": it.episode_number,
"prev_id": prev_id,
"next_id": next_id,
"collections": _collection_strips(db, it, user.id),
"collections": _collection_strips(db, it, user.id, lang),
# TMDB enrichment (E5 S2), served from the row — movies only (episodes aren't enriched).
"tmdb": plex_tmdb.serve_block(db, it) if it.kind == "movie" else None,
"tmdb": plex_tmdb.serve_block(db, it, lang=lang) if it.kind == "movie" else None,
}
+30 -20
View File
@@ -18,11 +18,12 @@ from fastapi import APIRouter, Depends, Query
from sqlalchemy import and_, func, or_, select
from sqlalchemy.orm import Session
from app.collation import sorted_words
from app.auth import current_user
from app.db import get_db
from app.models import PlexItem, PlexShow, PlexState, User
from app.plex import rollup
from app.plex.cards import leaf_card, movie_card, show_card
from app.plex.cards import display_title, leaf_card, movie_card, show_card
from app.plex.query import (
LibraryFilters,
apply_meta_filters,
@@ -61,14 +62,14 @@ def _movie_meta(q, p: dict, lang: str, f: LibraryFilters):
return q
def _movie_cards(db: Session, uid: int, items: list[PlexItem]) -> list[dict]:
def _movie_cards(db: Session, uid: int, items: list[PlexItem], lang: str) -> list[dict]:
sts = rollup.item_state_map(db, uid, [it.id for it in items])
return [movie_card(it, sts.get(it.id)) for it in items]
return [movie_card(it, sts.get(it.id), lang=lang) for it in items]
def _show_cards(db: Session, uid: int, shows: list[PlexShow]) -> list[dict]:
def _show_cards(db: Session, uid: int, shows: list[PlexShow], lang: str) -> list[dict]:
statuses = rollup.state_map(db, uid, [s.id for s in shows])
return [show_card(s, statuses.get(s.id, "new")) for s in shows]
return [show_card(s, statuses.get(s.id, "new"), lang=lang) for s in shows]
def _recently_added(
@@ -123,8 +124,8 @@ def _recently_added(
# share one), so the merge doesn't depend on object identity staying alive.
top_movies = [o for _, kind, o in top if kind == "movie"]
top_shows = [o for _, kind, o in top if kind == "show"]
mcards = {m.rating_key: c for m, c in zip(top_movies, _movie_cards(db, uid, top_movies))}
scards = {s.rating_key: c for s, c in zip(top_shows, _show_cards(db, uid, top_shows))}
mcards = {m.rating_key: c for m, c in zip(top_movies, _movie_cards(db, uid, top_movies, lang))}
scards = {s.rating_key: c for s, c in zip(top_shows, _show_cards(db, uid, top_shows, lang))}
return [mcards[o.rating_key] if kind == "movie" else scards[o.rating_key] for _, kind, o in top]
@@ -156,10 +157,10 @@ def _recently_watched(
.order_by(PlexState.watched_at.desc().nullslast())
.limit(RAIL_LIMIT)
).all():
rows.append((st.watched_at, leaf_card(it, st)))
rows.append((st.watched_at, leaf_card(it, st, lang=lang)))
if show_ids:
eq = (
select(PlexItem, PlexState, PlexShow.title, PlexShow.rating_key)
select(PlexItem, PlexState, PlexShow)
.join(PlexState, and_(PlexState.item_id == PlexItem.id, PlexState.user_id == uid))
.join(PlexShow, PlexShow.id == PlexItem.show_id)
.where(
@@ -170,8 +171,11 @@ def _recently_watched(
.order_by(PlexState.watched_at.desc().nullslast())
.limit(RAIL_LIMIT)
)
for ep, st, sh_title, sh_rk in db.execute(eq).all():
rows.append((st.watched_at, leaf_card(ep, st, show_title=sh_title, show_rk=sh_rk)))
for ep, st, sh in db.execute(eq).all():
card = leaf_card(
ep, st, show_title=display_title(sh, lang), show_rk=sh.rating_key, lang=lang
)
rows.append((st.watched_at, card))
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
return [card for _, card in rows[:RAIL_LIMIT]]
@@ -214,7 +218,7 @@ def _continue_watching(
.limit(RAIL_LIMIT)
).all()
for it, st in mrows:
rows.append((st.progress_updated_at, movie_card(it, st)))
rows.append((st.progress_updated_at, movie_card(it, st, lang=lang)))
if show_ids:
# Reuse the shared per-show episode rollup (total/watched/inprog) rather than re-inlining it, plus
@@ -234,7 +238,7 @@ def _continue_watching(
.limit(RAIL_LIMIT)
).all()
for sh, resumed_at in srows:
rows.append((resumed_at, show_card(sh, "in_progress")))
rows.append((resumed_at, show_card(sh, "in_progress", lang=lang)))
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
return [card for _, card in rows[:RAIL_LIMIT]]
@@ -261,7 +265,8 @@ def _genre_tiles(
expanded = base.subquery()
for g, c in db.execute(select(expanded.c.g, func.count()).group_by(expanded.c.g)).all():
counts[g] = counts.get(g, 0) + c
top = sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))[:GENRE_TILES]
alpha = {w: i for i, w in enumerate(sorted_words(db, list(counts), lang))}
top = sorted(counts.items(), key=lambda kv: (-kv[1], alpha[kv[0]]))[:GENRE_TILES]
# A 2×2 poster collage per tile: the top movies in that genre (color hash is the client fallback).
art = top_art_keys(
db,
@@ -310,7 +315,7 @@ def _because_you_watched(
)
if anchor is None:
return None
resolved = resolve_recs(db, anchor.recommendations)
resolved = resolve_recs(db, anchor.recommendations, lang=lang)
# Preserve TMDB's recommendation order, keep only in-library hits, and drop the anchor itself.
rks = [
r["rating_key"]
@@ -346,9 +351,11 @@ def _because_you_watched(
ordered_shows = [shows[rk] for rk in rks if rk in shows]
# rating_key is globally unique in Plex, so it keys the serialized cards back safely (no id()).
mcards = {
m.rating_key: c for m, c in zip(ordered_movies, _movie_cards(db, uid, ordered_movies))
m.rating_key: c for m, c in zip(ordered_movies, _movie_cards(db, uid, ordered_movies, lang))
}
scards = {
s.rating_key: c for s, c in zip(ordered_shows, _show_cards(db, uid, ordered_shows, lang))
}
scards = {s.rating_key: c for s, c in zip(ordered_shows, _show_cards(db, uid, ordered_shows))}
items: list[dict] = []
for rk in rks:
if rk in mcards:
@@ -357,7 +364,10 @@ def _because_you_watched(
items.append(scards[rk])
if not items:
return None
return {"anchor": {"id": anchor.rating_key, "title": anchor.title}, "items": items}
return {
"anchor": {"id": anchor.rating_key, "title": display_title(anchor, lang)},
"items": items,
}
@router.get("/discover")
@@ -456,8 +466,8 @@ def surprise(
.scalars()
.all()
)
mcards = _movie_cards(db, user.id, movies)
scards = _show_cards(db, user.id, shows)
mcards = _movie_cards(db, user.id, movies, lang)
scards = _show_cards(db, user.id, shows, lang)
# Interleave movie/show cards, then trim to `limit`.
items: list[dict] = []
for i in range(max(len(mcards), len(scards))):
+41 -19
View File
@@ -7,7 +7,8 @@ from fastapi import APIRouter, BackgroundTasks, Depends
from sqlalchemy import func, or_, select
from sqlalchemy.orm import Session
from app import logs
from app.collation import collated
from app import logs, sysconfig
from app.auth import current_user
from app.db import SessionLocal, get_db
from app.models import (
@@ -18,22 +19,28 @@ from app.models import (
)
from app.plex import rollup
from app.plex import tmdb as plex_tmdb
from app.plex.cards import movie_card, show_card
from app.plex.query import content_libs, split_csv
from app.plex.cards import movie_card, norm_lang, show_card
from app.plex.query import content_libs, split_csv, title_col
router = APIRouter(prefix="/api/plex", tags=["plex"])
log = logs.get_logger(__name__)
# How many "Known for" titles the person page shows (the stored pool is deeper; see plex.tmdb).
_KNOWN_FOR_SHOWN = 15
def _resolve_known_for(
db: Session,
known_for: list | None,
uid: int,
movie_lib_ids: list[int],
show_lib_ids: list[int],
lang: str,
) -> list[dict]:
"""The person's TMDB "Known for" credits (top by popularity) that are IN our library, as cards, in
popularity order. Matched by the canonical guid `tmdb:<id>`; out-of-library credits are dropped."""
"""The person's TMDB "Known for" credits (most-voted first) that are IN our library, as cards, in
that order, at most _KNOWN_FOR_SHOWN. Matched by the canonical guid `tmdb:<id>`; out-of-library
credits are dropped."""
if not known_for:
return []
# Match each credit to a library row by guid, bucketed by media_type (movie ids and tv ids collide
@@ -50,10 +57,10 @@ def _resolve_known_for(
# A credit resolves against its OWN media_type's bucket only.
if k.get("media_type") == "tv":
if g in shows:
out.append(show_card(shows[g], show_statuses.get(shows[g].id, "new")))
out.append(show_card(shows[g], show_statuses.get(shows[g].id, "new"), lang=lang))
elif g in movies:
out.append(movie_card(movies[g], m_states.get(movies[g].id)))
return out
out.append(movie_card(movies[g], m_states.get(movies[g].id), lang=lang))
return out[:_KNOWN_FOR_SHOWN]
@router.get("/people/cards")
@@ -159,6 +166,7 @@ def _enrich_person_bg(person_id: int) -> None:
def person_detail(
name: str,
background_tasks: BackgroundTasks,
lang: str = "en",
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
@@ -171,18 +179,24 @@ def person_detail(
E5 S6: the first time a page is opened, the person's TMDB bio + "Known for" credits are lazily
fetched (from the S2-linked `tmdb_person_id`) and persisted, then served from the row on every open.
The fetch runs OFF the request path (a background task); the first open serves whatever's in the row
(bio possibly still null) and the bio lands from the row by the next open.
with `enriching` set, and the page asks again until the bio lands. E30 S6: the people_enrich job
fetches everyone ahead of time, so this lazy path is only for a person the job hasn't reached yet.
"""
lang = norm_lang(lang)
person = db.execute(select(PlexPerson).where(PlexPerson.name == name)).scalars().first()
# Lazy one-time bio enrich, scheduled AFTER the response so the TMDB call is never on the serve path.
# Gated by a per-person cooldown so concurrent/rapid opens (or repeated opens while TMDB is degraded)
# can't pile background tasks onto the shared threadpool.
if (
# `enriching` tells the page the bio is on its way (E30 S6): it asks again shortly instead of showing
# an empty page until the next open. True while a fetch is pending, including one an earlier open
# already reserved.
enriching = (
person is not None
and person.tmdb_person_id
and bool(person.tmdb_person_id)
and person.tmdb_enriched_at is None
and _should_schedule_enrich(person.id)
):
and bool(sysconfig.get_str(db, "tmdb_api_key"))
)
if enriching and person is not None and _should_schedule_enrich(person.id):
background_tasks.add_task(_enrich_person_bg, person.id)
movie_lib_ids, show_lib_ids = content_libs(db)
@@ -199,14 +213,16 @@ def person_detail(
PlexItem.directors.contains([name]),
),
)
.order_by(PlexItem.year.desc().nullslast(), func.lower(PlexItem.title))
.order_by(
PlexItem.year.desc().nullslast(), collated(title_col(PlexItem, lang), lang)
)
.limit(_PERSON_SECTION_MAX)
)
.scalars()
.all()
)
states = rollup.item_state_map(db, user.id, [it.id for it in items])
movies = [movie_card(it, states.get(it.id)) for it in items]
movies = [movie_card(it, states.get(it.id), lang=lang) for it in items]
shows: list[dict] = []
if show_lib_ids:
@@ -220,28 +236,34 @@ def person_detail(
PlexShow.directors.contains([name]),
),
)
.order_by(PlexShow.year.desc().nullslast(), func.lower(PlexShow.title))
.order_by(
PlexShow.year.desc().nullslast(), collated(title_col(PlexShow, lang), lang)
)
.limit(_PERSON_SECTION_MAX)
)
.scalars()
.all()
)
statuses = rollup.state_map(db, user.id, [s.id for s in show_rows])
shows = [show_card(sh, statuses.get(sh.id, "new")) for sh in show_rows]
shows = [show_card(sh, statuses.get(sh.id, "new"), lang=lang) for sh in show_rows]
known_for = _resolve_known_for(
db, person.known_for if person else None, user.id, movie_lib_ids, show_lib_ids
db, person.known_for if person else None, user.id, movie_lib_ids, show_lib_ids, lang
)
return {
"name": name,
"photo": person.photo_thumb if person else None,
"biography": person.biography if person else None,
# The Hungarian biography for a Hungarian reader, the instance-language one when TMDB has none.
"biography": ((person.biography_hu if lang == "hu" else None) or person.biography)
if person
else None,
"birthday": person.birthday if person else None,
"deathday": person.deathday if person else None,
"place_of_birth": person.place_of_birth if person else None,
"known_for_department": person.known_for_department if person else None,
"also_known_as": (person.also_known_as if person else None) or [],
"known_for": known_for,
"enriching": enriching,
"movies": movies,
"shows": shows,
}
+22 -16
View File
@@ -1,6 +1,5 @@
"""Plex playback: image/subtitle proxies, HLS streaming, and watch-state writes."""
import hashlib
import threading
from datetime import datetime, timedelta, timezone
@@ -66,7 +65,7 @@ def image(rating_key: str, request: Request, variant: str = "thumb") -> Response
brief session only to resolve the image key + read the Plex config, then RELEASES it before the
fetch. Auth is the signed-session check (no DB user-load)."""
_require_session(request)
v = "art" if variant == "art" else "thumb"
v = variant if variant in imgcache.SIZES else "thumb"
w, h = imgcache.SIZES[v]
# Size is part of the cache key: Plex resizes server-side (see PlexClient.image_bytes), so we cache
# the small variant. The width tag also means changing a target size just re-caches, never serves a
@@ -78,7 +77,7 @@ def image(rating_key: str, request: Request, variant: str = "thumb") -> Response
# Cold miss: short-lived session for the key + Plex client config, released before the slow fetch.
try:
with SessionLocal() as db:
key = imgcache.key_for(db, str(rating_key), v)
key = imgcache.key_for(db, str(rating_key), imgcache.picture(v))
if not key:
raise not_found("image")
plex = PlexClient(db) # reads base URL + token now; the fetch below needs no DB
@@ -214,24 +213,31 @@ def _resume_push_superseded(user_id: int, item_id: int, intent: datetime) -> boo
@router.get("/person-image")
def person_image(u: str, request: Request) -> Response:
"""Proxy a cast/crew photo from Plex's public metadata CDN (host-whitelisted; no token needed).
Signed-session auth (no DB) so it, like /image, never holds a pool connection during the fetch."""
"""Proxy a cast/crew photo from Plex's public metadata CDN (only a headshot-shaped URL on its host;
no token needed), resized through Plex and cached (plex.people.cache_portrait; the people_enrich job
fills the same cache ahead of time). Signed-session auth; like /image, the DB session is released
before the fetch."""
_require_session(request)
if not u.startswith(plex_people.PERSON_IMG_HOST):
if not plex_people.is_portrait_url(u):
raise refused(400, "image_host")
cache_key = "person_" + hashlib.sha1(u.encode()).hexdigest()
hit = imgcache.read(cache_key)
hit = plex_people.portrait_cached(u)
if hit:
return imgcache.response(*hit)
try:
with httpx.Client(timeout=10.0, follow_redirects=True) as c:
r = c.get(u)
r.raise_for_status()
except httpx.HTTPError:
raise refused(502, "image_fetch_failed")
ct = r.headers.get("content-type", "image/jpeg")
imgcache.write(cache_key, r.content, ct)
return imgcache.response(r.content, ct)
with SessionLocal() as db:
plex: PlexClient | None = PlexClient(db)
# silent-ok: Plex off is a setting; without its resize the original comes straight from the CDN.
except PlexNotConfigured:
plex = None
try:
if plex is None:
data, ct = plex_people.cache_portrait(None, u)
else:
with plex:
data, ct = plex_people.cache_portrait(plex, u)
except (PlexError, httpx.HTTPError):
raise refused(502, "image_fetch_failed") from None
return imgcache.response(data, ct)
def _push_resume_guarded(
+8 -3
View File
@@ -19,7 +19,7 @@ from app.models import (
PlexState,
User,
)
from app.plex.cards import leaf_card
from app.plex.cards import display_title, leaf_card, norm_lang
from app.plex.lookups import by_rating_key
from app.routes._common import owned_or_404, not_found
from app.refusal import refused
@@ -137,8 +137,12 @@ def create_playlist(
@router.get("/playlists/{pid}")
def playlist_detail(
pid: int, user: User = Depends(current_user), db: Session = Depends(get_db)
pid: int,
lang: str = "en",
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
lang = norm_lang(lang)
pl = owned_or_404(db, PlexPlaylist, pid, user, what="playlist")
rows = db.execute(_playlist_item_query(pid)).all()
items = [it for (_pi, it) in rows]
@@ -163,8 +167,9 @@ def playlist_detail(
leaf_card(
it,
states.get(it.id),
shows[it.show_id].title if it.show_id in shows else None,
display_title(shows[it.show_id], lang) if it.show_id in shows else None,
shows[it.show_id].rating_key if it.show_id in shows else None,
lang=lang,
)
for it in items
]
+7 -3
View File
@@ -71,7 +71,7 @@ def sync_status(user: User = Depends(current_user), db: Session = Depends(get_db
def my_status(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
"""Per-user sync progress: how far the channels *this* user subscribes to have synced."""
sub_join = and_(Subscription.channel_id == Channel.id, Subscription.user_id == user.id)
total, details_synced, recent_synced, deep_done = db.execute(
total, details_synced, recent_synced, all_deep_done = db.execute(
select(
func.count(Channel.id),
func.count(Channel.details_synced_at),
@@ -82,7 +82,7 @@ def my_status(user: User = Depends(current_user), db: Session = Depends(get_db))
.join(Subscription, sub_join)
).one()
total = total or 0
deep_done = int(deep_done or 0)
all_deep_done = int(all_deep_done or 0)
# Per-user full-history (deep) backfill: channels this user has opted in, and an ETA
# for the still-pending ones (quota-bound estimate).
@@ -97,6 +97,10 @@ def my_status(user: User = Depends(current_user), db: Session = Depends(get_db))
.all()
)
deep_eta = estimate_deep_backfill(db, deep_requested_channels)
# "done / requested" is shown as one ratio, so both halves count the SAME channels: the ones this
# user asked full history for. Counting every subscribed channel whose back-catalog happens to be
# complete (another user asked, or it was short) gave "342/329".
deep_done = sum(1 for c in deep_requested_channels if c.backfill_done)
my_videos = db.scalar(
select(func.count(Video.id)).join(
@@ -117,7 +121,7 @@ def my_status(user: User = Depends(current_user), db: Session = Depends(get_db))
"channels_recent_synced": recent_synced,
"channels_deep_done": deep_done,
"channels_recent_pending": total - recent_synced,
"channels_deep_pending": total - deep_done,
"channels_deep_pending": total - all_deep_done,
"channels_deep_requested": len(deep_requested_channels),
"deep_pending_count": deep_eta["channels_pending"],
"deep_eta_seconds": deep_eta["eta_seconds"],
+67 -11
View File
@@ -18,6 +18,7 @@ from app.downloads.gc import run_download_gc
from app.models import SchedulerSetting, User
from app.plex.sync import sync as run_plex_sync
from app.plex.tmdb import run_enrich as run_tmdb_enrich
from app.plex.tmdb import run_people_enrich
from app.plex.watch_sync import run_plex_watch_reconcile, run_plex_watch_sync
from app.notifications import create_notification
from app.state import is_sync_paused
@@ -69,6 +70,7 @@ JOB_INTERVALS: dict[str, int] = {
"plex_watch_sync": settings.plex_watch_sync_interval_min,
"plex_watch_reconcile": settings.plex_watch_reconcile_interval_min,
"tmdb_enrich": settings.tmdb_enrich_interval_min,
"people_enrich": settings.people_enrich_interval_min,
"audit_gc": settings.audit_gc_minutes,
"log_gc": settings.log_gc_minutes,
}
@@ -115,7 +117,9 @@ def _record(name: str, **fields) -> None:
_activity.setdefault(name, {}).update(fields)
def _notify_done(db, actor_id: int, job_id: str, status: str, summary: str | None) -> None:
def _notify_done(
db, actor_id: int, job_id: str, status: str, summary: str | None, result: dict | None = None
) -> None:
"""Post a completion notice to the triggering admin's inbox (manual runs only). Best
effort — a notification failure must never mask the job's own outcome."""
try:
@@ -125,7 +129,7 @@ def _notify_done(db, actor_id: int, job_id: str, status: str, summary: str | Non
type="scheduler",
title=f"{job_id}: {status}", # English fallback; UI renders translated from data
body=summary,
data={"job_id": job_id, "status": status, "summary": summary},
data={"job_id": job_id, "status": status, "summary": summary, "result": result},
)
except Exception:
db.rollback()
@@ -140,7 +144,12 @@ def _run_changed(result) -> bool:
if result is None:
return False
if isinstance(result, dict):
return any(isinstance(v, (int, float)) and v for v in result.values())
# A phased job (backfill's recent/deep, people_enrich's bios/portraits) nests a result per
# phase: a count inside a phase is a change too.
return any(
_run_changed(v) if isinstance(v, dict) else isinstance(v, (int, float)) and bool(v)
for v in result.values()
)
if isinstance(result, (int, float)):
return result != 0
return bool(result)
@@ -152,6 +161,9 @@ def _audit_run(db, actor_id, job_id, status, summary, result) -> None:
they changed something, so routine no-op polls don't spam the trail. Best-effort."""
if actor_id is None and status == "ok" and not _run_changed(result):
return
# The result as data too (E30): the audit log names it in the reader's language, as the inbox
# does; `summary` stays the English line of the row's own summary column.
data = _result_data(result)
try:
audit.record(
db,
@@ -160,7 +172,7 @@ def _audit_run(db, actor_id, job_id, status, summary, result) -> None:
target_type="job",
target_id=job_id,
summary=f"{job_id}: {status}" + (f" — {summary}" if summary else ""),
after={"status": status, "summary": summary},
after={"status": status, "summary": summary, **({"result": data} if data else {})},
)
db.commit()
except Exception:
@@ -192,28 +204,49 @@ def _job(name: str, fn) -> None:
status="skipped",
last_finished=_now_iso(),
last_result="paused",
result=_PAUSED,
progress=None,
)
if actor_id is not None:
_notify_done(db, actor_id, name, "skipped", "sync paused")
_audit_run(db, actor_id, name, "skipped", "sync paused", None)
_notify_done(db, actor_id, name, "skipped", "sync paused", _PAUSED)
_audit_run(db, actor_id, name, "skipped", "sync paused", _PAUSED)
return
with progress.bind(sink):
result = fn(db)
summary = _summarize(result)
data = _result_data(result)
log.info("job %s -> %s", name, result)
_record(name, status="ok", last_finished=_now_iso(), last_result=summary, progress=None)
_record(
name,
status="ok",
last_finished=_now_iso(),
last_result=summary,
result=data,
progress=None,
)
if actor_id is not None:
_notify_done(db, actor_id, name, "ok", summary)
_notify_done(db, actor_id, name, "ok", summary, data)
_audit_run(db, actor_id, name, "ok", summary, result)
except Exception as exc:
db.rollback()
log.exception("job %s failed", name, extra=logs.event("scheduler.job.failed", job=name))
err = str(exc) or exc.__class__.__name__
_record(name, status="error", last_finished=_now_iso(), last_error=err, progress=None)
# A failure the job named (progress.JobFailed) rides along as data, as a result does, so the
# client says it in the reader's language; `err` stays the English line.
failure = (
{"failure": exc.reason, **exc.params} if isinstance(exc, progress.JobFailed) else None
)
_record(
name,
status="error",
last_finished=_now_iso(),
last_error=err,
result=failure,
progress=None,
)
if actor_id is not None:
_notify_done(db, actor_id, name, "error", err)
_audit_run(db, actor_id, name, "error", err, None)
_notify_done(db, actor_id, name, "error", err, failure)
_audit_run(db, actor_id, name, "error", err, failure)
finally:
# ALWAYS release the claim, even if the body raised before its own status update — so a
# failure can never wedge the job as permanently "running".
@@ -221,6 +254,21 @@ def _job(name: str, fn) -> None:
db.close()
# A skipped run's result, in the same shape as a job's own `{"skipped": …}`.
_PAUSED = {"skipped": "sync paused"}
def _result_data(result) -> dict | None:
"""A job's result as the client names it in the reader's language (E30): the dict itself (counts,
a `skipped` / `reason` word, a nested dict per phase), `enrich`'s bare count as one. The one-line
`_summarize` is the English fallback."""
if isinstance(result, dict):
return result or None
if isinstance(result, int) and not isinstance(result, bool):
return {"enriched": result}
return None
def _summarize(result) -> str | None:
"""Compact one-line rendering of a job's return value for the dashboard."""
if result is None:
@@ -259,6 +307,7 @@ def scheduler_snapshot() -> dict:
"last_started": a.get("last_started"),
"last_finished": a.get("last_finished"),
"last_result": a.get("last_result"),
"result": a.get("result"),
"last_error": a.get("last_error"),
"progress": a.get("progress"),
}
@@ -369,6 +418,12 @@ def _tmdb_enrich_job() -> None:
_job("tmdb_enrich", run_tmdb_enrich)
def _people_enrich_job() -> None:
# Make the people local (E30 S6): a batch of person bios from TMDB and a batch of portraits resized
# through Plex, so a person page and a cast strip never wait on an outside service.
_job("people_enrich", run_people_enrich)
def _audit_gc_job() -> None:
# Prune audit-log rows older than the admin-set retention (audit_retention_days; 0 = keep
# forever). Pure DB, no quota.
@@ -415,6 +470,7 @@ JOB_FUNCS: dict[str, Callable[[], None]] = {
"plex_watch_sync": _plex_watch_sync_job,
"plex_watch_reconcile": _plex_watch_reconcile_job,
"tmdb_enrich": _tmdb_enrich_job,
"people_enrich": _people_enrich_job,
"audit_gc": _audit_gc_job,
"log_gc": _log_gc_job,
}
+3 -1
View File
@@ -115,7 +115,9 @@ _TOPIC_SLUGS = {
"Tourism": "Travel",
"Fashion": "Fashion & Beauty",
"Physical_attractiveness": "Fashion & Beauty",
"Humor": "Comedy",
# YouTube links the British page (…/wiki/Humour); the key was "Humor", so no channel ever got Comedy
# from its topic (E30).
"Humour": "Comedy",
"Entertainment": "Entertainment",
"Film": "Film",
"Television_program": "TV",
+21 -6
View File
@@ -83,6 +83,8 @@ def run_rss_poll(db: Session, channels: list[Channel] | None = None) -> dict[str
# outage from one permanently dead feed id — which is the whole difference on an instance with
# a single subscription, where BOTH render as 100% failed.
first_error: str | None = None
# The same first failure as data, for the reader's language: whose feed, and why (`reasons`' key).
first_failure: tuple[str, str] | None = None
# What failed, as counts per reason (`http_404`, `transport`, `apply`, …) plus a few examples —
# the ONE warning a poll writes. One line per failed channel was what turned YouTube's RSS outage
# of 2026-09-16 into 11k warnings in four hours; each channel's own line is DEBUG now.
@@ -92,18 +94,19 @@ def run_rss_poll(db: Session, channels: list[Channel] | None = None) -> dict[str
# hand the same rows to the same broken write) — the feed fallback's candidates.
unread: list[str] = []
def fail(reason: str, detail: str, *, unexpected: str | None = None) -> None:
def fail(reason: str, channel_id: str, detail: str, *, unexpected: str | None = None) -> None:
"""Count a channel that did not get polled. `unexpected` is the message for a failure that
is a bug on our side: its FIRST occurrence per reason gets a traceback at ERROR, the rest
DEBUG — a read-only database fails every channel the same way, and 979 identical tracebacks
tell nobody anything the first one didn't. Call it from inside the `except` block."""
nonlocal failed, first_error
nonlocal failed, first_error, first_failure
if unexpected is not None:
level = logging.ERROR if reasons[reason] == 0 else logging.DEBUG
log.log(level, unexpected, exc_info=True)
failed += 1
reasons[reason] += 1
first_error = first_error or detail
first_failure = first_failure or (channel_id, reason)
if len(examples) < RSS_FAILURE_EXAMPLES:
examples.append(detail)
@@ -125,13 +128,16 @@ def run_rss_poll(db: Session, channels: list[Channel] | None = None) -> dict[str
# bug on our side and earns a traceback (the first of its kind per poll).
except RssError as exc:
log.debug("RSS poll skipped for channel %s: %s", channel.id, exc)
fail(exc.reason, f"{channel.id}: {exc}")
fail(exc.reason, channel.id, f"{channel.id}: {exc}")
unread.append(channel.id)
# silent-ok: `fail(unexpected=…)` is the report — it logs at ERROR with the
# traceback, once per reason per poll, which is the flood control this loop needs.
except Exception as exc: # noqa: BLE001 — one channel must not stop the poll
fail(
"error", f"{channel.id}: {exc}", unexpected=f"RSS fetch failed for {channel.id}"
"error",
channel.id,
f"{channel.id}: {exc}",
unexpected=f"RSS fetch failed for {channel.id}",
)
else:
try:
@@ -144,6 +150,7 @@ def run_rss_poll(db: Session, channels: list[Channel] | None = None) -> dict[str
db.rollback()
fail(
"apply",
channel.id,
f"{channel.id} (apply): {exc}",
unexpected=f"RSS apply failed for {channel.id}",
)
@@ -180,9 +187,17 @@ def run_rss_poll(db: Session, channels: list[Channel] | None = None) -> dict[str
# "fix this subscription"; `Connection refused` says "the egress is down".
# The fallback's count rides along: a red card during an RSS outage should still say
# whether the uploads are arriving another way.
raise RuntimeError(
# As data too (E31): the audit log and the inbox say it in the reader's language, the first
# failure as its channel and its reason code (`http_404` vs `transport` is that difference).
first_channel, first_reason = first_failure or ("", "error")
raise progress.JobFailed(
"rss_all_failed",
f"RSS poll failed for all {total} channels — first: {first_error}"
f" (read through the API instead: {fallback['read']})"
f" (read through the API instead: {fallback['read']})",
total=total,
channel=first_channel,
cause=first_reason,
read=fallback["read"],
)
return {"new": new + fallback["new"], "failed": failed, "fallback": fallback["read"]}
+33 -8
View File
@@ -10,6 +10,7 @@ stored in the DB and stay env-only).
from dataclasses import dataclass
from sqlalchemy import select
from sqlalchemy.orm import Session
from app import logs, security
@@ -45,8 +46,8 @@ class ConfigSpec:
# Registry of DB-overridable keys. Add a key here + wire its read through get_*() to move it
# off env into the admin UI; the admin page renders it automatically from this list. The ORDER is the
# admin page's tab order (first appearance of each group), so keep a group's keys together.
# off env into the admin UI; the admin page renders it automatically from this list. A group's keys
# stay together, in the order its section lists them; a NEW group also needs its place in SECTIONS.
SPECS: tuple[ConfigSpec, ...] = (
# --- Access / registration ---
ConfigSpec("allow_registration", "bool", "access", "allow_registration"),
@@ -219,6 +220,16 @@ SPECS: tuple[ConfigSpec, ...] = (
# is a config.py default — both stay out of SPECS, like plex_sync_interval_min.)
)
# The admin page's sections (E31, the owner's pick): sixteen groups as tabs wrapped to three rows at 133%
# zoom, so they are a vertical list under four headings, in this order. Every group of SPECS sits in
# exactly one section (test-held), so a new group cannot fall off the page.
SECTIONS: tuple[tuple[str, tuple[str, ...]], ...] = (
("access", ("access", "google", "email")),
("youtube", ("youtube", "quota", "backfill", "shorts", "batch", "explore")),
("media", ("downloads", "plex", "tmdb")),
("system", ("maintenance", "audit", "diagnostics", "issues")),
)
_BY_KEY: dict[str, ConfigSpec] = {s.key: s for s in SPECS}
@@ -297,7 +308,11 @@ def _raw_override(db: Session, key: str) -> str | None:
per key so the operator gets a real signal without a per-request traceback. Recovery is documented in
docs/self-hosting.md: the key must never change once secrets are stored. (B-core-2)
"""
row = db.get(SystemConfig, key)
return _raw_of(db.get(SystemConfig, key), key)
def _raw_of(row: SystemConfig | None, key: str) -> str | None:
"""`_raw_override` over a row already read (describe() reads them all in one query)."""
if row is None or row.value is None:
return None
if not row.encrypted:
@@ -317,8 +332,11 @@ def _raw_override(db: Session, key: str) -> str | None:
def get(db: Session, key: str):
"""Effective value: the DB override (typed) if set, else the env/config default."""
s = _BY_KEY[key]
raw = _raw_override(db, key)
return _effective(_BY_KEY[key], _raw_override(db, key))
def _effective(s: ConfigSpec, raw: str | None):
"""`get` over an override already read."""
if raw is None:
return _default(s)
if raw == "" and not s.allow_empty:
@@ -410,8 +428,12 @@ def describe(db: Session) -> dict:
"""Admin-UI metadata + current values, grouped. Secret values are never echoed —
only whether an override is set and whether an env default exists."""
groups: dict[str, list[dict]] = {}
# Every override in one query: a db.get per key was one round trip each (two for a key with no
# row, which the session cannot cache), ~40 ms of the page's ~55 on prod (E31 S4d).
rows = {r.key: r for r in db.scalars(select(SystemConfig))}
for s in SPECS:
is_set = db.get(SystemConfig, s.key) is not None
row = rows.get(s.key)
is_set = row is not None
item: dict = {
"key": s.key,
"type": s.type,
@@ -426,7 +448,10 @@ def describe(db: Session) -> dict:
if s.secret:
item["default_is_set"] = bool(_default(s))
else:
item["value"] = get(db, s.key)
item["value"] = _effective(s, _raw_of(row, s.key))
item["default"] = _default(s)
groups.setdefault(s.group, []).append(item)
return {"groups": groups, "secrets_manageable": secrets_manageable()}
sections = [
{"id": sid, "groups": [g for g in members if g in groups]} for sid, members in SECTIONS
]
return {"groups": groups, "sections": sections, "secrets_manageable": secrets_manageable()}
+10 -2
View File
@@ -60,16 +60,24 @@ def now_utc() -> datetime:
return datetime.now(timezone.utc)
# The shortest last word that is prefix-matched. A 1–2 letter prefix matches a large share of the
# library and ts_rank then scores every match: "m" took ~5–8 s on prod against ~100 ms for a whole
# word (E31 S4e). A shorter last word is matched whole instead ("tv", "4k" still find themselves).
MIN_PREFIX_LEN = 3
def to_tsquery_str(q: str) -> str | None:
"""Turn free-text into a to_tsquery() string: word runs ANDed together, the LAST word
prefix-matched (`:*`) for as-you-type search — e.g. "ti amo magyar" → "ti & amo & magyar:*".
prefix-matched (`:*`) for as-you-type search — e.g. "ti amo magyar" → "ti & amo & magyar:*" —
once it has MIN_PREFIX_LEN characters ("ti amo m" → "ti & amo & m").
Only `\\w` runs containing an alphanumeric are kept, so no tsquery operators can leak in
(the string is built from sanitised tokens, never the raw query). Shared by the YouTube feed
search and the Plex library search, hence this neutral home rather than either domain module."""
tokens = [t for t in re.findall(r"\w+", q, flags=re.UNICODE) if any(c.isalnum() for c in t)]
if not tokens:
return None
tokens[-1] = tokens[-1] + ":*"
if len(tokens[-1]) >= MIN_PREFIX_LEN:
tokens[-1] = tokens[-1] + ":*"
return " & ".join(tokens)
+36
View File
@@ -0,0 +1,36 @@
"""E30: an admin action that counts something keeps the count as data in its audit row's `after`, under
the key the audit log names in the reader's language (`scheduler.result.<key>`, frontend
auditColumns.tsx) — not only in the English `summary`. A renamed key would fall back to the raw word.
DB lane (Postgres).
"""
from sqlalchemy import select
from app.models import AuditLog
def _after(db, action: str) -> dict:
db.expire_all()
return db.execute(select(AuditLog.after).where(AuditLog.action == action)).scalar_one()
def test_clearing_the_audit_log_keeps_the_count_as_data(as_user, db, make_user):
admin = make_user(email="boss@example.com", role="admin")
db.add_all([AuditLog(action="sync_pause"), AuditLog(action="sync_resume")])
db.commit()
assert as_user(admin).delete("/api/admin/audit").json()["cleared"] == 2
assert _after(db, "audit_clear") == {"cleared": 2}
def test_a_demo_reset_keeps_the_count_as_data(as_user, db, make_user):
admin = make_user(email="boss@example.com", role="admin")
seeded = as_user(admin).post("/api/admin/demo/reset").json()["playlists_seeded"]
assert _after(db, "demo_reset") == {"playlists_seeded": seeded}
def test_a_discovery_purge_keeps_its_counts_as_data(as_user, db, make_user):
admin = make_user(email="boss@example.com", role="admin")
body = as_user(admin).post("/api/admin/purge-discovery").json()
body.pop("ok")
assert _after(db, "discovery_purge") == body
assert {"videos_deleted", "channels_deleted", "search_videos_deleted"} <= set(body)
+24
View File
@@ -0,0 +1,24 @@
"""A YouTube topic URL's slug → the auto-tag it gives (pure). The slugs are the ones YouTube actually
links (measured on the owner's channels, 2026-09-29): a key spelled differently from YouTube's page
never matches, silently — "Humor" sat in the map for months while YouTube links …/wiki/Humour."""
import pytest
from app.sync.autotag import map_topic_slug
@pytest.mark.parametrize(
("slug", "tag"),
[
("Humour", "Comedy"),
("Video_game_culture", "Gaming"),
("Electronic_music", "Music"),
("Association_football", "Sports"),
("Knowledge", "Education"),
("Physical_fitness", "Health & Fitness"),
("Television_program", "TV"),
("Lifestyle_(sociology)", None), # YouTube's catch-all: deliberately no tag
],
)
def test_a_youtube_topic_slug_gives_its_tag(slug, tag):
assert map_topic_slug(slug) == tag
@@ -0,0 +1,83 @@
"""The channel manager's sums and the feed sidebar's counts read the index alone (E31 S4d).
`GET /api/channels` counts each subscribed channel's videos and sums their durations; `/feed/summary`
and `/feed/years` count the reader's videos per channel and per year under the default filters. They
used to read every video's heap page for that (681 MB on prod), a 2–6 s first open once something else
had pushed the table out of memory. Migration 0095 makes `ix_videos_channel_pub_cover` carry every
column those queries read, so Postgres answers them with Index Only Scans. This test holds the queries
and the index together: a column added to one and not the other fails it. DB lane.
"""
import re
from sqlalchemy import text
from sqlalchemy.dialects import postgresql
from app.feedquery import filtered_query
from app.models import Video
from app.routes.channels import channel_aggregates
INDEX = "ix_videos_channel_pub_cover"
# The feed's own defaults (routes/feed.py _feed_params): what the sidebar asks on a plain open.
FEED_DEFAULTS = {
"tags": [],
"tag_modes": {"language": "or", "topic": "or", "other": "or"},
"untagged": False,
"channel_ids": [],
"q": None,
"min_duration": None,
"max_duration": None,
"max_age_days": None,
"published_after": None,
"published_before": None,
"show_normal": True,
"include_shorts": False,
"include_live": False,
"show": "unwatched",
"scope": "my",
"library_source": "organic",
}
def _index_columns(db) -> list[str]:
# Key columns first, then the INCLUDE ones, in the index's own order.
return list(
db.execute(
text(
"SELECT a.attname FROM pg_index i "
"JOIN pg_class c ON c.oid = i.indexrelid "
"CROSS JOIN LATERAL unnest(i.indkey) WITH ORDINALITY AS k(attnum, n) "
"JOIN pg_attribute a ON a.attrelid = i.indrelid AND a.attnum = k.attnum "
"WHERE c.relname = :name ORDER BY k.n"
),
{"name": INDEX},
).scalars()
)
def _video_columns(stmt) -> set[str]:
# The columns of `videos` the statement as SENT reads: the compiled SQL, not the expression tree,
# which still carries the projection a with_only_columns replaced.
sql = str(stmt.compile(dialect=postgresql.dialect()))
return set(re.findall(r"\bvideos\.(\w+)", sql))
def _assert_covered(db, read: set[str]) -> None:
held = _index_columns(db)
assert held, f"{INDEX} is missing"
# It leads with channel_id, so it also serves every per-channel lookup the old index did.
assert held[0] == "channel_id"
assert read <= set(held), f"read but not in {INDEX}: {sorted(read - set(held))}"
def test_the_channel_sums_read_only_columns_the_index_holds(db):
_assert_covered(db, _video_columns(channel_aggregates(["UC_x"])))
def test_the_feed_sidebar_counts_read_only_columns_the_index_holds(db, make_user):
# feed_summary groups the default filter by channel, feed_years by publish date.
base, _rank = filtered_query(db, make_user(), **FEED_DEFAULTS)
_assert_covered(
db, _video_columns(base.with_only_columns(Video.channel_id, Video.published_at))
)
+302
View File
@@ -0,0 +1,302 @@
"""E30 S3: every A→Z list the server orders is in the reader's alphabet. The database image is Alpine,
whose `en_US.utf8` is BYTE order: "Életrajzi" and "Ötödik" sorted after "Zorró" in the feed, the channel
list, the Plex grid, the genre list. Hungarian puts "Ö" after "O" and "Cs" after "Cu". Needs the DB."""
from fastapi.testclient import TestClient
import app.state as state
from app.auth import current_user
from app.collation import sorted_words
from app.db import get_db
from app.main import app
from app.models import (
Channel,
Playlist,
PlaylistItem,
PlexCollection,
PlexItem,
PlexLibrary,
Subscription,
User,
Video,
)
from datetime import datetime, timezone
WORDS = ["Zorró", "Életrajzi", "Ötödik", "oroszlán", "Csillag", "Cukor"]
HU = ["Cukor", "Csillag", "Életrajzi", "oroszlán", "Ötödik", "Zorró"]
EN = ["Csillag", "Cukor", "Életrajzi", "oroszlán", "Ötödik", "Zorró"]
def _user(db, lang: str) -> User:
user = User(email=f"collate-{lang}@example.com", preferences={"language": lang})
db.add(user)
db.flush()
return user
def _client(db, user, monkeypatch, override_dep) -> TestClient:
monkeypatch.setattr(state, "_configured_cache", True)
override_dep(get_db, db.info["request"])
override_dep(current_user, lambda: user)
return TestClient(app)
def test_words_sort_in_the_readers_alphabet(db):
assert sorted_words(db, WORDS, "hu") == HU
assert sorted_words(db, WORDS, "en") == EN
assert sorted_words(db, WORDS, "xx") == EN # an unknown language sorts as English
assert sorted_words(db, [], "hu") == []
def _seed_feed(db, user):
db.add(Channel(id="UCc", title="Channel"))
db.flush()
db.add(Subscription(user_id=user.id, channel_id="UCc", priority=0, hidden=False))
db.add_all(
Video(
id=f"v{i}",
channel_id="UCc",
published_at=datetime(2024, 1, i + 1, tzinfo=timezone.utc),
title=w,
is_short=False,
live_status="none",
via_search=False,
via_explore=False,
)
for i, w in enumerate(WORDS)
)
db.commit()
def test_the_feed_title_sort_pages_in_the_readers_alphabet(db, monkeypatch, override_dep):
user = _user(db, "hu")
_seed_feed(db, user)
client = _client(db, user, monkeypatch, override_dep)
full = [it["title"] for it in client.get("/api/feed?sort=title&limit=50").json()["items"]]
assert full == HU
# The cursor compares with the same collated expression, so paging walks the same order.
walked: list[str] = []
cursor = None
for _ in range(len(WORDS)):
url = "/api/feed?sort=title&limit=2" + (f"&cursor={cursor}" if cursor else "")
body = client.get(url).json()
walked += [it["title"] for it in body["items"]]
cursor = body["next_cursor"]
if not body["has_more"]:
break
assert walked == HU
desc = client.get("/api/feed?sort=title_desc&limit=50").json()["items"]
assert [it["title"] for it in desc] == HU[::-1]
def test_the_channel_list_is_in_the_readers_alphabet(db, monkeypatch, override_dep):
user = _user(db, "hu")
db.add_all(Channel(id=f"UC{i}", title=w) for i, w in enumerate(WORDS))
db.flush()
db.add_all(
Subscription(user_id=user.id, channel_id=f"UC{i}", priority=0, hidden=False)
for i in range(len(WORDS))
)
db.commit()
r = _client(db, user, monkeypatch, override_dep).get("/api/channels")
assert [c["title"] for c in r.json()] == HU
def _seed_plex(db):
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
col = PlexCollection(rating_key="col1", library_id=lib.id, title="Set", child_count=len(WORDS))
db.add(col)
db.add_all(
PlexItem(
rating_key=f"m{i}",
library_id=lib.id,
kind="movie",
title=f"EN {i}",
title_hu=w,
year=2000,
genres_hu=[w],
collection_keys=["col1"],
)
for i, w in enumerate(WORDS)
)
db.commit()
def _plex(db, monkeypatch, override_dep) -> TestClient:
user = _user(db, "hu")
_seed_plex(db)
return _client(db, user, monkeypatch, override_dep)
def test_the_plex_grid_sorts_by_the_hungarian_title(db, monkeypatch, override_dep):
client = _plex(db, monkeypatch, override_dep)
grid = client.get("/api/plex/library?scope=movie&sort=title&sort_dir=asc&lang=hu").json()
assert [it["title"] for it in grid["items"]] == HU
def test_the_plex_genre_list_is_in_the_readers_alphabet(db, monkeypatch, override_dep):
client = _plex(db, monkeypatch, override_dep)
facets = client.get("/api/plex/facets?scope=movie&lang=hu").json()
assert [g["value"] for g in facets["genres"]] == HU
def test_a_collection_sorts_by_the_hungarian_title(db, monkeypatch, override_dep):
client = _plex(db, monkeypatch, override_dep)
members = client.get("/api/plex/collections/col1?sort=title&sort_dir=asc&lang=hu").json()
assert [it["title"] for it in members["items"]] == HU
# year ties everywhere, so the default (year) order falls through to the title tiebreak
default = client.get("/api/plex/collections/col1?lang=hu").json()
assert [it["title"] for it in default["items"]] == HU
def _seed_discovery(db, user, synced: bool):
"""Channels the user's playlist holds one video from each, not subscribed: discovery lists them,
count-tied, so the title decides. `synced=False` leaves them stubs, which sends the endpoint down
its enrich path — the one that re-sorts in Python and has to agree with the ORDER BY."""
pl = Playlist(user_id=user.id, name="Mix")
db.add(pl)
db.flush()
stamp = datetime(2024, 1, 1, tzinfo=timezone.utc) if synced else None
for i, w in enumerate(WORDS):
db.add(Channel(id=f"UCd{i}", title=w, details_synced_at=stamp))
db.flush()
db.add(
Video(
id=f"d{i}",
channel_id=f"UCd{i}",
published_at=datetime(2024, 1, 1, tzinfo=timezone.utc),
title="x",
is_short=False,
live_status="none",
via_search=False,
via_explore=False,
)
)
db.flush()
db.add(PlaylistItem(playlist_id=pl.id, video_id=f"d{i}", position=i))
db.commit()
def test_channel_discovery_is_in_the_readers_alphabet(db, monkeypatch, override_dep):
user = _user(db, "hu")
_seed_discovery(db, user, synced=True)
r = _client(db, user, monkeypatch, override_dep).get("/api/channels/discovery")
assert [c["title"] for c in r.json()] == HU
def test_channel_discovery_keeps_that_order_after_enriching(db, monkeypatch, override_dep):
import app.routes.channels as channels
class FakeYouTube:
def __init__(self, *a, **k): ...
def __enter__(self):
return self
def __exit__(self, *a):
return False
def get_channels(self, ids):
return []
monkeypatch.setattr(channels, "YouTubeClient", FakeYouTube)
monkeypatch.setattr(channels, "apply_channel_details", lambda *a, **k: None)
user = _user(db, "hu")
_seed_discovery(db, user, synced=False)
r = _client(db, user, monkeypatch, override_dep).get("/api/channels/discovery")
assert [c["title"] for c in r.json()] == HU
# --- The other lists the server puts in the reader's alphabet (E30 review): each call site, once. ---
def _others(db, n_offset: int = 0) -> list[User]:
users = [
User(email=f"{w.lower()}@example.com", display_name=w) for w in WORDS
] # emails and names in WORDS' order
db.add_all(users)
db.flush()
return users
def test_the_share_and_message_recipients_are_in_the_readers_alphabet(
db, monkeypatch, override_dep
):
from app.models import MessageKey
user = _user(db, "hu")
others = _others(db)
db.add_all(
MessageKey(
user_id=u.id,
public_key="k",
wrapped_private_key="w",
salt="s",
wrap_iv="i",
key_check="c",
)
for u in others
)
db.commit()
client = _client(db, user, monkeypatch, override_dep)
emails = [r["email"] for r in client.get("/api/downloads/recipients").json()]
assert emails == [f"{w.lower()}@example.com" for w in HU]
names = [r["name"] for r in client.get("/api/messages/users").json()]
assert names == HU
def test_the_blocked_channels_are_in_the_readers_alphabet(db, monkeypatch, override_dep):
from app.models import BlockedChannel
user = _user(db, "hu")
db.add_all(Channel(id=f"UC{i}", title=w) for i, w in enumerate(WORDS))
db.flush()
db.add_all(BlockedChannel(user_id=user.id, channel_id=f"UC{i}") for i in range(len(WORDS)))
db.commit()
r = _client(db, user, monkeypatch, override_dep).get("/api/channels/blocked/list")
assert [c["title"] for c in r.json()] == HU
def test_the_plex_libraries_and_collections_are_in_the_readers_alphabet(
db, monkeypatch, override_dep
):
user = _user(db, "hu")
libs = [
PlexLibrary(plex_key=str(i), title=w, kind="movie", enabled=True)
for i, w in enumerate(WORDS)
]
db.add_all(libs)
db.flush()
db.add_all(
PlexCollection(rating_key=f"c{i}", library_id=libs[0].id, title=w, child_count=1)
for i, w in enumerate(WORDS)
)
db.commit()
client = _client(db, user, monkeypatch, override_dep)
assert [lib["title"] for lib in client.get("/api/plex/libraries").json()["libraries"]] == HU
cols = client.get("/api/plex/collections").json()["collections"]
assert [c["title"] for c in cols] == HU
def test_a_persons_titles_of_one_year_are_in_the_readers_alphabet(db, monkeypatch, override_dep):
user = _user(db, "hu")
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
db.add_all(
PlexItem(
rating_key=f"m{i}",
library_id=lib.id,
kind="movie",
title=f"EN {i}",
title_hu=w,
year=2000,
cast_names=["Anna"],
)
for i, w in enumerate(WORDS)
)
db.commit()
body = _client(db, user, monkeypatch, override_dep).get("/api/plex/person/Anna?lang=hu").json()
assert [m["title"] for m in body["movies"]] == HU
+53 -39
View File
@@ -180,7 +180,7 @@ def test_continue_watching_merges_both_sources_by_recency(db):
)
db.commit()
got = rails.continue_watching(db, user)
got = rails.continue_watching(db, user, lang="en")
assert [(c["source"], c["id"]) for c in got] == [("plex", "pm1"), ("youtube", "vmed")]
assert got[0]["remaining_seconds"] == 100 * 60 - 600
@@ -208,7 +208,7 @@ def test_continue_watching_is_youtube_only_when_plex_is_off(db):
]
)
db.commit()
assert {c["source"] for c in rails.continue_watching(db, user)} == {"youtube"}
assert {c["source"] for c in rails.continue_watching(db, user, lang="en")} == {"youtube"}
def test_the_resume_rail_obeys_the_channel_blocklist(db):
@@ -227,13 +227,13 @@ def test_the_resume_rail_obeys_the_channel_blocklist(db):
)
)
db.commit()
assert [c["id"] for c in rails.continue_watching(db, user)] == ["vmed"]
assert [c["id"] for c in rails.continue_watching(db, user, lang="en")] == ["vmed"]
db.add(BlockedChannel(user_id=user.id, channel_id=ch.id))
db.commit()
assert rails.continue_watching(db, user) == []
assert rails.continue_watching(db, user, lang="en") == []
# And so the hero, whose resume tier draws from that rail, has nothing to offer either.
assert hero.pick(db, user, "any", 0) is None
assert hero.pick(db, user, "any", 0, lang="en") is None
def test_a_null_timestamp_sorts_last(db):
@@ -263,7 +263,7 @@ def test_a_null_timestamp_sorts_last(db):
]
)
db.commit()
got = rails.continue_watching(db, user)
got = rails.continue_watching(db, user, lang="en")
assert got[0]["id"] == "vshort"
assert len(got) == 3
@@ -283,7 +283,7 @@ def test_hero_prefers_resume_and_labels_the_tier(db):
)
)
db.commit()
got = hero.pick(db, user, "any", seed=1)
got = hero.pick(db, user, "any", seed=1, lang="en")
assert got["tier"] == "resume"
assert got["card"]["id"] == "vmed"
@@ -302,7 +302,7 @@ def test_hero_skips_a_resume_that_does_not_fit_the_budget(db):
)
)
db.commit()
got = hero.pick(db, user, "short", seed=1)
got = hero.pick(db, user, "short", seed=1, lang="en")
assert got["tier"] == "surprise"
assert got["card"]["duration_seconds"] <= BANDS["short"][1]
@@ -312,7 +312,7 @@ def test_hero_falls_back_to_the_shelf_before_guessing(db):
user, ch = _seed_yt(db)
db.add(ComfortPin(user_id=user.id, video_id="vmed", position=0))
db.commit()
got = hero.pick(db, user, "medium", seed=1)
got = hero.pick(db, user, "medium", seed=1, lang="en")
assert got["tier"] == "comfort"
assert got["card"]["id"] == "vmed"
@@ -332,7 +332,7 @@ def test_hero_resolves_a_show_to_the_episode_that_would_play(db):
)
)
db.commit()
got = hero.pick(db, user, "any", seed=1)
got = hero.pick(db, user, "any", seed=1, lang="en")
assert got["tier"] == "resume"
assert got["card"]["kind"] == "episode"
assert got["card"]["id"] == "ps1e2" # ep1 is finished → ep2 is what plays
@@ -345,17 +345,19 @@ def test_hero_is_none_when_nothing_fits(db):
user, ch = _seed_yt(db)
db.execute(text("DELETE FROM videos WHERE id = 'vlong'"))
db.commit()
assert hero.pick(db, user, "long", seed=1) is None
assert hero.pick(db, user, "long", seed=1, lang="en") is None
def test_surprise_is_reproducible_for_a_seed(db):
"""The reroll button depends on this: one seed drives both sources, so the same seed must give
the same answer and a different seed must be free to give another."""
user, _ = _seed_yt(db)
a = rails.surprise(db, user, "any", seed=7, limit=3)
b = rails.surprise(db, user, "any", seed=7, limit=3)
a = rails.surprise(db, user, "any", seed=7, limit=3, lang="en")
b = rails.surprise(db, user, "any", seed=7, limit=3, lang="en")
assert [c["id"] for c in a] == [c["id"] for c in b]
seeds = {tuple(c["id"] for c in rails.surprise(db, user, "any", s, 3)) for s in range(12)}
seeds = {
tuple(c["id"] for c in rails.surprise(db, user, "any", s, 3, lang="en")) for s in range(12)
}
assert len(seeds) > 1 # the seed actually reaches the ordering
@@ -374,7 +376,9 @@ def test_surprise_never_offers_something_already_started(db):
)
)
db.commit()
assert "vlong" not in {c["id"] for c in rails.surprise(db, user, "long", seed=3, limit=5)}
assert "vlong" not in {
c["id"] for c in rails.surprise(db, user, "long", seed=3, limit=5, lang="en")
}
def test_the_seed_decides_which_source_leads(db):
@@ -384,7 +388,9 @@ def test_the_seed_decides_which_source_leads(db):
user, _ = _seed_yt(db)
_seed_plex(db, user)
_enable_plex(db)
leaders = {rails.surprise(db, user, "any", s, limit=1)[0]["source"] for s in range(6)}
leaders = {
rails.surprise(db, user, "any", s, limit=1, lang="en")[0]["source"] for s in range(6)
}
assert leaders == {"youtube", "plex"}
@@ -413,10 +419,10 @@ def test_the_reroll_must_be_able_to_move_past_the_resume_tier(db):
)
db.commit()
first = hero.pick(db, user, "any", seed=1)
first = hero.pick(db, user, "any", seed=1, lang="en")
assert first["card"]["id"] == "vmed"
# Same seed, same everything — only the exclusion differs.
second = hero.pick(db, user, "any", seed=1, exclude={"youtube:vmed"})
second = hero.pick(db, user, "any", seed=1, exclude={"youtube:vmed"}, lang="en")
assert second["card"]["id"] == "vshort"
@@ -438,7 +444,7 @@ def test_the_reroll_rotates_which_tier_answers(db):
)
db.add(ComfortPin(user_id=user.id, video_id="vlong", position=0))
db.commit()
tiers = [hero.pick(db, user, "any", seed=1, start=n)["tier"] for n in range(4)]
tiers = [hero.pick(db, user, "any", seed=1, start=n, lang="en")["tier"] for n in range(4)]
assert tiers == ["resume", "comfort", "surprise", "resume"]
@@ -447,7 +453,7 @@ def test_the_rotation_still_falls_through_an_empty_tier(db):
Here there is no shelf at all, so the comfort-led press has to answer from somewhere else."""
user, _ = _seed_yt(db)
assert db.execute(select(func.count()).select_from(ComfortPin)).scalar_one() == 0
got = hero.pick(db, user, "any", seed=1, start=1)
got = hero.pick(db, user, "any", seed=1, start=1, lang="en")
assert got is not None and got["tier"] != "comfort"
@@ -468,10 +474,10 @@ def test_excluding_a_show_also_excludes_the_episode_it_resolves_to(db):
)
db.commit()
shown = hero.pick(db, user, "any", seed=1)
shown = hero.pick(db, user, "any", seed=1, lang="en")
assert shown["card"]["id"] == "ps1e1" # the show resolved to its in-progress episode
# Excluding what the USER saw — the show — must move the tier on, not return the episode again.
after = hero.pick(db, user, "any", seed=1, exclude={"plex:ps1"})
after = hero.pick(db, user, "any", seed=1, exclude={"plex:ps1"}, lang="en")
assert after is None or after["card"]["id"] != "ps1e1"
@@ -518,7 +524,7 @@ def test_shelf_resolves_a_channel_to_its_newest_unwatched_video(db):
user, ch = _seed_yt(db)
db.add(ComfortPin(user_id=user.id, channel_id=ch.id, position=0))
db.commit()
(card,) = pins.shelf(db, user)
(card,) = pins.shelf(db, user, lang="en")
assert card["kind"] == "channel"
assert card["unwatched_count"] == 3
assert card["next"]["id"] == "vshort" # newest
@@ -538,7 +544,7 @@ def test_shelf_resolves_a_playlist_to_its_first_unwatched_item(db):
db.add(VideoState(user_id=user.id, video_id="vshort", status="watched", watched_at=NOW))
db.add(ComfortPin(user_id=user.id, playlist_id=pl.id, position=0))
db.commit()
(card,) = pins.shelf(db, user)
(card,) = pins.shelf(db, user, lang="en")
assert card["kind"] == "playlist"
assert card["item_count"] == 2
assert card["next"]["id"] == "vmed"
@@ -585,7 +591,7 @@ def test_the_shelf_resolves_each_channel_and_playlist_on_its_own_terms(db):
got = [
(c.get("unwatched_count", c.get("item_count")), (c["next"] or {}).get("id"))
for c in pins.shelf(db, user)
for c in pins.shelf(db, user, lang="en")
]
assert got == [(3, "vshort"), (1, "vother"), (1, "vlong"), (0, None)]
@@ -601,14 +607,14 @@ def test_the_shelf_costs_the_same_queries_for_one_pin_or_many(db):
db.add(ComfortPin(user_id=user.id, channel_id=ch.id, position=0))
db.add(ComfortPin(user_id=user.id, playlist_id=lists[0].id, position=1))
db.commit()
few = _count_statements(db, lambda: pins.shelf(db, user))
few = _count_statements(db, lambda: pins.shelf(db, user, lang="en"))
for i, c in enumerate(extra):
db.add(ComfortPin(user_id=user.id, channel_id=c.id, position=10 + i))
for i, p in enumerate(lists[1:]):
db.add(ComfortPin(user_id=user.id, playlist_id=p.id, position=20 + i))
db.commit()
assert _count_statements(db, lambda: pins.shelf(db, user)) == few
assert _count_statements(db, lambda: pins.shelf(db, user, lang="en")) == few
def test_a_pinned_video_survives_leaving_the_feed_scope(db):
@@ -619,7 +625,7 @@ def test_a_pinned_video_survives_leaving_the_feed_scope(db):
db.commit()
db.execute(delete(Subscription))
db.commit()
(card,) = pins.shelf(db, user)
(card,) = pins.shelf(db, user, lang="en")
assert card["id"] == "vmed"
assert card["next"]["id"] == "vmed"
@@ -633,7 +639,9 @@ def test_the_plex_surprise_obeys_the_time_budget(db):
_seed_plex(db, user)
_enable_plex(db)
drawn = lambda budget: { # noqa: E731
(c["source"], c["id"]) for s in range(6) for c in rails.surprise(db, user, budget, s, 5)
(c["source"], c["id"])
for s in range(6)
for c in rails.surprise(db, user, budget, s, 5, lang="en")
}
assert ("plex", "pm1") not in drawn("short")
# …and it IS offered where it fits, so this cannot pass by the rail being empty.
@@ -658,7 +666,9 @@ def test_the_plex_surprise_never_offers_something_already_started(db):
)
db.commit()
drawn = {
(c["source"], c["id"]) for s in range(6) for c in rails.surprise(db, user, "long", s, 5)
(c["source"], c["id"])
for s in range(6)
for c in rails.surprise(db, user, "long", s, 5, lang="en")
}
assert ("plex", "pm1") not in drawn
@@ -682,7 +692,7 @@ def test_a_new_pin_lands_at_the_end_even_after_a_hole(db):
db.add(ComfortPin(user_id=user.id, channel_id=ch.id, position=pins.next_position(db, user)))
db.commit()
assert [c["kind"] for c in pins.shelf(db, user)] == ["video", "channel"]
assert [c["kind"] for c in pins.shelf(db, user, lang="en")] == ["video", "channel"]
positions = (
db.execute(select(ComfortPin.position).order_by(ComfortPin.position)).scalars().all()
)
@@ -695,7 +705,7 @@ def test_a_self_playing_pin_does_not_reference_itself(db):
user, _ = _seed_yt(db)
db.add(ComfortPin(user_id=user.id, video_id="vmed", position=0))
db.commit()
(card,) = pins.shelf(db, user)
(card,) = pins.shelf(db, user, lang="en")
assert card["next"] is not card
assert "next" not in card["next"]
@@ -718,7 +728,7 @@ def test_the_resume_rail_says_what_would_actually_play(db):
)
)
db.commit()
rail = rails.continue_watching(db, user)
rail = rails.continue_watching(db, user, lang="en")
tile = next(c for c in rail if c["kind"] == "show")
assert tile["id"] == "ps1" # still the show
assert tile["next"]["kind"] == "episode"
@@ -762,7 +772,7 @@ def test_only_a_show_row_pays_for_a_resolution(db):
)
db.commit()
rail = rails.continue_watching(db, user)
rail = rails.continue_watching(db, user, lang="en")
# A video, a movie and a show — one row of each kind that can appear here, so the assertion below
# actually exercises both sides of the equality.
assert {c["kind"] for c in rail} == {"video", "movie", "show"}
@@ -778,7 +788,7 @@ def test_next_episode_takes_an_episode_with_no_state_row_at_all(db):
_, show, ep1 = _seed_plex(db, user)
db.add(PlexState(user_id=user.id, item_id=ep1.id, status="watched", watched_at=NOW))
db.commit() # ep2 has NO PlexState row
assert queries.next_episode(db, user.id, show.id)["id"] == "ps1e2"
assert queries.next_episode(db, user.id, show.id, lang="en")["id"] == "ps1e2"
def test_a_finished_show_starts_again_from_the_top(db):
@@ -793,7 +803,7 @@ def test_a_finished_show_starts_again_from_the_top(db):
]
)
db.commit()
assert queries.next_episode(db, user.id, show.id)["id"] == "ps1e1"
assert queries.next_episode(db, user.id, show.id, lang="en")["id"] == "ps1e1"
def test_next_episode_agrees_with_the_show_page_when_two_are_in_progress(db):
@@ -825,12 +835,16 @@ def test_next_episode_agrees_with_the_show_page_when_two_are_in_progress(db):
)
db.commit()
got = queries.next_episode(db, user.id, show.id)
got = queries.next_episode(db, user.id, show.id, lang="en")
assert got["id"] == "ps1e2"
# Asserted against the other implementation rather than against a literal, so the two cannot
# drift apart again without this failing.
ordered = [
leaf_card(ep, db.execute(select(PlexState).where(PlexState.item_id == ep.id)).scalar_one())
leaf_card(
ep,
db.execute(select(PlexState).where(PlexState.item_id == ep.id)).scalar_one(),
lang="en",
)
for ep in (ep1, ep2)
]
assert rollup.rollup(ordered)["resume"]["id"] == got["id"]
@@ -848,7 +862,7 @@ def test_next_episode_prefers_the_one_in_progress(db):
]
)
db.commit()
assert queries.next_episode(db, user.id, show.id)["id"] == "ps1e2"
assert queries.next_episode(db, user.id, show.id, lang="en")["id"] == "ps1e2"
# ── HTTP surface ──────────────────────────────────────────────────────────────────────────────────
@@ -0,0 +1,41 @@
"""E30: the "shared with you" notice carries the download's title in its data, so the client can name
the notice in the reader's language. Its stored title is an English sentence ("Cats was shared with
you") — the only place the title lived, so a Hungarian reader got the English. Needs the DB."""
from sqlalchemy import select
from app.models import DownloadJob, MediaAsset, Notification
def test_a_share_notice_carries_the_title_and_the_sharer(db, make_user, as_user):
owner = make_user("owner@example.com")
friend = make_user("friend@example.com")
asset = MediaAsset(
source_kind="url", source_ref="share-e30", format_sig="sig", status="ready", ref_count=1
)
db.add(asset)
db.flush()
job = DownloadJob(
user_id=owner.id,
asset_id=asset.id,
source_kind="url",
source_ref="share-e30",
status="done",
queue_pos=1,
profile_snapshot={},
display_name="Cats",
)
db.add(job)
db.commit()
r = as_user(owner).post(f"/api/downloads/{job.id}/share", json={"email": "friend@example.com"})
assert r.status_code == 200, r.text
notice = db.execute(select(Notification).where(Notification.user_id == friend.id)).scalar_one()
assert notice.type == "download_shared"
assert notice.data == {
"kind": "download_shared",
"from": "owner@example.com",
"job_id": job.id,
"title": "Cats",
}
+133
View File
@@ -0,0 +1,133 @@
"""Relevance ranks only the newest RANK_WINDOW matches (E31), DB-lane.
A broad word ("a") matched most of the prod library and ts_rank read every match's tsvector: 6.8 s a page.
The ranking now covers the newest RANK_WINDOW matches and the rest follow newest first. The window's start
rides the cursor, so a video ingested mid-scroll cannot push a row behind the cursor and repeat it.
"""
import base64
import json
from datetime import datetime, timezone
import pytest
from app import feedquery
from app.models import Channel, Subscription, User, Video
def _video(vid: str, day: int, title: str) -> Video:
return Video(
id=vid,
channel_id="UCw",
published_at=datetime(2024, 1, day, tzinfo=timezone.utc),
title=title,
is_short=False,
live_status="none",
via_search=False,
via_explore=False,
)
def _seed(db) -> User:
db.add(Channel(id="UCw", title="Window"))
user = User(email="window@example.com", email_verified=True)
db.add(user)
db.flush()
db.add(Subscription(user_id=user.id, channel_id="UCw", priority=0, hidden=False))
# m1 is the oldest and the strongest match, m4 the strongest of the newest three; the rest tie on one
# title "foo" each (ts_rank counts occurrences, it does not normalise by length).
db.add_all(
[
_video("m1", 1, "foo foo foo foo foo"),
_video("m2", 2, "foo and some other words here"),
_video("m3", 3, "foo plus some filler words"),
_video("m4", 4, "foo foo foo"),
_video("m5", 5, "foo with a longer title"),
_video("m6", 6, "foo with yet another title"),
]
)
db.commit()
return user
def _ids(client, cursor=None, limit=50):
url = f"/api/feed?q=foo&sort=relevance&limit={limit}"
if cursor:
url += f"&cursor={cursor}"
resp = client.get(url)
assert resp.status_code == 200, resp.text
body = resp.json()
return [it["id"] for it in body["items"]], body["next_cursor"]
def test_a_specific_word_ranks_every_match(db, as_user):
client = as_user(_seed(db))
ids, _ = _ids(client)
assert ids[:2] == ["m1", "m4"], ids
def test_a_broad_word_ranks_the_newest_window_and_the_rest_follow_newest_first(
db, as_user, monkeypatch
):
monkeypatch.setattr(feedquery, "RANK_WINDOW", 3)
client = as_user(_seed(db))
ids, _ = _ids(client)
assert ids[0] == "m4", ids # the strongest of the newest three
assert sorted(ids[:3]) == ["m4", "m5", "m6"], ids
assert ids[3:] == ["m3", "m2", "m1"], ids # past the window: newest first, m1 not ranked
def test_paging_keeps_the_window_page_one_ranked_against(db, as_user, monkeypatch):
monkeypatch.setattr(feedquery, "RANK_WINDOW", 3)
client = as_user(_seed(db))
full, _ = _ids(client)
first, cursor = _ids(client, limit=2)
assert first == full[:2]
# A newer match lands mid-scroll. Ranked against a recomputed window, m4 (shown on page one) would
# fall out of it to rank 0, sort behind the cursor and come back on page two.
db.add(_video("m7", 7, "foo arriving while the reader scrolls"))
db.commit()
walked = list(first)
while cursor:
page, cursor = _ids(client, cursor, limit=2)
walked.extend(page)
assert len(walked) == len(set(walked)), walked
assert set(full) <= set(walked), walked
@pytest.mark.parametrize(
("window", "limit"),
# 3: the page is read as its two halves (the window ranked, the rest newest first), and pages of 2
# and 3 cross the window's edge mid-page and exactly at it. 2000: one ranked query.
[(3, 2), (3, 3), (2000, 2)],
)
def test_cursor_walk_matches_the_unpaged_order(db, as_user, monkeypatch, window, limit):
monkeypatch.setattr(feedquery, "RANK_WINDOW", window)
client = as_user(_seed(db))
full, _ = _ids(client)
if window == 3:
assert full == ["m4", "m6", "m5", "m3", "m2", "m1"]
walked, cursor = _ids(client, limit=limit)
while cursor:
page, cursor = _ids(client, cursor, limit=limit)
walked.extend(page)
assert walked == full
@pytest.mark.parametrize(
"payload",
[
{"k": [5, None], "id": "m1", "w": "2024-01-01T00:00:00+00:00"}, # null published_at
{"k": [5, "2024-01-03T00:00:00+00:00"], "id": "m1", "w": "2024-01-01T00:00:00"}, # naive w
{"k": [5, "2024-01-03T00:00:00"], "id": "m1", "w": "2024-01-01T00:00:00+00:00"}, # naive k
],
)
def test_a_hand_made_relevance_cursor_is_refused_not_a_crash(db, as_user, payload):
# The page halves compare the cursor's timestamp with the window's in Python: a null or a naive
# one raised there, past the route's 400 — a 500 (E31 review 2).
client = as_user(_seed(db))
cursor = base64.urlsafe_b64encode(json.dumps(payload).encode()).decode()
resp = client.get(f"/api/feed?q=foo&sort=relevance&limit=2&cursor={cursor}")
assert resp.status_code == 400, resp.text
+100
View File
@@ -0,0 +1,100 @@
"""Every feed sort on a videos column rides an index, and an ICU upgrade rebuilds the title ones (E31 S4e).
Without an index in the exact ORDER BY, each feed page read the whole videos heap and kept a top-N
(prod: 350 ms a page for views/duration, 550–780 ms for title). Migration 0096 adds one per direction
and per reader language. The planner is told it may not sort or scan the table, so a plan that still
sorts it whole means no index matches the ORDER BY `feedquery.order_by` builds: a key, direction, NULLS
rule or collation changed on one side only. An Incremental Sort is fine: the index gives the key's order
and only its ties are sorted by id (`newest`/`oldest` ride the published_at indexes that way). DB lane.
"""
import re
import pytest
from sqlalchemy import select, text
from sqlalchemy.dialects import postgresql
from app import collation
from app.db import engine
from app.feedquery import order_by, sort_keys
from app.models import Video
VIDEO_SORTS = [
"newest",
"oldest",
"views",
"views_asc",
"duration_desc",
"duration_asc",
"title",
"title_desc",
]
def _plan(db, sort: str, lang: str) -> str:
stmt = select(Video.id).order_by(*order_by(sort_keys(sort, 0, None, lang))).limit(60)
sql = str(stmt.compile(dialect=postgresql.dialect(), compile_kwargs={"literal_binds": True}))
db.execute(text("SET LOCAL enable_seqscan = off"))
db.execute(text("SET LOCAL enable_sort = off"))
db.execute(text("SET LOCAL enable_bitmapscan = off"))
return "\n".join(db.execute(text(f"EXPLAIN {sql}")).scalars())
@pytest.mark.parametrize("lang", sorted(collation.ICU))
@pytest.mark.parametrize("sort", VIDEO_SORTS)
def test_the_sort_walks_an_index(db, sort, lang):
plan = _plan(db, sort, lang)
db.rollback()
full_sort = re.search(r"(?<!Incremental )Sort \(", plan)
assert not full_sort, f"{sort} ({lang}) sorts the table:\n{plan}"
assert "Index Scan" in plan or "Index Only Scan" in plan, plan
def _drifted(db) -> list[str]:
return list(
db.execute(
text(
"SELECT collname FROM pg_collation WHERE collname = ANY(:names) "
"AND collversion IS DISTINCT FROM pg_collation_actual_version(oid)"
),
{"names": list(collation.ICU.values())},
).scalars()
)
def _files(db) -> dict[str, int]:
return dict(
db.execute(
text("SELECT relname, relfilenode FROM pg_class WHERE relname LIKE 'ix_videos_title_%'")
).all()
)
def test_an_icu_upgrade_rebuilds_the_title_indexes(db):
# What an image update with a newer ICU looks like from inside: the recorded version no longer
# matches the library's (the test database runs as the bootstrap superuser, as the app's does).
db.execute(text("UPDATE pg_collation SET collversion = '0.0' WHERE collname = 'hu-x-icu'"))
db.commit()
try:
assert _drifted(db) == ["hu-x-icu"]
before = _files(db)
# A concurrent rebuild waits for every open transaction, this session's too.
db.rollback()
rebuilt = collation.heal_drift(engine)
assert sorted(rebuilt) == ["ix_videos_title_hu", "ix_videos_title_hu_desc"]
after = _files(db)
# Rebuilt = written to new files; the English pair, on an unchanged ICU, is left alone.
assert {n for n in before if before[n] != after[n]} == set(rebuilt)
assert _drifted(db) == []
finally:
db.execute(
text(
"UPDATE pg_collation SET collversion = pg_collation_actual_version(oid) "
"WHERE collname = 'hu-x-icu'"
)
)
db.commit()
def test_no_drift_rebuilds_nothing(db):
assert collation.heal_drift(engine) == []
+53
View File
@@ -0,0 +1,53 @@
"""E31 S1: the welcome message is rendered once, when the account is created, in the language of that day.
An account that switched language afterwards kept reading it in the old one (the owner's: English under a
Hungarian UI). It is served in the reader's language now; any other system message stays as stored.
Needs the DB."""
from fastapi.testclient import TestClient
import app.state as state
from app.auth import current_user
from app.db import get_db
from app.main import app
from app.models import Message, User
from app.routes.messages import WELCOME
def _client(db, user, monkeypatch, override_dep) -> TestClient:
monkeypatch.setattr(state, "_configured_cache", True)
override_dep(get_db, db.info["request"])
override_dep(current_user, lambda: user)
return TestClient(app)
def _user_with_english_welcome(db) -> User:
"""An account whose welcome was written in English, then switched to Hungarian."""
user = User(email="welcome-hu@example.com", preferences={"language": "hu"})
db.add(user)
db.flush()
db.add(Message(kind="system", sender_id=None, recipient_id=user.id, body=WELCOME["en"]))
db.commit()
return user
def test_the_welcome_is_read_in_the_readers_language(db, monkeypatch, override_dep):
user = _user_with_english_welcome(db)
client = _client(db, user, monkeypatch, override_dep)
thread = client.get("/api/messages/conversations/0").json()["items"]
assert [m["body"] for m in thread] == [WELCOME["hu"]]
convs = client.get("/api/messages/conversations").json()["items"]
assert [c["last_message"]["body"] for c in convs if c["partner"]["id"] == 0] == [WELCOME["hu"]]
def test_another_system_message_is_served_as_stored(db, monkeypatch, override_dep):
user = _user_with_english_welcome(db)
db.add(
Message(kind="system", sender_id=None, recipient_id=user.id, body="Maintenance tonight.")
)
db.commit()
client = _client(db, user, monkeypatch, override_dep)
bodies = [m["body"] for m in client.get("/api/messages/conversations/0").json()["items"]]
assert bodies == [WELCOME["hu"], "Maintenance tonight."]
@@ -0,0 +1,53 @@
"""A concurrent index build that failed part-way, then re-run (E31 S5 review), DB-lane.
CREATE INDEX CONCURRENTLY that fails leaves an INVALID index under its name, and alembic_version is not
stamped, so the next start runs the migration again. `IF NOT EXISTS` matched the name and kept the
leftover, which the planner never uses; 0095 then dropped the old channel_id index, leaving `videos`
with no usable one. A re-run now rebuilds the leftover and drops the old index only once the new one is
valid.
"""
from alembic import command
from alembic.config import Config
from sqlalchemy import text
from app.db import engine
def _index(name: str) -> tuple[bool, str] | None:
with engine.connect() as conn:
row = conn.execute(
text(
"SELECT i.indisvalid, pg_get_indexdef(i.indexrelid) FROM pg_index i "
"JOIN pg_class c ON c.oid = i.indexrelid WHERE c.relname = :name"
),
{"name": name},
).first()
return None if row is None else (bool(row[0]), row[1])
def _leave_invalid(name: str) -> None:
"""What a build that failed part-way leaves behind: the name taken, the index INVALID."""
with engine.connect().execution_options(isolation_level="AUTOCOMMIT") as conn:
conn.execute(text(f"CREATE INDEX {name} ON videos (id)"))
conn.execute(
text(f"UPDATE pg_index SET indisvalid = false WHERE indexrelid = '{name}'::regclass")
)
def test_a_rerun_rebuilds_an_invalid_leftover_before_dropping_the_old_index(db):
db.rollback() # CONCURRENTLY waits for every open transaction, this session's too
cfg = Config("alembic.ini")
command.downgrade(cfg, "0094_plex_genres_hu_everywhere")
try:
_leave_invalid("ix_videos_channel_pub_cover")
_leave_invalid("ix_videos_views_desc")
finally:
command.upgrade(cfg, "head")
cover = _index("ix_videos_channel_pub_cover")
assert cover is not None and cover[0], cover
assert "INCLUDE (duration_seconds" in cover[1]
assert _index("ix_videos_channel_pub_desc") is None
views = _index("ix_videos_views_desc")
assert views is not None and views[0] and "view_count DESC NULLS LAST" in views[1], views
+33
View File
@@ -0,0 +1,33 @@
"""The Stats page's "Full history: done / requested" ratio counts the SAME channels on both sides — the
ones this user asked full history for. The numerator used to count every subscribed channel whose
back-catalog was complete (another user had asked, or the channel was short), so the page read
"342/329". "Channels without full history yet" still counts every subscribed channel. Needs the DB."""
from app.models import Channel, Subscription
def test_full_history_ratio_counts_only_the_requested_channels(db, make_user, as_user):
user = make_user("deep@example.com")
db.add_all(
[
Channel(id="UCreqdone", title="Requested, done", backfill_done=True),
Channel(id="UCreqopen", title="Requested, pending", backfill_done=False),
Channel(id="UCdone", title="Not requested, done anyway", backfill_done=True),
Channel(id="UCopen", title="Not requested, pending", backfill_done=False),
]
)
db.flush()
db.add_all(
[
Subscription(user_id=user.id, channel_id="UCreqdone", deep_requested=True),
Subscription(user_id=user.id, channel_id="UCreqopen", deep_requested=True),
Subscription(user_id=user.id, channel_id="UCdone"),
Subscription(user_id=user.id, channel_id="UCopen"),
]
)
db.commit()
body = as_user(user).get("/api/sync/my-status").json()
assert body["channels_deep_requested"] == 2
assert body["channels_deep_done"] == 1 # never more than it is out of
assert body["channels_deep_pending"] == 2 # subscribed channels without full history yet
+4 -4
View File
@@ -11,21 +11,21 @@ from app.plex.cards import movie_card, show_card
def test_movie_card_emits_art_when_art_key_present():
it = PlexItem(rating_key="m1", kind="movie", title="M", year=2020, art_key="/library/art/1")
card = movie_card(it, None)
card = movie_card(it, None, lang="en")
assert card["thumb"] == "/api/plex/image/m1"
assert card["art"] == "/api/plex/image/m1?variant=art"
def test_movie_card_art_is_none_without_art_key():
it = PlexItem(rating_key="m2", kind="movie", title="M2")
assert movie_card(it, None)["art"] is None
assert movie_card(it, None, lang="en")["art"] is None
def test_show_card_emits_art_when_art_key_present():
sh = PlexShow(rating_key="s1", title="S", art_key="/library/art/2")
assert show_card(sh)["art"] == "/api/plex/image/s1?variant=art"
assert show_card(sh, lang="en")["art"] == "/api/plex/image/s1?variant=art"
def test_show_card_art_is_none_without_art_key():
sh = PlexShow(rating_key="s2", title="S2")
assert show_card(sh)["art"] is None
assert show_card(sh, lang="en")["art"] is None
+49
View File
@@ -352,3 +352,52 @@ def test_show_detail_lazy_fills_when_unenriched(db, monkeypatch, override_dep):
assert show.imdb_id == "tt0068646" # persisted for next time
assert show.people_enriched_at is not None
assert [c["name"] for c in show.full_cast] == ["Marlon Brando", "Al Pacino"]
def test_item_detail_names_tracks_from_their_language_not_plexs_title(
db, monkeypatch, override_dep
):
"""E30: Plex titles a track in the Plex SERVER's language ("English (AC3 5.1)", "magyar Forced",
"中文 (EAC3 Stereo)"), so a Hungarian reader got a mix. The route sends what the client names it
from instead — the language tag, the forced / SDH flags, the codec tail — keeping Plex's title
only as the fallback, and no invented English "Audio 2" for a track with neither."""
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
db.add(PlexItem(rating_key="m1", library_id=lib.id, kind="movie", title="T"))
db.commit()
user = _user(db, "tracks@example.com")
db.commit()
streams = [
{"streamType": 2, "displayTitle": "English (AC3 5.1)", "languageTag": "en-US"},
{"streamType": 2, "displayTitle": "中文 (EAC3 Stereo)", "languageTag": "zh"},
{"streamType": 2},
{
"streamType": 3,
"displayTitle": "magyar Forced",
"languageTag": "hu",
"forced": True,
"codec": "srt",
},
{
"streamType": 3,
"displayTitle": "English SDH",
"languageTag": "en",
"hearingImpaired": True,
"codec": "srt",
},
]
meta = {**RICH_META, "Media": [{"Part": [{"Stream": streams}]}]}
monkeypatch.setattr(detail_routes, "PlexClient", lambda db: _StubClient(meta))
body = _get(db, user, monkeypatch, override_dep, "/api/plex/item/m1").json()
naming = ("label", "lang_tag", "forced", "hearing_impaired", "detail")
assert [tuple(a[k] for k in naming) for a in body["audio_streams"]] == [
("English (AC3 5.1)", "en-US", False, False, "AC3 5.1"),
("中文 (EAC3 Stereo)", "zh", False, False, "EAC3 Stereo"),
(None, None, False, False, None),
]
assert [tuple(s[k] for k in naming) for s in body["subtitle_streams"]] == [
("magyar Forced", "hu", True, False, None),
("English SDH", "en", False, True, None),
]
@@ -214,3 +214,28 @@ def test_hidden_movie_absent_from_grid_and_facets(db, as_user):
facets = client.get("/api/plex/facets?scope=movie&show=all")
crs = {c["value"]: c["count"] for c in facets.json()["content_ratings"]}
assert crs.get("R") == 1, crs
def test_a_watch_state_scope_rolls_the_episodes_up_once(db, as_user):
"""The per-user episode rollup behind a show's watch state is a pass over every episode; each show
facet group asked it again (E31 S4e: half of a filtered facets call). It is asked once now."""
from sqlalchemy import event
from app.db import engine
seed = _seed(db)
client = as_user(seed["user"])
rollups: list[str] = []
def count(conn, cursor, statement, *_):
if "GROUP BY plex_items.show_id" in statement:
rollups.append(statement)
event.listen(engine, "before_cursor_execute", count)
try:
for show in ("watched", "in_progress", "unwatched"):
rollups.clear()
assert client.get(f"/api/plex/facets?scope=both&show={show}").status_code == 200
assert len(rollups) == 1, show
finally:
event.remove(engine, "before_cursor_execute", count)
+77
View File
@@ -0,0 +1,77 @@
"""A card asks the Plex image proxy for a small cut; a detail page keeps the full one (E31 S4e).
Cards fetched the 400×600 poster for a ~144px picture and the 1280×720 backdrop for a ~212px shelf
picture. The `_sm` variants are cut by Plex at a card's size and cached apart from the full ones; an
unknown variant still gets the full poster. DB lane (the proxy opens its own short session).
"""
from fastapi.testclient import TestClient
import app.state as app_state
from app.main import app
from app.models import PlexItem, PlexLibrary
from app.plex import imgcache
from app.routes.plex import playback
class _FakePlex:
calls: list[tuple[str, int | None, int | None]] = []
def __init__(self, db) -> None:
pass
def __enter__(self):
return self
def __exit__(self, *exc) -> None:
return None
def image_bytes(self, key, width=None, height=None):
_FakePlex.calls.append((key, width, height))
return b"\xff\xd8jpeg", "image/jpeg"
def _setup(db, make_user, monkeypatch, tmp_path) -> TestClient:
user = make_user()
lib = PlexLibrary(plex_key="90", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
db.add(
PlexItem(
rating_key="pm1",
library_id=lib.id,
kind="movie",
title="A Movie",
thumb_key="/library/metadata/pm1/thumb/1",
art_key="/library/metadata/pm1/art/1",
)
)
db.commit()
monkeypatch.setattr(app_state, "_configured_cache", True)
monkeypatch.setattr(playback, "resolved_user_id", lambda request: (user.id, True))
monkeypatch.setattr(playback, "PlexClient", _FakePlex)
monkeypatch.setattr(imgcache, "CACHE_DIR", tmp_path)
_FakePlex.calls = []
return TestClient(app)
def test_each_variant_is_cut_at_its_own_size(db, make_user, monkeypatch, tmp_path):
client = _setup(db, make_user, monkeypatch, tmp_path)
for variant in ("thumb", "thumb_sm", "art", "art_sm"):
assert client.get(f"/api/plex/image/pm1?variant={variant}").status_code == 200
assert _FakePlex.calls == [
("/library/metadata/pm1/thumb/1", 400, 600),
("/library/metadata/pm1/thumb/1", 240, 360),
("/library/metadata/pm1/art/1", 1280, 720),
("/library/metadata/pm1/art/1", 480, 270),
]
# Each cut is cached under its own name: asking again reaches Plex no more.
for variant in ("thumb", "thumb_sm", "art", "art_sm"):
client.get(f"/api/plex/image/pm1?variant={variant}")
assert len(_FakePlex.calls) == 4
def test_an_unknown_variant_gets_the_full_poster(db, make_user, monkeypatch, tmp_path):
client = _setup(db, make_user, monkeypatch, tmp_path)
assert client.get("/api/plex/image/pm1?variant=huge").status_code == 200
assert _FakePlex.calls == [("/library/metadata/pm1/thumb/1", 400, 600)]
+182 -3
View File
@@ -12,7 +12,19 @@ import app.state as state
from app.auth import current_user
from app.db import get_db
from app.main import app
from app.models import PlexItem, PlexLibrary, User
from app import sysconfig
from app.plex.cards import display_summary, display_title
from app.models import (
PlexCollection,
PlexItem,
PlexLibrary,
PlexPlaylist,
PlexPlaylistItem,
PlexSeason,
PlexShow,
PlexState,
User,
)
def _seed(db):
@@ -30,7 +42,9 @@ def _seed(db):
summary="English summary.",
summary_hu="Magyar összefoglaló.",
genres=["Drama"],
genres_hu=["Dráma"],
genres_hu=["Dráma", "Akció"],
tagline="English tagline.",
tagline_hu="Magyar mottó.",
rating=6.5,
imdb_rating=None,
year=2001,
@@ -44,7 +58,9 @@ def _seed(db):
library_id=lib.id,
kind="movie",
title="EN Only",
genres=["Comedy"], # no HU twin
summary="English only.",
tagline="English only tagline.",
genres=["Comedy"], # no HU twin, never reached by a sync since: no genres_hu
year=2002,
added_at=datetime(2024, 1, 2, tzinfo=timezone.utc),
people_enriched_at=datetime(2024, 1, 2, tzinfo=timezone.utc),
@@ -102,12 +118,52 @@ def test_genre_filter_matches_active_language_vocabulary(db, monkeypatch, overri
assert miss.json()["items"] == []
def test_a_genre_filter_saved_in_english_still_matches_in_hungarian(db, monkeypatch, override_dep):
"""E30: the HU column holds one Hungarian word per genre. A filter saved while the HU list still
offered "Action" is read through the same map, so it finds the "Akció" title instead of nothing."""
user = _seed(db)
resp = _get(
db, user, monkeypatch, override_dep, "/api/plex/library?scope=movie&lang=hu&genres=Action"
)
assert _titles(resp) == {"Magyar Cím"}
def test_a_genre_the_hungarian_list_offers_in_english_still_matches(db, monkeypatch, override_dep):
"""E30 review: a row the sync has not given Hungarian genres yet is listed under its English word
(the facets read coalesce(genres_hu, genres)); filtering by that word must find it."""
user = _seed(db)
facets = _get(db, user, monkeypatch, override_dep, "/api/plex/facets?scope=movie&lang=hu")
assert "Comedy" in {g["value"] for g in facets.json()["genres"]}
resp = _get(
db, user, monkeypatch, override_dep, "/api/plex/library?scope=movie&lang=hu&genres=Comedy"
)
assert _titles(resp) == {"EN Only"}
def test_a_title_without_a_hungarian_twin_reads_in_english(db, monkeypatch, override_dep):
user = _seed(db)
body = _get(db, user, monkeypatch, override_dep, "/api/plex/item/m2?lang=hu").json()
assert (body["title"], body["summary"], body["tagline"]) == (
"EN Only",
"English only.",
"English only tagline.",
)
row = PlexItem(title="T", summary="S")
assert (display_title(row, "hu"), display_summary(row, "hu")) == ("T", "S")
row.title_hu, row.summary_hu = "HT", "HS"
assert (display_title(row, "hu"), display_summary(row, "hu")) == ("HT", "HS")
assert (display_title(row, "en"), display_summary(row, "en")) == ("T", "S")
def test_item_detail_language_and_rating_split(db, monkeypatch, override_dep):
user = _seed(db)
resp = _get(db, user, monkeypatch, override_dep, "/api/plex/item/m1?lang=hu")
body = resp.json()
assert body["title"] == "Magyar Cím"
assert body["summary"] == "Magyar összefoglaló."
assert body["tagline"] == "Magyar mottó." # E30: the HU overlay's tagline
en = _get(db, user, monkeypatch, override_dep, "/api/plex/item/m1").json()
assert en["tagline"] == "English tagline."
# imdb_rating is IMDb-only (None here); the audience score is served separately as `rating`.
assert body["imdb_rating"] is None
assert body["rating"] == 6.5
@@ -147,3 +203,126 @@ def test_rating_filter_prefers_imdb_over_audience(db, monkeypatch, override_dep)
assert "High IMDb" in _titles(hit)
miss = _get(db, user, monkeypatch, override_dep, "/api/plex/library?scope=movie&rating_min=9")
assert "High IMDb" not in _titles(miss)
def _seed_rails(db):
"""`_seed` plus what every card rail reads: a HU-overlaid show with a HU-overlaid episode in
progress, a cast name, a collection, and a playlist holding the movie and the episode."""
user = _seed(db)
m1 = db.query(PlexItem).filter_by(rating_key="m1").one()
m1.cast_names = ["Anna"]
m1.collection_keys = ["c1"]
db.add(PlexCollection(rating_key="c1", library_id=m1.library_id, title="Coll"))
tv = PlexLibrary(plex_key="2", title="TV Shows", kind="show", enabled=True)
db.add(tv)
db.flush()
show = PlexShow(
rating_key="s1",
library_id=tv.id,
title="English Show",
title_hu="Magyar Sorozat",
added_at=datetime(2024, 1, 3, tzinfo=timezone.utc),
)
db.add(show)
db.flush()
season = PlexSeason(rating_key="s1s1", show_id=show.id, title="Season 1", season_number=1)
db.add(season)
db.flush()
ep = PlexItem(
rating_key="e1",
library_id=tv.id,
kind="episode",
show_id=show.id,
season_id=season.id,
title="Pilot",
title_hu="Kezdet",
season_number=1,
episode_number=1,
duration_s=3000,
)
db.add(ep)
db.flush()
db.add(
PlexState(
user_id=user.id,
item_id=ep.id,
status="in_progress",
position_seconds=600,
progress_updated_at=datetime(2024, 2, 1, tzinfo=timezone.utc),
)
)
pl = PlexPlaylist(user_id=user.id, title="Mix")
db.add(pl)
db.flush()
db.add_all(
[
PlexPlaylistItem(playlist_id=pl.id, item_id=m1.id, position=0),
PlexPlaylistItem(playlist_id=pl.id, item_id=ep.id, position=1),
]
)
# TMDB extras: m2 recommends m1 (held) and a title we lack; m1, watched, anchors the recs rail.
m1.guid = "tmdb:1"
m1.tmdb_enriched_at = datetime(2024, 1, 1, tzinfo=timezone.utc)
m1.recommendations = [{"tmdb_id": 2, "title": "EN Only", "media_type": "movie"}]
m2 = db.query(PlexItem).filter_by(rating_key="m2").one()
m2.guid = "tmdb:2"
m2.tmdb_enriched_at = datetime(2024, 1, 1, tzinfo=timezone.utc)
m2.recommendations = [
{"tmdb_id": 1, "title": "English Title", "media_type": "movie"},
{"tmdb_id": 77, "title": "Elsewhere", "media_type": "movie"},
]
db.add(
PlexState(
user_id=user.id,
item_id=m1.id,
status="watched",
watched_at=datetime(2024, 3, 1, tzinfo=timezone.utc),
)
)
sysconfig.set_value(db, "plex_enabled", True)
db.commit()
return user, pl.id
def test_every_card_rail_serves_the_readers_language(db, monkeypatch, override_dep):
"""E30: the grid spoke Hungarian, but every other card surface (Discover rails, show page,
search, collection, person page, playlist, Home) served the English title to a HU reader."""
user, pid = _seed_rails(db)
def get(path):
return _get(db, user, monkeypatch, override_dep, path).json()
def titles(cards):
return {c["title"] for c in cards}
rails = get("/api/plex/discover?scope=both&lang=hu")
assert {"Magyar Cím", "Magyar Sorozat"} <= titles(rails["recently_added"])
assert "Magyar Sorozat" in titles(rails["continue_watching"])
assert titles(get("/api/plex/show/s1?lang=hu")["seasons"][0]["episodes"]) == {"Kezdet"}
found = get("/api/plex/library?scope=both&lang=hu&q=Kezdet")["episodes"]
assert [(e["title"], e["show_title"]) for e in found] == [("Kezdet", "Magyar Sorozat")]
assert titles(get("/api/plex/collections/c1?lang=hu")["items"]) == {"Magyar Cím"}
assert titles(get("/api/plex/person/Anna?lang=hu")["movies"]) == {"Magyar Cím"}
items = get(f"/api/plex/playlists/{pid}?lang=hu")["items"]
assert [(c["title"], c.get("show_title")) for c in items] == [
("Magyar Cím", None),
("Kezdet", "Magyar Sorozat"),
]
# A recommendation we hold is named as the library names it; the rail's anchor too.
recs = get("/api/plex/item/m2?lang=hu")["tmdb"]["recommendations"]
assert [(r["title"], r["in_library"]) for r in recs] == [
("Magyar Cím", True),
("Elsewhere", False),
]
byw = get("/api/plex/discover?scope=both&lang=hu")["because_you_watched"]
assert byw["anchor"]["title"] == "Magyar Cím"
home = get("/api/discover?lang=hu")["continue_watching"]
plex = [c for c in home if c["source"] == "plex"]
assert [c["title"] for c in plex] == ["Magyar Sorozat"]
assert plex[0]["next"]["title"] == "Kezdet"
# EN (the default) still serves English on the same rails.
items = get(f"/api/plex/playlists/{pid}")["items"]
assert [(c["title"], c.get("show_title")) for c in items] == [
("English Title", None),
("Pilot", "English Show"),
]
+174 -1
View File
@@ -6,11 +6,16 @@ Uses the DB lane (`db` fixture) — skipped when no Postgres is reachable. Runs
(migrations 0063/0064) so the JSONB + STORED generated columns are exercised.
"""
from alembic import command
from alembic.config import Config
from sqlalchemy import func, select
from sqlalchemy.orm import Session
import app.plex.sync as syncmod
from app import sysconfig
from app.models import PlexItem, PlexLibrary
from app.db import engine
from app.models import PlexItem, PlexLibrary, PlexShow
from app.plex.genres import hu_genre, hu_genres
from app.plex.sync import _apply_item, _canonical_guid, _sync_overlay
from app.plex.query import content_libs
@@ -244,3 +249,171 @@ def test_content_libs_excludes_overlay_libraries(db):
assert en_lib.id in movie_ids
assert overlay.id not in movie_ids # the HU overlay is never browsable
assert show_ids == [en_shows.id]
class _KindStub:
"""An overlay section listing per Plex type (2 = show, 4 = episode), like a HU TV section."""
def __init__(self, by_type: dict[int, list[dict]]):
self._by = by_type
def section_items(self, key, item_type, start, size):
items = self._by.get(item_type, [])
return items[start : start + size], len(items)
def test_a_show_overlay_writes_hungarian_episode_titles_and_summaries(db):
"""E30: the HU TV section lists every episode in Hungarian; they were never read, so a Hungarian
reader saw every episode title and summary in English."""
en_lib = PlexLibrary(plex_key="1", title="TV Shows", kind="show")
hu_lib = PlexLibrary(plex_key="9", title="Sorozatok", kind="show", metadata_lang="hu")
db.add_all([en_lib, hu_lib])
db.flush()
show = PlexShow(rating_key="s1", library_id=en_lib.id, title="3 Body Problem", guid="tmdb:10")
db.add(show)
db.flush()
ep = PlexItem(
rating_key="e1",
library_id=en_lib.id,
kind="episode",
show_id=show.id,
title="Countdown",
summary="EN.",
guid="tmdb:501",
)
db.add(ep)
db.commit()
stub = _KindStub(
{
syncmod._TYPE_SHOW: [_hu_movie("hs", "A 3-test-probléma", "HU.", [], "tmdb://10")],
syncmod._TYPE_EPISODE: [
_hu_movie("he", "Visszaszámlálás", "Magyar epizód.", [], "tmdb://501")
],
}
)
_sync_overlay(db, stub, hu_lib, en_lib.id, {})
db.commit()
db.refresh(ep)
db.refresh(show)
assert (ep.title_hu, ep.summary_hu) == ("Visszaszámlálás", "Magyar epizód.")
assert show.title_hu == "A 3-test-probléma"
def test_the_overlay_gives_every_row_one_hungarian_word_per_genre(db):
"""E30: Plex's HU agent mixes "Action" with "Dráma", and a title with no HU twin kept only its
English list — so the HU genre list offered "Drama" beside "Dráma"."""
en_lib = PlexLibrary(plex_key="4", title="Movies", kind="movie")
hu_lib = PlexLibrary(plex_key="9", title="Filmek", kind="movie", metadata_lang="hu")
db.add_all([en_lib, hu_lib])
db.flush()
twin = PlexItem(
rating_key="m1",
library_id=en_lib.id,
kind="movie",
title="Heat",
guid="tmdb:1",
genres=["Action", "Crime"],
)
lone = PlexItem(
rating_key="m2",
library_id=en_lib.id,
kind="movie",
title="Solo",
guid="tmdb:2",
genres=["Drama", "Science Fiction"],
)
db.add_all([twin, lone])
db.commit()
meta = _hu_movie(
"h1", "Szemtől szemben", "HU.", ["Action", "Bűnügyi", "Drama", "Dráma"], "tmdb://1"
)
meta["tagline"] = "Magyar mottó."
_sync_overlay(db, _StubPlex([meta]), hu_lib, en_lib.id, {})
db.commit()
db.refresh(twin)
db.refresh(lone)
assert twin.genres_hu == ["Akció", "Bűnügyi", "Dráma"]
assert twin.tagline_hu == "Magyar mottó."
assert lone.genres_hu == ["Dráma", "Sci-Fi"] # no twin: its own English list, in Hungarian
assert lone.tagline_hu is None
def test_a_twin_without_genres_keeps_the_rows_own_hungarian_list(db):
en_lib = PlexLibrary(plex_key="4", title="Movies", kind="movie")
hu_lib = PlexLibrary(plex_key="9", title="Filmek", kind="movie", metadata_lang="hu")
db.add_all([en_lib, hu_lib])
db.flush()
row = PlexItem(
rating_key="m1",
library_id=en_lib.id,
kind="movie",
title="Heat",
guid="tmdb:1",
genres=["Crime"],
)
db.add(row)
db.commit()
_sync_overlay(
db,
_StubPlex([_hu_movie("h1", "Szemtől szemben", "HU.", [], "tmdb://1")]),
hu_lib,
en_lib.id,
{},
)
db.commit()
db.refresh(row)
assert (row.title_hu, row.genres_hu) == ("Szemtől szemben", ["Bűnügyi"])
def test_every_synced_row_gets_hungarian_genres():
"""E30 review: not only an overlaid row — the HU facets and filter read genres_hu for all."""
from app.plex.sync import _apply_facet_fields
row = PlexItem(title="Solo")
_apply_facet_fields(row, {"Genre": [{"tag": "Drama"}, {"tag": "Action"}]})
assert (row.genres, row.genres_hu) == (["Drama", "Action"], ["Dráma", "Akció"])
def test_the_genre_map_is_idempotent_and_keeps_an_unknown_word():
assert hu_genre("science fiction") == "Sci-Fi" # case-insensitive
assert hu_genre("Dráma") == "Dráma"
assert hu_genre("Space Opera") == "Space Opera" # unknown: shown as Plex gave it, never dropped
once = hu_genres(["Drama", "Action", "Dráma"])
assert once == ["Dráma", "Akció"]
assert hu_genres(once) == once
assert hu_genres(None) is None and hu_genres([]) is None
def test_0094_gives_every_row_without_a_hungarian_list_one(db):
cfg = Config("alembic.ini")
command.downgrade(cfg, "0093_plex_person_known_for_votes")
upgraded = False
try:
with Session(engine) as s:
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
s.add(lib)
s.flush()
rows = {
"bare": PlexItem(genres=["Action", "Biography"]),
"overlaid": PlexItem(
genres=["Action"], genres_hu=["Kaland"]
), # its twin's own list
"genreless": PlexItem(genres=[]),
}
for key, row in rows.items():
row.rating_key, row.library_id, row.kind, row.title = key, lib.id, "movie", key
show = PlexShow(rating_key="s", library_id=lib.id, title="s", genres=["Comedy"])
s.add_all([*rows.values(), show])
s.commit()
ids = {k: r.id for k, r in rows.items()}
show_id = show.id
command.upgrade(cfg, "head")
upgraded = True
with Session(engine) as s:
got = {k: s.get(PlexItem, i).genres_hu for k, i in ids.items()} # type: ignore[union-attr]
show_hu = s.get(PlexShow, show_id).genres_hu # type: ignore[union-attr]
assert got == {"bare": ["Akció", "Életrajzi"], "overlaid": ["Kaland"], "genreless": None}
assert show_hu == ["Vígjáték"]
finally:
if not upgraded:
command.upgrade(cfg, "head")
+463
View File
@@ -0,0 +1,463 @@
"""E30 S6: the people are made local ahead of time. The people_enrich job fetches linked people's TMDB
bios (the most-credited first) and caches their portraits resized through Plex; the person page says
when a bio is still on its way (`enriching`), so it can ask again instead of staying empty.
DB lane (Postgres); the portrait cache runs in a temp directory.
"""
from datetime import datetime, timezone
import httpx
import pytest
from app import scheduler
from app.models import PlexItem, PlexLibrary, PlexPerson, User
from app.plex import imgcache
from app.plex import people as plex_people
from app.plex import tmdb as plex_tmdb
CDN = plex_people.PERSON_IMG_HOST + "people/"
DONE = datetime(2024, 1, 1, tzinfo=timezone.utc)
def _movie(db, lib, key, cast):
db.add(PlexItem(rating_key=key, library_id=lib.id, kind="movie", title=key, cast_names=cast))
def _library(db):
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
return lib
class _Tmdb:
"""A TmdbClient stand-in: records the ids asked for, fails on `fail_on`."""
def __init__(self, fail_on=()):
self.asked: list[int] = []
self.fail_on = set(fail_on)
def fetch_person(self, pid):
self.asked.append(pid)
if pid in self.fail_on:
raise plex_tmdb.TmdbError("429")
return {"biography": f"Bio {pid}."}
def test_the_batch_takes_the_most_credited_linked_people_first(db):
lib = _library(db)
_movie(db, lib, "a", ["Often", "Sometimes"])
_movie(db, lib, "b", ["Often", "Sometimes"])
_movie(db, lib, "c", ["Often", "Unlinked"])
db.add_all(
[
PlexPerson(name="Sometimes", tmdb_person_id=2),
PlexPerson(name="Often", tmdb_person_id=1),
PlexPerson(name="Unlinked"), # no TMDB id: nothing to fetch
PlexPerson(name="Done", tmdb_person_id=3, tmdb_enriched_at=DONE),
PlexPerson(name="Rare", tmdb_person_id=4), # credited nowhere: last
]
)
db.commit()
tmdb = _Tmdb()
assert plex_tmdb.enrich_people_batch(db, tmdb, 2) == {"enriched": 2} # type: ignore[arg-type]
assert tmdb.asked == [1, 2]
db.expire_all()
often = db.query(PlexPerson).filter_by(name="Often").one()
assert often.biography == "Bio 1." and often.tmdb_enriched_at is not None
assert db.query(PlexPerson).filter_by(name="Rare").one().tmdb_enriched_at is None # next run
def test_a_failed_lookup_is_skipped_and_the_rest_still_land(db):
db.add_all([PlexPerson(name="A", tmdb_person_id=1), PlexPerson(name="B", tmdb_person_id=2)])
db.add(PlexPerson(name="C", tmdb_person_id=3))
db.commit()
tmdb = _Tmdb(fail_on={2})
assert plex_tmdb.enrich_people_batch(db, tmdb, 10) == {"enriched": 2, "failed": 1} # type: ignore[arg-type]
assert tmdb.asked == [1, 2, 3] # B refused: skipped, not the end of the batch
db.expire_all()
stamped = {p.name: p.tmdb_enriched_at is not None for p in db.query(PlexPerson)}
assert stamped == {"A": True, "B": False, "C": True} # B is tried again next run
def test_failures_in_a_row_end_the_batch(db):
db.add_all([PlexPerson(name=f"P{i}", tmdb_person_id=i) for i in range(1, 7)])
db.commit()
tmdb = _Tmdb(fail_on={1, 2, 3, 4, 5, 6}) # TMDB is down
assert plex_tmdb.enrich_people_batch(db, tmdb, 10) == {"enriched": 0, "failed": 3} # type: ignore[arg-type]
assert tmdb.asked == [1, 2, 3]
def test_failures_between_successes_do_not_end_the_batch(db):
db.add_all([PlexPerson(name=f"P{i}", tmdb_person_id=i) for i in range(1, 9)])
db.commit()
tmdb = _Tmdb(fail_on={1, 3, 5, 7}) # fail, ok, fail, ok, …: never three in a row
assert plex_tmdb.enrich_people_batch(db, tmdb, 10) == {"enriched": 4, "failed": 4} # type: ignore[arg-type]
assert tmdb.asked == list(range(1, 9))
def test_the_batch_stores_the_hungarian_biography(db):
db.add(PlexPerson(name="A", tmdb_person_id=1))
db.commit()
class _WithHu(_Tmdb):
def fetch_person(self, pid):
return {
"biography": "English.",
"translations": {
"translations": [{"iso_639_1": "hu", "data": {"biography": "Magyar."}}]
},
}
assert plex_tmdb.enrich_people_batch(db, _WithHu(), 10) == {"enriched": 1} # type: ignore[arg-type]
db.expire_all()
person = db.query(PlexPerson).one()
assert (person.biography, person.biography_hu) == ("English.", "Magyar.")
class _Plex:
"""A PlexClient stand-in. `down`: urls Plex does not answer for (True: none); `dead`: urls Plex
answers `gone` for (a dead CDN file: "not found"); `status`: an error status Plex answers every
other url with."""
def __init__(self, down=False, dead=(), status=None, gone=404):
self.asked: list[tuple[str, int | None]] = []
self.down = down
self.dead = set(dead)
self.status = status
self.gone = gone
def __enter__(self):
return self
def __exit__(self, *a):
return False
def image_bytes(self, url, width=None, height=None):
self.asked.append((url, width))
if self.down is True or (self.down and url in self.down):
raise plex_people.PlexError("down") from httpx.ConnectError("refused")
code = self.gone if url in self.dead else self.status
if code:
req = httpx.Request("GET", url)
status = httpx.HTTPStatusError(
str(code), request=req, response=httpx.Response(code, request=req)
)
raise plex_people.PlexError(str(code)) from status
return b"small", "image/jpeg"
@pytest.fixture
def cache_dir(tmp_path, monkeypatch):
monkeypatch.setattr(imgcache, "CACHE_DIR", tmp_path)
return tmp_path
@pytest.fixture
def plex_on(monkeypatch):
"""The Plex module switched on (the job's gate), with `plex` as its client."""
state = {"enabled": True, "plex": _Plex()}
monkeypatch.setattr(plex_people.sysconfig, "get_bool", lambda d, k: state["enabled"])
monkeypatch.setattr(plex_people, "PlexClient", lambda d: state["plex"])
return state
def _faces(db, n):
db.add_all(
[
PlexPerson(name=f"P{i}", photo_thumb=plex_people.person_thumb_url(CDN + f"{i}.jpg"))
for i in range(n)
]
)
db.commit()
def test_portraits_are_cached_small_and_replace_the_full_size_original(db, cache_dir, plex_on):
lib = _library(db)
_movie(db, lib, "a", ["Often"])
_movie(db, lib, "b", ["Often", "Cached"])
db.add_all(
[
PlexPerson(name="Often", photo_thumb=plex_people.person_thumb_url(CDN + "often.jpg")),
PlexPerson(name="Cached", photo_thumb=plex_people.person_thumb_url(CDN + "cached.jpg")),
PlexPerson(name="Faceless"),
]
)
db.commit()
small, original = plex_people._portrait_keys(CDN + "often.jpg")
imgcache.write(original, b"a big original", "image/jpeg") # cached without Plex
imgcache.write(plex_people._portrait_keys(CDN + "cached.jpg")[0], b"x", "image/jpeg")
plex = plex_on["plex"]
assert plex_people.warm_portraits(db, 10) == {"cached": 1}
assert plex.asked == [
(CDN + "often.jpg", plex_people.PORTRAIT_BOX)
] # resized, the cached one skipped
assert imgcache.read(small) == (b"small", "image/jpeg")
assert not imgcache.has(original)
def test_portraits_wait_for_plex(db, cache_dir, plex_on, monkeypatch):
_faces(db, 1)
def not_configured(d):
raise plex_people.PlexNotConfigured()
monkeypatch.setattr(plex_people, "PlexClient", not_configured)
assert plex_people.warm_portraits(db, 10) == {"cached": 0, "skipped": "disabled"}
assert list(cache_dir.iterdir()) == [] # no full-size originals behind Plex's back
def test_portraits_wait_for_the_plex_module_to_be_on(db, cache_dir, plex_on):
_faces(db, 1)
plex_on["enabled"] = False # switched off, its server still set up
assert plex_people.warm_portraits(db, 10) == {"cached": 0, "skipped": "disabled"}
assert plex_on["plex"].asked == []
def test_plex_not_answering_ends_the_portrait_run(db, cache_dir, plex_on):
_faces(db, 8)
plex_on["plex"] = _Plex(down=True)
assert plex_people.warm_portraits(db, 100) == {"cached": 0, "failed": 3}
assert len(plex_on["plex"].asked) == 3
assert not list(cache_dir.glob("*.miss")) # Plex was down, the files are not to blame
@pytest.mark.parametrize("code", [401, 403, 429, 500, 503])
def test_plex_refusing_is_plex_failing_not_a_dead_file(db, cache_dir, plex_on, code):
_faces(db, 8)
plex_on["plex"] = _Plex(status=code) # a rotated token, a busy or broken transcoder
assert plex_people.warm_portraits(db, 100) == {"cached": 0, "failed": 3}
assert len(plex_on["plex"].asked) == 3
assert not list(cache_dir.glob("*.miss")) # the files are not to blame: asked again next run
def test_plex_failing_between_successes_does_not_end_the_run(db, cache_dir, plex_on):
_faces(db, 8)
plex_on["plex"] = _Plex(down={CDN + f"{i}.jpg" for i in (0, 2, 4, 6)}) # down, ok, down, ok, …
assert plex_people.warm_portraits(db, 100) == {"cached": 4, "failed": 4}
def test_plex_failing_between_dead_files_does_not_end_the_run(db, cache_dir, plex_on):
_faces(db, 8)
plex_on["plex"] = _Plex( # down, dead, down, dead, …: Plex answers in between
down={CDN + f"{i}.jpg" for i in (0, 2, 4, 6)}, dead={CDN + f"{i}.jpg" for i in (1, 3, 5, 7)}
)
assert plex_people.warm_portraits(db, 100) == {"cached": 0, "failed": 8}
assert len(plex_on["plex"].asked) == 8
@pytest.mark.parametrize("gone", [404, 410])
def test_dead_headshots_are_left_alone_and_the_rest_still_land(db, cache_dir, plex_on, gone):
_faces(db, 8)
dead = {CDN + f"{i}.jpg" for i in range(6)} # more dead files than any failure limit
plex_on["plex"] = _Plex(dead=dead, gone=gone)
assert plex_people.warm_portraits(db, 100) == {"cached": 2, "failed": 6}
plex_on["plex"] = again = _Plex(dead=dead, gone=gone)
assert plex_people.warm_portraits(db, 100) == {"cached": 0}
assert again.asked == [] # dead ones marked, live ones cached: nothing to ask for
def test_the_person_image_route(db, cache_dir, monkeypatch):
from fastapi.testclient import TestClient
import app.state as state
from app.main import app
from app.routes.plex import playback
monkeypatch.setattr(state, "_configured_cache", True)
monkeypatch.setattr(playback, "_require_session", lambda request: 1)
client = TestClient(app)
url = CDN + "face.jpg"
small, original = plex_people._portrait_keys(url)
def get(u):
return client.get("/api/plex/person-image", params={"u": u})
# Only a headshot-shaped url on the CDN host: no query variants of one file, no spellings an HTTP
# client folds onto it, no other host.
for bad in (
url + "?n=1",
url + "#x",
CDN + "x/../face.jpg",
CDN + "./face.jpg",
plex_people.PERSON_IMG_HOST + "people//face.jpg",
CDN + ".face.jpg",
"https://example.com/people/a.jpg",
):
assert get(bad).status_code == 400, bad
assert plex_people.person_thumb_url(bad) is None, bad # never stored either
# No Plex: the original from the CDN, kept under its own key (the job resizes it later).
def not_configured(d):
raise plex_people.PlexNotConfigured()
class _Cdn:
def __init__(self, *a, **k):
pass
def __enter__(self):
return self
def __exit__(self, *a):
return False
def get(self, u):
return httpx.Response(
200,
content=b"original",
headers={"content-type": "image/jpeg"},
request=httpx.Request("GET", u),
)
monkeypatch.setattr(playback, "PlexClient", not_configured)
monkeypatch.setattr(plex_people.httpx, "Client", _Cdn)
resp = get(url)
assert (resp.status_code, resp.content) == (200, b"original")
assert imgcache.has(original) and not imgcache.has(small)
monkeypatch.setattr(plex_people.httpx, "Client", None) # the CDN is not asked again
assert get(url).content == b"original" # the kept original answers
# With Plex: resized, and it replaces the original.
monkeypatch.setattr(playback, "PlexClient", lambda d: _Plex())
imgcache.drop(original)
resp = get(url)
assert (resp.status_code, resp.content) == (200, b"small")
assert imgcache.has(small)
assert get(url).content == b"small" # the cache answers now
monkeypatch.setattr(playback, "PlexClient", lambda d: _Plex(down=True))
assert get(CDN + "other.jpg").status_code == 502
def test_the_people_job_runs_its_halves_without_a_tmdb_key(db, cache_dir, plex_on, monkeypatch):
_faces(db, 1)
monkeypatch.setattr(plex_tmdb.sysconfig, "get_str", lambda d, k: "")
assert plex_tmdb.run_people_enrich(db) == {
"bios": {"skipped": "tmdb_api_key not set"},
"portraits": {"cached": 1},
}
def test_a_phase_count_is_a_change_worth_an_audit_row():
assert scheduler._run_changed({"bios": {"enriched": 3}, "portraits": {"cached": 0}})
assert scheduler._run_changed({"recent": {"videos_added": 2}, "deep": {}})
assert not scheduler._run_changed({"bios": {"skipped": "x"}, "portraits": {"cached": 0}})
def test_the_person_page_says_a_bio_is_on_its_way(db, monkeypatch, override_dep):
from fastapi.testclient import TestClient
import app.state as state
from app.auth import current_user
from app.db import get_db
from app.main import app
from app.routes.plex import people
db.add_all(
[
PlexPerson(name="Pending", tmdb_person_id=1),
PlexPerson(name="Known", tmdb_person_id=2, tmdb_enriched_at=DONE, biography="Bio."),
PlexPerson(name="Nobody"),
]
)
user = User(email="enriching@example.com")
db.add(user)
db.commit()
monkeypatch.setattr(state, "_configured_cache", True)
override_dep(get_db, db.info["request"])
override_dep(current_user, lambda: user)
monkeypatch.setattr(people, "_enrich_person_bg", lambda pid: None)
key = {"value": "key"}
monkeypatch.setattr(people.sysconfig, "get_str", lambda d, k: key["value"])
def enriching(name):
resp = TestClient(app).get(f"/api/plex/person/{name}")
assert resp.status_code == 200, resp.text
return resp.json()["enriching"]
assert enriching("Pending") is True
assert enriching("Known") is False
assert enriching("Nobody") is False
key["value"] = "" # no TMDB key: nothing will ever arrive, so the page must not wait
assert enriching("Pending") is False
def test_known_for_is_the_most_voted_roles_not_guest_visits():
payload = {
"combined_credits": {
"cast": [
# Popular this week, but a visit as himself: not what he is known for.
{
"id": 1,
"media_type": "tv",
"name": "Late Show",
"character": "Himself - Guest",
"popularity": 900,
"vote_count": 50,
},
{
"id": 2,
"media_type": "tv",
"name": "Talk Hour",
"character": "",
"genre_ids": [10767],
"popularity": 800,
"vote_count": 400,
},
{
"id": 3,
"media_type": "movie",
"title": "Small Film",
"character": "Bob",
"popularity": 90,
"vote_count": 300,
},
{
"id": 4,
"media_type": "movie",
"title": "Big Film",
"character": "Hero",
"popularity": 10,
"vote_count": 9000,
},
],
"crew": [
{
"id": 5,
"media_type": "movie",
"title": "His Own Film",
"job": "Director",
"vote_count": 1000,
},
{
"id": 6,
"media_type": "movie",
"title": "Thanked In",
"job": "Thanks",
"vote_count": 99999,
},
],
}
}
kf = plex_tmdb.parse_person(payload)["known_for"]
assert [k["title"] for k in kf] == ["Big Film", "His Own Film", "Small Film"]
def test_the_page_shows_at_most_fifteen_known_for_titles(db):
from app.routes.plex import people
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
for i in range(1, 21): # TMDB ids start at 1
db.add(
PlexItem(
rating_key=f"k{i}", library_id=lib.id, kind="movie", title=f"T{i}", guid=f"tmdb:{i}"
)
)
db.commit()
pool = [{"tmdb_id": i, "media_type": "movie", "title": f"T{i}"} for i in range(1, 21)]
cards = people._resolve_known_for(db, pool, 0, [lib.id], [], "en")
assert [c["title"] for c in cards] == [f"T{i}" for i in range(1, 16)]
+1
View File
@@ -133,6 +133,7 @@ def test_person_detail_unknown_name_is_empty(db, monkeypatch, override_dep):
"place_of_birth": None,
"known_for_department": None,
"also_known_as": [],
"enriching": False,
"known_for": [],
"movies": [],
"shows": [],
+73 -1
View File
@@ -191,11 +191,12 @@ def test_person_detail_schedules_enrich_off_request_path(db, monkeypatch, overri
scheduled: list[int] = []
monkeypatch.setattr(people, "_enrich_person_bg", lambda pid: scheduled.append(pid))
monkeypatch.setattr(people.sysconfig, "get_str", lambda d, k: "key")
people._enrich_inflight.clear() # ensure the per-person cooldown guard doesn't suppress this open
resp = _get(db, user, monkeypatch, override_dep, "/api/plex/person/Lazy P")
assert resp.status_code == 200, resp.text
assert resp.json()["biography"] is None # not enriched yet — the bio lands by the next open
assert resp.json()["biography"] is None # not enriched yet — the page asks again (`enriching`)
assert len(scheduled) == 1 # the enrich was scheduled off the request path
@@ -332,3 +333,74 @@ def test_person_detail_serves_bio_and_resolves_known_for(db, monkeypatch, overri
assert j["also_known_as"] == ["Alt Name"]
# Only the in-library known_for credit resolves to a card; the out-of-library one is dropped.
assert [c["title"] for c in j["known_for"]] == ["Known Movie"]
# --- E30 S4: the biography in the reader's language --------------------------------------------------
# The same /person request appends `translations`; the Hungarian one is kept beside the instance-language
# biography, and a Hungarian reader is served it, falling back to the English one when TMDB has none.
def _translation(lang: str, bio: str) -> dict:
return {"iso_639_1": lang, "iso_3166_1": lang.upper(), "data": {"biography": bio}}
def test_parse_person_keeps_the_hungarian_biography():
payload = {
**PAYLOAD,
"translations": {
"translations": [
_translation("de", "Ein Schauspieler."),
_translation("hu", " Ünnepelt színész. "),
]
},
}
assert plex_tmdb.parse_person(payload)["biography_hu"] == "Ünnepelt színész."
def test_parse_person_without_a_hungarian_biography():
assert plex_tmdb.parse_person(PAYLOAD)["biography_hu"] is None
blank = {**PAYLOAD, "translations": {"translations": [_translation("hu", " ")]}}
assert plex_tmdb.parse_person(blank)["biography_hu"] is None
def test_fetch_person_asks_for_the_translations():
client = plex_tmdb.TmdbClient.__new__(plex_tmdb.TmdbClient)
seen: list[dict] = []
client._get = lambda path, params=None: seen.append(params or {}) or {} # type: ignore[method-assign]
client.fetch_person(500)
assert "translations" in seen[0]["append_to_response"].split(",")
def test_person_detail_serves_the_biography_in_the_readers_language(db, monkeypatch, override_dep):
from datetime import datetime, timezone
done = datetime(2024, 1, 1, tzinfo=timezone.utc)
db.add(PlexLibrary(plex_key="7", title="Movies", kind="movie", enabled=True))
db.add_all(
[
PlexPerson(
name="Both P",
tmdb_person_id=1,
tmdb_enriched_at=done,
biography="English bio.",
biography_hu="Magyar életrajz.",
),
PlexPerson(
name="Only EN P", tmdb_person_id=2, tmdb_enriched_at=done, biography="Only English."
),
]
)
user = User(email="biolang@example.com")
db.add(user)
db.commit()
def bio(name: str, lang: str) -> str | None:
resp = _get(db, user, monkeypatch, override_dep, f"/api/plex/person/{name}?lang={lang}")
assert resp.status_code == 200, resp.text
return resp.json()["biography"]
assert bio("Both P", "hu") == "Magyar életrajz."
assert bio("Both P", "en") == "English bio."
assert (
bio("Only EN P", "hu") == "Only English."
) # TMDB has no Hungarian one: the English stands in
+73 -4
View File
@@ -49,7 +49,7 @@ MOVIE_PAYLOAD = {
],
},
"production_companies": [{"name": "Paramount"}, {"name": "Alfran"}],
"production_countries": [{"name": "United States of America"}],
"production_countries": [{"iso_3166_1": "US", "name": "United States of America"}],
"recommendations": {
"results": [
{
@@ -126,7 +126,8 @@ def test_parse_movie_pulls_all_fields():
},
] # the Gaffer is filtered; the two Coppola jobs survive (dedup is by name+job)
assert d["studios"] == ["Paramount", "Alfran"]
assert d["production_countries"] == ["United States of America"]
# E30: the code beside the English name, so the client can name the country in the reader's language.
assert d["production_countries"] == [{"code": "US", "name": "United States of America"}]
assert [r["tmdb_id"] for r in d["recommendations"]] == [240, 999]
assert d["backdrops"] == ["/bd1.jpg", "/bd2.jpg"]
# cast + crew names → tmdb person ids for the plex_people foundation
@@ -255,13 +256,13 @@ def test_serve_block_none_until_enriched_and_resolves_recs(db):
db.add_all([sequel, fresh])
db.commit()
assert tmdb.serve_block(db, fresh) is None # not enriched yet
assert tmdb.serve_block(db, fresh, lang="en") is None # not enriched yet
tmdb.apply_tmdb_detail(fresh, tmdb.parse_tmdb(MOVIE_PAYLOAD, "movie"))
fresh.tmdb_enriched_at = datetime.now(timezone.utc)
db.commit()
block = tmdb.serve_block(db, fresh)
block = tmdb.serve_block(db, fresh, lang="en")
assert block is not None
assert block["budget"] == 6000000
assert block["keywords"] == ["mafia", "crime family"]
@@ -409,6 +410,7 @@ def test_resolve_recs_matches_by_media_type_not_bare_id(db):
{"tmdb_id": 500, "title": "Movie 500", "media_type": "movie"},
{"tmdb_id": 900, "title": "Show 900", "media_type": "tv"},
],
lang="en",
)
}
assert miss[500]["in_library"] is False and miss[500]["rating_key"] is None
@@ -423,7 +425,74 @@ def test_resolve_recs_matches_by_media_type_not_bare_id(db):
{"tmdb_id": 900, "media_type": "movie"},
{"tmdb_id": 500, "media_type": "tv"},
],
lang="en",
)
}
assert hit[900]["in_library"] is True and hit[900]["rating_key"] == "m900"
assert hit[500]["in_library"] is True and hit[500]["rating_key"] == "s500"
# ---- E30: countries by ISO code -----------------------------------------------------------------
class FakeCountries(FakeTmdb):
"""TMDB's /configuration/countries, counting the requests."""
def __init__(self):
super().__init__()
self.calls = 0
def countries(self):
self.calls += 1
return [
{"iso_3166_1": "US", "english_name": "United States of America"},
{"iso_3166_1": "YU", "english_name": "Yugoslavia"},
]
def test_countries_stored_before_codes_get_them_from_tmdbs_list(db):
lib = _lib(db)
old = PlexItem(
rating_key="m1",
library_id=lib.id,
kind="movie",
title="Old",
production_countries=["Yugoslavia", "Atlantis"],
tmdb_enriched_at=datetime.now(timezone.utc),
)
show_lib = _lib(db, key="2", kind="show")
show = PlexShow(
rating_key="s1",
library_id=show_lib.id,
title="Show",
production_countries=["United States of America"],
)
new = PlexItem(
rating_key="m2",
library_id=lib.id,
kind="movie",
title="New",
production_countries=[{"code": "US", "name": "United States of America"}],
)
db.add_all([old, show, new])
db.commit()
# Served before the backfill: the bare name, with no code — never a crash on the old shape.
assert tmdb.serve_block(db, old, lang="hu")["production_countries"] == [
{"code": None, "name": "Yugoslavia"},
{"code": None, "name": "Atlantis"},
]
fake = FakeCountries()
assert tmdb.backfill_country_codes(db, fake) == 2
db.refresh(old)
db.refresh(show)
assert old.production_countries == [
{"code": "YU", "name": "Yugoslavia"},
{"code": None, "name": "Atlantis"}, # not in TMDB's list → no code, the name kept
]
assert show.production_countries == [{"code": "US", "name": "United States of America"}]
# Nothing left to code: no request at all.
assert tmdb.backfill_country_codes(db, fake) == 0
assert fake.calls == 1
+91
View File
@@ -2,7 +2,10 @@
through (at the top of _job): holding _activity_lock across the check-and-set means exactly one
caller can hold a job's "running" claim at a time, and it's claimable again once released."""
from sqlalchemy import select
from app import scheduler
from app.models import Notification
_JOB = "unit_test_claim" # not a real job id — exercises the primitive without touching live jobs
@@ -23,3 +26,91 @@ def test_claim_is_exclusive_until_released():
# ...and now it can be claimed again.
assert scheduler._claim_running(_JOB) is True
_release()
# ---- E30: the result as data -------------------------------------------------------------------
def _snapshot_job(job_id):
return next(j for j in scheduler.scheduler_snapshot()["jobs"] if j["id"] == job_id)
def test_a_run_keeps_its_result_as_data_for_the_readers_language(db, make_user):
"""The dashboard and the manual-run notice showed `_summarize`'s English one-liner ("new=3,
failed=0", a Python repr for a nested phase). The result rides along as data, so the client names
it; the one-liner stays as the fallback and the audit text."""
admin = make_user("sched@example.com", role="admin")
token = scheduler._manual_actor.set(admin.id)
try:
scheduler._job("rss_poll", lambda _db: {"new": 3, "failed": 0, "fallback": 1})
scheduler._job("enrich", lambda _db: 5) # enrich returns a bare count
finally:
scheduler._manual_actor.reset(token)
rss = _snapshot_job("rss_poll")
assert rss["result"] == {"new": 3, "failed": 0, "fallback": 1}
assert rss["last_result"] == "new=3, failed=0, fallback=1"
assert _snapshot_job("enrich")["result"] == {"enriched": 5}
notices = db.execute(
select(Notification).where(Notification.user_id == admin.id).order_by(Notification.id)
).scalars()
assert [n.data["result"] for n in notices] == [
{"new": 3, "failed": 0, "fallback": 1},
{"enriched": 5},
]
def test_a_failed_run_drops_the_previous_result(db):
scheduler._job("rss_poll", lambda _db: {"new": 1})
scheduler._job("rss_poll", lambda _db: (_ for _ in ()).throw(RuntimeError("boom")))
rss = _snapshot_job("rss_poll")
assert rss["last_error"] == "boom"
assert rss["result"] is None
def test_a_job_runs_audit_row_carries_its_result_as_data(db):
"""E30 review: the audit log names a run's result in the reader's language, as the inbox does —
from data, not from the English one-liner (a nested phase there was a Python dict repr)."""
from app.models import AuditLog, User
admin = User(email="auditrun@example.com")
db.add(admin)
db.commit()
scheduler._audit_run(db, None, "people_enrich", "ok", "x", {"bios": {"enriched": 2}})
scheduler._audit_run(db, admin.id, "rss_poll", "skipped", "sync paused", scheduler._PAUSED)
scheduler._audit_run(db, admin.id, "rss_poll", "error", "boom", None)
rows = db.execute(select(AuditLog).order_by(AuditLog.id)).scalars().all()
assert [r.after for r in rows] == [
{"status": "ok", "summary": "x", "result": {"bios": {"enriched": 2}}},
{"status": "skipped", "summary": "sync paused", "result": {"skipped": "sync paused"}},
{"status": "error", "summary": "boom"},
]
def test_a_failure_the_job_names_rides_along_as_data(db, make_user):
"""E31: a failed RSS poll reached the Hungarian audit log as the server's English sentence. A
failure the job names (`progress.JobFailed`) is stored as data beside that sentence — on the
dashboard, the manual-run notice and the audit row — so the client says it in the reader's
language. Any other failure keeps only its own text (the test above)."""
from app import progress
from app.models import AuditLog
def outage(_db):
raise progress.JobFailed("rss_all_failed", "RSS poll failed", total=2, cause="transport")
admin = make_user("failure@example.com", role="admin")
token = scheduler._manual_actor.set(admin.id)
try:
scheduler._job("rss_poll", outage)
finally:
scheduler._manual_actor.reset(token)
named = {"failure": "rss_all_failed", "total": 2, "cause": "transport"}
rss = _snapshot_job("rss_poll")
assert rss["last_error"] == "RSS poll failed"
assert rss["result"] == named
notice = db.execute(select(Notification).where(Notification.user_id == admin.id)).scalar_one()
assert notice.data["result"] == named
row = db.execute(select(AuditLog).order_by(AuditLog.id.desc())).scalars().first()
assert row is not None and row.after["result"] == named
+9 -1
View File
@@ -6,6 +6,7 @@ import logging
import httpx
import pytest
from app import progress
from app.sync import runner
from app.sync.playlists import FAILED, _Breaker
from app.sync.subscriptions import should_prune
@@ -224,8 +225,15 @@ class TestRssPollResult:
def dead(cid):
raise rss.RssError(f"RSS fetch for {cid} returned HTTP 404", "http_404")
with pytest.raises(RuntimeError, match="HTTP 404"):
with pytest.raises(RuntimeError, match="HTTP 404") as err:
self._run(monkeypatch, fetch=dead, apply_=lambda db, ch, e: 1, channels=("UC_dead",))
# E31: and as data, for the reader's language — whose feed, and the reason code that tells a
# dead feed (`http_404`) from a dead egress (`transport`).
assert isinstance(err.value, progress.JobFailed)
assert (err.value.reason, err.value.params) == (
"rss_all_failed",
{"total": 1, "channel": "UC_dead", "cause": "http_404", "read": 0},
)
def test_a_read_only_database_is_a_total_outage_too(self, monkeypatch):
# Every feed reads fine, nothing can be stored: just as un-polled, just as red.
@@ -0,0 +1,39 @@
"""The admin Configuration page reads every override in one query (E31 S4d).
`sysconfig.describe()` did a `db.get` per key to tell whether an override was set, and `get()` did one
more for its value: ~55 keys, and two round trips for a key with no row, since the session cannot cache
an absence. That was ~40 ms of the page's ~55 on prod. It now reads the table once. DB lane.
"""
from sqlalchemy import event
from app import sysconfig
def _statements(db, fn) -> int:
seen: list[str] = []
engine = db.get_bind()
listener = lambda *a, **k: seen.append(a[2]) # noqa: E731 — (conn, cursor, statement, …)
event.listen(engine, "before_cursor_execute", listener)
try:
fn()
finally:
event.remove(engine, "before_cursor_execute", listener)
return len(seen)
def test_describe_reads_the_overrides_once(db):
sysconfig.set_value(db, "quota_daily_budget", 5000)
db.commit()
assert _statements(db, lambda: sysconfig.describe(db)) == 1
def test_describe_shows_the_stored_value_and_the_default(db):
sysconfig.set_value(db, "quota_daily_budget", 5000)
db.commit()
items = {i["key"]: i for g in sysconfig.describe(db)["groups"].values() for i in g}
assert items["quota_daily_budget"]["is_set"] is True
assert items["quota_daily_budget"]["value"] == 5000
other = next(i for k, i in items.items() if k != "quota_daily_budget" and not i["secret"])
assert other["is_set"] is False
assert other["value"] == other["default"]
+28 -7
View File
@@ -1,8 +1,10 @@
"""The sysconfig registry: its order is the admin page's tab order, and a closed-set knob is a CHOICE.
"""The sysconfig registry: a group's keys stay together, every group has its section, and a closed-set
knob is a CHOICE.
The admin Configuration page used to keep its own hand-written group order, which had already drifted
(`explore` was missing, so its tab sorted last). The page now follows SPECS, so SPECS has to read in
tab order: one run of keys per group.
(`explore` was missing, so its tab sorted last). The page now follows the registry: SECTIONS orders the
groups under their headings (E31), and a group missing there would fall off the page — so each group
of SPECS sits in exactly one section.
A closed-set admin knob is a CHOICE, not free text (E22 S2 follow-up).
@@ -22,7 +24,7 @@ from app.config import Settings, choices_of
CHOICE_KEYS = {s.key for s in sysconfig.SPECS if s.type == "choice"}
def test_each_group_is_one_run_of_keys_so_the_registry_reads_in_tab_order():
def test_each_group_is_one_run_of_keys():
runs = [
s.group
for i, s in enumerate(sysconfig.SPECS)
@@ -32,6 +34,13 @@ def test_each_group_is_one_run_of_keys_so_the_registry_reads_in_tab_order():
assert runs[:2] == ["access", "email"]
def test_every_group_sits_in_exactly_one_section():
listed = [g for _, members in sysconfig.SECTIONS for g in members]
assert len(listed) == len(set(listed)), listed
assert set(listed) == {s.group for s in sysconfig.SPECS}
assert [sid for sid, _ in sysconfig.SECTIONS] == ["access", "youtube", "media", "system"]
def test_every_closed_set_knob_is_a_choice_whose_options_come_from_config():
assert CHOICE_KEYS == {
"search_source",
@@ -58,19 +67,23 @@ def test_the_env_accepts_any_letter_case_and_rejects_an_unknown_value():
class _Row:
def __init__(self, value):
self.value, self.encrypted = value, False
def __init__(self, key, value):
self.key, self.value, self.encrypted = key, value, False
class _FakeDb:
"""Just enough of a Session for sysconfig's reads: one stored override row per key."""
def __init__(self, **rows):
self.rows = {k: _Row(v) for k, v in rows.items()}
self.rows = {k: _Row(k, v) for k, v in rows.items()}
def get(self, _model, key):
return self.rows.get(key)
def scalars(self, _stmt):
# describe() reads every override at once (E31 S4d).
return list(self.rows.values())
def test_a_stored_value_reads_back_canonical_and_a_stale_one_falls_back_to_the_default():
assert sysconfig.get_str(_FakeDb(plex_transcode_hw="OFF"), "plex_transcode_hw") == "off"
@@ -85,6 +98,14 @@ def test_describe_hands_the_picker_its_options():
assert items["quota_daily_budget"]["choices"] == []
def test_describe_lists_every_group_under_its_section_in_order():
d = sysconfig.describe(_FakeDb())
assert [s["id"] for s in d["sections"]] == ["access", "youtube", "media", "system"]
assert d["sections"][0]["groups"] == ["access", "google", "email"]
listed = [g for s in d["sections"] for g in s["groups"]]
assert sorted(listed) == sorted(d["groups"])
def test_set_value_stores_the_canonical_spelling_and_refuses_anything_else(db):
sysconfig.set_value(db, "search_source", "API")
assert sysconfig.get_str(db, "search_source") == "api"
+18 -2
View File
@@ -1,8 +1,8 @@
"""iso(): the single date/datetime→JSON serializer the routes share (R6 S1a)."""
"""iso(): the single date/datetime→JSON serializer the routes share (R6 S1a); to_tsquery_str()."""
from datetime import date, datetime, timezone
from app.utils import iso, normalize_description
from app.utils import iso, normalize_description, to_tsquery_str
def test_none_maps_to_none():
@@ -56,3 +56,19 @@ def test_normalize_description_single_line_unchanged():
def test_normalize_description_folds_crlf_line_endings():
# splitlines() folds CR/CRLF to \n — a documented, benign line-ending normalization.
assert normalize_description("a\r\nb\r\n\r\nc") == "a\nb\nc"
def test_tsquery_prefix_matches_a_last_word_of_three_or_more():
assert to_tsquery_str("ti amo magyar") == "ti & amo & magyar:*"
assert to_tsquery_str("min") == "min:*"
def test_tsquery_matches_a_short_last_word_whole():
# A 1-2 letter prefix made ts_rank score most of the library (5-8 s on prod, E31 S4e).
assert to_tsquery_str("m") == "m"
assert to_tsquery_str("minecraft 4k") == "minecraft & 4k"
assert to_tsquery_str("ti amo m") == "ti & amo & m"
def test_tsquery_without_a_word_is_none():
assert to_tsquery_str("?! --") is None
+67 -1
View File
@@ -38,6 +38,11 @@ runs this check before it is called done (E28 S3g):
which the measure cannot trace to a component) and the playlist table's own rows.
4. **Seen.** A screenshot of every changed state, thin edges at 1:1; the tablet widths (1024 × 768, a
narrow split screen) when the layout can reflow. Phones are the E29 epic's, not this check's.
5. **Judged as a whole** (E30 S6). For each screenshot, one written line on its composition: where the empty
space is, what hugs one side, what the eye lands on first. Then the screen at 125% and 133% browser
zoom and at the largest text size. All are held to [Composition and the reader](#composition-and-the-reader--decided-e30-s6).
A rule that holds is not the same as a screen that works: the person page passed every rule above with
half its panel empty.
## The design lab
@@ -435,6 +440,37 @@ pick a density step; it never changes the anatomy or loses an action. [Nielsen #
- **Reading measure.** Prose (a synopsis, a biography, a channel's description) is capped at ~65
characters a line (the `measure` utility) wherever its container can be wider. [WCAG 1.4.8]
## Composition and the reader — _Decided_ (E30 S6)
The rules above say how each piece looks; these say whether the screen works as a whole, for the one
reader it is built for. Found when the person page passed every rule with its biography in the left half
of a full-width panel and nothing on the right.
- **The space is used.** A narrow element in a wide container (prose at `measure`, a form, a short list)
comes with a decision about the rest of the width: a second column with related content (the person
page: facts and bio left, "Known for" right), a container sized to its content, or a layout that stacks.
A band of nothing wider than about a third of a panel, beside content that stops short, is a finding.
[Material layout, Gestalt proximity]
- **One focal point, then an order.** A screen has one thing the eye lands on first (the page heading, the
hero, the player), then the sections, then the metadata. A heading level keeps one look across screens;
a second focal point competing with the first is a finding. [Nielsen #8, HIG hierarchy]
- **Worst real data, not the demo row.** A screen is judged with the longest title, the Hungarian label,
zero items, the largest list, a missing picture and a missing field. None of them clips, overlaps or
leaves a hole where a value was. [Nielsen #9, WCAG 1.4.10]
- **Where am I.** The browser tab names the place and the object in it ("Nicolas Cage · Plex · Siftlode"),
the rail marks the place, and Back returns to the same view at the same scroll position with focus
where it was. [Nielsen #1, WCAG 2.4.2]
- **A click answers at once.** A control shows its pressed or changed state within ~100ms (an optimistic
update, a spinner in the control); what a screen opens with is the app's own data — an outside service
(TMDB, YouTube) is fetched ahead of time by a job, not on the click, so a page never waits on it.
Late content keeps its place (B8). [Nielsen #1, RAIL]
- **Zoom and text size are first-class.** The owner reads with the browser zoomed and a large text size.
Every screen holds at 125% and 133% browser zoom on a 1080p desktop window (what the owner uses: the
layout of a 1536 × 729 and a 1444 × 685 viewport) and at the largest Settings text size at 1024 wide:
nothing clipped or overlapping, no sideways page scroll, every control reachable and at
least its rung, text wrapping rather than cut. Measured, not assumed, on every UI change. [WCAG 1.4.4,
1.4.10, 1.4.12]
## Interaction rules — _Decided_
- **Keyboard:** everything works by keyboard, in visual order, with no traps. A shortcut is a row in
@@ -531,7 +567,7 @@ pick a density step; it never changes the anatomy or loses an action. [Nielsen #
- **Messages.** A toast or error of one sentence has no full stop and no "✓": "Couldn't sync
subscriptions" / "Nem sikerült szinkronizálni". Two sentences keep their stops. Help text is prose.
- **Quotes** around a name are curly: “abc” in English, „abc” in Hungarian.
- **Dates**, in the UI language, only from `lib/format.ts` (lint: `local/date-format`): recency in a list
- **Dates**, in the UI language, only from `lib/format.ts` (lint: `local/locale-format`): recency in a list
is relative ("3d ago" / "3 napja") through `ui/RelativeTime`, which carries the exact minute on hover;
a date is "Sep 27, 2026" / "2026. szept. 27."; a moment "Sep 27, 2026, 08:30 AM" /
"2026. szept. 27. 08:30"; seconds only in the server log, the audit and the recorder.
@@ -540,6 +576,17 @@ pick a density step; it never changes the anatomy or loses an action. [Nielsen #
label holding one is written in that same form ("Under 90 min"). Never a bare letter ("90m", "24h",
"7d"; test-held). In a sentence, and in a list of time spans named in words ("24 hours", "1 week"), the
unit is a word. A clock-style length on a picture stays "2:34:11".
- **Numbers** in the UI language, from `lib/format.ts` (same lint): the decimal mark and grouping are the
language's ("8.3" / "8,3", "1,234" / "1 234"); a big count is compact ("1.2K" / "1,2 E"); a size keeps
its symbol ("5.0 GB" / "5,0 GB"), as do a stream's (Mbps) and "%" / "px". Never `toFixed` on screen.
- **Order.** An A→Z list is in the UI language's alphabet ("Ö" after "O", "Cs" after "Cu" in Hungarian),
by the name the reader SEES: `compareText` (`lib/collate.ts`) on the client, the ICU collation of
`app/collation.py` on the server, so both sort alike.
- **Hungarian words, not English ones** (E30 D3). Kept as Hungarian users say them: Feed, Shorts, Admin,
Token, Port, and technical tokens (log levels, ffmpeg presets, remux). Everything else is Hungarian:
közvetítés (stream), Prioritás, Feladat (job), Eseménynapló (audit), Demó, Automatikus, Infó,
Felület / Szerver (frontend / backend), videók behúzása (backfill), átkódolás (transcode), e-mail.
A longer Hungarian word gets its room — a column is sized to it, a pill truncates — never clipped.
- A label says what happens ("Delete playlist"), not "OK"; a destructive confirm names the object.
## Anti-patterns
@@ -550,6 +597,9 @@ pick a density step; it never changes the anatomy or loses an action. [Nielsen #
- A click handler on text that swallows the user's text selection.
- A colour literal, a px font size, a px-frozen box.
- An empty state shown for an error.
- Half a panel left empty beside content that stops short.
- A page that waits on an outside service when the user opens it.
- A screen checked only at 100% zoom and the default text size.
## Decision log
@@ -624,6 +674,9 @@ pick a density step; it never changes the anatomy or loses an action. [Nielsen #
| 2026-09-27 | C4 as built | `initialFocus` required on every `Modal`; the commit row in `ModalFooter`, portalled under the body from any depth; the fades driven by the body's scroll timeline; `interactive-widget=resizes-content` | An optional prop defaulting to the ✕; a `footer` prop (the issue composer's buttons live three components down); always-on fades | A default is how 21 dialogs came to open on the ✕; lifting the composer's state to reach a prop is the larger change; a resting fade dimmed the first line |
| 2026-09-27 | C3 + C5 as built | A tap toggles the bar through `pointerType` from `pointerdown`; a double-tap takes back its first tap's toggle; the card's pair is the video card's `MediaResumeButtons` (Continue / Restart), a sibling over the picture; the YT description is an ⓘ click popover | A delayed single tap (wait out the double-tap window); new "Resume / From the start" words for Plex; the pair inside the picture's button | A delayed tap feels dead; one media card says one pair of words; a button inside a button is invalid and the hover row already sits beside the picture |
| 2026-09-27 | C2 as built | The side panel's drawer is `ui/Modal` `drawer-start`; drawer = a coarse pointer or below 768; the pin offered in its header ≥ 1024 | A drawer of its own inside `SidePanel` | Esc, Back, the scrim, the trap and the scroll lock are the dialog's; a second implementation is how two shells drift |
| 2026-09-30 | Composition and the reader (E30 S6) | A chapter and a Conformance step: the space is used, one focal point, worst real data, where am I, a click answers at once, zoom and text size first-class; the person page as two columns (facts and bio left, "Known for" right) | Rules for pieces only; the bio in a full-width panel; a narrower panel | The person page passed every piece rule with half its panel empty; the owner reads zoomed in, so zoom is a requirement, not a nicety |
| 2026-09-30 | Hungarian words (E30 S5) | The D3 list: Feed, Shorts, Admin, Token, Port stay; stream, Prio, Job, Audit, Demo, Auto, Info, Frontend/Backend, Backfill, transcode get the Hungarian word; one name per thing (kérelem, behúzás, one "Test connection") | Keep every loanword; translate Feed/Shorts too | The owner chose the list (D3); the final sweep found the same feature under two names in three places |
| 2026-09-29 | Numbers and order in the UI language (E30 S3) | Every number through `lib/format.ts` in the UI language (decimal mark, grouping, compact, units); A→Z by the shown name in the UI language's alphabet, client and server alike; the language comes off `useTranslation`, never the singleton | `toFixed`, hand-built "1.2K", `localeCompare` in the browser's locale, byte-order server sorts | A Hungarian reader saw "8.3", "1.2K", "Életrajzi" after "Z"; the singleton read also left values in the old language after a switch (React Compiler) |
| 2026-09-27 | Narrow rows by container | The download row, the playlist table and the Channels tag row stack by a container query; a fact line never wraps | Breakpoints on the window | The iPad zoomed in, split screen and a column beside a rail are narrow while the window is not; the zoomed download row broke "18.9 / MB" apart |
| 2026-09-27 | Phones deferred | A phone-only layout (the hamburger nav, a folded feed band) waits for its own epic; E28 stays desktop + tablet | Fix the phone views inside E28 | Principle 2; the first phone attempts made the feed worse, and the epic's review grows with every sprint |
| 2026-09-27 | Conformance check (S3g) | A four-step check before a UI change is done: primitive at a named rung, the chapter re-read, the rendered page measured by kind and look, every state seen | A reviewer's eye per sprint | The S1 probe found 69 button looks; after S3 every control kind is drawn by its primitive, and the check keeps a new look from arriving unnoticed |
@@ -636,3 +689,16 @@ pick a density step; it never changes the anatomy or loses an action. [Nielsen #
| 2026-09-27 | Match switch on the panel | `ui/MatchToggle` above the picker from two picks, everywhere; one wording (Any / All) | The picker's popover footer (Feed, Plex collection) | A closed picker hid an active "all"; six pickers had drawn it two ways |
| 2026-09-27 | Missing picture, bare link | A glyph of the kind for no picture; a download with no name is titled by its short link, once | An empty box; the URL as title, channel and link | The owner's picks 4A and 3A: an empty box read as loading, the URL showed three times |
| 2026-09-27 | Copy on a selection | Right-click on selected words keeps the browser's menu; no app Copy row | An app menu with Copy first | A real browser's menu has copy, search and translate; only the desktop pane's is thin |
| 2026-10-01 | Zoom target 125–133% | Screens are held at 125% and 133% browser zoom (1536 × 729, 1444 × 685 on a 1080p window) and the largest text size | 200% browser zoom | The owner: 200% was a guess; 125–133% is what they read Siftlode and most sites at, so 200% would be an upper bound nobody uses |
| 2026-10-01 | E31 Nav rail at short heights | One "Admin" rail item opening a panel of the admin pages | A denser rail when short; a visible "+4" hint | The rail fits at 133–150% without a hidden scroll; the admin pages are rare and admin-only |
| 2026-10-01 | E31 Configuration sections | A vertical, grouped section list, as Settings has | Group tabs + sub-tabs; one long page with a contents list | Sixteen tabs wrapped to three rows at 133%; one pattern for both settings pages |
| 2026-10-01 | E31 Statistics overview | Number tiles, and the per-operation counts as horizontal bars | Three narrow cards; a 30-day chart | The label sits by its value; what spends the quota is seen at a glance |
| 2026-10-01 | E31 Channel page header | The full banner, folding to one row once the grid scrolls | Always compact; no banner | The header is fixed and took 40% of a 685px window; the channel keeps its face at first sight |
| 2026-10-01 | E31 Channel page watch state | A fresh channel page shows the unwatched videos; a picked state is remembered | Show all, forgotten on reload | The owner: open a channel to see what is new; a choice made there should stick |
| 2026-10-01 | E31 Audit log runs | A job's successful runs fold into one row that opens, whatever ran between; only that job's failure (never folded, danger colour) breaks it | A filter defaulting to admin actions; both | Nothing hidden, the noise collapsed, the failure seen |
| 2026-10-01 | E31 Users row | Role and state as badges, the actions in the row's one menu | An inline role toggle; icon-only buttons | Action placement: the row shows state, rare and destructive actions live in the menu |
| 2026-10-01 | E31 YouTube player | Video beside an info panel at 1280px and wider (the video as large as the height allows); stacked below | Today's stack with a one-line bar; full-window like Plex | The owner's eyesight: about 1460px of video on 1080p instead of 1008, the actions still visible |
| 2026-10-01 | E31 Plex info and show pages | A second column for the facts, keywords and crew, as the person page; a collection's hero sized to its content | Full-width text; a larger poster and fanart | The right ~40% was empty; one composition for every media page |
| 2026-10-01 | E31 Nav rail rows are links | Each page row is a real link: a middle-click or Ctrl-click opens the page in a new tab | A button | The owner: open a page in a tab of its own, as on any site |
| 2026-10-02 | E31 Plex detail second column (refined) | The facts over the key crew beside the text; the keywords, chips that filter like the genres, under the text; the show page gets all three too | All three in the column, as the mockup drew it | At 20 keywords and 12 crew the column ran to twice the text's height (849 vs ~400 px at 133%): a hole under the text instead of beside it |
| 2026-10-02 | E31 Plex detail poster and crew | The movie and show poster keeps its 2:3 and grows a rung in a wide hero (w-56 at a 64rem panel); the crew shows its first 6 names and "Show N more" (a list of 8 or fewer shows whole) | Poster stretched to the row; a "More" on the keywords | The row is as tall as the crew column, so the poster drew ~175×590 and cropped; the crew, not the keywords, set the hero's height |
+15 -9
View File
@@ -106,28 +106,34 @@ test.describe("channel view", () => {
await expectContentType(page, "includeLive", false);
});
// Platform S4b: the whole channel chrome (banner, identity, tabs, toolbar) is fixed — it stays
// put while only the video grid scrolls.
test("the channel chrome stays fixed while the video grid scrolls", async ({ page }) => {
// Platform S4b + E31 S4c: the channel chrome sits outside the scroller. Scrolling into the videos
// folds the banner and identity once (the name moves into the tab row); from then on the folded
// chrome stays put while only the video grid scrolls.
test("the channel chrome folds once, then stays fixed while the grid scrolls", async ({
page,
}) => {
await openApp(page);
await page.getByTestId("video-card-channel").first().click();
const count = page.getByTestId("item-count");
const title = page.getByTestId("channel-title");
await expect(count).toBeVisible();
await expect(page.getByTestId("video-card").first()).toBeVisible();
const titleTop = await title.evaluate((el) => el.getBoundingClientRect().top);
const countTop = await count.evaluate((el) => el.getBoundingClientRect().top);
await expect(page.getByTestId("channel-folded")).toHaveCount(0);
// Keep nudging until the grid is tall enough to actually scroll (it may still be filling in).
await expect
.poll(async () => {
await page.getByRole("main").evaluate((el) => el.scrollBy(0, 1500));
await page.getByRole("main").evaluate((el) => el.scrollBy(0, 600));
return page.getByRole("main").evaluate((el) => el.scrollTop);
})
.toBeGreaterThan(400);
const folded = page.getByTestId("channel-folded");
await expect(folded).toBeVisible();
await expect(folded.getByTestId("channel-title")).toBeVisible();
const titleTop = await folded.evaluate((el) => el.getBoundingClientRect().top);
const countTop = await count.evaluate((el) => el.getBoundingClientRect().top);
// The chrome didn't move — banner title and toolbar are in exactly the same place.
expect(await title.evaluate((el) => el.getBoundingClientRect().top)).toBe(titleTop);
await page.getByRole("main").evaluate((el) => el.scrollBy(0, 400));
expect(await folded.evaluate((el) => el.getBoundingClientRect().top)).toBe(titleTop);
expect(await count.evaluate((el) => el.getBoundingClientRect().top)).toBe(countTop);
});
});
+65 -15
View File
@@ -82,33 +82,66 @@ const focusSafeDisabled = {
};
},
};
// E28 S3g (docs/DESIGN.md "Copy"): the app shows a date in three forms plus the relative one, all
// built in lib/format.ts in the UI language. A bare `toLocaleString()` formats in the BROWSER's locale
// and as numbers with seconds; seven call sites had drifted into five shapes. Outside lib/format.ts
// a `toLocale*String` call or an `Intl.DateTimeFormat` / `Intl.RelativeTimeFormat` is refused (a
// number goes through formatNumber, which the same rule covers).
const dateFormat = {
// E28 S3g + E30 S3 (docs/DESIGN.md "Copy"): a date, a number and an alphabetical order are all in the
// UI language — dates and numbers built in lib/format.ts, text compared in lib/collate.ts. A bare
// `toLocaleString()` formats in the BROWSER's locale (seven date sites had drifted into five shapes), a
// bare `toFixed` writes the English decimal point, a bare `localeCompare` sorts in the browser's
// alphabet. So outside those files a `toLocale*String` call or an Intl date/number formatter is refused,
// an `Intl.Collator` outside collate.ts, and a `localeCompare` anywhere.
//
// And in a component, the language itself must come off `useTranslation()`, never the i18n singleton
// (`import i18n from "../i18n"`): the React Compiler memoizes on what it can see, and a singleton read
// is invisible to it — a card kept "2 hr 4 min" and a Plex query its English after a switch to
// Hungarian. The singleton is resolved by SCOPE, so `const { i18n } = useTranslation()` passes.
const localeFormat = {
meta: {
type: "problem",
messages: {
date: "Format dates and numbers through lib/format.ts (formatDate, formatDateTime, formatDateTimeSec, relativeTime, formatNumber), in the UI language.",
date: "Format dates and numbers through lib/format.ts (formatDate, formatDateTime, formatNumber, formatDecimal, …), in the UI language.",
order:
"Compare text with compareText (lib/collate.ts), in the UI language — a bare localeCompare sorts in the browser's.",
singleton:
"Take the language from useTranslation() (`const { i18n } = useTranslation()`), not the i18n singleton — the React Compiler cannot see a singleton read, so the value stays in the old language after a switch.",
},
},
create(context) {
const path = context.filename.split("\\").join("/");
// lib/format.ts is where the forms live; a test computes its expectations; the Plex player's
// on-screen clock (lib/plexPlayerPrefs.ts) is a clock face with its own 12/24h switch, not a date.
if (/\/src\/lib\/(format|plexPlayerPrefs)\.ts$|\.test\.tsx?$/.test(path)) return {};
if (/\.test\.tsx?$/.test(path)) return {};
// lib/format.ts is where the forms live; the Plex player's on-screen clock (lib/plexPlayerPrefs.ts)
// is a clock face with its own 12/24h switch, not a date.
const formats = /\/src\/lib\/(format|plexPlayerPrefs)\.ts$/.test(path);
const collates = /\/src\/lib\/collate\.ts$/.test(path);
const component = /\/src\/components\//.test(path);
const importedFromI18n = (node) => {
for (let scope = context.sourceCode.getScope(node); scope; scope = scope.upper) {
const v = scope.set.get(node.name);
if (v)
return v.defs.some(
(d) => d.type === "ImportBinding" && /\/i18n$|^i18next$/.test(d.parent.source.value),
);
}
return false;
};
return {
"CallExpression > MemberExpression.callee[property.name=/^toLocale(Date|Time)?String$/]"(
node,
) {
context.report({ node, messageId: "date" });
if (!formats) context.report({ node, messageId: "date" });
},
"MemberExpression[object.name='Intl'][property.name=/^(DateTimeFormat|RelativeTimeFormat)$/]"(
"MemberExpression[object.name='Intl'][property.name=/^(DateTimeFormat|RelativeTimeFormat|NumberFormat)$/]"(
node,
) {
context.report({ node, messageId: "date" });
if (!formats) context.report({ node, messageId: "date" });
},
"MemberExpression[object.name='Intl'][property.name='Collator']"(node) {
if (!collates) context.report({ node, messageId: "order" });
},
"CallExpression > MemberExpression.callee[property.name='localeCompare']"(node) {
context.report({ node, messageId: "order" });
},
"MemberExpression[property.name='language'][object.type='Identifier']"(node) {
if (component && importedFromI18n(node.object))
context.report({ node, messageId: "singleton" });
},
};
},
@@ -382,7 +415,7 @@ const localRules = {
"jsx-comment-text": jsxCommentText,
"focus-safe-disabled": focusSafeDisabled,
"button-primitive": buttonPrimitive,
"date-format": dateFormat,
"locale-format": localeFormat,
"input-primitive": inputPrimitive,
"surface-role": surfaceRole,
"literal-a11y-text": literalA11yText,
@@ -395,6 +428,12 @@ const BOX_PX =
"/(^|[\\s:])(min-|max-)?(w|h|size|left|right|top|bottom|inset|gap|p[xytrbl]?|m[xytrbl]?)-\\[[\\d.]+px\\]/";
const BOX_PX_MESSAGE =
"px-frozen box size doesn't scale with --font-scale: use the spacing scale (w-37.5, max-w-60), not w-[Npx].";
// …and a grid's auto-fill column MINIMUM in px (`minmax(150px,1fr)`): the cards' text grows with the
// slider while the column floor stays, so the cards get cramped at a large text size (E30 review). A
// fixed track (`grid-cols-[1fr_240px]`) is left alone, as is VirtualGrid's measured `minCol` (a number).
const GRID_MIN_PX = "/minmax\\([\\d.]+px,/";
const GRID_MIN_PX_MESSAGE =
"px-frozen box size doesn't scale with --font-scale: give a grid column's minimum in rem (minmax(9.375rem,1fr)), not minmax(Npx,…).";
// A stacking order outside the layer scale (DESIGN.md Tokens: `base < menu < chrome < … < tooltip` in
// index.css). A numeric `z-10` equals a level today but doesn't follow it when the scale moves; name it.
const Z_RAW = "/(^|[\\s:])-?z-([1-9]\\d*|\\[[^\\]]+\\])(?![\\w-])/";
@@ -477,7 +516,7 @@ export default tseslint.config(
"local/jsx-comment-text": "error",
"local/focus-safe-disabled": "error",
"local/button-primitive": "error",
"local/date-format": "error",
"local/locale-format": "error",
"local/input-primitive": "error",
"local/surface-role": "error",
"local/literal-a11y-text": "error",
@@ -508,6 +547,15 @@ export default tseslint.config(
// lib/*.contract.test.ts).
"no-restricted-syntax": [
"error",
// A next-page fetch called bare CANCELS the one in flight and sends the same cursor again;
// our requests ignore the abort, so the server ran both (E31 S4e: every feed page 2–3×).
// lib/nextPage.ts NEXT_PAGE makes the second call join the first.
{
selector:
"CallExpression[arguments.length=0]:matches([callee.name='fetchNextPage'], [callee.property.name='fetchNextPage'])",
message:
"fetchNextPage() cancels and re-sends a page already loading: call fetchNextPage(NEXT_PAGE) (lib/nextPage.ts).",
},
// `enabled:` sees only the native attribute; a FocusSafeButton that holds the focus is
// disabled by aria-disabled, so a bare `enabled:hover:` lit it up on hover as if it worked.
{
@@ -554,6 +602,8 @@ export default tseslint.config(
// 150px at the default size) — 32 of these were swept in the 2026-09 debt burn.
{ selector: `Literal[value=${BOX_PX}]`, message: BOX_PX_MESSAGE },
{ selector: `TemplateElement[value.raw=${BOX_PX}]`, message: BOX_PX_MESSAGE },
{ selector: `Literal[value=${GRID_MIN_PX}]`, message: GRID_MIN_PX_MESSAGE },
{ selector: `TemplateElement[value.raw=${GRID_MIN_PX}]`, message: GRID_MIN_PX_MESSAGE },
{ selector: `Literal[value=${Z_RAW}]`, message: Z_RAW_MESSAGE },
{ selector: `TemplateElement[value.raw=${Z_RAW}]`, message: Z_RAW_MESSAGE },
{ selector: `Literal[value=${HOVER_SWAP}]`, message: HOVER_SWAP_MESSAGE },
+6 -5
View File
@@ -1,5 +1,6 @@
import { lazy, Suspense, useEffect, useRef, useState } from "react";
import { qk } from "./lib/queryKeys";
import { saveLanguage } from "./lib/saveLanguage";
import { useTranslation } from "react-i18next";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { api, setUnauthorizedHandler, type Me } from "./lib/api";
@@ -15,7 +16,7 @@ import {
type PanelLayout,
} from "./lib/panelLayout";
import { notify, removeByMetaKind } from "./lib/notifications";
import { LS, setAccountRaw } from "./lib/storage";
import { LS, setTabRaw } from "./lib/storage";
import { usePref } from "./lib/usePref";
import { useVerifyEntry } from "./lib/useVerifyEntry";
import { useNavigation, useNavigationActions } from "./components/navigationContext";
@@ -248,7 +249,7 @@ export default function App() {
if (result === "ok") {
notify({ level: "success", message: t("settings.account.googleLink.linked") });
void refetchMe();
setAccountRaw(LS.settingsTab, "account");
setTabRaw(LS.settingsTab, "account");
setPage("settings");
} else {
const key = result === "conflict" || result === "mismatch" ? result : "error";
@@ -333,9 +334,9 @@ export default function App() {
setLanguage(code);
// Also from the signed-out login page, where the save 401s — `req` marks that as handled, so
// the reporter stays quiet there.
api
.savePrefs({ language: code })
.catch((e: unknown) => notifyActionError(e, t("errors.writeFailed.prefs")));
saveLanguage(queryClient, code).catch((e: unknown) =>
notifyActionError(e, t("errors.writeFailed.prefs")),
);
}
// First-run: a fresh instance is in setup mode (all other routes are locked) → show the wizard.
+5 -4
View File
@@ -8,10 +8,10 @@ import { formatDateTime } from "../lib/format";
import { Button } from "./ui/Button";
import Modal from "./ui/Modal";
function fmtDate(d?: string | null): string {
function fmtDate(d: string | null | undefined, lang: string): string {
if (!d) return "—";
const t = new Date(d);
return isNaN(t.getTime()) ? d : formatDateTime(d);
return isNaN(t.getTime()) ? d : formatDateTime(d, lang);
}
// About dialog: app + build + schema versions (frontend/backend/database), plus a
@@ -23,7 +23,8 @@ export default function About({
onClose: () => void;
onOpenReleaseNotes: () => void;
}) {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
const { data } = useQuery({
queryKey: qk.version(),
queryFn: api.version,
@@ -40,7 +41,7 @@ export default function About({
data ? data.app_version + (data.git_sha !== "unknown" ? ` · ${data.git_sha}` : "") : "…",
],
[t("about.database"), data?.db_revision ?? "…"],
[t("about.buildDate"), fmtDate(data?.build_date ?? FRONTEND_BUILD_DATE)],
[t("about.buildDate"), fmtDate(data?.build_date ?? FRONTEND_BUILD_DATE, lang)],
];
return (
+3 -2
View File
@@ -55,7 +55,8 @@ export default function AccountPanel({
user: AdminUserRow;
onClose: () => void;
}) {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
const qc = useQueryClient();
const [link, setLink] = useState<{ url: string; expires_at: string | null } | null>(null);
const [copied, setCopied] = useState(false);
@@ -182,7 +183,7 @@ export default function AccountPanel({
label={t("users.account.lastLogin")}
value={
a.last_login_at
? formatDateTime(a.last_login_at)
? formatDateTime(a.last_login_at, lang)
: a.is_demo
? t("users.account.lastLoginDemo")
: t("users.account.lastLoginNone")
+10 -6
View File
@@ -13,6 +13,7 @@ import { api, type LogFacet, type LogQuery, type ServerLogRow } from "../lib/api
import { copyText } from "../lib/clipboard";
import { formatDateTimeSec } from "../lib/format";
import { nextNavId, notifyNav, rememberNavPosition } from "../lib/history";
import { nextPageOnClick } from "../lib/nextPage";
import { LEVEL_GROUPS, type LevelGroup } from "../lib/log";
import {
DEFAULT_LOGS_STATE,
@@ -74,7 +75,8 @@ function toQuery(state: LogsUrlState, q: string): LogQuery {
}
export default function AdminLogs() {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
const qc = useQueryClient();
const [state, setState] = useState<LogsUrlState>(() =>
logsParamsToState(new URLSearchParams(window.location.search)),
@@ -155,7 +157,8 @@ export default function AdminLogs() {
? "countsFailed"
: null;
const retryFailed = () => {
if (logs.isFetchNextPageError) void logs.fetchNextPage();
if (logs.isFetchNextPageError)
void logs.fetchNextPage(nextPageOnClick(logs.isFetchingNextPage));
else if (logs.isError) void logs.refetch();
if (facets.isError) void facets.refetch();
};
@@ -189,7 +192,7 @@ export default function AdminLogs() {
const loadOlder = () => {
setLive(false);
void logs.fetchNextPage();
void logs.fetchNextPage(nextPageOnClick(logs.isFetchingNextPage));
};
const copy = async () => {
@@ -406,7 +409,7 @@ export default function AdminLogs() {
level={row.level}
event={row.event ?? "—"}
msg={row.msg}
title={formatDateTimeSec(row.ts)}
title={formatDateTimeSec(row.ts, lang)}
open={open === row.id}
onToggle={() => setOpen(open === row.id ? null : row.id)}
>
@@ -468,14 +471,15 @@ function RowDetail({
onRequest: (id: string) => void;
onJob: (id: number) => void;
}) {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
const pivotCls =
"font-mono underline decoration-dotted underline-offset-2 hover:decoration-solid";
return (
<div className="flex flex-col gap-2">
<dl className="grid grid-cols-[max-content_minmax(0,1fr)] gap-x-3 gap-y-0.5 font-sans text-xs">
<dt>{t("adminLogs.fields.when")}</dt>
<dd className="tabular-nums">{formatDateTimeSec(row.ts)}</dd>
<dd className="tabular-nums">{formatDateTimeSec(row.ts, lang)}</dd>
<dt>{t("adminLogs.fields.source")}</dt>
<dd className="font-mono">
{row.source} · {row.logger}
@@ -93,7 +93,7 @@ describe("the last sign-in column (E22 S1)", () => {
</QueryClientProvider>,
);
expect(await screen.findByText(relativeTime(at))).toBeTruthy();
expect(await screen.findByText(relativeTime(at, "en"))).toBeTruthy();
expect(screen.getAllByText("—")).toHaveLength(2);
});
});
@@ -116,13 +116,63 @@ describe("a user row", () => {
});
});
/** The open menu's rows (a closed popover's are hidden from the accessibility tree in jsdom). */
const menuRows = () => screen.queryAllByRole("menuitem", { hidden: true });
/** The row's actions live in its "…" menu (E31): open it, then pick the row. */
async function chooseInRowMenu(label: string) {
fireEvent.click(await screen.findByRole("button", { name: "users.rowMenu" }));
const item = await waitFor(() => {
const found = menuRows().find((m) => m.textContent?.startsWith(label));
if (!found) throw new Error(`no menu row "${label}"`);
return found;
});
fireEvent.click(item);
}
// E31 (DESIGN § Action placement): the role, the suspension and the delete moved off the row into
// ONE menu — the row's "…", which a right-click on the row opens too. A locked one stays listed with
// its reason.
describe("a user row's menu", () => {
it("holds the role, the suspension and the delete; the row shows no action button", async () => {
mount();
const open = await screen.findByTitle("users.account.open");
const card = open.parentElement!;
const buttons = [...card.querySelectorAll("button")].filter((b) => b !== open);
expect(buttons.map((b) => b.getAttribute("aria-label"))).toEqual(["users.rowMenu"]);
fireEvent.contextMenu(card, { clientX: 5, clientY: 5 });
await waitFor(() => expect(menuRows().length).toBe(3));
expect(menuRows().map((i) => i.textContent)).toEqual([
"users.roles.makeAdmin",
"users.suspend.action",
"users.delete.action",
]);
});
it("lists your own row's actions as locked, with the reason", async () => {
vi.spyOn(api, "adminUsers").mockResolvedValue([{ ...row, id: 1, email: "admin@x" }] as never);
render(
<QueryClientProvider client={new QueryClient()}>
<AdminUsers />
</QueryClientProvider>,
);
fireEvent.click(await screen.findByRole("button", { name: "users.rowMenu" }));
await waitFor(() => expect(menuRows().length).toBe(3));
const [role, suspend, del] = menuRows();
expect(role!.hasAttribute("disabled")).toBe(true);
expect(role!.textContent).toBe("users.roles.makeAdminusers.roles.selfLocked");
expect(suspend!.textContent).toBe("users.suspend.actionusers.suspend.selfLocked");
expect(del!.textContent).toBe("users.delete.actionusers.delete.locked");
});
});
describe("a user action that fails", () => {
it("says a user who is gone, and refetches the list", async () => {
const users = mount();
vi.spyOn(globalThis, "fetch").mockResolvedValue(
answer(404, { code: "not_found", reason: "user" }),
);
fireEvent.click(await screen.findByText("users.roles.makeAdmin"));
await chooseInRowMenu("users.roles.makeAdmin");
await waitFor(() =>
expect(notify).toHaveBeenCalledWith({
level: "error",
@@ -136,7 +186,7 @@ describe("a user action that fails", () => {
it("is not toasted on top of the error dialog the request already opened", async () => {
mount();
vi.spyOn(globalThis, "fetch").mockResolvedValue(answer(409, "Cannot demote the last admin"));
fireEvent.click(await screen.findByText("users.roles.makeAdmin"));
await chooseInRowMenu("users.roles.makeAdmin");
const { reportError } = await import("../lib/errorDialog");
await waitFor(() => expect(reportError).toHaveBeenCalled());
expect(notify.mock.calls.filter(([n]) => n.level === "error")).toEqual([]);
+87 -92
View File
@@ -41,11 +41,15 @@ import AccountPanel from "./AccountPanel";
import { useConfirm } from "./confirmContext";
import Tabs from "./ui/Tabs";
import { useTabPanel } from "./ui/tabPanel";
import { useAccountPersistedState } from "../lib/storage";
import { useTabPersistedState } from "../lib/storage";
import { PageToolbar } from "./ui/PageShell";
import { useMe } from "../lib/useMe";
import { notifyActionError, notifyKnownFailure } from "../lib/actionErrors";
import { Pressable } from "./ui/Pressable";
import { useContextMenu } from "./ui/useContextMenu";
import { ContextMenu } from "./ui/ContextMenu";
import { MoreButton } from "./ui/MoreButton";
import type { ActionMenuItem } from "./ui/ActionMenu";
import { Input } from "./ui/Input";
import { SURFACE } from "./ui/surfaces";
@@ -65,7 +69,7 @@ const ROW_OPEN =
export default function AdminUsers() {
const me = useMe();
const { t } = useTranslation();
const [tab, setTab] = useAccountPersistedState(ADMIN_USERS_TAB_KEY, "roles");
const [tab, setTab] = useTabPersistedState(ADMIN_USERS_TAB_KEY, "roles");
const panel = useTabPanel();
const tabs = [
{ id: "roles", label: t("users.tabs.roles") },
@@ -103,7 +107,8 @@ export default function AdminUsers() {
}
function UsersRoles({ me }: { me: Me }) {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
const qc = useQueryClient();
const confirm = useConfirm();
const q = useQuery({ queryKey: qk.adminUsers(), queryFn: api.adminUsers });
@@ -205,10 +210,58 @@ function UsersRoles({ me }: { me: Me }) {
if (ok) del.mutate({ id: u.id, email: u.email });
};
// The row's actions are ONE menu (DESIGN § Action placement, E31): the role, the suspension and
// the delete — rare and two of them destructive — open from the row's "…", a right-click on the row,
// a long-press or Shift+F10. The row keeps what the admin reads: the name, the role and state
// badges, the last sign-in. A locked action (the demo, yourself) stays listed with its reason.
const rowMenu = useContextMenu<AdminUserRow>({ keyOf: (u) => u.id });
const menuItems = (u: AdminUserRow): ActionMenuItem[] => {
const self = u.id === me.id;
const pending = pendingIds.has(u.id);
const locked = (demoNote: string, selfNote: string) =>
u.is_demo ? demoNote : self ? selfNote : undefined;
const roleNote = locked(t("users.roles.demoLocked"), t("users.roles.selfLocked"));
const suspendNote = locked(t("users.suspend.demoLocked"), t("users.suspend.selfLocked"));
const deleteNote = u.is_demo || self ? t("users.delete.locked") : undefined;
return [
{
id: "role",
label: u.role === "admin" ? t("users.roles.makeUser") : t("users.roles.makeAdmin"),
icon: <Shield />,
disabled: pending || !!roleNote,
note: roleNote,
onSelect: () => onToggle(u),
},
{
id: "suspend",
label: u.is_suspended ? t("users.suspend.undoAction") : t("users.suspend.action"),
icon: u.is_suspended ? <RotateCcw /> : <Ban />,
disabled: pending || !!suspendNote,
note: suspendNote,
onSelect: () => onSuspend(u),
},
{
id: "delete",
label: t("users.delete.action"),
icon: <Trash2 />,
danger: true,
separatorBefore: true,
disabled: pending || !!deleteNote,
note: deleteNote,
onSelect: () => onDelete(u),
},
];
};
const rows = q.data ?? [];
return (
<Section card title={t("users.roles.title")}>
<ContextMenu
menu={rowMenu}
ariaLabel={t("users.rowMenu")}
items={rowMenu.target ? menuItems(rowMenu.target) : []}
/>
{panelFor && <AccountPanel user={panelFor} onClose={() => setPanelFor(null)} />}
<p className="text-xs text-muted leading-relaxed mb-3">{t("users.roles.intro")}</p>
{q.isError && !q.data ? (
@@ -231,6 +284,7 @@ function UsersRoles({ me }: { me: Me }) {
// stretched layer. The hover used to be the name button's own and ended at the first badge.
<div
key={u.id}
{...rowMenu.triggerProps(u)}
className={`${SURFACE.card} ${ROW_LAYER} flex items-center gap-2.5 p-2.5`}
>
<Pressable
@@ -255,97 +309,30 @@ function UsersRoles({ me }: { me: Me }) {
{u.is_active && !u.email_verified && !u.has_google && (
<Badge tone="warn">{t("users.roles.unverified")}</Badge>
)}
<Tooltip
always={u.is_demo || self}
hint={
u.is_demo
? t("users.roles.demoLocked")
: self
? t("users.roles.selfLocked")
: u.role === "admin"
? t("users.roles.makeUser")
: t("users.roles.makeAdmin")
}
>
<Button
variant="glass"
size="xs"
className="relative shrink-0"
onClick={() => onToggle(u)}
// Locked (demo / self) is unavailable WITH a reason, the tooltip's: aria-disabled keeps it
// focusable so a keyboard user reaches the reason. A pending write is transient.
aria-disabled={u.is_demo || self}
disabled={pendingIds.has(u.id)}
icon={<Shield className="w-3.5 h-3.5" />}
>
{u.role === "admin" ? t("users.roles.makeUser") : t("users.roles.makeAdmin")}
</Button>
</Tooltip>
<Tooltip
always={u.is_demo || self}
hint={
u.is_demo
? t("users.suspend.demoLocked")
: self
? t("users.suspend.selfLocked")
: u.is_suspended
? t("users.suspend.undoAction")
: t("users.suspend.action")
}
>
<IconButton
noTitle
label={
u.is_suspended ? t("users.suspend.undoAction") : t("users.suspend.action")
}
size="sm"
variant={u.is_suspended ? "outline-warn" : "outline"}
className="relative"
onClick={() => onSuspend(u)}
aria-disabled={u.is_demo || self}
disabled={pendingIds.has(u.id)}
>
{u.is_suspended ? (
<RotateCcw className="w-4 h-4" />
) : (
<Ban className="w-4 h-4" />
)}
</IconButton>
</Tooltip>
<Tooltip
always={u.is_demo || self}
hint={u.is_demo || self ? t("users.delete.locked") : t("users.delete.action")}
>
<IconButton
noTitle
label={t("users.delete.action")}
size="sm"
variant="outline-danger"
className="relative"
onClick={() => onDelete(u)}
aria-disabled={u.is_demo || self}
disabled={pendingIds.has(u.id)}
>
<Trash2 className="w-4 h-4" />
</IconButton>
</Tooltip>
{/* How long ago this account last signed in. LAST in the row on purpose: the badges and
the role button before it vary in width row to row, so only the row's own right
edge lines the values up into one column. The exact moment is the hint here and a
{/* How long ago this account last signed in. At the row's right end on purpose (only the
fixed-width "…" after it): the badges before it vary in width row to row, so only
the right edge lines the values up into one column. The exact moment is the hint here and a
line in the account panel; below `sm` the row has no room, and the panel still says it. */}
<Tooltip
hint={
u.last_login_at
? t("users.roles.lastLogin", { time: formatDateTime(u.last_login_at) })
? t("users.roles.lastLogin", { time: formatDateTime(u.last_login_at, lang) })
: u.is_demo
? t("users.roles.lastLoginDemo")
: t("users.roles.lastLoginNone")
}
>
<span className="relative hidden sm:inline-block w-20 shrink-0 text-right text-2xs text-muted tabular-nums">
{u.last_login_at ? relativeTime(u.last_login_at) : "—"}
{u.last_login_at ? relativeTime(u.last_login_at, lang) : "—"}
</span>
</Tooltip>
<MoreButton
label={t("users.rowMenu")}
size="md"
expanded={rowMenu.open && rowMenu.target?.id === u.id}
menuProps={rowMenu.buttonProps(u)}
className="relative"
/>
</div>
);
})}
@@ -440,7 +427,8 @@ function PendingAccessRowView({
onActivate: () => void;
onResend: () => void;
}) {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
return (
<div className={`${SURFACE.card} flex items-center gap-2 p-2.5`}>
<div className="min-w-0 flex-1">
@@ -449,7 +437,7 @@ function PendingAccessRowView({
{row.created_at && (
<span>
{t("users.pendingAccess.signedUp", {
time: formatDateTime(row.created_at),
time: formatDateTime(row.created_at, lang),
})}
</span>
)}
@@ -536,6 +524,7 @@ function InviteLangPicker({
function AdminInvites() {
const { t, i18n } = useTranslation();
const lang = i18n.language;
const qc = useQueryClient();
const confirm = useConfirm();
const me = useMe();
@@ -626,7 +615,8 @@ function AdminInvites() {
// is also where the "you already wrote to them" warning belongs, next to the button that would
// do it a second time, rather than only in a tooltip the admin may never hover.
const onInvite = async (inv: Invite) => {
let lang: LangCode = LANGUAGES.some((l) => l.code === inv.lang)
// The language the INVITE is written in — not the admin's `lang`, which dates the sentence below.
let mailLang: LangCode = LANGUAGES.some((l) => l.code === inv.lang)
? (inv.lang as LangCode)
: ((i18n.language.split("-")[0] === "hu" ? "hu" : "en") as LangCode);
const ok = await confirm({
@@ -640,16 +630,16 @@ function AdminInvites() {
: "settings.invites.inviteConfirm",
{
email: inv.email,
time: formatDateTime(inv.invited_at ?? new Date(0).toISOString()),
time: formatDateTime(inv.invited_at ?? new Date(0).toISOString(), lang),
},
)}
</p>
<InviteLangPicker initial={lang} onPick={(v) => (lang = v)} />
<InviteLangPicker initial={mailLang} onPick={(v) => (mailLang = v)} />
</>
),
confirmLabel: t("settings.invites.inviteSend"),
});
if (ok) invite.mutate({ inv, lang });
if (ok) invite.mutate({ inv, lang: mailLang });
};
const add = useMutation({
@@ -728,7 +718,11 @@ function AdminInvites() {
<div key={i.id} className="flex items-center gap-2">
<span className="truncate flex-1">{i.email}</span>
<span className={i.status === "approved" ? "text-accent" : "text-danger"}>
{i.status}
{t(
i.status === "approved"
? "settings.invites.status.approved"
: "settings.invites.status.denied",
)}
</span>
{/* Only an approved row with NOBODY behind it yet: this button means "the door is
open, come in", which is meaningless to someone already inside — and half the
@@ -740,7 +734,7 @@ function AdminInvites() {
hint={
i.invited_at
? t("settings.invites.invitedAt", {
time: formatDateTime(i.invited_at),
time: formatDateTime(i.invited_at, lang),
})
: t("settings.invites.inviteHint")
}
@@ -927,7 +921,8 @@ function InviteRow({
onDelete: () => void;
busy: boolean;
}) {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
return (
<div className={`${SURFACE.card} flex items-center gap-2 p-2.5`}>
<div className="min-w-0 flex-1">
@@ -935,7 +930,7 @@ function InviteRow({
{inv.requested_at && (
<div className="text-2xs text-muted">
{t("settings.invites.requested", {
time: formatDateTime(inv.requested_at),
time: formatDateTime(inv.requested_at, lang),
})}
</div>
)}
@@ -0,0 +1,48 @@
import { useState } from "react";
import { useTranslation } from "react-i18next";
import clsx from "clsx";
import { ChevronRight } from "lucide-react";
import type { AuditRow } from "../lib/api";
import { Button } from "./ui/Button";
import { RelativeTime } from "./ui/RelativeTime";
/** A folded run of one job's successful runs (lib/auditGroups, E31): the row shows the newest, this
* opens the earlier ones, each with its moment and its result (`detailOf`, the row's own wording). */
export default function AuditEarlierRuns({
runs,
detailOf,
}: {
runs: AuditRow[];
detailOf: (run: AuditRow) => string | null;
}) {
const { t } = useTranslation();
const [open, setOpen] = useState(false);
const earlier = runs.slice(1);
return (
<div className="mt-1">
<Button
variant="link-muted"
aria-expanded={open}
onClick={() => setOpen(!open)}
icon={<ChevronRight className={clsx("transition-transform", open && "rotate-90")} />}
>
{t("audit.earlierRuns", { count: earlier.length })}
</Button>
{open && (
<ul className="mt-1 space-y-0.5 text-2xs text-muted">
{earlier.map((run) => (
<li key={run.id} className="flex gap-2">
<RelativeTime
at={run.created_at}
seconds
empty="—"
className="shrink-0 tabular-nums"
/>
<span className="min-w-0 wrap-break-word">{detailOf(run)}</span>
</li>
))}
</ul>
)}
</div>
);
}
+12 -6
View File
@@ -13,23 +13,29 @@ import DataTable from "./ui/DataTable";
import { LoadingState, StateMessage } from "./ui/QueryState";
import { auditColumns } from "./auditColumns";
import { notifyActionError } from "../lib/actionErrors";
import { compareText } from "../lib/collate";
import { foldJobRuns } from "../lib/auditGroups";
// Admin-only audit trail: an append-only who/what/when log of admin actions + scheduler run
// summaries (never per-row). Read-only apart from Clear. Client-side sort/filter/paginate via
// DataTable; the API returns the most recent window (capped) — the log stays small by design.
export default function AuditLog() {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
const qc = useQueryClient();
const confirm = useConfirm();
const q = useQuery({ queryKey: qk.adminAudit(), queryFn: () => api.adminAudit() });
const data = q.data;
const rows = useMemo(() => data?.items ?? [], [data]);
// Back-to-back successful runs of one job fold into one row (lib/auditGroups, E31).
const rows = useMemo(() => foldJobRuns(data?.items ?? []), [data]);
const actionOptions = useMemo(() => {
const present = Array.from(new Set(rows.map((r) => r.action))).sort();
return present.map((a) => ({ value: a, label: t(`auditActions.${a}`, a) }));
}, [rows, t]);
const columns = useMemo(() => auditColumns(t, actionOptions), [t, actionOptions]);
// By what the reader sees — the named action, in their alphabet — not by the code.
return Array.from(new Set(rows.map((r) => r.action)))
.map((a) => ({ value: a, label: t(`auditActions.${a}`, a) }))
.sort((a, b) => compareText(a.label, b.label, lang));
}, [rows, t, lang]);
const columns = useMemo(() => auditColumns(t, lang, actionOptions), [t, lang, actionOptions]);
const clear = useMutation({
mutationFn: () => api.clearAudit(),
+23 -1
View File
@@ -8,7 +8,7 @@
//
// Real NavigationProvider, real storage and i18n; only the two boot probes are stubbed. The probe
// records EVERY render, so "never showed the wrong page, not even for one render" is what is asserted.
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { describe, it, expect, vi, beforeEach, afterEach, type MockInstance } from "vitest";
import { render, screen, cleanup, act, fireEvent, waitFor } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { useEffect } from "react";
@@ -47,6 +47,7 @@ function mount() {
return qc;
}
const account = (over: Partial<Me> = {}) => ({ id: 5, preferences: {}, ...over }) as Me;
let savePrefs: MockInstance<typeof api.savePrefs>;
const retryButton = () => screen.findByRole("button", { name: i18n.t("common.retry") });
beforeEach(async () => {
@@ -57,6 +58,7 @@ beforeEach(async () => {
mounts = 0;
await i18n.changeLanguage("en");
vi.spyOn(api, "setupStatus").mockResolvedValue({ configured: true });
savePrefs = vi.spyOn(api, "savePrefs").mockResolvedValue({ ok: true } as never);
});
afterEach(() => {
cleanup();
@@ -85,6 +87,26 @@ describe("BootGate", () => {
expect(renders.map((r) => r.lang)).toEqual(renders.map(() => "hu"));
});
it("an account with no language stored stores the one it is shown in, once", async () => {
await i18n.changeLanguage("hu"); // the browser's language, never saved
vi.spyOn(api, "me").mockResolvedValue(account());
const qc = mount();
await screen.findByTestId("app");
await waitFor(() => expect(savePrefs).toHaveBeenCalledWith({ language: "hu" }));
expect(savePrefs).toHaveBeenCalledTimes(1);
// The server now sorts in Hungarian, and so does what reads the stored language.
await waitFor(() => expect(qc.getQueryData<Me>(qk.me())?.preferences.language).toBe("hu"));
});
it("an account with its language stored stores nothing", async () => {
vi.spyOn(api, "me").mockResolvedValue(account({ preferences: { language: "hu" } }));
mount();
await screen.findByTestId("app");
expect(savePrefs).not.toHaveBeenCalled();
});
it("mounts nothing of the app while the identity is still unknown", async () => {
vi.spyOn(api, "me").mockReturnValue(new Promise<Me | null>(() => {}));
mount();
+21 -4
View File
@@ -1,7 +1,9 @@
import { useLayoutEffect, useRef, useState, type ReactNode } from "react";
import { useQuery } from "@tanstack/react-query";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { useTranslation } from "react-i18next";
import { api, getActiveAccount, setActiveAccount } from "../lib/api";
import { log } from "../lib/log";
import { saveLanguage } from "../lib/saveLanguage";
import { qk } from "../lib/queryKeys";
import i18n, { isSupported, setLanguage } from "../i18n";
import { LoadingState, StateMessage } from "./ui/QueryState";
@@ -38,6 +40,7 @@ export function BootScreen() {
export default function BootGate({ children }: { children: ReactNode }) {
const { t } = useTranslation();
const qc = useQueryClient();
// First-run install gate: a fresh instance reports configured=false and runs in setup mode, where
// every other route is locked — so `me` is held back until the instance is known to be set up
// (otherwise it would 503 against the setup-mode lock).
@@ -81,7 +84,10 @@ export default function BootGate({ children }: { children: ReactNode }) {
// not is corrected here — otherwise the tab would run as the default account on the server while
// reading and writing the gone account's local state.
// • Adopt the account's language, which follows the account across devices — the local copy only
// knows what this browser last showed.
// knows what this browser last showed. An account with none stored yet (the setup wizard's admin,
// an account older than the stamp sign-up gives) stores the language it is shown in: the server
// orders its lists and writes its mail in the stored one, else English, and a Hungarian screen
// over English-ordered lists breaks "A→Z in the UI language's alphabet".
const [readyFor, setReadyFor] = useState<number | null | undefined>(undefined);
// `me` is read for the identity it produced, so it rides a ref: a background refetch of the same
// account must not re-run the adoption over a language the user has just changed. That rule used
@@ -104,10 +110,21 @@ export default function BootGate({ children }: { children: ReactNode }) {
const account = meRef.current;
if (account && getActiveAccount() !== account.id) setActiveAccount(account.id);
const lang = account?.preferences?.language;
if (typeof lang === "string" && isSupported(lang) && lang !== i18n.language) setLanguage(lang);
if (typeof lang === "string" && isSupported(lang)) {
// eslint-disable-next-line local/locale-format -- a one-shot read in an effect, compared before adopting the account's language; nothing renders it, so there is no memoized value to go stale.
if (lang !== i18n.language) setLanguage(lang);
} else if (account) {
// eslint-disable-next-line local/locale-format -- a one-shot read of the language the app is about to be shown in, stored once; nothing renders it.
const shown = i18n.language;
saveLanguage(qc, shown).catch((e: unknown) =>
log.warn("boot.language_store_failed", "the shown language could not be stored", {
error: String(e),
}),
);
}
// eslint-disable-next-line react-hooks/set-state-in-effect -- the adoption latch. It cannot be derived: the whole point is that the tree mounts only AFTER the pin and the language have been applied, and both are writes to things outside React (sessionStorage, i18n). A layout effect so the frame that first shows the app already has them.
setReadyFor(identity);
}, [identity, readyFor]);
}, [identity, readyFor, qc]);
// No answer at all (a network or 5xx failure on the very first setup or account probe — past the
// one retry `req` already makes): say so, with a retry. An unanswered setup probe is NOT "signed
+12 -20
View File
@@ -3,35 +3,27 @@ import { ExternalLink } from "lucide-react";
import type { ChannelDetail } from "../lib/api";
import { linkify } from "../lib/linkify";
import { Badge } from "./ui/Badge";
import { languageName, regionName, vocab } from "../lib/vocab";
// --- About-tab metadata helpers (shared by the channel page and the manager's About overlay) ---
function displayName(code: string, lang: string, type: "region" | "language"): string {
try {
return (
new Intl.DisplayNames([lang], { type }).of(type === "region" ? code.toUpperCase() : code) ??
code
);
} catch {
return code;
}
}
// YouTube keywords are ONE space-separated string with multi-word tags quoted: gaming "let's play".
function parseKeywords(s: string): string[] {
return (s.match(/"[^"]+"|\S+/g) ?? []).map((k) => k.replace(/^"|"$/g, "")).filter(Boolean);
}
// topicCategories are Wikipedia URLs (…/wiki/Music) → a readable label, de-duplicated.
function topicLabels(urls: string[]): string[] {
// topicCategories are Wikipedia URLs (…/wiki/Music) → the page slug ("Pop_music"), de-duplicated. The
// slug is the vocabulary word (E30): named in the reader's language, else shown with spaces.
function topicSlugs(urls: string[]): string[] {
const seen = new Set<string>();
for (const url of urls) {
const seg = url.split("/wiki/")[1] ?? url.split("/").pop() ?? url;
let label = seg;
let slug = seg;
try {
label = decodeURIComponent(seg);
slug = decodeURIComponent(seg);
} catch {
/* keep raw */
}
label = label.replace(/_/g, " ").trim();
if (label) seen.add(label);
slug = slug.trim();
if (slug) seen.add(slug);
}
return [...seen];
}
@@ -43,7 +35,7 @@ function topicLabels(urls: string[]): string[] {
*/
export default function ChannelAboutContent({ ch }: { ch: ChannelDetail }) {
const { t, i18n } = useTranslation();
const topics = ch.topic_categories?.length ? topicLabels(ch.topic_categories) : [];
const topics = ch.topic_categories?.length ? topicSlugs(ch.topic_categories) : [];
const keywords = ch.keywords ? parseKeywords(ch.keywords) : [];
const hasAboutDetails =
!!ch.country || !!ch.default_language || topics.length > 0 || keywords.length > 0;
@@ -86,14 +78,14 @@ export default function ChannelAboutContent({ ch }: { ch: ChannelDetail }) {
aria-hidden
/>
)}
{displayName(ch.country, i18n.language, "region")}
{regionName(t, ch.country, i18n.language)}
</span>
</div>
)}
{ch.default_language && (
<div className="flex gap-2">
<span className="text-muted w-24 shrink-0">{t("channel.language")}</span>
<span>{displayName(ch.default_language, i18n.language, "language")}</span>
<span>{languageName(ch.default_language, i18n.language, ch.default_language)}</span>
</div>
)}
{topics.length > 0 && (
@@ -102,7 +94,7 @@ export default function ChannelAboutContent({ ch }: { ch: ChannelDetail }) {
<div className="flex flex-wrap gap-1.5">
{/* Future search epic: these topic chips become clickable filters. */}
{topics.map((tp) => (
<Badge key={tp}>{tp}</Badge>
<Badge key={tp}>{vocab(t, "topic", tp, tp.replace(/_/g, " "))}</Badge>
))}
</div>
</div>
+6 -4
View File
@@ -41,7 +41,8 @@ export default function ChannelDiscovery({
onOpenWizard: () => void;
onViewChannel: (id: string, name: string) => void;
}) {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
const qc = useQueryClient();
const confirm = useConfirm();
// The table/cards density is shared with the Subscriptions tab (ChannelsProvider). The sort is
@@ -148,7 +149,7 @@ export default function ChannelDiscovery({
hideInCard: true,
render: (c) => <AboutCell text={c.description} channelId={c.id} />,
},
subsColumn<DiscoveredChannel>(t),
subsColumn<DiscoveredChannel>(t, lang),
{
key: "videos",
header: t("channels.discovery.cols.totalVideos"),
@@ -157,7 +158,7 @@ export default function ChannelDiscovery({
sortable: true,
sortValue: (c) => c.video_count ?? -1,
render: (c) => (
<span className="text-muted tabular-nums">{formatCountOrDash(c.video_count)}</span>
<span className="text-muted tabular-nums">{formatCountOrDash(c.video_count, lang)}</span>
),
},
{
@@ -186,7 +187,7 @@ export default function ChannelDiscovery({
// computed for cards — DataTable sorts and slices internally, so doing it in table mode too would
// copy+sort the whole list every render for a result nothing reads.
const isCards = layout === "cards";
const sortedRows = isCards ? sortRows(rows, columns, sort) : rows;
const sortedRows = isCards ? sortRows(rows, columns, sort, lang) : rows;
const {
paged: pagedRows,
pagerProps,
@@ -241,6 +242,7 @@ export default function ChannelDiscovery({
onSortChange={setSort}
emptyText={t("channels.discovery.empty")}
keyboardNav
narrowedBy={q}
/>
) : (
<ChannelLayoutGrid
@@ -0,0 +1,132 @@
// @vitest-environment jsdom
//
// E31 (the owner's pick): the channel page's chrome is fixed, and its banner + identity block kept ~40% of a
// 133% screen before the first video for as long as the page was open. Once the grid scrolls they fold
// away and the name rides in the tab row; back at the top they return. The Back button is one element in
// both states, so a focused Back keeps its focus through the fold.
import type { ReactNode } from "react";
import { describe, it, expect, vi, beforeAll, afterEach } from "vitest";
import { render, cleanup, screen, act, waitFor } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
vi.mock("react-i18next", async (importOriginal) => ({
...(await importOriginal<typeof import("react-i18next")>()),
useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }),
}));
vi.mock("./navigationContext", () => ({
useNavigationActions: () => ({ closeChannel: vi.fn(), openChannel: vi.fn() }),
}));
vi.mock("./confirmContext", () => ({ useConfirm: () => async () => true }));
vi.mock("./feedFiltersContext", () => ({
useFeedFilters: () => ({ includeNormal: true, includeShorts: false, includeLive: false }),
}));
vi.mock("./Feed", () => ({ default: () => null }));
vi.mock("./ChannelAbout", () => ({ default: () => null }));
vi.mock("./ui/PageShell", async (importOriginal) => ({
...(await importOriginal<typeof import("./ui/PageShell")>()),
PageToolbar: ({ children }: { children?: ReactNode }) => <>{children}</>,
}));
import ChannelPage from "./ChannelPage";
import { CONTENT_ID } from "./ui/landmarks";
import { api, type ChannelDetail, type Me } from "../lib/api";
beforeAll(() => {
class Stub {
observe() {}
unobserve() {}
disconnect() {}
takeRecords() {
return [];
}
}
globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver;
globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver;
});
afterEach(() => {
cleanup();
document.getElementById(CONTENT_ID)?.remove();
vi.restoreAllMocks();
});
const CH = "UCqoAEDirJPjEUFcF2FklnBA";
const BANNER = 'img[src^="https://yt3."]';
// The page's one scroller (PageScroller's <main>), with a scroll position jsdom does not lay out.
function scroller() {
const main = document.createElement("main");
main.id = CONTENT_ID;
let top = 0;
Object.defineProperty(main, "scrollTop", { get: () => top, configurable: true });
Object.defineProperty(main, "scrollHeight", { value: 6000, configurable: true });
Object.defineProperty(main, "clientHeight", { value: 600, configurable: true });
document.body.appendChild(main);
return (to: number) =>
act(() => {
top = to;
main.dispatchEvent(new Event("scroll"));
});
}
// Renders the page and waits for the detail to land (the banner shows).
async function renderPage() {
vi.spyOn(api, "channelDetail").mockResolvedValue({
id: CH,
title: "StarTalk",
banner_url: "https://yt3.googleusercontent.com/banner",
subscribed: true,
blocked: false,
hidden: false,
deep_requested: false,
backfill_done: true,
stored_videos: 10,
tag_ids: [],
} as unknown as ChannelDetail);
vi.spyOn(api, "tags").mockResolvedValue([] as never);
const view = render(
<QueryClientProvider
client={new QueryClient({ defaultOptions: { queries: { retry: false } } })}
>
<ChannelPage
channelId={CH}
initialName="StarTalk"
me={{ id: 1, can_write: true } as unknown as Me}
view="cards"
setView={() => {}}
onShowInFeed={() => {}}
/>
</QueryClientProvider>,
);
await waitFor(() => expect(view.container.querySelector(BANNER)).not.toBeNull());
return view;
}
describe("ChannelPage — the header folds while the grid scrolls", () => {
it("drops the banner and puts the name in the tab row once scrolled, and brings them back at the top", async () => {
const scrollTo = scroller();
const { container } = await renderPage();
const banner = () => container.querySelector(BANNER);
expect(banner()).not.toBeNull();
expect(screen.queryByTestId("channel-folded")).toBeNull();
scrollTo(400);
expect(banner()).toBeNull();
expect(screen.getByTestId("channel-folded").textContent).toContain("StarTalk");
scrollTo(10);
expect(screen.queryByTestId("channel-folded")).not.toBeNull();
scrollTo(0);
expect(banner()).not.toBeNull();
expect(screen.queryByTestId("channel-folded")).toBeNull();
});
it("keeps a focused Back focused through the fold", async () => {
const scrollTo = scroller();
await renderPage();
const back = screen.getByTestId("channel-back");
back.focus();
scrollTo(400);
expect(screen.getByTestId("channel-folded")).toBeTruthy();
expect(document.activeElement).toBe(back);
});
});
+283 -225
View File
@@ -1,6 +1,6 @@
import { useCallback, useEffect, useRef, useState } from "react";
import { useTranslation } from "react-i18next";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { useMutation, useQueryClient } from "@tanstack/react-query";
import { useLiveQuery } from "../lib/useLiveQuery";
import { channelDetailPollMs, useBackfillFeedRefresh } from "../lib/channelDetailPoll";
import { qk } from "../lib/queryKeys";
@@ -24,24 +24,29 @@ import {
import Avatar from "./ui/Avatar";
import Tabs from "./ui/Tabs";
import { useTabPanel } from "./ui/tabPanel";
import { useScrollFold } from "./ui/scrollFold";
import Feed from "./Feed";
import ChannelAboutContent from "./ChannelAbout";
import { PageToolbar } from "./ui/PageShell";
import { PageToolbarSlot } from "./ui/pageToolbarSlot";
import { useFeedFilters } from "./feedFiltersContext";
import { SHOW_IDS } from "./feedFiltersInit";
import { usePref } from "../lib/usePref";
import { LS } from "../lib/storage";
import { useConfirm } from "./confirmContext";
import { notify } from "../lib/notifications";
import { notifyYouTubeActionError } from "../lib/youtubeErrors";
import { notifyActionError } from "../lib/actionErrors";
import { api, DEFAULT_TAG_MODES, type FeedFilters, type Me, type ChannelDetail } from "../lib/api";
import type { FeedView } from "../lib/feedView";
import { channelYouTubeUrl, formatDate, formatViews } from "../lib/format";
import { channelYouTubeUrl, formatDate, formatCompact } from "../lib/format";
import { ChannelTagsControl } from "./ChannelTags";
import { useChannelTagActions } from "../lib/useChannelTagActions";
import { Button } from "./ui/Button";
import { IconButton } from "./ui/IconButton";
import { IconLink } from "./ui/LinkButton";
import { PageHeading } from "./ui/Headings";
import { useTags } from "../lib/useTags";
// A dedicated channel page: an "About"-style header (banner, avatar, stats, subscribe) over the
// channel's videos (the catalog filtered to this channel). For an un-subscribed channel it
@@ -69,6 +74,7 @@ export default function ChannelPage({
onShowInFeed: () => void;
}) {
const { t, i18n } = useTranslation();
const lang = i18n.language;
const { closeChannel } = useNavigationActions();
const qc = useQueryClient();
const confirm = useConfirm();
@@ -84,6 +90,10 @@ export default function ChannelPage({
// where the channelScoped Feed portals ITS toolbar (via the overridden PageToolbarSlot below), so
// the toolbar lands under the tabs inside the fixed band rather than in the scroller.
const [chromeSlot, setChromeSlot] = useState<HTMLDivElement | null>(null);
// The banner and the identity block fold away once the grid scrolls (E31, the owner's pick): the chrome
// is fixed, so unfolded it kept ~40% of a 133% screen for good. Folded, the name rides in the tab row.
const foldable = useRef<HTMLDivElement>(null);
const folded = useScrollFold(foldable);
// The banner comes from Google's image CDN, which refuses any request carrying a Referer from our
// origin — every time, not only in a burst (measured 2026-09-26) — so the <img> sends none
// (`referrerPolicy`, as ui/Avatar). A load can still drop, and the browser NEGATIVE-CACHES the miss,
@@ -230,11 +240,7 @@ export default function ChannelPage({
// invalidates the manager list so a tag added here shows there too.
// Only fetch the tag catalog when the editor will actually render (subscribed, non-demo) — an
// unsubscribed/demo channel page never shows the control, so the request would be thrown away.
const tagsQuery = useQuery({
queryKey: qk.tags(),
queryFn: api.tags,
enabled: !!ch?.subscribed && !me.is_demo,
});
const tagsQuery = useTags({ enabled: !!ch?.subscribed && !me.is_demo });
const userTags = (tagsQuery.data ?? []).filter((tg) => !tg.system);
const { toggleTag, createAndAttachTag } = useChannelTagActions({
optimisticToggle: (id, tagId, attach) =>
@@ -262,7 +268,13 @@ export default function ChannelPage({
// Channel-scoped feed: the whole catalog filtered to this channel (scope=all + source=all so
// the explored, per-user-private videos show — the backend gates them to this explorer).
const [filters, setFilters] = useState<FeedFilters>(() => ({
// The watch state is the one filter a channel page remembers (the owner, E31): a fresh page shows
// what is still unwatched, and a change sticks — across channels, reloads and devices.
const [show, setShow] = usePref<string>("channelShow", "unwatched", {
localKey: LS.channelShow,
parse: (v) => (typeof v === "string" && SHOW_IDS.includes(v) ? v : "unwatched"),
});
const [ownFilters, setOwnFilters] = useState<FeedFilters>(() => ({
tags: [],
tagModes: DEFAULT_TAG_MODES,
q: "",
@@ -273,9 +285,14 @@ export default function ChannelPage({
includeNormal: globalFilters.includeNormal,
includeShorts: globalFilters.includeShorts,
includeLive: globalFilters.includeLive,
show: "all",
show: "unwatched",
channelIds: [channelId],
}));
const filters: FeedFilters = { ...ownFilters, show };
const setFilters = (next: FeedFilters) => {
if (next.show !== show) setShow(next.show);
setOwnFilters(next);
};
const name = ch?.title ?? initialName ?? channelId;
const ytUrl = channelYouTubeUrl(channelId, ch?.handle);
@@ -284,7 +301,7 @@ export default function ChannelPage({
const metaParts = [
ch?.handle ? `@${ch.handle.replace(/^@/, "")}` : null,
ch?.subscriber_count != null
? t("channel.subscribers", { formatted: formatViews(ch.subscriber_count) })
? t("channel.subscribers", { formatted: formatCompact(ch.subscriber_count, lang) })
: null,
// Show "stored / total" when we only hold a subset of a subscribed channel's uploads, so the
// gap between the browsable videos and YouTube's raw count is honest rather than looking like
@@ -295,7 +312,7 @@ export default function ChannelPage({
: t("channel.videoCount", { count: ch.video_count })
: null,
ch?.total_view_count != null
? t("channel.totalViews", { formatted: formatViews(ch.total_view_count) })
? t("channel.totalViews", { formatted: formatCompact(ch.total_view_count, lang) })
: null,
// Per-type breakdown (R14 moved this off the manager table's N/S/L column). Only meaningful
// once we actually hold some of the channel's videos.
@@ -308,251 +325,292 @@ export default function ChannelPage({
: null,
joined ? t("channel.joined", { date: joined }) : null,
].filter(Boolean) as string[];
// The page's name and its state, shown in the identity block or, folded, in the tab row.
const heading = (
<PageHeading
data-testid="channel-title"
className={`${folded ? "text-base" : "text-xl"} font-semibold truncate`}
>
{name}
</PageHeading>
);
const stateBadge = ch?.blocked ? (
<span className="shrink-0 text-xs px-2 py-0.5 rounded-full bg-danger/15 text-danger">
{t("channel.blockedBadge")}
</span>
) : (
ch?.explored &&
!ch?.subscribed && (
<span className="shrink-0 text-xs px-2 py-0.5 rounded-full bg-warn/15 text-warn">
{t("channel.exploringBadge")}
</span>
)
);
// The channel page owns no scroller of its own (PageShell provides the one app scroller). Its
// whole chrome is fixed — wrapped in <PageToolbar> it portals into the shell's fixed band, so the
// banner/identity/tabs/toolbar stay put and only the video grid scrolls.
// tabs/toolbar stay put and only the video grid scrolls; the banner/identity fold away meanwhile.
const bannerSrc = ch?.banner_url
? `${ch.banner_url}=w1707${bannerAttempt ? `?r=${bannerAttempt}` : ""}`
: null;
return (
<>
<PageToolbar>
{/* Banner + back — OPTION C: inset, rounded card (not full-bleed) */}
<div className="relative px-4 sm:px-6 pt-3">
{bannerSrc && bannerAttempt <= MAX_BANNER_RETRIES ? (
// Match YouTube's banner: the stored bannerExternalUrl is the full 16:9 template at a
// low default size, so request a crisp wide version (=w1707) and object-cover the
// desktop "safe area" — the centre 2560×423 (~6:1) band.
<div
className="w-full overflow-hidden rounded-2xl bg-surface"
style={{ aspectRatio: "2560 / 423", maxHeight: "150px" }}
>
<img
// key forces a fresh <img> per attempt; the ?r= cache-buster on a retry bypasses the
// browser's negative cache (googleusercontent accepts an extra query param).
key={bannerAttempt}
src={bannerSrc}
referrerPolicy="no-referrer"
alt=""
className={`w-full h-full object-cover object-center ${
bannerLoaded === bannerSrc ? "" : "opacity-0"
}`}
onLoad={() => setBannerLoaded(bannerSrc)}
onError={() => {
if (bannerAttempt <= MAX_BANNER_RETRIES) {
window.setTimeout(
() => setBannerAttempt((a) => a + 1),
500 * (bannerAttempt + 1),
);
}
}}
/>
</div>
) : (
// No banner (or a dead one): a neutral bar the SAME height as a real banner, so the
// overlapping avatar + the Back button have the same room and don't collide (a short bar
// let the -mt-8 avatar ride up into the Back button).
<div
className="w-full rounded-2xl bg-surface"
style={{ aspectRatio: "2560 / 423", maxHeight: "150px" }}
/>
)}
<Button
variant="media"
data-testid="channel-back"
icon={<ArrowLeft />}
onClick={closeChannel}
className="absolute top-5 left-7 sm:left-9 z-base backdrop-blur-xs"
>
{t("channel.back")}
</Button>
</div>
<div className="px-4 sm:px-6">
{/* Identity + actions. relative+z-base so the avatar that overlaps up into the banner
paints ABOVE it (the banner's positioned container would otherwise cover it). */}
<div className="relative z-base flex items-end gap-4 -mt-8">
<Avatar
src={ch?.thumbnail_url ?? null}
fallback={name}
className="w-20 h-20 rounded-full border-4 border-bg shrink-0 bg-bg"
/>
<div className="flex-1 min-w-0 pb-1">
<div className="flex items-center gap-2 flex-wrap">
<PageHeading data-testid="channel-title" className="text-xl font-semibold truncate">
{name}
</PageHeading>
{ch?.blocked ? (
<span className="text-xs px-2 py-0.5 rounded-full bg-danger/15 text-danger">
{t("channel.blockedBadge")}
</span>
<div className="relative">
{!folded && (
<div ref={foldable}>
{/* Banner — OPTION C: inset, rounded card (not full-bleed) */}
<div className="px-4 sm:px-6 pt-3">
{bannerSrc && bannerAttempt <= MAX_BANNER_RETRIES ? (
// Match YouTube's banner: the stored bannerExternalUrl is the full 16:9 template at a
// low default size, so request a crisp wide version (=w1707) and object-cover the
// desktop "safe area" — the centre 2560×423 (~6:1) band.
<div
className="w-full overflow-hidden rounded-2xl bg-surface"
style={{ aspectRatio: "2560 / 423", maxHeight: "150px" }}
>
<img
// key forces a fresh <img> per attempt; the ?r= cache-buster on a retry bypasses the
// browser's negative cache (googleusercontent accepts an extra query param).
key={bannerAttempt}
src={bannerSrc}
referrerPolicy="no-referrer"
alt=""
className={`w-full h-full object-cover object-center ${
bannerLoaded === bannerSrc ? "" : "opacity-0"
}`}
onLoad={() => setBannerLoaded(bannerSrc)}
onError={() => {
if (bannerAttempt <= MAX_BANNER_RETRIES) {
window.setTimeout(
() => setBannerAttempt((a) => a + 1),
500 * (bannerAttempt + 1),
);
}
}}
/>
</div>
) : (
ch?.explored &&
!ch?.subscribed && (
<span className="text-xs px-2 py-0.5 rounded-full bg-warn/15 text-warn">
{t("channel.exploringBadge")}
</span>
)
// No banner (or a dead one): a neutral bar the SAME height as a real banner, so the
// overlapping avatar + the Back button have the same room and don't collide (a short bar
// let the -mt-8 avatar ride up into the Back button).
<div
className="w-full rounded-2xl bg-surface"
style={{ aspectRatio: "2560 / 423", maxHeight: "150px" }}
/>
)}
</div>
<div className="text-sm text-muted mt-0.5 flex flex-wrap gap-x-1.5 gap-y-0.5">
{metaParts.map((part, i) => (
<span key={i}>
{i > 0 && <span className="mr-1.5">·</span>}
{part}
</span>
))}
</div>
</div>
</div>
{exploring && (
<div className="flex items-center gap-2 text-xs text-muted mt-2">
<Loader2 className="w-3.5 h-3.5 animate-spin" />
{t("channel.loadingVideos")}
<div className="px-4 sm:px-6">
{/* Identity. relative+z-base so the avatar that overlaps up into the banner paints
ABOVE it (the banner's positioned container would otherwise cover it). */}
<div className="relative z-base flex items-end gap-4 -mt-8">
<Avatar
src={ch?.thumbnail_url ?? null}
fallback={name}
className="w-20 h-20 rounded-full border-4 border-bg shrink-0 bg-bg"
/>
<div className="flex-1 min-w-0 pb-1">
<div className="flex items-center gap-2 flex-wrap">
{heading}
{stateBadge}
</div>
<div className="text-sm text-muted mt-0.5 flex flex-wrap gap-x-1.5 gap-y-0.5">
{metaParts.map((part, i) => (
<span key={i}>
{i > 0 && <span className="mr-1.5">·</span>}
{part}
</span>
))}
</div>
</div>
</div>
</div>
</div>
)}
{/* Tabs — the channel actions ride along here rather than off in the header's top-right
<div className="px-4 sm:px-6">
{exploring && (
<div className="flex items-center gap-2 text-xs text-muted mt-2">
<Loader2 className="w-3.5 h-3.5 animate-spin" />
{t("channel.loadingVideos")}
</div>
)}
{/* Tabs — the channel actions ride along here rather than off in the header's top-right
corner: there are only ever two tabs, so there's room, and it keeps the things you click
within a few pixels of each other instead of across the page. */}
<div className="flex items-center gap-2 mt-3 border-b border-border pb-2">
<Tabs
idBase={panel.idBase}
className="flex gap-1"
active={tab}
onChange={(id) => setTab(id as "videos" | "about")}
tabs={[
{ id: "videos", label: t("channel.tabVideos") },
{ id: "about", label: t("channel.tabAbout") },
]}
/>
<div className="ml-2 flex items-center gap-2">
<IconLink
href={ytUrl}
target="_blank"
rel="noreferrer"
label={t("channels.row.openOnYouTube")}
size="md"
variant="outline"
{/* Wraps rather than overflows: at a tablet width (1024) the actions + tags ran past the edge and
the whole page scrolled sideways (E31 S1). The tag lane takes what is left and folds into "+N". */}
<div
className={`flex flex-wrap items-center gap-2 border-b border-border pb-2 ${folded ? "pt-3" : "mt-3"}`}
>
{/* One Back button in both states, so a focused Back keeps its focus when the header folds:
over the banner's corner (absolute to the chrome) while unfolded, first in this row folded. */}
<Button
variant={folded ? "outline" : "media"}
data-testid="channel-back"
icon={<ArrowLeft />}
onClick={closeChannel}
className={folded ? "" : "absolute top-5 left-7 sm:left-9 z-base backdrop-blur-xs"}
>
<ExternalLink />
</IconLink>
{ch?.subscribed && (
<IconButton
label={t("channel.showInFeed")}
size="md"
variant="outline"
onClick={onShowInFeed}
{t("channel.back")}
</Button>
{folded && (
<div
className="flex min-w-0 max-w-[40%] items-center gap-2"
data-testid="channel-folded"
>
<SlidersHorizontal className="w-4 h-4" />
</IconButton>
)}
{ch?.subscribed && !me.is_demo && (
<div className="inline-flex items-center gap-1.5 h-9 px-2.5 rounded-lg border border-border">
<span className="text-xs text-muted">{t("channels.cols.priority")}</span>
<PriorityStepper
priority={ch.priority}
onPriority={(d) => bumpPriority(channelId, d)}
onSet={(v) => setPriority(channelId, v)}
size="md"
hint={t("channels.row.priorityHint")}
<Avatar
src={ch?.thumbnail_url ?? null}
fallback={name}
className="w-8 h-8 rounded-full shrink-0"
/>
{heading}
{stateBadge}
</div>
)}
{/* Only the recent subset is stored → offer to pull the channel's full back-catalogue.
<Tabs
idBase={panel.idBase}
className="flex gap-1"
active={tab}
onChange={(id) => setTab(id as "videos" | "about")}
tabs={[
{ id: "videos", label: t("channel.tabVideos") },
{ id: "about", label: t("channel.tabAbout") },
]}
/>
{/* Unfolded, the actions share the tabs' line and the tag lane gives way ("+N"). Folded, Back
and the name take that room, so at a tablet width the actions wrap onto a line of their
own rather than push the page sideways (min-w-0 let them overflow: 1072 px at 1024). */}
<div className={`ml-2 flex flex-1 items-center gap-2 ${folded ? "" : "min-w-0"}`}>
<IconLink
href={ytUrl}
target="_blank"
rel="noreferrer"
label={t("channels.row.openOnYouTube")}
size="md"
variant="outline"
>
<ExternalLink />
</IconLink>
{ch?.subscribed && (
<IconButton
label={t("channel.showInFeed")}
size="md"
variant="outline"
onClick={onShowInFeed}
>
<SlidersHorizontal className="w-4 h-4" />
</IconButton>
)}
{ch?.subscribed && !me.is_demo && (
<div className="inline-flex items-center gap-1.5 h-9 px-2.5 rounded-lg border border-border">
<span className="text-xs text-muted">{t("channels.cols.priority")}</span>
<PriorityStepper
priority={ch.priority}
onPriority={(d) => bumpPriority(channelId, d)}
onSet={(v) => setPriority(channelId, v)}
size="md"
hint={t("channels.row.priorityHint")}
/>
</div>
)}
{/* Only the recent subset is stored → offer to pull the channel's full back-catalogue.
Once queued (deep_requested) it shows as loading; hidden once the backfill is done.
While loading, the label carries the live stored/total count (the detail poll refreshes
it) so a large channel's paced tail reads as visible progress rather than a stuck
spinner. */}
{ch?.subscribed && !ch.backfill_done && !me.is_demo && (
<Button
variant="outline"
onClick={() => !ch.deep_requested && deepBackfill.mutate()}
disabled={ch.deep_requested || deepBackfill.isPending}
loading={ch.deep_requested}
icon={<History className="w-4 h-4" />}
title={
ch.deep_requested
? t("channel.fullHistoryLoadingHint")
: t("channel.fullHistory")
}
>
<span className="hidden sm:inline whitespace-nowrap">
{!ch.deep_requested
? t("channel.fullHistory")
: ch.video_count != null && ch.stored_videos < ch.video_count
? t("channel.fullHistoryProgress", {
stored: ch.stored_videos,
total: ch.video_count,
})
: t("channel.fullHistoryLoading")}
</span>
</Button>
)}
{!me.is_demo && (
<IconButton
// Busy blocks the click, not the focus (FocusSafeButton). Nor does a busy toggle
// light up on hover, as if a click would do something.
size="md"
variant={ch?.blocked ? "danger-soft" : "outline-danger"}
onClick={() => block.mutate()}
loading={block.isPending}
label={ch?.blocked ? t("channel.unblock") : t("channel.block")}
>
<Ban />
</IconButton>
)}
{/* Hide-from-feed toggle — subscription-scoped, so offered only on a followed channel
{ch?.subscribed && !ch.backfill_done && !me.is_demo && (
<Button
variant="outline"
onClick={() => !ch.deep_requested && deepBackfill.mutate()}
disabled={ch.deep_requested || deepBackfill.isPending}
loading={ch.deep_requested}
icon={<History className="w-4 h-4" />}
title={
ch.deep_requested
? t("channel.fullHistoryLoadingHint")
: t("channel.fullHistory")
}
>
<span className="hidden sm:inline whitespace-nowrap">
{!ch.deep_requested
? t("channel.fullHistory")
: ch.video_count != null && ch.stored_videos < ch.video_count
? t("channel.fullHistoryProgress", {
stored: ch.stored_videos,
total: ch.video_count,
})
: t("channel.fullHistoryLoading")}
</span>
</Button>
)}
{!me.is_demo && (
<IconButton
// Busy blocks the click, not the focus (FocusSafeButton). Nor does a busy toggle
// light up on hover, as if a click would do something.
size="md"
variant={ch?.blocked ? "danger-soft" : "outline-danger"}
onClick={() => block.mutate()}
loading={block.isPending}
label={ch?.blocked ? t("channel.unblock") : t("channel.block")}
>
<Ban />
</IconButton>
)}
{/* Hide-from-feed toggle — subscription-scoped, so offered only on a followed channel
(mirrors the manager's Eye/EyeOff). Keeps the subscription; only drops its videos
from the feed. EyeOff = currently hidden (click to unhide). */}
{ch?.subscribed && !me.is_demo && (
<IconButton
size="md"
variant={ch.hidden ? "outline-accent" : "outline"}
onClick={() => toggleHidden.mutate()}
loading={toggleHidden.isPending}
label={ch.hidden ? t("channel.unhide") : t("channel.hide")}
title={ch.hidden ? t("channel.hiddenHint") : t("channel.hideHint")}
>
{ch.hidden ? <EyeOff /> : <Eye />}
</IconButton>
)}
{!me.is_demo &&
!ch?.blocked &&
(ch?.subscribed ? (
<Button
variant="outline"
onClick={onUnsubscribe}
loading={unsubscribe.isPending}
title={t("channel.unsubscribe")}
icon={<Check className="w-4 h-4" />}
{ch?.subscribed && !me.is_demo && (
<IconButton
size="md"
variant={ch.hidden ? "outline-accent" : "outline"}
onClick={() => toggleHidden.mutate()}
loading={toggleHidden.isPending}
label={ch.hidden ? t("channel.unhide") : t("channel.hide")}
title={ch.hidden ? t("channel.hiddenHint") : t("channel.hideHint")}
>
{t("channel.subscribedState")}
</Button>
) : (
<Button
variant="outline"
onClick={() => subscribe.mutate()}
loading={subscribe.isPending}
icon={<Plus className="w-4 h-4" />}
>
{t("channel.subscribe")}
</Button>
))}
{/* Tagging is a subscription-scoped action (attach_tag → _user_subscription 404s
{ch.hidden ? <EyeOff /> : <Eye />}
</IconButton>
)}
{!me.is_demo &&
!ch?.blocked &&
(ch?.subscribed ? (
<Button
variant="outline"
onClick={onUnsubscribe}
loading={unsubscribe.isPending}
title={t("channel.unsubscribe")}
icon={<Check className="w-4 h-4" />}
>
{t("channel.subscribedState")}
</Button>
) : (
<Button
variant="outline"
onClick={() => subscribe.mutate()}
loading={subscribe.isPending}
icon={<Plus className="w-4 h-4" />}
>
{t("channel.subscribe")}
</Button>
))}
{/* Tagging is a subscription-scoped action (attach_tag → _user_subscription 404s
otherwise), and personal tags are for your subscriptions — so only offer it on a
channel you actually follow. */}
{ch?.subscribed && !me.is_demo && (
<ChannelTagsControl
userTags={userTags}
attachedIds={ch.tag_ids}
onToggle={(tagId) => toggleTag(channelId, tagId, ch.tag_ids.includes(tagId))}
onCreate={(name) => createAndAttachTag(channelId, name)}
// The header row has free width — show tags inline instead of collapsing to "+N".
bounded={false}
/>
)}
{ch?.subscribed && !me.is_demo && (
<div className="min-w-0 flex-1">
<ChannelTagsControl
userTags={userTags}
attachedIds={ch.tag_ids}
onToggle={(tagId) => toggleTag(channelId, tagId, ch.tag_ids.includes(tagId))}
onCreate={(name) => createAndAttachTag(channelId, name)}
// The header row's leftover width: tags inline, "+N" only when they do not fit.
bounded={false}
/>
</div>
)}
</div>
</div>
</div>
</div>
@@ -0,0 +1,109 @@
// @vitest-environment jsdom
//
// E31 (the owner): a freshly opened channel page shows what is still unwatched, and a watch state the
// viewer picks there sticks — a reload (a fresh mount, seeded from the stored preference) keeps it.
import type { ReactNode } from "react";
import { describe, it, expect, vi, beforeAll, afterEach } from "vitest";
import { render, cleanup, act } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { FeedFilters } from "../lib/api";
vi.mock("react-i18next", async (importOriginal) => ({
...(await importOriginal<typeof import("react-i18next")>()),
useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }),
}));
vi.mock("./navigationContext", () => ({
useNavigationActions: () => ({ closeChannel: vi.fn(), openChannel: vi.fn() }),
}));
vi.mock("./confirmContext", () => ({ useConfirm: () => async () => true }));
vi.mock("./feedFiltersContext", () => ({
useFeedFilters: () => ({ includeNormal: true, includeShorts: false, includeLive: false }),
}));
const feed = vi.hoisted(() => ({
last: null as null | { filters: FeedFilters; setFilters: (f: FeedFilters) => void },
}));
vi.mock("./Feed", () => ({
default: (p: { filters: FeedFilters; setFilters: (f: FeedFilters) => void }) => {
feed.last = p;
return null;
},
}));
vi.mock("./ChannelAbout", () => ({ default: () => null }));
vi.mock("./ui/PageShell", async (importOriginal) => ({
...(await importOriginal<typeof import("./ui/PageShell")>()),
PageToolbar: ({ children }: { children?: ReactNode }) => <>{children}</>,
}));
import ChannelPage from "./ChannelPage";
import { api, type ChannelDetail, type Me } from "../lib/api";
import { qk } from "../lib/queryKeys";
beforeAll(() => {
class Stub {
observe() {}
unobserve() {}
disconnect() {}
takeRecords() {
return [];
}
}
globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver;
globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver;
});
afterEach(() => {
cleanup();
vi.restoreAllMocks();
feed.last = null;
});
const CH = "UCqoAEDirJPjEUFcF2FklnBA";
function mount(preferences: Record<string, unknown> = {}) {
vi.spyOn(api, "channelDetail").mockResolvedValue({
id: CH,
title: "StarTalk",
subscribed: true,
stored_videos: 10,
} as unknown as ChannelDetail);
vi.spyOn(api, "tags").mockResolvedValue([] as never);
const save = vi.spyOn(api, "savePrefs").mockResolvedValue({} as never);
const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
qc.setQueryData(qk.me(), { id: 1, preferences } as unknown as Me);
render(
<QueryClientProvider client={qc}>
<ChannelPage
channelId={CH}
initialName="StarTalk"
me={{ id: 1, can_write: true, can_read: true } as unknown as Me}
view="cards"
setView={() => {}}
onShowInFeed={() => {}}
/>
</QueryClientProvider>,
);
return save;
}
describe("ChannelPage — the watch state", () => {
it("shows only the unwatched videos on a freshly opened page", () => {
mount();
expect(feed.last?.filters.show).toBe("unwatched");
});
it("stores a picked watch state", () => {
const save = mount();
act(() => feed.last!.setFilters({ ...feed.last!.filters, show: "all" }));
expect(feed.last?.filters.show).toBe("all");
expect(save).toHaveBeenCalledWith({ channelShow: "all" });
});
it("keeps the stored watch state after a reload", () => {
mount({ channelShow: "watched" });
expect(feed.last?.filters.show).toBe("watched");
});
it("falls back to unwatched for a stored value it does not know", () => {
mount({ channelShow: "bogus" });
expect(feed.last?.filters.show).toBe("unwatched");
});
});
@@ -14,12 +14,16 @@ import { FloatingTree } from "@floating-ui/react";
// (i18n/index.ts calls i18n.use(initReactI18next) at import time).
vi.mock("react-i18next", async (importOriginal) => {
const actual = await importOriginal<typeof import("react-i18next")>();
return { ...actual, useTranslation: () => ({ t: (key: string) => key }) };
return {
...actual,
useTranslation: () => ({ i18n: { language: "en" }, t: (key: string) => key }),
};
});
import ChannelStatsPopover from "./ChannelStatsPopover";
import { type MyStatus } from "../lib/api";
import { formatEta, formatNumber } from "../lib/format";
import type { TFunction } from "i18next";
beforeAll(() => {
class Stub {
@@ -74,7 +78,7 @@ describe("ChannelStatsPopover", () => {
expect(screen.getByText("channels.stats.recentSynced")).toBeTruthy();
expect(screen.getByText("3/10")).toBeTruthy(); // recent-synced ratio
expect(screen.getByText("1/4")).toBeTruthy(); // full-history ratio
expect(screen.getByText(formatNumber(1234))).toBeTruthy(); // my_videos, locale-agnostic
expect(screen.getByText(formatNumber(1234, "en"))).toBeTruthy(); // my_videos, locale-agnostic
});
it("shows the 'left' ETA row only while a backfill is pending, with the formatted ETA value", () => {
@@ -86,7 +90,9 @@ describe("ChannelStatsPopover", () => {
// The value is the FORMATTED ETA (via formatEta), not raw seconds — asserting it guards the
// formatEta wiring, which is untested anywhere else. Using the same formatEta keeps this
// locale/impl-agnostic (expected === what the component renders).
expect(screen.getByText(formatEta(120))).toBeTruthy();
expect(
screen.getByText(formatEta(((k: string) => k) as unknown as TFunction, 120)),
).toBeTruthy();
expect(screen.queryByText("120")).toBeNull(); // never the raw seconds
});
@@ -22,7 +22,8 @@ import { SURFACE } from "./ui/surfaces";
// with a leading icon. Kept in the manager's own toolbar (the header's SyncStatus is global; these
// are channel-catalog figures).
export default function ChannelStatsPopover({ status }: { status: MyStatus }) {
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
const [open, setOpen] = useState(false);
const pop = useAnchoredPopover({
open,
@@ -58,7 +59,7 @@ export default function ChannelStatsPopover({ status }: { status: MyStatus }) {
{
icon: Clock,
label: t("channels.stats.left"),
value: formatEta(status.deep_eta_seconds),
value: formatEta(t, status.deep_eta_seconds),
hint: t("channels.stats.leftHint", { count: status.deep_pending_count }),
},
]
@@ -66,13 +67,13 @@ export default function ChannelStatsPopover({ status }: { status: MyStatus }) {
{
icon: Film,
label: t("channels.stats.myVideos"),
value: formatNumber(status.my_videos),
value: formatNumber(status.my_videos, lang),
hint: t("channels.stats.myVideosHint"),
},
{
icon: Zap,
label: t("channels.stats.quotaLeft"),
value: formatNumber(status.quota_remaining_today),
value: formatNumber(status.quota_remaining_today, lang),
hint: t("channels.stats.quotaLeftHint"),
},
];
+11 -4
View File
@@ -54,6 +54,7 @@ import { useNavigationActions } from "./navigationContext";
import { useFeedFilters, useFeedFiltersActions } from "./feedFiltersContext";
import { useWizard } from "./wizardContext";
import { useMe } from "../lib/useMe";
import { useTags } from "../lib/useTags";
export type ChannelsView = "subscribed" | "discovery" | "hidden" | "blocked";
@@ -106,7 +107,8 @@ export default function Channels() {
const { openChannel: onViewChannel, setPage } = useNavigationActions();
const feedFilters = useFeedFilters();
const { setFilters: setFeedFilters } = useFeedFiltersActions();
const { t } = useTranslation();
const { t, i18n } = useTranslation();
const lang = i18n.language;
const qc = useQueryClient();
// Clicking a tag chip filters the feed by that tag and jumps there. Internalised (was an App
@@ -118,7 +120,7 @@ export default function Channels() {
};
const channelsQuery = useQuery({ queryKey: qk.channels(), queryFn: api.channels });
const tagsQuery = useQuery({ queryKey: qk.tags(), queryFn: api.tags });
const tagsQuery = useTags();
const statusQuery = useQuery({ queryKey: qk.myStatus(), queryFn: api.myStatus });
const blockedQuery = useQuery({ queryKey: qk.blockedChannels(), queryFn: api.blockedChannels });
const [tagManagerOpen, setTagManagerOpen] = useState(false);
@@ -169,7 +171,10 @@ export default function Channels() {
const listRef = useRef<HTMLDivElement>(null);
const pickPriority = (c: ManagedChannel, next: number) => {
setPriority(c.id, next);
qc.setQueryData<ManagedChannel[]>(qk.channels(), (old) => old && placeByPriority(old, c.id));
qc.setQueryData<ManagedChannel[]>(
qk.channels(),
(old) => old && placeByPriority(old, c.id, me.preferences),
);
setFollow((f) => ({ key: c.id, seq: (f?.seq ?? 0) + 1 }));
};
@@ -363,6 +368,7 @@ export default function Channels() {
const columns = buildChannelColumns({
t,
lang: i18n.language,
prioOptions,
userTags,
bumpPriority,
@@ -470,7 +476,7 @@ export default function Channels() {
// Cards have no column headers, so apply the shared column filters (sync + any funnel set in table
// mode) here via the exported predicate — the same filterRows DataTable runs internally — then sort.
const sortedChannels = isCards
? sortRows(filterRows(syncedChannels, columns, filters), columns, sort)
? sortRows(filterRows(syncedChannels, columns, filters), columns, sort, lang)
: syncedChannels;
const {
paged: pagedCards,
@@ -748,6 +754,7 @@ export default function Channels() {
keyboardNav
reveal={follow}
surfacedFilters={SURFACED_FILTERS}
narrowedBy={q}
/>
) : (
<>
@@ -72,6 +72,7 @@ vi.mock("../lib/api", async (importOriginal) => {
choice("log_level", LEVELS, "INFO"),
],
},
sections: [{ id: "s", groups: ["g"] }],
secrets_manageable: true,
}),
setConfig: (key: string, value: unknown) => setConfig(key, value),
@@ -46,6 +46,7 @@ vi.mock("../lib/api", async (importOriginal) => {
groups: {
g: [text("site_name"), { ...text("signups_open"), type: "bool", value: "false" }],
},
sections: [{ id: "s", groups: ["g"] }],
secrets_manageable: true,
}),
},
@@ -0,0 +1,92 @@
// @vitest-environment jsdom
//
// E31 (the owner's pick): Configuration's sixteen group tabs wrapped to three rows at 133% zoom, so they
// are a side list under four headings, as Settings has. The registry owns the order: the groups follow
// its sections, not the order the groups object happens to arrive in.
import { afterEach, describe, expect, it, vi } from "vitest";
import { cleanup, fireEvent, render, screen } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { ConfigItem } from "../lib/api";
vi.mock("react-i18next", async (importOriginal) => {
const actual = await importOriginal<typeof import("react-i18next")>();
return {
...actual,
useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }),
};
});
const text = (key: string, group: string): ConfigItem => ({
key,
type: "str",
group,
secret: false,
min: null,
max: null,
is_set: false,
allow_empty: true,
choices: [],
value: "",
default: "",
});
vi.mock("../lib/api", async (importOriginal) => {
const actual = await importOriginal<typeof import("../lib/api")>();
return {
...actual,
api: {
...actual.api,
adminConfig: () =>
Promise.resolve({
groups: { b: [text("kb", "b")], c: [text("kc", "c")], a: [text("ka", "a")] },
sections: [
{ id: "one", groups: ["a", "b"] },
{ id: "two", groups: ["c"] },
],
secrets_manageable: true,
}),
},
};
});
vi.mock("../lib/useMe", () => ({ useMe: () => ({ email_configured: true }) }));
import ConfigPanel from "./ConfigPanel";
afterEach(() => {
cleanup();
localStorage.clear();
});
const renderPanel = () =>
render(
<QueryClientProvider client={new QueryClient()}>
<ConfigPanel />
</QueryClientProvider>,
);
describe("Configuration sections (E31)", () => {
it("lists the groups in the registry's sections, each heading once above its run", async () => {
renderPanel();
const list = await screen.findByRole("tablist");
expect(list.getAttribute("aria-orientation")).toBe("vertical");
// The list's own lines, top to bottom: headings and tabs.
const lines = Array.from(list.children).map((el) =>
el.getAttribute("role") === "tab" ? `tab:${el.textContent}` : `head:${el.textContent}`,
);
expect(lines).toEqual([
"head:config.sections.one",
"tab:config.groups.a",
"tab:config.groups.b",
"head:config.sections.two",
"tab:config.groups.c",
]);
});
it("opens on the first group and switches to the one picked", async () => {
renderPanel();
expect(await screen.findByText("config.fields.ka.label")).toBeTruthy();
fireEvent.click(screen.getByRole("tab", { name: "config.groups.c" }));
expect(screen.getByText("config.fields.kc.label")).toBeTruthy();
expect(screen.queryByText("config.fields.ka.label")).toBeNull();
});
});
+129 -124
View File
@@ -11,12 +11,11 @@ import { useMe } from "../lib/useMe";
import Tabs from "./ui/Tabs";
import { useTabPanel } from "./ui/tabPanel";
import { LoadingState, StateMessage } from "./ui/QueryState";
import { PageToolbar } from "./ui/PageShell";
import { Switch } from "./ui/form";
import SegmentedToggle from "./ui/SegmentedToggle";
import FilterSelect from "./ui/FilterSelect";
import { DraftSaveBar, type SaveState } from "./ui/DraftSaveBar";
import { LS, useAccountPersistedState } from "../lib/storage";
import { LS, useTabPersistedState } from "../lib/storage";
import { notifyActionError } from "../lib/actionErrors";
import { Button } from "./ui/Button";
import { CheckRow } from "./ui/NativeControls";
@@ -57,7 +56,7 @@ export default function ConfigPanel() {
const [reseedToken, setReseedToken] = useState(0);
// One tab per config group (persisted). Called before the loading guard so hook order is
// stable; the active id is clamped to a real group at render time once data has loaded.
const [tab, setTab] = useAccountPersistedState(LS.configTab);
const [tab, setTab] = useTabPersistedState(LS.configTab);
// A stable signature of the config SHAPE — changes only when keys are added/removed, not when an
// unchanged refetch produces a new baseline object. Re-seed the draft on a shape change (initial
@@ -143,7 +142,7 @@ export default function ConfigPanel() {
notifyActionError(e, t("config.plexTestFailed"));
},
});
// Bug reports' "Check the connection" (E16): the repository answers, takes issues, is private.
// Bug reports' "Test connection" (E16): the repository answers, takes issues, is private.
const testIssues = useMutation({
mutationFn: () => api.testIssues(),
onSuccess: (r) =>
@@ -176,146 +175,152 @@ export default function ConfigPanel() {
return <LoadingState label={t("config.loading")} />;
}
const groupKeys = Object.keys(data.groups);
// The groups in the registry's sections, in order (E31: sixteen tabs wrapped to three rows at 133%
// zoom, so they are a side list under four headings, as Settings has).
const groupKeys = data.sections.flatMap((s) => s.groups);
// Clamp the persisted tab to a real group (the registry can change between sessions). The
// tab pill carries a dot when that group has unsaved edits, so a draft in a hidden tab is
// never silently lost behind the global Save bar.
const activeGroup = groupKeys.includes(tab) ? tab : groupKeys[0];
const dirtyByGroup = new Set(dirtyKeys.map((k) => byKey[k]?.group).filter(Boolean));
const groupTabs = groupKeys.map((g) => ({
id: g,
label: `${t(`config.groups.${g}`, g)}${dirtyByGroup.has(g) ? " •" : ""}`,
}));
const groupTabs = data.sections.flatMap((s) =>
s.groups.map((g) => ({
id: g,
group: t(`config.sections.${s.id}`, s.id),
label: `${t(`config.groups.${g}`, g)}${dirtyByGroup.has(g) ? " •" : ""}`,
testId: `config-tab-${g}`,
})),
);
return (
<>
{/* The tab bar is the page's band (B4); the intro scrolls with the fields — in the band it
took three lines of a short window for text read once. */}
<PageToolbar>
<div className="px-4 pt-3 max-w-3xl w-full mx-auto">
<div className="px-4 pb-24 pt-3 max-w-4xl w-full mx-auto">
<p className="text-xs text-muted mb-3 leading-relaxed">{t("config.intro")}</p>
{/* The groups as a side list under their headings, the fields beside it: Settings' own
layout, one pattern for both settings pages (E31). */}
<div className={`${SURFACE.panel} mb-4 flex flex-col overflow-hidden sm:flex-row`}>
<Tabs
idBase={panel.idBase}
orientation="vertical"
className="flex gap-1 overflow-x-auto border-b border-border/60 p-2 sm:w-52 sm:shrink-0 sm:flex-col sm:border-b-0 sm:border-r"
tabs={groupTabs}
active={activeGroup ?? ""}
onChange={setTab}
/>
</div>
</PageToolbar>
<div className="px-4 pb-24 pt-1 max-w-3xl w-full mx-auto">
<p className="text-xs text-muted mb-3 leading-relaxed">{t("config.intro")}</p>
{activeGroup && (
<div {...panel.panelProps(activeGroup ?? "")} className={`${SURFACE.panel} p-4 mb-4`}>
<div className="divide-y divide-border/60">
{(data.groups[activeGroup] ?? []).map((item) => (
<Field
key={item.key}
item={item}
value={draft[item.key] ?? ""}
secretsManageable={data.secrets_manageable}
pendingSecretReset={!!secretReset[item.key]}
onChange={(v) => setDraft((d) => ({ ...d, [item.key]: v }))}
onResetField={() => {
if (item.secret) setSecretReset((s) => ({ ...s, [item.key]: !s[item.key] }));
else setDraft((d) => ({ ...d, [item.key]: "" }));
}}
/>
))}
</div>
{activeGroup && (
<div {...panel.panelProps(activeGroup ?? "")} className="min-w-0 flex-1 p-4">
<div className="divide-y divide-border/60">
{(data.groups[activeGroup] ?? []).map((item) => (
<Field
key={item.key}
item={item}
value={draft[item.key] ?? ""}
secretsManageable={data.secrets_manageable}
pendingSecretReset={!!secretReset[item.key]}
onChange={(v) => setDraft((d) => ({ ...d, [item.key]: v }))}
onResetField={() => {
if (item.secret) setSecretReset((s) => ({ ...s, [item.key]: !s[item.key] }));
else setDraft((d) => ({ ...d, [item.key]: "" }));
}}
/>
))}
</div>
{activeGroup === "email" && (
<div className="mt-3 pt-3 border-t border-border/60 space-y-3">
{/* The one place an operator can see that mail is off. Sending is fail-soft, so a
{activeGroup === "email" && (
<div className="mt-3 pt-3 border-t border-border/60 space-y-3">
{/* The one place an operator can see that mail is off. Sending is fail-soft, so a
half-filled config costs verification links, resets and access-request notices
with no error anywhere — say so next to the fields that fix it. */}
{!me.email_configured && <InlineNotice>{t("config.emailOff")}</InlineNotice>}
<Tooltip hint={t("config.testEmailHint")}>
<Button
variant="glass"
icon={<Send />}
loading={testEmail.isPending}
aria-disabled={dirty}
onClick={() => testEmail.mutate()}
>
{t("config.testEmail")}
</Button>
</Tooltip>
{dirty && <p className="text-2xs text-muted">{t("config.testDirty")}</p>}
</div>
)}
{!me.email_configured && <InlineNotice>{t("config.emailOff")}</InlineNotice>}
<Tooltip hint={t("config.testEmailHint")}>
<Button
variant="glass"
icon={<Send />}
loading={testEmail.isPending}
aria-disabled={dirty}
onClick={() => testEmail.mutate()}
>
{t("config.testEmail")}
</Button>
</Tooltip>
{dirty && <p className="text-2xs text-muted">{t("config.testDirty")}</p>}
</div>
)}
{activeGroup === "issues" && (
<div className="mt-3 pt-3 border-t border-border/60">
<Tooltip hint={t("config.issuesTestHint")}>
<Button
variant="glass"
icon={<Plug />}
loading={testIssues.isPending}
aria-disabled={dirty}
onClick={() => testIssues.mutate()}
>
{t("config.issuesTest")}
</Button>
</Tooltip>
{dirty && <p className="text-2xs text-muted mt-2">{t("config.testDirty")}</p>}
</div>
)}
{activeGroup === "issues" && (
<div className="mt-3 pt-3 border-t border-border/60">
<Tooltip hint={t("config.issuesTestHint")}>
<Button
variant="glass"
icon={<Plug />}
loading={testIssues.isPending}
aria-disabled={dirty}
onClick={() => testIssues.mutate()}
>
{t("config.issuesTest")}
</Button>
</Tooltip>
{dirty && <p className="text-2xs text-muted mt-2">{t("config.testDirty")}</p>}
</div>
)}
{activeGroup === "plex" && (
<div className="mt-3 pt-3 border-t border-border/60">
<Tooltip hint={t("config.plexTestHint")}>
<Button
variant="glass"
icon={<Plug />}
loading={testPlex.isPending}
aria-disabled={dirty}
onClick={() => testPlex.mutate()}
>
{t("config.plexTest")}
</Button>
</Tooltip>
{dirty && <p className="text-2xs text-muted mt-2">{t("config.testDirty")}</p>}
{plexResult && (
<div className="mt-3">
<p className="text-xs text-muted mb-2">
{t("config.plexConnected", {
name: plexResult.server_name,
version: plexResult.version ?? "",
})}
</p>
{plexResult.media_check?.checked &&
(plexResult.media_check.ok ? (
<p className="text-2xs text-success mb-2">{t("config.plexMediaOk")}</p>
) : (
<p className="text-2xs text-danger mb-2">
{t("config.plexMediaMissing", {
path: plexResult.media_check.local_path ?? "",
})}
</p>
))}
<p className="text-2xs text-muted mb-1.5">{t("config.plexPickLibraries")}</p>
<div className="flex flex-col gap-1.5">
{plexResult.sections.map((s) => (
<CheckRow
key={s.key}
checked={plexLibs.length === 0 || plexLibs.includes(s.key)}
onChange={() => togglePlexLib(s.key)}
className="text-sm"
>
<span>{s.title}</span>
<span className="text-2xs text-muted">
{s.type === "movie" ? t("config.plexMovies") : t("config.plexShows")}
{s.count != null ? ` · ${s.count}` : ""}
</span>
</CheckRow>
))}
{activeGroup === "plex" && (
<div className="mt-3 pt-3 border-t border-border/60">
<Tooltip hint={t("config.plexTestHint")}>
<Button
variant="glass"
icon={<Plug />}
loading={testPlex.isPending}
aria-disabled={dirty}
onClick={() => testPlex.mutate()}
>
{t("config.plexTest")}
</Button>
</Tooltip>
{dirty && <p className="text-2xs text-muted mt-2">{t("config.testDirty")}</p>}
{plexResult && (
<div className="mt-3">
<p className="text-xs text-muted mb-2">
{t("config.plexConnected", {
name: plexResult.server_name,
version: plexResult.version ?? "",
})}
</p>
{plexResult.media_check?.checked &&
(plexResult.media_check.ok ? (
<p className="text-2xs text-success mb-2">{t("config.plexMediaOk")}</p>
) : (
<p className="text-2xs text-danger mb-2">
{t("config.plexMediaMissing", {
path: plexResult.media_check.local_path ?? "",
})}
</p>
))}
<p className="text-2xs text-muted mb-1.5">{t("config.plexPickLibraries")}</p>
<div className="flex flex-col gap-1.5">
{plexResult.sections.map((s) => (
<CheckRow
key={s.key}
checked={plexLibs.length === 0 || plexLibs.includes(s.key)}
onChange={() => togglePlexLib(s.key)}
className="text-sm"
>
<span>{s.title}</span>
<span className="text-2xs text-muted">
{s.type === "movie" ? t("config.plexMovies") : t("config.plexShows")}
{s.count != null ? ` · ${s.count}` : ""}
</span>
</CheckRow>
))}
</div>
<p className="text-2xs text-muted mt-2">{t("config.plexPickHint")}</p>
</div>
<p className="text-2xs text-muted mt-2">{t("config.plexPickHint")}</p>
</div>
)}
</div>
)}
</div>
)}
)}
</div>
)}
</div>
)}
</div>
<DraftSaveBar
variant="floating"

Some files were not shown because too many files have changed in this diff Show More