Round-1 review fixes on the E3 branch:
- prune partial-fetch (correctness): `_paginate` now reports whether it reached
the server's reported `total`; `_prune_stale` prunes only on a COMPLETE
non-empty listing. A truncated fetch (a mid-sequence empty/short page before
total — a transient Plex hiccup) no longer deletes the rows it merely failed
to list. New test: a listing that claims 5 but returns 1 prunes nothing.
- `_prune_stale` (robustness): `stats.get("pruned", 0) + 1` — no KeyError when a
caller's stats dict omits the key.
- service transitions (conventions): pause/resume/redownload use the JobStatus
StrEnum instead of loose "queued"/"paused"/"error" literals, matching cancel.
- ffmpeg progress writer (correctness): a stuck-percent heartbeat now runs on its
OWN 1/interval budget, so it can't consume the progress-write slot and delay a
real percent change by up to an interval.
- backend/app/README.md (conventions): reword the transaction-ownership rule to
quote and extend CLAUDE.md's canonical ("handler/job-wrapper commits; helpers
flush; only quota owns its session") instead of stating a contradicting
"helpers MAY commit" carve-out.
ruff clean, full DB-lane 364 pass.
A sync was upsert-only: a movie/show/season/episode removed from Plex stayed
browsable forever and 404'd on playback. Each section now diffs the rating_keys
it just listed against the stored ones and deletes the stale rows (watch-state /
playlist links cascade via the existing FKs), mirroring the collections prune
that already shipped.
Safety: the prune is GUARDED on a non-empty listing (_prune_stale returns early
when `seen` is empty), so a transient Plex glitch that yields an empty page can
never mass-delete a library — the metadata is cheap to re-sync, an accidental
wipe is the only real risk. Shows prune leaf-first (episodes -> seasons -> shows)
so a stale row is deleted directly before its parent's cascade reaches it.
Out of scope (separate follow-up): library-level reconcile of a whole removed
section + retiring the dead `PlexLibrary.enabled` column.
Tests: movie prune + watch-state cascade, the empty-listing guard (mutation-
verified — removing the guard wipes everything), show/season/episode prune.
351 DB-lane pass, ruff clean.
Review round on the whole E1 stack (main..HEAD). 6 findings, all low/medium,
none blocking; fixed for a clean gate-green handoff.
- plex/sync (correctness): the un-enriched cast guard compared the stored cast
against THIS listing's length, so a trimmed Plex cast on a not-yet-enriched
row was never applied. Compare against the listing CAP (_LISTING_CAST_CAP=20)
instead: >cap = enriched (keep, survives a watermark reset), <=cap =
listing-sourced (track edits in both directions). +2 regression tests.
- main (security): add X-Frame-Options: DENY as the legacy-browser fallback for
the CSP frame-ancestors clickjacking guard.
- downloads/service (altitude): finalize_if_running now uses a SQLAlchemy 2.0
update() construct instead of a hand-built text() UPDATE string — same atomic
WHERE status='running', idiomatic, no identifier interpolation, no manual
StrEnum coercion. Drops the now-unused text import.
- auth/admin (reuse): extract the SEC-10 "drop untrusted preset password + bump
epoch" invariant into one helper (revoke_preset_password_if_unverified) so the
Google-callback and admin-activation paths can't drift.
- tests/migrations (robustness): guarantee the shared schema is restored to head
in a finally, so a downgrade/re-upgrade failure can't cascade into later tests.
- worker (simplification): align _finish_from_cache's display_name fill with the
_download path's guard (no behavior change).
Gate green: ruff clean, 334 pytest (0 skip), frontend unchanged.
The union form fixed the downgrade but removed the per-sync self-healing: a name
Plex removed from an unenriched title lingered in cast_names/people_text until
enrichment. Replace the union with a length guard — take the fresh listing
unless it is shorter than what is stored. A stored cast longer than the ~20-cap
listing can only be an enriched fuller list (kept, so a watermark reset never
downgrades it); at or below the cap the stored value is itself listing-sourced,
so replacing it with the current listing stays self-correcting.
_apply_facet_fields seeded cast_names from the cheap top-~20 listing only while
people_enriched_at was NULL, relying on "a sync never clears the watermark" to
protect an enriched fuller cast. Migration 0065 broke that assumption by nulling
people_enriched_at on every movie/show row, so the first post-deploy sync
downgraded enriched casts back to the listing top-20 — silently dropping
person-page cast-filter names for weeks until the backfill caught up (the S5b
regression, re-opened by the migration).
Guard structurally instead of trusting the watermark: union the listing cast
onto whatever is stored, never dropping a name already present. A fresh row
still seeds from the listing; a row whose watermark was reset but whose fuller
cast survives keeps it. Robust to any future watermark reset, not just 0065.
Address the /code-review high finding on S5b: gating the info-page cast enrich on
`people_enriched_at is None` removed the old per-view regrow of `cast_names`, so a routine full sync
that reset it to the listing top-20 (`_apply_facet_fields`) — while never clearing people_enriched_at —
left it permanently truncated, silently dropping cast past the top ~20 from the S4 person-page filter.
Root fix: `_apply_facet_fields` now only takes the listing's cast for a NOT-yet-enriched row; an
enriched row keeps its fuller stored cast (people_text still rebuilt from the updated directors + the
preserved cast). Makes the people_enriched_at flag the endpoints/backfill trust actually reliable.
Tests: cast_names preserved for an enriched row vs applied for a fresh one; show_detail lazy-fill
persists imdb/full_cast on first view. Full suite green.
Persist the info-page rich fields that were previously read LIVE from plex.metadata() on every view —
imdb_rating (IMDb-specific, vs `rating`=audienceRating), imdb_id, tagline, and full_cast
([{name,role,thumb}]) — onto plex_items/plex_shows, and serve the detail endpoints from the row.
- migration 0065: ADD imdb_rating/imdb_id/tagline/full_cast (nullable → no rewrite/lock, unlike 0064);
NULL people_enriched_at on movie/show rows so the existing cast backfill re-fetches each once and
fills the new columns.
- new app/plex/detail.py: rich_detail(meta) (moved out of browse._rich_meta so sync + serve share one
parser) + apply_rich_detail(row, meta); wired into _enrich_people_batch and the info-page lazy fill.
- item_detail: movies serve cast/IMDb/tagline/genres from the row and keep plex.metadata(markers=True)
ONLY for markers + audio/subtitle streams (so detail survives a live-Plex outage); episodes (never
enriched) still parse detail from that live meta.
- show_detail: serves cast/IMDb from the row; `related` stays live best-effort and also lazily fills a
not-yet-enriched show.
- tests: rich_detail/apply_rich_detail parsing + persistence, backfill stamping, item_detail
outage-resilience + lazy fill + episode-live, show_detail row serving. Full suite 285 pass / 2 skip;
migration verified reversible (up→down→up).
Precondition for S6's pure-DB language-aware serving.
- Correctness: _sync_overlay now first-wins on a guid (a `seen` set) so a duplicate HU entry
sharing a guid can't overwrite the applied Hungarian metadata or double-count `overlaid`.
- Test-coverage: cover the earlier dup-guid fix (both editions of a shared-guid pair get *_hu,
counted once) and the guid-keep guard (_apply_item preserves a prior guid when the listing
carries none, still updates on a real one).
Full backend suite 276 passed / 2 skipped.
- Correctness (stale HU): _sync_overlay is now a full reconcile — it clears every guid-bearing EN
row's *_hu first, then repopulates the matches, so a title whose HU twin vanished or was re-guid'd
falls back to EN instead of keeping stale Hungarian metadata (only rows that held HU data go dirty).
- Correctness (dup guid): by_guid maps a guid to the LIST of EN rows carrying it, so two editions of
one film both receive the overlay instead of just the last-iterated one.
- Robustness (guid wipe): _apply_item/_sync_shows keep a prior guid when a listing carries none
(`_canonical_guid(meta) or row.guid`), so a server ignoring includeGuids can't erase the match key.
- Reuse: merged _sync_overlay's movie/show branches (only the select + item_type differ).
- Test-coverage: a full-sync test drives sync() over a stub Plex (sections + section_items +
overlay_map) asserting pass-2 designation (metadata_lang/overlay_of), guid capture, the HU write,
and browse exclusion; plus a reconcile test that a vanished twin clears stale *_hu.
Full backend suite 274 passed / 2 skipped.
Adds the storage + sync for the Hungarian-metadata overlay (the browsable EN sections gain HU
twin fields written from a paired HU section, matched by guid; no browsable HU rows).
Schema (2 migrations, ADD-only; run before the first HU sync while *_hu is NULL so the generated
column build is trivial):
- 0063: plex_libraries.metadata_lang ('en' default) + self-FK overlay_of.
- 0064: plex_items/plex_shows.guid (indexed) + title_hu/summary_hu/genres_hu (+ GIN) + a STORED
generated search_vector_hu mirroring the EN vector (title_hu A / people_text B / summary_hu C) + GIN.
Sync:
- includeGuids=1 on section_items so the canonical guid rides the cheap listing.
- _canonical_guid(meta) (tmdb > imdb > tvdb) captured onto every EN row in _apply_item/_sync_shows.
- Two-pass sync(): pass 1 mirrors browsable (non-overlay) sections; pass 2 reads plex_overlay_map
('huKey:enKey', new sysconfig), designates each HU section (metadata_lang='hu' + overlay_of), and
_sync_overlay writes title_hu/summary_hu/genres_hu onto the guid-matched EN rows.
Browse: _content_libs now excludes overlay libraries (overlay_of is None); the unified_library and
facets inline library partitions fold into it, and list_libraries/collections exclude overlays too.
Tests: DB-lane coverage of the overlay write (matched vs unmatched rows, no browsable HU rows, the
generated search_vector_hu populates), the guid parse, and the browse exclusion. Full suite 272
passed; migrations verified reversible (upgrade→downgrade→upgrade). S5b (persist the live detail
fields + serve item/show detail from DB) is the follow-up.
- by_rating_key: generic-typed (type[_M] -> _M | None) so call sites keep their
Model | None static type.
- Migrate the last 2 open-coded PlexCollection-by-rating_key lookups (sync.py
resync_collection + delete_collection_local) to by_rating_key; zero open-coded
rating_key lookups now remain in the package.
- Add test_playlist_bulk_add_and_remove covering the bulk add/remove endpoints
and pinning the remove-bulk synchronize_session=False path.
Full suite 258 passed, 2 skipped; ruff clean.
Mechanical 1.x->2.0 conversion of the self-contained plex modules, behaviour-
equivalent (the characterization net + existing plex tests stay green, 39
passed): app/plex/{sync(15),watch_sync(9),people(1),imgcache(4)} and
app/routes/plex/{admin(4),playback(8),_shared(1)}. Query builders become
select(); .first()/.all()->execute().scalars(); a bulk .update() ->
execute(update()); a single-column .first() -> db.scalar().
browse.py (61) + rollup.agg_subquery (the atomic unit with unified_library)
remain for wave 2.
Transaction ownership (C-A5):
- quota.record_usage / log_action write in their OWN SessionLocal(), committed
immediately, and no longer take/commit the caller's db. The spend stays DURABLE
regardless of the caller (read-only YouTube paths never commit; a failed job rolls
back) — under-counting the shared daily budget would risk overspending the real
YouTube quota — AND quota stops flushing a job's partial state mid-run behind its
own `except: db.rollback()`. units_used_today switches to a scalar SELECT so
measured()'s before/after diff sees the separate session's commits (READ COMMITTED).
Call sites updated (youtube/client.py ×3, routes/search.py ×1).
- plex sync commits PER SECTION (moved the end-of-loop commit inside), so a PlexError
on a later library keeps the ones already mirrored (idempotent; next run reconciles).
The sync jobs did NOT rely on record_usage's incidental commit for persistence — their
per-item functions (apply_rss_feed, backfill_channel_recent, import_subscriptions,
sync_user_playlists, apply_channel_autotags) all commit internally, so the in-loop
rollback handlers already isolate a failed item. No loop restructuring needed.
DB-lane tests (test_quota_ownership, 4): spend survives a caller rollback while the
caller's uncommitted row does NOT leak; accumulation; measured-style mid-txn read.
Gate: ruff clean; DB lane 180 green.
Lazy enrichment only covers viewed titles, so an actor's count/results were
partial. Add a background backfill: each plex_sync rich-fetches a bounded batch
(60) of not-yet-enriched movies/shows and persists their FULL cast, stamping
people_enriched_at so each is fetched once — the whole library is covered over
successive runs (migration 0057 adds the marker). Extracted the enrichment into a
shared app/plex/people.py (cast_from_meta / enrich_row_people / prune_orphan_people
+ the person-image host/proxy) so the info-page lazy path and the batch share ONE
implementation — no duplication; the info page also stamps the marker so the batch
skips it. Verified: an 8-item batch grows those movies' cast_names 3->24 and fills
plex_people (32->224); the info-page cast is intact after the _rich_meta refactor.
(Plex cast-explore S1b)
Guard the prune's NOT IN against the classic NULL trap: a JSON-null cast array
element would surface as SQL NULL in the referenced set, making name NOT IN
(…, NULL) evaluate NULL for every row → silently prune nothing. Filter NULLs out
of the subquery. (Plex cast-explore S4 review)
The plex_people headshot cache can outlive its subjects — when a title leaves the
Plex library, cast/crew who appear in nothing else are orphaned. prune_orphan_people
deletes rows whose name no longer appears in ANY movie/show cast_names/directors,
run at the end of every plex_sync (best-effort; a jsonb_typeof='array' guard skips
scalar/null JSONB values). The filterable cast_names self-clean with their row;
this cleans the only per-person cache. Verified: an injected orphan is dropped, real
people kept. (Plex cast-explore S4)
Closes the Plex #9 backend backlog (frontend was the prior follow-up). Behavior-
neutral cleanup + two two-way-sync bug fixes; PW2 verified a non-bug.
fix(plex): watch-sync PW1 pagination + PW3 lost-update guard
- PW1 watch_history: the history is a GLOBAL cross-account feed read as a single
500-row page, then filtered to the owner — on a busy family server the owner's
recent views can sit past page 1 and be missed. Now paginate (desc) until the
since-cutoff, bounded by max_pages (daily reconcile is the backstop; logs if hit).
- PW3 push_state_to_plex: it flagged synced_to_plex=True unconditionally after a
push, so a local edit landing between the push and the flag-set was marked clean
and never pushed (lost update). Capture the row's freshest timestamp when the
push is scheduled (_push_watch) and only settle the flag if the row is unchanged.
- PW2 was flagged as "accountID hardcoded to 1" but is NOT a bug: on a PMS account 1
is reserved for the owner/admin and an owner link always holds the admin token —
added a clarifying comment so it isn't re-flagged.
chore(plex): backend dedup
- PC2 unified_library + facets shared a ~13-field filter Query signature + p-dict →
one LibraryFilters dataclass injected via Depends(); as_dict() feeds the builders.
- PS-C1 sync.py collection upsert dup (resync_collection ≈ _sync_collections) →
_upsert_collection().
- PS-C2 sync.py 8-field filterable-metadata block dup (_apply_item ≈ _sync_shows) →
_apply_facet_fields().
- PW-C1 _repush_dirty read duration from the mirrored PlexItem.duration_s instead of
a per-row plex.metadata() round-trip.
- PW-C2 rk→id map built inline in two places → _rk_to_id() helper.
Give TV shows the same filterable metadata as movies so the TV grid can be
filtered/sorted, not just library+sort. Backend: migration 0052 adds
rating/content_rating/studio/originally_available_at/genres/directors/cast_names/
people_text to plex_shows (+ GIN/indexes, people_text folded into search_vector);
_sync_shows populates them cheaply from the show section listing (no per-item
calls). /browse show-branch gains the movie filter set (minus duration) plus an
aggregate per-user watch-state (a show rolls up its episodes: all watched=watched,
any progress=in_progress, none=new) with year/rating/release sorts; /facets returns
show facets. Frontend: PlexSidebar renders the metadata filters + watch-state for TV
libraries (duration stays movie-only); show cards show a watched/in-progress badge.
i18n plex.inProgress (en/hu/de). Needs a Plex re-sync to populate the new columns.
Admins can curate Plex collections from a movie's info page ("Collections"
button → PlexCollectionEditor dialog): create a collection (seeded with the
movie), add/remove the movie to/from editable collections, delete a collection,
and "take over" an existing plain Plex collection (mark it editable). All writes
go to Plex (POST/PUT/DELETE via new PlexClient methods) and are reflected on
every client; a targeted single-collection re-sync (sync.resync_collection /
delete_collection_local) updates the local mirror without the full ~4-min sync.
Gating (per the design decisions): editing is ADMIN-ONLY (collections are shared
library-wide); only plain manual collections are editable — smart + external
auto-lists (IMDb/TMDb/…) are always read-only (can_edit = editable && !smart &&
source=="collection"). New admin endpoints under /api/plex/collections
(create/items add+remove/rename/delete/editable). Verified end-to-end incl. the
live Plex write API (create/add/rename/remove/delete all 200, self-cleaned) and
the editor UI (create + delete with confirm) on localdev.
Backend (migration 0047_plex_collections): a plex_collections table mirrors every Plex
collection (card metadata + smart flag + an editable flag reserved for Phase 2); membership
is stored as GIN-indexed collection_keys on member movies (plex_items) and shows (plex_shows).
The background sync (plex_sync) now fetches each library's collections + their children and
rebuilds membership — so ALL reads are local (Plex's dual-language collection queries are slow;
this trades a ~4-min background sync for instant reads). New /api/plex/collections endpoint;
/browse gains a combinable filter; item_detail returns the movie's collection
'strips' (sibling titles as playable cards, smallest/most-specific collection first) — all pure
local lookups.
Frontend: PlexSidebar gains a searchable Collection picker + an active-collection chip;
PlexInfo renders the collection strips (playable posters + 'Browse collection' → sets the
filter); the collection is part of PlexFilters (persisted). i18n en/hu/de.
Phase 2 (create/edit collections with write-back to Plex) is separate.
Backend (migration 0046_plex_people_search): a new people_text column (cast +
director names, de-duped) is folded into the generated search_vector at weight B,
so the top search box finds titles by an actor/director name and ranks them above
summary-only mentions. New /api/plex/people endpoint returns the cast/crew matching
the term (name prefix or word-start) with a film count + a headshot (pulled from one
representative film's live metadata; image bytes disk-cached).
Frontend: PlexBrowse shows matching people as virtual cards above the grid; clicking
one adds them to the actor/director filter (multi-value, from the C1 work) and clears
the search box so you land on exactly that person's films. Answers the 'why does
"drew" match Rambo?' confusion — it was matching the word in the synopsis; now names
are searchable. i18n en/hu/de.
⚠️ Prod needs a Plex re-sync after deploy to populate people_text (search_vector
regenerates automatically once the column has data).
Backend (migration 0045_plex_filter_meta): plex_items gains rating (audienceRating
~IMDb), content_rating, studio, originally_available_at, and GIN-indexed genres /
directors / cast_names — all mirrored from the cheap section listing (no per-item API
calls; they also seed a future watch-habit recommender). /browse gains genre / content-
rating / year / rating / duration / added-within / director / actor / studio filters
(@> containment, GIN) + sort by year|rating|duration|release; new /facets endpoint
returns available genres+ratings (with counts) and the year/rating/duration bounds. A
thin on-disk image cache (.plex-img-cache) serves posters/art/cast photos from local
disk after first fetch (~7-14x faster repeat loads).
Frontend: PlexSidebar grows the full filter set (facet-driven genre/age chips, rating
steps, year range inputs, duration buckets, added-within, active people/studio chips,
clear-all); filters persist per-account as one JSON blob. PlexInfo metadata (year,
genre, director, cast, studio, IMDb score) is clickable → sets the matching filter and
returns to the filtered grid (page variant only; the in-player overlay stays read-only
so a stray click can't stop playback). i18n en/hu/de.
- app/plex/sync.py: full reconcile of enabled movie/show sections into plex_*
(upsert by rating_key). Reuses Plex's own codec info to classify browser
playability (direct/remux/transcode) — no ffprobe. Movies + episodes become
playable leaves; shows/seasons mirrored for the drill-down.
- routes/plex.py: GET /libraries, /browse (FTS search + sort + paging, movie
leaves or show cards), /show/{rk} (seasons+episodes, per-user state), /image
(proxies Plex art, no duplication); admin POST /sync. All auth'd, demo-allowed.
- scheduler: plex_sync job (interval settings.plex_sync_interval_min).
- Verified against the real server: 2 libs, 3206 movies, 260 shows/961 seasons/
14768 episodes synced in 13s; FTS search + drill-down work. Playability:
~1% direct, 89% remux, 6% transcode (informs P2/P3).
peter