Review follow-up. The docstring said "channel/video description" but the helper is
wired only to the channel store sites; clarify it is deliberately channel-only
(video descriptions are user-facing prose where paragraph breaks carry meaning,
unlike a channel's metadata block) so it doesn't read as an asymmetric fix. Also
document that str.splitlines() folds CR/CRLF and Unicode line separators to "\n",
and lock it with a test.
YouTube channel descriptions pad their contact/link blocks with runs of blank
lines, which the About popover renders verbatim (whitespace-pre-wrap) — wasted
space and storage, and noise in the search_vector. Add normalize_description()
and apply it at both channel store sites (full details + initial import) so the
DB holds the clean text. New channels are clean immediately; existing rows heal
on their next metadata re-sync. Unit-tested in test_utils.
Round-3 review of the S3 config-fold, all in routes/scheduler.py:
- stale docstring: "Persisted to app_state" -> "Persisted via sysconfig" (the
batch moved stores in B-core-11; the comment didn't follow).
- drop the manual min/max check in the PATCH — sysconfig.set_value already
validates against the same spec bounds; catch its ValueError -> 400, matching
the config PATCH pattern (bounds enforced in one place).
- fetch sysconfig.spec(...) once into a local in the GET instead of twice.
No behaviour change beyond the 400 message wording (untested path). 368 pass.
Restores CLAUDE.md's canonical transaction rule ("the handler commits; helpers
flush; only quota owns its session") on the config path: sysconfig.set_value /
sysconfig.reset and state.set_sync_paused now flush instead of committing, so a
change lands in ONE transaction with the audit row the route records for it —
previously the helper's internal commit split them, and a crash between left the
mutation persisted with no audit trail.
All callers already commit (the config PATCH/DELETE + audit, the sync pause/resume
+ audit, the scheduler maintenance PATCH); the setup wizard, which relied on the
internal commit, gets an explicit db.commit() after its loop. apply_interval is
deliberately left committing — it performs a live APScheduler reschedule that must
follow the DB persist, so it owns its own unit of work (a legitimate exception,
like the download service transitions). New test locks the flush-not-commit
contract (a rollback before the caller commits discards the change). 368 pass.
The admin-tunable maintenance re-validation batch lived in its own app_state
column with a hand-rolled override-or-default reader in app.state — a fourth
config store duplicating the sysconfig SPECS pattern. Move it to a sysconfig key
(`maintenance` group, min=100 max=100_000): the reader (sync/maintenance.py) and
the Scheduler dashboard (routes/scheduler.py PATCH + GET) now go through
sysconfig.get_int / set_value / spec, and app.state loses get/set_maintenance_batch
+ the MIN/MAX constants + the settings import. Migration 0070 carries any existing
override into the system_config store before dropping the app_state column
(reversible; restores + copies back on downgrade). Reuses the existing
sysconfig.spec() for the dashboard's min/max, so the bounds have one source.
Full lane 365 pass (migration drift + reversibility exercised), ruff clean.
None are referenced by the SPA or tests; the Scheduler dashboard's "run now"
covers every manual trigger. Removed: POST /api/sync/{rss,backfill,enrich}
(+ the now-dead _user_channels helper and three sync.runner imports),
POST /api/tags/recompute (+ its admin_user / run_autotag_all imports), and
GET /auth/me (a lesser duplicate of the canonical /api/me in routes/me.py).
This also retires a latent bug: POST /api/sync/backfill reset backfill_done /
backfill_cursor on already-fully-backfilled channels (tipping them back to
"needs full history") for up to 25 channels per curl. Full lane 365 pass.
This codebase leans on "why" comments as its architecture doc, so a header that
claims the opposite of the code below it is worse than none. Fix the seven the
review flagged: channels.py (subscribe/unsubscribe shipped), playlists.py +
sync/playlists.py (YouTube two-way sync + write-back shipped), setup.py (drop the
"epic 6c" ref — the wizard steps exist), config.py ("auth._client_ip" ->
"routes.auth._client_ip"), db.py ("routes/plex.py" -> "routes/plex/" after the
package split), sysconfig.py (the "Phase C" note — the watch-sync job that reads
plex_watch_sync_interval_min has landed; clarified it's a boot default, not a knob).
Docs only.
Round-2 review of the R1 fixes found that the two-independent-limiter progress
writer issued a redundant heartbeat every interval during an ACTIVE download: a
percent change writes progress (which already refreshes updated_at, the stall
watchdog's liveness source), then the sub-integer calls that follow fire the
separate beat budget — ~2 DB writes/interval instead of the throttle's intended 1.
Replace the second limiter with a shared `last_touch` clock (last DB write OR
beat). Progress writes keep their own `allow_write` budget (so a beat still can't
delay a real write — the R1 concern), but the heartbeat now fires only when
nothing touched the row for a full interval. Active download → ~1 write/interval;
a genuinely stuck re-encode → 1 heartbeat/interval. New regression test:
an unchanged-percent call in the same interval as a write does NOT heartbeat.
ruff clean, full DB-lane 365 pass.
Round-1 review fixes on the E3 branch:
- prune partial-fetch (correctness): `_paginate` now reports whether it reached
the server's reported `total`; `_prune_stale` prunes only on a COMPLETE
non-empty listing. A truncated fetch (a mid-sequence empty/short page before
total — a transient Plex hiccup) no longer deletes the rows it merely failed
to list. New test: a listing that claims 5 but returns 1 prunes nothing.
- `_prune_stale` (robustness): `stats.get("pruned", 0) + 1` — no KeyError when a
caller's stats dict omits the key.
- service transitions (conventions): pause/resume/redownload use the JobStatus
StrEnum instead of loose "queued"/"paused"/"error" literals, matching cancel.
- ffmpeg progress writer (correctness): a stuck-percent heartbeat now runs on its
OWN 1/interval budget, so it can't consume the progress-write slot and delay a
real percent change by up to an interval.
- backend/app/README.md (conventions): reword the transaction-ownership rule to
quote and extend CLAUDE.md's canonical ("handler/job-wrapper commits; helpers
flush; only quota owns its session") instead of stating a contradicting
"helpers MAY commit" carve-out.
ruff clean, full DB-lane 364 pass.
The layering rule ("request-only logic in routes/; scheduler+worker-shared logic
in the plex|downloads|sync packages; cross-cutting primitives as flat core
modules; extract at the SECOND consumer") was real but unwritten — it lived only
in three module docstrings and a one-line README mention. Collect it, plus the
transaction-ownership rule (a mid-operation helper stages, the caller commits),
the atomic-counter / JobStatus-StrEnum conventions, and the one-home-per-config
rule, into backend/app/README.md. Add docs/development.md (run locally + the gate
commands + branching), so a new session/contributor has one place to look instead
of copying whichever pattern they happen to find first.
worker.py (~1090 L) mixed the PARENT supervisor (claim + supervise + the process
loop) with the CHILD pipeline (the yt-dlp/ffmpeg download+edit that runs in the
spawned subprocess). Extract them:
worker.py 1090 -> 475 parent: claim/supervise/loop
downloads/pipeline.py (new 592) the child pipeline (_download/_process_edit +
hooks/progress/ffmpeg helpers + _Aborted)
downloads/jobdb.py (new 73) the short-lived-session helpers both sides
share (_set_job/_job_status/_touch_job/
_requeue_if_running/_reset_asset_pending +
the staging-dir paths)
The graph is one-directional (runner -> pipeline -> jobdb; worker -> jobdb; no
worker<->pipeline edge) — the parent never calls the child directly, it spawns
runner.py as a subprocess. Consumers rewired: runner import, and two tests follow
the moved symbols (_media_info/_produced_file -> pipeline; the ffmpeg-callback
patches target pipeline where the names are looked up).
Pure code motion, verbatim bodies. ruff clean, full lane 363 pass (unchanged).
request-access / demo / register / verify / verify-resend / password-login /
password-reset(request|confirm) / set-password each took an untyped `payload:
dict` and hand-picked fields with `.get(...)`. Replace with five small BaseModels
(EmailIn, EmailPasswordIn, TokenIn, TokenPasswordIn, SetPasswordIn) so FastAPI
validates the body shape and publishes the schema. Fields are `str | None` to
preserve exact behaviour — a missing/null field still coerces via the existing
`(... or "")`, no stricter 422s. Green under the auth-session suite (7), ruff clean.
pause/resume/cancel/delete/redownload each hand-rolled their asset bookkeeping
inline in the route (release, failed-asset reset, the 20-line fresh-asset
rebuild), duplicating logic the worker/GC also touch. Move each transition into a
service.* method (service.pause/resume/cancel/delete/redownload) that owns the
mutation + its commit; the routes keep only auth, the HTTP 409 guards, and
serialization. Drops the now-dead _set_status route helper and two stale imports.
Pure behaviour-preserving code motion (routes -56 lines). Green under the E3 S2
transition net + the full downloads suite (363 pass), ruff clean.
The supervisor kills a job that writes nothing for download_stall_timeout_seconds
(150s) and labels it a "network" stall. But the ffmpeg progress writer only wrote
on an integer-percent CHANGE, so a re-encode that sits on one percent for minutes
(AV1/VP9 -> H.264 or a crop on the CPU-only prod host) produced no row write and
was killed mid-encode as "stalled" — and the retry stalled the same way.
Fix: the writer now heartbeats (`_touch_job`, bumping updated_at without touching
the progress bar) once per interval even when the whole percent is unchanged, so
a live-but-quiet re-encode keeps a fresh updated_at. Reorders the rate-limit gate
first so a burst still costs one DB op — either a progress write or a heartbeat.
Pure-logic test with an injected clock: three stuck-at-10% intervals write no new
progress but heartbeat each time. Full lane green, ruff clean.
Two jobs on the same shared asset (same video+format queued twice, or by two
users) starved the worker pool: worker A downloads the asset; worker B claims the
sibling job, finds the asset `downloading`, requeues it, sleep(2)s, and re-claims
it on the next tick — burning a whole worker slot behind one in-flight asset for
the entire download (at concurrency=2, the queue stalls).
Fix: the claim SQL now excludes any job whose asset is already `downloading`
(`NOT EXISTS … media_assets.status='downloading'`), so a competing sibling simply
waits until the asset settles and the workers pick up unrelated jobs meanwhile.
The `_process_job` busy/lost-race branches drop their `time.sleep(2)` — the
exclusion keeps a requeued job out of the claim window, so there's no spin.
TDD: the new test failed on the old claim (claimed the sibling), passes now.
Full lane green, ruff clean.
MediaAsset.ref_count (how many jobs hold a shared cache asset) was a Python
read-modify-write at four sites (enqueue, enqueue_edit, release_asset, the
redownload route). Two concurrent enqueues of the same (source,format) both read
the same value and write the same +1 → the count ends up one short → the first
release drives it to zero and deletes the file out from under a live holder.
Replace all four with atomic DB ops: incr_ref_count (`ref_count = ref_count + 1`)
and decr_ref_count (`greatest(ref_count - 1, 0)` RETURNING the new value, so the
release decides whether to free the asset from the post-decrement count while
holding the row's write lock — Postgres serialises concurrent releases on that
lock, closing the double-free). Pure DB-atomicity change, no semantic shift.
Green under the E3 S2 ref_count net (5) + full lane (356 pass), ruff clean.
A sync was upsert-only: a movie/show/season/episode removed from Plex stayed
browsable forever and 404'd on playback. Each section now diffs the rating_keys
it just listed against the stored ones and deletes the stale rows (watch-state /
playlist links cascade via the existing FKs), mirroring the collections prune
that already shipped.
Safety: the prune is GUARDED on a non-empty listing (_prune_stale returns early
when `seen` is empty), so a transient Plex glitch that yields an empty page can
never mass-delete a library — the metadata is cheap to re-sync, an accidental
wipe is the only real risk. Shows prune leaf-first (episodes -> seasons -> shows)
so a stale row is deleted directly before its parent's cascade reaches it.
Out of scope (separate follow-up): library-level reconcile of a whole removed
section + retiring the dead `PlexLibrary.enabled` column.
Tests: movie prune + watch-state cascade, the empty-listing guard (mutation-
verified — removing the guard wipes everything), show/season/episode prune.
351 DB-lane pass, ruff clean.
routes/plex/browse.py was a 1693-line god-module (libraries + facets +
collections + playlists + show/item detail + people). Split it into focused
sibling routers under the same /api/plex prefix — pure code motion, no SQL /
route-path / response-shape changes:
browse.py 1693 -> 469 (list_libraries, unified_library, facets)
collections.py (new 253) browse + admin collection editor; owns _collection_source
playlists.py (new 405) the user's orderable playlists
detail.py (new 438) show/item detail + enrichment; _lib_key lives here
people.py (new 139) cast/crew chips + person filmography
Cross-module helpers _norm_lang + _content_libs move to _shared.py (their one
home); detail imports _collection_source from collections (one-way, no cycle).
__init__.py includes the four new routers. Three DB-lane tests follow the moved
symbols (import path for _content_libs; monkeypatch targets for PlexClient /
_show_enrichment now on detail).
Behaviour-equivalent: ruff clean, 348 DB-lane pass (unchanged), parity net green.
The `show` scope (all|watched|in_progress|unwatched) filter and the status CASE
were hand-duplicated between unified_library (the grid) and facets.wsq — the
exact drift risk the review flagged (a hidden movie kept out of the grid must
not inflate facets). Extract them into four shared query.py helpers
(movie_status_case, show_status_case, apply_movie_state_filter,
apply_show_state_filter), model-agnostic (callers pass the joined state
expressions), and point all three call sites at them.
Pure behaviour-equivalence refactor: browse.py -67 lines, no query changes.
Green under the E3 S1 parity net + query_conversion/facets/field-scoped/lang
tests (348 DB-lane pass, ruff clean).
Two scheduler jobs full-seq-scanned the 257k-row videos table every run:
- PERF-17 enrich_pending (`WHERE enriched_at IS NULL LIMIT N`, every few minutes):
44,680 buffers even with 0 pending -> a partial ix_videos_enrich_pending makes
it a 1-buffer index scan (empty when nothing pending).
- PERF-18 maintenance revalidate (`WHERE unavailable_since IS NULL ORDER BY
last_checked_at NULLS FIRST LIMIT N`, ~300 pages/run): 44,754 buffers + top-N
sort -> a partial ix_videos_recheck_order in that exact order = a 42-buffer
index scan. The order_by is rewritten to `asc().nulls_first()` (equivalent to
the old `(IS NULL) DESC, ASC`) so it rides the index; the unused plain
ix_videos_last_checked_at is dropped.
PERF-3: drop the never-picked boolean btrees (is_short, via_explore) and slim the
useful ones to partials over the queried subset (via_search=true, NOT
shorts_probed, unavailable_since IS NOT NULL) — less write amplification on the
biggest table, same lookups. Model: those 6 columns lose their inline index=True
(the partials are migration-defined, like the search_vector GIN); live_status
left alone.
Measured on the prod-sized localdev reseed: enrich 44,680 -> 1 buffer (25.9 ->
0.02 ms), maintenance 44,754 -> 42 buffers (35.5 -> 0.17 ms); all 11 CONCURRENTLY
index ops on 257k rows in 0.75s. Gate green (346 pytest incl. drift + downgrade
round-trip).
The tags_by_category extraction dropped feed_summary's `if selected:` guard, so
a /feed/summary request with no tag filter (the common case) ran a pointless
`SELECT ... WHERE id IN ()` round-trip. Early-return {} for empty tag_ids —
restores the no-query hot path and covers filtered_query + future callers too.
Gate green (346 pytest).
Review round on the E2 stack (S1-S5). No correctness bugs; the surviving
low/medium findings addressed:
- altitude (#1): extract the shared `tags_by_category` helper used by BOTH
filtered_query (SQL tag WHERE) and feed_summary (Python facet/count), removing
the category-grouping duplication, and add bidirectional "keep in lockstep —
equivalence-tested by test_feed_summary_matches_count_plus_facets" comments so
the SQL<->Python tag rule can't silently drift.
- test-coverage (#2): the /feed/summary equivalence test seeded only system tags,
leaving the `ChannelTag.user_id == user.id` visibility branch untested. Add a
user-owned Tag + link + two combos; the suite is now 346 (was 344) and the new
cases pass (feed_summary agrees with /feed/count + /facets on user-visible tags).
- dead-code (#3): annotate get_feed_count, get_facets, and the FE api.feedCount/
facets as "SUPERSEDED by /feed/summary — kept only as the equivalence oracle;
retire together" so the intentionally-retained surface reads as deliberate.
Not changed (confirmed no-op): the search channel-name resolve (#4) is
once-per-request on a 633-row table and a deliberate pre-resolve for PERF-12's
BitmapOr plan; the facet loop (#5) already computes passes() once per
(channel, category) with a distinct exclude each, so there is no recomputation.
Gate green: ruff + 346 pytest; frontend tsc/knip/prettier (FE change is a comment).
The FE fired /feed/count (1 pass) + /facets (3 passes) on every filter change =
4 full scans of the subscribed set. New /api/feed/summary does ONE grouped query
(the base filter with tags stripped, GROUP BY channel_id -> per-channel video
counts) and derives the total count + every tag's distinct-channel facet count in
Python on the ~few-hundred-row channel map, faithfully mirroring filtered_query's
channel-based tag logic + the /facets drill-down (own category dropped, unless a
conjunctive topic-AND). Returns {count, facets} — the same numbers the two old
endpoints returned.
FE: a single shared query (Feed reads .count, Sidebar reads .facets); same key +
filters, so React Query dedupes to ONE request. Invalidations repointed to
qk.feedSummary. The old /feed/count + /facets endpoints (and their now-unused FE
api/qk) are kept as the equivalence-test oracle + safety net; a later cleanup can
drop them.
New equivalence test (test_feed_views, 8 filter combos incl. cross-category,
disjunctive/conjunctive topic modes, search): /feed/summary count == /feed/count
and facets == /facets, byte-identical. Structural win: 2 requests / 4-6 queries
-> 1 request / 2-3 queries (the review measured 530 -> 144 ms). Gate green:
344 pytest, frontend tsc/eslint/knip/prettier/222 vitest.
PERF-12: the search ORed `unaccent(channels.title) ILIKE '%q%'` straight into
the per-video predicate, which forced a full scan + detoast of every
search_vector and a per-candidate channel fetch. Pre-resolve the matching
channel ids (a few-hundred-row scan) and match `Video.channel_id.in_(ids)`
instead, so the planner keeps the search_vector GIN as a BitmapOr branch
(alongside ix_videos_channel_pub_desc). Empty match -> false() collapses the OR.
PERF-16: show=watched|hidden|in_progress used an outer join + coalesce(status,
'new'), so the planner couldn't start from the user's small video_states set.
Those views only return rows that HAVE a state, so inner-join video_states and
filter the raw column (status is NOT NULL on a present row, so it's identical).
'unwatched'/'all' keep the outer join. Dropped the now-dead position_expr.
New regression tests (test_feed_views): search returns both text- and channel-
name matches; the four watch-state views return the exact expected sets.
Measured on the prod-sized localdev reseed (prod planner, 0067/0068 applied):
- search /feed/count q=magyar: 663,463 -> 19,206 buffers (1,406 -> 31 ms; OLD
seq-scan+detoast -> NEW BitmapOr GIN+channel index)
- search feed page: 2,908 -> 1,479 buffers (32.6 -> 1.4 ms)
- show=watched (user with 64 watched): 40,661 -> 264 buffers (~154x)
Gate green (336 pytest, incl. the 2 new equivalence tests).
A nullable published_at forced the feed keyset into a non-sargable
`... OR published_at IS NULL` disjunction: with the 0067 index the planner still
applied it as a FILTER and scanned from the top of the index, so a deep `newest`
page discarded every earlier row (O(depth)).
- Migration 0068: backfill any NULLs (0 in prod) -> SET NOT NULL + DEFAULT now().
- Model: Video.published_at is now non-Optional, nullable=False, server_default now().
- feedquery: the `pub` key is nullable=False, and keyset_predicate is rebuilt
recursively so a non-nullable leading key gets a non-strict range bound
(`published_at <= v`) as an AND conjunct — an index range Cond instead of a
filter. Nullable keys keep the NULL-aware OR form (a `<= v` bound would drop
their NULL group). Equivalence proven by test_feed_keyset (all 13 sorts,
cursor-walk == unpaged).
- Ingest paths (stub_from_playlist_item, apply_video_details, playlists sync)
fall back to now() so they never write an explicit NULL.
Measured on the prod-sized localdev reseed at a ~50k-deep cursor (0067 index +
prod planner): OLD `Rows Removed by Filter: 50001`, 25,687 buffers / 9.16 ms ->
NEW `Index Cond: published_at <= v`, 1 row filtered, 47 buffers / 0.037 ms. SET
NOT NULL on 257k rows: 0.32s. Gate green (334 pytest).
The feed's default `newest` sort is `published_at DESC NULLS LAST, id ASC`, but
the only published_at index was a plain ASC btree — read backwards it satisfies
DESC NULLS FIRST, not NULLS LAST — so every feed page fell back to a full
`videos` seq scan + top-N sort. Migration 0067 adds the two matching indexes
(global + per-channel) CONCURRENTLY and drops the now-redundant single-column
ix_videos_channel_id (the channel composite leads with channel_id, a strict
superset).
Measured on the prod-sized localdev reseed (257,312 videos, prod planner):
- global newest: 44,747 buffers / 154 ms -> 60 buffers / 0.32 ms
- per-channel(12.6k): 7,604 buffers / 96 ms -> 54 buffers / 0.14 ms
CONCURRENTLY build of both on 257k rows: 1.7s total (~0 downtime). This is the
likely root cause of the "prod feed is slow" symptom.
Model: drop the inline index=True on Video.channel_id (the composite supersedes
it); the composites are migration-defined like the search_vector GIN. Gate green
(334 pytest incl. the migration drift + downgrade/re-upgrade round-trip).
Review round on the whole E1 stack (main..HEAD). 6 findings, all low/medium,
none blocking; fixed for a clean gate-green handoff.
- plex/sync (correctness): the un-enriched cast guard compared the stored cast
against THIS listing's length, so a trimmed Plex cast on a not-yet-enriched
row was never applied. Compare against the listing CAP (_LISTING_CAST_CAP=20)
instead: >cap = enriched (keep, survives a watermark reset), <=cap =
listing-sourced (track edits in both directions). +2 regression tests.
- main (security): add X-Frame-Options: DENY as the legacy-browser fallback for
the CSP frame-ancestors clickjacking guard.
- downloads/service (altitude): finalize_if_running now uses a SQLAlchemy 2.0
update() construct instead of a hand-built text() UPDATE string — same atomic
WHERE status='running', idiomatic, no identifier interpolation, no manual
StrEnum coercion. Drops the now-unused text import.
- auth/admin (reuse): extract the SEC-10 "drop untrusted preset password + bump
epoch" invariant into one helper (revoke_preset_password_if_unverified) so the
Google-callback and admin-activation paths can't drift.
- tests/migrations (robustness): guarantee the shared schema is restored to head
in a finally, so a downgrade/re-upgrade failure can't cascade into later tests.
- worker (simplification): align _finish_from_cache's display_name fill with the
_download path's guard (no behavior change).
Gate green: ruff clean, 334 pytest (0 skip), frontend unchanged.
SEC-10 (account pre-emption / "pre-hijack"): a password set on an account whose
email was never verified is untrusted — whoever registered the address (to
pre-empt the real owner) could have chosen it. Both the Google-adoption callback
and admin activation (_activate_user_by_email) now drop that pre-set password and
bump the session epoch when the account isn't email_verified, so a later Google
login / admin approval can't hand the pre-set password a working login. No-op for
a verified account (the legit no-SMTP flow). +2 tests.
SEC-11 (link/upgrade bound to the session default): the OAuth link/upgrade GETs
resolved to the session DEFAULT account, so a tab acting as a secondary account
would link Google / the YouTube grant to the wrong account. The frontend now
threads the active account through the navigation URL (withActiveAccount ->
?account=<active>), and _complete_link accepts any wallet account, not only the
default, so the identity lands on the account that started the flow.
A small, safe set applied to every response via middleware: X-Content-Type-Options
nosniff, Referrer-Policy strict-origin-when-cross-origin, CSP frame-ancestors
'none' (anti-clickjacking; unrelated to the YouTube/Plex embeds we host), and
Permissions-Policy disabling camera/mic/geolocation. A full CSP and HSTS are
deferred (HSTS belongs on the Caddy edge).
The union form fixed the downgrade but removed the per-sync self-healing: a name
Plex removed from an unenriched title lingered in cast_names/people_text until
enrichment. Replace the union with a length guard — take the fresh listing
unless it is shorter than what is stored. A stored cast longer than the ~20-cap
listing can only be an enriched fuller list (kept, so a watermark reset never
downgrades it); at or below the cap the stored value is itself listing-sourced,
so replacing it with the current listing stays self-correcting.
Follow-up to the /code-review high on the S1 branch. The per-site
`if job.status == 'running'` guards were a non-atomic read-then-write (TOCTOU):
a cancel committing between the child's status read and its commit could still
be overwritten with done/error, re-opening the double-release window.
Introduce service.finalize_if_running(db, job_id, **fields): one atomic
`UPDATE download_jobs SET ... WHERE status='running'` that returns whether it
fired. It flushes pending asset writes first so the asset-then-job row-lock
order matches the cancel route (no deadlock), and runs in the caller's session
so it commits with the asset writes in one transaction. All five worker sites
(download/edit completion, both error paths, cache finish) now go through it,
so the invariant lives in one place. The 'job done' logs move inside the
finalized branch so a job stopped mid-completion no longer logs a false 'done'.
Add tests/test_downloads_finalize.py: running -> finalizes; canceled/paused ->
no-op.
POST /api/me/switch validated wallet membership, access and suspension but not
the target account's session epoch. Switching to an account revoked elsewhere (a
password reset/change or "log out everywhere" bumped its epoch) made it the
session default, so the next request's current_user hit the epoch mismatch on
the default identity and cleared the WHOLE wallet session — logging out every
account in the browser. Mirror the suspend guard: drop the stale account from
the switch list and return 403 instead of switching.
A rotated or missing TOKEN_ENCRYPTION_KEY makes every stored secret undecryptable.
sysconfig._raw_override called security.decrypt() with no guard, and it sits on
hot paths (/api/me via google_enabled, login, every YouTube scheduler job,
account purge) — so a key mismatch raised InvalidToken and 500'd the whole app,
the SPA wouldn't even boot, with no documented recovery.
Catch the decrypt failure in _raw_override -> warn once per key and fall back to
the env/config default. The YouTube client's refresh-token decrypt now raises a
clear YouTubeError ("reconnect") that every caller already handles, and
purge_user uses decrypt_optional so a best-effort revoke can't 500 the delete.
Document that the key must never change once secrets are stored (recovery: clear
the affected system_config rows and re-enter).
/api/search/youtube raised 502 on an upstream error, but the frontend treats
502/503/504 as a gateway dropout: it auto-retries the GET (a second quota-costed
search) and shows a "connection lost" banner instead of the real message. Every
sibling upstream-error site already returns 422 for exactly this reason. Reserve
5xx for the raw byte-proxies.
_apply_facet_fields seeded cast_names from the cheap top-~20 listing only while
people_enriched_at was NULL, relying on "a sync never clears the watermark" to
protect an enriched fuller cast. Migration 0065 broke that assumption by nulling
people_enriched_at on every movie/show row, so the first post-deploy sync
downgraded enriched casts back to the listing top-20 — silently dropping
person-page cast-filter names for weeks until the backfill caught up (the S5b
regression, re-opened by the migration).
Guard structurally instead of trusting the watermark: union the listing cast
onto whatever is stored, never dropping a name already present. A fresh row
still seeds from the listing; a row whose watermark was reset but whose fuller
cast survives keeps it. Robust to any future watermark reset, not just 0065.
The cancel route writes status 'canceled' but the supervisor compared against
'cancelled', so its "a route stopped it -> kill the child, free the asset"
branch was dead code. A canceled job's child ran to completion and wrote 'done'
unconditionally over the route's 'canceled'; since 'done' holds the asset but
'canceled' does not, a later delete released the asset a second time, dropping
ref_count below the live holder count and deleting a file still in use.
Add app/downloads/status.py as the single source of truth for the lifecycle
literals (JobStatus StrEnum + the HOLDING / STOPPED_BY_ROUTE sets) and use it at
the safety-critical sites: the supervisor stop check, the route cancel, and
release_asset's holding set. Make the worker's terminal done/error writes
conditional on the row still being 'running' so a child finishing just after a
route cancel/pause can no longer resurrect it. Add the missing 'canceled'
supervisor regression test.
(The full literal sweep across the downloads surface is left for the E3
downloads owner-unification; this is the data-safety subset.)
1. (correctness) Show hero rating regression: PlexBrowse's show hero rendered a
rating only when show.imdb_rating was set, so after the S6 IMDb-only change a
show with no imdb:// entry showed no rating. Now falls back to the audience
`rating` like PlexInfo.
2. (correctness) ratingMin column mismatch: the rating filter and the /facets
rating_max now key off coalesce(imdb_rating, rating) — the SAME "best score"
the badge displays — so a ratingMin picked from a shown IMDb score narrows the
right column instead of the raw audience `rating`.
3. (altitude) HU search gap: new query.vector_match() matches EITHER vector in HU
(or_(search_vector_hu, search_vector)) so a row with no HU twin stays findable
by its English title; ranking still uses the HU vector. field_recheck unions
the HU+EN per-field tsvectors in HU, staying a strict subset of the prefilter.
4. (test-coverage) Extracted the badge logic to lib/plexRating.pickRatingBadge
(shared by PlexInfo + the show hero) with a unit test (imdb-preferred, audience
fallback, 0-score, none). Backend tests added for the HU union search + the
IMDb-preferred rating filter.
5. (efficiency) Language switch no longer double-fetches the grid: browseQ is
gated on a committedLang state that lags `lang` by one render, so the
genre-clear effect settles into a single fetch instead of one throwaway fetch
with stale-language genres plus a second.
DB-lane suite 295 passed / 2 skip; FE tsc/knip/lint clean, 217 vitest pass.
E5 S6 backend half (no migration — the *_hu / rich columns exist from S5a/S5b).
Language-aware serving (?lang=hu serves the HU overlay twin, EN fallback per row):
- query.py: column pickers title_col / genres_col / vector_col; field_recheck and
meta_filter_conds take `lang`. DISPLAY/FILTER pickers coalesce(*_hu, *) (EN
fallback); SEARCH pickers use the raw *_hu column mirroring the generated
search_vector_hu (coalesce to '', not EN) so the field recheck stays a strict
subset of the GIN vector (the S2 invariant). genres_col type_coerces the coalesce
to JSONB so the GIN `.contains` survives.
- browse.py: `lang` param on /library, /facets, /item, /show (validated to en|hu).
unified_library title/vector/rank/field-scope + meta filters are language-aware
for both the movie and show branches; facets unnest the active-language genre
vocabulary (jsonb_typeof guard kept); detail endpoints serve HU title/summary/
genres/show_title with EN fallback. Episodes stay EN (not overlay-localized).
IMDb-only rating (folded-in S5b review #5):
- detail.py rich_detail: drop the audienceRating/rating fallback → imdb_rating is
NULL when there's no imdb:// Rating entry (stop mislabelling a Plex audience score
as IMDb). item_detail now also serves the `rating` column (audience score;
show_detail already did) so the UI can show it when there's no IMDb score.
Tests: test_plex_lang.py (HU title serving + EN fallback, language-specific facet
vocabulary, genre-filter language matching, item_detail lang + rating split);
test_plex_detail updated for IMDb-only. DB-lane suite 293 passed / 2 skipped; ruff clean.
Second-round /code-review high follow-ups:
- test_feed_cursor.py: GET /api/feed?cursor=<garbage> must return HTTP 400
(not 500) — locks the cursor-error path the refactor split across modules
(feedquery.decode_cursor raises ValueError, get_feed translates to 400).
Plus a no-cursor 200 sanity check. DB lane.
- get_feed: `raise HTTPException(...) from None` so the expected bad-input
ValueError isn't chained as __context__.
DB-lane suite 289 passed / 2 skipped; ruff clean.
Applies the three findings from the /code-review high pass on this stack:
1. to_tsquery_str moved from app.feedquery to app.utils — it's a generic
tsquery sanitizer with no feed dependency; housing it in the feed
read-model coupled the Plex browse module to the feed domain. Both feed
and plex now import it from the neutral util module.
2. feedquery.decode_cursor no longer raises fastapi.HTTPException — it raises
a plain ValueError, and routes/feed.get_feed translates it to the HTTP 400.
The read-model no longer imports fastapi, matching its "no request state"
docstring.
3. Dropped the leading underscore on the helpers that are now a cross-module
public API: feedquery {filtered_query, sort_keys, order_by, keyset_predicate,
encode_cursor, decode_cursor}; plex.query {TS_CONFIG, FIELD_COLUMNS,
field_recheck, split_csv, added_cutoff, meta_filter_conds, apply_meta_filters,
like_escape}; plex.cards {movie_card, show_card, episode_card, leaf_card}.
Genuinely-internal helpers (_channel_url, _saved_expr, _ADDED_WINDOWS,
_lib_key) keep their underscore.
Pure code motion + rename. DB-lane suite 287 passed / 2 skipped; ruff clean.
Second half of the 2026-08-17 review's "Plex query/cards extraction" — slim
the 1841-line browse.py god-module ahead of S6 (which reworks the library
query's column selection for the HU overlay).
- app/plex/cards.py: the pure grid/leaf card serializers (_movie_card,
_show_card, _episode_card, _leaf_card).
- app/plex/query.py: the library query-shaping layer — field-scope tsvector
rechecks (_TS_CONFIG, _FIELD_COLUMNS, _field_recheck) and the shared sidebar
metadata filters (_csv, _added_cutoff, _meta_filter_conds, _apply_meta_filters,
LibraryFilters, _like_escape). This is the seam S6 will touch.
browse.py imports both and keeps its route handlers; the entangled
collection/playlist card cluster (_collection_source/_collection_editable)
stays put deliberately. All helpers were browse-internal, so no external
importers change. Pure code motion — DB-lane suite 287 passed / 2 skipped;
ruff clean. browse.py 1841 -> 1667 lines.
B-core-3 (2026-08-17 review): decouple two things from router modules.
1. Feed read-model → new app/feedquery.py: the pure query-building and
row-serialization layer (feed_columns, serialize, _filtered_query,
to_tsquery_str, and the keyset-pagination helpers) moves out of
routes/feed.py, which keeps only the route handlers, _feed_params, and
the request models. The three external consumers (playlists, search,
plex/browse) now import from app.feedquery instead of reaching into the
feed router.
2. admin_user sweep: 7 modules imported admin_user via app.routes.admin
(an incidental re-export of app.auth.admin_user). Repoint them all to
the definition in app.auth (audit, channels, config, quota, scheduler,
plex/admin, plex/browse); routes.admin keeps its own import for its use.
Pure code motion — no behaviour change. Full DB-lane suite 287 passed /
2 skipped; ruff clean.
Address the /code-review high finding on S5b: gating the info-page cast enrich on
`people_enriched_at is None` removed the old per-view regrow of `cast_names`, so a routine full sync
that reset it to the listing top-20 (`_apply_facet_fields`) — while never clearing people_enriched_at —
left it permanently truncated, silently dropping cast past the top ~20 from the S4 person-page filter.
Root fix: `_apply_facet_fields` now only takes the listing's cast for a NOT-yet-enriched row; an
enriched row keeps its fuller stored cast (people_text still rebuilt from the updated directors + the
preserved cast). Makes the people_enriched_at flag the endpoints/backfill trust actually reliable.
Tests: cast_names preserved for an enriched row vs applied for a fresh one; show_detail lazy-fill
persists imdb/full_cast on first view. Full suite green.
Persist the info-page rich fields that were previously read LIVE from plex.metadata() on every view —
imdb_rating (IMDb-specific, vs `rating`=audienceRating), imdb_id, tagline, and full_cast
([{name,role,thumb}]) — onto plex_items/plex_shows, and serve the detail endpoints from the row.
- migration 0065: ADD imdb_rating/imdb_id/tagline/full_cast (nullable → no rewrite/lock, unlike 0064);
NULL people_enriched_at on movie/show rows so the existing cast backfill re-fetches each once and
fills the new columns.
- new app/plex/detail.py: rich_detail(meta) (moved out of browse._rich_meta so sync + serve share one
parser) + apply_rich_detail(row, meta); wired into _enrich_people_batch and the info-page lazy fill.
- item_detail: movies serve cast/IMDb/tagline/genres from the row and keep plex.metadata(markers=True)
ONLY for markers + audio/subtitle streams (so detail survives a live-Plex outage); episodes (never
enriched) still parse detail from that live meta.
- show_detail: serves cast/IMDb from the row; `related` stays live best-effort and also lazily fills a
not-yet-enriched show.
- tests: rich_detail/apply_rich_detail parsing + persistence, backfill stamping, item_detail
outage-resilience + lazy fill + episode-live, show_detail row serving. Full suite 285 pass / 2 skip;
migration verified reversible (up→down→up).
Precondition for S6's pure-DB language-aware serving.
- Correctness: _sync_overlay now first-wins on a guid (a `seen` set) so a duplicate HU entry
sharing a guid can't overwrite the applied Hungarian metadata or double-count `overlaid`.
- Test-coverage: cover the earlier dup-guid fix (both editions of a shared-guid pair get *_hu,
counted once) and the guid-keep guard (_apply_item preserves a prior guid when the listing
carries none, still updates on a real one).
Full backend suite 276 passed / 2 skipped.
- Correctness (stale HU): _sync_overlay is now a full reconcile — it clears every guid-bearing EN
row's *_hu first, then repopulates the matches, so a title whose HU twin vanished or was re-guid'd
falls back to EN instead of keeping stale Hungarian metadata (only rows that held HU data go dirty).
- Correctness (dup guid): by_guid maps a guid to the LIST of EN rows carrying it, so two editions of
one film both receive the overlay instead of just the last-iterated one.
- Robustness (guid wipe): _apply_item/_sync_shows keep a prior guid when a listing carries none
(`_canonical_guid(meta) or row.guid`), so a server ignoring includeGuids can't erase the match key.
- Reuse: merged _sync_overlay's movie/show branches (only the select + item_type differ).
- Test-coverage: a full-sync test drives sync() over a stub Plex (sections + section_items +
overlay_map) asserting pass-2 designation (metadata_lang/overlay_of), guid capture, the HU write,
and browse exclusion; plus a reconcile test that a vanished twin clears stale *_hu.
Full backend suite 274 passed / 2 skipped.
Adds the storage + sync for the Hungarian-metadata overlay (the browsable EN sections gain HU
twin fields written from a paired HU section, matched by guid; no browsable HU rows).
Schema (2 migrations, ADD-only; run before the first HU sync while *_hu is NULL so the generated
column build is trivial):
- 0063: plex_libraries.metadata_lang ('en' default) + self-FK overlay_of.
- 0064: plex_items/plex_shows.guid (indexed) + title_hu/summary_hu/genres_hu (+ GIN) + a STORED
generated search_vector_hu mirroring the EN vector (title_hu A / people_text B / summary_hu C) + GIN.
Sync:
- includeGuids=1 on section_items so the canonical guid rides the cheap listing.
- _canonical_guid(meta) (tmdb > imdb > tvdb) captured onto every EN row in _apply_item/_sync_shows.
- Two-pass sync(): pass 1 mirrors browsable (non-overlay) sections; pass 2 reads plex_overlay_map
('huKey:enKey', new sysconfig), designates each HU section (metadata_lang='hu' + overlay_of), and
_sync_overlay writes title_hu/summary_hu/genres_hu onto the guid-matched EN rows.
Browse: _content_libs now excludes overlay libraries (overlay_of is None); the unified_library and
facets inline library partitions fold into it, and list_libraries/collections exclude overlays too.
Tests: DB-lane coverage of the overlay write (matched vs unmatched rows, no browsable HU rows, the
generated search_vector_hu populates), the guid parse, and the browse exclusion. Full suite 272
passed; migrations verified reversible (upgrade→downgrade→upgrade). S5b (persist the live detail
fields + serve item/show detail from DB) is the follow-up.
- Efficiency: cap each person section at _PERSON_SECTION_MAX (200) so a prolific person can't
return an unbounded list the client mounts a card per row for.
- Reuse: extract the optimistic watch-toggle (flip → persist → resync) into a shared
toggleWatchState helper; the grid and the person page each supply their own cache updater.
- Correctness (scroll): onOpenPerson now stashes infoScrollRef ONLY when leaving an info page
(not from a show/grid context), and a jumped-to info page resets it — so a stale offset can't
wrongly restore a fresh info page.
- Test-coverage: add PlexPersonView.dom.test.tsx (export CastStrip + PlexPersonView) covering
the cast-row rewiring (open-person vs filter fallback) and the person page's sections + filter
+ empty state.
The name-as-identity finding is inherent to Plex cast carrying no per-person id (PlexPerson is
name-keyed too); documented in the endpoint docstring, not structurally fixable in S4.
Backend: new GET /api/plex/person/{name} returns a cast/crew member's cached headshot plus
the movies (PlexItem, kind=movie) and shows (PlexShow) in the enabled libraries whose
cast_names OR directors include the name — the inverse of the cast → filter link. Introduce a
shared _content_libs(db) helper (enabled-library partition) and reuse it in people_cards;
S5 will extend it with the HU-overlay exclusion. Watch-state comes from PlexState / rollup
(reusing _movie_card / _show_card / state_map). DB-lane tests cover cast+director credits,
library scoping, the kind filter, unknown-name empties, and the partition helper.
Frontend: a new `person` history-subview. A cast-row click (PlexInfo cast row + the show
page's CastStrip) now opens the person's filmography page instead of filtering; the filter
gesture moves to a "Filter library by {name}" button on the page itself. PlexPersonView
mirrors PlexInfoView's fetch-wrapper + glass hero (headshot + name + credit counts) and
renders Movies / Shows as PlexPosterCard grids, with an optimistic per-card watch toggle on
the person cache. New plexPerson query key + api client + PlexPersonDetail type; HU/EN copy.
peter