Compare commits

...
444 Commits
Author SHA1 Message Date
peter 94c62ca699 release: 0.78.6 (feat/admin-invite-delete-email-health)
gate / frontend (push) Failing after 1m43s
gate / backend (push) Successful in 2m53s
2026-09-02 00:36:52 +02:00
peter 1fe24d374a docs(tests): correct the deny-vs-delete rationale in the invites test file
Round 4's only finding: the file header and one comment still said Deny stops a
refused person re-asking. It does not — upsert_pending_invite reopens a denied
row as pending — and the delete_invite docstring in the same branch already says
so. The real difference is that Deny records an auditable decision and mails the
requester; Delete forgets the address.
2026-09-02 00:15:21 +02:00
peter 2cba6b9fca docs: round-3 copy corrections, and pin the two behaviours nothing was guarding
Round 3's correctness/security pass came back clean; what was left said more than
the code does.

- emailOff promised the access-request list is where sign-ups show up. A sign-up
  on an address that is already approved or already pending adds nothing to it
  (roadmap UX-25), and "only" was falsified by this release's own inbox notice.
- deleteConfirm claimed the admin was dropping "an approval you already gave" —
  it is the confirmation for pending and denied rows too.
- sendFailed blamed the mail server for rejecting a message; the reason covers
  connect, DNS, TLS and login failures alike, so it now describes the symptom and
  leaves the diagnosis to the log.
- ADMIN_EMAILS re-promotion happens in the Google callback only, not on any
  sign-in.

Tests: the decided-list copy of the delete affordance was never rendered (both
new cases fail if that branch is dropped), and the throttle release — a
deliberate behaviour the notes advertise — was asserted by nothing; a drop with
no admin yet, or with a failing write, must not silence the following hour.
2026-09-02 00:06:01 +02:00
peter 8d7aaad9a8 fix: warn before the delete, not after — plus round-2 copy and a11y corrections
Round 2 on the feature diff (15 → 12):

- still_allowed_by_env now rides the invite LIST as well as the delete response,
  so the confirmation warns while the admin can still change their mind. It
  mattered most for a DENIED row on an env-allow-listed address: deleting it
  silently converted a refusal into a grant, and the only warning arrived after
  the irreversible click.
- Both inbox bodies claimed to name "the most recent" dropped message. The
  throttle records the FIRST of each hour and never updates it, so they now say
  "one of them", and the not-configured body finally names where to fix it.
- settings.invites.emailOff told the admin they "get no notice when someone
  signs up" — rendered directly above the queue that shows exactly that. Only
  the email is lost, and it now says so.
- The disabled delete buttons put their reason out of reach: `disabled` drops a
  control from the tab order and suppresses its title. Switched to
  aria-disabled + preventDefault, the pattern Playlists.tsx already established
  for the same reason.
- deleteInvite is quiet, so the 409 the UI handles with a translated toast no
  longer also raises the server's English detail as a modal.
- After a completed password reset, Back landed on the reset form again with a
  consumed token — the history change made that state reachable. The token is
  cleared on success so the reset view stops being the card's root.
- The hu deleteHasAccount named a tab that renders under a different label.

Tests: a dom test for the Access-requests panel covering both has_account
variants and all three delete outcomes, /api/me flag cases, an ADMIN_EMAILS
case, and the env-allowlist test now patches the raw setting so the parsing
stays under test.
2026-09-01 23:50:29 +02:00
peter 85043013f0 fix: address the feature review — accurate copy, reachable controls, real coverage
Round 1 on the feature diff found 15; the substantive ones:

- The delete control VANISHED on any invite whose address had registered, which
  is the ordinary sign-up shape (register writes the User and the pending invite
  together). The guard stays broad — that row is the account's approval gate, and
  orphaning it strands someone who can then neither sign in nor be approved — but
  the control is now a disabled button carrying the reason, on both lists, so an
  admin never sees a row silently missing a button its neighbours have.
- Deleting an entry is not always a revoke: is_allowed falls back to the env
  ALLOWED_EMAILS / ADMIN_EMAILS lists once no row exists, and an ADMIN_EMAILS
  address is force-promoted on its next Google sign-in. The route now reports
  still_allowed_by_env and the UI says so instead of claiming access was removed.
- "Deny keeps the row so they can't immediately re-ask" was simply false —
  upsert_pending_invite reopens a denied row. Corrected everywhere it was stated.
- The Configuration warning reads me.email_configured, which save() did not
  invalidate, so it kept warning about fields the admin had just filled in.
- The read-only variant was an aria-hidden span explained only by a Tooltip that
  the hints toggle can switch off; now a disabled button with title + aria-label.
- The delete error path threw away the 409's reason (the stale-list case) and
  left the stale row on screen.
- The throttle window was spent even when no notice was written, silencing the
  next hour on the path whose job is to stop silent drops.
- email_enabled() opened a second session and decrypted the SMTP password on
  every /api/me poll; it now takes the caller's session.
- welcomeMode.dom.test.ts never rendered the card, so reverting the history
  wiring to useState left it green. A render test now drives the real links and
  fails on that revert (verified).

Plus /api/me flag tests, the registration-shaped pending case, the env-allowlist
case, and a release note that matches what the code does.
2026-09-01 23:31:49 +02:00
peter 44f626f732 feat(admin): delete a whitelist entry, and stop hiding a broken mail setup
Three operator gaps, all found the same way — a person could not sign up and the
app showed nobody anything.

- DELETE /api/admin/invites/{id} with a delete control on both the pending
  requests and the previously-decided list. Deny keeps the row so the person
  cannot immediately re-ask; delete forgets the address. Refused with 409 while
  a User row exists for it, because is_allowed reads that row and removing it
  would revoke a live user's access with no other sign — the list ships
  has_account so the UI withholds the button instead of offering one that fails.
- Email is fail-soft by design, which had come to mean invisible. _send() now
  records an email_undelivered notification for every admin when a message is
  skipped or rejected, naming the message that was lost, throttled to one an
  hour. /api/me carries email_configured, driving a standing warning on
  Configuration → Email / SMTP and above the access-request list.
- The sign-in card's Create account / Forgot password / Didn't get the email
  views are history entries (useHistorySubview), so Back returns to the previous
  view instead of only moving the address bar.

Also: the per-test rate-limiter reset sweeps every loaded app.* module rather
than a hand-kept list of three — this change added a limiter outside it.
2026-09-01 23:09:25 +02:00
peter 24c93cbb83 docs: name the Configuration tab as it is rendered, drop an unused fixture
Round 6's two remaining nits. The operator note said "Configuration → Email"
but the group renders as "Email / SMTP" in both locales, and
test_an_admin_can_activate_a_pre_approved_registrant requested make_user without
using it — a leftover from the reverted rounds that made the test look like it
seeded an admin when it did not.
2026-09-01 22:15:56 +02:00
peter cb1f4f00b0 docs(auth): correct the last round of over-claiming comments and notes
Round 5's correctness/security pass came back clean; what was left was text that
said more than the code does.

- The register-path notify comment (and its test) claimed the admin "is still
  told". The call fires, but on a no-SMTP instance _send drops it — which is the
  open half of the problem, not something this branch fixed. Both now say so.
- The release notes tell admins what to actually do: an account whose address was
  whitelisted in advance has no pending request to approve, so the whitelist box
  is what activates it. That instruction only existed in the roadmap and in
  chores, which the dialog does not render.
- "That one line is the whole production change" now accounts for the test-only
  RateLimiter.reset() that also ships.
- The rate-limiter test claimed to pin that blocked attempts are not counted, on
  an assertion that held either way. Replaced with one that drives the clock
  across the window boundary and fails if hammering extends it.
2026-09-01 22:02:43 +02:00
peter f6e1d1e69f fix(auth): keep the admin notice unconditional on the register path
Round 4 showed the `inv is not None` gate added in b5a1002 removed a signal that
matters: for an address the admin had already approved, that email is their only
notice that someone has now created an account on it — the invite is not pending
so nothing lands in the Access-requests queue either. Suppressing it left the
registrant at "awaiting admin approval" with the admin unaware they existed. The
duplicate it avoided (Request access, then sign up) is mild noise by comparison.

The production change is now one db.commit(), which is all the reported bug ever
needed. Release notes corrected again: a previously-denied address was also
unaffected, the pending-request case never lost its queue entry (only the
account behind it), and the test inventory was missing a guard.
2026-09-01 21:48:40 +02:00
peter b64c3f514d fix(auth): drop the invite reopen — keep the fix to the lost account alone
Round 3 of the review killed the reopen from b5a1002: flipping an approved
invite back to pending let an unauthenticated POST /auth/register REVOKE a
standing approval for any whitelisted-but-unclaimed address, which then fails
is_allowed() and locks the real owner out of Google sign-in — the primary way
people get in here. It also re-armed the pre-emption window it was meant to
close, because with no SMTP email_verified is set True and the SEC-10 guard the
design leaned on never fires.

Two attempts at fixing the stranded-account UX in the same change both came back
worse than the wart. What remains is the narrow, real bug: the account was
silently rolled back and is now committed. An admin still binds the approval to
the account, exactly as before this branch. UX-25 in the roadmap records the
follow-up and both dead ends so they are not retried.

Also from round 3: the upsert docstring no longer claims the admin UI cannot
reach these accounts (re-entering the address in the same panel's whitelist box
does, and a test now pins it), the release notes are scoped to what the diff
actually changes with the operator note moved where the dialog renders it, and
RateLimiter has unit tests so reset() is not production code only a fixture
calls.
2026-09-01 21:34:38 +02:00
peter b5a1002412 fix(auth): reopen a pre-dated approval instead of auto-granting it
Replaces the activation added in ea931dc, which round 2 of the review showed was
a real weakening: on a no-SMTP instance it made a standing whitelist entry a
bearer grant — the first anonymous party to register an approved-but-unclaimed
address got an immediately usable account under that identity, with no
attestation, no admin action and no signal anywhere. The verify-time activation
had the same shape one step removed, and neither site ran the SEC-10 preset-
password guard or left an audit trail.

Keep a human in the loop instead. An approval granted before any account existed
was a decision about the ADDRESS, so registration reopens the invite
(upsert_pending_invite gains revive_approved, used only by the register worker).
The request lands in the Access-requests queue, approving it runs
_activate_user_by_email — and with it revoke_preset_password_if_unverified — and
the account activates. One mechanism, already audited, no new grant path, and
the "awaiting admin approval" copy is honest again.

The explicit commit from the previous commit still matters: with an invite
already pending, upsert returns without committing and _issue_token is gated on
SMTP, so a no-SMTP registration was still being rolled back.

Also from round 2: gate the admin notice on a fresh decision (matching the two
other upsert callers) so a retry no longer re-pings every admin, and reset the
module-level rate limiters between tests — the suite pooled every login into one
300s window and was two logins short of failing an unrelated test with a 429.
2026-09-01 21:18:14 +02:00
peter ea931dca30 fix(auth): honour an approval granted before sign-up instead of parking the account
Follow-up to the commit fix, from the deep-review. Persisting the account was
only half the story: when the admin had approved the address BEFORE the person
registered, the account landed is_active=False against an invite that is no
longer pending — absent from the Access-requests queue, and approve_invite 409s
on an already-decided invite. The sign-up worked and still could not sign in.

Derive activation from is_allowed() at the moment ownership is attested:
- verify_email_confirm activates a pre-approved account when the emailed link is
  confirmed (only the mailbox owner can follow it), so SEC-10 stays intact;
- _register_account activates only when there is no SMTP to verify with, where
  the admin whitelist is by design the sole gate — the same one add_invite
  already activates on.

Tests: the pre-approved no-SMTP case and the verify-activates case are real
regression tests (both fail without this change); guards cover the unknown-email
and denied->pending branches, verification of an unapproved account, and the
first end-to-end coverage of the /auth/register route and its
allow_registration gate. Release notes rewritten — the previous entry miscounted
its regression tests and advertised an email fix this release has no code for.
2026-09-01 20:56:43 +02:00
peter d954e71a77 fix(auth): commit the account on registration so a pre-approved sign-up isn't lost
_register_account() flushed the new User but relied on someone else to commit:
upsert_pending_invite() commits only when it actually writes an invite row, and
_issue_token() runs only when SMTP is enabled. For an email an admin had ALREADY
approved, on an instance where email is unavailable, neither fired — the session
closed and the account was rolled back. The would-be user got the uniform "ok"
and no account, could not sign in, and re-registering was a silent no-op.

Commit explicitly after the invite upsert (a no-op when upsert already committed
the same transaction). Three DB-lane regression tests cover the pre-approved
no-SMTP path (fails without the fix), the unknown-email path that always worked,
and the pre-approved path with SMTP on.
2026-09-01 16:25:05 +02:00
peter 602f62967a release: 0.78.4 (bug/plex-transcode-hls-startposition)
gate / frontend (push) Failing after 1m55s
gate / backend (push) Successful in 2m57s
2026-08-30 20:33:15 +02:00
peter ba00b50dea release: 0.78.4 — fix Plex transcode playback stall/rebuffer
hls.js startPosition:0 so the transcode/remux HLS session starts VOD-style
at the requested position instead of chasing the live edge of the growing
EVENT playlist.
2026-08-30 20:32:56 +02:00
peter 92adb593c8 fix(plex): start hls.js at position 0 for the transcode/remux HLS session
Our HLS is a growing EVENT playlist with no ENDLIST, which hls.js reads as
LIVE. Its default startPosition (-1) then starts at the live edge and hugs it
(no forward buffer → rebuffer spinners), and on a resume/seek session the
segments' absolute PTS (-copyts, offset by the server's -ss) don't line up
with the live-edge timeline → 0 fragments load → permanent stall. The
seek-restart model makes fragment 0 the requested content position, so start
VOD-style at 0 and buffer forward. Fixes the transcode playback stall/rebuffer
found in 0.78.2 UAT.
2026-08-30 20:09:00 +02:00
peter 66132b6dda release: 0.78.3 (feature/plex-playback-class-filter)
gate / frontend (push) Failing after 1m36s
gate / backend (push) Successful in 2m35s
2026-08-30 19:09:22 +02:00
peter d09f021f0c release: 0.78.3 — filter the Plex feed by playback class 2026-08-30 19:09:08 +02:00
peter d7e9240e54 merge main (0.78.2) into playback-class before release
# Conflicts:
#	backend/app/plex/stream.py
2026-08-30 19:08:08 +02:00
peter f2fa49b778 release: 0.78.2 (feature/plex-hdr-hw-tonemap)
gate / frontend (push) Failing after 1m37s
gate / backend (push) Successful in 2m24s
2026-08-30 18:56:32 +02:00
peter d4ed45b2eb release: 0.78.2 — HW HDR→SDR tonemap for 10-bit/HDR Plex transcode 2026-08-30 18:56:18 +02:00
peter 375f6e58be merge main (0.78.1) into tonemap before release 2026-08-30 18:55:14 +02:00
peter 1a102ce81a release: 0.78.1 (bug/download-mediakind-probe)
gate / frontend (push) Failing after 1m36s
gate / backend (push) Successful in 2m22s
2026-08-30 18:47:05 +02:00
peter bfd45a155e release: 0.78.1 — download media_kind from a file probe (BUG 2) 2026-08-30 18:46:33 +02:00
peter 1102ad6a28 fix(downloads): make the probe verdict authoritative over yt-dlp info by probe-ran, not per-field
Deep-review round 1 on bug/download-mediakind-probe found a real HIGH the prior
review missed:

- resolve_media_facts used a per-field `probe or info` fallback, so a probe that
  RAN and deliberately found no video track (cover-art-only audio: attached_pic
  skipped; a still image: format-guarded) had its None video overridden by
  yt-dlp's stale vcodec. An audio-mode download from a progressive source (info
  carries the source's video codec, embed-thumbnail on) was then classified VIDEO
  -> a Play button + a broken in-app <video> with no video track. Only the
  download path regressed (upload calls probe_media directly).

  Fix: gate precedence on whether ffprobe RAN, not per-field. If the probe read
  the file, its codec verdict wins outright (a None means 'no such track'); only a
  full ffprobe failure falls back to info. Dimensions still gap-fill from info.
  Refactored the parsing into _facts_from_probe so resolve_media_facts can see the
  raw probe-ran signal; probe_media keeps its 5-tuple (upload + TestProbeClassifier
  unchanged).

- tests: the cover-art-audio and still-image resurrection cases, a differing
  populated-info precedence case (the prior-flagged gap), and the ffprobe-failed
  info fallback — now stubbing run_ffprobe_json so the real parsing + precedence
  run together.

Backend gate green (ruff + full pytest).
2026-08-30 18:09:26 +02:00
peter c1a3b7b935 fix(plex): gate the part-key re-probe to single-version items; drop dead i18n key; cover cross-facet narrowing
Deep-review round 2 on the round-1 fixes (all low):

- part_key churn (state): round 1's part-key reset re-probed EVERY sync for a
  multi-version item (an original + a Plex 'Optimized' copy), because
  _media_facts reads media[0]/part[0] whose order Plex doesn't guarantee stable,
  oscillating transcode_hw between the two versions. Gate the part-key signal on
  a single-version item (len(Media)<=1); codec-change reset still applies to all.
  A multi-version same-codec re-encode not self-healing is an acceptable rare
  edge; a per-sync re-probe loop is not.

- dead i18n key: this diff's playableClass() collapses to transcode_hw/sw, so the
  bare plex.playable.transcode key it orphaned is now unused (knip can't see
  dynamic keys). Dropped from both locales, parity kept.

- test: cross-facet narrowing by an active playable class (a co-facet's year
  range collapses to the selected subset) — the flip side of self-exclusion,
  previously unpinned.

The aria-label+title double-announcement on the card dot is a deliberate accepted
trade-off (reliable SR name + mouse tooltip). Backend + FE gates green.
2026-08-30 17:58:38 +02:00
peter bf98b48517 fix(plex): re-probe on file replace, HU parity, public class const, playback-class test gaps
Deep-review round 1 on feature/plex-playback-class-filter:

- Sticky watermark (medium): _apply_item reset the bit-depth probe watermark
  only on a codec-string change, so a same-codec re-encode (8-bit->10-bit HEVC)
  kept a stale video_bit_depth/transcode_hw forever (the backfill only re-probes
  media_probed_at IS NULL rows). Also reset when the Plex Part key changes (the
  file was replaced), not just the codec.

- i18n parity (medium): the HU playback-class labels were mixed-language (only
  'direct' translated). Translated remux/transcode_hw/transcode_sw and the
  tooltips consistently to Hungarian (Kozvetlen streameles / Atkodolas).

- Layering (low): _PLAYBACK_CLASSES was module-private yet imported across into
  routes.plex.browse; renamed to public PLAYBACK_CLASSES (it is the shared
  source of truth for facet order + zero-fill).

- a11y (low): the browse-card playback-class dot conveyed meaning only via a
  mouse-hover title; added role=img + aria-label so SR/touch users get the class.

- tests: the NULL/unprobed transcode_hw -> SW fold (filter + facet), facet
  self-exclusion under an active playable filter, the migration codec-set ==
  HW_DECODABLE_CODECS drift guard, the probe-batch bound + oldest-first order,
  the transient probe-failure retry path, and the same-codec re-encode re-probe.

Backend gate green (ruff + full pytest); FE gate green (tsc/lint/knip/prettier/
vitest). NOTE: the facet-vs-runtime disagreement for 10-bit HEVC (C1) is the
known merge-order constraint — this branch's runtime is codec-only; the facet
becomes truthful once BUG1/tonemap's bit-depth-aware routing lands with it. Left
as a sequencing constraint, not code-changed here.
2026-08-30 17:51:33 +02:00
peter b563155a37 docs(plex): correct the cull-yield fallback comment to name the real route 404s
Deep-review round 3: the only surviving finding — a comment-drift nit. The
fallback's cull-yield comment named the route's 'Not available yet' path, but a
culled-key segment request actually returns 'No active playback session' (the
substantive invariant — never a None that 404s as file-not-found — is unchanged).
Reworded to match the route. Concurrency + correctness came back clean.
2026-08-30 15:18:26 +02:00
peter 915319fa35 fix(plex): make the HW->SW fallback supersede-check atomic; honest cull yield
Deep-review round 2 on the round-1 fix:

- Race (medium): the round-1 identity guard did the supersede check and the
  respawn in TWO separate _lock sections. A newer same-key session installed in
  the gap was still torn down by _spawn's unconditional pop+kill (the round-1
  comment even had the direction backwards). Fold the check into _spawn via a
  new replace_only= param: the identity test and the pop+replace now share ONE
  lock, so the respawn is a true no-op when superseded/culled.

- Honest states (low): the superseded branch returned current_session(key),
  which is None when the key was cap-culled during our out-of-lock wait; the
  route turns None into a dishonest 'Media file not found on disk' 404 for a
  file that exists. Yield 'current_session(key) or s' so a culled session
  degrades to the route's retryable 'not available yet' path instead.

- Hygiene: _want_hw docstring corrected (it is a test-only shim now; production
  decides via _select_encode) and the two stale _want_hw references refreshed
  (stream.py header, config.py plex_transcode_hw).

- tests: the cap-cull no-respawn/no-None sub-case; the _opencl_present except
  OSError branch (glob raising, not just empty); tightened a tautological -vf
  assertion. Backend gate green (ruff + full pytest).
2026-08-30 15:11:37 +02:00
peter a2a8f5681d fix(plex): don't tonemap 10-bit SDR; identity-guard the HW→SW fallback
Deep-review round 1 on feature/plex-hdr-hw-tonemap:

- F1 (correctness): _select_encode gated tonemap on bit-depth alone, so a
  10-bit *SDR* (bt709) source got the HDR hable tone curve applied, visibly
  altering an SDR picture where only a 10->8-bit downconvert was needed. Split
  the ffprobe gate into a 3-way classifier (_probe_video_kind: hwsafe / hdr /
  sdr10); only true-HDR (PQ/HLG) takes the OpenCL tonemap graph, 10-bit SDR
  drops to software libx264 (an exact downconvert). Applies to both auto and on.

- F2 (race): the runtime HW->SW fallback popped+killed _sessions[key] with no
  identity check. Because wait_for blocks OUTSIDE _lock, a concurrent same-key
  _spawn (rapid seek-restart) or _enforce_cap cull could kill this session; the
  fallback then respawned software over the NEWER session and breached the
  transcode cap. Guard the respawn: only fall back if _sessions[key] is still
  this session, else yield the currently-registered one.

- tests: 3-way classifier (incl. HLG + empty-probe), real _opencl_present glob,
  real wait_for proc fast-fail branch, tonemap-variant fallback, the F1
  regressions (auto + on), the supersede/no-respawn race, and the on/opencl
  matrix cell. Backend gate green (ruff + full pytest).
2026-08-30 15:02:41 +02:00
peter 4827aebb7f feat(plex): HW HDR→SDR tonemap for 10-bit/HDR transcode (OpenCL on the iGPU)
Upgrade mode-3's 10-bit/HDR branch from "route to software libx264" (BUG 1's
behaviour) to a full-GPU pipeline: VAAPI decode → OpenCL tonemap (hable,
HDR10/HLG → bt709 SDR nv12) → h264_vaapi. Gen 9.5 (UHD 630) can't tonemap in
the VAAPI VPP (tonemap_vaapi fails), so OpenCL is the only HW route — proven on
the prod iGPU at ~4.2x realtime, ~1 CPU core.

- stream.py: `_select_encode` returns (hw, tonemap); a 10-bit/HDR HW-decodable
  source picks the tonemap graph when the OpenCL runtime is present, else drops
  to software. `_opencl_present` gates on a registered ICD. `_venc_opts` /
  `_hwaccel_in_opts` gain the tonemap variant (named vaapi+opencl devices,
  `-filter_hw_device ocl`, hwmap→tonemap_opencl→hwmap reverse). `_want_hw` kept
  as the hw-half wrapper. The runtime HW→SW fallback still backstops a session
  that emits no segment.
- Dockerfile: add the Intel NEO OpenCL runtime (intel-opencl-icd). It was removed
  from Debian trixie, so it's pulled from bookworm (22.43 still supports Gen 9.5;
  the .deb runs on the trixie base — verified live). Harmless without /dev/dri.
- route + FE: thread `tonemap` through the session response; the stats overlay
  mode row reads "transcode · GPU · HDR→SDR" for the tonemap path.
- tests: tonemap command branch, the (hw, tonemap) selection matrix (auto/on ×
  8-bit/10-bit × opencl), start_session tonemap wiring, and the fmtMode tag.
2026-08-30 06:02:45 +02:00
peter 840526b1b8 feat(plex): filter the feed by playback class (Direct Play / Direct Stream / Transcode HW/SW)
Adds a movie-scoped "Playback" filter to the Plex feed so a user can isolate items by how
they play — the groundwork for collecting HW-transcodable (or no-transcode) versions.

Four classes derived from the stored `playable` + a new `transcode_hw` facet:
- Direct Play (direct), Direct Stream (remux)
- Transcode · HW  — playable=transcode AND the iGPU can hardware-transcode it
- Transcode · SW  — playable=transcode, software libx264 only

The HW/SW split is precise, not codec-only: a 10-bit/HDR HEVC is HW-decodable yet breaks the
VAAPI nv12 pipeline (the same case plex/stream special-cases), so it must count as SW.

Backend:
- New PlexItem columns (migration 0077): transcode_hw, video_bit_depth, media_probed_at.
  Migration backfills transcode_hw codec-only so the filter works immediately.
- `plex.sync.transcode_hw_capable` is the single classifier; the HW-decodable codec set moves
  to sync as `HW_DECODABLE_CODECS` and plex.stream imports it, so the catalog facet and the
  runtime encoder choice can't disagree by codec.
- Bit-depth backfill (`_probe_media_batch`, wired into sync): probes only HW-decodable transcode
  items' video Stream for bit-depth + HDR and refines a 10-bit/HDR HEVC to software. Bounded
  per run, watermarked by media_probed_at. A codec change resets the watermark (stale probe).
- Filter + self-excluded facet on a shared `playback_class_expr` CASE (movie-leg only — a show
  is an aggregate with no single class). transcode_hw threaded onto the feed/detail/card JSON.

Frontend:
- Multi-select "Playback" Combobox in the sidebar (movie scope, facet-empty drop-out), active
  chips, 4-way corner-dot badge (green/blue/amber/red) + tooltip, and an info-page playback chip
  with a codec · bit-depth subtitle. New `bg-info` (blue) token, i18n HU/EN parity.
- Fix a regression where the card's playback dot lost its hover tooltip: the full-card reveal
  overlay swallowed the cursor; the dot now sits above it (z-20).

Gate: backend full lane + frontend typecheck/lint/knip/prettier/vitest all green. Browser-UAT'd
on localdev (filter narrows to the SW set, facet counts, badge colour + tooltip, info chip).
2026-08-30 04:45:11 +02:00
peter 10d69cb316 fix(plex): route 10-bit/HDR HEVC to software + runtime VAAPI→libx264 fallback
A 10-bit/HDR HEVC source (e.g. an HDR10 x265 file) died at scale_vaapi=format=nv12
on the UHD 630 (Gen 9.5 VPP: "VAProfile not supported"); ffmpeg produced 0 segments
and the item never played, with no fallback — a retry just re-picked the failing HW
path. An 8-bit HEVC and the same file on libx264 both work.

Two defenses:
- `_want_hw` auto mode now also gates on an 8-bit-SDR source via a new
  `_probe_video_hwsafe` ffprobe (bit depth from pix_fmt/bits_per_raw_sample, HDR from
  color_transfer) — a 10-bit/HDR source takes the software libx264 path, which
  downconverts to yuv420p cleanly. The Plex mirror stores no bit-depth signal, so this
  is probed at session start. "on" still trusts the admin and skips the gate.
- `start_session` adds a runtime HW→SW fallback: a HW session that emits no first
  segment respawns on libx264 and reports hw=False (the route/stats overlay honestly
  show CPU). `wait_for` gained an optional `proc` so a dead ffmpeg fails fast instead
  of burning the 25s budget before the fallback fires.

Tests: TestProbeVideoHwsafe (parsing + probe-failure-assumes-safe), the 10-bit auto
gate + "on" skip in TestWantHw, and the no-segment HW→SW respawn in
TestTranscodeStartSession. Backend lane green.
2026-08-30 03:53:31 +02:00
peter a86131e062 fix(downloads): classify media_kind from a file probe, not yt-dlp info
A downloaded video was silently demoted to a download-only "file" (no
in-app play/edit) whenever yt-dlp's info dict omitted vcodec/acodec — which
it does for Facebook and other sources. Finalize derived media_kind from
those nulls (`derive(None, None)` -> file), so a genuine H.264/AAC MP4
became non-playable in the library.

Fix: probe the finished file on disk (the ground truth) at finalize and
derive from that, falling back to the info dict only for a field the probe
couldn't read. This is the same ffprobe the upload ingest already used, now
extracted to a shared downloads/probe.py so the download worker and the
upload path share one classifier (and can't drift). Also backfills the
stored vcodec/acodec/width/height from the real file.

Tests: TestResolveMediaFacts pins probe-wins-over-info (incl. the Facebook
no-codec case and yt-dlp's literal "none"); upload's probe tests repoint to
the shared module.
2026-08-30 03:32:14 +02:00
peter a84f2db310 release: 0.78.0 (feature/e6-plex-player-polish)
gate / frontend (push) Failing after 1m35s
gate / backend (push) Successful in 2m20s
2026-08-30 02:26:25 +02:00
peter 872d91f540 release: 0.78.0 — Plex transcode playback + player polish (E6) 2026-08-30 02:25:20 +02:00
peter 4eaa68ceeb feat(plex): S1a — add intel-media-va-driver for VAAPI transcode (E6)
The runtime image ships ffmpeg with h264_vaapi/libva2 but no VA driver, so
h264_vaapi can't initialise and a Plex transcode silently falls back to
libx264. Add the iHD driver (intel-media-va-driver) so the UHD 630 iGPU can
hardware-encode once /dev/dri is passed into LXC 118 (host + compose side,
tracked in siftlode-ops). Harmless where no render node is present.
2026-08-30 02:14:06 +02:00
peter 00539a4d97 fix(plex): deep-review follow-ups on E6 transcode + player polish
Eight verified findings from the sprint deep-review:

- Transcode cap: _enforce_cap now counts only transcode sessions, so a
  live remux (stream-copy) stream can no longer be LRU-evicted mid-play —
  matching the config comment + the HU/EN Config hint that promise remux
  is uncapped.
- Player Back: the browser/mouse Back interceptor (BackClose) now also
  mounts for the stats overlay and up-next card, so Back closes them first
  instead of ejecting the whole player (keyboard/Esc already did).
- Honest cold-start: loadSession clears sessionIsTranscode up front, so a
  transcode->direct/remux switch no longer flashes a stale "Preparing
  stream..." during the session request.
- Stats overlay a11y: the close (X) buttons in the stats, help and info
  overlays now read as "Close"/"Bezaras" (new plex.player.close key), not
  "Back"; dropped aria-live from the per-second stat grid so a screen
  reader is not flooded with telemetry.

Tests: route-level transcode routing (transcode item -> HLS re-encode
session, remux stays copy, direct opens none); start_session threading the
preset/crf/max_height knobs + hw/preexec into the ffmpeg argv; UpNextCard
countdown auto-advance, Play-now, Cancel and the S?-E? heading fallback.

Also brings UpNextCard.tsx / plexStats.ts / plexStats.test.ts to prettier.
2026-08-30 02:04:17 +02:00
peter dc24d353e0 feat(plex): honest transcode cold-start + encoder in stats (E6 S1d)
Surface the transcode session's runtime facts in the player. The session
response's transcode/hw flags now thread through: the stats overlay's mode row
reads "transcode · GPU" (VAAPI) or "transcode · CPU" (libx264) — the at-a-glance
confirmation of which S1 path engaged — and the initial-load overlay says
"Preparing stream…" instead of "Loading…" for a transcode session, since its
first segment is genuinely slower than a remux and shouldn't read as a hang.

fmtMode (pure, unit-tested) appends the encoder only for a transcode with a known
encoder. i18n HU/EN for the new preparing label. Browser-UAT'd on the XviD
fixture: mode row reads "transcode · CPU" live (software, no render node here).
2026-08-30 01:30:53 +02:00
peter ce96032edb feat(plex): transcode resource discipline + admin knobs (E6 S1c)
The software-fallback re-encode now runs deprioritized and admin-configurable.
The SW transcode ffmpeg child is niced (preexec_fn → os.nice, applied per-child
because the Popen lives in the long-lived API process, which must NOT itself be
niced — unlike the download worker's whole-process _deprioritize) so a CPU-bound
re-encode can't starve request handling or playback; the VAAPI path needs none of
it.

New sysconfig knobs (plex group, rendered on the admin Configuration page, i18n
HU/EN): plex_transcode_hw (auto/on/off, layered onto _want_hw), plex_transcode_
preset, plex_transcode_crf, plex_transcode_max_height (software-only downscale-to
cap; the GPU path encodes at source resolution). plex_max_transcodes stays the
concurrency cap (comment corrected — the fallback is shipped, not "later").

pytest extends the command-builder + _want_hw coverage (preset/crf/max_height
plumbing, the on/off/auto policy). Browser-UAT'd: max_height=240 drops the live
resolution to 330x240; the SW ffmpeg child runs at nice 10 (ppid=api process).
2026-08-30 01:22:30 +02:00
peter 0658f9550b feat(plex): transcode playback mode — VAAPI/libx264 re-encode (E6 S1b)
Non-h264 items (playable="transcode": HEVC/VP9/AV1/MPEG-2/XviD…) now play by
re-encoding the video to H.264 inside the existing HLS session, instead of the
old HTTP 501. VAAPI hardware encode (h264_vaapi, full GPU decode→scale→encode
pipeline) where the render node is present and the source is HW-decodable;
libx264 software fallback otherwise (yuv420p 8-bit, -threads reserve-one-core).

The re-encode mints a keyframe at every segment boundary (-force_key_frames) so
the HLS muxer still cuts independent segments and seeking works; everything else
— -copyts, the HLS tail, multi-audio renditions, the AAC audio path — is the
same machinery as remux, so hls.js/subtitles/seeking are unchanged. The session
route returns transcode/hw so the player can surface the active mode.

pytest pins both command branches (software + VAAPI shape) and the HW-selection
gate (render node present AND source HW-decodable). Browser-UAT'd on the XviD
fixture (Winning London): 501→plays, real 528×384 H.264 frames, 0 dropped, seek
lands clean; stats overlay Mode row reads "transcode".
2026-08-30 01:06:08 +02:00
peter f162788135 feat(plex): up-next end-card at episode end (E6 S4)
At episode end, when a next item exists and the pref is on, show an up-next
card (poster + title + ~10s countdown + Cancel) instead of the old instant jump,
so binge stays in the viewer's control (D6-A). Countdown or Play now advances;
Cancel/✕ dismisses and stays on the finished frame. Lives only on the
play-to-the-end path — credits-auto-skip advances earlier and never reaches the
card, so the two reconcile by construction (no double-advance). New pref
'upNext' (default on) with a Playback-tab toggle; the card closes on item change
and joins the Back/Escape cascade. i18n HU/EN.
2026-08-30 00:28:50 +02:00
peter 968674bc6a feat(plex): stats-for-nerds debug overlay (E6 S2)
Add a toggleable stats overlay to the Plex player: active mode
(direct/remux/transcode), transport, resolution, codec, bitrate, HLS level,
dropped frames, buffer-ahead, session offset and position — sampled once a
second off the live videoRef + hlsRef (no data layer). Non-modal corner panel,
aria-live polite, dismissable; toggled with 'd', a gauge control-bar button, or
its X. Number-crunching lives in a pure, unit-tested plexStats module. Doubles
as the S1 transcode-verification surface. i18n HU/EN.
2026-08-30 00:17:34 +02:00
peter 90ae9f6e88 feat(plex): on-screen ±10s skip buttons flanking play/pause (E6 S3)
Add discoverable rewind-10 / forward-10 controls to the Plex player control
bar, flanking play/pause (D5-A). New Skip10 glyph: an outline circular arrow
with a centred '10', deliberately distinct from the episode-level
SkipBack/SkipForward and from Restart's plain RotateCcw. Reuses seekTo; mirrors
the keyboard ←/→ (seekStep=10). i18n HU/EN aria-labels.
2026-08-30 00:09:36 +02:00
peter 3682e6a01f release: 0.76.0 (feature/e7-downloads-media-library)
gate / frontend (push) Failing after 1m36s
gate / backend (push) Successful in 2m26s
2026-08-29 21:54:19 +02:00
peter aca355f43b chore(release): 0.76.0 — E7 Downloads & Media Library
Bundles the whole E7 epic (library metadata, crop fix, share-by-default, in-app player,
browser-playable guarantee, local upload) plus the deep-review hardening. Bump VERSION and
add the 0.76.0 release-notes entry (RELEASE_NOTES[0] must match VERSION).
2026-08-29 21:52:21 +02:00
peter f41c81eb56 fix(downloads): media-library UI review fixes (E7)
Deep-review remediation on the frontend, all findings adversarially verified:

- LibraryPlayer picks audio vs video from the canonical media_kind (single source), not a
  hardcoded container whitelist, so an uploaded audio file with an uncommon/absent container
  renders in <audio> instead of a broken video box (legacy null media_kind keeps the heuristic).
- UploadDialog aborts the in-flight upload when the dialog is closed by X/ESC/backdrop/Back
  (not only the footer Cancel); drag highlight uses an enter/leave depth counter to stop the
  flicker; the progress bar gets a stable role=progressbar aria-label with the value on aria-valuenow.
- DownloadCenter's Shared tab renders honest loading/error states instead of a blank panel, and
  the library/shared row-action gating moves to a single tested helper (lib/downloadActions).

Tests: downloadActions gating matrix, the uploadReq XHR path (abort/401/422-localize/onerror),
and the media_kind-driven audio/video branch.

i18n: downloads.upload.uploadProgress (HU + EN).
2026-08-29 21:52:13 +02:00
peter 614c5528ad fix(downloads): harden upload serving + ingest against stored-XSS and leaks (E7 review)
Deep-review remediation for the E7 upload path, all findings adversarially verified:

- Serve every library file with a Content-Type chosen from the ffprobe-verified media_kind
  + a strict container allow-list (storage.served_media_type), never guessed from the
  user-controlled file extension. Closes a stored-XSS vector where a media/HTML polyglot
  uploaded as "evil.html" was served inline as text/html on the login-free /watch surface.
  Applied to both the authenticated /file and the public watch_file endpoints.
- Classify uploaded still images (jpg/png/gif/webp/…) as media_kind='file' so they are
  force-downloaded and don't get a broken in-app video player.
- Roll back the on-disk side of an upload when the ingest transaction fails after place_file
  (unlink the placed file + its deterministic poster path), so a failed commit can't orphan
  bytes in the GC-skipped .uploads tree.
- Floor download_upload_max_bytes at min=1 so a stored 0 can't silently disable the per-file cap.
- Dedupe the ffprobe-JSON invocation into edit.run_ffprobe_json (shared by probe_codecs and
  upload._probe) and the upload extension into storage._clean_upload_ext; drop the now-unused
  parse_probe_streams.

Tests: public + authenticated force-download and the polyglot-not-served-as-HTML guard, the
real _probe classifier (image/cover-art/dims/duration), the ingest rollback/cleanup path, the
max_jobs branch, the cap-floor invariant, and run_ffprobe_json error handling.
2026-08-29 21:52:02 +02:00
peter 104bd1e3a6 feat(downloads): public generic-file watch surface (E7 S5d)
The /watch/{token} SPA gets a third branch: media_kind==='file' renders an
info card (name / TYPE · size) + a Download button, no player. The <video>/
<audio> branches now guard on media_kind!=='file' so legacy links
(media_kind null) still play. Download is gated on the link's allow_download,
mirroring the media branch; the backend serves a file force-download
(attachment + octet-stream) regardless, so the gate only reflects owner ACL.

FE-only (public_meta already emits media_kind/size_bytes/container since S5a).
i18n HU/EN/DE 'file' fallback label. Two DOM tests: file -> info card +
Download, no player; view-only file -> no Download.
2026-08-29 19:51:58 +02:00
peter 584e9ced0c feat(downloads): local-upload UI — drag-drop dialog + progress (E7 S5c)
UploadDialog (drag-drop + native picker + optional display name) opened
from a new Upload button in the queue add-row. Uploads stream through a
new XHR helper (uploadReq in api/core) so the dialog shows real 0-100%
progress with a cancel — fetch/req can't report upload progress. The
upload lands as an already-done job; the S5b file-kind row gates it to
download+share. i18n HU/EN; upload cap/empty errors localize via the
shared structured-detail path.
2026-08-29 18:53:53 +02:00
peter b4ad59274b feat(downloads): gate library row actions by media_kind (E7 S5b)
Surface the S5a media_kind discriminator in the library UI. A generic
uploaded file (media_kind='file') is download+share only — no player and
no timeline to trim, so Play/Edit/Edit-details are media-only; it shows a
neutral file glyph instead of an (always-empty) thumbnail and no duration
chip (container/size still read). The "Shared with me" tab's file row is
gated the same way. Re-download is now hidden for edit clips and uploads
too (neither has a re-fetchable source — the backend already rejects both
with 409), removing a latent click-to-409 on edit rows.

FE-only: the serializer already emits media_kind (S5a). Adds media_kind to
the DownloadAsset type and "upload" to the job_kind union; DOM tests cover
the file-row presentation (glyph, no img, no duration).
2026-08-29 18:38:21 +02:00
peter fe0b34cd00 feat(downloads): local file upload ingest — media library backend (E7 S5a)
Add the non-yt-dlp ingest path so a user-uploaded file joins the same download
library and inherits its function set. Any file type is accepted: media
(video/audio) gets the full set (metadata, share, play/edit), a generic file
(pdf/zip/docx/…) is download-only.

- MediaAsset.media_kind ("video"|"audio"|"file") — the stored discriminator that
  gates library actions. Set at every finalize (download probe, editor carries the
  source's kind, upload probe); consolidates the ad-hoc audio-detect that lived only
  in links.py into one invariant (downloads.mediakind.derive). Migration 0076 adds
  the column + backfills existing rows.
- downloads/upload.py: ingest_upload probes the file (ffprobe → codecs/dims/duration),
  places it in a per-user .uploads tree, and creates an already-ready asset + done job
  (the same cache-hit shortcut enqueue uses). Uploads never expire (irreplaceable).
- POST /api/downloads/upload: streaming multipart with the per-file cap
  (download_upload_max_bytes, default 500 MiB) enforced on actual bytes read; quota
  charges the known size up front (quota.check_upload).
- Security: a generic file is always served force-download (attachment +
  application/octet-stream) on both the in-app and public endpoints, so uploaded
  HTML/SVG can't execute inline. Edit + redownload reject non-media / upload sources.
- python-multipart added (first multipart endpoint in the app).
2026-08-29 08:02:11 +02:00
peter 9eaa01617c feat(downloads): in-app player for library items (E7 S4b)
Play a completed library item in-app: a Play button on Library rows (and
Shared rows with download access) opens a modal that streams the local file
into a native <video controls> (or <audio> for audio-only assets).

Rather than the YouTube IFrame PlayerModal or the Plex HLS PlexPlayer — both
bound to their own domain APIs and unable to point at a local file — this
reuses the shared Modal a11y shell (focus-trap, ESC, layer-stack, portal) and
the range-aware /api/downloads/{id}/file endpoint. S4a's universal-codec
guarantee (mp4/H.264/AAC + m4a/mp3) makes native playback always work, so seek,
volume, fullscreen and speed come for free with zero transcode.
2026-08-29 07:18:58 +02:00
peter fdca12968b feat(downloads): one-click Share + downloadable-by-default links (E7 S3)
D2-A: the library row's Share button becomes a single action — quickShareLink()
ensures a public link (minting on demand), copies it to the clipboard, and toasts
with an "Open in new tab" action. A copy failure still toasts (warning) with that
action so the link is never a dead end. Advanced controls (password/expiry/revoke/
user-ACL) stay in the existing ShareDialog, reached via a new "Manage sharing" row
button.

D1 (forward-only): CreateLinkIn.allow_download defaults to true, so a newly minted
link is downloadable by default. The default lives in the API, not a migration —
the download_links column default stays false, so no existing link flips. The
ShareDialog create-default is flipped to match.

Tests: TestCreateLinkDefault (default true; explicit false honored); 4 vitest cases
for quickShareLink (mint-vs-reuse, copy ok/fail, create failure). HU/EN parity.
2026-08-29 06:36:07 +02:00
peter 94fcc40bde feat(downloads): guarantee every download is browser-playable (mp4/H.264/AAC)
The Download module's purpose is download -> share -> play in-browser, so the
output must play in EVERY browser (iOS/Safari/WebKit included). The only
universally-safe combo is mp4 + H.264 + AAC/MP3.

- formats.normalize() now CLAMPS every spec to that profile (container->mp4,
  vcodec->None, audio_format opus->m4a) at the single point all paths flow
  through (picker, stored profile, raw API) — not just by hiding UI options. A
  source exposing no H.264 is still caught by the worker's browser-safe
  re-encode, so playability is guaranteed end-to-end.
- build_ydl_opts drops the now-unreachable non-compat branches (+ _VCODEC_SORT).
- _SIG_VERSION 3->4 so assets cached under an old non-compat spec — or a stale
  non-H.264 mp4 that slipped the v3 net (the HEVC Facebook dls) — re-derive.
- ProfileEditor drops the container/codec pickers and Opus; audio is M4A/MP3.
- i18n: remove the now-unused container/vcodec/any keys (HU+EN parity).
2026-08-29 06:08:21 +02:00
peter fb2c843fa9 fix(downloads): anchor crop overlay to the letterboxed picture (E7 S2)
The video preview is object-fit: contain (the browser default for <video>), so a
non-16:9 source is letter/pillarboxed inside the element box. The crop marquee was
drawn in fractions of the whole box (bars included) and those fractions were then
multiplied by the source resolution, so the emitted ffmpeg crop diverged from what
the selector showed. Anchor the overlay to the computed content rect (and set
object-contain explicitly) so a marquee fraction is a fraction of the real picture;
extract the display-px<->source-px math into cropGeometry.ts with unit tests, and
clamp the even-rounded window in-bounds so a full-frame crop can't overflow.
2026-08-29 05:43:41 +02:00
peter bb6cb6e335 style(downloads): bump library-row meta text + action buttons a step (E7 S1)
Enlarge every non-title text on the card (channel, status/format/size/date meta,
source-ref line, progress readout) one step (xs→sm) plus their inline icons, and
grow the row action buttons (padding + icon) via the actions wrapper so the shared
IconBtn stays smaller in the modals. Buttons keep the title's line — the wrapper is
shrink-0/no-wrap, so the text column shrinks instead.
2026-08-29 05:29:19 +02:00
peter bdbff89507 feat(downloads): show container/format chip in library rows (E7 S1)
The completed download rows already surfaced size + duration; add the concrete
container (MP4/M4A/WebM…) as a compact chip so the library reads as a media list
with a visible format — the audio-vs-video distinction the running-only phase pill
can't show at rest. Pure display of the already-serialized asset.container; locked
by a small DOM test (present when ready, absent before the worker fills it).
2026-08-29 05:21:54 +02:00
peter 9b87838e3a release: 0.75.0 (improvement/quickfix-batch-2)
gate / frontend (push) Failing after 1m34s
gate / backend (push) Successful in 2m16s
2026-08-29 03:55:15 +02:00
peter f014416e31 chore(release): 0.75.0 — quickfix batch #2 (Plex/downloads/playlists) 2026-08-29 03:54:08 +02:00
peter 3369587412 fix(review): resolve deep-review findings across quickfix batch #2
- Button: `loading` always disables (|| not ??) — stops a double-submit
  when a call-site passes an explicit boolean `disabled` (VideoEditor).
- playlists: track un-ingestable-but-live YouTube ids in the sync baseline
  so a linked playlist containing a private/deleted video converges instead
  of re-classifying it as a pending pull forever; re-attempt ingestion each
  sync so a restored (private->public) video heals, and prune ids that
  leave YouTube.
- SettingsPanel: download-auth section renders an honest error/loading
  state (StateMessage + retry) instead of a bare empty section on a failed
  credentials fetch; HU+EN loadError string.
- tests: reorder-order, partial-failure baseline hold, un-ingestable
  converge/heal/prune, Button loading-disables, download-auth error/retry.
2026-08-29 03:52:13 +02:00
peter 9af4e8f4ff feat(playlists): sentence-style two-way sync preview (only changed sides)
The confirm dialog spelled out both sides even when one was "+0 / −0", which read
as engineer shorthand. Build the message from one sentence per side that actually
changes and add a short "stays unchanged" line for the untouched side; the zero
side is no longer shown as a bare +0/−0 to decode. Assembled in the component
from granular i18n clauses (add/remove per side, reorder, rename) with count
plurals; HU+EN. Replaces the single pushPlanMerge template.
2026-08-28 05:20:34 +02:00
peter a0b725d779 feat(playlists): two-way (3-way) merge YouTube sync + mandatory preview (S7)
"Sync to YouTube" was a one-directional destructive push (local wins, deletes
from YouTube every item not in the local list), so items added on YouTube were
lost and a dirty pull was skipped entirely. Replace it with a real 3-way merge:
each item is classified against the last-synced baseline as a local-add, a
YouTube-add, or a genuine delete, so an add or delete on either side survives.

- Persist the ordered baseline id set (new playlists.synced_video_ids, migration
  0075); the fingerprint hash alone couldn't drive a merge. NULL baseline (fresh
  export / pre-S7 link) unions losslessly on the first sync and records one.
- Pure merge_plan(baseline, local, youtube) -> add/remove per side + merged order
  (local order, YouTube-only additions appended). merge_playlist applies the
  local side directly and the YouTube side through the failure breaker; the
  baseline only advances on full success so a partial failure re-converges.
- plan_merge/push-plan now report what each side gains and loses; the confirm
  dialog shows the two-way plan before applying (pushPlanMerge, HU+EN).
- Tests: 10 new (pure classification incl. delete-vs-stale-copy; DB apply proving
  a YouTube-add and a Siftlode-add both survive on both sides, a genuine delete
  propagates without collateral loss, no-baseline never deletes, create path).
2026-08-28 04:20:29 +02:00
peter bc33b2ed16 feat(downloads): per-user, per-host yt-dlp cookie/auth support (S6)
Each user stores their own login cookies (Netscape cookies.txt) for an
auth-walled download host (Facebook, Instagram, X, TikTok, Reddit) in
Settings; a download of an auth-walled link on that host is retried with
the acting user's cookie.

Two load-bearing security controls, both in downloads.credentials as the
single source of truth so enqueue and the worker can't drift:
- cache privacy: a cookie-authed download salts its MediaAsset format_sig
  with the user id, so the authed asset is private and never cache-served
  to another user. The worker attaches a cookie ONLY when the asset it is
  downloading into IS that private, salted identity.
- host scope (SEC-12): a cookie is resolved only for a URL whose netloc
  the credential owns (netloc-scoped _host_is); a look-alike/unrelated
  host gets none.

cookies_enc is Fernet-encrypted at rest, only decrypted in-worker into a
0600 file under staging/.auth/ (survives the retry wipe + _produced_file
scan) and unlinked in finally. The blob is never returned by a route,
logged, audited, or written to a job's error/snapshot. cookiefile is set
after the adapter's extra merge so an adapter can't clobber it. Settings
UI is write-only (host + set/updated only, never echoed). New
download_credentials table + migration 0074; per-user routes; HU+EN i18n.

Tests: host-scope, cache-privacy (no cross-user serve), encrypt-at-rest,
write-only API, worker attach-gate; FE dom test for the write-only field.
2026-08-28 03:53:04 +02:00
peter 5280949667 feat(ui): migrate the canonical action-button surface onto <Button>
Sweep the rounded-lg solid/ghost/outline action-button family onto the shared primitive:
ConfirmProvider, ErrorDialog, DownloadDialog, DownloadCenter, downloadModals, downloadAdmin,
ProfileEditor, PersonalViewsModal, SettingsPanel, WatchPage, DraftSaveBar. Deliberately left
on their own styles per the agreed scope: login/onboarding rounded-xl CTAs (Welcome/Onboarding/
Setup/KeyGate), glass Plex surfaces, segmented pills/toggles/switches/chips, dropdown triggers,
data-driven Banner/tone maps, text-xs dense buttons, and tiny inline p-1/p-1.5 icon controls —
none map 1:1 to a variant+size without a visual change, and the repo has no tailwind-merge.
2026-08-28 03:12:29 +02:00
peter 4892c38466 feat(ui): shared <Button> primitive + migrate ShareDialog/VideoEditor/ChannelPage proofs
Introduce frontend/src/components/ui/Button.tsx — the one action-button primitive
(variants primary|danger|outline|ghost|glass|danger-outline; sizes xs|sm|md|lg|icon;
icon/iconOnly/label/loading/fullWidth/forwardRef). No focus-visible (already centralized
in index.css @layer base). Migrate the cleanest proof sites off btnCls, and flip the
channel Subscribe button to the outline variant so it matches Full history (the user's ask).
2026-08-28 02:59:21 +02:00
peter 9ae6c47d6c perf(downloads): de-prioritize worker transcodes + idle-back-off polls to stop LAN slowdown
The download/edit child (yt-dlp + ffmpeg) shares one LAN box with the API,
Postgres, and Plex playback. A CPU re-encode pegged every core for ~30s and
starved web requests + playback until it finished. Full-scope S4 mitigations:

- Child self-de-prioritizes at startup (inherited by every ffmpeg/yt-dlp
  descendant): CPU nice increment + best-effort idle ionice class + an
  OMP_NUM_THREADS budget, all config-gated and independently disable-able.
- Cap the browser-safe re-encode's encoder threads (`-threads`) so a transcode
  reserves a core for foreground work instead of claiming all of them.
- Throttle the retention GC's whole-tree orphan sweep (yield IO after each
  batch of files) so its rglob can't monopolize disk IO in the API process.
- Idle-back-off the Download Center (2s→15s when nothing queued/running) and
  UsageBar (2s→15s when running_jobs is 0) polls; both snap back to 2s on the
  next fetch after an enqueue (invalidateDownloads refetches immediately).

Tests: encode_thread_cap (pin/reserve-core/edge), -threads flag presence,
_deprioritize (nice/ionice/OMP branches), GC sweep throttle batching.
2026-08-28 02:37:56 +02:00
peter b7a6451dba fix(downloads): iPad audio playback + honest non-retryable failure for auth-walled hosts
5a — the public watch page rendered every asset in a <video>, but WebKit (all
iOS browsers) refuses to play an audio-only mp3 in <video>, so shared audio
links were broken on iPad while working on desktop Chromium. public_meta now
flags audio-only assets (codec present, no video stream) and WatchPage renders
a native <audio> in a padded surface card (no artwork exists for these).

5c — a login-walled Facebook post failed as a bare 'unknown', which offered a
misleading Retry. Classify closed-group / cookie-suggestion auth walls as
login_required and genuinely unsupported sites as a new unsupported_host class
(both non-retryable), and drop unknown from the modal's retryable set — an
unclassified failure is most often an unparsed auth wall. The queue row keeps
its own force-retry for real transients.
2026-08-28 02:22:28 +02:00
peter 11e32163b8 fix(ui): SortControl direction toggle no longer wraps on narrow popovers
The asc/desc buttons in the sort popover footer share one component across
Feed and Plex browse, but rendered differently: the Plex popover is narrower
(short option labels), so "Descending" wrapped icon-over-text and made the
active button taller. Add whitespace-nowrap to the direction buttons and bump
the popover min-width (min-w-52 -> min-w-56) so both call-sites render an
identical single-line toggle.
2026-08-28 02:06:06 +02:00
peter 257dc5741d fix(plex): resolve movies stuck "in progress" on the 15-min incremental sync
The incremental watch-sync (run_plex_watch_sync) learned a movie's watched
transition only from the history feed + onDeck. When a movie is finished on
Plex (commonly by scrubbing through the end credits) Plex sets viewCount>0,
clears viewOffset and DROPS it from Continue-Watching, so onDeck no longer
lists it (and onDeck can only ever ADD an in-progress row, never resolve one),
while a scrub-to-end frequently emits no history session row. The stale local
status='new', position>0 row then survived until the DAILY full reconcile —
which is why movies kept showing "in progress".

Add _reconcile_inprogress_movies: for each local in-progress MOVIE that Plex's
onDeck no longer lists, read its current per-account viewCount/viewOffset with
one metadata() call and apply it via the shared _pull_apply, flipping to
watched when Plex says so. Bounded by a probe cap (100), most-recently-touched
first; episodes are left to onDeck/history. No-signal rows are preserved (an
un-watch stays the daily reconcile's job).

Tests: watched-flip, onDeck+episode skip, and no-signal-preserved.
2026-08-28 01:34:02 +02:00
peter f52bc5898b release: 0.74.0 (improvement/quickfix-batch)
gate / frontend (push) Failing after 1m28s
gate / backend (push) Successful in 2m15s
2026-08-28 00:05:21 +02:00
peter d0033617e2 chore(release): 0.74.0 — quickfix batch (Plex sync/browse + Channel Manager) 2026-08-28 00:03:13 +02:00
peter f80ed115ad fix(plex,channels): deep-review fixes — durable watch-sync + no shared-link clobber
Deep-review of the quickfix batch surfaced two data/preference-loss bugs plus
smaller items; all fixed with tests.

- watch_sync _pull_apply: don't let the Plex-authoritative pull overwrite a DIRTY
  local row (a change whose immediate push failed) — that stranded the value
  forever since _repush_dirty only retries dirty rows. Leave it dirty for re-push.
- PlexProvider / ChannelsProvider: a partial shared link opened in a fresh tab
  persisted mount-time DEFAULTS over the account's stored scope/sort/filters
  (base read before the account was pinned). Persist/keep only URL-present dims;
  re-hydrate the omitted ones from storage.
- playback resume write-back: guard against out-of-order delivery (a superseded
  older push now drops) and evict the throttle map so it can't grow unbounded.
- PlexPlayer: announce mid-playback buffering to assistive tech (role=status +
  sr-only label); new plex.player.buffering (HU+EN).
- columnSort: hoist the shared key:dir sort codec used by both URL-state modules.
- Tests: dirty-guard + flag-settle + per-key throttle scoping + superseded-push
  (backend); hold-repeat/suppress-click/unmount + partial-link no-clobber +
  refetch-doesn't-clobber-undo (frontend). Backend pytest 475, FE vitest 519.
2026-08-27 23:54:44 +02:00
peter 7fbb996297 feat(channels): shareable manager view in the URL — tab+sort+column-filters (S8)
The Channels-manager content state now round-trips through the address bar,
so a manager view is bookmarkable/shareable — the S4 pattern applied to the
last content page the audit flagged.

- New lib/channelsUrlState.ts (channelsParamUpdates / channelsParamsToState /
  hasChannelsParams) under NAMESPACED keys ctab/csort/cf — channels-unique
  like ?v=/?pl/pscope, so no other provider reads or clears them and the
  feed's URL-stripping can't delete them. The mixed scalar/array FilterMap
  (sync/priority scalars + tags multi) rides in `cf` as a JSON blob so both
  shapes round-trip exactly. Only non-default values emit.
- ChannelsProvider mirrors {view, sort, filters} via useUrlMirror, page-gated
  on channels (off-page the keys clear; localStorage still holds the state and
  it re-mirrors on return). A shared link wins over stored state: readChannels
  UrlSeed captured once, applied before paint through the slice setters (so the
  account-pin re-hydrate sees them dirty and won't reload storage over the
  shared view), then persisted to the account once pinned so it survives a
  later plain reload — mirrors PlexProvider's seed/rehydrate. The name search
  stays ephemeral (out of the shared contract).
- NavigationProvider's page-switch nulls ctab/csort/cf alongside the plex keys
  so switching modules resets to the manager root.

Tests: channelsUrlState.dom.test.ts (9 round-trip) + ChannelsProvider.dom
seed/mirror (2); the existing tab-history test's beforeEach now resets the
query string too (the mirror writes it). FE vitest 503→514. FE-only.
2026-08-27 04:37:53 +02:00
peter 9cd4ae09d4 feat(views): editable saved-view detail — removable filter chips + undo + save-current (S7)
The Personal-views manager's detail pane becomes editable instead of
read-only:

- Captured filters render as REMOVABLE chips. Removing one prunes the set
  and writes it straight back to the stored view (PATCH filters), via the
  same live-row builders (useActiveFilters / buildPlexActiveFilters) that
  already compute the pruned snapshot — so the modal and the live chip row
  can't drift. The prune is wrapped in useUndoable + the shared UndoToolbar,
  so undo/redo re-persist the same way. The undoable draft is the single
  source of truth for the chip row, so a chip-prune refetch never clobbers
  the history.
- A "Save current" button beside Apply stores the module's currently-applied
  filters INTO the view (the live→stored inverse of Apply). Disabled when the
  view already equals the live filters (it's the active one). On save the
  draft re-seeds to the saved snapshot and clears the stale undo history.
- Rename gains a visible commit affordance: a check button beside the name
  input that enables once the name changes (Enter and blur still commit).

Provider gains two methods mirroring `rename`: updateFilters(id, filters)
and saveCurrent(id). One shared provider serves feed + plex, so both get it.
PATCH filters was already supported end-to-end — FE-only.

New i18n: views.saveCurrent / saveCurrentHint / saveCurrentUpToDate (HU+EN).
Tests: 5 new in PersonalViewsModal.dom.test.tsx (chip-remove persists, undo
re-persists, save-current fires + disables when up-to-date, rename commit).
2026-08-27 04:22:07 +02:00
peter 00527d0cf3 feat(channels): leading '+ Tag' pill + detail-page Unhide affordance (S6)
Item 6 — the tag add-trigger now LEADS the chip row (a stable left anchor)
instead of trailing it, so on the detail page (bounded=false) adding chips
grows the row to its right and the '+'/its editor popover no longer drift.
Compact icon in the width-bounded manager cell; a labelled '+ Tag' pill
where there's free width (the detail header).

Item 7 — surface the per-subscription 'hidden' flag on the detail GET
(channel_detail was omitting it) and add an Eye/EyeOff hide/unhide toggle
next to Block on the channel page, mirroring the manager. FE ChannelDetail
gains 'hidden'; new channel.* i18n keys (HU+EN).

Tests: +3 backend (channel_detail surfaces hidden, both states + unsubscribed
default). Gate green: backend pytest 471, FE vitest 498.
2026-08-27 04:02:25 +02:00
peter 8dc4976c2c feat(channels): horizontal −/+ priority stepper with pop-free inline edit (S5)
Redesign the shared PriorityStepper (channel manager cell + detail header)
from a value-plus-stacked-arrows control into a horizontal [−] value [+]
segmented stepper. The value now sits centred between the buttons and opens
an inline editor in a FIXED-width slot, so click-to-type no longer "pops"
the control wider (the old absolutely-positioned overlay grew w-6 → w-14).

One `size` prop drives the two surfaces: `sm` draws its own bordered pill
for the standalone table cell; `md` drops the outer border and keeps only
the segment dividers, blending into the detail header's lozenge instead of
double-bordering it. Hold-to-repeat, negatives, and the debounced-flush
persistence hook are unchanged.

The manager priority column widens 3.5rem → 5.5rem to fit the horizontal
control under fixed table layout.

Tests: 7 new DOM tests for the stepper interaction contract.
2026-08-27 03:29:42 +02:00
peter 56a2ecdec3 feat(plex): shareable Plex-browse state in the URL — scope+filters+sort+show (S4)
Mirror the Plex-browse content state into the address bar so a browse view is
bookmarkable/shareable, the Plex analogue of the feed's urlState. Keys are
NAMESPACED (pscope/psort/pshow/pf) rather than reusing the feed's bare
scope/sort/show: FeedFiltersProvider actively strips its own KEYS from the URL
whenever off the feed page, so a shared key would be deleted out from under the
Plex page. The namespaced keys are Plex-unique (like ?v=/?pl), so no other
provider ever reads or clears them.

- New lib/plexUrlState.ts: plexParamUpdates / plexParamsToState / hasPlexParams.
  The pf filter blob reuses the SAME appendPlexFilters wire serializer (new
  public plexFiltersToParams + inverse parsePlexFilters, kept next to each other
  so a new wire key can't be added on one side only).
- PlexProvider: URL-seed wins over localStorage at mount; the account-pin
  rehydrate persists the seed instead of clobbering it (mirrors feed's
  initialHadUrlFilters); useUrlMirror mirrors/clears the 4 keys, gated on
  page===plex. q/field stay out of the shareable contract.
- NavigationProvider: a page switch now also nulls the 4 plex keys.

Tests: 8 new round-trip cases in plexUrlState.dom.test.ts (FE vitest 483→491).
2026-08-27 03:14:54 +02:00
peter 8299ea72b7 feat(plex): 'last watched' browse sort + auto-apply in the in-progress view (S3)
Add a 'lastwatched' sort key to the unified Plex browse. Movies rank by
coalesce(progress_updated_at, watched_at); shows by max(coalesce(...)) across
their episodes (rollup.resumed_at, now broadened from progress-only so watched
episodes count too). Never-watched titles sink via nullslast.

Frontend auto-applies lastwatched:desc when entering the in-progress watch-state
view and restores the pre-entry sort on exit; a manual sort change while there
still wins. i18n HU+EN. Backend tests cover both union legs + mixed nullslast.
2026-08-27 02:50:41 +02:00
peter 6c918d1dff docs(plex): refresh watch-sync job docstrings for Plex-authoritative pull (S2)
The incremental/reconcile job docstrings still described last-write-wins and a
30-min cadence; update them to match the S2 behavior (Plex wins on divergence,
15-min incremental) so the "why" doesn't drift from the code.
2026-08-27 02:33:42 +02:00
peter 3f8257059d feat(plex): Plex-authoritative watch-sync — resume write-back + faster pull (S2)
Make Plex the source of truth for watch state:

- Write-back during playback: item_progress now pushes the resume position to
  Plex on the periodic checkpoint too (not only the final flush), throttled to
  one push per item every ~10s (in-memory, single-process rate limiter). The FE
  now also fires an immediate (debounced) push on seek.
- Faster incremental pull: 30 → 15 min (plex_watch_sync_interval_min).
- Plex-authoritative pull: _pull_apply resolves a genuine divergence to Plex's
  state instead of last-write-wins; the _same_state + synced_to_plex short-circuit
  still prevents a freshly-pushed row from bouncing. Removed the now-unused
  _LWW_TOLERANCE_S / _sift_ts.

Tests: 5 new in test_plex_watch_sync.py (Plex-wins resume/watched, no-bounce,
throttle final vs periodic).
2026-08-27 02:07:45 +02:00
peter 12d9561ba1 fix(plex): warm up HLS buffer on session start + player loading/buffering spinner
The Plex player stalled ~2s a few seconds into every auto-advanced next
episode. Root cause: each auto-advance spins up a cold ffmpeg HLS remux
session, and for a fresh start (start_s==0) start_session returned
immediately with zero buffer — hls.js drained the first segment, then
rebuffered waiting for the still-warming ffmpeg to write the next one.
Once ffmpeg gets ahead (stream-copy is faster than realtime) it never
stalls again.

- stream.py: before reporting a session ready, wait for the first
  _WARMUP_SEGMENTS (3) video segments to exist — the seg0 wait is now
  unconditional (was start_s>0 only) and warm-up runs for all starts.
  Best-effort and bounded: stop at the first segment not ready in time.
- PlexPlayer.tsx: a loading/buffering spinner. The existing overlay only
  covered the INITIAL load (!ready); add a `buffering` state driven by the
  video's waiting/stalled/seeking (set) and playing/canplay/seeked/pause
  (clear) events so a mid-playback stall or in-flight seek reads as
  "loading" instead of a frozen frame, and surface a fatal loadError even
  once playback has started. Spinner uses the shared Loader2.

Merges roadmap E6·item1 (spinner while loading/seeking) into this fix,
since the server-side warm-up deliberately holds ready a beat longer.

Tests: 3 new in test_plex_stream.py (warm-up waits for the first few
segments; stops at the first missing one; no warm-up when seg0 never
arrives). Gate: backend pytest 460 (Docker lane), FE tsc/lint/knip/
prettier + vitest 483.
2026-08-27 00:50:58 +02:00
peter fe09879edd release: 0.73.0 (feature/e8-feed-filtering)
gate / frontend (push) Failing after 1m22s
gate / backend (push) Successful in 2m4s
2026-08-26 05:28:11 +02:00
peter 7104767ff9 fix(e8): deep-review hardening + cut 0.73.0
Fixes from several multi-agent deep-review passes over the E8 feed-filtering &
channel-manager epic (S1-S4):
- feed_years buckets by UTC year to match the UTC-pinned year-filter bounds
- tag suggestions: examples ∩ subscribed (drop tagged-then-unsubscribed channels
  from the profile denominator); exclude hidden channels as suggestion targets
- channel manager: restore ghost chips + surface an error toast on a failed tag accept
- year filter: debounce the search term so the per-year facet doesn't refetch per keystroke
- channel tags: aria-label the "+N" overflow pills
- extract useTagSuggestions hook; add backend + hook tests (profile boundaries, cap,
  language exclusion, unsubscribed/hidden handling, UTC bucketing, bulk-accept)

Bump VERSION to 0.73.0 + release notes.
2026-08-26 05:25:47 +02:00
peter 17f8885fb1 fix(channels): collapse overflowing tag suggestions into a "+N" pill
With 2+ suggestions the narrow tags cell clipped the second ghost chip
mid-word right next to the edit "+", reading as the icon covering it. Route
the ghost chips through the same fit machinery the attached chips use: a
dedicated SuggestedChipsLane fits what the cell holds and collapses the rest
into a "+N" pill whose hover popover still one-tap accepts / dismisses each.

The lane is attached-XOR-ghost (suggestions are untagged-only, and accepting
one now drops the whole channel from the suggestion cache), so each lane owns
the full cell width instead of clipping the other.
2026-08-26 02:39:00 +02:00
peter 549aae67cc feat(channels): suggest your own tags for untagged channels from your tagging pattern (E8·S4)
Learn, per user tag, which auto TOPIC tags are characteristic of the channels
you've tagged it, then suggest that tag for still-untagged subscribed channels
whose topics cover enough of the profile. Language auto-tags are excluded from
the signal — near-universal within a language, they flooded every channel.

New GET /api/tags/suggestions (read-only, deterministic) backed by
app/sync/tagsuggest.py. Surfaced in the Channel Manager as dashed "ghost" chips
in the tags cell — one tap accepts (reuses the attach-tag endpoint), × dismisses
for the session — plus a header badge and a bulk "Accept N confident" (the
full-profile matches) behind a confirm. No silent writes (D4 = suggest-and-confirm).

Also reflow a pre-existing prettier drift in dateFilter.ts from E8·S2.
2026-08-26 02:18:34 +02:00
peter fbb961b5b7 feat(feed): hover infobox on feed cards — full title, description, tags (E8·S3) 2026-08-26 00:44:27 +02:00
peter edcea87ea5 feat(feed): filter by year — per-year count facet + year control (E8·S2)
Add a Year filter to the feed rail: quick chips for recent years plus a
'More…' popover listing all years with per-year video counts, sortable by
year or count. Selecting a year narrows the feed to that whole calendar
year (a published_after/before range applied in the api layer) and is
mutually exclusive with the Upload-date control.

Backend: new /api/feed/years endpoint + feed_years() — one grouped query
that honours every filter EXCEPT the publish-date narrowing (dropped), so
the year list stays a live drill-down and switching years never zeroes the
others. FE keys the list on the date-stripped filters, so picking a year
re-queries the feed but not the list.

Wires year through FeedFilters, the filterParams/URL (?year) round-trip,
the active-filter chip row, and the panel layout. i18n HU+EN.

Tests: feed_years count query (date narrowing ignored, other filters
honoured); dateFilter year-clearing; urlState year round-trip.
2026-08-25 02:55:12 +02:00
peter 2c843e549b fix(channels): tag-filter row folds overflow into "+N", keeps "…" pinned
The channel manager's tag-filter row wrapped the trailing "…" catalog-actions
menu onto a second line once the chips + Clear outgrew the width. Rework it as
one non-wrapping row: the TAGS editor trigger (left), a single-line chip lane
that collapses overflow into a "+N" pill whose popover holds the rest (still
toggleable), then Clear and the "…" menu pinned right — so "…" always stays on
the first line.

New TagFilterChips reuses the shared measure core (useChipOverflowFit +
fitChipsWithPill), the same compression the feed's applied-filter row uses.
2026-08-25 02:30:28 +02:00
peter 4ee0dab68f feat(feed,channels): "No tag" filter for untagged channels (E8·S1)
Filter to channels you haven't personally tagged in the channel manager
(no user-owned ChannelTag; auto language/topic tags don't count):

- Feed: a "No tag" sentinel in the "Your tags" sidebar combobox (exclusive
  with real tag picks) + a removable "Untagged" chip; new `untagged` bool
  feed param / `notag` URL key, persisted + coerced on every filter path.
- Channel manager: a virtual "No tag" chip in the tag-chip row filters the
  table to untagged channels (client-side, shared NO_TAG_SENTINEL).

Backend `untagged` is a plain channel-level narrowing, so feed_summary keeps
it applied in the base pass — the count/facet lockstep holds with no mirror.

Note: item E (per-tag counts) and item D (Any/All toggle) were already
shipped; S1 delivers only item B.
2026-08-25 01:44:34 +02:00
peter 5d7be687d3 test(views,layer): close the two polish-tail deep-review test gaps
UX-8 layerStack: add an integration test wired through the real useLayer
hook (popover document-listener + stamp) and a PlayerModal-shaped window
listener, driven by a real dispatched keydown. The popover close is wrapped
in flushSync so its layer is popped synchronously between the two listeners
-- the production mid-dispatch race -- making the escapeHandled stamp the
only thing that keeps the player from also closing. Mutation-checked:
removing the guard fails exactly this case.

SavedViews: cover feedCanSeedDefault's "yield to a bookmarked filter URL"
branch (a filter-bearing search string -> no seed) and the per-source
seededRef guard against the 60s refetch (a changed-list refetch re-runs the
seed effect; the guard blocks re-seed). The refetch test waits for the
changed list to reach the component so the effect has its chance to wrongly
re-fire. Both mutation-checked.

Frontend gate green: 464 vitest (+4). Backend untouched.
2026-08-24 23:41:49 +02:00
peter e2a8503496 release: 0.72.1 (improvement/polish-tail)
gate / frontend (push) Failing after 1m20s
gate / backend (push) Successful in 2m0s
2026-08-24 23:14:42 +02:00
peter a389306885 chore(release): 0.72.1 — polish tail (a11y, saved-view default, show-collection Play-all) 2026-08-24 23:14:07 +02:00
peter 32c05897f4 test(plex): cover Play-all leaf/show-expansion + honest empty notice
The S4 Play-all wrapper had no test — a regression inverting the empty-queue
guard or dropping the notice would re-introduce the silent no-op it was added
to fix, and stay green. Adds a PlexCollectionView DOM test (spied api +
notify) asserting: a leaf queues as itself, a show expands to its episodes in
season->episode order, and an empty expansion surfaces plex.collection.playEmpty
instead of no-op'ing.
2026-08-24 22:38:35 +02:00
peter 2e319be569 refactor(plex): share showEpisodeKeys + honest empty Play-all
Extract the show->ordered-episode-keys flatten into one pure, unit-tested
helper (lib/plexQueue.ts) and use it at all three sites — the collection
Play-all (S4) and the show/season play-through pages in plexViews — so the
season-then-episode queue-order invariant lives in one place instead of three
hand-rolled copies. Behaviour-identical (the helper's ?? [] is a safe superset
of the collection site's previously non-null read).

Also: a show-collection whose shows expand to zero episodes had an enabled
Play-all that silently no-op'd — the same 'does nothing' state S4 removed for
the empty-queue case. Surface an honest info notice (plex.collection.playEmpty,
HU+EN) instead.
2026-08-24 22:09:52 +02:00
peter 852b227ea2 fix(views): seed the default view only after the tab-account pins
The S2 seed effect ran on the saved-views query's first resolve, but that
query is ungated and — since SavedViewsProvider mounts above <App>, which pins
the tab-account only after /api/me — resolves before the account is known on a
fresh tab. It then evaluated canSeedDefault() against a null account (which
reports 'stored, do not seed') and burned its one-shot guard, so the starred
default was never applied until a reload: exactly the fresh-tab miss S2 set out
to fix. Gate the effect on the reactive account id so it defers instead of
consuming the guard, and re-fires when the account pins. Covers feed + plex.

Adds a dom test for the defer-then-seed-on-pin path the old test masked by
pre-pinning the account.
2026-08-24 22:09:39 +02:00
peter 06b85586e6 feat(plex): expand shows to episodes so a show-collection's Play-all plays (S4)
A collection's Play-all/Shuffle queue held only playable leaves
(movies/episodes), filtering out shows — so on a show-collection the queue
was empty and Play-all was hidden/dead. Build the queue by expanding each
show member into its episodes (season then episode order), concatenated in
member order; leaves still queue as themselves. Shows are fetched lazily on
click (parallel plexShow fetches), not on page load. Gate Play-all/Shuffle
and the poster on "has any playable member" (leaves OR shows), and disable
them while the queue builds. Scope: option A — no member manual-order
migration. Adds plex.collection.playFailed (HU/EN).
2026-08-23 23:30:21 +02:00
peter fc32188fd2 refactor(chips): share the chip-overflow measure core between the two lanes (S3)
ActiveFilterChips (feed applied-filter row) and ChannelTagsControl both
hand-rolled the same single-line chip-fit measurer: a hidden mirror + a
layout-effect + ResizeObserver reading chip right edges, the real rem gap,
the pill width and the lane width. The policy half was already extracted
(decideChipLayout); the DOM-measurement half was still duplicated.

Promote that half to useChipOverflowFit(rowRef, chipsRef, pillRef, deps,
decide, initial): it owns the effect + observer + initial measure and reads
the common geometry, then hands it to each caller's own policy. The feed
row layers its leading label + trailing "Clear all" reservation on top and
keeps decideChipLayout; the tag cell's chips-only policy is extracted as
fitChipsWithPill (pure, unit-tested) alongside it. Add a jsdom test for the
hook (stubbed offsets, re-measure on deps). Behaviour unchanged — verified
in the browser: feed all/count modes + popover, tag "+N" fit + hover reveal.
2026-08-23 23:22:52 +02:00
peter 9538fb1b7a fix(views): apply the starred default view on first resolve, for feed AND plex (S2)
The feed applied its starred default via a localStorage mirror
(LS.defaultViewFilters) written by SavedViewsProvider and read by
FeedFiltersProvider on load. That raced the login effect: on a genuinely
fresh browser (no LS.filters and no mirror yet) the feed could seed
DEFAULT_FILTERS and never re-apply the default until a reload. Plex had no
default-application at all.

Replace the mirror with a single path: SavedViewsProvider applies the
starred default via cfg.setFilters on the views query's FIRST resolve, but
only into a genuinely fresh module (no stored filters for the account, and
— feed only — no URL filter override). Applying persists it, so it happens
once and the last-view-wins path takes over, matching the documented
storage semantics. Because it runs off cfg.setFilters it delivers the Plex
port for free. Extract pickDefaultToSeed (pure, unit-tested) and cover the
first-resolve/stored/no-default cases for both modules in a provider test.

Removes the now-dead LS.defaultViewFilters key and its mirror read.
2026-08-23 23:05:24 +02:00
peter 5051809940 fix(a11y): touch/keyboard reveal tail + single-Escape layer coordination (UX-7/UX-8)
UX-8: one Escape must close only the topmost layer, never both an open
popover AND the player beneath it. The popover's Escape listener is on
`document` and fires before the player's on `window`; closing the popover
flushes its layer off the shared stack synchronously, so the player's
`anyLayerAbove()` lookup read "nothing above" and closed too. Add a
consumed-event flag to layerStack (`markEscapeHandled`/`escapeHandled`):
the acting topmost layer stamps the shared KeyboardEvent, and both players
check the stamp before the (now-mutated) stack. Immune to listener order
and mid-dispatch stack mutation. Cover with layerStack.dom.test.ts.

UX-7: the E4 `.reveal` sweep already covered the card action rows; two
residual hover-only traps remained — invisible-but-tappable/Tab-focusable
on touch. Add a group-agnostic `.reveal-hover` recipe (display:none under
`(hover: none)`) paired with each site's own hover/focus-within reveal:
- VideoCard play/Continue/Restart overlay: also reveal on
  group-focus-within (keyboard), and drop it on touch — the card tap
  already plays/resumes, so no permanent scrim over every mobile thumbnail.
- Carousel prev/next arrows: gone on touch, where the strip swipes.
2026-08-23 21:59:58 +02:00
peter 64dde6fd36 chore(format): fix prettier drift on plex pickers landed on main
`prettier --check` was red on main for these two files (drifted in the
collection-review-tail merge); no logic change, format-only.
2026-08-23 21:59:41 +02:00
peter 38921fb997 Merge chore/collection-review-tail: shared PlexLibraryPicker + scroll/create tests + 409->nameTaken sweep
gate / frontend (push) Failing after 36s
gate / backend (push) Successful in 2m3s
2026-08-23 18:42:56 +02:00
peter 2d1a7ff374 fix(plex): complete the 409->nameTaken sweep + cover PlexLibraryPicker
The notifyCollectionEditError helper localizes the backend's duplicate-name
409 to plex.collection.nameTaken; it was applied to the New-collection panel
but two sibling collection-write sites still routed 409 through
notifyActionError, surfacing the server's raw English detail under the HU UI:

- PlexCollectionView.doRename (rename collision)
- PlexCollectionEditor.createNew (create-in-editor duplicate name)

Both now use notifyCollectionEditError, matching the create path. Also adds a
direct dom test for the extracted PlexLibraryPicker covering the enabled gate,
emptyHint, results, honest error, and no-results branches.
2026-08-23 18:39:22 +02:00
peter 361ca7872e docs: add collaboration section to CLAUDE.md, document ruff format gate
gate / frontend (push) Failing after 1m16s
gate / backend (push) Successful in 2m16s
- CLAUDE.md: new "Collaboration" section (plan-first, options-with-marked-
  optimal-and-why, mockups for design decisions, challenge unsound requests,
  visual UAT before done, terse otherwise).
- CLAUDE.md: clarify release is user-owned in timing, agent-run in execution.
- CLAUDE.md + docs/development.md: document the ruff format --check gate step
  (already enforced in .gitea/workflows/check.yml, was missing from the docs).
- CLAUDE.md: refresh the stale footer to point at the now-existing
  docs/development.md and backend/app/README.md.
2026-08-23 17:17:02 +02:00
peter 81b7660232 test(plex): cover create write-through + the 409->nameTaken mapping (F-F)
Backend: create-collection happy path (write-through + editable) and the
create resync-failure 422 (honest reconcile detail), mirroring the rename/remove
tests. Frontend: extract notifyCollectionEditError (the 409->nameTaken vs
generic-fallback mapping that was inline in the New-collection create()) and
unit-test it; wire create() through it.
2026-08-23 08:08:34 +02:00
peter 7caa2636e4 test(nav): cover PageScroller save/restore keying (F12)
Renders PageScroller and drives restoreScroll's rAF synchronously with fake
metrics: asserts a forward nav (new entry id) opens at top and Back restores
the offset, and that a new content key within one entry resets to top while
returning restores — the per-(entry,key) contract only the id primitives were
covering before.
2026-08-23 08:04:00 +02:00
peter 1e894d378d refactor(plex): extract shared PlexLibraryPicker (F6)
AddItemPanel (Collection page) and NewCollectionPanel (browse) had near-
duplicate debounced library-search + result-list rendering. Extract it into
one PlexLibraryPicker that owns the query, the loading/error/empty states and
the row markup; callers keep their own chrome and pass term + renderAction.
No behaviour change.
2026-08-23 08:00:54 +02:00
peter 6fb369ffcc Merge chore/ruff-format: adopt ruff format (line-length 100)
gate / frontend (push) Failing after 1m24s
gate / backend (push) Successful in 2m3s
2026-08-23 07:50:25 +02:00
peter a3500340cf chore: add the ruff-format pass to .git-blame-ignore-revs
So git blame skips the mechanical line-length-100 reformat (b1832af).
2026-08-23 07:43:25 +02:00
peter b1832afd18 style: apply ruff format across backend + scripts (line-length 100)
Mechanical, whole-tree `ruff format` pass — no behaviour change. Recorded in
.git-blame-ignore-revs (next commit) so git blame skips it.
2026-08-23 07:43:06 +02:00
peter 434a98d6ed chore(lint): set ruff line-length 100 and gate ruff format --check
Adds ruff's formatter to the gate. line-length 100 matches the frontend
prettier printWidth. The reformat itself lands as the next commit (kept
separate so it can go in .git-blame-ignore-revs).
2026-08-23 07:42:42 +02:00
peter cc1f196a2c fix(build): drop the unused readMerged export so the publish gate passes
gate / frontend (push) Failing after 1m22s
gate / backend (push) Successful in 2m5s
knip flagged src/lib/storage.ts readMerged as an unused export — its only
caller is readAccountMerged in the same module, so it doesn't need to be
exported. This aborted the 0.72.0 publish gate after the merge; no behaviour
change.
2026-08-23 07:31:24 +02:00
peter e3dac34bb0 release: 0.72.0 (feature/plex-filter-bar-overhaul)
gate / frontend (push) Failing after 41s
gate / backend (push) Successful in 2m6s
2026-08-23 07:27:18 +02:00
peter 938135bb59 chore(release): 0.72.0 — Plex browsing overhaul + editable Collection page
Plex filter-bar overhaul, in-header Discover/Browse + Playlist/Collection
modes, filter-aware Discover rails, poster-collage tiles, Recently-watched
rail, capped unseen Surprise, saved views on Plex, per-page scroll memory, and
a full Collection info page with write-through Plex editing (create/rename/
add/remove/delete). Migration 0073 (saved_views.source). Two deep-review rounds.
2026-08-23 07:26:43 +02:00
peter 7d2bee59a3 fix(plex): round-2 deep-review follow-ups on the review fixes
- collections: the resync-failed-after-successful-write path now returns 422
  (honest "reconcile on next sync" detail), NOT 502. The frontend treats
  502/503/504 as gateway-unreachable and raises a sticky offline banner
  (api/core.ts RETRIABLE_GATEWAY); the app answered, so a 422 action-error
  toast is correct. Removes the false-offline regression the 502 introduced,
  and — since 422 isn't retried — the create double-write vector with it.
- tests: make the mutate-then-resync rollback assertion non-vacuous — the
  resync stub now flushes a partial local change before raising, so the test
  proves db.rollback() actually undoes it; split into a write-failed (422)
  case and a resync-failed (422 + reconcile detail + rollback) case.
- New-collection dup hint is now muted-tone with its own nameMaybeTaken copy
  (per-library uniqueness means the name may be free here) instead of the red
  authoritative nameTaken, which the still-enabled create flow contradicts.
- history: cover the navCounter post-reload collision guard (ensureNavId
  advances the counter past a higher persisted sfScroll id).
2026-08-23 07:22:53 +02:00
peter 6af238c2ea fix(plex): resolve deep-review findings on the Discover/Collections batch
Round of USER-triggered deep-review fixes (11 CONFIRMED + 1 PLAUSIBLE, all
verified) on the 22-commit branch:

- collections: split the mutate-then-resync error surface. A resync READ that
  fails after a successful Plex mutation no longer reports a false 422 "Plex
  write failed" + rollback; it returns 502 "sync pending" (the next full sync
  reconciles the mirror). New _write_or_pending_error helper; delete is exempt
  (no Plex read). Covered by new happy-path / 422-rollback / 502 tests.
- New-collection dup-name guard is now advisory only: existingNames is the
  search-scoped, all-library union, but the backend enforces uniqueness PER
  library — so it no longer blocks a valid same-name collection in another
  library, and the backend 409 is localized to nameTaken.
- member-remove now confirms before the write-through (matches delete-collection).
- add-title / new-collection library search render an error state on request
  failure instead of a misleading "No matches".
- history: seed navCounter past the current entry's persisted sfScroll on load
  so post-reload ids can't collide with back-stack entries and mis-restore scroll.
- PageScroller: LRU-cap the navId-keyed savedScroll map (was unbounded).
- share isIntrinsicReadOnly between the browse tile and the info page so the
  lock/edit affordance can't drift.
2026-08-23 06:57:33 +02:00
peter b71b33654a feat(plex): forbid duplicate collection names + a "New collection" button
Two follow-ups after testing the collection editing against real Plex, which revealed that
Plex CONFLATES same-named collections (two rating_keys resolve to one merged member list) —
a genuine footgun.

- Duplicate-name guard: create and rename now reject (409) a title that already exists in the
  target section (case-insensitive), before any Plex write. create() can also derive the
  library from the seed item, so the caller needn't know the section key. The create dialogs
  surface the 409, and the New-collection panel warns inline before submit.
- A "New collection" button on the Collections browse surface (admin only) opens a panel:
  a name field (with the live duplicate check) + a search to pick the FIRST title — Plex can't
  create an empty collection, it needs a seed. Creating opens the new collection's detail page
  straight away, so you can keep adding titles there. This is the B-route the user wanted,
  vs. the only prior path (a movie's "add to collection").

Verified live: the 409 guard (creating an existing name is blocked, no Plex write), and the
full New-collection flow (name + first title → create → lands on the detail page); test
collections created and deleted, real account left clean.

Gate: tsc/lint/knip/prettier clean, 424 vitest; backend ruff clean, 7 collections tests
(+ a duplicate-name-rejection assertion).
2026-08-23 05:51:48 +02:00
peter 5d99a5f253 feat(plex): Collection info page — hero, play/shuffle, and write-through editing (B)
Clicking a collection tile now opens a dedicated Collection info page (like the Plex-web
one) instead of just applying it as a filter. New GET /collections/{rk} returns the hero
aggregates (year range, most-restrictive member cert, total runtime) + the members ordered
by release year then title; a new PlexCollectionView renders the hero (art/title/meta/
summary+More) and the member grid, with a Play-all and Shuffle over the playable leaves
(reusing the player's existing queue + auto-advance).

For a non-external, admin-enabled collection the page offers rename / delete / add-title
(a search picker) / remove-item, all through the pre-existing admin endpoints that write
THROUGH to Plex (each edit awaits and re-fetches — no optimism, since a Plex write can 422).
An eligible collection shows "Enable editing" first (the DB editable flag). Read-only ones
(smart / imdb / tmdb / … auto-lists) are marked with a lock on both the info page AND the
browse tile, up front.

New `{ kind: "collection" }` sub-view wired through plexSubUrl (bookmarkable ?v=collection:rk),
PlexBrowse (render branch, Backspace back-nav, tile handler), with a round-trip test.

Verified live: tile→page, hero + chronological members (James Bond 1962→2021, PG-13, 55h),
Play/Shuffle, Enable editing → edit controls + remove-X, the add search (marks already-in
titles "Added"), and the read-only lock on external collections. Real Plex mutations wired
but not executed against the live account.

Gate: tsc/lint/knip/prettier clean, 424 vitest; backend ruff clean, 6 collections tests
(+ a detail-hero/ordering assertion).
2026-08-23 04:40:09 +02:00
peter c0d869089d feat(plex): add a "Recently watched" rail to Discover (D)
A new Discover rail listing what the user last FULLY watched, newest first: a movie as its
own card, a show as the specific EPISODE watched (carrying its show title via leaf_card). Sits
above Recently added. Movies honour the sidebar filters like the other rails; episodes carry no
filterable metadata so they don't. Clicking a card plays it (the existing onCard path already
routes episodes to the player). Bilingual heading added.

Verified live: the rail renders watched movies + episodes with Watched badges.

Gate: tsc/lint/knip/prettier clean, 423 vitest; backend ruff clean, 13 discover tests (+ a
recently-watched movie+episode assertion).
2026-08-23 04:08:28 +02:00
peter 15f5a5694a feat(plex): Surprise-me shows only unseen titles, capped 3 movies + 3 shows (C)
The Surprise rail drew a random 20+20 sample including already-watched titles. Its point is
to surface something to WATCH, so exclude seen titles (a watched movie, a fully-watched show
— a partially-watched or never-started show still counts as unseen) and cap at 3 movies + 3
shows (6 total), interleaved.

Verified live: /surprise returns 6 items, 3+3, none watched.

Gate: backend ruff clean, 12 discover tests (+ a watched-exclusion/cap assertion).
2026-08-23 04:01:22 +02:00
peter 14a44bff6e fix(plex): show-library collections get montage art too (A3)
The collection browse-tile montage queried only movie members, so show-library
collections (IMDb Popular, TMDb Trending, … in the TV library) rendered as bare colour
tiles. Union the art from PlexShow members as well (a collection lives in one library, so
exactly one side has art); fall back to the collection's own poster, then the colour hash.
Runtime stays movie-only (shows have no single duration).

Verified live: every collection now shows a montage (0 art-less tiles, was ~all the show
collections).

Gate: backend ruff clean, 5 collections tests (+ a show-collection art assertion).
2026-08-23 03:57:20 +02:00
peter 37d11b602f feat(plex): redesign browse tiles — 2-poster montage top, readable label band
The full-bleed 4-poster collage drowned the on-tile text. Split PosterTile in two: the top
half shows up to 2 montage posters, the bottom half is a solid colour-hash band that carries
the label, so the text is always readable. Genre tiles drop the count (just the label now);
collection tiles show the name on one truncated line plus a sub-line of item count · total
runtime. A group with no art falls back to an all-colour tile.

Backend: `/collections` cards gain `total_runtime_s` (a new `runtime_by_key` grouped sum) and
their montage art is capped at 2. Frontend: PosterTile reworked; a `collectionMeta` helper
builds the count·runtime sub-line via the existing formatRuntime.

Verified live: collection and genre tiles render 2 posters over a readable colour band with
the right sub-lines; colour fallback intact for art-less groups.

Gate: tsc/lint/knip/prettier clean, 423 vitest; backend ruff clean, collections tests green
(+ a new art/runtime assertion).
2026-08-23 03:52:21 +02:00
peter da3cb5e694 feat(plex): poster collages on the genre + collection browse tiles
The Browse-by-genre tiles and the Collections browse tiles were flat color-hash
gradients. Give each a 2×2 collage of its top films' poster art, with the color hash
kept as the fallback so a group with no matching art (a show-only or empty collection)
never renders blank.

Backend: a shared `top_art_keys` windowed query returns, per key (a genre value or a
collection rating_key), the top few items' rating_keys — best-rated then newest, in one
query. Wired into the Discover genre tiles and the collections union endpoint; the client
renders the posters from the existing /api/plex/image/{key} endpoint, so no new image
plumbing. Frontend: a shared PosterTile component (collage + scrim + label, color-hash
fallback) replaces the two ad-hoc colored-tile implementations.

Verified live on localdev: genre tiles and collection tiles render real poster collages,
with the color fallback correctly kicking in for groups without art.

Gate: tsc/lint/knip/prettier clean, 423 vitest; backend ruff clean, 11 discover tests
(incl. a new genre-art assertion).
2026-08-23 03:10:36 +02:00
peter 1e5232f47a feat(nav): restore scroll position per history entry, open forward views at the top
The app scrolls one persistent <main> (PageScroller); its saved offsets were keyed by a
content `scrollKey` that is the same for a Plex page's Discover and grid views, so opening
a Browse-by-genre tile inherited the scrolled-down Discover offset (landing mid-page) and
Back couldn't tell the two views apart.

Tie scroll memory to the history entry instead: every pushState that opens a main-content
view stamps a fresh `sfScroll` id into history.state (setPage, Plex sub-view open); the
initial/legacy entries get one lazily via replaceState. PageScroller now saves/restores per
`${entryId}:${scrollKey}` — the entry id makes Back restore exactly where you left an entry
and a forward navigation (new id) open at the top, while the scrollKey still resets a new
feed filter to the top within the same entry. A small nav bus lets a Plex sub-view open
(which doesn't re-render the scroller) trigger the restore; `history.scrollRestoration` is
set to manual since the browser can't see our inner scroller.

Verified live: genre tile opens the grid at the top and Back returns to the Discover scroll;
feed→plex→Back restores the feed scroll; a feed filter change still resets to the top.

Gate: tsc/lint/knip/prettier clean, 5 new nav-id unit tests, full vitest green.
2026-08-23 02:55:59 +02:00
peter dddaeb9a38 fix(plex): make the header Discover/Browse toggle clickable
The floating header wrapper is pointer-events-none (so clicks fall through its empty
area to the content below); every interactive child must re-enable pointer events.
PlexViewToggle rendered its SegmentedToggle without the pointer-events-auto wrapper the
sibling PlexScopeToggle has, so the whole Discover/Browse switch was click-dead — the
buttons rendered and were correctly wired, but no real mouse click ever reached them.

Add `className="pointer-events-auto shrink-0"` to the toggle and a regression test that
asserts the root re-enables pointer events (plus the render/onChange/hide-on-deeper-view
contract). Verified live on localdev: real clicks now switch Discover⟷Browse (?v=browse)
in both directions; a header-wide hit-test audit shows no other dead controls.

Gate: tsc/lint/knip/prettier clean, 418 vitest.
2026-08-23 02:23:09 +02:00
peter 70a74f8629 fix(plex): round-3 deep-review follow-ups
Third review round, scoped to the round-2 fix commit. The production ternary was
confirmed correct; two sibling-surface findings:

- The sidebar CollectionPicker's clear-all still wrote collectionTitles:{} while the
  chip-row remove (fixed in round 2) wrote undefined — a divergence that leaves the
  {}-vs-absent cache split on the picker path. Collapse an empty map to undefined there
  too, so both no-collection routes key-match the pristine state.
- The queryKeys lang test was near-tautological (a variadic-spread factory can't drop a
  passed arg), and nothing asserted the CALL SITE keys carry lang. Add a real guard in
  PlexDiscover.dom.test.tsx that inspects the registered query keys for scope+lang, and
  reword the factory test's comment to say it only shape-pins the segment pass-through.

Gate: tsc/lint/knip/prettier clean, 413 vitest.
2026-08-23 02:09:14 +02:00
peter 8f1309fa18 fix(plex): round-2 deep-review follow-ups on the review fixes
Second review round, scoped to the round-1 fix commit (where round-1's own fixes
introduced new code):

- The collection chip-remove pruned collectionTitles to an empty {} on last removal,
  which still hashes differently from the pristine no-collection state (key absent) in
  the React Query keys → a residual cache split. Collapse an empty map to undefined so
  the key-shape matches the pristine one.
- Replace the tautological "order-independent signature" test (both objects spread from
  EMPTY_PLEX_FILTERS inherited the same key order, so it proved nothing) with full
  literals in genuinely different insertion order that would fail an order-sensitive
  JSON.stringify signature.
- Cover the collectionTitles chip-remove prune (usePlexActiveFilters.dom.test.tsx) and
  the plexSurprise/plexDiscover lang-key split (queryKeys.test.ts) with regression tests.

Gate: tsc/lint/knip/prettier clean, 412 vitest.
2026-08-23 01:53:33 +02:00
peter b7a4eaff8d fix(plex): resolve deep-review findings on the filter-bar overhaul
Self-review of the branch diff (feature/plex-filter-bar-overhaul) against main:

- Surprise rail query key omitted `lang` while the backend /surprise now filters
  against the language-specific genre column → a HU/EN switch under an active genre
  filter left the rail stale. Key it by scope+lang+filters, matching plexDiscover.
- Chip-removing a collection left its entry in `collectionTitles` (hashed into the
  plex query keys), splitting the cache from the pristine no-collection state and
  growing the titles map unboundedly. Prune the title on remove, like CollectionPicker.
- The saved-views default-mirror effect gained `cfg` in its deps (re-memoized on every
  live filter edit) → a synchronous localStorage write per keystroke. Depend on the
  module-stable mirror fn instead.
- Collections-browse search input had no accessible name (placeholder only). Add
  aria-label + type="search".
- Extract the tile hue-hash + gradient duplicated between PlexDiscover and
  plexBrowseLists into a shared lib/hueTile so the two surfaces can't drift.
- Document migration 0073 downgrade() as lossy and de-dup non-feed rows before
  recreating the narrow (user_id, name) constraint (verified by the drift test).
- Add unit tests for coercePlexFilters (legacy single-collection migration) and
  plexFiltersSignature (order-independence, title-ignoring).

Gate: tsc/lint/knip/prettier clean, 410 vitest; backend ruff + migration drift +
saved-views tests green.
2026-08-23 01:39:29 +02:00
peter a20e27800c feat(plex): port the saved-views (personal views) UI to Plex (frontend)
Plex now has the same personal-views system as the feed — the rail header icon
opens the quick-pick popover, the rail title opens the manager modal (list /
create / rename / delete / reorder / set-default / apply), and the active view
leads the chip row with a "modified" dot.

One SavedViewsProvider serves both modules: it switches `source` (feed ⟷ plex)
by the active page (both filter providers are global, so it reads whichever the
page owns) and runs the same logic on per-source wiring — the live filters,
apply, a stable signature (feed filtersToParams / plex plexFiltersSignature),
coercion (asFilters / coercePlexFilters), and what to strip on save. Moved below
PlexProvider in the tree so it can read plex state; resets sticky/modal on a
source switch. The shared popover/modal/chip are unchanged; the modal describes
a plex view's captured filters via a pure buildPlexActiveFilters (extracted from
usePlexActiveFilters — no duplication). api.savedViews/createView take a source;
`SavedView.filters` is now an opaque blob. New coercePlexFilters/plexFilters-
Signature (also dedups PlexProvider's collection migration). Empty-state copy
genericised (no longer says "feed"). A Plex view captures the rail filters (not
watch-state/sort/scope).

Verified end-to-end on localdev: create/apply/preview/default/delete on plex,
feed views stay separate + unregressed (402 vitest), no console errors.
2026-08-22 19:16:30 +02:00
peter fa7a09a420 feat(saved-views): scope saved views by module source (backend)
Saved "smart views" were feed-only; the Plex module is getting them too. Add a
`source` discriminator to saved_views ("feed" | "plex"), default "feed" so every
existing row and any source-less client stays feed. Widen the per-user unique
name constraint to (user_id, source, name) so a feed view and a Plex view can
share a name, and scope create's position numbering + the is_default "clear the
others" to (user, source).

Migration 0073 (ADD source NOT NULL server_default 'feed' + swap the unique
constraint; reversible). Endpoints: GET/POST take `source` (default feed).
Tests: source isolation, per-source default, per-source name uniqueness.
2026-08-22 18:56:52 +02:00
peter f983a491c0 feat(plex): add Playlist/Collection browse modes to the header pill
The header library-scope pill gains two entry points — Playlist and Collection —
next to All/Movies/Shows, a second way into each area (the rail filter is the
other). Picking Playlist shows a grid of the user's playlists (a tile opens that
playlist's view); Collection shows a searchable grid of collections (a tile
filters the title grid by that collection). Pure front-end — reuses the existing
plexPlaylists / plexCollections(union) list endpoints.

Two new top-level sub-views (`playlists` / `collections`, bookmarkable as
?v=playlists / ?v=collections) rendered by PlexBrowse via the new plexBrowseLists
surfaces (hue-tiled like the Discover genre tiles). The PlexViewBridge now carries
the view `kind`; PlexScopeToggle is a 5-way that derives its active segment from
it (browse-mode when in one, else scope) and drives the two views through the
bridge — picking a scope from a browse mode returns to the title grid. The
Discover/Browse toggle hides in these modes; Backspace + the search→grid flip
cover them.

Tests: PlexScopeToggle (5-way + browse-mode drive/active), plexSubUrl round-trip
for the two new views. Gate: tsc/lint/knip/prettier + 402 vitest.
2026-08-22 18:19:08 +02:00
peter f85dc12819 feat(plex): move the Discover/Browse toggle into the header
The Discover ⟷ Browse switch now sits in the header next to the library-scope
pill (All / Movies / Shows), instead of on the chip row / atop the Discover
surface.

It's driven through a small one-way PlexViewBridge: PlexBrowse still OWNS the
sub-view (useHistorySubview) and publishes its top-level view + opener to the
bridge; the always-mounted Header reads it to render PlexViewToggle. A global
sub-view provider was deliberately avoided — PlexBrowse's hook remounts on every
plex entry and re-seeds from history (a page switch clears ?v + drops _sub), so
each entry starts at the Discover root; lifting it out would let the view go
stale across navigation. The toggle hides on deeper views (show/info/person/…)
and on non-plex pages (bridge cleared on unmount). sortView is just SortControl
now; the grid toolbar no longer carries the toggle.

Verified: toggle in the header, switches discover⟷grid, hides on info + on Feed,
and Feed→Plex resets to Discover (no stale view); no console errors.
2026-08-22 18:02:55 +02:00
peter 9798e506cf feat(plex): apply the sidebar filters to the Discover rails too
Discover now behaves like the Browse grid — the active metadata filters (genre /
rating / year / collection / studio / …) narrow every rail: Continue watching,
Recently added, Surprise me, the genre tiles, and Because you watched. Same
LibraryFilters / meta_filter_conds the grid uses, applied to each rail's movie +
show query (movie duration included); a rail may come back empty, which the client
already hides. Genre-language matches the grid.

Because Discover honours filters itself, adding a filter no longer force-flips the
Discover landing to the grid (only a header SEARCH does, since Discover has no
search rail) — so you stay on Discover with narrowed rails.

Frontend threads `filters` into api.plexDiscover/plexSurprise + their query keys
(plexSurprise is variadic now) and the watched-toggle setQueryData; PlexDiscover
reads them from usePlex. Tests: two discover filter-narrowing cases; PlexDiscover
dom test updated for the provider default + 3-arg call.
2026-08-22 17:35:37 +02:00
peter a60164edc4 feat(plex): make the Collection filter multi-select with Any/All
A title can belong to several Plex collections, so the collection filter becomes
multi-select combining with Any (OR, default) / All (AND) — the same shape as the
genre/actor filters.

Backend (query.py): `collection` (single) → `collections` CSV + `collection_mode`
on LibraryFilters; meta_filter_conds ORs/ANDs `collection_keys.contains` per id.
Frontend: PlexFilters `collection`/`collectionTitle` → `collections` + optional
`collectionMode` + `collectionTitles` map; CollectionPicker is now a multi-select
Combobox with the any/all match footer; one chip per collection (usePlexActive-
Filters); the info-page "Browse collection" strip unions into the list; applyFilter
now shallow-merges label maps so adding one keeps the others' titles. PlexProvider
migrates a legacy persisted single `collection` into the new shape on read.

Test: test_plex_collections_filter (single / any-union / all-intersection).
2026-08-22 08:18:25 +02:00
peter 6c7038cf1f feat(plex): make the playlists rail widget a compact combobox
The playlists section used to render an always-open list plus a dedicated
create input, growing the rail taller with every playlist. Replace it with a
single Combobox field (reusing the shared component, like the feed's "Your
tags"): the list of playlists only appears when the dropdown is opened, and
typing a name that doesn't exist offers a "Create …" row. Picking a playlist
opens it (the existing playlistOpen hand-off); the create row creates then opens
the new one. `selected` stays empty — this picks an action, not a persisted value.
2026-08-22 07:59:41 +02:00
peter f8e144a658 fix(combobox): stop a single chip wrapping the chevron to a second line
The field's input carried min-w-16 (64px). Flex line-packing sizes a flex item
at its min-width, so in a narrow rail field (~202px) one chip (~119px) + the
64px-reserved input + the chevron overflowed the line, wrapping the chevron
down while the (grown, empty-looking) input filled the first line — the "breaks
to two rows even though it'd fit" glitch seen on the Plex Collection widget and
the feed's channel picker (both are Combobox). Lowering the floor to min-w-8
(32px) keeps chip + input + chevron on one line; flex-1 still grows the input to
fill the remaining width. Verified in-browser: field height 54px→36px (one row).
2026-08-22 07:56:05 +02:00
peter b4984dc38b feat(plex): move Watch-state out of the rail into a toolbar pill
The "Show" (watch-state) filter leaves the left rail and becomes a FilterSelect
pill next to the item count, mirroring the feed's Watch-state control — a
top-level "what am I looking at" choice, always in reach, not buried in the rail.
It still emits a removable chip when non-default (same as the feed).
2026-08-22 07:51:54 +02:00
peter 21c9b34540 feat(plex): port the filter bar to the YT-feed pattern
Bring the Plex module's filter surface in line with the feed's final shape:

- Flatten the rail: PlexSidebar passes `collapsible={false}` to FilterRail so
  the widgets become plain always-open label + control rows (no collapse
  chevrons; reorder/hide in edit mode stay). Drop the rail's own "Clear all" —
  the chip row above the grid already owns it.
- Move Sort + the Discover/Browse toggle into the applied-filter chip row,
  pinned right directly above the grid (a `sortView` fragment, like the feed's),
  and give ActiveFilterChips `flex-1` so it gets the shared responsive overflow
  (wide: fit + "+N" pill; narrow: single "N filters" count pill) — the chips can
  no longer crowd the controls off-screen.

ItemCount leads its own toolbar row, matching the feed. No behaviour change to
Discover, which keeps its standalone view toggle.
2026-08-22 07:23:20 +02:00
peter 041cd526b7 Merge: YT feed filter-bar overhaul + 3 deep-review rounds
gate / frontend (push) Failing after 38s
gate / backend (push) Successful in 1m57s
Flattened, more compact YT feed filter rail and a responsive applied-filter row:
- Rail: removed the in-rail Clear all; Upload date is now a single-select dropdown
  (Any time + more presets + reveal-on-select Custom range); widgets are plain
  label + control rows (non-collapsible), with the "Your tags" title opening the
  tag manager.
- Sort + View moved into the applied-filter row (pinned right, above the grid)
  with responsive overflow: "+N" pill on wide lanes, a single count pill on narrow.
- Earlier branch work also merges here: typeable combobox filters, hidden-channel
  empty->Library hint, Mine/Library scope toggle in the header, per-category
  any/all tag match, and the saved-views redesign (popover + modal + chip).

Three USER-triggered deep-review rounds (trend 14 -> 4 -> 1); round 2 caught a real
coerce-on-seed migration bug. Gate-green: tsc, 397 vitest, lint, prettier, backend
pytest. Not yet released -- prod stays 0.71.0.
2026-08-22 07:09:25 +02:00
peter 9477fea3dc docs(test): correct the feed-views fixture comment on topic tag distribution
chA/chB each carry a different topic tag (T1 vs T2), not both — the old comment
implied they shared T1/T2. Assertions unchanged (equivalence test).
2026-08-22 07:04:00 +02:00
peter e4b19b5320 fix(feed): round-2 deep-review remediations (incl. a real coerce-on-seed bug)
The re-review of the fix round surfaced 4 findings; fixing the coverage one
exposed that the C1 migration was ineffective:
- FeedFiltersProvider.loadAccountFilters: run coerceTagModes on the RAW stored
  blob, not the object merged over DEFAULT_FILTERS. The merge injects
  DEFAULT_TAG_MODES (all "or"), which coerceTagModes prefers over the legacy
  scalar `tagMode` — so the migration never fired. Now a persisted legacy
  tagMode:'and' actually migrates into tagModes.topic (proven by a new test).
- test_feed_views: the tag_mode_other=and case was vacuous (only chC carried
  either "other" tag, so AND == OR). Give chA an "other" tag too so OR-other
  {chA,chC} differs from AND-other {chC} — the conjunctive path now narrows.
- DateFilterSelect: reset the custom reveal in useLayoutEffect (not useEffect)
  so the correction commits before paint — no one-frame "Custom range" flicker
  on the trigger after an external clear.
- SavedViewsProvider: extract the sticky-view tie-break to a pure, tested
  lib/savedViews.ts (preferSticky).

New tests: savedViews (preferSticky), FeedFiltersProvider.dom (legacy tagMode
migration on the localStorage/default-view seed).
2026-08-22 06:55:16 +02:00
peter 7692253636 test(feed): cover the filter-bar gaps deep-review flagged
- test_feed_views: extend _seed_tagged so a channel carries 2+ tags in a
  non-topic category, and add language-AND / other-AND equivalence cases — the
  per-category any/all generalization was only exercised for topic.
- tagModes.test: coerceTagModes migration (legacy global tagMode -> topic),
  per-category preservation, invalid-value coercion, defaults.
- urlState.dom.test: per-category tag-mode URL round-trip + legacy ?tagmode=and
  migration into the topic slot.
- Combobox.dom.test: the any/all match footer (hidden below 2 selections, reports
  a mode change at 2+).
- chipOverflow.test: the all/some/count fit boundaries and the +N reservation.
2026-08-22 06:33:48 +02:00
peter e90603d4db fix(feed): deep-review remediations for the filter-bar overhaul
Correctness / behaviour:
- Move the Personal Views modal from inside the collapsible filter rail to the
  feed page (moduleRegistry FeedPage), so opening it from the always-visible
  toolbar view chip works while the rail is unpinned (its default) instead of
  no-op'ing and latching modalOpen into a later stale pop-open.
- Migrate a persisted legacy global `tagMode` into per-category `tagModes` on the
  localStorage seed paths too (loadAccountFilters), matching the saved-view and
  URL paths — an upgrading user's topic any/all no longer silently reverts to 'or'.
- DateFilterSelect: reset the custom reveal on an external clear while the popover
  is closed (guard on !open, not maxAgeDays), so the trigger reverts to "Any time"
  instead of lingering on "Custom range" with empty fields.
- PersonalViewsModal delete: only advance the detail selection when the delete
  actually happened (remove() now resolves a boolean) — cancelling the confirm no
  longer switches the selected view.
- SavedViewsProvider: prefer the explicitly-applied (sticky) view among several
  that capture identical filters, so apply(B) isn't re-attributed to an earlier
  duplicate A (wrong "modified" chip name).

A11y:
- Give the clickable panel title a descriptive accessible name (titleActionLabel,
  wired to "Personal views") instead of announcing only "Filters".
- Give the "+N" overflow pill an aria-label ("N more filters").
- Use t("views.reorder") for the saved-view drag handle instead of a hardcoded
  English "reorder" literal.

Also extract the chip-fit decision into a pure, testable lib/chipOverflow.ts and
measure the inter-chip gap live (rem-safe) instead of a frozen 6px constant.
2026-08-22 06:33:48 +02:00
peter dd76e9dcf7 feat(feed): move sort + view into the chip row with responsive overflow
Sort and the view switcher are tightly bound to the grid, so pin them to the
right of the applied-filter row (directly above the cards) instead of the filter
toolbar; they render there on both the feed and channel-scoped views.

ActiveFilterChips now shares that row with the controls and trims itself so it
never crowds them, measuring fit off a hidden mirror of the full chip set:
- wide  -> show as many chips as fit, then a "+N" pill whose popover holds the rest;
- narrow -> collapse every chip into one "N filters" pill whose popover holds them all.
Both popovers list the (overflowed / all) chips, each removable, plus Clear all.
2026-08-22 05:42:45 +02:00
peter fca989c098 feat(feed): flatten the filter rail — dropdown upload-date, non-collapsible widgets, title triggers
Rework the feed filter rail into simpler label + control rows:
- Remove the rail's own "Clear all" (the chip row above the cards already
  surfaces one whenever any filter is active).
- Replace the Upload date chip group with a single-select dropdown
  (DateFilterSelect): an "Any time" clear, more presets (2 weeks / 3 months /
  2 years / 5 years) and a "Custom range" that reveals from/to on select; the
  trigger then reads the chosen range. Selection logic is the pure lib/dateFilter
  (applyDateChip -> selectDateChip, no toggle-off; tests updated).
- Widgets are no longer collapsible: PanelGroup gains `collapsible` (feed passes
  false) so groups render as always-open label + control rows; reorder + hide in
  edit mode stay. Other rails (Plex/Playlists) keep the collapsible islands.
- A non-collapsible group's title can act as its own trigger (PanelItem
  onTitleClick): "Your tags" opens the tag manager, replacing the inline Manage
  button.
2026-08-22 05:42:30 +02:00
peter 7ebbf45921 feat(feed): saved views as a header popover + Personal Views modal + a view chip
Retire the standalone 'Saved views' sidebar panel. The saved-views feature now
lives in three lighter surfaces, coordinated by a new SavedViewsProvider (query +
CRUD + the active/sticky view + modal state):

- the filter panel's header ICON opens a quick-pick popover (apply a view, active
  one checked, plus 'Manage views…');
- the header TITLE ('Filters') opens a new Personal Views modal — a master/detail
  manager: left = the reorderable list + a save-current-filters input, right = the
  selected view's name, its captured filters (shown as the same chips the toolbar
  row uses), default toggle, apply, and delete;
- a SelectedViewChip leads the applied-filter row, naming the active view with a
  warn-coloured 'modified' dot once the live filters diverge from it; clicking it
  reopens the picker.

SidePanel/FilterRail gain optional iconExpanded + onTitleClick slots (feed-only,
hidden for the demo account); the collapsed tab keeps its plain pin behaviour.
SavedViewsWidget is deleted and dropped from the panel layout.

Verified live: icon popover lists HH + Manage; applying HH surfaces the chip and
switches scope; removing a filter lights the dirty dot; the title opens the
master/detail modal with the captured-filters summary. 374 vitest, tsc, eslint
(0 err).
2026-08-22 04:03:58 +02:00
peter 829cadf5fb feat(feed): per-category any/all tag match, in each combobox's popover footer
The tag 'Match: Any/All' toggle was a single GLOBAL setting that only ever
affected topics, shown as its own row under the Topic widget. Make it
per-category — language, topic, and 'your tags' each get their own independent
any/all — and move it into each filter combobox's popover footer (like
SortControl's direction segment), reclaiming the standalone Match row.

Backend: the feed query already groups selected tags by category; replace the
single tag_mode with per-category tag_mode_language/topic/other params and apply
the conjunctive (HAVING all) branch per category. Mirrored across filtered_query,
feed_summary.passes(), and the /facets drill-down exclusion (kept in lockstep;
equivalence test updated to tag_mode_topic). No schema change.

Frontend: FeedFilters.tagMode -> tagModes {language,topic,other} (pure leaf
lib/api/tagModes with a coerceTagModes migration for the legacy global value);
serialized to the API and the shareable URL per category (legacy ?tagmode=and
still resolves into the topic slot); saved-view blobs migrate on read. Combobox
gains an optional matchMode/onMatchModeChange footer segment, shown once 2+
values are picked; the three tag comboboxes wire it to their category.

Verified: backend feed tests green (12, incl. the count/facets equivalence net);
live, the footer appears at 2+ selections, toggles aria-pressed, serializes
tagmode_topic=and to the URL, and the request carries tag_mode_topic=and (200).
374 vitest, tsc, ruff, eslint (0 err).
2026-08-22 03:48:23 +02:00
peter 8f08073564 feat(feed): move the Mine/Library scope toggle beside the header search box
The scope toggle used to sit at the top of the filter rail, which is buried when
the rail is collapsed/unpinned. Relocate it to the header, just right of the feed
search box — mirroring Plex's scope toggle placement — so the primary 'what am I
browsing' control is always visible.

New FeedScopeToggle adapter (SegmentedToggle over filters.scope, same as the Plex
one over its scope); rendered in Header on the feed page between the search box and
the sync chip. Removed the toggle from the Sidebar rail's now-unused `top` slot and
dropped the dead `top` prop from FilterRail. Same filters.scope field, so behaviour
is unchanged.

Verified live: the toggle sits beside the feed search and switches scope; the rail
no longer carries it.
2026-08-22 03:29:15 +02:00
peter af08a7850b feat(feed): guide a hidden-channel empty result to the Library
Filtering by a channel under the Mine scope returns nothing when that channel's
subscription is hidden (or you don't subscribe to it) — its videos only live in
the shared Library, since the Mine feed excludes hidden subscriptions by design.
Previously this left a bare 'No videos match these filters', which reads as a
bug when you know you're subscribed.

Add an empty-state hint + a 'View in Library' button (switches scope to all) when
there's a channel filter under Mine and zero results. Bilingual EN/HU.

Verified live: ?channel=<ColdFusion>&show=all under Mine shows the hint; the
button switches to Library and surfaces the 36 videos.
2026-08-22 03:25:49 +02:00
peter d2193ccd1b fix(combobox): merged input ref so the field is typeable by mouse
The <input> carried only floating-ui's reference ref, so the component's own
inputRef stayed null. The field container's onMouseDown then preventDefault()-ed
every press (its `e.target !== inputRef.current` guard was always true) and the
compensating inputRef.current?.focus() was a no-op — so clicking the box opened
the list but never focused the input, and mouse users could not type to filter.
Tab focus still worked, which is why it read as 'click, can't type'.

Feed a merged callback ref that assigns the node to both floating-ui's
setReference and inputRef. Fixes the Channel, Your tags, Topic, and Language
sidebar filters at once (all use Combobox); the channel list's 50-row cap is now
reachable past 50 via search. Single-select commit no longer re-focuses (that now
actually fires and would re-open the just-closed list). Regression test presses
the field chrome and asserts the input receives focus.

Verified live: clicking the Channel field focuses it and typing 'cold' narrows
the list to ColdFusion / wehatethecold.
2026-08-22 03:22:47 +02:00
peter cda70164ba Merge: toolbar/header UI-consistency refactor + a11y fixes
gate / frontend (push) Failing after 1m13s
gate / backend (push) Successful in 1m52s
Collects the toolbar/header/layout redesign and assorted bugfixes on the
bug/channel-back-header-overlap branch: new FilterSelect / ItemCount /
pill-SortControl primitives, removed ModuleName, the --sbw / --content-mw /
--rail-pinned-w chrome-alignment machinery, the centred search box, and the
two deep-review rounds (menu-role a11y, sort aria-pressed, rem rail threshold,
accessible trigger names, e2e helper fix, added coverage).

Not a release: VERSION stays 0.71.0 and this is not deployed to prod.
2026-08-22 02:07:20 +02:00
peter 5e1cc4d981 fix(a11y,test): deep-review round 2 — e2e helper attr, trigger names, observer test
Second gate round over the full branch surfaced 4 verified findings (1 was a
regression the round-1 role change introduced); a 5th (--sbw load jump) was
dropped as refuted (the header only mounts after auth+data, so --sbw is set
before it ever paints).

- e2e (medium): the round-1 switch of FilterSelect rows from role=option to
  menuitemcheckbox moved their state from aria-selected to aria-checked, but
  the Playwright helper expectContentType still asserted aria-selected, so the
  channel specs would fail whenever the e2e lane runs (not part of the fast
  gate, so it stayed green). Assert aria-checked.
- a11y (low): the FilterSelect and SortControl pill triggers set
  aria-label={label}, which overrides the visible "Label: value" text, so a
  screen reader announced only the group name, never the current selection.
  Carry the value (and, for sort, the direction) in the accessible name,
  matching ViewSwitcher. Updated the unit-test trigger name queries to match.
- tests (low): cover the PageShell font-scale MutationObserver recompute path
  (a post-mount root font-size change → --rail-pinned-w re-computes once, no
  loop) — previously only the pre-render and resize paths were exercised.

Verified live: the triggers now announce e.g. "Sort: Date, Descending" /
"Watch state: All". Gate green: tsc (app+node+e2e), 373 vitest, prettier,
eslint (0 err).
2026-08-22 01:06:22 +02:00
peter 4ee452b854 fix(a11y,layout): deep-review fixes — menu roles, sort aria-pressed, rem rail threshold
Epic-end deep-review of the branch surfaced six verified findings; fix them.

a11y (medium): FilterSelect and SortControl opened their AnchoredPopover in
mode:"listbox" but never passed a role, so the surface fell through to
role="menu" while the rows rendered role="option"+aria-selected — options
orphaned outside a listbox, so AT could not announce the selected value.
Switch both to the house menu pattern (as ViewSwitcher already uses): a menu
surface with menuitemradio (single) / menuitemcheckbox (multi) rows carrying
aria-checked, scoped by a role=group. Trigger stays a button+haspopup=menu.

a11y (low): the SortControl asc/desc direction segment signalled the active
direction by colour alone. Add aria-pressed so the current direction is
programmatically exposed, matching SegmentedToggle.

correctness (low): PageShell converted a rem maxWidth to px with a hardcoded
16 to decide the pinned-rail overlay threshold, but index.css scales the root
font by --font-scale (default 1.06 ~ 17px, slider 0.9-1.3), so the threshold
was wrong at every text size on the rem pages (playlists is the only one with
a pinned rail). Use the live root font-size, scale the 36rem slack the same
way, and recompute when --font-scale changes (a guarded MutationObserver — the
slider fires no resize).

tests: cover the two gaps the review flagged — ItemCount (plural, 1M+ overflow,
undefined icon-only, tooltip) and the PageShell rail-pinned-w/content-mw
threshold math incl. the font-scale scaling. Updated the FilterSelect/
SortControl role queries to the new menuitemradio/checkbox roles.

Verified in the running app: menu surfaces carry role=group + menuitemradio/
checkbox with 0 orphaned options, and the sort direction segment exposes
aria-pressed. Dropped two review findings (max-width duplication, pill-class
reuse) as refuted on adversarial verification. Gate green: tsc, 372 vitest,
prettier, eslint (0 err).
2026-08-21 22:41:57 +02:00
peter edbc9dede5 style(header): centre the search-box group over the page content
The floating header band already centres on --content-mw (the width the
page's cards use), but its search box + scope-toggle/sync-chip group was
packed to the left edge (default justify-start), leaving it offset from
the centred card grid on max-width pages (Plex/Channels/Playlists).

Add justify-center to the band so the group centres within it: once the
column is wider than the box's max-w, the box sits centred over the
cards instead of hugging the left edge. Shared header row, so it applies
to every module with a SearchBar. Verified at 1920px on Plex: group
centre == content-band centre (431px margin each side).
2026-08-21 22:08:23 +02:00
peter 53ca4e2849 feat(layout): a pinned filter rail no longer pushes content when there's room for it
On a page that centres its content at a max width, if the window is wide enough for the rail to fit
in the content's LEFT margin without overlapping it, pinning the rail now leaves it OVERLAY — the
content doesn't shift. It reverts to the old in-flow "push" only when there isn't room: a full-width
page (feed) or a window too narrow for the rail to clear the centred content.

PageShell computes --rail-pinned-w from the page's maxWidth vs the viewport (innerWidth >= maxWidth +
~576px → overlay 2.25rem, else in-flow 16.75rem) on a resize listener; the SidePanel slot reads it.

Verified at 1920px: pinning the playlists rail leaves the table at left 487 (moved 0); the rail sits
256px wide ending at 312 — 175px clear of the content. Feed (full width) still pins in-flow.
2026-08-21 21:53:24 +02:00
peter 1801a58f6b feat(layout): centre the header on each page's content width, centre the playlist table
Every page's floating header (search bar + sync chip) and the toolbar band now centre on the SAME
max-width the content column uses, instead of spanning the whole column while a max-w page centres
its grid. So the search bar lines up with the left edge of the cards/table on every page, and sits
centred with the content — not against the full page width.

- PageShell takes a `maxWidth`, published as --content-mw; the header adds max-w-[--content-mw] mx-auto
  inside its left-4 / right-(1rem+scrollbar) span, so it centres on that width (`none` = full width).
- moduleRegistry PAGE_CONTENT carries each page's content max width (plex 1600px, channels 96rem,
  playlists 64rem, downloads 56rem, notifications/users 48rem; feed stays full width) — one source.
- Playlists: its content (header, filter row, table) was left-aligned max-w-5xl → now mx-auto, so the
  table is centred and the header lines up with it.

Verified at 1920px: header left edge == content left edge — feed 340, channels 213, playlists 487.
2026-08-21 21:38:49 +02:00
peter 011acd919e feat(toolbar): lead the Plex toolbar with ItemCount too, unify its gaps
Plex browse now uses the same fixed-width ItemCount pill as the feed (its title count → "N items"),
placed as the leading element, and the toolbar gap tightened 2 → 1.5 to match. One consistent
toolbar shape across the source modules.
2026-08-21 20:37:36 +02:00
peter 45d8b47366 feat(toolbar): fixed-width ItemCount pill, lead the toolbar with it, align chrome to the cards
- New ItemCount: a generic, FIXED-WIDTH count pill (glass, non-interactive) — "192,050 items"
  (neutral noun, dynamic plural), min-w-[9rem] so a changing number never shifts the controls
  beside it; a million or more collapses to "1M+" with the exact figure in the title tooltip.
- Feed toolbar: ItemCount is the fixed leading element now; dropped the mx-1 separators so every
  gap is a uniform 1.5 (Content-types↔Source used to be wider than Watch-state↔Content-types).
- Chrome-to-card alignment (app-wide, via the shared Header + PageShell): the floating header and
  the toolbar band now span exactly the card grid's width — left-4, and a right edge of
  1rem + the scrollbar gutter (measured once into --sbw), so the fixed chrome lines up with the
  scrolled cards instead of running ~15px past them. Wraps cleanly to multiple rows on mobile.

Renames the count testid feed-result-count → item-count (E2Es updated).
2026-08-21 20:33:55 +02:00
peter c3820a0b44 refactor(header): remove the ModuleName module-switcher pill
The user no longer wants the header's "FEED" pill (module label + accent dot + ◀/▶ module-step
arrows). Drop it everywhere — the header now starts with the module's SearchBar; module switching
stays on the nav rail.

Deletes ModuleName.tsx, the now-unused stepModule() in lib/modules, and the Header's
module-order/label/setPage plumbing that only fed it.
2026-08-21 19:58:19 +02:00
peter c7b393901f style(type): correct the trigger text from 16px back to the view-switcher's real 14px
The earlier bump measured the view-switcher's BUTTON (16px, inherited) instead of its visible
label span (text-sm = 14px), and 14px is the size the user actually finds comfortable. Walk the
over-sized triggers back to text-sm: FilterSelect, SortControl, ViewSwitcher, and the Combobox
input (which was 14px before the bump anyway). The secondary text already at text-sm/text-xs
(count, chips, widget headers, section labels) was correct and stays.
2026-08-21 19:53:14 +02:00
peter f1e14c94c5 style(viewswitcher): unify the ViewSwitcher trigger with the FilterSelect pill
The user prefers FilterSelect's rounded-full glass pill over the ViewSwitcher's squarer
rounded-lg trigger. Give the ViewSwitcher trigger the same look — glass-card/glass-hover,
rounded-full, px-3 py-1, open→border-accent, and a 16px label (was text-sm) — so the two read
as one control family. Applies to every ViewSwitcher consumer (feed, channel manager, Plex
playlists).

Native <select>s, the Combobox input box, and playlist/list cards are a different control class
(form fields / variable-height inputs / cards) and keep their own shape.
2026-08-21 19:49:10 +02:00
peter ddb171fca3 style(type): raise the too-small toolbar and filter text toward the view-switcher size
The view-switcher's 16px reads comfortably; the toolbar and filter controls sat at 12px. Bring
the marked controls up to match, and lift the general scale on the shared filter surfaces.

- Toolbar (feed + channel + Plex, via the shared components): FilterSelect and SortControl trigger
  pills text-xs → text-base (16px, the view-switcher size); the result count → text-sm; the
  active-filter chips → text-sm with the "FILTERED BY" label text-2xs → text-xs.
- Filterbar: the Combobox picker input → text-base (matching the toolbar pills) + its chips → text-sm;
  the panel/widget headers (SidePanel collapsed-rail + panel title, PanelGroup widget titles) up one step.

Channel info and the manager tables were already at 14–16px (readable), so they were left as-is.
2026-08-21 19:37:11 +02:00
peter ec3bbc78dc style(layout): tighten the oversized vertical gaps around the page toolbars
The floating-header → toolbar → filter-chips → content stack carried ~2× the whitespace it
needed. Halve it, app-wide where the chrome is shared:
- PageShell: the --hdr-h gap below the floating header 8px → 4px (every page's header).
- Feed: toolbar wrapper pt-4 → pt-2, the toolbar's own pb-3 → pb-1.5, and the grid's top pt-2
  → pt-1. Measured on the feed: header→toolbar 24→12px, toolbar→chips 12→6px, chips→cards 12→8px.
- Plex / Playlists / ChannelDiscovery / DownloadCenter / Notifications / AdminUsers: toolbar
  wrapper pt-3 → pt-2, so every page's top band matches the tighter feed.

PageShell.dom.test updated for the 4px gap.
2026-08-21 19:13:38 +02:00
peter 1be1dc9d06 feat(sort): give SortControl the FilterSelect pill look with a footer direction segment
Rebuilds SortControl's internals (API unchanged, so all five consumers — feed, Plex,
playlists, and the two table/card ColumnSortControl sites — get it for free) as a labeled
"Label: key ↓ ▾" pill on the AnchoredPopover platform, matching FilterSelect. The listbox
lists the sort keys; an asc/desc segment sits in the panel footer (with the trailing slot —
feed reshuffle, playlists dirty-first — beside it).

Two-dimensional key+direction, so it stays its own component rather than reusing FilterSelect:
the trigger is a single plain button (no nested button), picking a key keeps the panel open so
key AND direction can be set in one visit, and picking "no sort" (tables' defaultLabel) closes
it. Directionless keys (shuffle) hide the segment and drop the trigger arrow.

Drops the now-redundant external "Sort" label spans in Feed and PlexBrowse (the trigger carries
the label now). Rewrites SortControl.dom.test for the popover UI.
2026-08-21 18:11:45 +02:00
peter 6109e0c6f1 feat(feed): convert the Source filter to the FilterSelect dropdown
Replaces the native <select> "Source" control with a labeled FilterSelect pill, matching
the Watch state / Content types pills — one consistent toolbar idiom. Single-select over
the three librarySource values (organic / all / search-only).

Sort and View are intentionally left as-is: Sort is a two-dimensional key+direction control
(plus reshuffle and a tables-only "no sort") shared across five surfaces, and View is already
on the same AnchoredPopover platform with an icon-only trigger tuned for this packed toolbar —
FilterSelect's text trigger would regress it. Neither fits FilterSelect's one-dimensional model.
2026-08-21 17:36:33 +02:00
peter 5851ddd329 feat(feed): compress the toolbar chip groups into labeled FilterSelect dropdowns
Adds a reusable FilterSelect: a "Label: value ▾" pill that opens a listbox (a checkmark
row for single-select, a checkbox row for multi), on the AnchoredPopover platform. It
replaces the two feed-toolbar chip walls:
  - Watch state (single-select, default All) — picking a row replaces + closes.
  - Content types (multi-select, default Normal; requireOne keeps at least one on) — rows
    toggle and the panel stays open.
The trigger summarises multi picks as "First +N". The component is source-agnostic, meant
for reuse anywhere a small known option set currently renders as a chip group.

FilterSelect takes a `testId` → the trigger gets it, each option gets `${testId}-${value}`,
so the feed/channel E2Es keep stable selectors. They now open the dropdown and assert the
option's aria-selected instead of the old inline aria-pressed chips (new expectContentType/
toggleContentType helpers).
2026-08-21 17:28:57 +02:00
peter ad0cf5827b feat(feed): use the Combobox picker for Channel/Your tags/Topic/Language filters
Replaces the hand-rolled channel search list and the flat tag chip walls with the
searchable, ARIA-correct Combobox built for the Plex filter bar (E5), so the YT feed
filters get the same input+listbox+chips UX with facet counts.

- ChannelPicker now wraps Combobox (selectedLabels keeps picks named); drops its bespoke
  input + scroll-fade list.
- Sidebar language/topic/tags: each Combobox owns only its slice of the shared
  filters.tags list (setTagSelection replaces that list's ids, leaves the rest). Topic
  keeps its any/all match toggle, Your tags keeps Manage. TagChip removed.
2026-08-21 17:01:41 +02:00
peter f6192aa6bf fix(feed): re-measure header on feed↔channel Back so --hdr-h can't stick at 8px
PageShell is reused (never remounted) across feed↔channel navigation — same
component position in App.tsx — so a one-shot useLayoutEffect kept its
ResizeObserver bound to the FIRST header node. On channel-open that node detached
and fired the observer with a 0 box, rewriting --hdr-h to 0+0+8=8px on the
still-mounted content div. Back restored the pt-(--hdr-h) pad class with that
stale value, sliding the content toolbar up under the header — stuck until reload.

Hold both the header and content nodes in state (callback refs) and key the
measurement effect to the live nodes, so it re-runs every time the header
mounts/unmounts; drop the stale pad when the header is absent (channel page).

Adds a regression test: header removed then restored must re-measure, not 8px.
2026-08-21 16:51:05 +02:00
peter 6b4febcc95 release: 0.71.0 (improvement/e5-s1-plex-platform)
gate / frontend (push) Failing after 1m9s
gate / backend (push) Successful in 1m57s
2026-08-21 08:26:31 +02:00
peter cb5ac09a80 chore(e5): bump to 0.71.0 + release notes for the Plex Experience epic 2026-08-21 08:21:21 +02:00
peter bad2aae753 fix(e5): deep-review round — combobox Enter, parse_person collision, enrich hardening + coverage
Full-epic /deep-review (11 finders + 12 adversarial verifiers) surfaced no HIGH; every
mechanism finding calibrated to LOW on verification. Fix all 13:

Correctness / reliability:
- Combobox: Enter now commits the VISUALLY highlighted row (highlightIndex ?? activeIndex),
  not the bare keyboard activeIndex — so arrow-then-hover-then-Enter no longer toggles a
  different option than the one painted, and hover-with-no-arrow is no longer a silent no-op.
- parse_person: dedup the known_for pool by (id, media_type), not the bare id — TMDB movie
  and tv id spaces are independent, so a same-id movie+show pair is two distinct credits
  (matches the collision-safe key the rest of the module already uses).
- person-enrich: a per-person in-flight + 60s cooldown guard (people._should_schedule_enrich)
  dedups concurrent opens and rate-limits re-triggering while TMDB is degraded (a transient
  TmdbError leaves the watermark NULL), so bursts can't pile 8s tasks onto the shared threadpool.
- PlexInfo compact()/money(): format with i18n.language (keeping currency USD) instead of a
  hardcoded en/en-US, matching the app's Intl convention.
- delete the dead `plex.filter.add` i18n key (both locales) — the +Add button that used it is gone.

Reuse:
- _continue_watching reuses rollup.agg_subquery(include_resumed=True) + query.apply_show_state_filter
  instead of a third hand-inlined copy of the per-show rollup + in_progress predicate, so the
  Discover rail can't drift from the grid's definition.

Test coverage (the gaps a regression would ship through):
- resolve_library_guids scope-exclusion (a credit in a non-scoped/disabled library is dropped)
  + the []-vs-None distinction; parse_person cross-media-id retention; enrich_person transient
  (NULL watermark) and 404 (stamp) branches; _enrich_person_bg body (fresh session close/rollback,
  None-guard) + the cooldown dedup; continue_watching fully-watched-show exclusion; Combobox
  Enter-commits-highlighted + async/controlled-query mode; PopoverItem aria-checked contract;
  RatingsRow missing-IMDb + imdb==audience branches.

Backend lane 413→420, FE vitest 351→357; all gates green. No migration.
2026-08-21 08:11:18 +02:00
peter 8f6714e678 refactor(e5): deferred review-tail (frontend) — Carousel reuse + shared scroll core
The last of the deep-review deferred reuse items:

- extract the scroll-affordance machinery (watch the scroller AND its swappable/
  lazily-growing children, diff two edge booleans) that useScrollFade (vertical) and
  Carousel's useHScroll (horizontal) each hand-duplicated into one shared
  lib/useScrollEdges(axis); both hooks now just map start/end to their own shape.

- migrate PlexInfo's two remaining hand-rolled `flex overflow-x-auto` strips — the
  cast row and the collection strips — to the S1 <Carousel> primitive, so they get the
  edge fade + hover/focus scroll buttons (only on the clipped side) for free, matching
  the Discover rails / CastStrip / RelatedStrip.

Pure refactor, no behaviour change. Gate green (tsc, vitest 351, eslint, prettier,
knip). Browser-verified on a movie info page: cast row + collection strip render as
Carousels with the edge fade and the right-scroll button on the clipped cast row.
2026-08-21 06:16:15 +02:00
peter 13fb2b4e61 refactor(e5): deferred review-tail (backend) — enrich off request path, reuse, tests
Address the deep-review's deferred backend items so E5 is review-complete:

- person-enrich OFF the request path (ABUSE finding): person_detail no longer makes
  the TMDB call inline (even bounded to 8s it sat on the serve thread, un-throttled,
  demo-reachable — it violated the module's "no live TMDB call at serve time"
  invariant). It now schedules a FastAPI BackgroundTask (people._enrich_person_bg,
  fresh session, best-effort) after the response; the first open serves the row as-is
  and the bio lands by the next open. The inline try/rollback recovery is gone with it.

- reuse: a shared tmdb.resolve_library_guids owns the collision-safe guid→library
  bucketing (movie vs tv ids are independent sequences) that resolve_recs and
  people._resolve_known_for each hand-duplicated; a new rollup.item_state_map dedups
  the "PlexState-by-item_id" fetch across the person + discover paths.

- test gaps: enrich_batch transient TmdbError leaves the watermark NULL to retry;
  guid_to_tmdb_id resolves a tvdb source + tv media_type via /find's tv_results;
  _continue_watching includes a NULL-status started movie and excludes a hidden one;
  /api/search/youtube returns 422 (not 5xx) when ingest fails (route-level, B-core-1).

Backend lane 403 → 413, ruff clean. No migration; serve contracts unchanged.
2026-08-21 06:09:17 +02:00
peter 5316f48af2 feat(e5): first-class combobox platform mode + universal Combobox (A11Y-1)
The E5 filter pickers were a hand-rolled floating-ui combobox (MultiSelectChips)
with no combobox ARIA, no arrow-nav, and options in a body-portal — keyboard/SR
users could not operate it, and the bare role=listbox/option wiring actively
misled screen readers (deep-review A11Y-1, CONFIRMED 3/3 medium). Rather than
one-off-hack the a11y, add combobox as a first-class mode of the AnchoredPopover
platform and rebuild the pickers on a reusable component.

Platform (AnchoredPopover):
- new mode "combobox": reuses the whole positioning core (offset/flip/shift/size/
  hide, isPositioned + transform-clobber flash guards, FloatingPortal + fullscreen
  re-parent, topmost-only Escape layer). Differs only in the focus model — no
  useClick (open is field-driven), useRole role="combobox" so Floating UI wires the
  full ARIA tree (input role=combobox + aria-expanded/controls/autocomplete/
  activedescendant, listbox surface, option ids), and useListNavigation virtual:true
  so ↑/↓ move an aria-activedescendant pointer while DOM focus STAYS on the input.
- PopoverSurface: combobox surfaces host the item registry but are NOT focus-trapped.
- PopoverItem: virtual mode carries a stable id (the activedescendant target), never
  takes roving DOM focus, keeps the input focused on click (mousedown preventDefault),
  and its checked/selected truth wins over getItemProps' single-select guess.

Component (Combobox.tsx, new): the universal react-select/Overseerr combi-input —
chips-in-field + typed search + checklist. Supports multi/single select, client OR
server (async) filtering, custom rows, the big-vocabulary render cap + refine footer,
and a selected-label cache so a server-narrowed single-select still names its chip.

Rollout: MultiSelectChips (genre/age-rating/studio) → Combobox; the CollectionPicker
"+Add" popover → a single-select async Combobox (also fixes its arrow-keys-eaten
story), unifying the two filter idioms. PlexSearchBox is a documented future consumer.

Gate green (tsc, vitest 351, eslint, prettier, knip). Browser E2E on localdev:
role=combobox confirmed on both pickers; ↑/↓ moves aria-activedescendant with focus
staying on the input; Enter + click select; multi-select chips accumulate; Backspace
removes the last chip; async collection server-search picks + closes. No backend change.
2026-08-21 05:43:36 +02:00
peter 0c746f5bcd fix(e5): deep-review survivors — TMDB key log leak, person-enrich rollback + timeout
Three verified deep-review findings (the 4th, the filter combobox a11y gap, is
deferred to a dedicated session on the combobox architecture):

- SEC (medium): TmdbClient sent the api_key as a URL query param, so an HTTP
  error's httpx exception string embedded the full URL incl. the key, which
  enrich_batch logged. _get now raises status+path only (never str(e)/the URL)
  and severs the secret-bearing cause with `from None`. Regression test asserts
  neither the status nor the network error path contains the key.
- STATE (low): person_detail's lazy-enrich except now db.rollback()s, so a
  failed enrich_person commit can't poison the session and 500 the subsequent
  SELECTs (SQLAlchemy 2.x deactivates the tx on a commit-time flush failure and
  does not auto-reset it). Test locks that the page renders despite a poisoned
  enrich.
- ABUSE (low): the request-path lazy person enrich now caps its TMDB client at
  an 8s timeout (was the 30s default) so a slow TMDB can't tie up a worker;
  TmdbClient gained a `timeout` param (batch keeps 30s). The fuller fix (move
  person enrichment off the request path) is logged for later.
2026-08-21 04:42:07 +02:00
peter caced8664c fix(e5): stop the filter dropdown flashing at the origin before it positions
The combi-input popover flashed in the top-left corner for a frame, then
jumped into place. Two causes, both confirmed with a per-frame capture in
the browser:

1. The panel painted before floating-ui computed a position — gated its
   visibility on `isPositioned` (hidden until positioned), same as
   PopoverSurface.
2. The bigger one: the `popIn` entrance animation animates `transform`
   (scale/translateY), and it sat on the SAME element floating-ui positions
   via `transform` — so during the 0.16s animation the keyframes OVERRODE
   the translate(x,y) and the panel rendered at the origin. Moved the glass
   styling + popIn onto an inner child, leaving the positioned element's
   transform untouched.

Frame capture after the fix: frame 0 hidden, first visible frame already at
translate(82,474) — never visible at the origin. Also gave the size
middleware a min width for a narrow rail.
2026-08-21 04:00:54 +02:00
peter 29f2c9059e fix(e5): rebuild the filter combi-input on floating-ui primitives (focus, dismiss, z-order)
The AnchoredPopover-based version had three UAT-caught bugs: the field
showed an ugly "Add" placeholder; the first click couldn't type (the
platform's FloatingFocusManager + focusItemOnOpen pulled focus onto the
list, blurring the field until a second click); and an outside click didn't
close the list. A visual pass also caught the dropdown rendering BEHIND the
sidebar panel (z-menu 20 < z-panel 35).

Rebuilt on floating-ui primitives directly — no focus manager, so focus
stays in the field; open is driven by field focus; useDismiss closes on
outside-press/Escape; the list is a z-popover (60) FloatingPortal sized to
the field width. Placeholder is now a plain "Search" (empty once chips
exist). Verified in the browser: type-on-first-click, filter, chip add,
outside-close, and the list painting above the sidebar.
2026-08-21 03:37:14 +02:00
peter b83248c510 fix(e5): Plex detail/UI polish from review (backdrop reset, layout, combobox, labels)
User-reported issues after the E5 build:

- useCrossfadeBackdrop: the static branch (one image / crossfade off /
  reduced motion) painted #root but returned NO cleanup, so the backdrop
  persisted after leaving the page (e.g. switching modules). Every painted
  branch now returns clear() so unmount always resets the default backdrop.
- scheduler: give the tmdb_enrich job a proper label + description (was the
  raw "tmdb_enrich" id) in EN/HU.
- CastStrip: cap crew names to ONE line (truncate) so the row of circular
  photos sits on one baseline instead of going wavy on 2-line names.
- PlexInfo: lay Details / Keywords / Crew out as side-by-side boxes
  (items-start grid) instead of full-width stacked panels with dead space;
  narrower Crew grid for the boxed width; each panel independently hideable
  from the customize menu (new Details/Keywords/Crew toggles). Tighter
  vertical spacing.
- MultiSelectChips: reshape the trigger from a "+Add" button into an
  Overseerr-style combi input — removable chips inside a typeable field that
  filters the checklist, opening on focus (backspace removes the last chip).
- Discover / person rails: reduce the vertical gap between rails.
2026-08-21 03:16:40 +02:00
peter 2b5de89eda fix(e5): address code-review findings (TMDB guid media_type, search ingest guard, discover keying)
Follow-ups from the epic-end /code-review of the E5 branch:

- tmdb.resolve_recs + people._resolve_known_for: match recommendations/
  credits to the library bucketed by media_type. TMDB movie ids and tv ids
  are independent sequences, so a bare `tmdb:<id>` match could deep-link a
  movie rec to an unrelated show (or vice-versa) on a numeric collision.
  Regression test added (test_resolve_recs_matches_by_media_type_not_bare_id).
- routes/search: wrap the _ingest_candidates call so a DB/enrich failure
  degrades to a 422 with a message instead of a bare 500 hidden behind the
  FE's generic "search failed" (extends the B-core-1 rationale); rollback on
  failure. Stamp enriched_at with its own timestamp taken after the enrich,
  not the stub-insert placeholder.
- routes/plex/discover: key serialized cards back by rating_key (globally
  unique in Plex) instead of id(obj); fix the stale rows 3-tuple annotation.
2026-08-21 03:16:24 +02:00
peter ff45db70ae fix: live YouTube search 500 on the scrape source (published_at NOT NULL)
The scrape source exposes no publish date, so its search stub carried
published_at=None. Migration 0068 (PERF-15, E2/0.68.0) made
videos.published_at NOT NULL, so insert_stubs raised IntegrityError —
outside the route's page-fetch try/except — and the whole
/api/search/youtube request 500'd ("YouTube search failed."). Every
scrape search (the default source) had been broken since 0.68.0.

- routes/search._ingest_candidates: stub published_at falls back to
  now() (mirrors stub_from_playlist_item), satisfying the constraint.
- sync.videos.apply_video_details: prefer the enrich snippet's real
  date whenever present, so the now() placeholder is corrected instead
  of becoming the video's permanent date (the guard only filled a NULL,
  which a non-null placeholder no longer is). Never clobbers a real
  date with now().

Regression tests (DB lane): dateless scrape stub ingests and takes the
real enrich date; an enrich miss keeps a non-null placeholder, unkept.
2026-08-21 02:15:09 +02:00
peter 7d40a8780d E5 S8: crossfading art backdrop (detail + person pages)
The faint HTPC art backdrop now slowly cross-dissolves through a pool of images
instead of showing one static art — the last E5 sprint.

- useCrossfadeBackdrop (replaces useArtBackdrop): a double buffer — the SETTLED
  image lives on #root's background (the proven full-viewport layer); the next
  image cross-fades in on a z-index:-1 overlay layer, then is committed and the
  overlay reset. ⭐ #root is given `isolation: isolate` so it's a stacking context
  — without it the z:-1 overlay paints BEHIND #root's own background and is
  invisible (found + fixed live: a red test layer + a real TMDB backdrop both
  only showed once #root was isolated).
- Art pool: the item's own art + its TMDB backdrops (E5 S2, rendered straight
  from the TMDB CDN — the CSP allows it) + its collection siblings' art; the show
  page adds related art; the PERSON page cross-dissolves through the person's
  films' art (the roadmap's signature touch).
- HARD-OFF under prefers-reduced-motion (static, no overlay) + a new usePref
  "Crossfade backdrop" toggle in the detail customize menu (fade 1.6s / 7s cycle).

Wired into PlexInfo, PlexShowView, PlexSeasonView, PlexPersonView. Tests:
plexBackdrop.dom.test.tsx (5: paint+isolate+overlay, reduced-motion off, single
image, animate=false, disable/unmount cleanup). Browser-E2E'd (isolation +
overlay render + a real backdrop crossfade + the toggle). Gate: vitest 347,
backend lane 403 (unchanged — FE only).

▶ E5 · Plex Experience epic COMPLETE (S1–S8).
2026-08-21 01:00:11 +02:00
peter d56a791c04 E5 S7: Plex filter UI — listbox-select pickers + studio filter
Reworks the always-expanded filter chip walls into compact, Seerr-style
"+ Add" → searchable-popover pickers (the user's space-saving ask), and adds
the missing studio filter.

- New MultiSelectChips: selected values as removable chips + a "+ Add" trigger
  opening a searchable checklist (AnchoredPopover platform — menuitemcheckbox,
  arrow/Escape, anchor flip). Options carry facet counts; a large vocabulary
  renders a 50-row cap + "N more — refine with search" footer (studios run to
  ~1500). Replaces the flat Genre + Age-rating chip lists.
- New Studio filter via the same picker. Backend: /facets now aggregates the
  studio vocabulary (distinct studio + count, self-excluded like genres). The
  studio filter itself already existed (only addable from the detail page before).
- Collection converted to the same "+ Add" pattern (CollectionPicker): a
  single-select popover with server-side search + the 50-cap; the selected
  collection still shows as a removable chip. Retires the always-inline search+list.

New i18n plex.filter.studio/add/moreRefine + common.search/remove/noResults
(en+hu). Tests: MultiSelectChips.dom.test.tsx (5, incl. the cap) +
test_plex_facets studio-vocabulary. Browser-E2E'd: Genre→Horror(402),
Studio→A24(14, capped/searched), Collection→James Bond(26)/harry-search.
Gate: vitest 342, backend lane 403.
2026-08-21 00:40:31 +02:00
peter 187b8007f8 E5 S6: enriched person page (TMDB bio + birth facts + Known-for rail)
The person page now leads with a TMDB bio, lazily fetched the first time it's
opened (off the S2-linked `tmdb_person_id`) and then served from the row:

- Migration 0072: ADD-only nullable person columns on plex_people — biography,
  birthday, deathday, place_of_birth, known_for_department, also_known_as,
  known_for ([{tmdb_id, media_type, title}]), tmdb_enriched_at.
- tmdb.py: TmdbClient.fetch_person (/person/{id}+combined_credits), parse_person
  (bio + a popularity-ranked, deduped known_for), apply_person, enrich_person
  (lazy one-shot, key+id guarded, commits, 404 stamps the watermark).
- people.py person_detail: loads the full person row, lazily enriches (best-effort
  — never fails the page), serves the bio fields + resolves known_for to library
  rows by guid (in-library only, popularity order).
- FE PlexPersonView: bio/birth-facts/aka hero + a "Known for" Carousel rail above
  the existing Movies/Shows filmography; PlexPersonDetail type + i18n (en+hu).

Tests: test_plex_person_bio.py (6: parse / lazy-enrich guards / serve) + extended
PlexPersonView.dom.test.tsx + the unknown-name contract in test_plex_person.py.
Browser-E2E'd on Jim Carrey (live TMDB enrich → bio 1937 chars, born 1962-01-17
Newmarket, Known-for resolved). Gate: vitest 337, backend lane 402.
2026-08-21 00:08:19 +02:00
peter f10915c223 E5 S5: enriched Plex detail page (ratings + facts + keywords + crew)
Consumes the TMDB block the S2 engine already persists + serves (movie/show
detail's `tmdb`), turning the movie info page into a Seerr-style enriched detail
(page variant only — the player overlay stays lean):

- Multi-source ratings row: IMDb (links out) · TMDb (with vote count) · Plex
  audience — whichever are present. Only the primary score (IMDb, else audience)
  carries the ratingMin filter, mirroring the backend rating column.
- Facts panel: release · runtime · cert · budget · revenue · studios · countries.
- Keyword chips → tap to filter the library. Backend: a new `keywords` filter on
  LibraryFilters + meta_filter_conds (`keywords @> [kw]`, both models), wired
  through PlexFilters (appendPlexFilters / plexFilterCount / active-chip / clear).
- Full crew (TMDB key-crew) → tap a name to open their person page.

Tests: test_plex_keyword_filter.py (3) + PlexInfo.dom.test.tsx (5, incl. the
overlay-stays-lean guard). New i18n plex.info.facts.* / keywords / crew / audience
(en+hu). Gate: vitest 337, backend lane 396.
2026-08-20 23:50:50 +02:00
peter f9b63745f9 E5 S4: Plex Discover rail-surface (default landing)
A dedicated Plex "Discover" landing — a stack of horizontal Carousel rails
served from ONE consolidated read (serve-from-DB, no live Plex/TMDB call):

- GET /api/plex/discover → continue_watching (in-progress movies + shows, most
  recently progressed) · recently_added · genres (top-N tiles) ·
  because_you_watched (recs of the most-recent watched, TMDB-enriched movie,
  resolved to library rows by guid via serve_block's resolver).
- GET /api/plex/surprise → a fresh random, scope-bounded sample for the ↻ rail.

Frontend: PlexDiscover.tsx (rails reuse the S1 Carousel + S3 PlexPosterCard;
colored genre tiles → tap to filter the grid; ↻ reshuffles the surprise rail
alone). Discover is now the root sub-view; the flat grid becomes the bookmarkable
"Browse all" secondary view (?v=browse) reached by a header search/filter (a
browse-intent flip) or the new Discover⟷Browse toggle.

resolve_recs is promoted from _resolve_recs (now cross-module). Tests:
test_plex_discover.py (rail shape/order, continue-watching, because-you-watched
+ null, surprise bounds, scope) and PlexDiscover.dom.test.tsx; plexSubUrl
round-trip updated for discover-root/grid-browse. Backend lane 393, vitest 332.
2026-08-20 22:15:54 +02:00
peter 5b5e76b73d E5 S3: unified Plex card — scrim-reveal hover + rest polish
The poster card (grid + person page) gets the Seerr-style hover: on hover a
bottom scrim slides up with the title, a "year · runtime · ★rating" meta line,
and a 3-line synopsis — the info you'd otherwise click through for — plus a
less-flat play ring (radial-gradient + ring shadow) and a top-right quick-action
cluster (Info + watched-toggle, revealed on hover / always on touch). Rest-state
badges (season / type tag) yield to the scrim on hover; the playable-cost tint
stays. (§1 A + C from the enrichment study.)

Backend (the data the scrim needs, mirroring S1's `art`): the grid
(browse.unified_library) now emits a lang-coalesced, truncated `summary` (new
query.summary_col) + the Plex `rating`; movie_card/show_card do the same (EN) for
the detail/related/person cards. PlexCard.rating type added (summary already
existed for episodes).

Gate: tsc x3 / eslint / prettier / knip / vitest 326 green; ruff green; backend
lane full suite green. Self-E2E on the live dev server (rebuilt on this branch,
S2-enriched): the grid serves rating+summary (Shawshank 9.3, Godfather 9.2, real
synopses) and the card renders the play ring + scrim (The Expanse: ring, synopsis,
★8.1) — verified via the DOM (the pane wouldn't composite for a screenshot).
2026-08-20 20:30:30 +02:00
peter 91be3d0535 E5 S2: prioritise TMDB crew, and capture the TV showrunner
A live smoke against the real TMDB API showed two crew problems the mocked
tests missed: (1) a TV show's creator/showrunner lives in the payload's
top-level `created_by`, not the crew list, so it was dropped entirely; and
(2) TMDB's crew order is arbitrary — a TV show's per-episode Directors (dozens)
flooded the key-crew list and buried the show-level roles.

Fix: split the accepted crew jobs by media type — movies key on
Director/Writer/…; TV keys on Creator (from created_by) + the show-level
Executive Producer/Writer/Producer/Composer, deliberately excluding the
per-episode Director/Editor/DP noise. Stored crew is sorted by display
priority and capped, so the lead roles come first.

Verified live: Breaking Bad -> [Gilligan (Creator), Johnson (EP), … , Writer];
The Godfather -> [Coppola (Director), Puzo (Screenplay), …]. test_plex_tmdb
green (9).
2026-08-20 09:24:25 +02:00
peter 3bde114b14 E5 S2: TMDB enrichment engine (Plex catalog)
Matches each mirrored Plex movie/show to TMDB by its stored canonical guid,
fetches once (append_to_response), and PERSISTS the enrichment onto the row —
served from the DB, never a live TMDB call at serve time. Additive: the
Plex-sourced imdb_rating/rating are untouched; tmdb_rating is the distinct
TMDb community score (ratings scope = TMDb + Plex's IMDb/audience, no OMDb/RT).

- config.py: tmdb_api_key + api_base/language/interval/batch defaults.
- sysconfig SPECS: tmdb_api_key (secret, Fernet) + tmdb_language, auto-rendered
  in the admin Config page; read like the Plex token.
- migration 0071 (ADD-only nullable, reversible): plex_items + plex_shows gain
  tmdb_id / tmdb_enriched_at (watermark) / tmdb_rating / tmdb_vote_count /
  budget / revenue / keywords / crew / studios / production_countries /
  recommendations / backdrops; plex_people gains tmdb_person_id (the S2
  foundation for S6 person bios). No watermark reset needed.
- app/plex/tmdb.py: TmdbClient (mirrors PlexClient; throttled; 404->None,
  transient->TmdbError), parse_tmdb / apply_tmdb_detail (one-parser rule),
  guid_to_tmdb_id (tmdb: direct, imdb:/tvdb: via /find), enrich_batch (movies
  then shows, transient error leaves the watermark NULL to retry, definitive
  no-match still stamps so it isn't re-fetched), _link_people (upserts
  tmdb_person_id by name), serve_block (recommendations resolved to library
  rows by guid), run_enrich (no-op without a key).
- scheduler: tmdb_enrich nightly job (default 120 min, no YouTube quota),
  runtime-tunable + "run now" from the Scheduler dashboard.
- detail endpoints emit `tmdb` (movies + shows) so it's curl-verifiable now;
  the FE render lands in S5.

Note: _link_people inserts plex_people rows for TMDB cast/crew; unreferenced
crew (composers etc., not in cast_names/directors) get pruned by the orphan
sweep — acceptable, the person page is cast-driven and cast rows survive.

Gate: ruff + py_compile green; backend test lane FULL suite 387 passed
(incl. migration drift downgrade->re-upgrade + test_plex_tmdb 9 tests).
Frontend unchanged since S1. Not merged (epic branch, stacks on S1).
2026-08-20 09:17:43 +02:00
peter fca3fa13ef E5 S1: shared Carousel primitive + Plex card-art serializer
Wave 5a building blocks under the Seerr-inspired Plex enrichment (E5).

Carousel (frontend/src/components/Carousel.tsx): the one horizontal-rail
component, replacing the hand-rolled `flex overflow-x-auto` strips and the
home of the future Plex-Discover rows. Edge-fade mask, hover/focus scroll
buttons that appear only on the clipped side (useHScroll — the X-axis twin
of useScrollFade, same watch-the-children discipline), native scrollbar
hidden behind .no-scrollbar, reduced-motion-aware paging. First consumers:
CastStrip + RelatedStrip now scroll inside it (dedup + free affordances).
New i18n: common.seeAll / scrollLeft / scrollRight (en+hu).

Card-art serializer (backend/app/plex/cards.py): movie_card + show_card now
emit `art` (the wide backdrop, /api/plex/image/{rk}?variant=art) so rails and
the crossfading hero get a landscape image with no per-item fetch — gated on
art_key so an art-less item stays null and falls back to the poster. FE type
PlexCard.art?.

Tests: Carousel.dom.test (children/header/see-all, clipped-edge buttons +
paging), test_plex_cards (art present/absent by art_key). Added a
ResizeObserver stub to PlexPersonView.dom.test (it renders CastStrip, now a
Carousel consumer).

Gate: tsc x3 / eslint / prettier / knip / vitest 324 green; ruff green;
backend test lane (docker-compose.test.yml) green on the plex card tests.
2026-08-20 08:50:47 +02:00
peter df60bc5ff5 merge: E4 test-coverage backfill (deferred deep-review gaps + 2 review rounds)
Closes every ▶ DEFERRED test-coverage item from the E4 deep-review (Modal
focus-trap, useHistorySubview popstate URL-fallback, PLEX_SUB_URL round-trip,
loadInitialPage precedence + channelViewFrom, readSelectedId ?pl guard,
useUrlMirror dep-keying, usePref flush-on-unmount) + the a11y sweep, and the
findings from two /code-review high rounds. Test-only + behaviour-preserving
seams (parseId/navigationInit/playlistsInit/plexSubUrl extractions); no
user-facing change, no version bump. vitest 257→321. Not deployed.
2026-08-20 05:56:16 +02:00
peter 99d63d272e test(e4): lock accessible names on the Plex card buttons
Closes the re-review's test-coverage finding: a plexCards.dom.test asserting the
icon-only card controls (watched-toggle, Info, add-season/episode) each expose
their aria-label as an accessible name via getByRole({name}). A future edit that
drops a label now fails the gate instead of silently reopening the unnamed-button
gap the a11y sweep closed.

4 tests; full FE gate green (tsc/eslint/prettier/knip, 321 vitest).
2026-08-20 05:52:36 +02:00
peter 482dd28006 fix(e4): address code-review findings (a11y sweep + dedup guards)
Resolves the 5 /code-review high findings on this branch:
- a11y sweep: extend aria-label to the remaining icon-only Plex card buttons the
  first pass missed — PlexPosterCard Info, SeasonCard watched-toggle + add-season
  (EpisodeCard already had them). "Fix a principle everywhere, in one sweep."
- new shared lib/parseId.ts `parsePositiveIntId`, used by BOTH readSelectedId
  (now guarding the stored ?pl fallback the same as the URL — was unguarded) and
  plexSubUrl's ?v=playlist decode; removes the hand-duplicated guard.
- PAGE_KEY exported from navigationInit and imported by NavigationProvider (was
  defined in both).
- hoist a `watchedLabel` const per card so title + aria-label can't drift.

+8 tests (parseId unit + guarded-stored ?pl); full FE gate green (tsc/eslint/
prettier/knip, 317 vitest).
2026-08-20 05:40:59 +02:00
peter 0b7180ce8f a11y(e4): aria-label on the icon-only Plex watched-toggle buttons
The hover watched-toggle on Plex movie/episode cards is icon-only (WatchedIcon,
no text), so it carried a `title` but no accessible name for screen readers —
and the empty-Circle unwatched glyph could read like a radio. Mirror the title
into `aria-label` (the Modal close-button idiom) so the control announces its
action. The PlexInfo watched buttons already have visible text labels, so they
need nothing.

FE gate green (tsc/eslint/prettier/knip, 309 vitest).
2026-08-20 05:25:48 +02:00
peter d2f2085f1f test(e4): readSelectedId ?pl guard, useUrlMirror dep-keying, usePref flush
The remaining E4 deep-review deferred gaps, in one sprint:
- readSelectedId split into playlistsInit.ts (react-refresh-clean, testable) —
  ?pl accepted only as a positive integer; empty/0/negative/fractional/non-numeric
  fall back to the stored selection; null when neither is present.
- useUrlMirror: mounts write the param, a same-serialized rerender does NOT
  re-fire (stable dep), a null value clears the key, other params preserved.
- usePref: a pending debounced write is FLUSHED on unmount (not dropped); an
  unmount with nothing pending stays silent.

20 new test cases; full FE gate green (tsc/eslint/prettier/knip, 309 vitest).
2026-08-20 05:24:11 +02:00
peter 062015ba27 test(e4): loadInitialPage precedence + channelViewFrom (S1 fix)
Split the two URL-derived nav seeds out of NavigationProvider into
navigationInit.ts so they're unit-testable and the provider stays
react-refresh-clean (component-only exports). channelViewFrom also dedups the
identical _chan/?chan resolve the channelView seed and the onPop handler each
hand-rolled — locking the S1 fix (a popstate on a URL-only bookmark entry must
restore the channel from ?chan, not close it). Pure code motion.

Tests: the full loadInitialPage precedence matrix (?page > ?v > ?pl >
filters/?chan > stored landing > feed, with the beats-a-lower-param cases) and
channelViewFrom (_chan+name, ?chan fallback, state-over-url, closed → null).

15 tests; full FE gate green (tsc/eslint/prettier/knip).
2026-08-20 05:21:13 +02:00
peter 0401dae689 test(e4): extract PLEX_SUB_URL for round-trip coverage
Move `type Sub` + `PLEX_SUB_URL` out of the 538-line PlexBrowse into a small
plexSubUrl.ts so the encode/decode round-trip and its guards can be unit-tested
without dragging the component (and its lazy hls.js player) into the test graph.
Pure code motion — PlexBrowse imports the two symbols; the `SubviewUrl` import
moves with them.

Tests lock: encode of each bookmarkable kind, grid/player → null, encode→decode
identity, a person name containing ':' round-tripping, and the decode guards
(positive-integer playlist id; season needs both ids; non-empty show/info/person
arg; unknown kind/garbage → null).

12 tests; full FE gate green (tsc/eslint/prettier/knip).
2026-08-20 05:16:06 +02:00
peter e8a1c252a9 test(e4): useHistorySubview seed + popstate URL-fallback (S2 fix)
Backfills the E4 deep-review's deferred coverage for the subview history
primitive. The keeper case is the S2 bookmark fix: a popstate that lands on a
URL-only entry (no history.state._sub) must restore the sub-view from the ?v
param via the SAME fallback the mount seed uses, not collapse to root.

Covers: seed from URL vs state._sub vs root, the no-SubviewUrl state-only
path, open() push+mirror + encode→null key-clear, and popstate restore from
state / from URL (the fix) / collapse-to-root.

9 tests, gate green.
2026-08-20 05:14:12 +02:00
peter fa8bdf83a8 test(e4): Modal focus-trap coverage (FE-9 deferred gap)
Backfills the E4 deep-review's deferred Modal focus-management test. jsdom
returns null for every offsetParent (no layout), so focusablesIn() would see
zero controls; the test stubs HTMLElement.prototype.offsetParent to report
"visible" for connected nodes (what a real browser reports for the shown
dialog), and a second describe documents the raw-jsdom empty-guard path.

Covers: initial focus on open, Tab/Shift+Tab edge-wrap, pull-back on escaped
focus, the active===panel leak guard, no-trap in the interior, Escape closes
the topmost only, and restore-to-trigger on unmount. useBackToClose is mocked
(its history stack is covered by history.dom.test.ts).

10 tests; full FE gate green (tsc/eslint/prettier/knip, 257 vitest).
2026-08-20 05:12:33 +02:00
peter 868ff533ee style(e4): prettier-format two S6-touched files to unblock the 0.70.0 gate
gate / frontend (push) Successful in 1m2s
gate / backend (push) Successful in 2m1s
The reveal-class swap in S6 shortened two className lines (VideoCard action row,
SavedViewsWidget default-star), which prettier wants collapsed to one line. The
local `npm run lint` doesn't run prettier --check, so it slipped past the sprint
gate and only the release `siftlode check` caught it. Formatting only, no behavior
change. Fixes the pre-publish gate for the already-merged 0.70.0 on main.
2026-08-20 03:52:11 +02:00
peter 00bf9a39f7 release: 0.70.0 (improvement/e4-frontend-platform)
gate / frontend (push) Failing after 38s
gate / backend (push) Successful in 2m51s
2026-08-20 03:50:02 +02:00
peter 640a8efeed chore(release): 0.70.0 — E4 Frontend Platform (bookmarkable URLs + touch a11y)
Bump VERSION to 0.70.0 and add the user-facing release-notes entry for the E4
Frontend Platform epic: full bookmarkable/shareable address-bar URLs (retiring the
"Share view" button), touch-visible card action buttons with a shape (not
colour-only) watched marker, and the bookmark-page-closes-on-modal-close fix — over
a large internal front-end refactor (god-component splits, usePref, useUrlMirror,
FilterRail, Modal focus). No DB migration.
2026-08-20 03:49:07 +02:00
peter 23ac884b42 fix(e4): address deep-review findings — bookmark popstate, focus trap, touch a11y
Deep-review (6 dimensions, 3-verifier adversarial pass) surfaced 7 findings, all
verified; fixed here.

- NavigationProvider + history.ts (medium, correctness): the popstate handlers
  restored the open channel / Plex sub-view from history.state ONLY, but a fresh
  ?chan / ?v bookmark lives in the URL with no marker in state — so any overlay's
  programmatic history.back() (closing a modal/player) silently closed the
  bookmarked channel or collapsed the sub-view to the grid. Give onPop the same
  URL fallback the initializers already use (history.ts extracts a shared readView
  so seed and popstate can't diverge again).
- plexCards SeasonCard (medium, a11y): the "In progress" season badge was a
  reveal-idle swap-badge → display:none on touch, and a season has no progress bar,
  so an in-progress season was indistinguishable from unwatched on touch. Moved it
  to the top-left (the toggle owns top-right) and made it always visible.
- plexCards watched toggle (low, a11y / WCAG 1.4.1): watched state was color-only
  on touch (green vs white CheckCircle2). New WatchedIcon distinguishes by SHAPE —
  CheckCircle2 (watched) vs an empty Circle (not) — used by all three card toggles.
- Modal.tsx (low, a11y): the focus trap leaked on Shift+Tab when focus rested on
  the panel container itself (Node.contains(self) is true, so the pull-back branch
  never ran). Guard now also checks `active === panel`.
- index.css (low, a11y): .reveal-idle faded only on .group:hover though .reveal is
  revealed on hover AND focus-within → the badge overlapped the control on keyboard
  focus. Added the focus-within fade.
- PlexBrowse.tsx (low, correctness): the ?v=playlist decoder accepted a non-integer
  id (Number.isFinite); tightened to Number.isInteger(n) && n > 0, matching the ?pl
  guard.

Gate green (tsc, knip, lint 0-err, 247 vitest). Self-E2E (localdev): ?v=show and
?chan bookmarks survive a modal open+close (were collapsing/closing before); Modal
Shift+Tab from the panel stays inside; in-progress season badge visible on touch;
watched toggle shows an empty-circle-vs-check shape difference; 0 console errors.
2026-08-20 03:26:19 +02:00
peter 1579a81395 fix(e4): address code-review findings — usePref flush, ?pl guard, dedup (E4 review)
Applies the /code-review high findings on the E4 branch:

- usePref: flush the pending debounced server write on unmount instead of
  clearing it, so the last change isn't lost (and can't later be overwritten by
  the stale server value via the adopt effect). [correctness]
- PlaylistsProvider.readSelectedId: accept ?pl only when it's a positive integer
  — "?pl=" gave Number("")===0 (finite) and "?pl=1.5" a fractional id, both of
  which selected a non-existent playlist. [correctness]
- usePref: extract mergePrefsCache() so useSavePref and usePref.set share the
  one me-cache merge shape instead of duplicating it. [reuse]
- New lib/useUrlMirror: the shared replaceState(urlWithParams(...)) mirror
  effect, so FeedFiltersProvider and PlaylistsProvider stop hand-rolling the
  identical block (keyed on the serialized delta so it only fires on change).
  [reuse]
- plexCards: swapBadgeReveal() helper for the repeated
  reveal-idle/group-hover:opacity-0 swap-badge ternary. [simplification]

Gate green (tsc, knip, lint 0-err, 247 vitest). Self-E2E (localdev): feed filter
mirrors to ?show and clears on leaving the feed / re-mirrors on return via
useUrlMirror; ?pl= (empty) and ?pl=abc no longer select a bogus playlist (fall
back to the stored selection); Plex reveal-idle badges still display:none on
touch after the helper refactor; 0 console errors.
2026-08-20 02:13:49 +02:00
peter 4fe6f38f1b refactor(rail): extract shared FilterRail shell + name the MediaSource seam (E4 S7)
The three filter rails — feed Sidebar, PlexSidebar, PlaylistsRail — each rendered
the same SidePanel chrome (title bar, customize `editing` state, edit-toggle,
reset-to-layout-default, edit hint) around a PanelGroups body, differing only in
identity + filter content. Extracted that shell into components/FilterRail.tsx:
each rail now supplies only what differs (title/icon, active-filter count +
Clear-all action, its `items`, and optional content above/below the groups —
`top` for the feed's scope toggle, `children` for its tag manager). FilterRail
owns the `editing` state, so all three drop their duplicated useState + SidePanel
wiring.

MediaSource facade (lib/mediaSource.ts): a thin `MediaSourceKind = "feed" | "plex"`
naming the two browsable, filterable sources (a PanelId minus "playlists", which
is a rail but not a source). Already unified in practice — one SortControl, the
source-agnostic ActiveFilter[] chips, PanelId layouts, now the shared shell — so
this is formalization, not new capability: a single named seam for a future third
source or a cross-source feature. The two media-source rails annotate their `kind`
with it; PlaylistsRail passes the plain PanelId. Per the recon, the playlist-dialog
merge was deliberately NOT forced (it proved forced + low value). UX-3 control-vocab
sweep deliberately deferred — the rails already use the canonical primitives.

Behaviour-preserving. Gate green (tsc, knip, lint 0-err, 247 vitest). Self-E2E
(localdev): all three rails render via FilterRail — feed (scope toggle + panel
groups + Customize→editing with reset + 56 reorder affordances, exits clean), Plex
(filter groups + customize), Playlists (sections + customize); 0 JS console errors.
2026-08-20 00:47:09 +02:00
peter 94a608053a feat(a11y): touch-visible card controls + modal focus trap/restore (E4 S6)
UX-7 — hover-only card actions were `opacity-0 group-hover:opacity-100`: fine on
a mouse, but invisible AND still tappable on a touch device (no hover). New
`.reveal` primitive (index.css) gates the hide on `@media (hover: hover)`, so a
touch device keeps the control visible; its partner `.reveal-idle` is the idle
badge a control replaces (watched/in-progress pill under the watched-toggle),
faded on hover and dropped on touch so the two don't collide at the same corner.
Migrated the true controls — feed card action toolbar (VideoCard), Plex poster
card watched-toggle/info/add across all card variants + their swap badges
(plexCards), message pop-out (Messages), edit-interval pencil (Scheduler), and
saved-view star/rename/delete (SavedViewsWidget). Pure play-scrim overlays (the
whole card is the tap target) stay hover-only by design.

FE-9 — Modal.tsx had aria-modal but no focus management. Adds: a Tab/Shift+Tab
focus trap (topmost layer only, wraps at the edges and pulls escaped focus back);
initial focus into the dialog unless an autoFocus child already placed it; focus
restore to the trigger on close (captured in a useState initializer during render,
before commit runs autoFocus and moves activeElement inside); and aria-labelledby
wiring the dialog to its title.

Gate green (tsc, knip, lint 0-err, 247 vitest). Self-E2E (localdev): FE-9 trap
both directions + pull-back + Escape-restore-to-trigger + aria-labelledby verified;
UX-7 feed toolbar opacity 0 on desktop / 1 under mobile (hover:none) emulation,
Plex reveal buttons opacity 1 + reveal-idle badges display:none on touch.
2026-08-19 21:35:52 +02:00
peter aea99cc71a refactor(downloads): split DownloadCenter god-file into row + modals + admin (E4 S5)
Pure code-motion, DownloadCenter 1088->359 L. Group by kind, mirroring the S4
plexCards/plexViews split:

- components/downloadRow.tsx (309 L): the user-list presentation + shared row
  primitives — StatusPill, Thumb, IconBtn, SourceRef, ChannelLine, DownloadRow,
  UsageBar (+ jobTitle/jobChannel/displayUrl helpers, status/phase constants).
- components/downloadModals.tsx (178 L): the per-item user modals — MetaEditModal
  (rename/metadata) and ErrorModal (failure detail + retry).
- components/downloadAdmin.tsx (267 L): the admin "System" tab — AdminSystem plus
  QuotaModal and QuotaUserPicker.

Every extracted piece is prop-driven with no shell-state reach-back, so the shell
just imports them back; the tab logic, mutations and modal state stay in
DownloadCenter. Behaviour-preserving.

Gate green (tsc first try, knip, lint 0-err, 247 vitest). Self-E2E (localdev):
Queue/Library/Shared/System tabs render, UsageBar, admin storage/footprint/job
list, MetaEditModal open/close, 0 console errors.
2026-08-19 21:14:49 +02:00
peter 180ab56379 refactor(plex): split PlexBrowse god-file into cards + views + shared UI (E4 S4)
Pure behaviour-preserving code-motion — PlexBrowse 1751 -> 538 lines. The ~11 inlined
components are self-contained (every view is invoked with shell callbacks as props; no
view reaches back into shell state), so they just move to sibling files.

New files:
- lib/plexWatch.ts: toggleWatchState (shared by the grid + person page)
- components/plexCards.tsx: PlexPosterCard, SeasonCard, CastStrip, RelatedStrip, EpisodeCard
- components/plexViews.tsx: PlexInfoView, PlexPersonView, PlexShowView, PlexSeasonView
- lib/plexDetailUi.tsx (extended): BackBtn, ActionBtn, WatchedToggle, DetailHero, epLabel
  (the file's own remit — "shared detail-page UI")

PlexBrowse keeps the Sub router + PLEX_SUB_URL + PAGE + PlexBrowse() shell. Repointed
the only external importer (PlexPersonView.dom.test) to plexCards/plexViews.

Self-E2E (localdev): Plex grid renders (poster cards + episodes); The Expanse show-drill
renders the hero + action buttons + season cards; ?v=show URL-mirroring intact; 0 console
errors. Gate green: typecheck, knip, eslint (0 err), prettier, 247 vitest.
2026-08-19 06:10:48 +02:00
peter 0b592345e6 fix(player): remove the leaked window "blur" listener in useNativeControlsYield
The native-controls-yield effect registered addEventListener("blur", onBlur) but its
cleanup removed "blur-sm" (a stray Tailwind-class find-replace artifact), so the blur
listener was never removed — leaking one per player open/close. Match the names.
2026-08-19 05:56:20 +02:00
peter 4800df4208 refactor(player): split PlayerModal god-component into hooks + controls (E4 S3)
Mirror the Plex player split (plexPlayerHooks.ts / plexPlayerControls.tsx). The shell
keeps owning the shared refs and passes them into each hook; the once-bound keyboard
handler + layer-stack registration stay in the shell (they orchestrate every cluster).
Behaviour-preserving code-motion — PlayerModal 1332 -> 890 lines.

New lib/playerModalHooks.ts:
- useYtVolume (per-account wheel/arrow volume, on shared playerVolume.ts)
- useNativeControlsYield (iframe-focus overlay yield)
- useYtBigView (F-maximise + our true-fullscreen; YT-specific, not Plex's useFullscreen)
- useYtProgress (player create + resume + 5s checkpoint + auto-watch + auto-advance)

New components/playerModalControls.tsx: PlayerTransportControls (moved as-is) + the
AutoMode/LoopMode types.

Self-E2E (localdev): play a feed video -> plays; transport bar (Auto/Loop); wheel volume
persists ({"volume":100}); F maximises + docked bar; exit big view; Close destroys the
player. Gate green: typecheck, knip, eslint (0 err), prettier, 247 vitest.

Noted a pre-existing bug (kept byte-identical here, flagged separately): the native-yield
effect removes a "blur-sm" listener but adds "blur" — the blur listener leaks.
2026-08-19 05:41:52 +02:00
peter 8bfc01684f feat(nav): full bookmarkable URL model — retire minimalist-URL + Share-view (E4 S2)
The query string is now the source of truth for the current view, so every URL is
bookmarkable/shareable. Reverses the early minimalist-URL rule (user's call).

- urlState: urlWithParams (read-modify-write delta on the live query, each provider
  owns its keys) + clearParams (strip only transient auth params) replace the whole-
  query stripUrlParams; shareUrl deleted; filterParamUpdates helper.
- NavigationProvider: page + ?chan mirrored to the address bar (pushState URL arg;
  history.state mechanics untouched); loadInitialPage resolves deep links
  (?v->plex, ?pl->playlists, filter/?chan->feed, bare->stored landing).
- FeedFiltersProvider: feed filters mirrored to the URL, gated on page===feed.
- useHistorySubview: optional SubviewUrl mapping; PlexBrowse mirrors the sub-view to
  ?v=<kind>:<args> (show/season/info/person/playlist; grid+player stay ephemeral).
- PlaylistsProvider: selected playlist mirrored to ?pl on the Playlists page.
- App: the 5 stripUrlParams calls become targeted clearParams (link/oauth/admin/
  access/verify/reset); the share-link hydrate effect removed.
- Retire the "Share view" feature: Sidebar + SavedViewsWidget buttons + i18n keys.

Browser tab title stays the short module/channel name (document.title untouched) —
the query never leaks into it, per the user's request.

Gate green: typecheck, knip, eslint (0 err), prettier, 247 vitest (urlState.dom.test
+9). No browser-E2E yet (most visual sprint — flagged for the epic-end review).
2026-08-19 04:47:36 +02:00
peter 48aa1a25f5 feat(prefs): usePref — one per-account+server preference hook (E4 S1)
Fold the hand-rolled server-pref plumbing (savePrefs + local mirror + me-cache
merge + adopt-on-login) into a single usePref<T>(serverKey, default, opts) hook,
plus useSavePref for dynamic multi-key patches.

Migrated (6 savePrefs call sites + 2 duplicated adopt effects -> the hook):
- FeedViewProvider: view
- App: filterCollapsed, playlistsCollapsed (+ their adopt block removed)
- PlayerModal: playerAutoAdvance, playerLoop
- plexDetailUi: plexInfoArtBg/Cast/Related (savePref via useSavePref)

Left as-is by design: panel layouts (a dynamically-indexed Record, not one key),
language (i18n store, no React state), PrefsProvider (debounced multi-field
auto-save + baseline echo-suppression - a different model).

Behaviour-equivalent (net -84 LOC at the call sites). usePref.dom.test covers
the three seed sources, persist path, debounce, adopt, parse, and useSavePref.
Frontend gate green: typecheck, knip, eslint (0 err), prettier, 238 vitest.
2026-08-19 03:07:25 +02:00
peter c82cf25bcf release: 0.69.2 (bug/download-poster-account-scope)
gate / frontend (push) Successful in 56s
gate / backend (push) Successful in 1m43s
2026-08-19 02:00:44 +02:00
peter 634e9bb17c chore(release): 0.69.2 — fix multi-account Library thumbnails 2026-08-19 02:00:24 +02:00
peter ce2954cf2b fix(downloads): carry the active account on the Library poster URL
An <img> can't send the X-Siftlode-Account header, so the poster.jpg request
resolved to the session-default account and _accessible_job 404'd it for any
download owned by a different (per-tab) account — every card then fell back to
the source's remote thumbnail, which for Facebook is an expiring fbcdn URL, so
the Library showed blank thumbnails in a multi-account session. The 0.69.1
local-poster-primary change made this the primary image, exposing it broadly.

Append ?account= to the poster URL via a new api.posterImageUrl(), the same
wallet-gated header-less selection downloadFileUrl / storyboardImageUrl already
use (resolved_user_id reads ?account= for exactly these contexts).
2026-08-19 02:00:24 +02:00
peter 2229b01a91 release: 0.69.1 (bug/feed-scroll-thumbnails-about)
gate / frontend (push) Successful in 56s
gate / backend (push) Failing after 38s
2026-08-19 01:42:54 +02:00
peter b17b47f54e chore(release): 0.69.1 — feed read-ahead, download thumbnails, About popover
Three prod bug fixes: local-poster-primary Library thumbnails, viewport-scaled
feed read-ahead (fixes the fast-scroll "Load more" stall), and the channel About
popover fitting/blank-line cleanup.
2026-08-19 01:42:35 +02:00
peter 1e2b3c5d29 docs(sync): scope normalize_description to channels + lock line-ending behavior
Review follow-up. The docstring said "channel/video description" but the helper is
wired only to the channel store sites; clarify it is deliberately channel-only
(video descriptions are user-facing prose where paragraph breaks carry meaning,
unlike a channel's metadata block) so it doesn't read as an asymmetric fix. Also
document that str.splitlines() folds CR/CRLF and Unicode line separators to "\n",
and lock it with a test.
2026-08-19 01:32:26 +02:00
peter d065902404 refactor(feed): cap the read-ahead lookahead and extract it to a tested helper
Review follow-up. The viewport-scaled lookahead was unbounded, so a tall window
prefetched several feed pages on every mount / filter change before any scroll —
wasted work on the app's heaviest query. Clamp it to MAX_LOOKAHEAD_ROWS, and move
the policy into a pure readAhead.ts (readAheadLookahead / shouldPrefetchNext) with
unit tests covering the floor, the scaling, the cap boundary, and the fetch trigger
— the core of the feed-scroll fix now has coverage instead of only manual E2E.
2026-08-19 01:32:26 +02:00
peter a300977ed4 fix(feed): read further ahead so fast scrolling doesn't stall at "Load more"
Keyset pagination is strictly serial (a page needs the previous page's cursor)
and the read-ahead only fired within a fixed 4 rows of the loaded end, pulling
one 60-item page per cycle. A fast free-spin flick (MX Master & co.) out-scrolls
that tiny lookahead, lands on an empty bottom, and waits out a chain of serial
round-trips — the multi-second "Load more" hang. Scale the lookahead to the
rendered row count (PREFETCH_VIEWPORTS screenfuls); since the effect re-runs when
a page settles it chains, keeping a couple of screens buffered ahead of the fold
so the content is usually already there. Bounded, and a no-op for finite lists.
2026-08-19 01:13:57 +02:00
peter ddb7818d38 fix(sync): strip blank lines from channel descriptions before storing
YouTube channel descriptions pad their contact/link blocks with runs of blank
lines, which the About popover renders verbatim (whitespace-pre-wrap) — wasted
space and storage, and noise in the search_vector. Add normalize_description()
and apply it at both channel store sites (full details + initial import) so the
DB holds the clean text. New channels are clean immediately; existing rows heal
on their next metadata re-sync. Unit-tested in test_utils.
2026-08-19 01:13:56 +02:00
peter 48787af11c fix(channels): keep the About popover on-screen at high browser zoom
At elevated zoom the CSS viewport shrinks; a long channel About wanted its fixed
max-h-112 (448px) height, so when floating-ui flipped it above the trigger its
top rendered above the viewport edge, clipped and unreachable (the inner
overflow-y-auto can't scroll past its own top). Enable the popover's fitHeight so
the `size` middleware caps the panel to the available space. Because the surface
splits into an outer positioned element and an inner styled/scroll wrapper, the
middleware now also publishes the available height as a --fit-max-h custom
property the inner scroller consumes via maxHeight: min(28rem, var(--fit-max-h)),
keeping the 28rem design cap on a tall viewport and shrinking to fit on a short one.
2026-08-19 01:13:40 +02:00
peter b733ee84fa fix(downloads): use the local poster as the Library thumbnail primary
The Library card used the source's remote thumbnail_url as the primary <img>
src and only fell back to our self-hosted poster on error. For Facebook
downloads (the bulk of a typical library) that URL is an fbcdn signed link that
expires within days, and is also blocked by ad-blockers / Brave Shields on the
facebook domain — so the thumbnail broke even though the local poster was
present on disk. Flip the priority: serve /api/downloads/{id}/poster.jpg first
(always present, never expires, same-origin) and keep the remote as the fallback
for the rare poster-less asset.
2026-08-19 01:13:40 +02:00
peter 96413c3d5e release: 0.69.0 (improvement/e3-s1-plex-split)
gate / frontend (push) Successful in 59s
gate / backend (push) Successful in 1m51s
2026-08-19 00:14:03 +02:00
peter d9677e0e3a chore(release): 0.69.0 — E3 backend architecture consolidation
Bump VERSION + user-facing release notes for the E3 epic (Plex god-module split,
downloads owner-unification, backend contract & conventions). Observable wins:
Plex catalog prune, download reliability (no false stalls, fair queue, no
file-delete-under-live-download). 20 commits, four clean /code-review high rounds.
2026-08-19 00:13:00 +02:00
peter 2b804300d5 fix(scheduler): address /code-review R3 findings on the maintenance handler (3)
Round-3 review of the S3 config-fold, all in routes/scheduler.py:
- stale docstring: "Persisted to app_state" -> "Persisted via sysconfig" (the
  batch moved stores in B-core-11; the comment didn't follow).
- drop the manual min/max check in the PATCH — sysconfig.set_value already
  validates against the same spec bounds; catch its ValueError -> 400, matching
  the config PATCH pattern (bounds enforced in one place).
- fetch sysconfig.spec(...) once into a local in the GET instead of twice.

No behaviour change beyond the 400 message wording (untested path). 368 pass.
2026-08-18 23:54:32 +02:00
peter 3f047bbad1 refactor(config): config/state mutators flush, callers commit (E3 S3, B-core-10)
Restores CLAUDE.md's canonical transaction rule ("the handler commits; helpers
flush; only quota owns its session") on the config path: sysconfig.set_value /
sysconfig.reset and state.set_sync_paused now flush instead of committing, so a
change lands in ONE transaction with the audit row the route records for it —
previously the helper's internal commit split them, and a crash between left the
mutation persisted with no audit trail.

All callers already commit (the config PATCH/DELETE + audit, the sync pause/resume
+ audit, the scheduler maintenance PATCH); the setup wizard, which relied on the
internal commit, gets an explicit db.commit() after its loop. apply_interval is
deliberately left committing — it performs a live APScheduler reschedule that must
follow the DB persist, so it owns its own unit of work (a legitimate exception,
like the download service transitions). New test locks the flush-not-commit
contract (a rollback before the caller commits discards the change). 368 pass.
2026-08-18 23:46:48 +02:00
peter 8a97c79d37 refactor(config): fold maintenance_revalidate_batch from app_state into sysconfig (E3 S3, B-core-11)
The admin-tunable maintenance re-validation batch lived in its own app_state
column with a hand-rolled override-or-default reader in app.state — a fourth
config store duplicating the sysconfig SPECS pattern. Move it to a sysconfig key
(`maintenance` group, min=100 max=100_000): the reader (sync/maintenance.py) and
the Scheduler dashboard (routes/scheduler.py PATCH + GET) now go through
sysconfig.get_int / set_value / spec, and app.state loses get/set_maintenance_batch
+ the MIN/MAX constants + the settings import. Migration 0070 carries any existing
override into the system_config store before dropping the app_state column
(reversible; restores + copies back on downgrade). Reuses the existing
sysconfig.spec() for the dashboard's min/max, so the bounds have one source.

Full lane 365 pass (migration drift + reversibility exercised), ruff clean.
2026-08-18 23:40:45 +02:00
peter d37b6ef07f refactor(routes): delete 5 dead endpoints the frontend never calls (E3 S3, B-core-8)
None are referenced by the SPA or tests; the Scheduler dashboard's "run now"
covers every manual trigger. Removed: POST /api/sync/{rss,backfill,enrich}
(+ the now-dead _user_channels helper and three sync.runner imports),
POST /api/tags/recompute (+ its admin_user / run_autotag_all imports), and
GET /auth/me (a lesser duplicate of the canonical /api/me in routes/me.py).

This also retires a latent bug: POST /api/sync/backfill reset backfill_done /
backfill_cursor on already-fully-backfilled channels (tipping them back to
"needs full history") for up to 25 channels per curl. Full lane 365 pass.
2026-08-18 23:32:49 +02:00
peter 88c5d5c5cb docs: refresh 7 stale module headers that describe shipped work as "later phase" (E3 S3, B-core-12)
This codebase leans on "why" comments as its architecture doc, so a header that
claims the opposite of the code below it is worse than none. Fix the seven the
review flagged: channels.py (subscribe/unsubscribe shipped), playlists.py +
sync/playlists.py (YouTube two-way sync + write-back shipped), setup.py (drop the
"epic 6c" ref — the wizard steps exist), config.py ("auth._client_ip" ->
"routes.auth._client_ip"), db.py ("routes/plex.py" -> "routes/plex/" after the
package split), sysconfig.py (the "Phase C" note — the watch-sync job that reads
plex_watch_sync_interval_min has landed; clarified it's a boot default, not a knob).
Docs only.
2026-08-18 23:28:25 +02:00
peter cff8bed62b fix(downloads): heartbeat only when no write touched the row this interval (E3 review R2)
Round-2 review of the R1 fixes found that the two-independent-limiter progress
writer issued a redundant heartbeat every interval during an ACTIVE download: a
percent change writes progress (which already refreshes updated_at, the stall
watchdog's liveness source), then the sub-integer calls that follow fire the
separate beat budget — ~2 DB writes/interval instead of the throttle's intended 1.

Replace the second limiter with a shared `last_touch` clock (last DB write OR
beat). Progress writes keep their own `allow_write` budget (so a beat still can't
delay a real write — the R1 concern), but the heartbeat now fires only when
nothing touched the row for a full interval. Active download → ~1 write/interval;
a genuinely stuck re-encode → 1 heartbeat/interval. New regression test:
an unchanged-percent call in the same interval as a write does NOT heartbeat.

ruff clean, full DB-lane 365 pass.
2026-08-18 23:16:48 +02:00
peter bc79720e81 fix(e3): address /code-review high findings (5)
Round-1 review fixes on the E3 branch:

- prune partial-fetch (correctness): `_paginate` now reports whether it reached
  the server's reported `total`; `_prune_stale` prunes only on a COMPLETE
  non-empty listing. A truncated fetch (a mid-sequence empty/short page before
  total — a transient Plex hiccup) no longer deletes the rows it merely failed
  to list. New test: a listing that claims 5 but returns 1 prunes nothing.
- `_prune_stale` (robustness): `stats.get("pruned", 0) + 1` — no KeyError when a
  caller's stats dict omits the key.
- service transitions (conventions): pause/resume/redownload use the JobStatus
  StrEnum instead of loose "queued"/"paused"/"error" literals, matching cancel.
- ffmpeg progress writer (correctness): a stuck-percent heartbeat now runs on its
  OWN 1/interval budget, so it can't consume the progress-write slot and delay a
  real percent change by up to an interval.
- backend/app/README.md (conventions): reword the transaction-ownership rule to
  quote and extend CLAUDE.md's canonical ("handler/job-wrapper commits; helpers
  flush; only quota owns its session") instead of stating a contradicting
  "helpers MAY commit" carve-out.

ruff clean, full DB-lane 364 pass.
2026-08-18 22:47:25 +02:00
peter b02a9e37e8 docs: write the backend conventions + a development guide (E3 S3, B-core-20)
The layering rule ("request-only logic in routes/; scheduler+worker-shared logic
in the plex|downloads|sync packages; cross-cutting primitives as flat core
modules; extract at the SECOND consumer") was real but unwritten — it lived only
in three module docstrings and a one-line README mention. Collect it, plus the
transaction-ownership rule (a mid-operation helper stages, the caller commits),
the atomic-counter / JobStatus-StrEnum conventions, and the one-home-per-config
rule, into backend/app/README.md. Add docs/development.md (run locally + the gate
commands + branching), so a new session/contributor has one place to look instead
of copying whichever pattern they happen to find first.
2026-08-18 22:02:18 +02:00
peter b2013a079a refactor(downloads): split worker.py parent/child into pipeline.py + jobdb.py (E3 S2, B-sub-9)
worker.py (~1090 L) mixed the PARENT supervisor (claim + supervise + the process
loop) with the CHILD pipeline (the yt-dlp/ffmpeg download+edit that runs in the
spawned subprocess). Extract them:

  worker.py  1090 -> 475   parent: claim/supervise/loop
  downloads/pipeline.py  (new 592)  the child pipeline (_download/_process_edit +
                                    hooks/progress/ffmpeg helpers + _Aborted)
  downloads/jobdb.py     (new  73)  the short-lived-session helpers both sides
                                    share (_set_job/_job_status/_touch_job/
                                    _requeue_if_running/_reset_asset_pending +
                                    the staging-dir paths)

The graph is one-directional (runner -> pipeline -> jobdb; worker -> jobdb; no
worker<->pipeline edge) — the parent never calls the child directly, it spawns
runner.py as a subprocess. Consumers rewired: runner import, and two tests follow
the moved symbols (_media_info/_produced_file -> pipeline; the ffmpeg-callback
patches target pipeline where the names are looked up).

Pure code motion, verbatim bodies. ruff clean, full lane 363 pass (unchanged).
2026-08-18 21:58:23 +02:00
peter 0fab0ce779 refactor(auth): typed Pydantic request bodies for the 9 payload:dict handlers (E3 S3, B-core-6)
request-access / demo / register / verify / verify-resend / password-login /
password-reset(request|confirm) / set-password each took an untyped `payload:
dict` and hand-picked fields with `.get(...)`. Replace with five small BaseModels
(EmailIn, EmailPasswordIn, TokenIn, TokenPasswordIn, SetPasswordIn) so FastAPI
validates the body shape and publishes the schema. Fields are `str | None` to
preserve exact behaviour — a missing/null field still coerces via the existing
`(... or "")`, no stricter 422s. Green under the auth-session suite (7), ruff clean.
2026-08-18 21:53:57 +02:00
peter 4ace486618 refactor(downloads): service owns the job state transitions (E3 S2, B-sub-8)
pause/resume/cancel/delete/redownload each hand-rolled their asset bookkeeping
inline in the route (release, failed-asset reset, the 20-line fresh-asset
rebuild), duplicating logic the worker/GC also touch. Move each transition into a
service.* method (service.pause/resume/cancel/delete/redownload) that owns the
mutation + its commit; the routes keep only auth, the HTTP 409 guards, and
serialization. Drops the now-dead _set_status route helper and two stale imports.

Pure behaviour-preserving code motion (routes -56 lines). Green under the E3 S2
transition net + the full downloads suite (363 pass), ruff clean.
2026-08-18 21:46:52 +02:00
peter 3334a8e895 test(downloads): E3 S2 characterization net — job state transitions (B-sub-8)
Pins the observable HTTP behaviour of pause/resume/cancel/delete/redownload
(status changes, asset release, failed-asset reset, fresh-asset rebuild) before
the inline route logic moves into service.* methods. Green on the current routes.
2026-08-18 21:43:57 +02:00
peter e2362e1ed1 fix(downloads): heartbeat a slow re-encode so the stall watchdog can't false-kill it (E3 S2, B-sub-4)
The supervisor kills a job that writes nothing for download_stall_timeout_seconds
(150s) and labels it a "network" stall. But the ffmpeg progress writer only wrote
on an integer-percent CHANGE, so a re-encode that sits on one percent for minutes
(AV1/VP9 -> H.264 or a crop on the CPU-only prod host) produced no row write and
was killed mid-encode as "stalled" — and the retry stalled the same way.

Fix: the writer now heartbeats (`_touch_job`, bumping updated_at without touching
the progress bar) once per interval even when the whole percent is unchanged, so
a live-but-quiet re-encode keeps a fresh updated_at. Reorders the rate-limit gate
first so a burst still costs one DB op — either a progress write or a heartbeat.

Pure-logic test with an injected clock: three stuck-at-10% intervals write no new
progress but heartbeat each time. Full lane green, ruff clean.
2026-08-18 21:34:36 +02:00
peter 7d89bc8b8e fix(downloads): exclude in-flight assets from the claim, drop the busy-wait (E3 S2, B-sub-3)
Two jobs on the same shared asset (same video+format queued twice, or by two
users) starved the worker pool: worker A downloads the asset; worker B claims the
sibling job, finds the asset `downloading`, requeues it, sleep(2)s, and re-claims
it on the next tick — burning a whole worker slot behind one in-flight asset for
the entire download (at concurrency=2, the queue stalls).

Fix: the claim SQL now excludes any job whose asset is already `downloading`
(`NOT EXISTS … media_assets.status='downloading'`), so a competing sibling simply
waits until the asset settles and the workers pick up unrelated jobs meanwhile.
The `_process_job` busy/lost-race branches drop their `time.sleep(2)` — the
exclusion keeps a requeued job out of the claim window, so there's no spin.

TDD: the new test failed on the old claim (claimed the sibling), passes now.
Full lane green, ruff clean.
2026-08-18 21:31:18 +02:00
peter f0a0214d97 fix(downloads): atomic ref_count increment/decrement (E3 S2, B-sub-5)
MediaAsset.ref_count (how many jobs hold a shared cache asset) was a Python
read-modify-write at four sites (enqueue, enqueue_edit, release_asset, the
redownload route). Two concurrent enqueues of the same (source,format) both read
the same value and write the same +1 → the count ends up one short → the first
release drives it to zero and deletes the file out from under a live holder.

Replace all four with atomic DB ops: incr_ref_count (`ref_count = ref_count + 1`)
and decr_ref_count (`greatest(ref_count - 1, 0)` RETURNING the new value, so the
release decides whether to free the asset from the post-decrement count while
holding the row's write lock — Postgres serialises concurrent releases on that
lock, closing the double-free). Pure DB-atomicity change, no semantic shift.

Green under the E3 S2 ref_count net (5) + full lane (356 pass), ruff clean.
2026-08-18 21:27:39 +02:00
peter e824fe0ae9 test(downloads): E3 S2 characterization net — ref_count enqueue/release semantics (B-sub-5)
Locks the observable single-threaded ref_count accounting before the atomic-SQL
conversion: two enqueues of one (source,format) share a single asset and stack
ref_count to 2; release keeps the asset until the last holder then frees the
ready row; a non-holding (canceled) job's release is a no-op; the decrement
clamps at 0; an errored asset is kept at zero (only ready is freed). Green on the
current RMW code (5 pass) — the fix must keep every assertion.
2026-08-18 21:24:46 +02:00
peter 49011940d6 feat(plex): catalog prune — delete titles that disappeared from Plex (E3 S1, B-sub-6)
A sync was upsert-only: a movie/show/season/episode removed from Plex stayed
browsable forever and 404'd on playback. Each section now diffs the rating_keys
it just listed against the stored ones and deletes the stale rows (watch-state /
playlist links cascade via the existing FKs), mirroring the collections prune
that already shipped.

Safety: the prune is GUARDED on a non-empty listing (_prune_stale returns early
when `seen` is empty), so a transient Plex glitch that yields an empty page can
never mass-delete a library — the metadata is cheap to re-sync, an accidental
wipe is the only real risk. Shows prune leaf-first (episodes -> seasons -> shows)
so a stale row is deleted directly before its parent's cascade reaches it.

Out of scope (separate follow-up): library-level reconcile of a whole removed
section + retiring the dead `PlexLibrary.enabled` column.

Tests: movie prune + watch-state cascade, the empty-listing guard (mutation-
verified — removing the guard wipes everything), show/season/episode prune.
351 DB-lane pass, ruff clean.
2026-08-18 21:13:18 +02:00
peter 52b6535dc6 refactor(plex): split browse.py god-module into per-concern routers (E3 S1)
routes/plex/browse.py was a 1693-line god-module (libraries + facets +
collections + playlists + show/item detail + people). Split it into focused
sibling routers under the same /api/plex prefix — pure code motion, no SQL /
route-path / response-shape changes:

  browse.py  1693 -> 469  (list_libraries, unified_library, facets)
  collections.py  (new 253)  browse + admin collection editor; owns _collection_source
  playlists.py    (new 405)  the user's orderable playlists
  detail.py       (new 438)  show/item detail + enrichment; _lib_key lives here
  people.py       (new 139)  cast/crew chips + person filmography

Cross-module helpers _norm_lang + _content_libs move to _shared.py (their one
home); detail imports _collection_source from collections (one-way, no cycle).
__init__.py includes the four new routers. Three DB-lane tests follow the moved
symbols (import path for _content_libs; monkeypatch targets for PlexClient /
_show_enrichment now on detail).

Behaviour-equivalent: ruff clean, 348 DB-lane pass (unchanged), parity net green.
2026-08-18 21:06:04 +02:00
peter 9bf7da2cbe refactor(plex): dedup movie/show watch-state predicates into query.py (E3 S1)
The `show` scope (all|watched|in_progress|unwatched) filter and the status CASE
were hand-duplicated between unified_library (the grid) and facets.wsq — the
exact drift risk the review flagged (a hidden movie kept out of the grid must
not inflate facets). Extract them into four shared query.py helpers
(movie_status_case, show_status_case, apply_movie_state_filter,
apply_show_state_filter), model-agnostic (callers pass the joined state
expressions), and point all three call sites at them.

Pure behaviour-equivalence refactor: browse.py -67 lines, no query changes.
Green under the E3 S1 parity net + query_conversion/facets/field-scoped/lang
tests (348 DB-lane pass, ruff clean).
2026-08-18 20:52:39 +02:00
peter 3301976375 test(plex): E3 S1 characterization net — grid⟷facets watch-state parity
Coverage-first safety net for the E3 S1 Plex god-module split. The movie/show
watch-state predicates are currently hand-duplicated in unified_library (the
grid) and facets.wsq (the facet aggregation); this pins the invariant the
facets docstring promises — with no metadata filter active, a facet's genre /
content-rating counts equal the same counts over the grid's visible items — for
every (scope × show) combination, deriving the expected counts from the grid's
own response. Locks the two branch-sets together before they merge into one
shared helper, so the merge is provably behaviour-equivalent.

Mutation-verified: dropping the facets hidden-exclusion branch makes both tests
fail (Drama 3≠2, R 2≠1). Green on the current code (348 DB-lane pass).
2026-08-18 20:48:43 +02:00
peter fa176a3e6a release: 0.68.0 (improvement/e2-s6-job-index-hygiene)
gate / frontend (push) Successful in 52s
gate / backend (push) Successful in 1m37s
2026-08-18 19:20:43 +02:00
peter e8771f4d6b chore(release): 0.68.0 — E2 Feed & Search performance epic
Bump VERSION 0.67.0 -> 0.68.0 and add the user-facing release-notes entry for
the E2 epic (feed sort index, keyset/NOT NULL, search GIN + state inner-joins,
single /feed/summary pass, no card-action refetch storm, job partial indexes).
2026-08-18 19:20:18 +02:00
peter 94dbe4b228 fix(jobs): match the boolean partial predicates to the IS TRUE/IS FALSE queries (E2 S6 review)
The via_search / shorts_probed partials used `WHERE via_search` / `WHERE NOT
shorts_probed`, but the queries filter with `.is_(True)`/`.is_(False)` (SQL
`IS TRUE`/`IS FALSE`), which Postgres's partial-index prover does NOT match to a
bare-boolean predicate — so both fell back to a full seq scan (the shorts-probe
job kept full-scanning for its work queue, the exact thing PERF-3 meant to fix).

Align the predicates to the query form: `WHERE via_search IS TRUE` and
`WHERE shorts_probed IS FALSE`. Verified on the prod-sized localdev: both now
Index Only Scan (via_search 44,683 -> 3 buffers; shorts_probed 44,683 -> 1).
Gate green (346 pytest).
2026-08-18 18:31:38 +02:00
peter 0ed4ac6958 perf(jobs): partial indexes for the enrich/maintenance scanners + boolean cleanup (E2 S6, PERF-17/18/3)
Two scheduler jobs full-seq-scanned the 257k-row videos table every run:
- PERF-17 enrich_pending (`WHERE enriched_at IS NULL LIMIT N`, every few minutes):
  44,680 buffers even with 0 pending -> a partial ix_videos_enrich_pending makes
  it a 1-buffer index scan (empty when nothing pending).
- PERF-18 maintenance revalidate (`WHERE unavailable_since IS NULL ORDER BY
  last_checked_at NULLS FIRST LIMIT N`, ~300 pages/run): 44,754 buffers + top-N
  sort -> a partial ix_videos_recheck_order in that exact order = a 42-buffer
  index scan. The order_by is rewritten to `asc().nulls_first()` (equivalent to
  the old `(IS NULL) DESC, ASC`) so it rides the index; the unused plain
  ix_videos_last_checked_at is dropped.

PERF-3: drop the never-picked boolean btrees (is_short, via_explore) and slim the
useful ones to partials over the queried subset (via_search=true, NOT
shorts_probed, unavailable_since IS NOT NULL) — less write amplification on the
biggest table, same lookups. Model: those 6 columns lose their inline index=True
(the partials are migration-defined, like the search_vector GIN); live_status
left alone.

Measured on the prod-sized localdev reseed: enrich 44,680 -> 1 buffer (25.9 ->
0.02 ms), maintenance 44,754 -> 42 buffers (35.5 -> 0.17 ms); all 11 CONCURRENTLY
index ops on 257k rows in 0.75s. Gate green (346 pytest incl. drift + downgrade
round-trip).
2026-08-18 17:01:13 +02:00
peter 8b8b0ca5ab perf(feed): skip the empty IN() query in tags_by_category (E2 review round 2)
The tags_by_category extraction dropped feed_summary's `if selected:` guard, so
a /feed/summary request with no tag filter (the common case) ran a pointless
`SELECT ... WHERE id IN ()` round-trip. Early-return {} for empty tag_ids —
restores the no-query hot path and covers filtered_query + future callers too.
Gate green (346 pytest).
2026-08-18 16:49:29 +02:00
peter 6116ad4bf4 refactor(feed): address /code-review high findings on the E2 stack
Review round on the E2 stack (S1-S5). No correctness bugs; the surviving
low/medium findings addressed:

- altitude (#1): extract the shared `tags_by_category` helper used by BOTH
  filtered_query (SQL tag WHERE) and feed_summary (Python facet/count), removing
  the category-grouping duplication, and add bidirectional "keep in lockstep —
  equivalence-tested by test_feed_summary_matches_count_plus_facets" comments so
  the SQL<->Python tag rule can't silently drift.
- test-coverage (#2): the /feed/summary equivalence test seeded only system tags,
  leaving the `ChannelTag.user_id == user.id` visibility branch untested. Add a
  user-owned Tag + link + two combos; the suite is now 346 (was 344) and the new
  cases pass (feed_summary agrees with /feed/count + /facets on user-visible tags).
- dead-code (#3): annotate get_feed_count, get_facets, and the FE api.feedCount/
  facets as "SUPERSEDED by /feed/summary — kept only as the equivalence oracle;
  retire together" so the intentionally-retained surface reads as deliberate.

Not changed (confirmed no-op): the search channel-name resolve (#4) is
once-per-request on a 633-row table and a deliberate pre-resolve for PERF-12's
BitmapOr plan; the facet loop (#5) already computes passes() once per
(channel, category) with a distinct exclude each, so there is no recomputation.

Gate green: ruff + 346 pytest; frontend tsc/knip/prettier (FE change is a comment).
2026-08-18 16:39:04 +02:00
peter 943143ef45 perf(feed): stop the card-action + player-close feed refetch storm (E2 S5, PERF-2)
Marking a card watched/hidden/reset and closing the player each fired a full
`invalidateQueries(qk.feed())`, refetching EVERY loaded infinite-query page — 10
pages x a few quick marks = ~50 feed refetches, each a full scan (x PERF-1). The
optimistic `overrides` layer + the matchesView filter already render the correct
state (a hidden/watched card drops out instantly), so the refetch was pure waste.

Switch these paths to `refetchType: "none"`: the queries are marked stale but not
refetched now; the next filter change / remount pulls fresh server data. Applies
to the two Feed card-action paths (setState/clearState) and the PlayerModal
close (feed + playlists — progress is already persisted server-side, only the
on-screen resume bar waits for the next natural refetch).

Left as immediate refetch on purpose: channel/tag/playlist mutations (one-off, on
pages where the main feed is unmounted anyway) and the onboarding / notification-
undo paths (which must reveal the changed content right away). Gate green:
frontend tsc/eslint/knip/prettier/222 vitest (backend unchanged, 344 pytest).
2026-08-18 15:49:49 +02:00
peter 21a3aae700 perf(feed): collapse count + facets into one /feed/summary pass (E2 S4, PERF-13)
The FE fired /feed/count (1 pass) + /facets (3 passes) on every filter change =
4 full scans of the subscribed set. New /api/feed/summary does ONE grouped query
(the base filter with tags stripped, GROUP BY channel_id -> per-channel video
counts) and derives the total count + every tag's distinct-channel facet count in
Python on the ~few-hundred-row channel map, faithfully mirroring filtered_query's
channel-based tag logic + the /facets drill-down (own category dropped, unless a
conjunctive topic-AND). Returns {count, facets} — the same numbers the two old
endpoints returned.

FE: a single shared query (Feed reads .count, Sidebar reads .facets); same key +
filters, so React Query dedupes to ONE request. Invalidations repointed to
qk.feedSummary. The old /feed/count + /facets endpoints (and their now-unused FE
api/qk) are kept as the equivalence-test oracle + safety net; a later cleanup can
drop them.

New equivalence test (test_feed_views, 8 filter combos incl. cross-category,
disjunctive/conjunctive topic modes, search): /feed/summary count == /feed/count
and facets == /facets, byte-identical. Structural win: 2 requests / 4-6 queries
-> 1 request / 2-3 queries (the review measured 530 -> 144 ms). Gate green:
344 pytest, frontend tsc/eslint/knip/prettier/222 vitest.
2026-08-18 05:01:27 +02:00
peter 6a35e83ded perf(feed): un-block the search GIN + inner-join the state views (E2 S3, PERF-12/16)
PERF-12: the search ORed `unaccent(channels.title) ILIKE '%q%'` straight into
the per-video predicate, which forced a full scan + detoast of every
search_vector and a per-candidate channel fetch. Pre-resolve the matching
channel ids (a few-hundred-row scan) and match `Video.channel_id.in_(ids)`
instead, so the planner keeps the search_vector GIN as a BitmapOr branch
(alongside ix_videos_channel_pub_desc). Empty match -> false() collapses the OR.

PERF-16: show=watched|hidden|in_progress used an outer join + coalesce(status,
'new'), so the planner couldn't start from the user's small video_states set.
Those views only return rows that HAVE a state, so inner-join video_states and
filter the raw column (status is NOT NULL on a present row, so it's identical).
'unwatched'/'all' keep the outer join. Dropped the now-dead position_expr.

New regression tests (test_feed_views): search returns both text- and channel-
name matches; the four watch-state views return the exact expected sets.

Measured on the prod-sized localdev reseed (prod planner, 0067/0068 applied):
- search /feed/count q=magyar: 663,463 -> 19,206 buffers (1,406 -> 31 ms; OLD
  seq-scan+detoast -> NEW BitmapOr GIN+channel index)
- search feed page: 2,908 -> 1,479 buffers (32.6 -> 1.4 ms)
- show=watched (user with 64 watched): 40,661 -> 264 buffers (~154x)
Gate green (336 pytest, incl. the 2 new equivalence tests).
2026-08-18 04:41:16 +02:00
peter 8c08ea3133 perf(feed): published_at NOT NULL + range-anchored keyset for deep pages (E2 S2, PERF-15)
A nullable published_at forced the feed keyset into a non-sargable
`... OR published_at IS NULL` disjunction: with the 0067 index the planner still
applied it as a FILTER and scanned from the top of the index, so a deep `newest`
page discarded every earlier row (O(depth)).

- Migration 0068: backfill any NULLs (0 in prod) -> SET NOT NULL + DEFAULT now().
- Model: Video.published_at is now non-Optional, nullable=False, server_default now().
- feedquery: the `pub` key is nullable=False, and keyset_predicate is rebuilt
  recursively so a non-nullable leading key gets a non-strict range bound
  (`published_at <= v`) as an AND conjunct — an index range Cond instead of a
  filter. Nullable keys keep the NULL-aware OR form (a `<= v` bound would drop
  their NULL group). Equivalence proven by test_feed_keyset (all 13 sorts,
  cursor-walk == unpaged).
- Ingest paths (stub_from_playlist_item, apply_video_details, playlists sync)
  fall back to now() so they never write an explicit NULL.

Measured on the prod-sized localdev reseed at a ~50k-deep cursor (0067 index +
prod planner): OLD `Rows Removed by Filter: 50001`, 25,687 buffers / 9.16 ms ->
NEW `Index Cond: published_at <= v`, 1 row filtered, 47 buffers / 0.037 ms. SET
NOT NULL on 257k rows: 0.32s. Gate green (334 pytest).
2026-08-18 04:30:07 +02:00
peter 3af3799b25 perf(feed): index the default published_at DESC NULLS LAST sort (E2 S1, PERF-1)
The feed's default `newest` sort is `published_at DESC NULLS LAST, id ASC`, but
the only published_at index was a plain ASC btree — read backwards it satisfies
DESC NULLS FIRST, not NULLS LAST — so every feed page fell back to a full
`videos` seq scan + top-N sort. Migration 0067 adds the two matching indexes
(global + per-channel) CONCURRENTLY and drops the now-redundant single-column
ix_videos_channel_id (the channel composite leads with channel_id, a strict
superset).

Measured on the prod-sized localdev reseed (257,312 videos, prod planner):
- global newest:  44,747 buffers / 154 ms  ->  60 buffers / 0.32 ms
- per-channel(12.6k): 7,604 buffers / 96 ms -> 54 buffers / 0.14 ms
CONCURRENTLY build of both on 257k rows: 1.7s total (~0 downtime). This is the
likely root cause of the "prod feed is slow" symptom.

Model: drop the inline index=True on Video.channel_id (the composite supersedes
it); the composites are migration-defined like the search_vector GIN. Gate green
(334 pytest incl. the migration drift + downgrade/re-upgrade round-trip).
2026-08-18 04:16:08 +02:00
peter aff5a9922c release: 0.67.0 (security/e1-s5-auth-abuse)
gate / frontend (push) Successful in 51s
gate / backend (push) Successful in 1m26s
2026-08-18 03:43:03 +02:00
peter cace9a004d chore(release): 0.67.0 — E1 Foundation epic
Bump VERSION 0.66.0 -> 0.67.0 and add the user-facing release-notes entry for
the E1 Foundation epic (data-safety fixes, account-switch + search-error fixes,
graceful key-mismatch degrade; plus the security/test-net/CI/repro chores).
2026-08-18 03:41:28 +02:00
peter bfc104a9b4 test(migrations): only restore schema in finally when the re-upgrade failed
Review round 2: the unconditional `command.upgrade(cfg, "head")` in finally ran
a redundant no-op on the success path and, on a genuinely broken re-upgrade,
replaced the try's original exception with its own. Gate the restore on a
`reupgraded` flag so it fires only when the upgrade didn't complete — the
cascade-prevention intent is kept without the double-run or error-masking.
2026-08-18 03:40:11 +02:00
peter 0340d0981c fix(e1): address /code-review high findings on the E1 stack
Review round on the whole E1 stack (main..HEAD). 6 findings, all low/medium,
none blocking; fixed for a clean gate-green handoff.

- plex/sync (correctness): the un-enriched cast guard compared the stored cast
  against THIS listing's length, so a trimmed Plex cast on a not-yet-enriched
  row was never applied. Compare against the listing CAP (_LISTING_CAST_CAP=20)
  instead: >cap = enriched (keep, survives a watermark reset), <=cap =
  listing-sourced (track edits in both directions). +2 regression tests.
- main (security): add X-Frame-Options: DENY as the legacy-browser fallback for
  the CSP frame-ancestors clickjacking guard.
- downloads/service (altitude): finalize_if_running now uses a SQLAlchemy 2.0
  update() construct instead of a hand-built text() UPDATE string — same atomic
  WHERE status='running', idiomatic, no identifier interpolation, no manual
  StrEnum coercion. Drops the now-unused text import.
- auth/admin (reuse): extract the SEC-10 "drop untrusted preset password + bump
  epoch" invariant into one helper (revoke_preset_password_if_unverified) so the
  Google-callback and admin-activation paths can't drift.
- tests/migrations (robustness): guarantee the shared schema is restored to head
  in a finally, so a downgrade/re-upgrade failure can't cascade into later tests.
- worker (simplification): align _finish_from_cache's display_name fill with the
  _download path's guard (no behavior change).

Gate green: ruff clean, 334 pytest (0 skip), frontend unchanged.
2026-08-18 03:30:23 +02:00
peter 18edbecb1e fix(auth): close the account pre-emption + wrong-account link holes (E1 S5, SEC-10/11)
SEC-10 (account pre-emption / "pre-hijack"): a password set on an account whose
email was never verified is untrusted — whoever registered the address (to
pre-empt the real owner) could have chosen it. Both the Google-adoption callback
and admin activation (_activate_user_by_email) now drop that pre-set password and
bump the session epoch when the account isn't email_verified, so a later Google
login / admin approval can't hand the pre-set password a working login. No-op for
a verified account (the legit no-SMTP flow). +2 tests.

SEC-11 (link/upgrade bound to the session default): the OAuth link/upgrade GETs
resolved to the session DEFAULT account, so a tab acting as a secondary account
would link Google / the YouTube grant to the wrong account. The frontend now
threads the active account through the navigation URL (withActiveAccount ->
?account=<active>), and _complete_link accepts any wallet account, not only the
default, so the identity lands on the account that started the flow.
2026-08-18 03:02:34 +02:00
peter 8ffa65a9da fix(security): bump cryptography 46.0.7 -> 50.0.0 to clear 4 CVEs
gate / frontend (push) Successful in 54s
gate / backend (push) Successful in 1m36s
The pip-audit step added in E1 S4 flagged cryptography 46.0.7 for PYSEC-2026-3552
/3553/3554 and GHSA-537c-gmf6-5ccf; all are fixed by 50.0.0. The requirements.txt
ceiling `<47` was blocking the upgrade — relaxed to `>=50.0.0,<51`. The lock is a
surgical bump (only the cryptography line changed; cffi 2.1.1 already satisfies
it, no other package moved). pip-audit -r requirements.lock is now clean.

Add tests/test_security.py (there was none for the crypto primitives): the
Fernet encrypt/decrypt round-trip, decrypt_optional degrading on garbage and on a
key mismatch (the B-core-2 path), and the argon2 password hash/verify round-trip
— so a future crypto-lib bump can't silently break the token-at-rest path. Gate
green: 330 pytest.
2026-08-18 02:47:05 +02:00
peter f54dd98129 test(release): lock the VERSION file and RELEASE_NOTES[0] together (E1 S4, DX-6)
gate / frontend (push) Successful in 1m9s
gate / backend (push) Successful in 1m27s
A vitest that fails the gate when the newest release-notes entry's version
diverges from the repo-root VERSION file (the two hand-edited version sources),
plus a no-duplicate-versions check. Catches the "bumped one, forgot the other"
release mistake.
2026-08-18 02:34:45 +02:00
peter 44d4e4876d build(deps): pin a reproducible lock + pip-audit; bump node 20->22 (E1 S4, SEC-5/DX-3)
requirements.txt stays the source (floor ranges); requirements.lock pins the
exact resolved versions (generated in the python:3.12-slim prod base, extras like
psycopg-binary/uvloop included). The prod Dockerfile, Dockerfile.test and CI now
install the lock, so the image is reproducible and the versions are the ones
pip-audit scans. CI gains a non-blocking pip-audit step over the lock. The
frontend build + CI move from the EOL node 20 to node 22.
2026-08-18 02:34:45 +02:00
peter e4ac061721 feat(security): add baseline security response headers (E1 S4, SEC-1)
A small, safe set applied to every response via middleware: X-Content-Type-Options
nosniff, Referrer-Policy strict-origin-when-cross-origin, CSP frame-ancestors
'none' (anti-clickjacking; unrelated to the YouTube/Plex embeds we host), and
Permissions-Policy disabling camera/mic/geolocation. A full CSP and HSTS are
deferred (HSTS belongs on the Caddy edge).
2026-08-18 02:34:45 +02:00
peter 54ba92bb05 ci(gitea): add the server-side gate mirror (E1 S3, DX-1)
gate / frontend (push) Successful in 51s
gate / backend (push) Successful in 1m13s
Add .gitea/workflows/check.yml so the quality gate runs on the lab act_runner,
machine-independent, on every push/PR — the review's "no server-side CI" gap.
Two parallel jobs mirror the two local lanes:
- frontend (node:20-bookworm, matching the prod build image): tsc + eslint +
  prettier --check + knip + vitest.
- backend (nikolaik/python-nodejs python3.12+node): ruff + the full pytest suite
  against a real postgres:16 service, with ffmpeg installed and
  SIFTLODE_TEST_REQUIRE_DB=1 so a DB/ffmpeg that failed to come up is a hard
  error, never a silent skip.

Notes for this runner: pytest runs natively against the service (the
docker-compose bind-mount is unreliable under a mounted-socket runner), and the
backend uses a baked python+node image rather than actions/setup-python, which
can't reach GitHub's python-versions download from the lab.
2026-08-18 01:59:36 +02:00
peter 472cefed55 test(migrations)+fix: widen drift check to column shape; notifications.created_at NOT NULL (S2 review #2)
The drift check dropped every diff except table/column presence, so a column
type or nullability change would slip through. Add modify_type/modify_nullable
to the ops it fails on. That immediately surfaced a real pre-existing drift:
notifications.created_at was created NULLABLE by migration 0017 but the model
has always declared it NOT NULL (Mapped[datetime] + server_default=func.now()).
Migration 0066 enforces NOT NULL to match the model — safe, since the
server_default has filled every row since day one.
2026-08-18 01:39:46 +02:00
peter e764ca4568 test(fixtures): make the TestClient fixtures isolated and composable (S2 review)
Address the /code-review findings on the S2 test-net:
- as_user no longer sets a global `lambda: user` override that two users in one
  test would collide on; the shared current_user override now resolves the caller
  from an x-test-user header, and each returned client tags its own requests, so
  as_user(a)/as_user(b) are independent (#1).
- override_dep records the prior value per dependency key and restores it on
  teardown instead of a blanket dependency_overrides.clear(), so fixtures compose
  without clobbering each other (#5).
- a _configured fixture bakes the post-setup state into client/as_user, and the
  per-test inline monkeypatch / autouse duplicates are removed (#4).
- the feed keyset walk asserts all seeded rows surfaced before comparing, so a
  seeding regression can't make the sorts pass vacuously (#3).
2026-08-18 01:39:45 +02:00
peter 296af2a498 test(auth): session/epoch/wallet/token security core (E1 S2, TEST-1)
The auth security mag had zero tests. Add a DB-lane suite over the real
auth/session stack (TestClient carries the signed cookie): password login ->
/api/me authenticated; a wrong password and an unknown email return a
byte-identical 401 (no enumeration oracle); an X-Siftlode-Account header naming
a non-wallet account falls back to the default (impersonation gate); logout-others
bumps the epoch so a second session is rejected (401 on a protected endpoint)
while the caller stays in; and password-reset/confirm is single-use (replay ->
400) and bumps the epoch so pre-reset sessions die.
2026-08-18 01:23:07 +02:00
peter 66c00327b1 test(feed): keyset paging reproduces the unpaged order for every sort (E1 S2, TEST-2)
The feed's cursor paging (sort_keys/order_by/keyset_predicate/encode/decode) is
the NULLS-LAST + tiebreaker algebra where a page-boundary bug hides as a
duplicated or skipped row on the busiest endpoint, and had zero tests. Seed 7
videos on 2 channels with duplicate sort values and NULLs across
published_at/view_count/duration/title/subscriber_count, then assert — for every
sort — that walking the feed in pages of 2 via the cursor reproduces exactly the
order a single unpaged query returns, with no duplicates.
2026-08-18 01:23:07 +02:00
peter 31879b01f4 test(migrations): model<->schema drift + downgrade/re-upgrade (E1 S2, TEST-3)
Prod only ever runs `alembic upgrade head`; nothing checks that the migrated
schema still matches Base.metadata, and the downgrade path never runs. Add a
DB-lane test that (a) asserts no structural (table/column) drift between the ORM
models and the head schema — filtering the index/constraint/sequence noise that
compare_metadata reports for migration-only objects — and (b) exercises the head
migration's downgrade + re-upgrade so a broken downgrade is caught the sprint it
lands.
2026-08-18 01:23:07 +02:00
peter 15a59f4695 test(infra): fail-loud DB/ffmpeg lane + client/as_user/make_user fixtures (E1 S2, TEST-4)
The DB-backed lane and the two ffmpeg tests skipped SILENTLY when the DB was
unreachable or ffmpeg was absent, so `siftlode check` could print green while
~60 tests never ran. Make the compose lane authoritative:
- SIFTLODE_TEST_REQUIRE_DB=1 in docker-compose.test.yml; conftest.pytest_configure
  turns a would-be skip (no Postgres, or no ffmpeg) into a hard UsageError.
- install ffmpeg in Dockerfile.test so the thumbnail/poster tests actually run.
- `-rs` in pytest.ini so skip reasons are always visible.

Add the shared TestClient fixtures the risk-core suites build on: `make_user`
(create + commit a login-ready User), `client` (real auth/session stack, no
current_user override — for login/wallet/epoch), and `as_user` (a client
pre-authenticated as a given user, for tests that aren't exercising login).
2026-08-18 01:23:06 +02:00
peter 0653203d81 fix(plex): make the cast gate self-correcting as well as non-shrinking (B-sub-2 review)
The union form fixed the downgrade but removed the per-sync self-healing: a name
Plex removed from an unenriched title lingered in cast_names/people_text until
enrichment. Replace the union with a length guard — take the fresh listing
unless it is shorter than what is stored. A stored cast longer than the ~20-cap
listing can only be an enriched fuller list (kept, so a watermark reset never
downgrades it); at or below the cap the stored value is itself listing-sourced,
so replacing it with the current listing stays self-correcting.
2026-08-18 00:58:13 +02:00
peter 4ce0a702f4 refactor(downloads): route worker terminal writes through an atomic finalize_if_running (B-sub-1 review)
Follow-up to the /code-review high on the S1 branch. The per-site
`if job.status == 'running'` guards were a non-atomic read-then-write (TOCTOU):
a cancel committing between the child's status read and its commit could still
be overwritten with done/error, re-opening the double-release window.

Introduce service.finalize_if_running(db, job_id, **fields): one atomic
`UPDATE download_jobs SET ... WHERE status='running'` that returns whether it
fired. It flushes pending asset writes first so the asset-then-job row-lock
order matches the cancel route (no deadlock), and runs in the caller's session
so it commits with the asset writes in one transaction. All five worker sites
(download/edit completion, both error paths, cache finish) now go through it,
so the invariant lives in one place. The 'job done' logs move inside the
finalized branch so a job stopped mid-completion no longer logs a false 'done'.
Add tests/test_downloads_finalize.py: running -> finalizes; canceled/paused ->
no-op.
2026-08-18 00:58:13 +02:00
peter 9845c8d0f7 fix(auth): epoch-check the target on account switch to avoid a full wallet logout (B-core-9)
POST /api/me/switch validated wallet membership, access and suspension but not
the target account's session epoch. Switching to an account revoked elsewhere (a
password reset/change or "log out everywhere" bumped its epoch) made it the
session default, so the next request's current_user hit the epoch mismatch on
the default identity and cleared the WHOLE wallet session — logging out every
account in the browser. Mirror the suspend guard: drop the stale account from
the switch list and return 403 instead of switching.
2026-08-18 00:22:15 +02:00
peter 7c431d73fb fix(config): degrade gracefully on a TOKEN_ENCRYPTION_KEY mismatch instead of a 500 (B-core-2)
A rotated or missing TOKEN_ENCRYPTION_KEY makes every stored secret undecryptable.
sysconfig._raw_override called security.decrypt() with no guard, and it sits on
hot paths (/api/me via google_enabled, login, every YouTube scheduler job,
account purge) — so a key mismatch raised InvalidToken and 500'd the whole app,
the SPA wouldn't even boot, with no documented recovery.

Catch the decrypt failure in _raw_override -> warn once per key and fall back to
the env/config default. The YouTube client's refresh-token decrypt now raises a
clear YouTubeError ("reconnect") that every caller already handles, and
purge_user uses decrypt_optional so a best-effort revoke can't 500 the delete.
Document that the key must never change once secrets are stored (recovery: clear
the affected system_config rows and re-enter).
2026-08-18 00:22:15 +02:00
peter 67d32fecd6 fix(search): return 422 not 502 on an upstream YouTube failure (B-core-1)
/api/search/youtube raised 502 on an upstream error, but the frontend treats
502/503/504 as a gateway dropout: it auto-retries the GET (a second quota-costed
search) and shows a "connection lost" banner instead of the real message. Every
sibling upstream-error site already returns 422 for exactly this reason. Reserve
5xx for the raw byte-proxies.
2026-08-18 00:21:59 +02:00
peter f39c5bb25c fix(plex): never shrink cast_names on sync so a watermark reset can't regress it (B-sub-2)
_apply_facet_fields seeded cast_names from the cheap top-~20 listing only while
people_enriched_at was NULL, relying on "a sync never clears the watermark" to
protect an enriched fuller cast. Migration 0065 broke that assumption by nulling
people_enriched_at on every movie/show row, so the first post-deploy sync
downgraded enriched casts back to the listing top-20 — silently dropping
person-page cast-filter names for weeks until the backfill caught up (the S5b
regression, re-opened by the migration).

Guard structurally instead of trusting the watermark: union the listing cast
onto whatever is stored, never dropping a name already present. A fresh row
still seeds from the listing; a row whose watermark was reset but whose fuller
cast survives keeps it. Robust to any future watermark reset, not just 0065.
2026-08-18 00:21:59 +02:00
peter 367bcebc9f fix(downloads): revive the worker cancel branch and guard terminal writes (B-sub-1)
The cancel route writes status 'canceled' but the supervisor compared against
'cancelled', so its "a route stopped it -> kill the child, free the asset"
branch was dead code. A canceled job's child ran to completion and wrote 'done'
unconditionally over the route's 'canceled'; since 'done' holds the asset but
'canceled' does not, a later delete released the asset a second time, dropping
ref_count below the live holder count and deleting a file still in use.

Add app/downloads/status.py as the single source of truth for the lifecycle
literals (JobStatus StrEnum + the HOLDING / STOPPED_BY_ROUTE sets) and use it at
the safety-critical sites: the supervisor stop check, the route cancel, and
release_asset's holding set. Make the worker's terminal done/error writes
conditional on the row still being 'running' so a child finishing just after a
route cancel/pause can no longer resurrect it. Add the missing 'canceled'
supervisor regression test.

(The full literal sweep across the downloads surface is left for the E3
downloads owner-unification; this is the data-safety subset.)
2026-08-18 00:21:43 +02:00
peter 429fbde866 docs: add CLAUDE.md working agreement for AI agents
The codebase is fully AI-authored and developed across many agent sessions; the
canonical conventions have lived only in a private memory store, which the
2026-08-17 review flagged as a real gap (unwritten rules → asymmetric fixes and
comment-vs-code drift, e.g. the cancelled/canceled literal bug).

Add a short, auto-loaded rules-of-the-road: the local gate (never merge/ship
red, no --skip), the trunk workflow and no-AI-co-author commit rule, the
one-way architecture shape, a canonical-pattern table (the review's §6
consistency choices), and the house rules learned from real bugs. Kept
deliberately thin; slims to a pointer once docs/development.md lands.
2026-08-17 23:33:01 +02:00
peter 4c013595b5 release: 0.66.0 (e5-s3-toolbar-relocation) 2026-08-17 22:19:23 +02:00
peter b605c28b32 release: 0.66.0 — E5 Plex epic (scoped search, HU localization, toolbar/header)
Ships the E5 sprint stack (S1–S7): field-scoped Plex search + typeahead, cast &
crew pages, the HU-overlay Hungarian metadata + language-aware serving, the
IMDb/audience rating split, the toolbar relocation, and the narrow/mobile header
fix — plus the underlying Plex-module refactors (feedquery/plex.cards/plex.query
extraction, db.query→2.0 select).

- VERSION 0.65.2 → 0.66.0
- releaseNotes.ts: 0.66.0 user-facing entry
- prettier-format the E5-touched frontend files (release hygiene)

Merged main (0.65.2 download-gc hotfix) in first; no code conflicts (E5 is
Plex-only, the hotfix was notifications/downloads).
2026-08-17 22:18:07 +02:00
peter cee0bc95b6 Merge remote-tracking branch 'gitea/main' into e5-s3-toolbar-relocation 2026-08-17 22:14:50 +02:00
peter 5c6172754a refactor(header): move --hdr-h measurement into PageShell (S7 review)
Addresses the S7 mobile-fix code-review findings:

- (altitude) The header-height→content-pad contract lived across two files with
  an implicit DOM coupling: Header measured itself and wrote --hdr-h onto
  `el.parentElement`, assuming PageShell rendered it as a direct child of the
  padded column. Move the whole contract into PageShell, which owns BOTH the
  header slot and the padded content wrapper: Header is now forwardRef and
  PageShell attaches a ref (via cloneElement), measures the header with a
  ResizeObserver, and writes --hdr-h straight onto the content wrapper that
  consumes it — no parent assumption, producer and consumer together.
- (simplification) The `+ 8` gap is now documented as the header's own gap-y-2
  row gap reused below it, not a bare magic number.
- (test-coverage) PageShell.dom.test.tsx: stub ResizeObserver + the header's
  offset metrics and assert --hdr-h is published onto the content wrapper (and
  absent with no header).

Re-verified live at 440 (mobile, no overlap) and desktop (one row). FE gate:
tsc / knip / lint clean, 220 vitest pass.
2026-08-17 21:43:14 +02:00
peter 3575237382 fix(header): dynamic --hdr-h so a wrapped mobile header can't overlap content
The floating header pads the content below it by a FIXED `--hdr-h` (4.25rem,
sized for one row). On mobile the flex-wrap header grows to 2–3 rows, so the
content toolbar (sort / count / filter chips) slid up UNDER the header — the top
of the page was a pile of overlapping controls.

- Header.tsx: measure the real header box with a ResizeObserver and publish its
  height (offsetTop + offsetHeight + gap) as `--hdr-h` on the content column, so
  the content pad always tracks the actual (wrapped) header height at any width.
  Verified live at 440 (mobile) and desktop: the toolbar now sits cleanly below
  the header, no overlap; desktop unchanged (still ~4rem, one row).

Also addresses the S7 /code-review findings:
- App.tsx: note that `container-type: inline-size` makes the column a containing
  block for `position: fixed` descendants too (safe today — floating-ui menus
  body-portal, toasts are absolute).
- SearchBar.tsx: move the responsive-rules comment above `return` and note the
  rules are on the SHARED search box (every module, not just Plex).
- SearchBar.dom.test.tsx: lock the container-query classes (min-w floor +
  basis-full/order-last) so the wrap-not-collapse fix can't silently regress.

FE gate: tsc / knip / lint clean, 218 vitest pass.
2026-08-17 21:31:22 +02:00
peter 14913a6447 feat(header): S7 — container-query fix for the narrow Plex header
The floating header's search box collapsed to ~0 width when the filter sidebar
was open on a narrow window (~768px): the content column shrinks with the
SIDEBAR, not the viewport, so Tailwind viewport breakpoints can't see it, and
SearchBar's `flex-1 min-w-0` shrank rather than wrapped.

Fix with Tailwind v4 container queries (no plugin):
- App.tsx: mark the content column `@container/appmain` (container-type:
  inline-size) — the width that actually shrinks when the sidebar opens.
- SearchBar: once the column has room (≥420px) enforce a `min-w-[15rem]` floor
  so the box WRAPS to its own line via the header's flex-wrap instead of
  collapsing the input; on a narrow column (≤640px) it takes a full line of its
  own (basis-full) and sorts last, so the small pills (module name + Plex scope
  toggle / feed sync chip) share the line above it — a tidy two-row header.

Verified live at 375 / 768 / 1024 / 1280 on Plex (search usable, scope toggle
reachable at every width) and on Feed (shared SearchBar, no regression). Desktop
stays a single row. tsc / knip / lint clean.
2026-08-17 21:06:33 +02:00
peter 9fb4db2546 fix(plex): address S6 code-review — 5 findings
1. (correctness) Show hero rating regression: PlexBrowse's show hero rendered a
   rating only when show.imdb_rating was set, so after the S6 IMDb-only change a
   show with no imdb:// entry showed no rating. Now falls back to the audience
   `rating` like PlexInfo.
2. (correctness) ratingMin column mismatch: the rating filter and the /facets
   rating_max now key off coalesce(imdb_rating, rating) — the SAME "best score"
   the badge displays — so a ratingMin picked from a shown IMDb score narrows the
   right column instead of the raw audience `rating`.
3. (altitude) HU search gap: new query.vector_match() matches EITHER vector in HU
   (or_(search_vector_hu, search_vector)) so a row with no HU twin stays findable
   by its English title; ranking still uses the HU vector. field_recheck unions
   the HU+EN per-field tsvectors in HU, staying a strict subset of the prefilter.
4. (test-coverage) Extracted the badge logic to lib/plexRating.pickRatingBadge
   (shared by PlexInfo + the show hero) with a unit test (imdb-preferred, audience
   fallback, 0-score, none). Backend tests added for the HU union search + the
   IMDb-preferred rating filter.
5. (efficiency) Language switch no longer double-fetches the grid: browseQ is
   gated on a committedLang state that lags `lang` by one render, so the
   genre-clear effect settles into a single fetch instead of one throwaway fetch
   with stale-language genres plus a second.

DB-lane suite 295 passed / 2 skip; FE tsc/knip/lint clean, 217 vitest pass.
2026-08-17 20:41:13 +02:00
peter 2bfe7d53eb fix(plex): S6 — thread lang into the player's item_detail query
Found by the HU-overlay E2E: PlexPlayer fetched qk.plexItem(id) WITHOUT lang,
so the in-player title/summary/rich detail stayed English even in HU mode while
the grid showed Hungarian (e.g. the player header read "127 Hours" over a grid
of "127 óra"). The S6 FE pass threaded lang through the grid, facets, info-page
and show queries but missed the player's own detail fetch. Now keyed by
plexLang() like the rest, so the player matches the active language.

FE gate: tsc / knip / lint clean; queryKeys tests pass. Verified live — the
player header now reads "127 óra" in HU mode.
2026-08-17 20:21:29 +02:00
peter 4169cfe4ff feat(plex): S6 frontend — language threading, rating badge, i18n parity
E5 S6 frontend half.

Language threading (?lang=hu serves the HU overlay; en is the default):
- api/plex.ts: `lang` on plexLibrary / plexFacets / plexItem / plexShow (only
  sent for hu, so EN URLs stay byte-stable); PlexItemDetail gains `rating`.
- queryKeys: plexItem / plexShow take an optional trailing lang so a switch caches
  HU/EN separately; the ...rest keys (library/facets) already carry it. Prefix
  invalidations (qk.plexShow(id) / qk.plexItem(id)) still match either.
- i18n: plexLang() collapses the UI language to en|hu for non-component code.
- PlexBrowse / PlexSidebar: thread lang into the grid, facets, item and show
  queries (useTranslation re-renders on switch → the key changes → refetch).
- plexPlayerHooks: the optimistic resume setQueryData now keys by plexLang() so
  it still lands on the language-keyed detail query.
- Clear the genre-filter selection on a language switch — genre chips are
  language-specific strings, so a stale one silently empties the grid.

IMDb-only rating badge (folded-in S5b review #5):
- PlexInfo.tsx shows the IMDb badge + imdb.com link only when imdb_rating is
  present; otherwise it renders the Plex audience `rating` (no IMDb link/branding),
  so a non-IMDb score is never mislabelled. ratingMin filter keys off whichever
  score is shown.

Tests: i18n parity.test.ts (en/hu key sets identical per namespace — currently
green, no drift). FE gate green: tsc / knip / lint clean, 213 vitest tests pass.
2026-08-17 19:19:03 +02:00
peter 030ba7297a feat(plex): S6 backend — language-aware serving + IMDb-only rating
E5 S6 backend half (no migration — the *_hu / rich columns exist from S5a/S5b).

Language-aware serving (?lang=hu serves the HU overlay twin, EN fallback per row):
- query.py: column pickers title_col / genres_col / vector_col; field_recheck and
  meta_filter_conds take `lang`. DISPLAY/FILTER pickers coalesce(*_hu, *) (EN
  fallback); SEARCH pickers use the raw *_hu column mirroring the generated
  search_vector_hu (coalesce to '', not EN) so the field recheck stays a strict
  subset of the GIN vector (the S2 invariant). genres_col type_coerces the coalesce
  to JSONB so the GIN `.contains` survives.
- browse.py: `lang` param on /library, /facets, /item, /show (validated to en|hu).
  unified_library title/vector/rank/field-scope + meta filters are language-aware
  for both the movie and show branches; facets unnest the active-language genre
  vocabulary (jsonb_typeof guard kept); detail endpoints serve HU title/summary/
  genres/show_title with EN fallback. Episodes stay EN (not overlay-localized).

IMDb-only rating (folded-in S5b review #5):
- detail.py rich_detail: drop the audienceRating/rating fallback → imdb_rating is
  NULL when there's no imdb:// Rating entry (stop mislabelling a Plex audience score
  as IMDb). item_detail now also serves the `rating` column (audience score;
  show_detail already did) so the UI can show it when there's no IMDb score.

Tests: test_plex_lang.py (HU title serving + EN fallback, language-specific facet
vocabulary, genre-filter language matching, item_detail lang + rating split);
test_plex_detail updated for IMDb-only. DB-lane suite 293 passed / 2 skipped; ruff clean.
2026-08-17 19:07:43 +02:00
peter c0cda545e0 test(feed): cover malformed-cursor 400 + from-None on the re-raise
Second-round /code-review high follow-ups:

- test_feed_cursor.py: GET /api/feed?cursor=<garbage> must return HTTP 400
  (not 500) — locks the cursor-error path the refactor split across modules
  (feedquery.decode_cursor raises ValueError, get_feed translates to 400).
  Plus a no-cursor 200 sanity check. DB lane.
- get_feed: `raise HTTPException(...) from None` so the expected bad-input
  ValueError isn't chained as __context__.

DB-lane suite 289 passed / 2 skipped; ruff clean.
2026-08-17 18:46:48 +02:00
peter 2ab188d873 refactor: address code-review — layering + public API naming
Applies the three findings from the /code-review high pass on this stack:

1. to_tsquery_str moved from app.feedquery to app.utils — it's a generic
   tsquery sanitizer with no feed dependency; housing it in the feed
   read-model coupled the Plex browse module to the feed domain. Both feed
   and plex now import it from the neutral util module.

2. feedquery.decode_cursor no longer raises fastapi.HTTPException — it raises
   a plain ValueError, and routes/feed.get_feed translates it to the HTTP 400.
   The read-model no longer imports fastapi, matching its "no request state"
   docstring.

3. Dropped the leading underscore on the helpers that are now a cross-module
   public API: feedquery {filtered_query, sort_keys, order_by, keyset_predicate,
   encode_cursor, decode_cursor}; plex.query {TS_CONFIG, FIELD_COLUMNS,
   field_recheck, split_csv, added_cutoff, meta_filter_conds, apply_meta_filters,
   like_escape}; plex.cards {movie_card, show_card, episode_card, leaf_card}.
   Genuinely-internal helpers (_channel_url, _saved_expr, _ADDED_WINDOWS,
   _lib_key) keep their underscore.

Pure code motion + rename. DB-lane suite 287 passed / 2 skipped; ruff clean.
2026-08-17 18:23:03 +02:00
peter e7b5b8e87f refactor(plex): extract card serializers + query builder from browse.py
Second half of the 2026-08-17 review's "Plex query/cards extraction" — slim
the 1841-line browse.py god-module ahead of S6 (which reworks the library
query's column selection for the HU overlay).

- app/plex/cards.py: the pure grid/leaf card serializers (_movie_card,
  _show_card, _episode_card, _leaf_card).
- app/plex/query.py: the library query-shaping layer — field-scope tsvector
  rechecks (_TS_CONFIG, _FIELD_COLUMNS, _field_recheck) and the shared sidebar
  metadata filters (_csv, _added_cutoff, _meta_filter_conds, _apply_meta_filters,
  LibraryFilters, _like_escape). This is the seam S6 will touch.

browse.py imports both and keeps its route handlers; the entangled
collection/playlist card cluster (_collection_source/_collection_editable)
stays put deliberately. All helpers were browse-internal, so no external
importers change. Pure code motion — DB-lane suite 287 passed / 2 skipped;
ruff clean. browse.py 1841 -> 1667 lines.
2026-08-17 17:47:30 +02:00
peter 5f09259bbf refactor(feed): extract read-model to app.feedquery + admin_user sweep
B-core-3 (2026-08-17 review): decouple two things from router modules.

1. Feed read-model → new app/feedquery.py: the pure query-building and
   row-serialization layer (feed_columns, serialize, _filtered_query,
   to_tsquery_str, and the keyset-pagination helpers) moves out of
   routes/feed.py, which keeps only the route handlers, _feed_params, and
   the request models. The three external consumers (playlists, search,
   plex/browse) now import from app.feedquery instead of reaching into the
   feed router.

2. admin_user sweep: 7 modules imported admin_user via app.routes.admin
   (an incidental re-export of app.auth.admin_user). Repoint them all to
   the definition in app.auth (audit, channels, config, quota, scheduler,
   plex/admin, plex/browse); routes.admin keeps its own import for its use.

Pure code motion — no behaviour change. Full DB-lane suite 287 passed /
2 skipped; ruff clean.
2026-08-17 17:42:01 +02:00
peter c361fc2581 fix(gate): green the frontend gate at HEAD
- PlexPersonView.dom.test.tsx: variadic vi.fn mock for plexSetState so the
  spread call typechecks (TS2556 — zero-arg mock couldn't take ...args).
- plexSearch.ts: drop the unused `export` on PLEX_SEARCH_FIELDS (module-local
  only; knip flagged it as an unused export).

tsc / knip / lint all green; ruff clean.
2026-08-17 17:30:12 +02:00
peter 81827fcf28 fix(plex): S5b review — preserve enriched cast_names across full syncs
Address the /code-review high finding on S5b: gating the info-page cast enrich on
`people_enriched_at is None` removed the old per-view regrow of `cast_names`, so a routine full sync
that reset it to the listing top-20 (`_apply_facet_fields`) — while never clearing people_enriched_at —
left it permanently truncated, silently dropping cast past the top ~20 from the S4 person-page filter.

Root fix: `_apply_facet_fields` now only takes the listing's cast for a NOT-yet-enriched row; an
enriched row keeps its fuller stored cast (people_text still rebuilt from the updated directors + the
preserved cast). Makes the people_enriched_at flag the endpoints/backfill trust actually reliable.

Tests: cast_names preserved for an enriched row vs applied for a fresh one; show_detail lazy-fill
persists imdb/full_cast on first view. Full suite green.
2026-08-16 23:30:33 +02:00
peter 4e97fc1a84 feat(plex): S5b persist rich detail + serve item/show detail from DB
Persist the info-page rich fields that were previously read LIVE from plex.metadata() on every view —
imdb_rating (IMDb-specific, vs `rating`=audienceRating), imdb_id, tagline, and full_cast
([{name,role,thumb}]) — onto plex_items/plex_shows, and serve the detail endpoints from the row.

- migration 0065: ADD imdb_rating/imdb_id/tagline/full_cast (nullable → no rewrite/lock, unlike 0064);
  NULL people_enriched_at on movie/show rows so the existing cast backfill re-fetches each once and
  fills the new columns.
- new app/plex/detail.py: rich_detail(meta) (moved out of browse._rich_meta so sync + serve share one
  parser) + apply_rich_detail(row, meta); wired into _enrich_people_batch and the info-page lazy fill.
- item_detail: movies serve cast/IMDb/tagline/genres from the row and keep plex.metadata(markers=True)
  ONLY for markers + audio/subtitle streams (so detail survives a live-Plex outage); episodes (never
  enriched) still parse detail from that live meta.
- show_detail: serves cast/IMDb from the row; `related` stays live best-effort and also lazily fills a
  not-yet-enriched show.
- tests: rich_detail/apply_rich_detail parsing + persistence, backfill stamping, item_detail
  outage-resilience + lazy fill + episode-live, show_detail row serving. Full suite 285 pass / 2 skip;
  migration verified reversible (up→down→up).

Precondition for S6's pure-DB language-aware serving.
2026-08-16 23:11:15 +02:00
peter 02a2102523 fix(plex): S5a review round 2 — dup-HU guard + tests for dup-guid & guid-keep
- Correctness: _sync_overlay now first-wins on a guid (a `seen` set) so a duplicate HU entry
  sharing a guid can't overwrite the applied Hungarian metadata or double-count `overlaid`.
- Test-coverage: cover the earlier dup-guid fix (both editions of a shared-guid pair get *_hu,
  counted once) and the guid-keep guard (_apply_item preserves a prior guid when the listing
  carries none, still updates on a real one).

Full backend suite 276 passed / 2 skipped.
2026-08-16 22:38:59 +02:00
peter 704c6c35d4 fix(plex): address S5a review — overlay reconcile, dup-guid, guid-wipe, tests
- Correctness (stale HU): _sync_overlay is now a full reconcile — it clears every guid-bearing EN
  row's *_hu first, then repopulates the matches, so a title whose HU twin vanished or was re-guid'd
  falls back to EN instead of keeping stale Hungarian metadata (only rows that held HU data go dirty).
- Correctness (dup guid): by_guid maps a guid to the LIST of EN rows carrying it, so two editions of
  one film both receive the overlay instead of just the last-iterated one.
- Robustness (guid wipe): _apply_item/_sync_shows keep a prior guid when a listing carries none
  (`_canonical_guid(meta) or row.guid`), so a server ignoring includeGuids can't erase the match key.
- Reuse: merged _sync_overlay's movie/show branches (only the select + item_type differ).
- Test-coverage: a full-sync test drives sync() over a stub Plex (sections + section_items +
  overlay_map) asserting pass-2 designation (metadata_lang/overlay_of), guid capture, the HU write,
  and browse exclusion; plus a reconcile test that a vanished twin clears stale *_hu.

Full backend suite 274 passed / 2 skipped.
2026-08-16 22:31:08 +02:00
peter c959859d29 feat(plex): S5a HU-overlay backend — schema, guid, overlay sync
Adds the storage + sync for the Hungarian-metadata overlay (the browsable EN sections gain HU
twin fields written from a paired HU section, matched by guid; no browsable HU rows).

Schema (2 migrations, ADD-only; run before the first HU sync while *_hu is NULL so the generated
column build is trivial):
- 0063: plex_libraries.metadata_lang ('en' default) + self-FK overlay_of.
- 0064: plex_items/plex_shows.guid (indexed) + title_hu/summary_hu/genres_hu (+ GIN) + a STORED
  generated search_vector_hu mirroring the EN vector (title_hu A / people_text B / summary_hu C) + GIN.

Sync:
- includeGuids=1 on section_items so the canonical guid rides the cheap listing.
- _canonical_guid(meta) (tmdb > imdb > tvdb) captured onto every EN row in _apply_item/_sync_shows.
- Two-pass sync(): pass 1 mirrors browsable (non-overlay) sections; pass 2 reads plex_overlay_map
  ('huKey:enKey', new sysconfig), designates each HU section (metadata_lang='hu' + overlay_of), and
  _sync_overlay writes title_hu/summary_hu/genres_hu onto the guid-matched EN rows.

Browse: _content_libs now excludes overlay libraries (overlay_of is None); the unified_library and
facets inline library partitions fold into it, and list_libraries/collections exclude overlays too.

Tests: DB-lane coverage of the overlay write (matched vs unmatched rows, no browsable HU rows, the
generated search_vector_hu populates), the guid parse, and the browse exclusion. Full suite 272
passed; migrations verified reversible (upgrade→downgrade→upgrade). S5b (persist the live detail
fields + serve item/show detail from DB) is the follow-up.
2026-08-16 22:16:54 +02:00
peter 072d8fc88f fix(plex): address S4 review — cap filmography, dedupe toggle, scroll guard, tests
- Efficiency: cap each person section at _PERSON_SECTION_MAX (200) so a prolific person can't
  return an unbounded list the client mounts a card per row for.
- Reuse: extract the optimistic watch-toggle (flip → persist → resync) into a shared
  toggleWatchState helper; the grid and the person page each supply their own cache updater.
- Correctness (scroll): onOpenPerson now stashes infoScrollRef ONLY when leaving an info page
  (not from a show/grid context), and a jumped-to info page resets it — so a stale offset can't
  wrongly restore a fresh info page.
- Test-coverage: add PlexPersonView.dom.test.tsx (export CastStrip + PlexPersonView) covering
  the cast-row rewiring (open-person vs filter fallback) and the person page's sections + filter
  + empty state.

The name-as-identity finding is inherent to Plex cast carrying no per-person id (PlexPerson is
name-keyed too); documented in the endpoint docstring, not structurally fixable in S4.
2026-08-16 21:55:34 +02:00
peter 87ef075ea7 feat(plex): make the actor filter-chip open the person page
The person cards above the Plex grid (the active actor filters) now open that person's
filmography page on click, reusing S4's onOpenPerson; the ✕ still removes the filter. The
name/photo become a button with the viewPerson tooltip; the remove control stays separate.
2026-08-16 21:37:43 +02:00
peter 8b8c077a29 feat(plex): S4 actor detail page — filmography endpoint + person subview
Backend: new GET /api/plex/person/{name} returns a cast/crew member's cached headshot plus
the movies (PlexItem, kind=movie) and shows (PlexShow) in the enabled libraries whose
cast_names OR directors include the name — the inverse of the cast → filter link. Introduce a
shared _content_libs(db) helper (enabled-library partition) and reuse it in people_cards;
S5 will extend it with the HU-overlay exclusion. Watch-state comes from PlexState / rollup
(reusing _movie_card / _show_card / state_map). DB-lane tests cover cast+director credits,
library scoping, the kind filter, unknown-name empties, and the partition helper.

Frontend: a new `person` history-subview. A cast-row click (PlexInfo cast row + the show
page's CastStrip) now opens the person's filmography page instead of filtering; the filter
gesture moves to a "Filter library by {name}" button on the page itself. PlexPersonView
mirrors PlexInfoView's fetch-wrapper + glass hero (headshot + name + credit counts) and
renders Movies / Shows as PlexPosterCard grids, with an optimistic per-card watch toggle on
the person cache. New plexPerson query key + api client + PlexPersonDetail type; HU/EN copy.
2026-08-16 21:26:58 +02:00
peter 6465f0ce75 style: prettier-format the 0.65.2 release-notes entry
The release gate's prettier --check flagged the new entry; no content change.
2026-08-16 20:13:09 +02:00
peter ee356f8a1f release: 0.65.2 (fix/download-gc-title-truncation) 2026-08-16 20:10:55 +02:00
peter 588a1a82a1 release: bump VERSION to 0.65.2 + release notes (download-cleanup title-truncation fix) 2026-08-16 20:08:23 +02:00
peter 80ea1918e3 fix(downloads): derive title clamp from the column + real-Postgres regression
Address review findings on the title-clamp: TITLE_MAX_LEN now reads the length straight
from Notification.title so the clamp can't drift from the column a migration would change
(single source of truth). Add a db-lane test that actually commits a >255-char title into
Postgres (end-to-end proof the truncation is gone, not just the clamp logic) and one that
asserts the constant tracks the column width.
2026-08-16 19:40:17 +02:00
peter 83326ecf53 fix(downloads): clamp notification title to column limit so download_gc can't wedge
A downloaded FB/IG reel's "title" is the whole caption (up to ~1900 chars). It flowed
untrimmed into Notification.title (String(255)); at the batched download_gc commit that
raised psycopg StringDataRightTruncation in pass 1, aborting the whole run so expiry
deletion, LRU eviction and orphan reap never executed (13/110 prod assets already exceed
the limit). Clamp the title to 255 (with an ellipsis) centrally in create_notification —
defending all six callers — since the full title already reaches the client via `data`
and the column is only an English fallback. Add a regression test.
2026-08-16 19:24:51 +02:00
peter 1d6c5f051e fix(plex): address S3 review nits — tests + narrow-header wrap
- Add PlexScopeToggle.dom.test.tsx (renders the three scopes, current one
  pressed, click writes scope back to the provider) and
  usePlexActiveFilters.dom.test.tsx (emits filter + watch-state chips but never
  a sort chip, even with a non-default sort active) — the S3 relocation was
  untested.
- Narrow-width header: make the header row flex-wrap so the scope toggle drops
  to a second line instead of crowding the flex-1 search box to zero width, and
  let the field-scope pill shrink/truncate. Desktop stays a single line
  (verified: 1280px header unchanged, search 237px, no wrap).

The removed one-click sort-reset chip is intended (sort is an always-visible
control now, like the Feed) — no change. FE suite 174 green; typecheck + eslint
clean.
2026-08-16 19:02:02 +02:00
peter a702e72973 fix(plex): replace stray NUL byte with a space in the duration sentinel
PlexSidebar.tsx carried a literal 0x00 byte in the duration-filter
FilterChipGroup's off-bucket sentinel (`... ? "" : "\0"`), where a normal space
was intended. Functionally inert (both the NUL and a space match no bucket
option, so a set-but-off-bucket duration selects nothing), but a NUL in source
is corruption — replaced with an ASCII space. No NUL bytes remain in src.
2026-08-16 18:45:47 +02:00
peter d0f5559c49 feat(plex): S3 toolbar relocation — scope to header, sort above the cards
Move the two "what am I browsing" controls out of the filter sidebar to where
they're always in reach, matching the Feed's layout:

- Library scope (both/movie/show) → a compact SegmentedToggle in the header next
  to the search box (new PlexScopeToggle, reads the same provider state).
- Sort → a SortControl in the row above the cards (into the PageToolbar band,
  count on the left, sort on the right), like the Feed.

Sidebar cleanup: drop the `scope` and `sort` island cards; the panel badge now
counts only the filters still in the panel (expanded filters + watch-state);
remove the now-unused sort imports. panelLayout drops both from the default plex
order (stale saved-layout keys are auto-filtered, so existing layouts degrade
cleanly). usePlexActiveFilters no longer emits a sort chip — sort is an
always-visible control now, not a removable narrowing (same as the Feed).

FE-only, no backend/migration. typecheck + eslint clean; 171 vitest green;
self-E2E verified (header scope drives the grid, sort control reorders, sidebar
no longer shows the two cards, no sort chip).
2026-08-16 18:30:48 +02:00
peter 5134ba5433 fix(plex): address S2 round-2 review nits (typeahead highlight + tests)
- Reset `hi` on Escape and gate `activeOptionId` on `open`, so a dismissed
  panel never exposes an aria-activedescendant pointing at an unmounted option,
  and no stale highlight flashes on reopen (live-verified: aria-activedescendant
  is null when nothing is highlighted / the panel is closed).
- Strengthen the reopen test: setQ now writes back to the mock provider's q, so
  the test asserts a FRESH suggest query fires on retype (q="matrixx"), not just
  that a cached listbox re-appears.
- Add a combobox-ARIA test: role=combobox, aria-expanded toggles with the panel,
  and aria-controls associates the input with the listbox it opens.

Suite 171 green; typecheck + eslint clean.
2026-08-16 08:34:56 +02:00
peter c51df4e0b9 fix(plex): address S2 /code-review findings (typeahead focus + a11y + test)
- Decouple the typeahead's dismiss from real focus: a separate `dismissed`
  flag (set on Escape/pick, cleared on typing/refocus) replaces manually
  clearing `focused`, so closing the panel no longer wedges it shut while the
  input keeps DOM focus — typing again re-opens it (verified in-browser).
- Scroll the highlighted option into view during arrow-key nav (listRef +
  data-idx + scrollIntoView), instead of leaving listRef unused.
- Wire the input as a combobox for screen readers: role=combobox,
  aria-autocomplete/expanded/controls/activedescendant + per-option ids.
- Drop the redundant client-side .slice (the request already caps at limit=10).
- SearchBar gains an `inputProps` passthrough for the combobox ARIA.
- Add PlexSearchBox.dom.test.tsx (4 tests): per-scope pill options, suggestion
  render + keyboard pick→jumpTo, and the Escape→retype reopen regression.

Reviewer's scrollbar-blur concern was a false positive — the listbox container
is not focusable, so a mousedown on it never blurs the input.
2026-08-16 08:21:29 +02:00
peter f9681deecf feat(plex): field-scoped search (S2) + instant search typeahead
Add a search scope-pill to the Plex header search box (GitHub-style, one
field-scope at a time) and an instant typeahead of the top matches below it.

Backend (unified_library):
- New `field` param: all | title | people | summary | episode. `all` is
  byte-identical to the pre-scope behaviour (untouched weighted search_vector);
  a specific field keeps the GIN `search_vector @@ tsq` prefilter AND ANDs a
  per-column tsvector recheck, so scoped hits are a strict subset of Everywhere.
  Per-field expressions mirror the search_vector Computed def (same TS config,
  same left(summary,1000) truncation).
- `episode` scope empties the movie/show grid and title-only-matches the grouped
  Episodes section; the title/people/summary scopes suppress that section.
- Unknown field falls back to `all`. New DB-lane characterization tests
  (test_plex_field_scoped_search.py): each scope narrows correctly, field=all is
  identical to today, episode empties the grid and needs shows in scope.

Frontend:
- PlexSearchBox wraps the shared SearchBar with the scope-pill (reusing
  ActionMenu) and the typeahead; SearchBar gains generic leading/below/focus/
  keydown slots so other modules' search boxes are untouched.
- Episode scope offered only when shows are in scope (movie-only hides it),
  healed at the point of use like the sort state.
- Typeahead reuses /api/plex/library (limit=10), honours scope+field (not the
  sidebar filters), shows poster+title+year, keyboard nav (arrows/Enter/Esc),
  and jumps straight to a movie's info page or a show's drill-down via a new
  provider jumpTo hand-off (mirrors the playlistOpen pattern).
- `field` persisted per-account; en/hu labels added.
2026-08-16 08:05:18 +02:00
peter 73a58674cf fix(plex): address S1 review round-2 findings
- by_rating_key: generic-typed (type[_M] -> _M | None) so call sites keep their
  Model | None static type.
- Migrate the last 2 open-coded PlexCollection-by-rating_key lookups (sync.py
  resync_collection + delete_collection_local) to by_rating_key; zero open-coded
  rating_key lookups now remain in the package.
- Add test_playlist_bulk_add_and_remove covering the bulk add/remove endpoints
  and pinning the remove-bulk synchronize_session=False path.

Full suite 258 passed, 2 skipped; ruff clean.
2026-08-16 07:14:09 +02:00
peter e008c48284 fix(plex): address S1 code-review findings
- Restore synchronize_session=False on the remove-bulk Core delete (the 2.0
  default is 'auto'; keep the pre-conversion contract explicit).
- _episode_nav: list(...scalars()) instead of [r[0] for r in ...all()], matching
  the house style used across the conversion.
- Extract app/plex/lookups.by_rating_key() and migrate the get-by-rating_key
  pattern at 13 sites (imgcache x4, _shared.item_or_404, playback show/season,
  browse x6).
- Coverage: add TestFacetsWithData (the converted wsq aggregation with real
  data — the highest-risk untested builder), TestPlaylistAndCollectionEndpoints
  (playlist CRUD + collections list), and an item-state playback test.

Full suite 256 passed, 2 skipped; ruff clean.
2026-08-16 07:06:19 +02:00
peter f8df504d18 refactor(plex): convert browse.py + rollup db.query() to 2.0 select() (wave 2)
Completes the plex-package 1.x->2.0 conversion, behaviour-equivalent (full
suite 253 passed, 2 skipped; the characterization net locks every path):
- routes/plex/browse.py (61 sites): libraries, facets (the wsq builder +
  jsonb genre unnest), collections, playlists (+ bulk add/remove/reorder),
  seasons, people/cards, collection strips, episode-nav, item detail, and
  the unified_library union builder.
- plex/rollup.py: agg_subquery now returns a select(); state_map executes it
  (the atomic unit coupled to unified_library via .subquery()).

Query builders -> select(); .filter/.filter_by -> .where; terminals ->
db.execute(...).scalars()/.all()/db.scalar(...); bulk .delete()/.update() ->
execute(delete()/update()). Zero db.query()/.filter_by remain in the package.
2026-08-16 06:06:21 +02:00
peter 59d50615c4 refactor(plex): convert service-layer db.query() to 2.0 select() (wave 1)
Mechanical 1.x->2.0 conversion of the self-contained plex modules, behaviour-
equivalent (the characterization net + existing plex tests stay green, 39
passed): app/plex/{sync(15),watch_sync(9),people(1),imgcache(4)} and
app/routes/plex/{admin(4),playback(8),_shared(1)}. Query builders become
select(); .first()/.all()->execute().scalars(); a bulk .update() ->
execute(update()); a single-column .first() -> db.scalar().

browse.py (61) + rollup.agg_subquery (the atomic unit with unified_library)
remain for wave 2.
2026-08-16 05:52:40 +02:00
peter ecbf5737b6 test(plex): characterization net for the db.query->select conversion
Locks the observable behaviour of the previously-untested Plex query hot
paths before the 2.0 conversion, so it can be proven behaviour-equivalent:
- unified_library: both/movie/show scope, added-desc ordering, the four
  watch-state branches, search rank + episode grouping, pagination.
- rollup.state_map: new / in_progress / watched rollup.
- sync upsert idempotency: _sync_movies + _sync_shows run twice = no dup
  rows, in-place update, episode->show linking.

Green on the current 1.x code (9 passed). The conversion commits must keep
these identical.
2026-08-16 05:45:05 +02:00
peter f38833f104 release: 0.65.1 (fix/backfill-immediate-full-history) 2026-08-16 03:22:32 +02:00
peter 9105c4db3d release: 0.65.1 — immediate full-history finishes large channels
Bumps VERSION to 0.65.1 and adds the release note for the backfill fix
(immediate "get full history" now completes channels over ~3000 videos in
one run) plus the per-page durability + centralized reserve-guard hardening.
2026-08-16 03:22:12 +02:00
peter ee2bd81254 test(backfill): lock the per-page durability and reserve-guard properties
Deep-review round 1 found the new tests asserted end-state only, so both
safety mechanisms this change adds were untested (a regression would ship
green):

- Per-page cursor durability: add test_a_mid_run_failure_leaves_the_prior_page
  _durably_committed, which fails the 2nd page's fetch after page 1 committed
  and asserts the cursor + rows are already persisted (re-read via db.expire).
  Mutation-verified: reverting to a single end-of-loop commit makes it fail
  while the pre-existing paging tests stay green — the property the original
  test could not distinguish.
- Reserve guard: add test_backfill_reserve_reached_tracks_the_boundary, driving
  real record_usage spend across the 9000/2000 default boundary and asserting
  the un-mocked quota.backfill_reserve_reached() flips at exactly remaining<=reserve.

Full backend suite: 244 passed, 2 skipped (network-gated).
2026-08-16 03:02:37 +02:00
peter 48285a715b refactor(backfill): centralize the reserve-yield guard and drop the per-page connection hold
Addresses the code-review findings on the immediate full-history change:

- Extract quota.backfill_reserve_reached(): the remaining/reserve compare was
  open-coded at seven sites (recent/deep/immediate backfill, explore, rolling
  revalidation, and the new per-page guard). It reads in its OWN short-lived
  session (like record_usage/log_action), so a per-page guard no longer holds
  the caller's transaction open across the YouTube fetch that follows — the
  400-page immediate run no longer pins a pooled connection idle-in-transaction
  through each network round-trip. Migrated all call sites; dropped the now-dead
  sysconfig imports in channels.py and maintenance.py.
- Simplify the deep-backfill loop: channel.backfill_done = not page_token (done
  is False on entry) plus one break, replacing the doubled `if not page_token`.
- Tests patch the backfill_reserve_reached seam directly instead of mocking
  remaining_today internals, removing the fragile dead-value budget iterator.

Full backend suite green; ruff clean.
2026-08-16 02:41:49 +02:00
peter e3ee1baa94 fix(backfill): finish large channels in one immediate full-history run
The user-triggered "get full history" (backfill_channel_now) capped at
max_pages=60 = 3000 videos. A channel just over that (BBC Earth: 3403
uploads = 69 pages) stopped one page short, so its tail dribbled in over
the scheduler's paced 10-minute ticks (250 videos/tick) and the button
spinner looked stuck for ~18 minutes with no visible progress.

- Raise the immediate cap to max_pages=400 (20 000 videos) so any
  realistic channel completes in the single background run.
- Make a high cap safe: backfill_channel_deep now re-checks the shared
  quota before each page and yields at the backfill reserve (previously
  quota was checked once by the caller before the whole run), and commits
  the advanced cursor after every page so a restart resumes instead of
  re-fetching — and re-charging quota for — pages already stored.
- Channel page: while loading, the button shows the live stored/total
  count and a tooltip explaining the tail finishes in the background, so a
  very large channel's paced remainder reads as progress, not a hang.

Adds deep-backfill paging tests (per-page cursor commit + resume, and the
per-page quota yield).
2026-08-15 17:13:52 +02:00
peter a7b4b2ed79 release: 0.65.0 (r14-channel-table-redesign) 2026-08-15 07:26:23 +02:00
peter 7dc9f712a0 release: 0.65.0 — R14 channel-table redesign + immediate full-history backfill
Bump VERSION to 0.65.0 and add the user-facing release notes for R14
(zoom-fit manager table, ✓ charm, N/S/L moved to the channel page,
live-refresh, immediate "get full history").
2026-08-15 07:26:09 +02:00
peter 0842babc77 fix(channels): trim fixedLayout column widths to fit narrower viewports (R14 F5)
Shave the manager table's fixed column widths (fixed sum 720->632px) and
convert priority/actions from px to rem so they scale with the text-size
slider like their siblings: tags 12->9rem, actions 104px->5.5rem,
lastUpload 6.5->6rem, stored 5.5->5rem, length 5->4.5rem, priority
56px->3.5rem. Gives the fluid Channel/About columns ~90px more room, so the
table fits without a horizontal scrollbar down to ~700px containers instead
of overflowing at the md/iPad-portrait band.
2026-08-15 07:20:18 +02:00
peter 5720356191 fix(channels): address R14 deep-review findings
- Make the immediate-backfill guard SYMMETRIC: run_deep_backfill now also
  registers the channel it pages in _backfill_in_progress, so it and
  backfill_channel_now mutually exclude per-channel and never advance the
  same cursor concurrently (whichever started first). The interactive cap
  moves to a separate _immediate_running counter so the scheduler's channels
  don't false-trip it.
- Attribute the immediate backfill's spend: wrap it in
  quota.attribute(user_id, VIDEOS_BACKFILL_FULL) (route passes user.id), so
  it shows under the user's full-backfill action, not the OTHER/system bucket.
- Tests: the update_channel->background-task wiring + the demo guard
  (scheduled for a real user, NOT for demo); distinct N/S/L counts (2 short
  + 1 live) so a short/live swap fails; extracted channelDetailPollMs with a
  pure unit test of the poll-or-stop predicate.
- a11y: role="img" on the fully-synced charm so screen readers announce it.
2026-08-15 07:11:50 +02:00
peter 2cc77757d9 fix(channels): address R14 code-review findings
- list_channels no longer computes the Normal/Shorts/Live aggregation the
  frontend stopped reading when R14 dropped the N/S/L table column; the dead
  ManagedChannel.count_* fields are gone too.
- Immediate "get full history" backfill now coordinates with the paced
  scheduler via a small in-flight set: capped at 2 concurrent + per-channel
  dedup, and run_deep_backfill skips a channel an immediate backfill owns —
  killing the cursor race and bounding the quota-drain burst, without losing
  the speed win.
- Add test_channel_backfill.py: the channelDetail N/S/L counts + the guard's
  no-op / cap / scheduler-skip behaviour.
- Inline the syncedChannels useMemo (its dep is a fresh array each render, so
  it never cached); fix the stale "~2-min ticks" comment (it's 10 minutes).
2026-08-15 06:38:19 +02:00
peter a342e15e05 feat(channel): backfill a channel's full history immediately on request
Clicking "get full history" now kicks an immediate background deep
backfill of that one channel (backfill_channel_now: own session, up to
~3000 videos in one pass, quota-guarded) instead of waiting for the
scheduler's paced 10-minute ticks (5 pages/channel/tick). The request
returns instantly and the channel page's live poll shows the count fill
in seconds. A very large channel's tail still falls to the scheduler
(deep_requested stays set).
2026-08-15 06:01:42 +02:00
peter be738d3791 feat(channel): live-refresh the channel page while full-history backfills
The channel detail page now polls (via useLiveQuery) while THIS user's
full-history backfill is running (deep_requested && !backfill_done), so
the stored count and the N/S/L breakdown climb on their own as the
scheduler adds videos — no manual F5 — and it stops polling once the
back-catalogue is in. Widen useLiveQuery's interval callback to allow
returning `false` (stop) so "poll while active, stop when done" is a
first-class, reusable cadence.
2026-08-15 05:50:11 +02:00
peter 95aa57a2ff style: prettier-format the DataTable fixedLayout test 2026-08-15 05:31:55 +02:00
peter b4dae04631 test(channels): cover applySyncFilter + DataTable fixedLayout
- applySyncFilter: the fully/needsFull/all predicate the top chips drive
  (moved off the retired Sync column) — guards which channels show.
- DataTable fixedLayout: asserts the colgroup + table-fixed appear only
  when opted in, with widths on fixed columns and none on fluid ones.
2026-08-15 05:30:40 +02:00
peter 7ee0e9cd02 feat(channels): R14 zoom-fit table — fluid columns + column fogyókúra
- DataTable: opt-in `fixedLayout` (colgroup + table-layout:fixed) so the
  table fits its container at any zoom/text-size; only the channel manager
  opts in — playlists/discovery/Plex keep auto-sizing.
- Remove the N/S/L and Sync columns from the manager table. Fully-synced
  status becomes a ✓ charm on the Stored count; the sync filter moves to an
  `applySyncFilter` pre-filter still driven by the top chips.
- Channel column is now fluid: ChannelLink truncates with the full name on
  hover (title). Data columns get fixed widths so only Channel/About flex.
- N/S/L (normal/short/live) breakdown moves to the channel page: add the
  three counts to the channelDetail endpoint (one grouped query, same
  filters as the list) + the ChannelDetail type + a meta-line entry.
- Drop the now-dead sync/types i18n keys (en+hu).
2026-08-15 05:28:59 +02:00
peter ea7bf593cb release: 0.64.0 (r13-sort-filter-unification) 2026-08-15 04:54:08 +02:00
peter 19f1905da9 release: 0.64.0 — R13 sort/filter idiom unification + Hidden tab
Bump VERSION to 0.64.0 and add the user-facing release-notes entry for
the R13 sprint (unified sort/filter controls, Hidden channels tab,
channel-manager toolbar declutter, filter-rail jitter fix).
2026-08-15 04:53:08 +02:00
peter 5838cf1c56 test(channels): cover ActionMenu keyboard-skip + stats ETA value; drop redundant disabledIndices
Deep-review follow-up on R13 S3.

- Add a roving-keyboard-nav test proving a disabled ActionMenu item is
  skipped by arrow keys, and assert the stats "left" row renders the
  formatEta value (formatEta had no coverage anywhere).
- Remove the disabledIndices plumbing from AnchoredPopover + ActionMenu:
  it was redundant. useListNavigation already skips items via their DOM
  disabled/aria-disabled attribute, and PopoverItem renders a native
  disabled button — verified against @floating-ui/react 0.27
  isListIndexDisabled and by a mutation test. Behaviour is unchanged and
  now guarded by the new test.
2026-08-15 04:45:32 +02:00
peter bb2d6999fd fix(channels): address R13 S3 review findings
- ActionMenu: skip disabled items in roving keyboard nav (plumb
  disabledIndices through the AnchoredPopover platform); a pending
  action no longer strands focus on an unfocusable button.
- ActionMenu + ChannelStatsPopover: add type="button" to the popover
  triggers so they can't submit an enclosing form.
- ChannelStatsPopover: restore the per-stat explanations as a native
  title on each row — the guidance the old inline stats showed is back.
- i18n: drop the now-dead channels.tags.yourTags / .manage keys (en+hu);
  the stats *Hint keys are live again via the restored row titles.
- test: cover ActionMenu (open/select/close/disabled) and
  ChannelStatsPopover (rows, conditional ETA row, per-row hint title).
2026-08-15 04:09:30 +02:00
peter a26d96630d i18n(channels): add missing HU actionsLabel (R13 S3)
The ⋯ menu aria-label was added to en only; add the Hungarian so every
user-facing string ships in both languages.
2026-08-15 03:42:29 +02:00
peter b52c87b336 polish(ui): tighter header gap, stronger row hover, readable tag chips (R13 S3)
- Channel-manager tab row: pt-4 -> pt-2, closing the too-wide gap under the
  fixed header.
- Row hover was barely visible (--card sits close to --bg in dark schemes). New
  shared --hover token (a 9% foreground mix, clear in every scheme + theme,
  auto-adapting via --fg); the .selectable recipe and the DataTable row both use
  it, so every hoverable row app-wide (tables, nav rail, menu rows) reads.
- Channel tag chips: text-3xs -> text-2xs + font-medium, so the tiny labels stay
  legible on the accent fill across all colour schemes.

Self-E2E'd: tighter gap, a table-row hover now clearly visible. gate green
(tsc/eslint-0/vitest 147/knip/prettier/build).
2026-08-15 03:40:35 +02:00
peter 6de6dece03 refactor(channels): collapse the stats row into a Stats hover chip (R13 S3)
The six inline catalog stats crammed into the tab row become one compact
"Stats" chip that reveals the numbers on hover — a new ChannelStatsPopover on
the AnchoredPopover hover mode, one icon-per-row (channels, recent synced, full
history, [ETA], my videos, quota left). Kept in the manager's own toolbar (the
header SyncStatus is global; these are channel-catalog figures). Retires the
local Stat helper + the now-unused formatEta import. i18n channels.stats.label
(HU+EN). Self-E2E'd: hover reveals the list. gate green (tsc/eslint-0/prettier).
2026-08-15 03:33:34 +02:00
peter ca7e4b16a0 refactor(channels): ⋯ overflow ActionMenu + TAGS-as-trigger (R13 S3)
Declutter the channel-manager tags row:
- New reusable ActionMenu (⋯) on the AnchoredPopover platform — a compact
  overflow menu of one-shot actions with optional pending/disabled + spinner.
  The two big glass buttons (Read subscriptions, Backfill everything) move into
  it, freeing the row.
- The "YOUR TAGS" caption becomes a "✎ TAGS" trigger button that opens the tag
  editor directly, retiring the separate "Manage tags" button.

Adds i18n (channels.actionsLabel, channels.tags.label; HU+EN). ActionMenu is
built to be reused for other overflow menus later. Self-E2E'd: ⋯ opens the two
actions, ✎ TAGS opens the manager. gate green (tsc/eslint-0/prettier).
2026-08-15 03:27:15 +02:00
peter 003043a989 test(sidebar): cover the date preset/custom selection logic (R13 S2c)
The deep-review flagged that the feed sidebar's new date single-select
derivation was the one bit of migrated logic without a test, while its sibling
primitives (FilterChipGroup/SegmentedToggle/SortControl) all gained one. Lift
the pure logic into lib/dateFilter (deriveDateKey + applyDateChip), same split
as columnSort, and cover it: dateKey for none/preset/custom/off-preset-value,
the preset toggle-off, the preset-not-active-while-custom-open case, and the
Custom open/close + maxAgeDays clearing. Sidebar now calls the helpers; the
onChange is behaviour-identical (self-E2E'd: preset on/off, Custom editor).
11 new tests (vitest 136 -> 147). gate green. Branch not merged.
2026-08-15 02:27:22 +02:00
peter 80cdafdbfe fix(ui): address self-review findings (R13 S2c)
Three low-severity findings from the /code-review high self-pass:
- Plex duration: the "Any" chip no longer false-highlights when durationMin/Max
  hold a set-but-off-bucket value (stale/version-skew state) — it selects
  nothing then, matching the pre-sweep behaviour, instead of colliding on the
  empty-string sentinel.
- FilterChipGroup: count badge now guards on count > 0 (was `!= null`), matching
  the Tabs badge so a future count:0 can't render a stray "0" pill.
- Extract the shared CountBadge component; Tabs and FilterChipGroup now render
  the count-badge token from one source (was two near-identical inline spans).

gate green (tsc/eslint-0/vitest 136/knip/prettier). Duration self-E2E'd:
bucket select + Any toggle intact. Branch not merged.
2026-08-15 01:37:05 +02:00
peter 9fdf274422 refactor(ui): ChannelPage Videos/About tabs -> shared Tabs (R13 S2c)
The channel page's underline tab pair was the last hand-rolled tab idiom on a
swept surface. It now uses the shared <Tabs> (rounded-lg pill, active accent
shadow) like every other tab-set, sitting inline with the channel action
buttons on the same separator row. Drops the bespoke tabClass helper. Only the
NavSidebar icon rail (aria-current, icon-only app nav) stays a distinct idiom.
Self-E2E'd: Videos/About switch, content flips. gate green (tsc/eslint-0/
prettier). Branch not merged.
2026-08-15 01:17:46 +02:00
peter 07e95ca8e8 refactor(ui): fold the feed TagChip onto the shared filterChipClass (R13 S2c)
The feed sidebar's multi-select TagChip (language/topic/tag chips) was the last
hand-rolled chip whose className duplicated the canonical geometry verbatim
(it's the reference look filterChipClass was derived from). Point it at the
shared constant; the only delta is dropping a one-off hover:shadow-sm so every
filter chip in the app now hovers identically. Badge markup unchanged.
gate green (tsc/eslint-0/prettier). Branch not merged.
2026-08-15 01:08:17 +02:00
peter dd2d53bb96 refactor(ui): fold Settings + Stats tabs into shared Tabs (R13 S2c step 6)
Tabs gains an `orientation` prop (+ optional per-tab `icon`/`testId`) so the
two hand-rolled NAV tab-sets that duplicated its active treatment now use it:
- SettingsPanel vertical rail (appearance/notifications/account) -> vertical
  Tabs with icons; settings-tab-* testIds preserved
- Stats overview/system tabs -> Tabs (rounded-full+border -> the rounded-lg
  tab canonical, per the agreed tab shape)

The underline ChannelPage tabs and the icon NavSidebar rail are intentionally
left as distinct idioms. Self-E2E'd: settings rail switches + persists with
icons, stats tabs toggle. gate green (tsc/eslint-0/vitest 136/knip/prettier).
Branch not merged.
2026-08-15 01:04:59 +02:00
peter b57aec113e refactor(ui): close the sort/layout reuse-gaps (R13 S2c step 5)
Two surfaces reinvented controls the app already consolidated:
- PlaylistsRail sort (a bespoke <select> + hand-rolled asc/desc flip) now uses
  the shared SortControl; "custom" is a directionless option (no arrow), the
  "unsynced first" pin rides the trailing slot as a pill toggle
- PlexPlaylistView Accordion/Tree layout (two hand-rolled toggle buttons) now
  uses the shared ViewSwitcher with a Rows3/ListTree option set

Adds aria-label i18n (plex.playlist.layoutLabel; HU+EN), drops the now-unused
ArrowUp/ArrowDown + ReactNode imports. Self-E2E'd the rail sort: key switch
reveals the arrow, asc<->desc flips, pin toggles, custom hides the arrow.
gate green (tsc/eslint-0/vitest 136/knip/prettier). Branch not merged.
2026-08-15 01:01:03 +02:00
peter 42d0173095 refactor(ui): unify Channels + Stats filter chips (R13 S2c step 4)
- channel-manager sync-status chips (F2) -> FilterChipGroup (rounded-lg -> the
  canonical rounded-full), controls-row alignment preserved via className
- channel-manager user-tag chips (multi-select) restyled to filterChipClass
  (rounded-lg -> rounded-full) so the whole manager chip set is one look
- stats system time-range chips (F4) -> FilterChipGroup (text-sm -> canonical)

Adds aria-label i18n (channels.filters.syncLabel, stats.rangeLabel; HU+EN).
Self-E2E'd: sync chips + tag chips rounded-full, view TABS stay rounded-lg
(their own canonical), stats 7/30/90d toggles. gate green (tsc/eslint-0/
vitest 136/knip/prettier). Branch not merged.
2026-08-15 00:53:12 +02:00
peter f26e329ecf refactor(ui): unify Plex filter chips; retire local Chip/ChipRow (R13 S2c step 3)
PlexSidebar's bespoke chip system is gone. The media scope (both/movie/show)
now uses the shared SegmentedToggle (fullWidth bar, matching the feed
Mine/Library scope), and every filter chip renders the canonical rounded-full
look:
- show / rating / duration / added (single-select) -> FilterChipGroup, with
  the null/"" "Any" sentinels mapped to string option values
- genres / content-ratings (multi-select) -> plain buttons on filterChipClass
  in their own labelled group (same look, independent semantics)

The local Chip (glass-card resting) and ChipRow helpers are deleted; the
unused ReactNode import dropped. Self-E2E'd on Plex: scope bar, show + rating
chips toggle, zero leftover glass-resting chips. gate green (tsc/eslint-0/
vitest 136/knip/prettier). Branch not merged.
2026-08-15 00:48:09 +02:00
peter 579af9cf89 refactor(ui): shared FilterChipGroup; unify feed filter chips (R13 S2c step 2)
New single-select FilterChipGroup primitive on the canonical rounded-full
"Filters bar" chip look (pill + shadow-xs + press feedback), plus an exported
filterChipClass so multi-select siblings share ONE geometry source of truth.

Migrations:
- feed watch-state chips (F1, Feed.tsx) -> FilterChipGroup, testIds preserved
- feed content-type toggles restyled to filterChipClass (stay multi-select) so
  the whole toolbar row is one chip look; wrapped in their own labelled group
- feed date presets + Custom (F3, Sidebar.tsx) -> FilterChipGroup with a
  behaviour-preserving dispatch (preset toggle-off + Custom editor intact)

Adds aria-label i18n (feed.showLabel/contentLabel, sidebar.datePresetLabel;
HU+EN) and FilterChipGroup.dom.test.tsx (6 tests). Self-E2E'd: watch switch,
date preset on/off toggle, Custom opens the range editor. gate green (vitest
136, tsc/eslint-0/knip/prettier). Branch not merged.
2026-08-15 00:35:22 +02:00
peter 61ee7af9e4 fix(ui): snap scaled root font-size to a whole pixel
The root font-size was calc(16px * var(--font-scale)); a non-integer scale
(1.02 -> 16.32px) made every rem-derived box height fractional. When a filter
section's visible row count changes on a facet refetch, its fractional height
shifts everything below it by the sub-pixel remainder — a deterministic ~0.5px
CSS move that snaps to a visible 1px jitter of the whole filter rail at DPR=1.

round(calc(16px * var(--font-scale)), 1px) keeps the font-scale feature but
quantises the root to an integer pixel, so metrics land on integers and the
sub-pixel reflow jitter is gone app-wide. Verified: root 16.32->16px, chip row
26.453->26px, build + Lightning CSS pass round() through intact.
2026-08-15 00:07:25 +02:00
peter 61e848e63a refactor(ui): shared SegmentedToggle for MATCH controls (R13 S2c step 1)
New source-agnostic SegmentedToggle primitive (rounded-full pill track, one
active option, aria-pressed drives the paint via the .pill recipe). Migrates
the three hand-rolled MATCH segments onto it:
- feed tag any/all "Match" toggle (Sidebar)
- feed Mine/Library scope bar (Sidebar; fullWidth + icons + per-option tips)
- Plex any/all combine toggle (PlexSidebar MatchToggle is now a thin binding)

Caption mode (justify-between + label) covers the match toggles; fullWidth
+ size="md" covers the scope bar. Adds SegmentedToggle.dom.test.tsx (6 tests).
Part of the app-wide single-select control unification; branch not merged.
2026-08-14 22:58:42 +02:00
peter ed1a2b7256 test(channels): cover the tab-history logic; memoize viewTabs (S2b review)
Addresses the 2 applied findings from /code-review high on 735d08c:
- Add ChannelsProvider.dom.test.tsx (4 tests): setView pushes a _chTab history entry +
  persists the tab, re-selecting the current tab pushes nothing, a genuine popstate carrying
  _chTab restores + re-persists it, and a popstate without _chTab is ignored. Real storage
  (jsdom) with the account pinned to 1; only NavigationProvider is mocked. vitest 120->124.
- Memoize viewTabs (keyed on the translation fn) so it isn't rebuilt on every Channels render.

The third finding (consolidate the hand-rolled single-select sync chips with <Tabs>) is
deferred by design into the S2c plan, where MatchToggle will be a shared segmented-control
primitive that can also absorb the sync chips.

gate-green: tsc + vitest 124 + eslint-0 + knip + prettier.
2026-08-14 04:01:38 +02:00
peter 735d08c91d refactor(channels): adopt shared Tabs + usePersistedTab for the manager tab row (R13 S2b)
Retire the hand-rolled channel-manager view pills for the shared <Tabs> component (given an
optional className so it embeds in the justify-between header without the default mb-4), and
back view + layout on usePersistedTab instead of manual getAccountRaw/setAccountRaw useState.
That subsumes the account-pin re-hydrate (usePersistedTab now re-reads on the pin), so
ChannelsProvider's bespoke markManagerDirty/useAccountRehydrate is gone; setView keeps the
history-tab logic (_chTab pushState + popstate) layered on the stable raw setter, and
useAccountPersistedState's setter is now memoized so that stable action holds.

Per the user's call, unify every channel-manager pill on the rounded-lg idiom: the view tabs
via <Tabs>, the sync chips and tag chips + the "manage tags" affordance from rounded-full to
rounded-lg — no more odd-one-out rounded-full.

gate-green: tsc + vitest 120 + eslint-0 + knip + prettier. Self-E2E'd: tab switch, browser
Back across tabs (Blocked->Hidden->Subscriptions with localStorage in sync), and a fresh tab
restoring view=hidden on the FIRST load; zero console errors.
2026-08-14 03:50:43 +02:00
peter e6688edbcc refactor(nav): drop redundant sub-view markDirty; add useFilterState re-hydrate tests
Addresses the 2 findings from /code-review high on 726c382:
- NavigationProvider: openChannel/enterYtSearch no longer markDirty. A sub-view is an
  overlay, not a page change — the page re-hydrate's replaceState already spreads the
  _chan/_yt marker, so the overlay survives on top of the RESTORED page. Removing the
  markDirty lets a fresh tab still restore the stored landing page under a pre-pin overlay
  (previously it stuck on the default). setPage keeps markDirty (a real navigation).
- useFilterState.dom.test.ts: model a mutable pinned account and add 3 integration tests —
  null->id re-hydrate adopts the stored map, the migrate seed carries a legacy value forward
  on re-hydrate when the key is empty, and a pre-pin edit is preserved (markDirty guard).

gate-green: vite build + vitest 120 + eslint-0 + knip + prettier. Fresh-tab restore re-E2E'd
(lands on Plex with full state, zero console errors).
2026-08-14 03:26:30 +02:00
peter 726c382524 refactor(account): address account-pin re-hydrate review — extract module, harden guards
Fixes the 5 findings from /code-review high on the account-pin work:
- Extract the reactive-account primitives (ACTIVE_ACCOUNT_KEY, activeAccountId,
  notifyActiveAccountChange, useActiveAccountId) into a new dependency-free
  lib/activeAccount.ts, breaking the storage.ts <-> useAccountRehydrate.ts import cycle.
  storage.ts / api/core.ts / useAccountRehydrate.ts now import from it.
- NavigationProvider: the page re-hydrate now PRESERVES existing history.state markers
  (spread, not overwrite), and openChannel/enterYtSearch markDirty so a sub-view opened
  before the account pins isn't clobbered by the page override.
- useAccountRehydrate: reset the dirty flag after each account transition, so a future
  non-reloading account switch can't skip the re-hydrate and leak the prior account's state.
- Doc-link useAccountRehydrate to the pre-existing useEffect([me?.id]) server-adopt pattern.
- Tests: model a mutable pinned account and add useSortState integration tests for the
  null->id re-hydrate + the markDirty no-clobber guard (vitest 115->117).

gate-green: vite build + vitest 117 + eslint-0 + knip + prettier. Self-E2E'd on a fresh
tab: still lands on Plex with Movies/In-progress/Year restored, zero console errors.
2026-08-14 03:12:50 +02:00
peter 9917d908eb fix(nav,plex,playlists): extend account-pin re-hydrate to every affected provider
The fresh-tab restore bug wasn't Channels-specific: any provider that mounts above
<App> and reads per-account state at useState-init hit it, since <App> pins the tab's
account only after /api/me. Audited all of them — FeedFilters/FeedView/Prefs/App already
self-correct via their own me?.id server-prefs adopt effects, so the still-broken ones
were the pure-localStorage providers with no server adopt: NavigationProvider (landing
page), PlexProvider (scope/show/sort/filters/q), PlaylistsProvider (selected playlist).
Each now adopts the shared useAccountRehydrate hook (markDirty in the setters; page uses
replaceState so the restore doesn't add a phantom Back step). Also baked re-hydrate into
useAccountPersistedState/Object so their consumers (persisted tabs, player prefs) are
robust even if one ever mounts before the pin. Self-E2E'd on a genuinely fresh tab: it
lands on Plex with Movies/In-progress/Year/search all restored on the FIRST load.
2026-08-14 02:56:55 +02:00
peter b57e4fb9bc fix(channels): re-hydrate per-account manager state once the account is pinned
The provider tree mounts above <App>, which pins this tab's account only after /api/me
resolves — so on a genuinely fresh tab (empty sessionStorage) the manager's sort, filters,
view and layout initialised from an unknown account and stayed at defaults until a second
reload. Make the active account reactive (a subscribable store + useActiveAccountId via
useSyncExternalStore; setActiveAccount/clearActiveAccount notify) and re-read the stored
value when the account transitions null→id, via a shared useAccountRehydrate hook. A
markDirty guard (called by each owning setter) prevents clobbering a value the user changed
in the brief pre-pin window; the effect is keyed on the account id only, so the user's own
writes can't loop it. useSortState, useFilterState and ChannelsProvider (view/layout) all
adopt it. Affects sort identically — this was a long-standing caveat, not new to R13.
2026-08-14 02:37:28 +02:00
peter ca9dd54923 fix(channels,datatable): address R13 S2a review + unify tag chips with the Tags column filter
Review fixes (user /code-review high):
- DataTable: hoist a stable EMPTY_FILTERS fallback so a controlled caller that omits
  `filters` doesn't churn a fresh {} each render.
- DataTable: the persistKey blob now stores only the slices this table OWNS — a
  controlled sort/filter is written as null/{} instead of a stale never-read duplicate.
- useFilterState: optional `migrate` seed (used only when storage is empty);
  ChannelsProvider carries the retired LS.channelFilter sync chip forward via migrateSyncChip.
- Channels: guard the sync-chip read with typeof instead of an `as string` cast.

Also unify the tag filter, same as the sync one: the top "YOUR TAGS" chip row is now a
controlled view over the Tags COLUMN filter (`filters.tags`), replacing its own tagFilter
state — so the chips and the Tags funnel share one state bidirectionally, the card layout
applies it via filterRows, and it persists with the rest of the map.
2026-08-14 02:19:44 +02:00
peter 837fdcd408 feat(channels,datatable): unify sync-status into one controlled DataTable filter (R13 S2a)
The top sync chips and the Sync column's own select filter were two independent
states with different vocab (all/needs_full/fully_synced vs fully/needsFull) and
subtly different semantics. DataTable gains a controlled filters/onFiltersChange
pair (symmetric to the existing controlled sort); ChannelsProvider now owns the
filter map via a new useFilterState hook, so the top chips, the Sync column
funnel, the card layout, and the header "full history" deep-link all drive ONE
filters.sync. The card layout reuses the exported filterRows predicate instead of
its own status pre-filter. Vocab unified on the column's fully = recent_synced &&
backfill_done, which now matches the Sync cell's own "fully synced" badge.
2026-08-14 02:00:17 +02:00
peter 665bc6f7b5 fix(plex,channels): address R13 S2 review findings
- Shared usePlexClearFilters hook replaces the duplicated clearAll (was copied in
  PlexBrowse and PlexSidebar) — one "reset every Plex narrowing" both call.
- The Plex duration chip's minutes unit is localized (plex.filter.durationUnit:
  "min"/"perc") instead of a hardcoded "m".
- The Plex collection chip falls back to the "Collection" label, not the raw
  rating_key id, when collectionTitle is missing.
- Dropped the dead rowClassName that dimmed hidden rows in the subscribed table
  (hidden channels are excluded there now — they live in the Hidden tab).
- Renamed the misleading `set` null-check predicate to `has`.

Gate green: tsc/eslint-0/prettier/knip clean, vitest 106.
2026-08-14 01:34:48 +02:00
peter 5e7169e064 feat(plex): active-filter chip row above the grid (R13 S2)
Plex now shows the same at-a-glance "what's narrowing this" row the feed has,
instead of the narrowing being visible only inside the collapsed sidebar. New
usePlexActiveFilters is the Plex analogue of useActiveFilters — it turns the Plex
state (expanded filters + watch-state + sort) into the source-agnostic
ActiveFilter[] the SHARED ActiveFilterChips renders. One removable chip per
dimension (genres/ratings/people emit one each), plus Clear-all resetting filters
+ watch-state + sort. Labels reuse the existing plex.* i18n keys; no new strings.

Lives in components (not lib) since it reads the Plex provider — lib stays
provider-free like useActiveFilters. Gate green: tsc/eslint-0/prettier/knip.
Self-E2E'd: a sort=year chip renders, removing it resets sort to title and hides
the row.
2026-08-14 00:56:03 +02:00
peter 002ef0ab84 feat(channels): proper channel cards for Hidden + Blocked tabs (R13 S2)
The Hidden and Blocked tabs were a bare row of name chips ("two withered
chips"). Both now render a responsive grid of real channel cards via a shared
ExcludedChannelList: avatar + name + open-on-YouTube (reusing ChannelLink) and
one labelled reversing action (Unhide / Unblock). One presentation, parameterised
by icon/label/action/copy — the two "excluded channel" tabs now match.

Gate green: tsc/eslint-0/prettier/knip clean. Self-E2E'd: Hidden tab renders both
channels as cards with avatars, YT links, and Unhide buttons.
2026-08-14 00:49:09 +02:00
peter 6062007947 feat(channels): add a Hidden tab; move hidden channels out of Subscriptions (R13 S2)
Hidden subscriptions get their own tab (mirroring Blocked): a flat list with an
Unhide action, instead of dimmed rows scattered through the Subscriptions table.
- ChannelsView += "hidden"; the subscribed table now EXCLUDES hidden channels,
  so hiding a channel (the eye button) moves it to the Hidden tab and unhiding
  (Eye) returns it — both via the existing patch mutation.
- The "Hidden" top-level status chip is dropped (ChannelStatusFilter loses
  "hidden"); the status row is now purely sync-status (all/needs_full/fully_synced).
- Bilingual i18n (en/hu): channels.tabs.hidden + a channels.hidden block; the now
  unused channels.filters.hidden removed.

Hidden = out of the personal (Mine) feed only, still in the shared Library — a
softer exclusion than Blocked (hidden everywhere). Gate green: tsc/eslint-0/
prettier/vitest 106. Self-E2E'd: hide → appears in Hidden tab → unhide → back,
all /api 200.
2026-08-14 00:38:47 +02:00
peter ccd7170a20 fix(sort): harden shared sort primitives (R13 S1 review round 2)
- SortControl no longer renders a blank select when an always-sorted view
  (no defaultLabel) is handed a key its options don't offer: it falls back to
  the first option instead of value="" (which also tripped React's controlled
  warning). Robustness now lives in the primitive, not each caller.
- PLEX_SORT_FALLBACK is Object.frozen: it's handed out by reference from
  healSort/healPlexSort and stored as provider state, so freezing turns any
  accidental in-place SortState mutation into a loud error instead of silently
  corrupting the shared fallback.
- Test: SortControl falls back to the first option for an unavailable key.

Gate green: tsc/eslint(0 err)/prettier/knip clean, vitest 106.
2026-08-14 00:10:13 +02:00
peter 02c8c71c53 fix(sort): address R13 S1 review findings
- PlexSidebar activeCount now compares the HEALED sort, so an out-of-scope key
  (movie "duration" after switching to shows) no longer inflates the
  active-filter badge while the control + grid show the default.
- PlexProvider strips the legacy `sortDir` from the filters blob, so it stops
  leaking into the facets query key and being re-persisted on every change.
- PlexProvider sort migration guards the fall-through: valid JSON that isn't a
  SortState now falls back instead of using the raw JSON string as the key.
- New shared healPlexSort(sort, scope) is the single heal both PlexBrowse and
  PlexSidebar use (removes the noop-label plexSortOptions call + the duplication).
- Tests: useSortState.dom.test.ts (hydrate/persist/heal round-trip via renderHook)
  and SortControl.dom.test.tsx (options, pick, direction toggle, directionless,
  trailing). vitest 94 -> 105.

Gate green: tsc/eslint(0 err)/prettier/knip clean, vitest 105.
2026-08-14 00:00:31 +02:00
peter 45bd588af7 refactor(channels): share useSortState across the manager + discovery (R13 S1)
ChannelsProvider and ChannelDiscovery each hand-rolled the same sort plumbing
(useState<SortState> + parseSortState + a persist key). Both now use the shared
useSortState hook. hydrateSort gains an empty-options rule: with no vocabulary
passed it parses without healing (these owners heal for display via
ColumnSortControl against the live columns), so the un-healed provider store is
preserved. plexSortKeys made module-private (knip).

Closes R13 S1: feed, Plex, and the two channel tables all sort through one
SortControl + one SortState model. Gate green: tsc/eslint(0 err)/prettier/knip
clean, vitest 94.
2026-08-13 23:41:12 +02:00
peter e4bdc5ea6a refactor(plex): unify Plex sort onto SortControl + a single SortState (R13 S1)
Plex sort was a bespoke chip-pill row plus a separate asc/desc chip pair, with
the key in PlexState.sort and the direction hidden in PlexFilters.sortDir. Now:
- sort is one SortState {key,dir}; sortDir is dropped from PlexFilters (and from
  the facet-key destructure). The wire call still splits it into sort + sort_dir
  at the server boundary, so the backend is untouched.
- The chip sort-row is replaced by the shared SortControl (select + arrow) — the
  same idiom the feed and channel manager use. New lib/plexSort.ts owns the
  scope-dependent vocabulary (movies add duration) + the title/asc fallback.
- PlexProvider migrates the old storage in place: a legacy bare-key plexSort
  string plus the old filters.sortDir become the {key,dir} state, so an existing
  sort preference survives the upgrade.
- PlexBrowse and PlexSidebar heal the stored sort against the current scope's
  options, so a movie-only key (duration) left over after switching to shows
  can't ask the server for an unavailable sort.

Gate green: tsc clean, eslint 0 errors, prettier clean, vitest 93.
2026-08-13 23:36:34 +02:00
peter 440bdafa74 refactor(feed): drive the feed sort toolbar through the shared SortControl (R13 S1)
The feed's bespoke select + asc/desc button is now the generic SortControl,
fed a SortOption[] built from the feed vocabulary (relevance offered only with
a query; shuffle/relevance directionless). The feedSort wire codec
(parseSort/buildSort) stays as the server-boundary translation, and the
reshuffle button rides SortControl's `trailing` slot so shuffle keeps its
re-seed affordance. Behaviour-equivalent; ArrowUp/ArrowDown imports dropped.
2026-08-13 23:28:47 +02:00
peter 7d9da80795 refactor(sort): source-agnostic SortControl + SortOption model (R13 S1 foundation)
Introduce the one sort control every content surface will share:
- columnSort.ts gains the SortOption vocabulary + pure pickSort/healSort/
  hydrateSort helpers (direction lives in SortState, never in the key).
- SortControl.tsx: the generic dropdown + asc/desc toggle, with an optional
  "no sort" choice (tables) and a trailing adornment slot (the feed's reshuffle).
- ColumnSortControl is now a thin wrapper over SortControl (API unchanged, so
  Channels + ChannelDiscovery are untouched); optionsFromColumns moved to
  lib/sortOptions.ts to keep the component file fast-refresh clean.
- useSortState.ts: the per-account persist/hydrate/heal plumbing the channel
  manager and discovery each hand-rolled, ready to replace both.

Pure logic is fully unit-tested (columnSort.test.ts 10 -> 23). Gate green:
tsc/eslint/prettier clean, vitest 93 pass. Feed/Plex/channels migration follows.
2026-08-13 23:24:59 +02:00
peter f3b95ed543 release: 0.63.3 (r12-s5-player-frontend) 2026-08-13 03:45:24 +02:00
peter 8ff28dcc69 release: 0.63.3 — shared player volume core + PlexPlayerSettings extraction (R12 S5) 2026-08-13 03:45:13 +02:00
peter 92ef6116f0 fix(player): hold last-known Plex volume on a corrupt read, normalise once (review)
- applyVolume/setVol now pass the element's CURRENT level as the normalize
  fallback instead of undefined, so a corrupt persisted volume holds the
  playing level rather than jumping to PLEX_SCALE.default=1 (full) — the "full
  blast" failure mode playerVolume.ts's header explicitly warns against. (The
  old bare clamp muted instead; both are wrong for corrupt data.) readMerged
  shallow-merges stored prefs without per-field validation, so a corrupt
  volume is reachable.
- setVol normalises the level ONCE and both applies + persists that same value,
  so the <video> element and the stored pref can never disagree.
- Test documents the hold-last-known contract (corrupt raw + valid fallback ->
  fallback; only an unusable fallback lands on the scale default).

Gate green (tsc / eslint-0 / prettier / knip / vitest 83 / pytest 234).
2026-08-13 03:43:02 +02:00
peter cba992379c refactor(player): share the volume core across both players + extract PlexPlayerSettings (R12 S5 frontend)
Volume-consolidation (conservative — no behaviour or prefs-format change):
- playerVolume.ts is now scale-generic (a VolumeScale parameterises range /
  default / integer rounding). YT_SCALE (0-100 ints) and PLEX_SCALE (0..1
  float) bind the same tested core; the YouTube wrappers keep the original API
  so the existing tests are unchanged.
- The Plex usePlayerVolume hook drops its bespoke clamp01 arithmetic for the
  shared normalizeVolumeIn / stepVolumeIn at PLEX_SCALE. Side effect (a latent
  bug fix): a corrupt persisted volume now falls back to the default instead of
  a bare clamp silently muting it. The separate .muted flag stays Plex-only.
- clamp01 stays for the seek-bar fraction (its non-volume use).
- Tests: the generic core is covered at both scales (float no-round, corrupt ->
  default, step/clamp) — the Plex-side arithmetic finally has unit coverage.

PlexPlayerSettings extraction (verbatim JSX move, behaviour-equivalent):
- The ~240-line gear/tuning panel (sync / playback / subtitle / clock tabs)
  moves out of PlexPlayer into PlexPlayerSettings.tsx (a pure prefs+patchPrefs
  view owning no state). PlexPlayer shrinks 1402 -> 1173 lines; Slider/Toggle
  imports it no longer uses are dropped.

Gate green (tsc / eslint-0 / prettier / knip / vitest 83 / pytest 234). Dev
server smoke: app builds + loads clean, no console errors. The interactive
settings panel needs a playing Plex video (unreachable on localdev) -> owed to
the user's UAT.
2026-08-13 03:20:01 +02:00
peter f3fce67e4c release: 0.63.2 (r12-s5-backend-residue) 2026-08-13 02:48:38 +02:00
peter b5f889aa33 release: 0.63.2 — download queue fairness + backend residue cleanup (R12 S5) 2026-08-13 02:48:18 +02:00
peter e025e0f719 fix(downloads): floor max_concurrent + stable share order (R12 S5 review)
- Floor the effective max_concurrent at 1 on the read path (quota.resolve)
  and in the claim SQL (GREATEST(..., 1)). admin_set_quota already floors on
  write and the sysconfig default's min is 1, but a stray stored 0 (direct DB
  edit) would diverge the two "at limit" checks — the SQL claim-window
  exclusion can't see a user "capped" with zero running jobs, while
  at_concurrency_limit can — reopening the starvation the fairness fix closes.
  Keeping the effective limit >= 1 makes the two checks provably agree.
- shared_with_me now orders by (created_at DESC, id DESC) to match
  list_downloads, so the newly-added 500 cap has a stable boundary among
  same-timestamp shares instead of a non-deterministic one.
- 0062 upgrade loop: rename the unused loop var "table" to "_table"
  (matching the underscore-marked "_col").

Test: a stored max_concurrent=0 flooder no longer wedges the claim loop.
2026-08-13 02:42:41 +02:00
peter 2a33fc0d42 fix(downloads): worker claim fairness + bounded job listings (R12 S5)
#2 — _claim_job excluded users at their concurrency limit only in Python,
AFTER a fixed LIMIT-50 fetch window. One user flooding the queue with >50
queued jobs while at their cap filled the whole window, chosen came back None,
and every other user's work past position 50 was never seen. Exclude capped
users IN THE SQL (per-user max_concurrent from the quota row, else the
sysconfig default; unlimited users never capped), keeping the Python re-check
as a race guard that can only reject, never over-claim.

#3 — list_downloads and shared_with_me loaded every job for a user with no
bound. Cap both at 500 newest-first (a shared _MAX_LIST, matching the admin
listing's existing cap); the Download Center renders newest-first and 500 is
far beyond any real manual-download history, so no frontend change is needed.

Tests: DB-backed — a capped user's 60-job flood no longer starves a user
behind the window; oldest-by-queue_pos claim, all-capped->None, unlimited
never excluded; and the list cap bounds the result set.
2026-08-13 02:02:20 +02:00
peter 222edaf99e perf(db): drop 10 redundant single-column FK indexes (R12 S5)
Each column already leads a composite UNIQUE constraint whose btree covers
leftmost-prefix lookups on it (and the ON DELETE CASCADE reverse lookups),
so the extra single-column index=True was pure write-amplification. Drop
index=True on the 10 columns + alembic 0062 drops the existing indexes
(DROP INDEX IF EXISTS; downgrade recreates). Verified: migration up/down
round-trips on localdev (10 indexes drop, composites intact), siftlode
check green (alembic upgrade head incl. 0062 + pytest 193). The 10 do not
appear in an alembic-check model/DB diff → models and migration agree.
2026-08-13 01:55:14 +02:00
peter 01ded15bb4 docs(progress): document the intentional feed-vs-Plex finish-margin divergence
R12 S5 item: the feed's FINISH_MARGIN_SECONDS (10) and the Plex player's
_FINISH_MARGIN_S (30) were flagged as drifted duplicates. They are a
deliberate divergence, not drift — the feed clears a short video's resume
bar, Plex marks long-form items (movies/episodes with credits) watched.
Cross-reference + document both so it reads as intentional.
2026-08-13 01:55:14 +02:00
peter 7928d81116 release: 0.63.1 (improvement/source-adapters) 2026-08-13 01:21:50 +02:00
peter ee8aa72abe release: 0.63.1 — SourceAdapter refactor + sidecar/nfo fixes 2026-08-13 01:21:22 +02:00
peter 2cd02eb1af fix(downloads): netloc-scoped adapter host match (review round 2)
resolve_adapter matched sources by loose substring over the netloc, so a
look-alike host ('myyoutube.com', 'youtube.com.evil') misresolved. Match the
host exactly or as a subdomain via a _host_is() helper, and identify from the
netloc of BOTH source_ref and yt-dlp's webpage_url — the latter by host only,
never a substring over its path. This also restores a host signal to the
early-fill path (resolve_adapter(None, None, info)), which has no source_ref,
so a Facebook asset's uploader_url no longer depends on extractor_key being
present in the progress-hook info_dict.

Tests: look-alike hosts stay generic while real subdomains resolve; a
webpage_url netloc identifies the source when extractor_key is absent.
2026-08-13 01:18:25 +02:00
peter 883e8515b9 fix(downloads): tighten SourceAdapter resolution + neutral nfo default (review)
- resolve_adapter: identify from the yt-dlp extractor_key only, not the full
  webpage_url (a sibling source's name in a URL path no longer misresolves);
  add youtube-nocookie.com to the YouTube host match so nocookie embeds keep
  the POT/player-client extraction strategy.
- _download: re-resolve the adapter with the post-extraction info dict so the
  .nfo uniqueid type and uploader_url agree on one source (dedups the inline
  resolve too).
- MediaMeta.uniqueid_type defaults to source-neutral "generic" instead of
  "youtube", so an un-updated caller can never falsely label a non-YouTube
  download as youtube.
- Tests: lock extractor-key-only resolution + nocookie host; default nfo type.
2026-08-13 01:09:38 +02:00
peter d4dca46d9f fix(downloads): keep sidecar basenames intact when the title contains a dot
write_sidecars / delete_asset_files computed `base = media.with_suffix("")` and then
`base.with_suffix(".nfo")`. Stripping the real `.mp4` first leaves a stem whose own last dot may
be inside the title (e.g. "Sonka Házilag."), which `with_suffix` then treats as the extension and
clobbers — writing the .nfo/.jpg under a TRUNCATED basename that no longer matches the media, so
Plex/Kodi never associate them (and delete_asset_files targets the wrong name, leaking orphans).
Swap the extension on the FULL media path via a shared `_sidecar` helper instead.

Pre-existing bug (predates the source-adapters work); found during its E2E and folded in here.
Verified end-to-end: a dotted-title Facebook reel now writes a single shared basename.
2026-08-13 00:47:51 +02:00
peter 8e4dee1bb6 refactor(downloads): introduce SourceAdapter — a thin per-source Strategy over yt-dlp
Consolidates source-specific policy (youtube / facebook / reddit / generic) that was scattered
across worker.py, backfill.py, service.py, formats.py and storage.py with no shared source
identity. app/downloads/adapters.py holds the base (= generic yt-dlp behaviour) + per-source
overrides; resolve_adapter(source_kind, source_ref, info) picks one by source_kind, the yt-dlp
extractor_key, or the URL host, falling back to generic. yt-dlp stays the engine — the adapter is
policy only.

Moved into adapters (behaviour-equivalent):
- uploader_url derivation (YouTube channel_url / Facebook uploader_id / reddit None) — removes the
  worker/backfill duplication (build_uploader_url + _should_refetch gone).
- the player-client fallback sets + retriable markers + YouTube extractor_args/POT (build_ydl_opts
  now takes source-agnostic opts plus an `extra` dict from the adapter).
- source_url reconstruction (service.source_url delegates to the adapter).

Two intentional (approved) behaviour changes:
- the player-client fallback is now YouTube-only; non-YouTube sources make a single extraction
  attempt instead of a pointless double attempt with youtube-only client args.
- the .nfo <uniqueid type=...> follows the source (adapter.nfo_uniqueid_type) instead of the
  hardcoded "youtube" that mislabelled every url-source download.

Deliberately left generic (documented non-moves): title cleaning (normalize_title is harmless
across sources), the classify marker table, thumbnail_to_jpg, the format selectors, and og.py's
YouTube-hqdefault og:image preference (a self-contained thumbnail heuristic on the share path).

Verified end-to-end on localdev across all four adapters: downloads succeed and uploader_url + the
.nfo uniqueid type are correct per source.
2026-08-13 00:34:42 +02:00
peter 44efdf328d release: 0.63.0 (improvement/download-subprocess-watchdog) 2026-08-12 23:43:28 +02:00
peter 099d6d1301 chore: bump VERSION 0.63.0 + release notes (supervised download-subprocess watchdog) 2026-08-12 23:43:15 +02:00
peter cce862cde7 fix(downloads): supervisor edge-cases from review round 3
- worker.py: keep an absolute wall-clock backstop for ANY still-alive child. A child that
  reached a terminal state (done/error) but wedges in teardown is now killed to free the worker
  slot ("abandon"), with its DB status left intact; only a still-'running' over-cap job is failed.
  Previously gating the wall-clock on status=='running' let a wedged finisher loop forever (#1).
- worker.py: reconcile the claimed asset on ANY clean child exit (not only non-zero), so a child
  that exits without finalizing it — e.g. an early return on a job DELETEd mid-flight — no longer
  leaves the asset stuck 'downloading' until boot recovery (#2).
- worker.py: extract _stop_and_free + _fail_after_kill; the exception / stall / timeout / cancel /
  shutdown / abandon paths now share one kill+free (+fail) tail instead of three copies (#3).
- tests: a wedged-finished child is killed by the wall-clock without a status change; clean exits
  now assert the reconciliation no-op (#1, #4).
2026-08-12 23:38:37 +02:00
peter b42b340594 fix(downloads): don't let the supervisor kill a finishing child (review round 2)
- worker.py: the watchdog loop now treats only 'paused'/'cancelled' (or a vanished row) as a
  stop; a terminal status (done/error, or a re-queued contended asset) means the child already
  wrote its outcome and is exiting, so keep polling instead of killing it. Previously a
  proc.wait timeout landing between the child's terminal commit and its exit was misread as a
  route-cancel and SIGKILLed a successful download (dropping its completion notification). The
  stall/wall-clock/shutdown checks are now gated on status=='running' (#1, #5 test).
- worker.py: guard the crash-path _job_liveness probe so a DB blip there can't propagate (the
  child is already dead; boot recovery is the backstop) (#2).
- worker.py: drop _stop_child's redundant poll guard — escalate_kill no-ops on an exited proc
  and getpgid yields None for a reaped pid (#3).
- worker.py: note the extraction heartbeat is per-attempt (a single over-timeout attempt is
  still killed, bounded by socket_timeout) (#4).
2026-08-12 23:15:41 +02:00
peter 657224c1ad refactor(downloads): address /code-review on the download supervisor
- worker.py: wrap the watchdog loop so a failure in it (e.g. a transient DB error in
  _job_liveness) still hard-kills the child + fails the job, instead of leaving the child
  running unsupervised — the exact wedge the feature prevents (#1).
- worker.py: _touch_job heartbeat bumps updated_at at each extraction attempt, so a slow /
  multi-client-set fallback extraction (which writes no progress, and re-writes identical
  values so ORM onupdate does not fire) is not false-killed as "stalled" (#2).
- edit.py: extract escalate_kill (terminate->wait->kill escalation); _stop and worker's
  _stop_child (process-group variant) now share it (#3).
- worker.py: document the edit path's second, tighter run_ffmpeg watchdog and that the
  supervisor is POSIX/Linux-container-only (#4, #6).
- tests: a long download stays alive while progress is fresh, and a watchdog-loop error
  still stops the child (#1, #5).
2026-08-12 23:04:48 +02:00
peter 0ed82fe6b6 feat(downloads): run each download/edit in a supervised child process with a hard-kill watchdog
Defense-in-depth after the ReDoS incident. The heavy work (yt-dlp download / ffmpeg edit)
now runs in a child process (app.downloads.runner) supervised by the worker thread, not in
the thread itself. A thread can't be killed and a C-level spin can't even be cooperatively
cancelled, so one wedged download used to pin a core, hold the GIL and freeze the whole
queue; a child process can be SIGTERM/SIGKILLed.

The supervisor kills the child (and its process group, so ffmpeg grandchildren die too)
when: a route pauses/cancels the job, it makes no progress for download_stall_timeout_seconds
(150s), it runs past download_wall_clock_max_seconds (2h), the worker is shutting down, or
the child crashes without setting state. On a kill it frees the asset and fails the job with
a retryable network error (stalls are not auto-requeued). A clean exit is left alone — the
child already wrote the terminal state, including a cooperative pause via the progress hook.

- config.py: stall / wall-clock / supervisor-poll / kill-grace settings.
- worker.py: _run_supervised + _spawn_download_child / _job_liveness / _stop_child /
  _cleanup_staging; _process_job claims the asset, then hands the heavy work to the child.
- app/downloads/runner.py: the child entrypoint.
- docker-compose.localdev.yml: init:true on the worker so tini (PID 1) reaps orphaned
  ffmpeg grandchildren left by a hard-kill (the worker reaps its own direct children).
- tests: watchdog decision logic + SIGTERM->SIGKILL escalation (fake proc, no real subprocess).
2026-08-12 22:39:17 +02:00
peter e395fb816d release: 0.62.2 (bug/download-worker-redos-hotfix) 2026-08-12 18:50:02 +02:00
peter 4af919ac4d chore: bump VERSION 0.62.2 + release notes (download ReDoS + Reddit thumbnail hotfix) 2026-08-12 18:49:42 +02:00
peter fea6f55d4f refactor(downloads): address /code-review on the download hotfix
- edit.py: extract _run_ffmpeg_best_effort (one run+timeout+swallow policy shared by
  thumbnail_to_jpg / ensure_poster / ensure_storyboard) + a build_thumbnail_jpg_cmd
  builder, matching the module's other ffmpeg commands.
- thumbnail_to_jpg: copy an already-JPEG thumbnail (Facebook, and Reddit's jpeg-under-
  a-.png-name) instead of re-encoding it; only webp/png-proper are decoded via ffmpeg.
- errors.py: drop the redundant "account authentication" classify marker
  ("authentication is required" already covers the known Reddit message).
- formats.py: document that the opt-in EmbedThumbnail is unaffected by dropping the
  convertor (it converts unsupported thumbs to PNG, not the crashing mjpeg2jpeg path).
- test_titles.py: guard the ReDoS regression with a thread + timeout (deterministic
  "must not hang") instead of a wall-clock assertion.
2026-08-12 18:47:01 +02:00
peter 37cca723fc fix(downloads): stop a ReDoS title and a fatal thumbnail convert from wedging/killing downloads
Three prod-observed download bugs, root-caused from a frozen prod queue with the
worker pinned at 100% CPU:

- ReDoS in normalize_title: _TRAILING_HASHTAGS had two overlapping unbounded
  quantifiers around the required letter, so a Facebook title with a long hashtag
  cluster that is NOT at the end ("... #a #b #c | Page") backtracked catastrophically.
  It runs inside the yt-dlp progress hook holding the GIL, so one such download froze
  the whole worker (and could stall feed enrichment too - sync/videos.py calls it).
  Rewrite with a zero-width "has a letter" lookahead + a possessive tag body: one
  unambiguous split, nothing to backtrack. 40-tag stress case: ~0.1ms.

- Reddit download killed by a cosmetic thumbnail convert: yt-dlp's
  FFmpegThumbnailsConvertor runs "-bsf:v mjpeg2jpeg" on a JPEG-served-under-a-.png-name
  (Reddit) and fails; being a postprocessor, it failed the WHOLE download even though
  the media was fine. Drop it and produce the <base>.jpg poster/og:image ourselves via
  editmod.thumbnail_to_jpg (plain ffmpeg decode->mjpeg, best-effort -> ensure_poster
  cuts a frame on failure). The FB Messenger / MS Teams share preview keeps working
  (posters verified as real JPEGs).

- classify: map "authentication is required" (Reddit's auth wall) to login_required
  instead of unknown, so the UI explains it rather than offering a pointless retry.

Tests: ReDoS-timing + numeric-marker regression, the Reddit classify case, and
thumbnail_to_jpg (real JPEG out / None on bad input, never raises).
2026-08-12 18:27:35 +02:00
peter 6883fedc1b release: 0.62.1 (r12-plex-layering) 2026-08-12 02:01:04 +02:00
peter d9a12144dc chore: bump VERSION 0.62.1 + release notes (R12 S4a PlexPlayer refactor) 2026-08-12 01:58:48 +02:00
peter 4c3ae00492 refactor(plex): address /code-review on the S4a decomposition
Two behaviour-equivalent cleanups from the review:
- Drop the redundant per-render `audioRef.current = audioOrd` mirror in
  PlexPlayer: the only two setAudioOrd callers (usePlexTracks.changeAudio
  + the detail effect) already set audioRef synchronously, so the mirror
  never carried a value they hadn't written.
- Narrow the time-tracking effect deps from [id, patchPrefs] to [id];
  patchPrefs was a vestigial, unused dependency (eslint confirms [id] is
  the complete set).

The third finding (usePlayerVolume duplicating playerVolume.ts) is the
tracked R12 volume-consolidation sprint, not a review fix.
2026-08-12 01:52:33 +02:00
peter f86313eaa9 refactor(plex): extract PlexPlayer track selection into usePlexTracks
S4a-4a of the PlexPlayer decomposition. Move audio/subtitle track
selection + subtitle rendering into usePlexTracks: the selected-ordinal
state, change/cycle audio+subtitle, the debounced sub-offset, the audio
A/V-sync offset, and the two subtitle effects (force-showing the mounted
track + positioning cues at the user's subPos). The shared media refs
(video/hls/audio/aoff/abs/multiAudio) stay component-owned and are passed
in, since loadSession + the detail effect write audioRef too; absRef is
hoisted above the hook call for that. No behaviour change; verified by
self-E2E (subtitle on/off via 'S' mounts/removes the <track>, mode
'showing', VTT 804 cues all positioned to line=subPos, subLang persists;
playback uninterrupted). Audio switch is single-track here so cycleAudio
correctly no-ops.
2026-08-12 01:21:20 +02:00
peter 521d2aeaa8 refactor(plex): extract PlexPlayer auto-skip into useAutoSkip
S4a-3b of the PlexPlayer decomposition. Move the intro/credits auto-skip
machinery (activeMarker detection, the skipProgress countdown, doSkip,
cancelAutoSkip + the running-interval and reset-on-id effects) into
useAutoSkip. The hook returns skipProgressRef / cancelAutoSkipRef — the
forward-declaration bridge the earlier keyboard/back handlers read to
cancel a running countdown; those two stable refs are now listed in the
backIntent/keyboard dep arrays (they were provably-stable local refs
before, params now). No behaviour change; verified by self-E2E (credits
marker detected -> countdown fill renders + advances while playing ->
auto-skip fires -> item marked Watched + player closes).
2026-08-12 00:46:54 +02:00
peter 68f4834695 refactor(plex): extract PlexPlayer progress + volume hooks
S4a-3a of the PlexPlayer decomposition. Move two byte-equivalent
behaviours into lib/plexPlayerHooks.ts: usePlexProgress (the 10s
interval flush + pause/pagehide beacon + reopen cache-patch checkpoint;
absRef stays in the component since seek/skip/ended mutate it too) and
usePlayerVolume (HTML5 <video> 0..1 volume + independent mute, persisted
+ re-applied per session). usePlayerVolume is the seam the later R12
volume-consolidation sprint will rework to share playerVolume.ts's
tested pure core with the YouTube player. No behaviour change; verified
by self-E2E (volume-bar sets + persists video.volume, 'm' toggles mute
keeping the level, pause fires plexProgress 200).
2026-08-12 00:32:44 +02:00
peter c7a69596ad refactor(plex): extract PlexPlayer leaf hooks (clock/fullscreen/chrome)
S4a-2 of the PlexPlayer decomposition. Move three self-contained,
byte-equivalent behaviours into lib/plexPlayerHooks.ts: useWallClock
(15s top-bar clock), useFullscreen (wrapper fs toggle + reactive flag),
and useControlsVisibility (auto-hiding control bar + wake). The ref
mirrors the hooks read (barAutoHide/menuOpen/video-paused) stay in the
component and are passed in, so the timer sees fresh state. No behaviour
change; verified by self-E2E (real HLS playback: clock renders, controls
auto-hide after 3s while playing, wake restores + stays while paused).
2026-08-12 00:15:41 +02:00
peter a0e9912886 refactor(plex): extract PlexPlayer pure helpers + control primitives
S4a-1 of the PlexPlayer decomposition. Move the module-level pure
helpers (prefs types/defaults, track-language resolution, subtitle
shadow CSS, wall-clock formatting, HLS_SUB_LAG, fmt) to
lib/plexPlayerPrefs.ts and the presentational building blocks
(Ctrl/Slider/Toggle/PanelHead) to plexPlayerControls.tsx. No hooks or
behaviour touched; PlexPlayer.tsx 1889->1661.
2026-08-12 00:06:21 +02:00
peter dd2118a547 refactor: address /code-review — rename uploader_url() to build_uploader_url()
The R12 S3 underscore-publish left worker.uploader_url() sharing a name with the
`uploader_url` model attribute and the yt-dlp `"uploader_url"` info-dict key. The
function is now build_uploader_url() — the attribute, dict key, log line and
docstrings keep the plain `uploader_url` name. Rename touches only the def, its
two call sites, and the backfill import; no behaviour change (ruff clean,
pytest 193).
2026-08-11 23:50:49 +02:00
peter c74f69babc refactor(plex): dedupe show/season heroes into DetailHero + WatchedToggle
R12 S4b. PlexShowView and PlexSeasonView were copy-paste siblings — the same
glass hero frame, play/resume poster button, customize-menu slot, and the
mark-watched action button, duplicated verbatim. Two shared components now own
that structure:

- DetailHero: the hero shell (frame + poster button + customize slot + info
  column via children); the season page passes a smaller poster size.
- WatchedToggle: the mark-whole-show/season watched action (icon + label flip);
  the differing labels are props.

Single source of truth for the hero — a future hero change touches one place
instead of two. Gate green (tsc / eslint 0 errors / prettier / knip / vitest 79)
+ self-E2E: the show detail and season detail pages both render their hero,
metadata, actions and watched-toggle identically to before (screenshot-verified;
the stale [hmr] PlexBrowse errors in the console buffer are from an intermediate
edit — the fresh Vite server logs clean).
2026-08-11 23:38:57 +02:00
peter 69d34f7fa3 refactor(channels): move cells + column defs into channelColumns.tsx
R12 S4c. Channels.tsx shed its ~150-line column block and 3 cell components
into the co-located channelColumns.tsx:

- SyncBadge / SyncCell / ActionsCell move verbatim (prop-driven, self-contained).
- the column array becomes a buildChannelColumns(deps) factory next to
  subsColumn — the manager owns the state/mutations and passes them in; the
  inline patch.mutate wrappers become onHide/onDeep callbacks so the factory
  doesn't couple to the mutation type.

Channels.tsx 905 -> 628 lines; no behaviour change. Gate green (tsc / eslint 0
errors / prettier / knip / vitest 79) + self-E2E: the manager renders every
column and cell (priority stepper, actions, sync badges, tags) with no console
errors.
2026-08-11 23:27:16 +02:00
peter 51463b226e refactor: R12 S3 — invert scheduler->routes, publish cross-module privates
Layering cleanup, no behaviour change (ruff clean backend+scripts, pytest 193):

- Invert the scheduler->routes dependency: reset_demo_state (+ its demo-seed
  helpers) move from routes/admin.py to a new service module app/demo.py.
  Both routes/admin.py and scheduler.py now import it from app.demo, so the
  scheduler no longer reaches into the routes layer — its lazy import (the
  routes<->scheduler cycle workaround) is gone, hoisted to a top-level import.

- Publish the private symbols that were imported across module boundaries (the
  underscore convention was lying): feed._serialize -> serialize,
  feed._to_tsquery_str -> to_tsquery_str, videos._insert_stubs -> insert_stubs,
  videos._stub_from_playlist_item -> stub_from_playlist_item,
  worker._uploader_url -> uploader_url, plex._shared._item_or_404 ->
  item_or_404. Scoped renames — the 5 other route modules keep their own truly
  private _serialize helpers.

- _release_asset moves from routes/downloads.py to app/downloads/service.py as
  the public release_asset (asset bookkeeping is a service concern); the routes
  call it via service.release_asset.
2026-08-11 22:58:10 +02:00
peter ac6383e92c refactor(auth): address /code-review on the S2 split
Cleanup only, no behaviour change (ruff clean + pytest 193):
- keep the whole OAuth scope vocabulary together in app/auth.py: the bundles
  (BASE_SCOPES/READ_SCOPES/WRITE_SCOPES/_YOUTUBE_SCOPES) move back next to the
  atoms (READ_SCOPE/WRITE_SCOPE) as core; routes/auth.py imports the bundles it
  needs instead of rebuilding them from the atoms.
- compact the const groups the split generator spread out (the six rate
  limiters, the TTLs, the demo/scope constants) back into blocks.
2026-08-11 22:43:33 +02:00
peter 1b07d54663 refactor(auth): split app/auth.py — endpoints to app/routes/auth.py
R12 S2. app/auth.py (1154 lines) mixed the core auth surface with 18 HTTP
endpoints on one /auth router. Split by dependency graph:

- app/auth.py (223 lines) keeps ONLY the core surface other modules import
  (current_user, resolved_user_id, admin_user, require_*, has_*_scope,
  purge_user, count_admins, get_or_create_demo_user, google_enabled,
  is_allowed, operator_contact, optional_current_user) plus everything it
  transitively depends on.
- app/routes/auth.py (883 lines, new) gets the router + all 18 endpoints +
  their endpoint-only helpers, importing the core surface from app.auth.
- main.py includes auth_routes.router (was app.auth.router).
- test_oauth_redirect imports _oauth_redirect_uri from its new home.

The STAY/MOVE partition is driven by an AST scan of the COMPLETE external
surface (every `from app.auth import` incl. multiline + `from app import
auth; auth.X`), so no still-referenced symbol was moved. Behaviour-equivalent:
every symbol relocated verbatim (get_source_segment equality), route set
proven identical, no undefined names (symtable sweep).

Gate: ruff clean (backend + scripts), pytest 193 passed.
2026-08-11 22:01:14 +02:00
peter f484de1008 refactor(plex): address /code-review on the R12 S1 split
Cleanup only, no behaviour change (ruff clean + pytest 193):
- imports hoisted above `router =` and grouped/sorted (isort) across the split
  files; ruff-format pass restores the 2-blank-line PEP8 spacing the generator
  collapsed to 1.
- the shared on-disk cache root moves to a neutral app/plex/cache.py, so
  subtitles.py no longer imports it from imgcache.py (drops the cross-import).
- the former plex.py module docstring's access-model note (read endpoints open
  to any authed user; admin endpoints test/sync) is restored in the package
  __init__.py.
2026-08-11 21:46:43 +02:00
peter 99befa6e38 refactor(plex): split routes/plex.py into an admin/browse/playback package
R12 S1 step B. The 2011-line routes/plex.py becomes an app/routes/plex/
package:

- admin.py     — connection test, catalog sync, owner watch-link config (5)
- browse.py    — libraries, facets, collections, playlists, show/item, people (23)
- playback.py  — image/subtitle proxies, HLS streaming, watch-state writes (10)
- _shared.py   — the two genuinely cross-module helpers (_item_or_404, log)
- __init__.py  — assembles `router` from the three sub-routers; the app wiring
                 (plex_routes.router in main.py) is unchanged.

Mechanically generated and proven behaviour-equivalent: every function/class
source relocated verbatim (get_source_segment equality) and the (method, path)
route set is identical to before. Endpoints keep the /api/plex prefix on each
sub-router. No SQL change (db.query untouched; 2.0 select() is the next step).

Gate: ruff clean, pytest 193 passed (pure-logic + DB lane).
2026-08-11 21:30:01 +02:00
peter 4106bceb08 refactor(plex): extract img/subtitle/rollup helpers into app/plex services
R12 S1 step A. Move the pure, leaf-level helpers out of the 2227-line
routes/plex.py into the app/plex service package with public names:

- app/plex/imgcache.py — on-disk poster/art cache + proxy (was _img_cache_*,
  _image_key, _img_response, _IMG_CACHE/_IMG_SIZES). Owns the shared cache dir.
- app/plex/subtitles.py — WebVTT convert/time-shift + subtitle cache (was
  _srt_to_vtt, _shift_vtt, _ffmpeg_to_vtt, _sub_cache_*, _BITMAP_SUBS).
- app/plex/rollup.py — show watch-state aggregation (was _rollup,
  _show_status, _show_agg_subq, _show_state_map).

Behaviour-equivalent move: bodies verbatim, only the public API renamed and
call sites re-qualified. No endpoint or SQL change (db.query kept as-is; the
2.0 select() conversion is a later step). plex.py 2227 -> 2011 lines.

Gate: ruff clean, pytest 193 passed (pure-logic + DB lane).
2026-08-11 21:16:44 +02:00
peter 2671ab4cc8 release: 0.62.0 (r11-feedback-symmetry) 2026-08-11 20:41:43 +02:00
peter c4fde626a4 chore(release): 0.62.0 — R11 feedback symmetry
VERSION bump + user-facing release notes for R11 (Plex error toasts,
download/edit ready notifications, saved-view rollbacks + retries, access-request
denial emails).
2026-08-11 20:40:46 +02:00
peter 359bac149b refactor: address /code-review on R11 feedback symmetry
- admin._decide: add an idempotency guard (409 unless status was 'pending') so a
  repeat/stale approve OR deny can't re-run its side effects — now that deny emails,
  this stops duplicate denial mails and a rejection mail to an already-approved user.
- SavedViewsWidget: extract runViewMutation(fn, failMsg) (returns success) — dedups
  the five try/catch+refresh blocks AND lets create/rename close their input only on
  success, so a failed create keeps the user's typed name.
- worker._notify_download_done: take user_id/title captured BEFORE the caller's commit
  (no post-commit refresh SELECT); add a  so the inbox labels an edit job
  'Edit ready' instead of 'Download ready' (new inbox.editDone i18n, HU+EN).

Gate green (ruff/tsc/eslint-0/prettier/knip/vitest 79/pytest 193); smoke-tested the
saved-views + notifications surfaces render clean on a fresh dev server.
2026-08-11 20:25:18 +02:00
peter e2fe6fa286 feat(ux): R11 feedback symmetry — Plex/download/rollback/deny notices
Give equal-weight actions equal-weight feedback (R11).

S1 Plex error toasts: the watched-toggles (card / season mark-all / episode /
info-panel / show mark-all) used to fail silently on 403/404/429; each now surfaces
the server reason via a shared notifyActionError() + resyncs the optimistic flip.
The playback auto-mark-watched stays intentionally silent (a mid-play toast is
noise; the next sync corrects it). New generic lib/actionErrors.ts (Plex + saved
views reuse it; notifyYouTubeActionError delegates its non-403 case to it).

S2 download-done inbox notification: worker.py emits a best-effort type=download_done
notice to the job owner on real completion (regular + edit jobs, not instant
cache-hits), rendered/localized in the inbox with an Open-Downloads link.

S3 optimistic rollbacks: SavedViewsWidget create/rename/toggle-default/remove/reorder
and the inbox revert-state no longer swallow failures — they toast + refetch to
resync (the reorder's optimistic order reverts to server truth).

S4 access-request deny: deny_invite now emails the requester (send_access_denied),
mirroring approve — no more silent limbo.
2026-08-11 05:05:50 +02:00
peter 354b3dd10c release: 0.61.1 (r10-btn-primary) 2026-08-11 04:24:22 +02:00
peter b5cf355ae4 chore(release): 0.61.1 — .btn-{tone} action-button recipe family
VERSION bump + chores release note for the R10 .btn-primary/.btn-danger/.btn-warn
recipe unification (refactor; a few primary buttons gained the standard hover).
2026-08-11 04:23:26 +02:00
peter fc186e1230 refactor(ui): address /code-review on .btn-primary — tone family + doc fixes
- Generalize the recipe into a .btn-{tone} family: add .btn-danger / .btn-warn
  siblings sharing one hover-dim, and migrate the two hand-rolled tone-action
  buttons (ConfirmProvider danger confirm, Banner warning action) onto them, so
  primary/danger/warn action paints can't drift.
- Reword the recipe comment: it owns paint + one standard hover-dim (not
  'PAINT-ONLY' — that mislabel obscured the hover behaviour).
- Drop the now-redundant hover:opacity-90 from Banner's action-button base (each
  tone's recipe supplies the hover).

The 3 previously hover-less primary buttons keep the standard hover-dim gained in
the prior commit (intentional consistency; now accurately documented). Gate green
(tsc/eslint-0/prettier/knip/vitest 79); self-E2E confirmed btn-danger/btn-warn
compute to their tone/tone-fg tokens.
2026-08-11 04:20:10 +02:00
peter 76f8de258c refactor(ui): R10 — .btn-primary recipe for the primary/action buttons
Add a paint-only .btn-primary recipe (solid accent bg + accent-fg text +
hover:opacity dim) in index.css @layer components, alongside the selection-paint
recipes (.pill/.chip/.is-current). Migrate ~38 primary/action buttons across 24
files off the hand-rolled 'bg-accent text-accent-fg hover:opacity-90', keeping
each site's geometry (padding/radius/font/gap/disabled/shadow).

Deliberately NOT migrated: accent BADGES/labels (spans), the chat message bubble,
selection toggles (tableHeader — R8 popover territory), and the round play FAB
(hover:scale, not the opacity-dim primary idiom). Three previously hover-less
primary buttons (SettingsPanel/SetupWizard/DraftSaveBar) gain the standard hover
dim for consistency.

Gate green (tsc/eslint-0/prettier/knip/vitest 79); self-E2E confirmed a migrated
button computes to accent/accent-fg and both recipe rules load.
2026-08-11 04:04:06 +02:00
peter 87da6155e1 release: 0.61.0 (fixpack-cards-nav-plex) 2026-08-11 03:37:57 +02:00
peter 3b41b68d22 chore(release): 0.61.0 — cards/nav/plex fixpack
VERSION bump + user-facing release notes for the 7-item UX batch (channel-page
priority stepper, cross-season Plex binge, unified card hover, slim icon nav,
scoped filter peek, full-viewport Plex backdrop, player end-of-queue exit).
2026-08-11 03:36:42 +02:00
peter 02754f479f refactor(ux): address /code-review high findings on the fixpack
- Extract the optimistic + debounced-flush feed-priority flow into a shared
  useChannelPriority hook (mirrors useChannelTagActions); the manager and the
  channel-detail page now share it instead of two drifting copies.
- NavSidebar: drop the always-true `slim` const and its dead expanded-rail
  branches; refresh the now-stale pinned/collapsible comments.
- Remove the orphaned `navCollapsed` localStorage key (no readers/writers left
  after the always-slim rail).
- Channel page: drop the wrapper native `title` that duplicated PriorityStepper's
  own Tooltip.

The keyboard-focus nav-label finding needed no change: the popover platform
already wires useFocus (visibleOnly), so a real Tab press shows the flyout label
(verified in-browser).
2026-08-11 03:29:08 +02:00
peter 71bbf35f5a fix(ux): dock filter tab flush to the nav + reliable full-viewport plex backdrop
Follow-ups on the 7-item batch:
- The slim nav slot was still w-20 while its card is w-14, leaving a ~24px dead
  gap the filter tab/bar couldn't dock against. Slot is now w-14 (exactly the
  card width), so the filter's flat-left tab sits flush on the rail's right edge
  (measured gap 0), both collapsed and expanded.
- The Plex art backdrop was invisible: a position:fixed layer at z-index -1 is
  painted BEHIND the body's opaque background here, so nothing showed. Paint the
  art on the app root (#root) instead — it's exactly the viewport and a real
  ancestor of the transparent nav/header/content, so the art fills the whole page
  edge-to-edge and the glass chrome floats over it (verified in-browser).
2026-08-11 02:53:55 +02:00
peter 9d7a2a75d8 feat(ux): card unification, slim icon nav, filter/plex fixes (7-item batch)
1A ChannelPage "Full history" button no longer wraps (whitespace-nowrap).
1B Extract the manager's press-and-hold priority stepper into a shared
   PriorityStepper and add it to the channel detail header (subscribed, non-demo);
   backend channel_detail now returns the per-subscription `priority`.
2  Unify YouTube + Plex media cards onto one look/behaviour: drop the whole-card
   hover LIFT (VideoCard) — which also fixes the clipped-top bug — and instead the
   inner thumbnail/poster ZOOMS on hover (both); give the flat Plex posters the same
   glass-card 3D depth + accent hover border as the feed cards (shadow-md is a
   transparent no-op here, so it's glass-card that provides the depth); clamp Plex
   card titles to a fixed 2-line height so 1- vs 2-line titles align.
3  Nav rail is now always a slim icon-only column, flush to the left edge; each icon
   reveals a floating name label on hover (Tooltip gains `always` + `openDelay`, and
   a right side). Removed the pin/unpin toggle, the expand-on-hover mode and the
   navCollapsed pref plumbing.
4  Scope the side-panel peek-open HOVER to the tab handle + the card (not the whole
   full-height slot next to the rail), so approaching the nav no longer pops the
   filter open; focus stays container-wide. useHoverFocusWithin gains a hover-out
   grace period + split hover/focus handler groups.
5  Plex art backdrop fills the whole viewport (a fixed inset-0 layer behind the
   transparent app) instead of only the inset <main> box.
6  Plex season binge continues across seasons: the season view's play queue now
   spans the whole show (allKeys), so a season's last episode rolls into the next
   season's first (seasonKeys stays for "add season to playlist").
7  When a Plex video/episode ends with nothing queued next, the player returns to
   the previous page (onEnded → onClose) instead of stranding a black screen.
2026-08-11 02:20:52 +02:00
peter e5d5735b9a release: 0.60.10 (r10-selected-paint) 2026-08-11 00:50:24 +02:00
peter a9b53bbc9d chore(release): 0.60.10 — VERSION bump + release note (R10 S4 selection-paint unification) 2026-08-11 00:48:26 +02:00
peter cb168c800a fix(ui): R10 S4 review r3 — preserve glass-card edge on pills, ease transform
Two low-severity follow-ups from the third /code-review round:
- .pill's resting border-color is now scoped `.pill:not(.glass-card)`, so a
  glass-card-resting pill (PlexSidebar scope/facet, PlexPlaylistView layout)
  keeps glass-card's own border incl. its lighter top-edge specular highlight,
  instead of having it flattened to a uniform var(--border).
- add `transform` to the recipe transition so the `active:translate-y-px` press
  nudge on the Sidebar tag/date pills eases again (it lost the utility transition).

Verified plain pills keep their resting border (Feed + YOUR TAGS) and active
pills still paint solid accent.
2026-08-11 00:45:14 +02:00
peter 90b33fc325 refactor(ui): R10 S4 — complete the selection-paint sweep (all pill idioms)
The first S4 pass migrated a subset, which left same-idiom selection pills
hand-rolled and made adjacent controls inconsistent (a /code-review finding).
This completes the sweep so every selected-among-options control shares the
recipe:

- plain pills → .pill (+ aria-pressed): Channels status filters + view tabs,
  Sidebar filter-tag chips + date presets + custom-dates, Stats tabs + ranges.
- glass-card-resting pills → .pill with the resting `glass-card glass-hover` kept
  in the non-pressed branch (solid accent when pressed): PlexSidebar library
  scope + the facet Chip (retired the round-1 "deferred" note — the recipe
  already expresses this via the resting branch, no new variant needed).
- custom-dates aria-pressed is coerced with !! so a date string can't leak into
  the attribute and miss the [aria-pressed="true"] paint.
- drop the dead `border` on ChannelTags' chip-solid (accent border on accent bg).

Out of scope (a separate concern, NOT selection-among-options): primary/action
buttons that share the accent fill — Banner action, ConfirmProvider primary,
PlexBrowse primary, PlayerModal watched-toggle — these want a future .btn-primary
recipe. Left by-design: tableHeader filter options (PopoverItem + keepSelectedPaint,
owned by the popover platform).
2026-08-11 00:27:04 +02:00
peter 7ff1ec1db0 fix(ui): R10 S4 review — guard .is-current against aria-current="false" + shadow transition
Follow-ups from the /code-review pass:
- .is-current[aria-current] now excludes aria-current="false" — the recipe is a
  shared primitive and a consumer using the common `x ? "true" : "false"` idiom
  would otherwise paint a non-current row as current.
- add box-shadow to the .pill/.is-current transition so the Tabs/SettingsPanel
  on-state shadow fades in instead of popping (it lost the utility `transition`).
- document that .chip-solid is intentionally paint-only (composes on a site's own
  geometry) so a truncating single-element chip isn't forced into inline-flex.

The 3 intentional visual changes (hover-border 55%, tag-filter transparent
resting, RemovableChip tint+border) were UAT-accepted as-is. PlexSidebar's facet
Chip (glass-card resting) deferred — wants a .pill soft-glass variant.
2026-08-10 23:25:21 +02:00
peter dcb91066f5 refactor(ui): R10 S4 — unify selected-paint into .pill/.is-current/.chip recipes
The ~14 hand-rolled "this is the selected/current/active one" idioms each
repeated the same `sel ? "bg-accent text-accent-fg border-accent" : "…"` inline
ternary, which had drifted. Three attribute-driven recipes in @layer components
now own that paint, keyed on the ARIA the control already carries: .pill
(aria-pressed segments/toggles), .is-current (aria-current nav/list items), and
.chip / .chip-solid (tag & filter tokens). The JSX just toggles the ARIA; the CSS
paints, so one control's selected look can't diverge from the next's. No
focus-visible in the recipes — the @layer base rule already rings every button/a.

Migrated 15 sites (Feed show/content, Channels tag filter, Sidebar or-and +
scope, PlexSidebar match, Tabs, SettingsPanel tab-rail, PlexPlaylistView layout
[soft], NavSidebar nav + settings, ActiveFilterChips, PlexSidebar RemovableChip,
ChannelTags, ChannelPicker). Deliberately left: PlexSidebar facet Chip and
PlexPlaylistView already covered; SavedViews row (paint split across two
elements), the scheme swatch (border-only on a coloured swatch), and
ChannelPicker's option rows (text-only selection) — noted for a follow-up.
2026-08-09 21:06:58 +02:00
peter 13120163a4 release: 0.60.9 (improvement/popover-selected-highlight) 2026-08-09 17:47:38 +02:00
peter ead617fd31 chore(release): 0.60.9 — VERSION bump + public release note (popover weak-ring) 2026-08-09 17:47:05 +02:00
peter 5c5c045ed9 fix(popover): R10 weak-ring — checked rows get the full highlight fill
A hovered/roving CHECKED row got only a thin ring (R9 split), reading as a weak
afterthought next to an unchecked row's vivid accent fill. Now selected rows get the
SAME bg-accent/35 fill + a STRONGER ring (ring-2 accent/80 vs the unselected ring-1
accent/60); the child checkmark/checkbox stays legible on the tint. The one idiom that
DID rely on the R9 ring-only — the table-filter options, whose selected item paints its
own solid bg-accent ROW — opts back in via a new `keepSelectedPaint` PopoverItem prop,
so its solid paint isn't downgraded to a 35% tint. Scopes the R9 exception to the one
menu that needs it instead of penalising every checkbox menu.
2026-08-09 17:46:13 +02:00
peter eee7443e6f release: 0.60.8 (r10-recipe-consolidation) 2026-08-09 17:45:02 +02:00
peter 84046a2874 chore(release): 0.60.8 — VERSION bump + public release note (R10 S3) 2026-08-09 17:44:29 +02:00
peter 45c95442b6 refactor(input): R10 S3 review — fold Welcome's local inputCls into the shared recipe
Welcome kept the last local `const inputCls` (rounded-xl, py-2.5) after S3b consolidated
the other three, leaving its auth fields the sole inputs off the unified rounded-lg recipe.
Import the shared inputCls and drop the local const — now every input is truly one recipe.
(Auth fields go rounded-xl→rounded-lg, py-2.5→py-2 to match.)
2026-08-09 17:43:43 +02:00
peter 5a26499f1d refactor(header): R10 S3c — SyncStatus adopts the shared header-pill shell
The header row's three pills now share one shell: SyncStatus was glass/rounded-xl while
ModuleName and SearchBar were glass-menu/rounded-2xl/h-11. Aligned it (glass-menu, rounded-2xl,
h-11) so the pill row reads as one system, per user call.
2026-08-09 17:43:43 +02:00
peter 7ea93925fb fix(ui): R10 S3c — Banner action buttons gain hover feedback
The accent + warning Banner action buttons had no hover state (flagged during S1 UAT).
Add the house hover:opacity-90 idiom. (The header-pill .glass-chrome consolidation is
deferred — 2 of the 3 pills already share the glass-menu/rounded-2xl look; unifying the
odd one, SyncStatus, needs a visual call on whether it becomes a full-height pill.)
2026-08-09 17:43:43 +02:00
peter 0559e1a503 refactor(input): R10 S3b — one input recipe, ring focus everywhere
Unify text inputs onto the form.tsx inputCls look (surface bg, rounded-lg, 2px accent
RING focus), replacing the settings-family border-focus style app-wide:
- deleted 3 duplicate local `const inputCls` (KeyGate/SettingsPanel/SetupWizard) → import
  the shared one; a 4th (Welcome, py-2.5) kept local but re-styled to match.
- ~30 inline settings inputs: default full-size fields → inputCls; size variants (pager/
  table-filter/chat textarea/plex/glass-card) keep their width/radius/padding but swap
  bg-card→bg-surface and border-focus→ring-focus. glass-card/bg-transparent bgs left as-is.
- refreshed the form.tsx doc-comment (the old two-style split is gone).

The ring is the more visible / a11y-correct focus cue, matching R9's focus foundation.
2026-08-09 17:43:42 +02:00
peter 1955fc87ee refactor(glass): R10 S3a — glass-card notification rows + perf-aware .overlay-backdrop
- NotificationsPanel per-row .glass -> glass-card (the rows are cards, not panel surface).
- New .overlay-backdrop recipe (4px blur = backdrop-blur-xs) with a data-perf override that
  DROPS the blur. The viewport-sized modal scrim blur (Modal/PlayerModal/Welcome) was a raw
  backdrop-blur-xs utility that survived perf mode while every .glass had its blur dropped —
  the most expensive blur in the app. Overlays keep their bg-black/NN.
- gotcha: write ONLY the standard backdrop-filter; a manual -webkit- line makes lightningcss
  drop the standard prop (webkit-only output -> no blur in Firefox).
2026-08-09 17:43:42 +02:00
peter 69077f9476 release: 0.60.7 (improvement/glass-over-image-tune) 2026-08-09 17:42:23 +02:00
peter d93c465176 style(glass): tune the over-image tier — cards let the image show through
The app-wide 'glass over image' tier: panels stay near-opaque (surface 90%, menu 100%)
so chrome text is readable, but glass-card drops to 50% so the per-scheme backdrop image
gently shows through the feed cards (blur 22, inset 10). User-tuned via the live GlassTuner
and validated ("the image just barely shows through, text still readable"); supersedes the
uniform-88% pass that had killed the image-through-cards look. Only the data-backdrop=on
tier — :root (image off) and .glass-media (Plex art) unchanged.
2026-08-09 06:07:15 +02:00
572 changed files with 50663 additions and 12812 deletions
+3
View File
@@ -4,3 +4,6 @@
#
# Prettier one-time repo-wide format pass (R2 S1b, 2026-07-21)
51c36c2071728d2cd7ae580b7bfdf21f6867a347
# ruff format one-time backend + scripts pass (line-length 100, 2026-08-23)
b1832afd180866b922521148603db447020c57a4
+71
View File
@@ -0,0 +1,71 @@
# Server-side mirror of the local `siftlode check` gate (E1 S3 / DX-1), so `main` is always green
# independent of any one machine. Two parallel jobs match the two local lanes:
# frontend — tsc (3 projects) + eslint + prettier --check + knip + vitest
# backend — ruff + the full pytest suite (pure-logic + the DB-backed lane) against a real Postgres
#
# Runner: the lab act_runner (LXC 115) mounts the Docker socket, so service containers work. The backend
# runs pytest NATIVELY against a `postgres` service (not the local docker-compose.test.yml, whose
# bind-mount is unreliable under a mounted-socket runner); fidelity is kept by installing the SAME pinned
# requirements, Postgres 16, and ffmpeg, and by setting SIFTLODE_TEST_REQUIRE_DB=1 so a DB/ffmpeg that
# failed to come up is a hard error, never a silent skip. node 22 matches the prod build image
# (Dockerfile `node:22-alpine`).
name: gate
on: [push, pull_request]
jobs:
frontend:
runs-on: ubuntu-latest
container: node:22-bookworm
defaults:
run:
working-directory: frontend
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm run typecheck
- run: npm run lint
- run: npm run format:check
- run: npm run knip
- run: npm run test
backend:
runs-on: ubuntu-latest
# A combined python+node image: python 3.12 (matches the prod Dockerfile) for the tests, and node
# so the JS actions (checkout) run. actions/setup-python is NOT usable here — the runner can't reach
# GitHub's python-versions download, so it must be baked into the image.
container: nikolaik/python-nodejs:python3.12-nodejs20
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: siftlode
POSTGRES_PASSWORD: siftlode
POSTGRES_DB: siftlode_test
options: >-
--health-cmd "pg_isready -U siftlode -d siftlode_test"
--health-interval 2s
--health-timeout 3s
--health-retries 30
env:
DATABASE_URL: postgresql+psycopg://siftlode:siftlode@postgres:5432/siftlode_test
SIFTLODE_TEST_REQUIRE_DB: "1"
steps:
- uses: actions/checkout@v4
# ffmpeg for the poster/thumbnail tests; postgresql-client for the readiness wait below.
- run: apt-get update && apt-get install -y --no-install-recommends ffmpeg postgresql-client
- name: Wait for Postgres
run: until pg_isready -h postgres -U siftlode -d siftlode_test; do sleep 1; done
- run: pip install --no-cache-dir -r backend/requirements.lock -r backend/requirements-dev.txt
- name: ruff
run: python -m ruff check --config backend/ruff.toml backend scripts
- name: ruff format --check
run: python -m ruff format --check --config backend/ruff.toml backend scripts
- name: pytest
run: python -m pytest
working-directory: backend
# Dependency CVE audit over the pinned lock. Non-blocking for now (informational) — a real CVE
# shows up in the run without failing the gate; tighten to a hard fail once the noise is known.
- name: pip-audit (advisory)
run: pip install --no-cache-dir pip-audit && pip-audit -r backend/requirements.lock
continue-on-error: true
+105
View File
@@ -0,0 +1,105 @@
# CLAUDE.md — working agreement for AI agents on Siftlode
This repository is **fully AI-authored** — every line was written by an AI agent, and it continues to
be developed that way. This file is the rules of the road: read it before you touch anything. It is
loaded into every agent session, so keep it **short and true**. A rule here that has gone stale is
worse than no rule — if you change how the code works, change this file in the same commit.
Siftlode is a self-hosted YouTube reader (+ a local Plex browser/player and a download center). It is a
**modular monolith**: one Postgres, one FastAPI process (uvicorn + APScheduler), one worker process
(yt-dlp/ffmpeg), one PO-token sidecar. React 18 + react-query frontend with a router-free navigation
provider.
## The gate — never merge or ship red
The local gate is **ruff (`check` + `format --check`) + typecheck (3 tsc projects) + eslint + prettier
--check + knip + vitest + pytest**. It runs on every `publish`, which **refuses to build on failure — there is deliberately no
`--skip` flag.** Run it and get it green **before** any merge, release, or handing work to review.
- Full gate (agents with the private ops dispatcher): `siftlode check`.
- Directly, without the dispatcher: `cd frontend && npm run typecheck && npm run lint && npm run knip && npm run test`, plus backend `ruff check backend` + `ruff format --check backend` and `pytest` (the DB-lane needs Postgres).
- New DB-backed test? It must actually run, not silently skip — assume `SIFTLODE_TEST_REQUIRE_DB=1`.
## Workflow
- **Trunk model.** One long-lived branch, `main`. Branch off it (`feature/`, `bug/`, `improvement/`),
keep commits logical and Conventional-Commit-styled.
- **No AI attribution in commits.** No `Co-Authored-By` trailer, no "generated by" lines. (This file
being AI-written is the intended, visible exception; commits are not.)
- **Release: user-owned in timing, agent-run in execution.** The user decides *when* to ship; on their
go-ahead *you* run `siftlode release` from the feature branch (merge to `main` → publish → deploy →
verify `/api/version`). Never self-ship — bring work to a green, committed state and stop for the
go-ahead first, then run it yourself.
- **Public repo.** Keep ops/topology/secrets out — those live in a separate private repo. Nothing here
should assume or leak them.
## Collaboration — how to work with the human
The human is the product owner: they bring ideas and requirements. You are the technical lead — you own
how it gets built. Concretely:
- **Plan before you build.** For anything non-trivial, present a short plan (sprint-level for larger
work) and get a go-ahead before writing code.
- **Offer choices, mark the optimal — and explain the trade-off.** When several solutions are viable,
list them and mark the recommended one. This is the *one* place to spend extra words: give the *why*
behind each option and the weight of the decision. These decision points are where the product owner
engages with the build and learns from it, so make them understandable — a reasoned menu, not a bare one.
- **Mockups for design/UX decisions.** For any visual or interaction choice, show mockup options to
choose from before building.
- **Challenge unsound requests.** If a request is technically ill-advised and you can back it with real
arguments, push back or propose an alternative — don't comply blindly.
- **Verify UI in the browser before calling it done.** tsc/unit/DOM tests miss visual and interaction
bugs; do a real visual check of any UI change.
- **Be terse everywhere else.** Outside those decision points, lead with the answer or decision. During
implementation and debugging especially: no restating the same point reworded, no filler.
## Architecture shape (don't fight it)
`routes/*` (HTTP only: validation, auth-dependency, serialization) → domain packages `sync/`, `plex/`,
`downloads/`, `youtube/` (shared by routes + scheduler + worker) → flat core (`auth`, `quota`,
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `realtime`, `ratelimit`, `progress`,
`security`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
Frontend: `main.tsx` provider tree → `App.tsx` shell → `moduleRegistry` route-chunks → components; data
via react-query (`qk` factory), per-account `localStorage`, and server prefs; API through `lib/api/*`
slices over `core.req<T>`.
**Not wanted** (reviewed and rejected as over-engineering for this size): microservices, event bus,
CQRS, a repository/DI framework, a router library, a generic CRUD-router factory or base serializer.
## Canonical patterns — when you have a choice, pick THIS
| Area | Use | Not |
|---|---|---|
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
| Request body | a Pydantic model | `payload: dict` |
| Ownership check | `owned_or_404` / `get_or_404` | ad-hoc `_own_playlist`/`_owned`/`_own_job` |
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`) | ad-hoc `{"unsubscribed"}`/`{"cleared": n}`/echo |
| Transaction commit | the handler/job-wrapper commits; helpers `flush` | helpers that commit (only `quota` owns its session) |
| Admin knob storage | `sysconfig` (values) | env / `app_state` / `scheduler_settings` scatter |
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
| FE per-account state | `usePref` | raw `readAccount`/`writeAccount` or bespoke hooks |
| FE control | the chip/segmented primitives (`Select` only for >6 options) | a native `<select>` |
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
## House rules that bite (learned from real bugs)
- **A rule stated in a comment must also be enforced in code.** The worst bugs here came from a comment
and the code/migration drifting apart (a `cancelled`≠`canceled` literal killed a cancel path). Put the
invariant in an enum/constant/shared function, not in prose.
- **Fix a principle everywhere, in one sweep.** If you correct one owner/commit/serialization site,
`grep` for the siblings and fix them too — asymmetric fixes are a recurring smell here.
- **Honest states.** An error must not render as an empty state; a swallowed failure must not look like
"no results". Reads degrade honestly; writes are conditional (`UPDATE … WHERE status='running'`).
- **i18n parity.** HU/EN, zero key drift. Every user-facing string is translated.
- **Comment the "why", keep it current.** This codebase uses comments as its architecture doc; a stale
"why" is a liability, so update it when the behavior changes.
---
*Run-it-locally and the full gate commands live in [`docs/development.md`](docs/development.md); backend
code conventions in [`backend/app/README.md`](backend/app/README.md). This file stays the source of
truth for the canonical-patterns table and the gate/workflow/collaboration rules above.*
+26 -7
View File
@@ -10,7 +10,7 @@ ARG GIT_SHA=unknown
ARG BUILD_DATE=
# Stage 1: build the frontend SPA
FROM node:20-alpine AS frontend
FROM node:22-alpine AS frontend
WORKDIR /fe
# Deps first, keyed on the lockfile, so editing app source never re-resolves node_modules.
COPY frontend/package.json frontend/package-lock.json ./
@@ -45,21 +45,40 @@ RUN adduser --disabled-password --gecos "" appuser \
&& mkdir -p /downloads \
&& chown appuser /app /downloads
# ffmpeg: merge separate video+audio streams + postprocessing. Deno: the JavaScript runtime
# yt-dlp uses to solve YouTube's "n" signature challenge for the web player clients (required
# alongside the PO token, else only storyboards are offered). yt-dlp auto-detects deno on PATH.
# ffmpeg: merge separate video+audio streams + postprocessing, and the Plex VAAPI transcode (E6 S1).
# intel-media-va-driver (iHD): the VA driver ffmpeg's h264_vaapi needs — libva2 ships in the base but
# no driver, so without this `vainfo`/`h264_vaapi` fail and transcode silently falls back to libx264.
# Harmless on hosts with no /dev/dri (the driver just goes unused). Deno: the JavaScript runtime yt-dlp
# uses to solve YouTube's "n" signature challenge for the web player clients (required alongside the PO
# token, else only storyboards are offered). yt-dlp auto-detects deno on PATH.
ARG DENO_VERSION=v2.9.1
RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg unzip curl ca-certificates \
&& apt-get install -y --no-install-recommends ffmpeg intel-media-va-driver unzip curl ca-certificates \
&& curl -fsSL "https://github.com/denoland/deno/releases/download/${DENO_VERSION}/deno-x86_64-unknown-linux-gnu.zip" -o /tmp/deno.zip \
&& unzip -o /tmp/deno.zip -d /usr/local/bin \
&& chmod +x /usr/local/bin/deno \
&& rm /tmp/deno.zip \
&& rm -rf /var/lib/apt/lists/*
COPY backend/requirements.txt .
# Intel NEO OpenCL runtime (intel-opencl-icd) — needed by the Plex transcode's HDR/10-bit HW path, which
# tone-maps HDR→SDR on the GPU via `tonemap_opencl` (Gen 9.5 / UHD 630 can't tonemap in the VAAPI VPP, so
# OpenCL is the only HW route; without this the 10-bit branch drops to CPU libx264). NEO was REMOVED from
# Debian trixie (this base = python:3.12-slim = trixie), so it's pulled from **bookworm** — the 22.43 there
# still supports Gen 9.5 and its .deb runs on trixie (verified live on the prod UHD 630). The bookworm apt
# source is added only for this one install and removed after, so it can't drift the rest of the image.
# Harmless on hosts with no /dev/dri or no OpenCL use (the runtime just sits unused).
RUN printf 'Types: deb\nURIs: http://deb.debian.org/debian\nSuites: bookworm\nComponents: main\nSigned-By: /usr/share/keyrings/debian-archive-keyring.pgp\n' > /etc/apt/sources.list.d/bookworm.sources \
&& apt-get update \
&& apt-get install -y --no-install-recommends -t bookworm intel-opencl-icd \
&& rm /etc/apt/sources.list.d/bookworm.sources \
&& rm -rf /var/lib/apt/lists/*
# Install from the PINNED lock (requirements.lock, generated from requirements.txt) so the image is
# byte-reproducible and the exact versions are the ones pip-audit scanned in CI — not whatever floats
# up from the floor ranges on build day. Regenerate the lock when requirements.txt changes (see its header).
COPY backend/requirements.lock .
RUN --mount=type=cache,target=/root/.cache/pip \
pip install --upgrade pip && pip install -r requirements.txt
pip install --upgrade pip && pip install -r requirements.lock
COPY --chown=appuser backend/ .
# Read at runtime by config.py as the app_version (a committed single source of truth).
+1 -1
View File
@@ -1 +1 @@
0.60.6
0.78.6
+10 -2
View File
@@ -8,5 +8,13 @@
# docker run --rm -v "<repo>/backend:/app" -w /app siftlode-test:local python -m pytest
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt requirements-dev.txt ./
RUN pip install --no-cache-dir -r requirements.txt -r requirements-dev.txt
# ffmpeg so the thumbnail/poster tests (needs_ffmpeg) actually run in the gate instead of skipping
# silently — the runtime image already ships ffmpeg, so the test image should too (see conftest's
# SIFTLODE_TEST_REQUIRE_DB, which now also refuses to run if ffmpeg is missing).
RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg \
&& rm -rf /var/lib/apt/lists/*
# Install the SAME pinned lock as the prod image (+ the test-only tooling), so the tests run against
# the exact dependency versions prod ships.
COPY requirements.lock requirements-dev.txt ./
RUN pip install --no-cache-dir -r requirements.lock -r requirements-dev.txt
+1
View File
@@ -4,6 +4,7 @@ Revision ID: 0001_initial
Revises:
Create Date: 2026-06-11
"""
from typing import Sequence, Union
from alembic import op
@@ -4,6 +4,7 @@ Revision ID: 0002_ingest_core
Revises: 0001_initial
Create Date: 2026-06-11
"""
from typing import Sequence, Union
from alembic import op
+1
View File
@@ -4,6 +4,7 @@ Revision ID: 0003_tags
Revises: 0002_ingest_core
Create Date: 2026-06-11
"""
from typing import Sequence, Union
from alembic import op
@@ -4,6 +4,7 @@ Revision ID: 0004_feed_state_prefs
Revises: 0003_tags
Create Date: 2026-06-11
"""
from typing import Sequence, Union
from alembic import op
@@ -4,6 +4,7 @@ Revision ID: 0005_shorts_probed
Revises: 0004_feed_state_prefs
Create Date: 2026-06-11
"""
from typing import Sequence, Union
from alembic import op
@@ -4,6 +4,7 @@ Revision ID: 0006_app_state
Revises: 0005_shorts_probed
Create Date: 2026-06-11
"""
from typing import Sequence, Union
from alembic import op
@@ -4,6 +4,7 @@ Revision ID: 0007_deep_requested
Revises: 0006_app_state
Create Date: 2026-06-11
"""
from typing import Sequence, Union
from alembic import op
+2 -3
View File
@@ -4,6 +4,7 @@ Revision ID: 0008_invites
Revises: 0007_deep_requested
Create Date: 2026-06-12
"""
from typing import Sequence, Union
from alembic import op
@@ -22,9 +23,7 @@ def upgrade() -> None:
"invites",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("email", sa.String(length=320), nullable=False),
sa.Column(
"status", sa.String(length=16), nullable=False, server_default="pending"
),
sa.Column("status", sa.String(length=16), nullable=False, server_default="pending"),
sa.Column("note", sa.Text(), nullable=True),
sa.Column(
"requested_at",
@@ -4,6 +4,7 @@ Revision ID: 0009_quota_events
Revises: 0008_invites
Create Date: 2026-06-12
"""
from typing import Sequence, Union
from alembic import op
@@ -4,6 +4,7 @@ Revision ID: 0010_watch_progress
Revises: 0009_quota_events
Create Date: 2026-06-14
"""
from typing import Sequence, Union
from alembic import op
+4 -9
View File
@@ -4,6 +4,7 @@ Revision ID: 0011_playlists
Revises: 0010_watch_progress
Create Date: 2026-06-15
"""
from typing import Sequence, Union
from alembic import op
@@ -26,16 +27,10 @@ def upgrade() -> None:
nullable=False,
),
sa.Column("name", sa.String(length=255), nullable=False),
sa.Column(
"kind", sa.String(length=16), nullable=False, server_default="user"
),
sa.Column(
"source", sa.String(length=16), nullable=False, server_default="local"
),
sa.Column("kind", sa.String(length=16), nullable=False, server_default="user"),
sa.Column("source", sa.String(length=16), nullable=False, server_default="local"),
sa.Column("yt_playlist_id", sa.String(length=64), nullable=True),
sa.Column(
"dirty", sa.Boolean(), nullable=False, server_default="false"
),
sa.Column("dirty", sa.Boolean(), nullable=False, server_default="false"),
sa.Column("position", sa.Integer(), nullable=False, server_default="0"),
sa.Column(
"created_at",
@@ -7,6 +7,7 @@ Create Date: 2026-06-15
Migrates the old per-video status='saved' into membership of a built-in 'watch_later'
playlist (one per user who had saved videos), then demotes those states to 'new'.
"""
from typing import Sequence, Union
from alembic import op
@@ -11,6 +11,7 @@ a YouTube read. Backfills the baseline for already-clean linked playlists (the c
state == the synced state for them); dirty ones are left NULL so they stay dirty until the
next real sync establishes a baseline.
"""
import hashlib
from typing import Sequence, Union
@@ -32,24 +33,18 @@ def _fingerprint(name: str, ids: list[str]) -> str:
def upgrade() -> None:
op.add_column(
"playlists", sa.Column("synced_fingerprint", sa.String(), nullable=True)
)
op.add_column("playlists", sa.Column("synced_fingerprint", sa.String(), nullable=True))
bind = op.get_bind()
# Backfill clean, YouTube-linked playlists: their current state IS the synced state.
rows = bind.execute(
sa.text(
"SELECT id, name FROM playlists "
"WHERE yt_playlist_id IS NOT NULL AND dirty = false"
)
sa.text("SELECT id, name FROM playlists WHERE yt_playlist_id IS NOT NULL AND dirty = false")
).fetchall()
for pid, name in rows:
ids = [
r[0]
for r in bind.execute(
sa.text(
"SELECT video_id FROM playlist_items "
"WHERE playlist_id = :pid ORDER BY position"
"SELECT video_id FROM playlist_items WHERE playlist_id = :pid ORDER BY position"
),
{"pid": pid},
).fetchall()
@@ -11,6 +11,7 @@ Adds the shared demo-account plumbing:
The demo user row itself is created lazily on first demo login (works across all three DBs
without a data migration), so this only adds the schema.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -47,9 +48,7 @@ def upgrade() -> None:
nullable=False,
),
)
op.create_index(
"ix_demo_whitelist_email", "demo_whitelist", ["email"], unique=True
)
op.create_index("ix_demo_whitelist_email", "demo_whitelist", ["email"], unique=True)
def downgrade() -> None:
@@ -7,6 +7,7 @@ Create Date: 2026-06-16
Adds scheduler_settings: a per-job run-interval override (minutes), editable from the admin
Scheduler dashboard. Absent row = use the env/config default, so this is purely additive.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -10,6 +10,7 @@ the validation lifecycle (last_checked_at for rolling re-validation; unavailable
unavailable_reason to flag a video as currently unplayable — hidden from the feed
immediately, hard-deleted after a grace period). Purely additive; all columns nullable.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -26,7 +27,9 @@ def upgrade() -> None:
op.add_column("videos", sa.Column("privacy_status", sa.String(length=16), nullable=True))
op.add_column("videos", sa.Column("upload_status", sa.String(length=16), nullable=True))
op.add_column("videos", sa.Column("last_checked_at", sa.DateTime(timezone=True), nullable=True))
op.add_column("videos", sa.Column("unavailable_since", sa.DateTime(timezone=True), nullable=True))
op.add_column(
"videos", sa.Column("unavailable_since", sa.DateTime(timezone=True), nullable=True)
)
op.add_column("videos", sa.Column("unavailable_reason", sa.String(length=24), nullable=True))
op.create_index("ix_videos_last_checked_at", "videos", ["last_checked_at"])
op.create_index("ix_videos_unavailable_since", "videos", ["unavailable_since"])
@@ -9,6 +9,7 @@ JSON payload column is named `data` (SQLAlchemy's declarative Base reserves `met
First producer is the maintenance job's batched "videos removed" notice. Coexists with the
existing client-side transient bell.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -36,18 +37,14 @@ def upgrade() -> None:
sa.Column("data", sa.JSON(), nullable=True),
sa.Column("read", sa.Boolean(), nullable=False, server_default="false"),
sa.Column("dismissed", sa.Boolean(), nullable=False, server_default="false"),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now()
),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
)
op.create_index("ix_notifications_user_id", "notifications", ["user_id"])
op.create_index("ix_notifications_type", "notifications", ["type"])
op.create_index("ix_notifications_read", "notifications", ["read"])
op.create_index("ix_notifications_created_at", "notifications", ["created_at"])
# Fast unread-badge count + inbox listing per user.
op.create_index(
"ix_notifications_user_read", "notifications", ["user_id", "read"]
)
op.create_index("ix_notifications_user_read", "notifications", ["user_id", "read"])
def downgrade() -> None:
@@ -8,6 +8,7 @@ Adds app_state.maintenance_revalidate_batch: an admin-tunable override (from the
dashboard) for how many videos the maintenance job re-validates per run. NULL = use the
env/config default. Purely additive.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -9,6 +9,7 @@ channels.list?mine=true and cached. Used to exclude the user's own channel from
playlist-discovery list (you don't subscribe to yourself). NULL = not resolved yet.
Purely additive.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -10,6 +10,7 @@ stored historical values to the canonical scheme defined in app.quota.QuotaActio
fully reversible. (Scheduler job ids and progress phase labels are a separate namespace and
are NOT touched.)
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -8,6 +8,7 @@ Adds the system_config table: admin overrides for operational config that otherw
from env/config defaults (see app.sysconfig for the registry of known keys). Secret values
are stored Fernet-encrypted (encrypted flag). Purely additive; absent rows = use defaults.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -9,6 +9,7 @@ google_sub nullable (password-only accounts have no Google sub until they link),
single-use auth_tokens table for email verification + password reset. Existing rows are all
Google/demo accounts → backfilled email_verified=true (is_active defaults true).
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -8,6 +8,7 @@ Adds users.is_suspended — an admin-controlled access block, distinct from is_a
approval lifecycle). A suspended account can't sign in (password or Google) and any live
session is rejected; the user is told why and given the operator's contact.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -9,6 +9,7 @@ email_verified=false (the callback didn't set it). Backfill those so the flag re
matters because password-login gates on email_verified, so a Google user who later adds a password
must count as verified. Demo/password-only rows (google_sub IS NULL) are left untouched.
"""
from typing import Sequence, Union
from alembic import op
@@ -8,6 +8,7 @@ Adds app_state.configured + setup_token_hash for the first-run install wizard. E
(any users present) are marked configured so they never see the wizard; a genuinely fresh DB (no
users) starts unconfigured and boots into setup mode.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -8,6 +8,7 @@ Adds the `messages` table for direct user-to-user messaging (notification module
A conversation is the set of rows between two users; both FKs cascade so a GDPR account
erasure removes that user's messages.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -12,6 +12,7 @@ hold each user's public key + passphrase-wrapped private key (the server can nev
Any pre-existing rows were plaintext with no keys, so they're dropped (the feature is unshipped;
prod has none).
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -11,6 +11,7 @@ the Library by default:
- channels.from_search — this channel exists only because search surfaced one of its videos
Both default false (every existing row is organic), indexed for the feed's exclude filter.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -25,16 +26,12 @@ depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"videos",
sa.Column(
"via_search", sa.Boolean(), nullable=False, server_default="false"
),
sa.Column("via_search", sa.Boolean(), nullable=False, server_default="false"),
)
op.create_index("ix_videos_via_search", "videos", ["via_search"])
op.add_column(
"channels",
sa.Column(
"from_search", sa.Boolean(), nullable=False, server_default="false"
),
sa.Column("from_search", sa.Boolean(), nullable=False, server_default="false"),
)
op.create_index("ix_channels_from_search", "channels", ["from_search"])
@@ -10,6 +10,7 @@ query wraps both sides in unaccent(); this migration just makes the function ava
`unaccent` ships with the official postgres image (postgresql-contrib). The DB role is a
superuser in our deployments, so CREATE EXTENSION succeeds; IF NOT EXISTS keeps it idempotent.
"""
from typing import Sequence, Union
from alembic import op
@@ -9,6 +9,7 @@ live YouTube search. This lets "videos I searched for" appear in that user's Min
Mine Source filter), independent of the global `videos.via_search` flag (which marks a video as
search-discovered by anyone, for the shared Library's Source filter).
"""
from typing import Sequence, Union
import sqlalchemy as sa
+2 -3
View File
@@ -15,6 +15,7 @@ search stays accent-insensitive ("tiesto" matches "Tiësto"). A GIN expression i
query uses the exact same expression so the planner picks the index. `unaccent` is already
enabled (migration 0029). The config-based 2-arg to_tsvector is IMMUTABLE, so it's index-safe.
"""
from typing import Sequence, Union
from alembic import op
@@ -26,9 +27,7 @@ depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.execute(
"CREATE TEXT SEARCH CONFIGURATION public.unaccent_simple (COPY = pg_catalog.simple)"
)
op.execute("CREATE TEXT SEARCH CONFIGURATION public.unaccent_simple (COPY = pg_catalog.simple)")
op.execute(
"""
ALTER TEXT SEARCH CONFIGURATION public.unaccent_simple
@@ -21,6 +21,7 @@ title-only expression index from 0031 is dropped (superseded). Existing rows get
description indexed immediately; keywords/search_terms fill in as videos are (re-)enriched and
searched. Uses the `unaccent_simple` config from 0031 (accent-insensitive).
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -48,9 +49,7 @@ def upgrade() -> None:
f"ALTER TABLE videos ADD COLUMN search_vector tsvector "
f"GENERATED ALWAYS AS ({_SEARCH_VECTOR_EXPR}) STORED"
)
op.execute(
"CREATE INDEX ix_videos_search_vector ON videos USING gin (search_vector)"
)
op.execute("CREATE INDEX ix_videos_search_vector ON videos USING gin (search_vector)")
def downgrade() -> None:
@@ -17,6 +17,7 @@ un-subscribed channels):
access to its ephemeral videos and drives the grace-period purge of un-kept channels.
All new flags default false / NULL, so every existing row stays organic.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -30,9 +31,7 @@ depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column("channels", sa.Column("total_view_count", sa.BigInteger(), nullable=True))
op.add_column(
"channels", sa.Column("published_at", sa.DateTime(timezone=True), nullable=True)
)
op.add_column("channels", sa.Column("published_at", sa.DateTime(timezone=True), nullable=True))
op.add_column("channels", sa.Column("banner_url", sa.String(length=1024), nullable=True))
op.add_column("channels", sa.Column("external_links", sa.JSON(), nullable=True))
op.add_column(
@@ -8,6 +8,7 @@ Adds the `blocked_channels` table: a per-user blocklist. Videos from a blocked c
never shown in that user's live YouTube search (and not ingested), and are hidden from their
feed / explore views. Per-user, so one user's block doesn't affect anyone else.
"""
from typing import Sequence, Union
import sqlalchemy as sa
+2 -3
View File
@@ -8,6 +8,7 @@ Adds the `saved_views` table: per-user named snapshots of the feed's filter/sort
row stores the serialized FeedFilters (JSON), a sidebar `position` for drag-reorder, and an
`is_default` flag (at most one per user, enforced by the API) for the view applied on load.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -26,9 +27,7 @@ def upgrade() -> None:
sa.Column("user_id", sa.Integer(), nullable=False),
sa.Column("name", sa.String(length=80), nullable=False),
sa.Column("filters", sa.JSON(), nullable=False),
sa.Column(
"position", sa.Integer(), server_default=sa.text("0"), nullable=False
),
sa.Column("position", sa.Integer(), server_default=sa.text("0"), nullable=False),
sa.Column(
"is_default",
sa.Boolean(),
@@ -10,6 +10,7 @@ FOR UPDATE SKIP LOCKED, live progress written back for polling); `download_profi
reusable format presets (builtins seeded in the next migration); `download_shares` grant a file
to another user; `download_quotas` hold admin-set per-user limits (default via sysconfig).
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -47,9 +48,16 @@ def upgrade() -> None:
sa.Column("ref_count", sa.Integer(), server_default="0", nullable=False),
sa.Column("last_access_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("source_kind", "source_ref", "format_sig", name="uq_media_asset_cache_key"),
sa.UniqueConstraint(
"source_kind", "source_ref", "format_sig", name="uq_media_asset_cache_key"
),
)
op.create_index("ix_media_assets_status", "media_assets", ["status"])
@@ -61,7 +69,12 @@ def upgrade() -> None:
sa.Column("spec", sa.JSON(), nullable=False),
sa.Column("is_builtin", sa.Boolean(), server_default="false", nullable=False),
sa.Column("sort_order", sa.Integer(), server_default="0", nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.ForeignKeyConstraint(["user_id"], ["users.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
@@ -84,8 +97,18 @@ def upgrade() -> None:
sa.Column("phase", sa.String(length=32), nullable=True),
sa.Column("error", sa.Text(), nullable=True),
sa.Column("queue_pos", sa.Integer(), server_default="0", nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.ForeignKeyConstraint(["user_id"], ["users.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["asset_id"], ["media_assets.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["profile_id"], ["download_profiles.id"], ondelete="SET NULL"),
@@ -102,7 +125,12 @@ def upgrade() -> None:
sa.Column("job_id", sa.Integer(), nullable=False),
sa.Column("from_user_id", sa.Integer(), nullable=False),
sa.Column("to_user_id", sa.Integer(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.ForeignKeyConstraint(["job_id"], ["download_jobs.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["from_user_id"], ["users.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["to_user_id"], ["users.id"], ondelete="CASCADE"),
@@ -120,7 +148,12 @@ def upgrade() -> None:
sa.Column("max_concurrent", sa.Integer(), nullable=False),
sa.Column("max_jobs", sa.Integer(), nullable=False),
sa.Column("unlimited", sa.Boolean(), server_default="false", nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.ForeignKeyConstraint(["user_id"], ["users.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("user_id"),
)
@@ -10,6 +10,7 @@ definition consumed by app.downloads.formats: mode ("av" audio+video / "v" video
the embed_*/sponsorblock postprocessor flags. Names are English (the UI default); the frontend
localizes builtin labels by a stable key derived from the name.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -7,6 +7,7 @@ Create Date: 2026-07-03
The download GC warns each holder once, a grace window before an asset's TTL expiry; this flag
records that so subsequent GC runs don't re-notify until the asset is deleted.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -9,6 +9,7 @@ display-friendly form (emoji stripped, ALL-CAPS de-shouted, trailing SEO hashtag
see app.titles). Reversible: original_title preserves the source, and title can be re-derived.
The generated search_vector column regenerates automatically as each title is updated.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -45,5 +46,7 @@ def upgrade() -> None:
def downgrade() -> None:
conn = op.get_bind()
conn.execute(sa.text("UPDATE videos SET title = original_title WHERE original_title IS NOT NULL"))
conn.execute(
sa.text("UPDATE videos SET title = original_title WHERE original_title IS NOT NULL")
)
op.drop_column("videos", "original_title")
@@ -11,6 +11,7 @@ rows so both fresh installs (0037 then this) and existing DBs converge on the sa
jobs are unaffected (they snapshot their spec; profile_id just SET NULLs if it pointed at a
replaced row).
"""
from typing import Sequence, Union
import sqlalchemy as sa
+6 -15
View File
@@ -10,6 +10,7 @@ still hangs off a `media_assets` row (with `source_kind='edit'` and a per-user c
existing file-serve / ref-count / GC / quota machinery is reused unchanged. `source_job_id` /
`source_asset_id` point at the parent download; `edit_spec` is the trim/crop recipe.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -24,16 +25,10 @@ depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"download_jobs",
sa.Column(
"job_kind", sa.String(length=16), server_default="download", nullable=False
),
)
op.add_column(
"download_jobs", sa.Column("source_job_id", sa.Integer(), nullable=True)
)
op.add_column(
"download_jobs", sa.Column("source_asset_id", sa.Integer(), nullable=True)
sa.Column("job_kind", sa.String(length=16), server_default="download", nullable=False),
)
op.add_column("download_jobs", sa.Column("source_job_id", sa.Integer(), nullable=True))
op.add_column("download_jobs", sa.Column("source_asset_id", sa.Integer(), nullable=True))
op.add_column("download_jobs", sa.Column("edit_spec", sa.JSON(), nullable=True))
op.create_foreign_key(
"fk_download_jobs_source_job",
@@ -54,12 +49,8 @@ def upgrade() -> None:
def downgrade() -> None:
op.drop_constraint(
"fk_download_jobs_source_asset", "download_jobs", type_="foreignkey"
)
op.drop_constraint(
"fk_download_jobs_source_job", "download_jobs", type_="foreignkey"
)
op.drop_constraint("fk_download_jobs_source_asset", "download_jobs", type_="foreignkey")
op.drop_constraint("fk_download_jobs_source_job", "download_jobs", type_="foreignkey")
op.drop_column("download_jobs", "edit_spec")
op.drop_column("download_jobs", "source_asset_id")
op.drop_column("download_jobs", "source_job_id")
@@ -9,6 +9,7 @@ can watch the file on the public `/watch/{token}` player page — no account nee
per-link controls: an expiry, an "allow download" toggle, and an (argon2-hashed) password. Revoke
= delete the row. Distinct from `download_shares` (an ACL grant to another *registered* user).
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -31,7 +32,12 @@ def upgrade() -> None:
sa.Column("password_hash", sa.Text(), nullable=True),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("view_count", sa.Integer(), server_default="0", nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.ForeignKeyConstraint(["job_id"], ["download_jobs.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["created_by"], ["users.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
@@ -8,6 +8,7 @@ Stores yt-dlp's canonical `webpage_url` for a download's source, so the Download
clean "downloaded from" reference (copyable + openable). Nullable — filled by the worker on
extraction; existing/queued rows fall back to a URL derived from source_kind+source_ref.
"""
from typing import Sequence, Union
import sqlalchemy as sa
+67 -16
View File
@@ -18,6 +18,7 @@ Plex server (playback is from the local physical file; Plex is metadata only). T
The generated `search_vector` columns + their GIN indexes are added via raw SQL (same approach
as migration 0032), reusing the `unaccent_simple` text-search config from migration 0031.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -52,8 +53,12 @@ def upgrade() -> None:
sa.Column("kind", sa.String(length=16), nullable=False),
sa.Column("enabled", sa.Boolean(), server_default="true", nullable=False),
sa.Column("synced_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
op.create_index("ix_plex_libraries_plex_key", "plex_libraries", ["plex_key"], unique=True)
@@ -61,7 +66,12 @@ def upgrade() -> None:
"plex_shows",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("rating_key", sa.String(length=32), nullable=False),
sa.Column("library_id", sa.Integer(), sa.ForeignKey("plex_libraries.id", ondelete="CASCADE"), nullable=False),
sa.Column(
"library_id",
sa.Integer(),
sa.ForeignKey("plex_libraries.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("title", sa.String(length=512), nullable=False),
sa.Column("summary", sa.Text(), nullable=True),
sa.Column("year", sa.Integer(), nullable=True),
@@ -69,8 +79,12 @@ def upgrade() -> None:
sa.Column("art_key", sa.String(length=512), nullable=True),
sa.Column("child_count", sa.Integer(), nullable=True),
sa.Column("added_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
op.create_index("ix_plex_shows_rating_key", "plex_shows", ["rating_key"], unique=True)
op.create_index("ix_plex_shows_library_id", "plex_shows", ["library_id"])
@@ -81,12 +95,21 @@ def upgrade() -> None:
"plex_seasons",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("rating_key", sa.String(length=32), nullable=False),
sa.Column("show_id", sa.Integer(), sa.ForeignKey("plex_shows.id", ondelete="CASCADE"), nullable=False),
sa.Column(
"show_id",
sa.Integer(),
sa.ForeignKey("plex_shows.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("title", sa.String(length=255), nullable=True),
sa.Column("season_number", sa.Integer(), nullable=True),
sa.Column("thumb_key", sa.String(length=512), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
op.create_index("ix_plex_seasons_rating_key", "plex_seasons", ["rating_key"], unique=True)
op.create_index("ix_plex_seasons_show_id", "plex_seasons", ["show_id"])
@@ -96,10 +119,25 @@ def upgrade() -> None:
"plex_items",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("rating_key", sa.String(length=32), nullable=False),
sa.Column("library_id", sa.Integer(), sa.ForeignKey("plex_libraries.id", ondelete="CASCADE"), nullable=False),
sa.Column(
"library_id",
sa.Integer(),
sa.ForeignKey("plex_libraries.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("kind", sa.String(length=16), nullable=False),
sa.Column("show_id", sa.Integer(), sa.ForeignKey("plex_shows.id", ondelete="CASCADE"), nullable=True),
sa.Column("season_id", sa.Integer(), sa.ForeignKey("plex_seasons.id", ondelete="CASCADE"), nullable=True),
sa.Column(
"show_id",
sa.Integer(),
sa.ForeignKey("plex_shows.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column(
"season_id",
sa.Integer(),
sa.ForeignKey("plex_seasons.id", ondelete="CASCADE"),
nullable=True,
),
sa.Column("season_number", sa.Integer(), nullable=True),
sa.Column("episode_number", sa.Integer(), nullable=True),
sa.Column("title", sa.String(length=512), nullable=False),
@@ -119,8 +157,12 @@ def upgrade() -> None:
sa.Column("probed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("markers", sa.JSON(), nullable=True),
sa.Column("added_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
op.create_index("ix_plex_items_rating_key", "plex_items", ["rating_key"], unique=True)
op.create_index("ix_plex_items_library_id", "plex_items", ["library_id"])
@@ -137,14 +179,23 @@ def upgrade() -> None:
op.create_table(
"plex_states",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("user_id", sa.Integer(), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False),
sa.Column("item_id", sa.Integer(), sa.ForeignKey("plex_items.id", ondelete="CASCADE"), nullable=False),
sa.Column(
"user_id", sa.Integer(), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False
),
sa.Column(
"item_id",
sa.Integer(),
sa.ForeignKey("plex_items.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("status", sa.String(length=12), server_default="new", nullable=False),
sa.Column("position_seconds", sa.Integer(), server_default="0", nullable=False),
sa.Column("watched_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("progress_updated_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("synced_to_plex", sa.Boolean(), server_default="false", nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.UniqueConstraint("user_id", "item_id", name="uq_plex_user_item"),
)
op.create_index("ix_plex_states_user_id", "plex_states", ["user_id"])
@@ -9,6 +9,7 @@ Plex section listing (no per-item API calls): rating (audienceRating ~ IMDb), co
release date, and GIN-indexed genre / director / cast arrays. These also double as the content
signals a future watch-habit recommender would use.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -33,9 +34,13 @@ def upgrade() -> None:
op.create_index("ix_plex_items_rating", "plex_items", ["rating"])
op.create_index("ix_plex_items_content_rating", "plex_items", ["content_rating"])
op.create_index("ix_plex_items_studio", "plex_items", ["studio"])
op.create_index("ix_plex_items_originally_available_at", "plex_items", ["originally_available_at"])
op.create_index(
"ix_plex_items_originally_available_at", "plex_items", ["originally_available_at"]
)
op.create_index("ix_plex_items_genres_gin", "plex_items", ["genres"], postgresql_using="gin")
op.create_index("ix_plex_items_directors_gin", "plex_items", ["directors"], postgresql_using="gin")
op.create_index(
"ix_plex_items_directors_gin", "plex_items", ["directors"], postgresql_using="gin"
)
op.create_index("ix_plex_items_cast_gin", "plex_items", ["cast_names"], postgresql_using="gin")
@@ -10,6 +10,7 @@ finds titles by an actor/director name and ranks those above films that merely m
their synopsis. A generated column's expression can't be altered in place, so we drop + recreate it
(and its GIN index). `people_text` is populated on the next Plex sync.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -10,6 +10,7 @@ A `plex_collections` table holds each collection's card metadata; membership is
combinable filter and a title's "collection strip" is a pure local lookup. `editable` marks the
(curated) collections we may later write back to Plex; everything is read-only by default.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -27,7 +28,12 @@ def upgrade() -> None:
"plex_collections",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("rating_key", sa.String(length=32), nullable=False),
sa.Column("library_id", sa.Integer(), sa.ForeignKey("plex_libraries.id", ondelete="CASCADE"), nullable=False),
sa.Column(
"library_id",
sa.Integer(),
sa.ForeignKey("plex_libraries.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("title", sa.String(length=512), nullable=False),
sa.Column("summary", sa.Text(), nullable=True),
sa.Column("thumb_key", sa.String(length=512), nullable=True),
@@ -35,16 +41,32 @@ def upgrade() -> None:
sa.Column("smart", sa.Boolean(), nullable=False, server_default="false"),
sa.Column("editable", sa.Boolean(), nullable=False, server_default="false"),
sa.Column("synced_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
)
op.create_index(
"ix_plex_collections_rating_key", "plex_collections", ["rating_key"], unique=True
)
op.create_index("ix_plex_collections_rating_key", "plex_collections", ["rating_key"], unique=True)
op.create_index("ix_plex_collections_library_id", "plex_collections", ["library_id"])
op.add_column("plex_items", sa.Column("collection_keys", postgresql.JSONB(), nullable=True))
op.create_index("ix_plex_items_collections_gin", "plex_items", ["collection_keys"], postgresql_using="gin")
op.create_index(
"ix_plex_items_collections_gin", "plex_items", ["collection_keys"], postgresql_using="gin"
)
op.add_column("plex_shows", sa.Column("collection_keys", postgresql.JSONB(), nullable=True))
op.create_index("ix_plex_shows_collections_gin", "plex_shows", ["collection_keys"], postgresql_using="gin")
op.create_index(
"ix_plex_shows_collections_gin", "plex_shows", ["collection_keys"], postgresql_using="gin"
)
def downgrade() -> None:
@@ -9,6 +9,7 @@ Plex account (like plex_states) — the "your own lists" counterpart to the shar
collections. `plex_playlists` is owned by a user; `plex_playlist_items` holds the ordered entries
(`position`). `plex_rating_key` is reserved for an optional later Plex-direction sync.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -24,11 +25,23 @@ def upgrade() -> None:
op.create_table(
"plex_playlists",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("user_id", sa.Integer(), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False),
sa.Column(
"user_id", sa.Integer(), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False
),
sa.Column("title", sa.String(length=512), nullable=False),
sa.Column("plex_rating_key", sa.String(length=32), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("now()"),
nullable=False,
),
)
op.create_index("ix_plex_playlists_user_id", "plex_playlists", ["user_id"])
@@ -36,12 +49,16 @@ def upgrade() -> None:
"plex_playlist_items",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column(
"playlist_id", sa.Integer(),
sa.ForeignKey("plex_playlists.id", ondelete="CASCADE"), nullable=False,
"playlist_id",
sa.Integer(),
sa.ForeignKey("plex_playlists.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"item_id", sa.Integer(),
sa.ForeignKey("plex_items.id", ondelete="CASCADE"), nullable=False,
"item_id",
sa.Integer(),
sa.ForeignKey("plex_items.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("position", sa.Integer(), nullable=False, server_default="0"),
sa.UniqueConstraint("playlist_id", "item_id", name="uq_plex_playlist_item"),
@@ -13,6 +13,7 @@ and captures yt-dlp's canonical channel/uploader URL so an auto-filled channel b
overrides (like display_name — display metadata only, never touches the on-disk file).
- download_jobs.extra_links: optional list of extra reference URLs shown on the card + watch page.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -26,7 +27,9 @@ depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column("media_assets", sa.Column("uploader_url", sa.Text(), nullable=True))
op.add_column("download_jobs", sa.Column("display_uploader", sa.String(length=255), nullable=True))
op.add_column(
"download_jobs", sa.Column("display_uploader", sa.String(length=255), nullable=True)
)
op.add_column("download_jobs", sa.Column("display_uploader_url", sa.Text(), nullable=True))
op.add_column("download_jobs", sa.Column("extra_links", sa.JSON(), nullable=True))
@@ -9,6 +9,7 @@ download shows a blank image box. The worker now extracts a representative frame
saves it as a poster sidecar; `poster_path` (relative to download_root) records it so the card /
watch page / link preview can fall back to it. Nullable — only set when there was no thumbnail.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -5,6 +5,7 @@ state with a Plex account. The owner MVP uses the server admin token (uses_admin
token); a personal token column is reserved for later friend-account linking (PIN OAuth). See the
PlexLink model docstring.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -11,6 +11,7 @@ blob folded into the generated `search_vector` (weight B). This lets the TV grid
sorted like the movie grid. A generated column's expression can't be altered in place, so the
search_vector is dropped + recreated. All new columns are populated on the next Plex sync.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -57,9 +58,13 @@ def upgrade() -> None:
op.create_index("ix_plex_shows_rating", "plex_shows", ["rating"])
op.create_index("ix_plex_shows_content_rating", "plex_shows", ["content_rating"])
op.create_index("ix_plex_shows_studio", "plex_shows", ["studio"])
op.create_index("ix_plex_shows_originally_available_at", "plex_shows", ["originally_available_at"])
op.create_index(
"ix_plex_shows_originally_available_at", "plex_shows", ["originally_available_at"]
)
op.create_index("ix_plex_shows_genres_gin", "plex_shows", ["genres"], postgresql_using="gin")
op.create_index("ix_plex_shows_directors_gin", "plex_shows", ["directors"], postgresql_using="gin")
op.create_index(
"ix_plex_shows_directors_gin", "plex_shows", ["directors"], postgresql_using="gin"
)
op.create_index("ix_plex_shows_cast_gin", "plex_shows", ["cast_names"], postgresql_using="gin")
# NB: ix_plex_shows_collections_gin already exists (created with collections in 0047).
@@ -5,6 +5,7 @@ current_user rejects any cookie whose epoch is stale. Bumped on password reset,
or "log out everywhere" — so a stolen/copied client-side cookie dies on those events instead of
staying valid until it expires. Existing rows default to 0 (matching a fresh cookie's absent/0 epoch).
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -4,6 +4,7 @@ Revision ID: 0054_audit_log
Revises: 0053_user_session_epoch
Create Date: 2026-07-12
"""
from typing import Sequence, Union
from alembic import op
@@ -4,6 +4,7 @@ Revision ID: 0055_channel_keywords
Revises: 0054_audit_log
Create Date: 2026-07-12
"""
from typing import Sequence, Union
from alembic import op
+7 -2
View File
@@ -4,6 +4,7 @@ Revision ID: 0056_plex_people
Revises: 0055_channel_keywords
Create Date: 2026-07-19
"""
from typing import Sequence, Union
from alembic import op
@@ -23,8 +24,12 @@ def upgrade() -> None:
# A representative proxied headshot URL (or NULL when none is available). Captured
# opportunistically from an item's rich metadata (info-page view + the rich re-sync).
sa.Column("photo_thumb", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
)
op.create_index("ix_plex_people_name", "plex_people", ["name"], unique=True)
@@ -4,6 +4,7 @@ Revision ID: 0057_plex_ppl_enriched
Revises: 0056_plex_people
Create Date: 2026-07-19
"""
from typing import Sequence, Union
from alembic import op
@@ -17,7 +18,9 @@ depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
for tbl in ("plex_items", "plex_shows"):
op.add_column(tbl, sa.Column("people_enriched_at", sa.DateTime(timezone=True), nullable=True))
op.add_column(
tbl, sa.Column("people_enriched_at", sa.DateTime(timezone=True), nullable=True)
)
def downgrade() -> None:
@@ -10,6 +10,7 @@ Best still one click away. Re-seeds the builtin rows so fresh installs and exist
(same mechanism as 0040). Existing jobs are unaffected (they snapshot their spec; a profile_id that
pointed at a replaced row just SET NULLs).
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -10,6 +10,7 @@ did NOT invalidate outstanding grants — they lived out their 6h TTL. Fold a pe
into the signed grant and bump it on every password change, so a rotation revokes old grants at once.
Small table (one row per share link), so this is a quick add-column with a default.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -9,6 +9,7 @@ This one-time data fixup rewrites those scalars to SQL NULL so the columns are u
forward-only: a SQL NULL is indistinguishable from the old JSON `null` to every reader (the guards
treat both as "not an array"), so there is nothing meaningful to downgrade. No schema change.
"""
from typing import Sequence, Union
from alembic import op
@@ -25,9 +26,7 @@ _COLS = ("genres", "directors", "cast_names", "collection_keys")
def upgrade() -> None:
for table in _TABLES:
for col in _COLS:
op.execute(
f"UPDATE {table} SET {col} = NULL WHERE jsonb_typeof({col}) = 'null'"
)
op.execute(f"UPDATE {table} SET {col} = NULL WHERE jsonb_typeof({col}) = 'null'")
def downgrade() -> None:
@@ -7,6 +7,7 @@ small category code (`unavailable` / `login_required` / `geo_blocked` / `format_
a retry makes sense, keeping the raw text as a technical-detail disclosure. Nullable; only set on the
error path. Forward-only — an existing errored row simply has NULL until its next failure/retry.
"""
from typing import Sequence, Union
import sqlalchemy as sa
@@ -0,0 +1,54 @@
"""drop 10 redundant single-column FK indexes (R12 S5)
Each of these columns already leads a composite UNIQUE constraint, whose btree covers leftmost-prefix
lookups on that column (and the ON DELETE CASCADE reverse lookups too) — so the extra single-column
`index=True` was pure write-amplification. Dropping them costs nothing on reads and saves an index
write per insert/update. Fast (index-only drop, no table rewrite). Reversible (downgrade recreates).
Redundant index -> the composite that already covers it:
ix_subscriptions_user_id <- uq_user_channel (user_id, channel_id)
ix_video_states_user_id <- uq_user_video (user_id, video_id)
ix_search_finds_user_id <- uq_user_search_video (user_id, video_id)
ix_explored_channels_user_id <- uq_user_explored_channel (user_id, channel_id)
ix_blocked_channels_user_id <- uq_user_blocked_channel (user_id, channel_id)
ix_saved_views_user_id <- uq_user_saved_view (user_id, name)
ix_playlist_items_playlist_id <- uq_playlist_video (playlist_id, video_id)
ix_download_shares_job_id <- uq_download_share (job_id, to_user_id)
ix_plex_states_user_id <- uq_plex_user_item (user_id, item_id)
ix_plex_playlist_items_playlist_id <- uq_plex_playlist_item (playlist_id, item_id)
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0062_drop_redundant_idx"
down_revision: Union[str, None] = "0061_dl_job_error_code"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# (index_name, table_name, column)
_REDUNDANT: list[tuple[str, str, str]] = [
("ix_subscriptions_user_id", "subscriptions", "user_id"),
("ix_video_states_user_id", "video_states", "user_id"),
("ix_search_finds_user_id", "search_finds", "user_id"),
("ix_explored_channels_user_id", "explored_channels", "user_id"),
("ix_blocked_channels_user_id", "blocked_channels", "user_id"),
("ix_saved_views_user_id", "saved_views", "user_id"),
("ix_playlist_items_playlist_id", "playlist_items", "playlist_id"),
("ix_download_shares_job_id", "download_shares", "job_id"),
("ix_plex_states_user_id", "plex_states", "user_id"),
("ix_plex_playlist_items_playlist_id", "plex_playlist_items", "playlist_id"),
]
def upgrade() -> None:
for idx, _table, _col in _REDUNDANT:
# IF EXISTS: a DB whose schema was stamped from a create_all (tests) won't have these once
# the models drop index=True, so a plain drop_index would error there.
op.execute(f"DROP INDEX IF EXISTS {idx}")
def downgrade() -> None:
for idx, table, col in _REDUNDANT:
op.create_index(idx, table, [col])
@@ -0,0 +1,40 @@
"""plex library HU-overlay columns (E5 S5a)
Adds `metadata_lang` (default 'en') and a self-referential `overlay_of` to plex_libraries. An
OVERLAY library is the Hungarian-metadata twin of another section: it is never browsed on its own —
its titles' HU fields are written onto the matched EN rows (paired by `guid`, see 0064). Cheap:
metadata-only, no table rewrite.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0063_plex_library_overlay"
down_revision: Union[str, None] = "0062_drop_redundant_idx"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"plex_libraries",
sa.Column("metadata_lang", sa.String(8), nullable=False, server_default="en"),
)
op.add_column(
"plex_libraries",
sa.Column(
"overlay_of",
sa.Integer(),
sa.ForeignKey("plex_libraries.id", ondelete="SET NULL"),
nullable=True,
),
)
op.create_index("ix_plex_libraries_overlay_of", "plex_libraries", ["overlay_of"])
def downgrade() -> None:
op.drop_index("ix_plex_libraries_overlay_of", table_name="plex_libraries")
op.drop_column("plex_libraries", "overlay_of")
op.drop_column("plex_libraries", "metadata_lang")
@@ -0,0 +1,59 @@
"""plex HU-overlay item/show columns + guid (E5 S5a)
On plex_items and plex_shows: `guid` (canonical external id, the EN<->HU pairing key), the Hungarian
twin fields `title_hu`/`summary_hu`/`genres_hu`, and a STORED generated `search_vector_hu` mirroring
the EN vector (title_hu A / people_text B / left(summary_hu,1000) C) + a GIN index. people_text is
language-neutral, reused for both vectors.
Run BEFORE the first HU overlay sync: while the *_hu columns are all NULL the generated-column build
is trivial. ADD only (no drop/rebuild of the EN search_vector). NOTE: adding a STORED generated
column + its GIN takes an ACCESS EXCLUSIVE lock and rewrites the table — brief, but flag it on prod.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0064_plex_hu_overlay_cols"
down_revision: Union[str, None] = "0063_plex_library_overlay"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
_TABLES = ("plex_items", "plex_shows")
# Mirror of the EN search_vector (models.py) but over the HU twin columns; people_text is shared.
_VECTOR_HU = (
"setweight(to_tsvector('public.unaccent_simple', coalesce(title_hu, '')), 'A') || "
"setweight(to_tsvector('public.unaccent_simple', coalesce(people_text, '')), 'B') || "
"setweight(to_tsvector('public.unaccent_simple', left(coalesce(summary_hu, ''), 1000)), 'C')"
)
def upgrade() -> None:
for t in _TABLES:
op.add_column(t, sa.Column("guid", sa.String(64), nullable=True))
op.add_column(t, sa.Column("title_hu", sa.String(512), nullable=True))
op.add_column(t, sa.Column("summary_hu", sa.Text(), nullable=True))
op.add_column(t, sa.Column("genres_hu", postgresql.JSONB(), nullable=True))
op.create_index(f"ix_{t}_guid", t, ["guid"])
op.create_index(f"ix_{t}_genres_hu_gin", t, ["genres_hu"], postgresql_using="gin")
# Generated column can't be expressed via add_column — raw DDL, matching the EN idiom (0044).
op.execute(
f"ALTER TABLE {t} ADD COLUMN search_vector_hu tsvector "
f"GENERATED ALWAYS AS ({_VECTOR_HU}) STORED"
)
op.execute(f"CREATE INDEX ix_{t}_search_vector_hu ON {t} USING gin (search_vector_hu)")
def downgrade() -> None:
for t in _TABLES:
op.execute(f"DROP INDEX IF EXISTS ix_{t}_search_vector_hu")
op.execute(f"ALTER TABLE {t} DROP COLUMN IF EXISTS search_vector_hu")
op.drop_index(f"ix_{t}_genres_hu_gin", table_name=t)
op.drop_index(f"ix_{t}_guid", table_name=t)
op.drop_column(t, "genres_hu")
op.drop_column(t, "summary_hu")
op.drop_column(t, "title_hu")
op.drop_column(t, "guid")
@@ -0,0 +1,46 @@
"""plex rich-detail item/show columns (E5 S5b)
On plex_items and plex_shows: the fields that were previously read LIVE from plex.metadata() on every
info-page view — `imdb_rating` (the IMDb-specific score, distinct from `rating` = audienceRating),
`imdb_id` (tt…), `tagline`, and `full_cast` (ordered [{name, role, thumb}]). Persisting them lets the
detail endpoints serve from the DB row instead of a live Plex call (outage-resilient, and the
precondition for S6's pure-DB language-aware serving).
These are filled by the same rich backfill that already fills the full cast (`_enrich_people_batch`),
so the data step below resets `people_enriched_at` to NULL on movie/show rows: the backfill re-fetches
each once (60/run, self-throttled) and stamps the new columns. ADD-only, all nullable → no table
rewrite, no generated column, no ACCESS EXCLUSIVE lock (unlike 0064).
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0065_plex_detail_fields"
down_revision: Union[str, None] = "0064_plex_hu_overlay_cols"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
_TABLES = ("plex_items", "plex_shows")
def upgrade() -> None:
for t in _TABLES:
op.add_column(t, sa.Column("imdb_rating", sa.Float(), nullable=True))
op.add_column(t, sa.Column("imdb_id", sa.String(16), nullable=True))
op.add_column(t, sa.Column("tagline", sa.Text(), nullable=True))
op.add_column(t, sa.Column("full_cast", postgresql.JSONB(), nullable=True))
# Force a one-time re-enrich so already-enriched rows gain the new columns (episodes are never
# enriched, so their NULL people_enriched_at is unaffected; they keep serving detail live).
op.execute("UPDATE plex_items SET people_enriched_at = NULL WHERE kind = 'movie'")
op.execute("UPDATE plex_shows SET people_enriched_at = NULL")
def downgrade() -> None:
for t in _TABLES:
op.drop_column(t, "full_cast")
op.drop_column(t, "tagline")
op.drop_column(t, "imdb_id")
op.drop_column(t, "imdb_rating")
@@ -0,0 +1,25 @@
"""notifications.created_at NOT NULL (align schema to the model)
The model has always declared `Notification.created_at: Mapped[datetime]` (NOT NULL) with
`server_default=func.now()`, but migration 0017 created the column without `nullable=False`, so the live
schema allowed NULL — a model/schema drift the new migration drift-test (E1 S2) surfaced. The
server_default has filled every row since day one, so no NULLs exist; enforcing NOT NULL just makes the
schema match the contract the model already promises. Pure DDL, reversible.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0066_notif_created_at_notnull"
down_revision: Union[str, None] = "0065_plex_detail_fields"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.alter_column("notifications", "created_at", nullable=False)
def downgrade() -> None:
op.alter_column("notifications", "created_at", nullable=True)
@@ -0,0 +1,56 @@
"""feed sort indexes: published_at DESC NULLS LAST composite(s) for the default feed (PERF-1)
The feed's default `newest` sort is `published_at DESC NULLS LAST, id ASC` (feedquery.order_by), but the
only published_at index was a plain ASC btree (`ix_videos_published_at`). A btree read backwards satisfies
`DESC NULLS FIRST`, not the `DESC NULLS LAST` the query asks for, so the planner could not use it for
ordering — every feed page fell back to a full `videos` scan + top-N sort. Measured on a prod-sized reseed
(257k videos): the default page went from ~44.8k shared buffers (dev planner) / ~169k (prod planner, the
per-channel nested loop that is the likely "prod feed is slow" root cause) down to ~624 buffers with the
matching index.
This adds two indexes in the exact feed order:
- ix_videos_published_at_desc (published_at DESC NULLS LAST, id ASC) — the global feed / Library sort.
- ix_videos_channel_pub_desc (channel_id, published_at DESC NULLS LAST, id ASC) — the per-channel feed
(channel page, channel_ids filter). It leads with channel_id, so it is a strict superset of the old
single-column `ix_videos_channel_id` (FK cascade, sync upsert, autotag, channel-detail all still ride
it) — that redundant index is dropped.
The plain ASC `ix_videos_published_at` stays: it already serves the `oldest` sort (ASC NULLS LAST = btree
default) as an Index Scan.
CONCURRENTLY (in alembic's autocommit_block, since it can't run inside a transaction) so a live prod builds
these without blocking reads/writes; downtime ~0. IF (NOT) EXISTS keeps the drift test's
downgrade->re-upgrade round-trip idempotent.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0067_feed_sort_indexes"
down_revision: Union[str, None] = "0066_notif_created_at_notnull"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
with op.get_context().autocommit_block():
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_published_at_desc "
"ON videos (published_at DESC NULLS LAST, id ASC)"
)
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_channel_pub_desc "
"ON videos (channel_id, published_at DESC NULLS LAST, id ASC)"
)
# Redundant now — ix_videos_channel_pub_desc leads with channel_id (a strict superset).
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_channel_id")
def downgrade() -> None:
with op.get_context().autocommit_block():
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_channel_id ON videos (channel_id)"
)
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_channel_pub_desc")
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_published_at_desc")
@@ -0,0 +1,35 @@
"""videos.published_at NOT NULL DEFAULT now() (PERF-15)
Every video in practice carries a publish date (0 NULLs in the 257k-row prod-sized reseed), but the
column was nullable, which forced the feed's keyset predicate into a non-sargable
`... OR published_at IS NULL` disjunction — blocking the ix_videos_published_at_desc range so deep
`newest` pages scanned from the top of the index and filtered (51.8k buffers). Making it NOT NULL lets
the keyset drop that disjunction and take a `published_at <=` range bound (deep page -> ~0.5k buffers).
server_default now() is the safety net for the rare date-less ingest path (the enrich overwrites it with
the real date); the app-side ingest paths also fall back to now() so they never write an explicit NULL.
Note: `SET NOT NULL` validates with a brief ACCESS EXCLUSIVE lock + one table scan (it cannot run
CONCURRENTLY). On the prod row count (~235k) that is sub-second; still, it is a short write-blocking lock
rather than the ~0 of 0067's CONCURRENTLY index build.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0068_published_at_not_null"
down_revision: Union[str, None] = "0067_feed_sort_indexes"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Backfill any stray NULLs first so SET NOT NULL can't fail (0 in prod, but be safe).
op.execute("UPDATE videos SET published_at = now() WHERE published_at IS NULL")
op.alter_column("videos", "published_at", nullable=False, server_default=sa.text("now()"))
def downgrade() -> None:
op.alter_column("videos", "published_at", nullable=True, server_default=None)
@@ -0,0 +1,94 @@
"""scheduler-job partial indexes + boolean-index cleanup (PERF-17/18/3)
Two scheduler jobs full-seq-scanned the 257k-row videos table every run:
- PERF-17: enrich_pending (`WHERE enriched_at IS NULL LIMIT N`, every few minutes) — 44.6k buffers /
run even when nothing is pending. A partial index over just the pending rows makes it a 1-buffer
(empty) index scan.
- PERF-18: maintenance revalidate (`WHERE unavailable_since IS NULL ORDER BY last_checked_at NULLS
FIRST LIMIT N`, ~300 pages/run) — 44.7k buffers/page; ix_videos_last_checked_at had idx_scan=0. A
partial index in that exact order serves it as an index scan (~36 buffers/page) and replaces the
unused plain one.
PERF-3: the videos table also carried plain btrees on low-cardinality boolean columns the planner never
picks (a `col IS false` majority predicate can't ride a btree). Drop the never-useful ones (is_short,
via_explore) and slim the rest to partial indexes over the small/shrinking subset the job queries target
(via_search=true, NOT shorts_probed, unavailable_since IS NOT NULL) — less write amplification on the
biggest table, same lookups. live_status is left alone (a separate finding).
All CONCURRENTLY (autocommit_block) — downtime ~0; IF (NOT) EXISTS keeps the drift test's
downgrade->re-upgrade idempotent.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0069_job_index_hygiene"
down_revision: Union[str, None] = "0068_published_at_not_null"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
with op.get_context().autocommit_block():
# PERF-17: enrich-pending partial (empty when nothing is pending -> 1-buffer scan).
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_enrich_pending "
"ON videos (id) WHERE enriched_at IS NULL"
)
# PERF-18: maintenance revalidate order, as a partial in the exact WHERE + ORDER BY shape.
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_recheck_order "
"ON videos (last_checked_at ASC NULLS FIRST) WHERE unavailable_since IS NULL"
)
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_last_checked_at")
# PERF-3: drop the never-picked boolean btrees; slim the rest to partials over the target subset.
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_is_short")
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_via_explore")
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_via_search")
# Predicate MUST match the query's `.is_(True)`/`.is_(False)` form (SQL `IS TRUE`/`IS FALSE`):
# Postgres's partial-index prover does NOT match a `col IS TRUE` clause to a bare `col` predicate,
# so `WHERE via_search` / `WHERE NOT shorts_probed` would leave the queries seq-scanning.
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_via_search "
"ON videos (id) WHERE via_search IS TRUE"
)
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_shorts_probed")
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_shorts_probed "
"ON videos (id) WHERE shorts_probed IS FALSE"
)
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_unavailable_since")
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_unavailable_since "
"ON videos (unavailable_since) WHERE unavailable_since IS NOT NULL"
)
def downgrade() -> None:
with op.get_context().autocommit_block():
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_enrich_pending")
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_recheck_order")
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_last_checked_at "
"ON videos (last_checked_at)"
)
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_is_short ON videos (is_short)"
)
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_via_explore ON videos (via_explore)"
)
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_via_search")
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_via_search ON videos (via_search)"
)
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_shorts_probed")
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_shorts_probed ON videos (shorts_probed)"
)
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_unavailable_since")
op.execute(
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_unavailable_since "
"ON videos (unavailable_since)"
)
@@ -0,0 +1,48 @@
"""Fold the maintenance re-validation batch from app_state into sysconfig (B-core-11)
The admin-tunable `maintenance_revalidate_batch` lived in its own `app_state` column with a
hand-rolled override-or-default reader in `app.state`, duplicating the sysconfig SPECS pattern. It's
now a sysconfig key (`system_config` store), so every admin knob lives in one place. This migration
carries any existing override into the sysconfig store, then drops the dead column. Reversible.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0070_fold_maintenance_batch"
down_revision: Union[str, None] = "0069_job_index_hygiene"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Preserve a customized override: copy it into the sysconfig store before the column goes.
op.execute(
"""
INSERT INTO system_config (key, value, encrypted)
SELECT 'maintenance_revalidate_batch', maintenance_revalidate_batch::text, false
FROM app_state
WHERE id = 1 AND maintenance_revalidate_batch IS NOT NULL
ON CONFLICT (key) DO NOTHING
"""
)
op.drop_column("app_state", "maintenance_revalidate_batch")
def downgrade() -> None:
op.add_column(
"app_state",
sa.Column("maintenance_revalidate_batch", sa.Integer(), nullable=True),
)
# Carry the value back onto the row, then drop the sysconfig override (symmetric restore).
op.execute(
"""
UPDATE app_state SET maintenance_revalidate_batch =
(SELECT value::int FROM system_config WHERE key = 'maintenance_revalidate_batch')
WHERE id = 1
AND EXISTS (SELECT 1 FROM system_config WHERE key = 'maintenance_revalidate_batch')
"""
)
op.execute("DELETE FROM system_config WHERE key = 'maintenance_revalidate_batch'")
@@ -0,0 +1,53 @@
"""plex TMDB enrichment columns (E5 S2)
Additive TMDB-sourced fields on plex_items + plex_shows — keywords, full crew, budget/revenue, TMDb
community score, production companies/countries, recommendations, and a backdrop-path pool — plus a
`tmdb_id` cache and a `tmdb_enriched_at` watermark. Filled by the new TMDB enrichment job (matched by
the stored canonical `guid`), SERVED FROM THE ROW (never a live TMDB call). Distinct from the existing
Plex-sourced `imdb_rating`/`rating`, which are left untouched. Plus `plex_people.tmdb_person_id` (the S2
foundation for S6's person bios). ADD-only, all nullable → no table rewrite, no ACCESS EXCLUSIVE lock.
No watermark reset needed: `tmdb_enriched_at` starts NULL, so every eligible row is picked up once.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0071_plex_tmdb_fields"
down_revision: Union[str, None] = "0070_fold_maintenance_batch"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
_TABLES = ("plex_items", "plex_shows")
# (name, type) — same set on both movie/leaf rows and show rows. budget/revenue stay NULL for shows.
_COLS: tuple[tuple[str, sa.types.TypeEngine], ...] = (
("tmdb_id", sa.Integer()),
("tmdb_enriched_at", sa.DateTime(timezone=True)),
("tmdb_rating", sa.Float()),
("tmdb_vote_count", sa.Integer()),
("budget", sa.BigInteger()),
("revenue", sa.BigInteger()),
("keywords", postgresql.JSONB()),
("crew", postgresql.JSONB()),
("studios", postgresql.JSONB()),
("production_countries", postgresql.JSONB()),
("recommendations", postgresql.JSONB()),
("backdrops", postgresql.JSONB()),
)
def upgrade() -> None:
for t in _TABLES:
for name, type_ in _COLS:
op.add_column(t, sa.Column(name, type_, nullable=True))
op.add_column("plex_people", sa.Column("tmdb_person_id", sa.Integer(), nullable=True))
def downgrade() -> None:
op.drop_column("plex_people", "tmdb_person_id")
for t in _TABLES:
for name, _ in reversed(_COLS):
op.drop_column(t, name)
@@ -0,0 +1,41 @@
"""plex person TMDB bio columns (E5 S6)
Additive TMDB-sourced person fields on plex_people — biography, birth/death dates, place of birth,
known-for department, also-known-as, and a `known_for` credit pool ([{tmdb_id, media_type, title}]
resolved to library rows by guid at serve time) — plus a `tmdb_enriched_at` watermark. Filled the
first time a person page is opened (from /person/{tmdb_person_id}), then SERVED FROM THE ROW. The
`tmdb_person_id` link itself already exists (0071). ADD-only, all nullable → no table rewrite, no
ACCESS EXCLUSIVE lock. `tmdb_enriched_at` starts NULL, so every person is enriched once on first view.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "0072_plex_person_bio"
down_revision: Union[str, None] = "0071_plex_tmdb_fields"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
_COLS: tuple[tuple[str, sa.types.TypeEngine], ...] = (
("biography", sa.Text()),
("birthday", sa.String(length=10)),
("deathday", sa.String(length=10)),
("place_of_birth", sa.Text()),
("known_for_department", sa.String(length=64)),
("also_known_as", postgresql.JSONB()),
("known_for", postgresql.JSONB()),
("tmdb_enriched_at", sa.DateTime(timezone=True)),
)
def upgrade() -> None:
for name, type_ in _COLS:
op.add_column("plex_people", sa.Column(name, type_, nullable=True))
def downgrade() -> None:
for name, _ in reversed(_COLS):
op.drop_column("plex_people", name)
@@ -0,0 +1,40 @@
"""saved_views.source (ask 4: per-module saved views)
Saved "smart views" were feed-only; the Plex module now has them too. Add a `source` discriminator
(default "feed" so every existing row stays a feed view) and widen the per-user unique name constraint
to (user_id, source, name) so a feed view and a Plex view can share a name. `source` is NOT NULL with a
server_default, so the ADD is a metadata-only rewrite-free change; the constraint swap is quick on a
small per-user table.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0073_saved_view_source"
down_revision: Union[str, None] = "0072_plex_person_bio"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"saved_views",
sa.Column("source", sa.String(length=16), nullable=False, server_default="feed"),
)
op.drop_constraint("uq_user_saved_view", "saved_views", type_="unique")
op.create_unique_constraint(
"uq_user_source_saved_view", "saved_views", ["user_id", "source", "name"]
)
def downgrade() -> None:
# LOSSY / effectively forward-only: upgrade lets a user hold a feed view and a plex view of the same
# name, so re-creating the narrow (user_id, name) constraint can raise IntegrityError on such a pair.
# De-dup by dropping the non-feed rows first (they can't exist in the pre-0073 schema anyway) so the
# narrow constraint re-applies cleanly for the drift test's downgrade→re-upgrade cycle.
op.execute("DELETE FROM saved_views WHERE source <> 'feed'")
op.drop_constraint("uq_user_source_saved_view", "saved_views", type_="unique")
op.create_unique_constraint("uq_user_saved_view", "saved_views", ["user_id", "name"])
op.drop_column("saved_views", "source")
@@ -0,0 +1,46 @@
"""download_credentials (S6: per-user, per-host yt-dlp cookie/auth store)
Each user stores their OWN login cookies (Netscape cookies.txt) for an auth-walled download host
(Facebook, Instagram, …), Fernet-encrypted at rest. A download of a yt-dlp-supported, auth-walled
link on that host is retried with the acting user's cookie. Unique per (user_id, host_key) so a
user holds at most one credential per host. Empty for every existing install; forward-only in
practice (a downgrade just drops the table + its secrets).
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0074_download_credentials"
down_revision: Union[str, None] = "0073_saved_view_source"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"download_credentials",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column(
"user_id",
sa.Integer(),
sa.ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("host_key", sa.String(length=64), nullable=False),
sa.Column("cookies_enc", sa.Text(), nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column(
"updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.UniqueConstraint("user_id", "host_key", name="uq_download_credential"),
)
op.create_index("ix_download_credentials_user_id", "download_credentials", ["user_id"])
def downgrade() -> None:
op.drop_index("ix_download_credentials_user_id", table_name="download_credentials")
op.drop_table("download_credentials")
@@ -0,0 +1,26 @@
"""playlist synced_video_ids (S7: 3-way merge baseline for two-way YouTube sync)
The one-directional "local wins" push is replaced by a 3-way merge that classifies each item
against the last-synced state. That needs the raw ordered id set as a baseline, not just the
`synced_fingerprint` hash. Add a nullable `synced_video_ids` (JSON list) column; NULL on every
existing row means "no baseline yet" — the first merge then unions losslessly and records one.
Forward-only in practice (a downgrade just drops the column).
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0075_playlist_sync_baseline"
down_revision: Union[str, None] = "0074_download_credentials"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column("playlists", sa.Column("synced_video_ids", sa.Text(), nullable=True))
def downgrade() -> None:
op.drop_column("playlists", "synced_video_ids")
@@ -0,0 +1,41 @@
"""media_assets.media_kind (E7 S5a: the video|audio|file discriminator for the media library)
Adds a stored media-kind to each asset so the library can gate its actions on it (play/edit for
media, download-only for a generic uploaded file) instead of re-deriving from codecs in prose. The
column is the single home of the invariant `downloads.mediakind.derive` computes at finalize.
Backfill for existing rows: a stored video codec -> 'video', an audio-only codec -> 'audio', an
editor derivative (source_kind='edit', which carries no codecs but is always a video clip) ->
'video'. Rows still pending/errored (no codecs, not an edit) stay NULL — kind is set when they
finalize. Empty `media_kind` never reaches the UI: only `ready`/`done` rows render actions, and
those are all backfilled here.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0076_media_asset_kind"
down_revision: Union[str, None] = "0075_playlist_sync_baseline"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column("media_assets", sa.Column("media_kind", sa.String(length=16), nullable=True))
op.execute(
"""
UPDATE media_assets
SET media_kind = CASE
WHEN vcodec IS NOT NULL THEN 'video'
WHEN acodec IS NOT NULL THEN 'audio'
WHEN source_kind = 'edit' THEN 'video'
ELSE NULL
END
"""
)
def downgrade() -> None:
op.drop_column("media_assets", "media_kind")
@@ -0,0 +1,54 @@
"""plex_items playback-class facet (E7: filter the feed by Direct Play / Direct Stream / Transcode)
Adds the columns the playback-class filter needs:
* transcode_hw — for a `playable="transcode"` item, whether the prod iGPU can hardware-transcode it
(codec HW-decodable AND 8-bit SDR) vs software-only. NULL for non-transcode items.
* video_bit_depth — the source video bit depth (from the Plex media Stream), NULL until probed.
* media_probed_at — watermark for the bit-depth backfill (NULL = not yet probed).
Backfill for existing rows: `transcode_hw` is derived from the ALREADY-stored `codec_video` alone (the
same HW-decodable allow-list `plex.sync.HW_DECODABLE_CODECS` / `plex.stream` use), so the filter works
immediately after migration. Bit depth is unknown for existing rows (the cheap Plex listing doesn't
carry it), so a 10-bit/HDR HEVC is provisionally 'hw' here and later refined to 'sw' by the bit-depth
backfill (`plex.sync._probe_media_batch`) once its Stream is probed. `video_bit_depth`/`media_probed_at`
start NULL.
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0077_plex_playback_class"
down_revision: Union[str, None] = "0076_media_asset_kind"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column("plex_items", sa.Column("transcode_hw", sa.Boolean(), nullable=True))
op.add_column("plex_items", sa.Column("video_bit_depth", sa.Integer(), nullable=True))
op.add_column(
"plex_items", sa.Column("media_probed_at", sa.DateTime(timezone=True), nullable=True)
)
op.create_index("ix_plex_items_transcode_hw", "plex_items", ["transcode_hw"])
# Codec-only derivation for existing rows — keep this codec set identical to
# plex.sync.HW_DECODABLE_CODECS. Bit depth is unknown here, so provisionally 8-bit (→ hw for a
# decodable codec); the bit-depth backfill downgrades 10-bit/HDR HEVC to sw later.
op.execute(
"""
UPDATE plex_items
SET transcode_hw = CASE
WHEN playable = 'transcode'
THEN lower(codec_video) IN ('h264','hevc','h265','vp9','vc1','mpeg2video')
ELSE NULL
END
"""
)
def downgrade() -> None:
op.drop_index("ix_plex_items_transcode_hw", table_name="plex_items")
op.drop_column("plex_items", "media_probed_at")
op.drop_column("plex_items", "video_bit_depth")
op.drop_column("plex_items", "transcode_hw")
+75
View File
@@ -0,0 +1,75 @@
# Backend conventions
The rules below are already followed across `backend/app` — they lived only in a few module
docstrings (`demo.py`, `userscope.py`, `routes/_common.py`) until this file collected them (E3 S3,
B-core-20). When you add code, match these so the next reader doesn't have to guess "which pattern do
I copy".
## Module layering
Three layers, by *who* runs the code — not by feature:
1. **`routes/…` — request-only.** A route module holds HTTP concerns: auth dependencies, request/
response models, status-code mapping, serialization. It owns logic that runs *only* on a request.
Business logic that a request happens to trigger does **not** live here if anything else runs it too.
2. **`plex/`, `downloads/`, `sync/` — domain logic shared by more than one caller.** Anything the
**scheduler or the worker** runs (not just a request) lives in one of these domain packages, so the
route, the worker, and the scheduler share one definition. Examples from this codebase:
`downloads/service.py` owns enqueue + the job state transitions (routes call it); `plex/sync.py` is
run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline.
3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth`, `sysconfig`, `quota`, `audit`,
`notifications`, `userscope`, `realtime`, `ratelimit`, `progress`, … are single-file primitives many
layers import. They depend on nothing above them (a core module must never import a `routes/…`
module).
**Extraction trigger: the SECOND consumer.** Keep new logic inline in its first caller. The moment a
*second* caller needs it (a worker as well as a route; a scheduler as well as a route), extract it down
a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't
pre-abstract for a single caller; don't leave shared logic duplicated once there are two.
## Transaction ownership
This follows CLAUDE.md's canonical rule verbatim — **"the handler/job-wrapper commits; helpers flush
(only `quota` owns its session)"** — so read it that way, not as an exception to it. The commit belongs
to the **operation owner**; anything called *inside* another transaction only stages its change
(`db.add` / `flush`) and lets the owner commit, so the whole operation is one transaction.
`audit.record()` is the reference: it adds the audit row and the caller commits it together with the
mutation it records, so an audit entry can never exist without its change (or vice-versa).
- The **operation owner** is usually the route handler. A `service.*` operation the handler delegates a
whole transition to (e.g. `service.cancel`) *is* that owner and commits its one transition — it isn't
a mid-operation helper. A helper that owns its OWN short-lived session (the worker's `jobdb` writes,
the same carve-out CLAUDE.md grants `quota`) commits that session. Everything else flushes.
- `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already
loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM
attribute — see `service.decr_ref_count`.
## Shared counters and status: do it atomically at the DB
- **Never read-modify-write a shared counter in Python** (`x.count = x.count + 1`). Concurrent sessions
lose an update. Use a DB-atomic statement — `col = col + 1`, or `greatest(col - 1, 0)` with
`RETURNING` so the caller branches on the post-value under the row lock (`service.incr_ref_count` /
`decr_ref_count`).
- **Download job status is a `JobStatus` StrEnum** (`downloads/status.py`), not bare strings. The
`HOLDING` set defines which states hold a `MediaAsset` (so release accounting stays correct);
`STOPPED_BY_ROUTE` is the route-initiated stop set. A worker child's terminal write goes through
`service.finalize_if_running` (an atomic `UPDATE … WHERE status='running'`) so a finishing child can't
resurrect a job a route just stopped.
## Config: one home per kind
- **`app.config.settings`** — boot/env config (DB URL, download root, worker concurrency): read once at
startup, not user-tunable at runtime.
- **`sysconfig.SPECS`** — the runtime admin knobs (DB-override-or-default). A new admin-tunable value
goes in `SPECS`; don't invent a fourth store. (`state.py`'s `maintenance_batch` predates this and is
slated to fold into `sysconfig` — do not add more like it.)
## The gate
Everything below must be green before a change is done (run via `siftlode check`):
`ruff` (backend) · `tsc` ×3 + `eslint` + `prettier` + `knip` (frontend) · `vitest` · `pytest`. The
pytest lane runs in `backend/docker-compose.test.yml` — an ephemeral Postgres + the backend tree
bind-mounted — so the DB-backed integration tests (the `db` fixture) actually run; `SIFTLODE_TEST_
REQUIRE_DB=1` there turns a would-be silent skip into a hard failure. See `docs/development.md`.
+5
View File
@@ -9,6 +9,7 @@ runs are always logged. Secrets are never stored (config changes log the key nam
mutation it describes, so the trail can't drift from reality (and a rolled-back mutation drops
its audit row too).
"""
from datetime import datetime, timedelta, timezone
from sqlalchemy import delete
@@ -29,6 +30,7 @@ class AuditAction:
INVITE_APPROVE = "invite_approve"
INVITE_DENY = "invite_deny"
INVITE_ADD = "invite_add"
INVITE_DELETE = "invite_delete"
DEMO_ADD = "demo_add"
DEMO_REMOVE = "demo_remove"
DEMO_RESET = "demo_reset"
@@ -43,6 +45,9 @@ class AuditAction:
SYNC_RESUME = "sync_resume"
# Scheduler job outcome / run-summary (actor_id set = a manual "run now"; NULL = automatic)
JOB_RUN = "job_run"
# Per-user download cookie/auth credential set or removed (the value is NEVER recorded)
DOWNLOAD_CREDENTIAL_SET = "download_credential_set"
DOWNLOAD_CREDENTIAL_DELETE = "download_credential_delete"
# The log itself was cleared (a single row survives the clear as tamper-evidence)
AUDIT_CLEAR = "audit_clear"
+18 -924
View File
File diff suppressed because it is too large Load Diff
+69 -4
View File
@@ -57,7 +57,7 @@ class Settings(BaseSettings):
# VPS Caddy front door. A request from any other peer (someone hitting the app port directly)
# is NOT trusted and its raw socket IP is used, so it can't spoof its rate-limit identity via
# XFF. Empty = trust no proxy (use the direct peer) — correct for a directly-exposed / local-dev
# deploy. Comma-separated. See auth._client_ip + the deploy docs.
# deploy. Comma-separated. See routes.auth._client_ip + the deploy docs.
trusted_proxy_ips: str = ""
# --- Outbound email (onboarding: access-request + approval notices) ---
@@ -169,10 +169,38 @@ class Settings(BaseSettings):
@property
def player_client_fallback_list(self) -> list[str]:
return self._clients(self.download_player_clients_fallback)
# How many downloads the worker runs concurrently (claimed via FOR UPDATE SKIP LOCKED).
download_worker_concurrency: int = 2
# How often (empty pending queue) the worker polls for a new job, in seconds.
download_poll_seconds: int = 5
# --- download-subprocess watchdog (each download/edit runs in a supervised child process so a
# wedged yt-dlp/ffmpeg can be hard-killed; a thread cannot) ---
# Kill a running download if its job row hasn't advanced (no progress written) for this long.
# Generous: a live download writes progress ~1/s, so only a genuine 0-progress stall trips it.
download_stall_timeout_seconds: int = 150
# Absolute ceiling on a single download/edit regardless of progress (a legitimately huge file
# still finishes well under this; a runaway is capped).
download_wall_clock_max_seconds: int = 7200 # 2 hours
# How often the supervisor checks the child's liveness + the job row.
download_supervisor_poll_seconds: int = 3
# Grace between SIGTERM and SIGKILL when stopping a child (matches run_ffmpeg's escalation).
download_kill_grace_seconds: int = 5
# --- worker-child resource de-prioritization ---
# The download/edit child (yt-dlp + ffmpeg) shares ONE LAN box with the API, Postgres, and Plex
# playback. Without a priority handicap a CPU re-encode pegs every core for ~30s and starves web
# requests + playback until it finishes. The child lowers its OWN priority at startup — inherited
# by every ffmpeg/yt-dlp descendant — so any foreground work always wins. Each knob is best-effort
# and independently disable-able with 0.
download_child_nice: int = 10 # CPU nice increment applied to the child; 0 = don't renice
download_child_ionice_class: int = 3 # best-effort IO class (3 = idle); 0 = leave IO untouched
# Cap the browser-safe re-encode's encoder threads so a transcode can't claim every core. 0 =
# reserve one core for foreground work (max(1, cpu_count - 1)); >0 pins exactly that many.
download_ffmpeg_threads: int = 0
# Throttle the retention GC's whole-tree orphan file sweep so its rglob can't monopolize disk IO
# in the API process: sleep `pause_ms` after every `batch` files touched. pause_ms=0 disables.
download_gc_sweep_pause_ms: int = 20
download_gc_sweep_batch: int = 400
# Total cache size cap across ALL users (bytes). When >0 and exceeded, the GC evicts the
# least-recently-accessed ready assets until under it. 0 = unlimited (disk is the limit).
download_total_max_bytes: int = 0
@@ -180,6 +208,10 @@ class Settings(BaseSettings):
download_default_max_bytes: int = 5_368_709_120 # 5 GiB
download_default_max_concurrent: int = 2
download_default_max_jobs: int = 50
# Per-file cap for local uploads (E7). Unlike a download (whose size is unknown until the worker
# finishes), an upload's size is known at ingest, so this bounds a single file directly; the
# user's overall byte footprint is still governed by download_default_max_bytes.
download_upload_max_bytes: int = 524_288_000 # 500 MiB
# Retention: an asset expires this many days after its last access; the GC job then
# deletes it (once no job references it) after an additional grace window.
download_retention_days: int = 30
@@ -209,26 +241,59 @@ class Settings(BaseSettings):
plex_path_map: str = ""
# Which Plex library section keys are exposed (comma-separated). Empty = all enabled sections.
plex_libraries: str = ""
# HU-overlay mapping (E5 S5): comma-separated `huSectionKey:enSectionKey` pairs. Each named HU
# section is synced as a non-browsable overlay whose Hungarian metadata is written onto the EN
# twin section's rows (matched by guid). Empty = no overlays. E.g. "3:4,5:1".
plex_overlay_map: str = ""
# Stable client identifier Siftlode presents to Plex (X-Plex-Client-Identifier).
plex_client_id: str = "siftlode-server"
# How often the catalog sync job mirrors Plex metadata, in minutes.
plex_sync_interval_min: int = 360
# How often the incremental Plex→Siftlode watch-state reconcile runs, in minutes (Phase C):
# a cheap history + on-deck pull plus a re-push of any states that failed their immediate push.
plex_watch_sync_interval_min: int = 30
# 15 min (S2): with Plex now the source of truth (prompt write-back on the ~10s checkpoint +
# Plex-authoritative pull), a faster incremental pull keeps Siftlode close to Plex-side changes.
plex_watch_sync_interval_min: int = 15
# How often the FULL watch-state reconcile runs, in minutes (Phase C). A whole-section rescan
# that catches what the incremental history feed can't — notably an un-watch done on the Plex
# side — so it runs far less often (daily) than the incremental pass.
plex_watch_reconcile_interval_min: int = 1440
# Max concurrent transcodes for the P3 fallback. Low by default — CPU-only transcode is
# expensive; direct-serve (browser-compatible files) has no such limit.
# Max concurrent transcode sessions (playable="transcode"). Low by default — the software-fallback
# re-encode is CPU-heavy (the VAAPI hardware path is far cheaper and could sustain more); direct-serve
# and remux (stream-copy) are not capped by this.
plex_max_transcodes: int = 4
# --- Transcode (playable="transcode") re-encode tuning (E6 S1) ---
# Hardware-encoder policy: "auto" = VAAPI (h264_vaapi) when the iGPU render node is present AND the
# source codec is hardware-decodable, else libx264 software; "on" = force VAAPI wherever the render
# node exists (skip the decodability gate); "off" = always software. See plex/stream.py `_select_encode`.
plex_transcode_hw: str = "auto"
# Software-fallback libx264 tuning (IGNORED on the VAAPI path). `preset` trades CPU for size; `crf`
# trades quality for size (lower = better + bigger). `max_height` caps the SOFTWARE transcode's output
# height (0 = keep source) so a CPU re-encode of a tall source stays affordable — the GPU path always
# encodes at source resolution.
plex_transcode_preset: str = "veryfast"
plex_transcode_crf: int = 20
plex_transcode_max_height: int = 1080
# Where on-the-fly HLS remux segments are written (transient, reaped). Empty → a `.plex-hls`
# dir under download_root. On localdev the download_root is a slow Windows bind-mount, so point
# this at a fast container-local path (e.g. /var/tmp/plex-hls); prod's download_root is fast
# local disk, so the default is fine there.
plex_hls_dir: str = ""
# --- TMDB enrichment (E5 S2; optional — enriches the Plex catalog from TMDB) ---
# TMDB API key (v3 api_key, 32 hex chars). Server-side only; stored encrypted when set via the
# admin Config page. Empty → the TMDB enrichment job no-ops. Feeds keywords / crew / budget /
# revenue / TMDb score / recommendations / backdrops onto the mirrored Plex rows, matched by the
# stored canonical guid, and SERVED FROM THE DB — never a live TMDB call at serve time.
tmdb_api_key: str = ""
tmdb_api_base: str = "https://api.themoviedb.org/3"
# Metadata language for TMDB fetches (keyword/overview locale). Stored EN by default.
tmdb_language: str = "en-US"
# How often the TMDB enrichment job fetches a batch of not-yet-enriched Plex titles, in minutes.
tmdb_enrich_interval_min: int = 120
# Titles fetched per TMDB enrichment run (TMDB is fast + generously rate-limited).
tmdb_enrich_batch: int = 100
@property
def allowed_email_set(self) -> set[str]:
return {e.strip().lower() for e in self.allowed_emails.split(",") if e.strip()}
+1 -1
View File
@@ -12,7 +12,7 @@ class Base(DeclarativeBase):
# Pool sized above the sync-endpoint threadpool (~40) so a burst of short DB sessions can't starve on
# checkout. The image proxies deliberately DON'T hold a connection during their slow external fetch
# (see routes/plex.py), so this headroom is only for the many brief sessions, not long ones.
# (see routes/plex/), so this headroom is only for the many brief sessions, not long ones.
engine = create_engine(
settings.database_url,
pool_pre_ping=True,
+82
View File
@@ -0,0 +1,82 @@
"""Demo-account service: reset the shared demo sandbox to a clean, seeded baseline.
Shared by the admin's manual reset endpoint (routes/admin.py) and the scheduled demo-reset job
(scheduler.py). Lives in the service layer, not routes, so the scheduler can call it without an
`app.routes -> app.scheduler` import cycle (the reason its import used to be lazy).
"""
from sqlalchemy import delete, func, select
from sqlalchemy.orm import Session
from app.models import Playlist, PlaylistItem, Subscription, User, Video, VideoState
def _seed_demo_playlists(db: Session, demo: User) -> int:
"""Seed a few sample playlists from the shared catalog so a freshly-reset demo isn't
empty — gives visitors something to play with. Best-effort: skips silently if the
catalog is too small."""
recent = (
db.execute(
select(Video.id)
.where(Video.is_short.is_(False), Video.live_status == "none")
.order_by(Video.published_at.desc().nullslast())
.limit(60)
)
.scalars()
.all()
)
if not recent:
return 0
samples = [
("Fresh picks", recent[0:10]),
("Quick watch", recent[10:18]),
("Saved for later", recent[18:26]),
]
created = 0
for pos, (name, vids) in enumerate(samples):
if not vids:
continue
pl = Playlist(user_id=demo.id, name=name, position=pos)
db.add(pl)
db.flush()
for ipos, vid in enumerate(vids):
db.add(PlaylistItem(playlist_id=pl.id, video_id=vid, position=ipos))
created += 1
return created
def _seed_demo_subscriptions(db: Session, demo: User, n: int = 14) -> int:
"""Seed the demo with a handful of catalog channels (the ones with the most videos) so its
Channel manager isn't empty — gives visitors something to explore and matches the landing-page
screenshot. Idempotent: clears the demo's subscriptions first, then re-adds the top N."""
db.execute(delete(Subscription).where(Subscription.user_id == demo.id))
top = (
db.execute(
select(Video.channel_id)
.where(Video.channel_id.is_not(None))
.group_by(Video.channel_id)
.order_by(func.count().desc())
.limit(n)
)
.scalars()
.all()
)
for channel_id in top:
db.add(Subscription(user_id=demo.id, channel_id=channel_id))
return len(top)
def reset_demo_state(db: Session, demo: User) -> int:
"""Wipe the demo account's per-user state (watch/save/hide states, playlists, preferences)
back to a clean baseline and re-seed a few sample playlists + channel subscriptions. Returns
the seeded playlist count. Shared by the admin's manual reset button and the scheduled demo
reset (see scheduler)."""
db.execute(delete(VideoState).where(VideoState.user_id == demo.id))
# Playlist items cascade on playlist delete (ondelete="CASCADE").
db.execute(delete(Playlist).where(Playlist.user_id == demo.id))
demo.preferences = {"language": "en"}
db.commit()
seeded = _seed_demo_playlists(db, demo)
_seed_demo_subscriptions(db, demo)
db.commit()
return seeded
+154
View File
@@ -0,0 +1,154 @@
"""Per-source adapters — a thin Strategy over yt-dlp, keyed by the media source (youtube / facebook
/ reddit / generic).
yt-dlp stays the download engine; an adapter only contributes source-specific POLICY around it. The
base class IS the generic behaviour (yt-dlp does everything, the ref is already a URL), and a
subclass overrides ONLY what its source genuinely needs. This replaces knowledge that used to be
scattered across worker.py / backfill.py / service.py with no shared source identity.
Adapters are stateless singletons; get one with `resolve_adapter(source_kind, source_ref, info)`.
Resolution works both before extraction (from source_kind / URL host) and after (from the yt-dlp
`extractor_key` in the info dict), so a non-YouTube "url" job still lands on the right adapter.
"""
from urllib.parse import urlparse
from app.config import settings
class SourceAdapter:
"""Generic source: yt-dlp handles everything and the ref is already a fetchable URL."""
key = "generic"
# --- extraction (worker._extract_with_fallback / backfill) ---------------------------------
def client_sets(self) -> list:
"""Player-client sets to attempt, in order. A generic source needs no client override, so a
single default attempt (`[None]`) — the per-client fallback is a YouTube-only concept."""
return [None]
def ydl_opts_extra(self, clients) -> dict:
"""yt-dlp options this source contributes (merged into `build_ydl_opts`). `clients` is the
current entry from `client_sets()`. Generic: nothing."""
return {}
def retriable(self, error_text: str) -> bool:
"""Retry the extraction with the NEXT client set on this error? Generic: no (single attempt)."""
return False
# --- metadata -----------------------------------------------------------------------------
def uploader_url(self, info: dict) -> str | None:
"""The channel/uploader page URL. Generic: whatever yt-dlp exposes directly (YouTube gives
`channel_url`), else None (reddit / a raw media URL have no derivable channel page)."""
return info.get("channel_url") or info.get("uploader_url")
def wants_metadata_backfill(self) -> bool:
"""Whether the one-off metadata backfill should re-fetch this source. Generic: no."""
return False
# --- source ref ↔ URL ---------------------------------------------------------------------
def source_url(self, source_ref: str) -> str:
"""A fetchable URL for `source_ref`. Generic: the ref is already a URL."""
return source_ref
def nfo_uniqueid_type(self) -> str:
"""The `<uniqueid type=...>` written into the Kodi/Plex `.nfo` sidecar."""
return self.key
class YouTubeAdapter(SourceAdapter):
key = "youtube"
# Errors worth retrying with a different player-client set (YouTube per-client flakiness).
_RETRIABLE = (
"not a bot",
"DRM protected",
"Requested format is not available",
"Only images are available",
"Sign in to confirm",
)
def client_sets(self) -> list:
return [
c for c in (settings.player_client_list, settings.player_client_fallback_list) if c
] or [None]
def ydl_opts_extra(self, clients) -> dict:
# == --extractor-args "youtube:player_client=…;youtubepot-bgutilhttp:base_url=…" (list values).
ea: dict = {}
if clients:
ea["youtube"] = {"player_client": clients}
if settings.download_pot_base_url:
ea["youtubepot-bgutilhttp"] = {"base_url": [settings.download_pot_base_url]}
return {"extractor_args": ea} if ea else {}
def retriable(self, error_text: str) -> bool:
return any(m in error_text for m in self._RETRIABLE)
def wants_metadata_backfill(self) -> bool:
return True
def source_url(self, source_ref: str) -> str:
return f"https://www.youtube.com/watch?v={source_ref}"
class FacebookAdapter(SourceAdapter):
key = "facebook"
def uploader_url(self, info: dict) -> str | None:
# Facebook gives neither channel_url nor uploader_url, but exposes a numeric `uploader_id`
# (the page id) that resolves at facebook.com/<id>.
base = super().uploader_url(info)
if base:
return base
uid = info.get("uploader_id")
return f"https://www.facebook.com/{uid}" if uid else None
def wants_metadata_backfill(self) -> bool:
return True
class RedditAdapter(SourceAdapter):
key = "reddit"
_YOUTUBE = YouTubeAdapter()
_FACEBOOK = FacebookAdapter()
_REDDIT = RedditAdapter()
_GENERIC = SourceAdapter()
def _host_is(host: str, *domains: str) -> bool:
"""True if `host` is exactly one of `domains` or a subdomain of it — a netloc-scoped match, NOT a
loose substring, so a look-alike host ('myyoutube.com', 'youtube.com.evil') never matches."""
return any(host == d or host.endswith("." + d) for d in domains)
def resolve_adapter(
source_kind: str | None, source_ref: str | None = None, info: dict | None = None
) -> SourceAdapter:
"""Pick the adapter for a job. Matches on `source_kind`, the yt-dlp `extractor_key` from `info`
(post-extraction, authoritative), and the netloc of any URL we hold — the `source_ref`
(pre-extraction) and yt-dlp's `webpage_url` (post-extraction). Host matching is netloc-scoped, and
the webpage_url is used by HOST only (never a substring over its path, which could carry a sibling
source's name). Falls back to the generic adapter — behaviour unchanged for any un-dedicated source."""
extractor = (info.get("extractor_key") or info.get("extractor") or "").lower() if info else ""
hosts = []
if source_ref:
hosts.append(urlparse(source_ref).netloc.lower())
if info and info.get("webpage_url"):
hosts.append(urlparse(info["webpage_url"]).netloc.lower())
def host_is(*domains: str) -> bool:
return any(_host_is(h, *domains) for h in hosts)
if (
source_kind == "youtube"
or "youtube" in extractor
or host_is("youtube.com", "youtu.be", "youtube-nocookie.com")
):
return _YOUTUBE
if "facebook" in extractor or host_is("facebook.com", "fb.watch"):
return _FACEBOOK
if "reddit" in extractor or host_is("reddit.com"):
return _REDDIT
return _GENERIC
+11 -23
View File
@@ -10,41 +10,28 @@ URL may have expired — those just get a locally-generated poster instead).
Run once per environment:
docker exec siftlode-worker-1 python -c "from app.downloads import backfill; print(backfill.run())"
"""
import logging
from urllib.parse import urlparse
import yt_dlp
from sqlalchemy import select
from app.config import settings
from app.db import SessionLocal
from app.downloads import adapters
from app.downloads import edit as editmod
from app.downloads import formats, service
from app.models import MediaAsset
from app.worker import _uploader_url
log = logging.getLogger("siftlode.backfill")
def _should_refetch(asset: MediaAsset) -> bool:
"""Only YouTube + Facebook sources have a derivable channel URL worth a network round-trip."""
if asset.source_kind == "youtube":
return True
if asset.source_kind == "url":
host = urlparse(asset.source_ref).netloc.lower()
return "facebook.com" in host or "fb.watch" in host
return False
def _extract_meta(url: str) -> dict | None:
"""Metadata-only yt-dlp extraction, trying the same client sets as the worker (YouTube needs the
POT/player-client strategy even just to extract)."""
client_sets = [
c for c in (settings.player_client_list, settings.player_client_fallback_list) if c
] or [None]
for clients in client_sets:
def _extract_meta(url: str, adapter: adapters.SourceAdapter) -> dict | None:
"""Metadata-only yt-dlp extraction, using the source adapter's client sets + options (YouTube
needs the POT/player-client strategy even just to extract)."""
for clients in adapter.client_sets():
opts = formats.build_ydl_opts(
{}, "%(id)s.%(ext)s", lambda d: None, settings.download_pot_base_url, clients
{}, "%(id)s.%(ext)s", lambda d: None, extra=adapter.ydl_opts_extra(clients)
)
opts["skip_download"] = True
try:
@@ -70,10 +57,11 @@ def run() -> dict:
.all()
)
for a in assets:
if a.uploader_url is None and _should_refetch(a):
adapter = adapters.resolve_adapter(a.source_kind, a.source_ref)
if a.uploader_url is None and adapter.wants_metadata_backfill():
url = a.source_webpage_url or service.source_url(a.source_kind, a.source_ref)
info = _extract_meta(url)
uu = _uploader_url(info) if info else None
info = _extract_meta(url, adapter)
uu = adapter.uploader_url(info) if info else None
if uu:
a.uploader_url = uu
filled_url += 1
+258
View File
@@ -0,0 +1,258 @@
"""Per-user, per-host download cookies — the single source of truth for "does the acting user's
cookie apply to this download, and how does that keep the result private".
Two call sites depend on ONE consistent answer (house rule: an invariant lives in shared code, not
in prose):
* `enqueue` salts the MediaAsset cache signature with the user id WHEN a cookie applies, so the
authed download becomes a PRIVATE asset that is never cache-served to another user.
* the worker attaches the cookie file to yt-dlp ONLY when a cookie still applies AND the asset it
is downloading into is that private, salted identity — so a cookie can never poison a shared
asset (the load-bearing cross-user privacy fix).
Both go through `cookie_salt` / `authed_format_sig` / `host_key_for_url` here, so they can't drift.
Security controls enforced here:
* **host scope (SEC-12)** — `host_key_for_url` matches the download URL's netloc against a curated
registry via the netloc-scoped `_host_is`, so a cookie is only ever offered to the exact site it
belongs to; an arbitrary `?url=` (or a look-alike host) resolves to no host_key and gets no
cookie.
* **secret handling** — cookies are Fernet-encrypted at rest; the blob is only ever decrypted
in-worker into a 0600 file that is unlinked in a `finally`, and is never returned by a route,
logged, or written to a job's error/snapshot.
"""
import logging
import os
from pathlib import Path
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.downloads import formats
from app.downloads.adapters import _host_is
from app.models import DownloadCredential
from app.security import decrypt_optional, encrypt
log = logging.getLogger("siftlode.downloads.credentials")
class HostSpec:
"""A supported cookie host: a short canonical `key` (stored on the credential + folded into the
cache salt), a human `label` for the UI, and the netloc `domains` it matches."""
__slots__ = ("key", "label", "domains")
def __init__(self, key: str, label: str, domains: tuple[str, ...]):
self.key = key
self.label = label
self.domains = domains
# Curated registry (user's call: seed the common auth-walled hosts now). One `key` can cover several
# domains of the same site. Add a host by adding a row — the FE dropdown, host-scope guard, and salt
# all derive from this, so there is nothing else to wire up.
SUPPORTED_HOSTS: tuple[HostSpec, ...] = (
HostSpec("facebook", "Facebook", ("facebook.com", "fb.watch")),
HostSpec("instagram", "Instagram", ("instagram.com",)),
HostSpec("twitter", "X (Twitter)", ("twitter.com", "x.com")),
HostSpec("tiktok", "TikTok", ("tiktok.com",)),
HostSpec("reddit", "Reddit", ("reddit.com",)),
)
_BY_KEY: dict[str, HostSpec] = {h.key: h for h in SUPPORTED_HOSTS}
def is_supported_host(host_key: str) -> bool:
return host_key in _BY_KEY
def host_label(host_key: str) -> str:
spec = _BY_KEY.get(host_key)
return spec.label if spec else host_key
def host_key_for_url(url: str | None) -> str | None:
"""The supported host_key whose domains own `url`'s netloc, or None. Netloc-scoped (via
`_host_is`), so a look-alike host ('myfacebook.com', 'facebook.com.evil') never matches. When
several specs could match, the most-specific (longest matched domain) wins so the result is
deterministic across the enqueue and worker call sites."""
if not url:
return None
from urllib.parse import urlparse
host = urlparse(url).netloc.lower()
if not host:
return None
best_key: str | None = None
best_len = -1
for spec in SUPPORTED_HOSTS:
for dom in spec.domains:
if _host_is(host, dom) and len(dom) > best_len:
best_key, best_len = spec.key, len(dom)
return best_key
def cookie_salt(user_id: int, host_key: str) -> str:
"""The cache-signature salt for a cookie-authed download. Keyed by user id + host so the authed
asset is private to that user (never dedup-shared) and stable across that user's own re-adds."""
return f"cookie:{user_id}:{host_key}"
def authed_format_sig(spec: dict, user_id: int, host_key: str) -> str:
"""The salted `format_sig` an authed download for (user, host) resolves to — the private asset
identity. Both enqueue (to create it) and the worker (to recognise it) compute it HERE."""
return formats.format_sig(spec, salt=cookie_salt(user_id, host_key))
def salt_for_enqueue(db: Session, user_id: int, url: str | None) -> str | None:
"""At enqueue: the cache salt to apply if this user has a cookie for `url`'s host, else None (a
normal shared download). Returns the salt string so the caller passes it straight to
`format_sig(spec, salt=...)`."""
host_key = host_key_for_url(url)
if host_key is None:
return None
if get_credential(db, user_id, host_key) is None:
return None
return cookie_salt(user_id, host_key)
# --- storage (CRUD) ---------------------------------------------------------------------------
def get_credential(db: Session, user_id: int, host_key: str) -> DownloadCredential | None:
return db.execute(
select(DownloadCredential).where(
DownloadCredential.user_id == user_id, DownloadCredential.host_key == host_key
)
).scalar_one_or_none()
class InvalidCookies(ValueError):
"""The pasted cookies aren't a usable Netscape cookies.txt — surfaced as a 400 by the route."""
def validate_cookies(raw: str) -> str:
"""Sanity-check a pasted cookies.txt (Netscape format) and return the cleaned text. yt-dlp reads
the Netscape format: comment/`#` lines plus TAB-separated cookie lines (domain, flag, path,
secure, expiry, name, value). We don't fully parse — just reject obvious non-cookie paste (empty,
or no tab-separated data line) so a mistaken paste fails HERE with a clear message rather than
silently producing a cookie file yt-dlp ignores."""
text = (raw or "").strip()
if not text:
raise InvalidCookies("empty")
has_data_line = any(
line.strip() and not line.lstrip().startswith("#") and "\t" in line
for line in text.splitlines()
)
if not has_data_line:
raise InvalidCookies("format")
# Netscape cookie files must end with a newline for some parsers; normalise.
return text if text.endswith("\n") else text + "\n"
def upsert_credential(db: Session, user_id: int, host_key: str, raw: str) -> DownloadCredential:
"""Create or replace this user's cookie blob for `host_key`. Caller has validated host_key is
supported and `raw` via `validate_cookies`. Does NOT commit (the route commits with the audit
row so the trail can't drift). The blob is encrypted here; the plaintext never leaves this call."""
cred = get_credential(db, user_id, host_key)
enc = encrypt(raw)
if cred is None:
cred = DownloadCredential(user_id=user_id, host_key=host_key, cookies_enc=enc)
db.add(cred)
else:
cred.cookies_enc = enc
return cred
def delete_credential(db: Session, user_id: int, host_key: str) -> bool:
"""Delete this user's cookie for `host_key`. Returns whether a row existed. Does NOT commit."""
cred = get_credential(db, user_id, host_key)
if cred is None:
return False
db.delete(cred)
return True
def status_list(db: Session, user_id: int) -> list[dict]:
"""The supported hosts with this user's configured/updated state — the write-only Settings view.
Never includes the blob."""
have = {
c.host_key: c
for c in db.execute(
select(DownloadCredential).where(DownloadCredential.user_id == user_id)
).scalars()
}
out = []
for spec in SUPPORTED_HOSTS:
cred = have.get(spec.key)
out.append(
{
"host_key": spec.key,
"label": spec.label,
"configured": cred is not None,
"updated_at": cred.updated_at.isoformat() if cred is not None else None,
}
)
return out
# --- worker: resolve + write the cookie file --------------------------------------------------
def resolve_cookiefile(
db: Session,
user_id: int,
url: str | None,
asset_format_sig: str,
spec: dict,
staging: Path,
) -> Path | None:
"""In the worker child: the cookies.txt to hand yt-dlp for this download, or None.
The double gate that prevents cross-user leakage:
1. a credential must still exist for `url`'s host (host-scoped via `host_key_for_url`), AND
2. the asset being downloaded into must BE this user's private, salted identity
(`asset_format_sig == authed_format_sig(...)`).
If (2) fails the asset is the SHARED (unsalted) one — attaching the cookie would poison the
global cache with a private-group video, so we attach nothing. An unreadable blob (rotated key →
InvalidToken) also yields None; the download then fails honestly on the login wall (S3's
`login_required`)."""
host_key = host_key_for_url(url)
if host_key is None:
return None
cred = get_credential(db, user_id, host_key)
if cred is None:
return None
if asset_format_sig != authed_format_sig(spec, user_id, host_key):
return None # shared asset — never attach a cookie to a non-private identity
raw = decrypt_optional(cred.cookies_enc)
if not raw:
log.warning(
"download cookie for user %s host %s could not be decrypted (rotated key?) — "
"proceeding without it",
user_id,
host_key,
)
return None
return _write_cookiefile(raw, staging)
def _write_cookiefile(raw: str, staging: Path) -> Path:
"""Write the cookie blob to a private 0600 file under `staging/.auth/` and return its path.
Deliberately NOT in the staging root: the download retry loop unlinks every file it finds there,
and `_produced_file` picks the largest file there as the output — either would corrupt a
cookies.txt in the root. A `.auth` SUBDIR survives both (they skip directories) and is still
removed by the parent worker's staging cleanup on a hard kill. The caller unlinks it in a
`finally` on the normal/exception paths."""
auth_dir = staging / ".auth"
auth_dir.mkdir(mode=0o700, exist_ok=True)
path = auth_dir / "cookies.txt"
# Open with 0600 from the start (don't create world-readable then chmod).
fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
try:
os.write(fd, raw.encode("utf-8"))
finally:
os.close(fd)
return path
+271 -61
View File
@@ -20,25 +20,41 @@ Also builds the editor **filmstrip** — a single tiled sprite of the source vid
scrub track. Cached on the source asset's `storyboard_path` (reserved in phase 1) and reusable
for a player hover-preview later.
"""
import hashlib
import json
import logging
import math
import os
import queue
import shutil
import subprocess
import threading
import time
from pathlib import Path
from app.config import settings
log = logging.getLogger("siftlode.edit")
def encode_thread_cap() -> int:
"""How many encoder threads a browser-safe re-encode may use. `download_ffmpeg_threads` when the
admin pinned one (>0), else reserve a core for foreground work (max(1, cpu_count - 1)). Shared by
the ffmpeg `-threads` flag and the child's `OMP_NUM_THREADS` so both honour the same budget."""
n = settings.download_ffmpeg_threads
if n > 0:
return n
return max(1, (os.cpu_count() or 2) - 1)
class EditAborted(Exception):
"""Raised by run_ffmpeg when the job was paused/canceled mid-encode."""
# --- edit spec ------------------------------------------------------------------------------
def normalize_edit_spec(spec: dict | None) -> dict:
"""Canonicalize a raw edit spec: clamp/round trim, coerce crop to ints, drop empties.
@@ -139,6 +155,7 @@ def clip_duration(spec: dict, source_duration: int | None) -> int | None:
# --- ffmpeg: trim / crop --------------------------------------------------------------------
def build_edit_cmd(src: Path, dest: Path, spec: dict, out_ext: str) -> list[str]:
"""ffmpeg command for a trim/crop edit. `-progress pipe:1` streams machine-readable progress.
@@ -152,8 +169,15 @@ def build_edit_cmd(src: Path, dest: Path, spec: dict, out_ext: str) -> list[str]
reencode = needs_reencode(spec)
cmd = [
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-nostdin",
"-progress", "pipe:1", "-nostats",
"ffmpeg",
"-y",
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-progress",
"pipe:1",
"-nostats",
]
if start:
cmd += ["-ss", f"{start:.3f}"]
@@ -169,8 +193,16 @@ def build_edit_cmd(src: Path, dest: Path, spec: dict, out_ext: str) -> list[str]
if crop:
cmd += ["-vf", f"crop={crop['w']}:{crop['h']}:{crop['x']}:{crop['y']}"]
cmd += [
"-c:v", "libx264", "-preset", "veryfast", "-crf", "20",
"-c:a", "aac", "-b:a", "192k",
"-c:v",
"libx264",
"-preset",
"veryfast",
"-crf",
"20",
"-c:a",
"aac",
"-b:a",
"192k",
]
else:
cmd += ["-c", "copy", "-avoid_negative_ts", "make_zero"]
@@ -203,8 +235,15 @@ def build_concat_plan(src: Path, dest: Path, spec: dict, out_ext: str, staging:
crop = spec.get("crop")
prep: dict[Path, str] = {}
head = [
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-nostdin",
"-progress", "pipe:1", "-nostats",
"ffmpeg",
"-y",
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-progress",
"pipe:1",
"-nostats",
]
if needs_reencode(spec):
@@ -217,9 +256,24 @@ def build_concat_plan(src: Path, dest: Path, spec: dict, out_ext: str, staging:
labels.append(f"[v{i}][a{i}]")
parts.append(f"{''.join(labels)}concat=n={len(segs)}:v=1:a=1[v][a]")
cmd = head + [
"-i", str(src), "-filter_complex", ";".join(parts),
"-map", "[v]", "-map", "[a]",
"-c:v", "libx264", "-preset", "veryfast", "-crf", "20", "-c:a", "aac", "-b:a", "192k",
"-i",
str(src),
"-filter_complex",
";".join(parts),
"-map",
"[v]",
"-map",
"[a]",
"-c:v",
"libx264",
"-preset",
"veryfast",
"-crf",
"20",
"-c:a",
"aac",
"-b:a",
"192k",
]
else:
listp = staging / "concat.txt"
@@ -287,23 +341,35 @@ def _drain_tail(stream, sink: list[str]) -> None:
pass
def _stop(proc: subprocess.Popen) -> None:
"""Terminate, then kill — never wait unbounded on a process we've given up on. Safe to call on
a process that already exited (the single exit path below may reach it after a clean wait)."""
def escalate_kill(proc: subprocess.Popen, terminate, kill, grace_s: float, label: str) -> None:
"""terminate → wait(grace) → kill → wait(grace) escalation for a process we've given up on, so it
is never waited on unboundedly. `terminate`/`kill` deliver the stop to whatever scope the caller
owns — a single process (`proc.terminate`/`proc.kill`) or its whole group (`os.killpg`). No-op if
the process already exited."""
if proc.poll() is not None:
return
proc.terminate()
terminate()
try:
proc.wait(timeout=5)
proc.wait(timeout=grace_s)
return
except subprocess.TimeoutExpired:
proc.kill()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
log.error("ffmpeg pid %s survived SIGKILL", proc.pid)
pass
kill()
try:
proc.wait(timeout=grace_s)
except subprocess.TimeoutExpired:
log.error("%s pid %s survived SIGKILL", label, proc.pid)
def run_ffmpeg(cmd, total_s, on_progress, should_cancel, stall_timeout_s=_FFMPEG_STALL_TIMEOUT_S) -> None:
def _stop(proc: subprocess.Popen) -> None:
"""Terminate, then kill — never wait unbounded on a process we've given up on. Safe to call on
a process that already exited."""
escalate_kill(proc, proc.terminate, proc.kill, grace_s=5, label="ffmpeg")
def run_ffmpeg(
cmd, total_s, on_progress, should_cancel, stall_timeout_s=_FFMPEG_STALL_TIMEOUT_S
) -> None:
"""Run an ffmpeg edit, reporting 0–100 progress and honouring cooperative cancel.
Both pipes are drained by threads (see `_drain_tail`) and the main loop polls on a 1s tick, so
@@ -372,18 +438,42 @@ _BROWSER_SAFE_VCODEC = ("h264",)
_BROWSER_SAFE_ACODEC = ("aac", "mp3")
def parse_probe_streams(payload: str) -> tuple[str | None, str | None]:
"""(video_codec, audio_codec) out of ffprobe's JSON stream list, lowercased.
JSON, not the flat `default=nw=1` output: there the fields arrive in ffprobe's own struct
order, which puts `codec_name` BEFORE `codec_type`, so a line-by-line "remember the last
codec_type" parser attributes every name to the PREVIOUS stream. Measured on real downloads:
an ordinary H.264+AAC mp4 probed as `v=aac a=None` — the browser-playability check then
believed the video stream was AAC and re-encoded a perfectly fine file on every download."""
def run_ffprobe_json(
path: Path,
*,
show_streams: bool = False,
show_format: bool = False,
entries: str | None = None,
) -> dict | None:
"""One ffprobe call → its parsed JSON dict, or None on any failure (timeout, non-zero exit,
missing binary, unparseable output). The SINGLE ffprobe-JSON-invocation point in the downloads
package: the browser-playability codec probe here and the shared media probe (`probe.probe_media`,
used by both the download worker's finalize and the upload ingest) both go through it, so the
subprocess flags/timeout/error handling live in one place instead of drifting copies."""
args = ["ffprobe", "-v", "error"]
if show_streams:
args.append("-show_streams")
if show_format:
args.append("-show_format")
if entries:
args += ["-show_entries", entries]
args += ["-of", "json", str(path)]
try:
streams = json.loads(payload or "{}").get("streams") or []
except ValueError:
return None, None
out = subprocess.run(args, capture_output=True, text=True, timeout=30, check=True).stdout
return json.loads(out)
except (subprocess.TimeoutExpired, subprocess.CalledProcessError, OSError, ValueError):
return None
def _codecs_from_streams(streams: list) -> tuple[str | None, str | None]:
"""(video_codec, audio_codec) lowercased from an ffprobe JSON stream list — the first of each kind.
Parsed from the JSON `streams` list, NOT ffprobe's flat `default=nw=1` output: there the fields
arrive in ffprobe's own struct order, which puts `codec_name` BEFORE `codec_type`, so a line-by-line
"remember the last codec_type" parser attributes every name to the PREVIOUS stream. Measured on real
downloads: an ordinary H.264+AAC mp4 probed as `v=aac a=None` — the browser-playability check then
believed the video stream was AAC and re-encoded a perfectly fine file on every download. Reading each
stream dict by its own `codec_type` is immune to that ordering."""
vcodec = acodec = None
for s in streams:
name = (s.get("codec_name") or "").lower() or None
@@ -398,15 +488,10 @@ def parse_probe_streams(payload: str) -> tuple[str | None, str | None]:
def probe_codecs(path: Path) -> tuple[str | None, str | None]:
"""Return (video_codec, audio_codec) lowercased via one ffprobe call; None for a missing
stream or if ffprobe fails (a failed probe is treated as 'leave it alone' upstream)."""
try:
out = subprocess.run(
["ffprobe", "-v", "error", "-show_entries", "stream=codec_type,codec_name",
"-of", "json", str(path)],
capture_output=True, text=True, timeout=30, check=True,
).stdout
except (subprocess.TimeoutExpired, subprocess.CalledProcessError, OSError):
data = run_ffprobe_json(path, entries="stream=codec_type,codec_name")
if data is None:
return None, None
return parse_probe_streams(out)
return _codecs_from_streams(data.get("streams") or [])
def probe_duration(path: Path) -> int:
@@ -414,16 +499,29 @@ def probe_duration(path: Path) -> int:
re-encode when the yt-dlp info dict carried no `duration` (some URL/live extractions)."""
try:
out = subprocess.run(
["ffprobe", "-v", "error", "-show_entries", "format=duration",
"-of", "default=nw=1:nk=1", str(path)],
capture_output=True, text=True, timeout=30, check=True,
[
"ffprobe",
"-v",
"error",
"-show_entries",
"format=duration",
"-of",
"default=nw=1:nk=1",
str(path),
],
capture_output=True,
text=True,
timeout=30,
check=True,
).stdout.strip()
return int(float(out)) if out else 0
except (subprocess.TimeoutExpired, subprocess.CalledProcessError, OSError, ValueError):
return 0
def browser_transcode_flags(vcodec: str | None, acodec: str | None, is_mp4: bool) -> tuple[bool, bool, bool]:
def browser_transcode_flags(
vcodec: str | None, acodec: str | None, is_mp4: bool
) -> tuple[bool, bool, bool]:
"""Decide what (if anything) makes a file universally browser-playable. Returns
(needs_change, reencode_video, reencode_audio). needs_change=False => already H.264 + AAC/MP3
in mp4. A None codec (no such stream, or a failed probe) is never re-encoded."""
@@ -432,16 +530,34 @@ def browser_transcode_flags(vcodec: str | None, acodec: str | None, is_mp4: bool
return (rv or ra or not is_mp4), rv, ra
def build_browser_safe_cmd(src: Path, dest: Path, reencode_video: bool, reencode_audio: bool) -> list[str]:
def build_browser_safe_cmd(
src: Path, dest: Path, reencode_video: bool, reencode_audio: bool, threads: int | None = None
) -> list[str]:
"""ffmpeg command that lands `src` as a browser-safe mp4 (faststart): re-encode only the
stream(s) that need it (libx264 / aac), stream-copy the rest, always into mp4. `-progress
pipe:1` streams machine-readable progress for run_ffmpeg."""
pipe:1` streams machine-readable progress for run_ffmpeg. `threads` caps the encoder so a
re-encode can't claim every core on the shared box (see the worker-child de-prioritization)."""
cmd = [
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-nostdin",
"-progress", "pipe:1", "-nostats", "-i", str(src),
"ffmpeg",
"-y",
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-progress",
"pipe:1",
"-nostats",
"-i",
str(src),
]
cmd += ["-c:v", "libx264", "-preset", "veryfast", "-crf", "20"] if reencode_video else ["-c:v", "copy"]
cmd += (
["-c:v", "libx264", "-preset", "veryfast", "-crf", "20"]
if reencode_video
else ["-c:v", "copy"]
)
cmd += ["-c:a", "aac", "-b:a", "192k"] if reencode_audio else ["-c:a", "copy"]
if threads and threads > 0:
cmd += ["-threads", str(threads)]
cmd += ["-movflags", "+faststart", str(dest)]
return cmd
@@ -475,20 +591,116 @@ def build_storyboard_cmd(src: Path, dest: Path, duration_s: int | None, geom: di
fps = geom["count"] / dur
vf = f"fps={fps:.6f},scale={_SB_TILE_W}:-2,tile={geom['cols']}x{geom['rows']}"
return [
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-nostdin",
"-i", str(src), "-frames:v", "1", "-q:v", "5", "-vf", vf, str(dest),
"ffmpeg",
"-y",
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-i",
str(src),
"-frames:v",
"1",
"-q:v",
"5",
"-vf",
vf,
str(dest),
]
def build_poster_cmd(src: Path, dest: Path, at_s: float) -> list[str]:
"""Grab one representative frame as a poster JPEG. `-ss` before `-i` seeks fast."""
return [
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error", "-nostdin",
"-ss", f"{max(0.0, at_s):.3f}", "-i", str(src),
"-frames:v", "1", "-q:v", "3", "-vf", "scale=640:-2", str(dest),
"ffmpeg",
"-y",
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-ss",
f"{max(0.0, at_s):.3f}",
"-i",
str(src),
"-frames:v",
"1",
"-q:v",
"3",
"-vf",
"scale=640:-2",
str(dest),
]
def build_thumbnail_jpg_cmd(src: Path, dest: Path) -> list[str]:
"""Decode a downloaded thumbnail (webp/png/…) to a single baseline JPEG frame."""
return [
"ffmpeg",
"-y",
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-i",
str(src),
"-frames:v",
"1",
"-update",
"1",
str(dest),
]
def _run_ffmpeg_best_effort(cmd: list[str], timeout_s: int) -> Exception | None:
"""Run a one-shot ffmpeg command, bounded by `timeout_s`. Returns None on success, or the
exception on a timeout/non-zero-exit/OS error — never raises. Shared by the best-effort image
steps (poster / thumbnail / storyboard) so the run+timeout+swallow policy lives in ONE place;
each caller logs its own context from the returned exception. (The download/transcode path uses
the richer `run_ffmpeg` instead, which streams progress and honours cooperative cancel.)"""
try:
subprocess.run(cmd, capture_output=True, timeout=timeout_s, check=True)
return None
except (subprocess.TimeoutExpired, subprocess.CalledProcessError, OSError) as exc:
return exc
def _is_jpeg(path: Path) -> bool:
"""True if the file starts with the JPEG SOI marker (FF D8 FF)."""
try:
with path.open("rb") as f:
return f.read(3) == b"\xff\xd8\xff"
except OSError:
return False
def thumbnail_to_jpg(src: Path | None, dest: Path, timeout_s: int = 30) -> Path | None:
"""Turn a downloaded thumbnail (webp / png / jpeg — whatever the source shipped) into a real
baseline JPEG for the `<name>.jpg` poster + og:image share sidecar.
Deliberately NOT yt-dlp's `FFmpegThumbnailsConvertor` (whose `-bsf:v mjpeg2jpeg` path crashes on
some already-JPEG thumbnails, e.g. Reddit's, and used to fail the entire download). A thumbnail
that is already JPEG is copied as-is; anything else is decoded via ffmpeg. Best-effort: returns
None on any failure, so the caller falls back to a cut video frame (`ensure_poster`) and the
download still succeeds."""
if src is None or not src.exists():
return None
if _is_jpeg(src): # already JPEG (e.g. Facebook) — no decode/re-encode needed
try:
shutil.copyfile(src, dest)
return dest
except OSError as exc:
log.info("thumbnail copy failed (%s); poster falls back to a cut frame", str(exc)[:120])
return None
err = _run_ffmpeg_best_effort(build_thumbnail_jpg_cmd(src, dest), timeout_s)
if err is not None:
log.info(
"thumbnail->jpg conversion failed (%s); poster falls back to a cut frame",
str(err)[:120],
)
return None
return dest if dest.exists() else None
def ensure_poster(asset, download_root: str, timeout_s: int = 30) -> str | None:
"""Generate a poster frame for a ready asset that has no thumbnail (e.g. a direct media URL
like a reddit HLS manifest, which yt-dlp can't get a thumbnail for). Written as the media's
@@ -506,10 +718,9 @@ def ensure_poster(asset, download_root: str, timeout_s: int = 30) -> str | None:
if dest.exists():
return rel
at = min(3.0, (asset.duration_s or 10) * 0.1)
try:
subprocess.run(build_poster_cmd(src, dest, at), capture_output=True, timeout=timeout_s, check=True)
except (subprocess.TimeoutExpired, subprocess.CalledProcessError, OSError) as exc:
log.info("poster gen skipped for asset %s: %s", asset.id, str(exc)[:120])
err = _run_ffmpeg_best_effort(build_poster_cmd(src, dest, at), timeout_s)
if err is not None:
log.info("poster gen skipped for asset %s: %s", asset.id, str(err)[:120])
return None
return rel if dest.exists() else None
@@ -538,10 +749,9 @@ def ensure_storyboard(asset, download_root: str, timeout_s: int = 90) -> dict |
return None
dest.parent.mkdir(parents=True, exist_ok=True)
cmd = build_storyboard_cmd(src, dest, asset.duration_s, geom)
try:
subprocess.run(cmd, capture_output=True, timeout=timeout_s, check=True)
except (subprocess.TimeoutExpired, subprocess.CalledProcessError, OSError) as exc:
log.info("storyboard gen skipped for asset %s: %s", asset.id, str(exc)[:120])
err = _run_ffmpeg_best_effort(cmd, timeout_s)
if err is not None:
log.info("storyboard gen skipped for asset %s: %s", asset.id, str(err)[:120])
return None
if not dest.exists():
return None
+17
View File
@@ -21,6 +21,19 @@ _RULES: tuple[tuple[str, str], ...] = (
("available to this channel's members", "login_required"),
("join this channel", "login_required"),
("this video is only available to", "login_required"),
# Some extractors (e.g. Reddit) gate content behind a logged-in account: "Account
# authentication is required." — an auth wall we deliberately don't cross, not a transient error.
("authentication is required", "login_required"),
# Closed-group / private-post auth walls (Facebook groups, Instagram private, …). yt-dlp phrases
# these many ways; the common thread is "you must be signed in" or its own suggestion to supply
# cookies. Treating the cookie suggestion as an auth signal is what makes a login-walled link read
# honestly (and stay non-retryable) when the acting user has no usable cookie for that host.
("log in to", "login_required"),
("login required", "login_required"),
("you must be logged in", "login_required"),
("only available for registered users", "login_required"),
("use --cookies", "login_required"),
("--cookies-from-browser", "login_required"),
# Geo-restricted.
("in your country", "geo_blocked"),
("available in your location", "geo_blocked"),
@@ -39,6 +52,10 @@ _RULES: tuple[tuple[str, str], ...] = (
("account associated with this video has been terminated", "unavailable"),
("this video is not available", "unavailable"),
("this video has been removed", "unavailable"),
# yt-dlp doesn't support this site at all ("Unsupported URL: …") — distinct from an auth wall on a
# supported site. Non-retryable: no cookie or retry can make an unsupported host downloadable.
("unsupported url", "unsupported_host"),
("no video could be found in this url", "unsupported_host"),
# DRM.
("drm protected", "drm"),
("drm-protected", "drm"),
+75 -67
View File
@@ -5,14 +5,18 @@ of truth for what file gets produced. `format_sig` hashes exactly the fields tha
OUTPUT BYTES, so two users picking equivalent settings share one cached MediaAsset.
`build_ydl_opts` turns a spec into a yt-dlp options dict (format selector + postprocessors).
spec shape (all keys optional; defaults applied by `normalize`):
spec shape (all keys optional; defaults + browser-playability clamp applied by `normalize`):
mode "av" (audio+video) | "v" (video-only) | "a" (audio-only)
max_height int | null (null = best)
container "mp4" | "mkv" | "webm" | null (merge/remux target for av/v)
audio_format "m4a" | "mp3" | "opus" | null (extract target for mode "a")
vcodec "h264" | "vp9" | "av1" | null (preference, best-effort)
container always coerced to "mp4" (the only universally browser-playable container)
audio_format "m4a" | "mp3" (mode "a" extract target; anything else — e.g. opus — → "m4a")
vcodec always coerced to null → H.264 selected (see `normalize`); no opt-out
embed_subs bool embed_chapters bool embed_thumbnail bool sponsorblock bool
`normalize` clamps container/vcodec/audio_format so EVERY download plays in-browser (E7); the
`container`/`vcodec` keys survive only so an old stored profile deserializes without error.
"""
import hashlib
import json
@@ -49,10 +53,12 @@ _DEFAULTS = {
# prefer H.264+AAC (browser/iOS-compatible) rather than yt-dlp's default VP9/Opus best.
# v3: mp4 compat now *selects* H.264 (avc1) with a progressive fallback + re-encodes the rare
# AV1/VP9-only result to H.264, so cached AV1 assets must re-derive (fixes iOS broken-player).
_SIG_VERSION = 3
# v4: EVERY download is clamped to the mp4/H.264/AAC compat profile (no more mkv/webm/VP9/AV1/Opus
# opt-out), so any asset cached under an old non-compat spec — or a stale non-H.264 mp4 that slipped
# the v3 net — must re-derive through the current browser-safe guarantee.
_SIG_VERSION = 4
_SPONSORBLOCK_CATEGORIES = ["sponsor", "selfpromo", "interaction"]
_VCODEC_SORT = {"h264": "vcodec:h264", "vp9": "vcodec:vp9", "av1": "vcodec:av01"}
def normalize(spec: dict) -> dict:
@@ -75,6 +81,18 @@ def normalize(spec: dict) -> dict:
# Audio-only files can't embed subtitles/thumbnails as video; keep the flags meaningful.
if s["mode"] == "a":
s["embed_subs"] = False
# Browser-playability guarantee (E7): the Download module exists to download → share → PLAY
# in-browser, so every output must play in EVERY browser, iOS/Safari/WebKit included. The only
# universally-safe combo is mp4 + H.264 + AAC/MP3, so we CLAMP the spec to it HERE — the single
# point every path (picker, stored profile, raw API) flows through — instead of only hiding
# options in the UI. Non-mp4 containers (mkv doesn't play at all; webm/VP9/AV1 fail iOS) and any
# explicit vcodec are coerced to the compat profile; Opus audio (Safari-flaky as a bare file)
# falls back to AAC/m4a. A source that exposes no H.264 is still caught by the worker's
# browser-safe re-encode (`_ensure_browser_playable`), so playback is guaranteed end-to-end.
s["container"] = "mp4"
s["vcodec"] = None
if s["audio_format"] not in ("m4a", "mp3"):
s["audio_format"] = "m4a"
return s
@@ -96,13 +114,18 @@ def build_ydl_opts(
spec: dict,
outtmpl: str,
progress_hook,
pot_base_url: str | None = None,
player_clients: list[str] | None = None,
extra: dict | None = None,
cookiefile: str | None = None,
) -> dict:
"""Assemble yt-dlp options for a single download to `outtmpl` (a full path template).
`pot_base_url` points the bgutil PO-token provider plugin at the sidecar server;
`player_clients` picks which YouTube clients to use (POT-capable ones like web use the token)."""
`extra` is source-specific options contributed by the SourceAdapter (e.g. YouTube's
`extractor_args` with the player-client set + bgutil PO-token base_url); it is merged onto the
generic options below. Everything here is source-agnostic.
`cookiefile` is the acting user's per-host cookies.txt path (S6), attached ONLY by the worker
after the host-scope + private-asset gate in `downloads.credentials`. It is set AFTER the `extra`
merge so a source adapter can never clobber it."""
s = normalize(spec)
h = s["max_height"]
opts: dict = {
@@ -127,16 +150,14 @@ def build_ydl_opts(
"socket_timeout": 30,
}
# Extractor args: which YouTube clients to use + where the bgutil PO-token sidecar lives
# (== --extractor-args "youtube:player_client=…;youtubepot-bgutilhttp:base_url=…"). Values
# are lists in the Python API.
ea: dict = {}
if player_clients:
ea["youtube"] = {"player_client": player_clients}
if pot_base_url:
ea["youtubepot-bgutilhttp"] = {"base_url": [pot_base_url]}
if ea:
opts["extractor_args"] = ea
# Source-specific options from the adapter (e.g. YouTube's extractor_args: player_client + POT).
if extra:
opts.update(extra)
# The acting user's per-host cookie (S6). AFTER the adapter merge on purpose: an adapter must not
# be able to override the authenticated session the worker resolved for this specific user.
if cookiefile:
opts["cookiefile"] = cookiefile
if s["mode"] == "a":
opts["format"] = "bestaudio/best"
@@ -149,51 +170,33 @@ def build_ydl_opts(
)
else:
hcap = f"[height<=?{h}]" if h else ""
# Compatibility-first for mp4 (the universal container): produce H.264 video + AAC audio
# so shared/downloaded files play in EVERY browser, including iOS/Safari. WebKit (which
# every iOS browser, Brave included, is forced to use) decodes only H.264/H.265 + AAC
# inside mp4 — a VP9/AV1-in-mp4 file shows a broken player there, even though Chromium
# plays it fine. A custom profile that explicitly sets `vcodec` opts out.
compat = s["container"] == "mp4" and s["vcodec"] is None
# Every video download is the mp4/H.264 compat profile (normalize() clamps container→mp4,
# vcodec→None), so shared/downloaded files play in EVERY browser, iOS/Safari included. WebKit
# (which every iOS browser, Brave included, is forced to use) decodes only H.264/H.265 + AAC
# inside mp4 — a VP9/AV1-in-mp4 file shows a broken player there — so H.264 is selected at the
# SELECTION level, not merely preferred: a sort key can't conjure an H.264 stream a source
# doesn't expose as video-only. A source that ships only non-H.264 is caught after download by
# the worker's browser-safe re-encode (`_ensure_browser_playable`).
if s["mode"] == "v":
if compat:
# Video-only: keep it audio-free. Prefer an H.264 *video-only* stream, then any
# video-only, and only fall to a progressive (audio-bearing) `best` as a last
# resort — same audio-free-when-possible guarantee as the old `bestvideo/best`.
opts["format"] = (
f"bestvideo[vcodec^=avc1]{hcap}/bestvideo{hcap}"
f"/best[vcodec^=avc1]{hcap}/best{hcap}"
)
else:
opts["format"] = f"bestvideo{hcap}/best{hcap}"
# Video-only: keep it audio-free. Prefer an H.264 video-only stream, then any video-only,
# and only fall to a progressive (audio-bearing) `best` as a last resort.
opts["format"] = (
f"bestvideo[vcodec^=avc1]{hcap}/bestvideo{hcap}/best[vcodec^=avc1]{hcap}/best{hcap}"
)
else: # av
if compat:
# Prefer H.264 at the SELECTION level, not just via format_sort: a sort key is only
# a *preference* and can't conjure an H.264 stream a source doesn't expose as
# video-only. A non-YouTube source (e.g. Facebook) may ship AV1 as the best
# video-only stream and H.264 only as a *progressive* (a+v) format — which
# `bestvideo` skips — so `bestvideo+bestaudio` picked AV1 and iOS broke. This chain
# tries avc1 video-only, then the avc1 progressive, then anything.
opts["format"] = (
f"bestvideo[vcodec^=avc1]{hcap}+bestaudio[acodec^=mp4a]"
f"/best[vcodec^=avc1]{hcap}"
f"/bestvideo{hcap}+bestaudio/best{hcap}"
)
else:
opts["format"] = f"bestvideo{hcap}+bestaudio/best{hcap}"
if s["container"]:
opts["merge_output_format"] = s["container"]
sort = []
# Keep the codec preference as a tiebreaker (ranks H.264/AAC above resolution among the
# selected candidates); the selector above is the hard guarantee, this is belt-and-braces.
if compat:
sort += ["vcodec:h264", "acodec:aac"]
# avc1 video-only + m4a audio, then the avc1 progressive, then anything (re-encoded).
opts["format"] = (
f"bestvideo[vcodec^=avc1]{hcap}+bestaudio[acodec^=mp4a]"
f"/best[vcodec^=avc1]{hcap}"
f"/bestvideo{hcap}+bestaudio/best{hcap}"
)
opts["merge_output_format"] = "mp4"
# Codec preference tiebreaker (ranks H.264/AAC among the selected candidates); the selector
# above is the hard guarantee, this is belt-and-braces.
sort = ["vcodec:h264", "acodec:aac"]
if h:
sort.append(f"res:{h}")
if s["vcodec"] in _VCODEC_SORT:
sort.append(_VCODEC_SORT[s["vcodec"]])
if sort:
opts["format_sort"] = sort
opts["format_sort"] = sort
# SponsorBlock first (marks segments), then strip them from the media + chapters.
if s["sponsorblock"]:
@@ -211,12 +214,17 @@ def build_ydl_opts(
opts["postprocessors"].append(
{"key": "FFmpegMetadata", "add_chapters": True, "add_metadata": True}
)
# Normalize the written thumbnail to jpg so we always have a `<name>.jpg` poster sidecar.
# `already_have_thumbnail` keeps that file on disk after embedding (default deletes it).
opts["postprocessors"].append({"key": "FFmpegThumbnailsConvertor", "format": "jpg"})
# NOTE: we deliberately do NOT add yt-dlp's `FFmpegThumbnailsConvertor` here. Its jpg conversion
# runs `ffmpeg -bsf:v mjpeg2jpeg` when the written thumbnail is already JPEG bytes (Reddit serves
# a JPEG under a `.png` name), and that bitstream filter *fails* on some such images — and since a
# postprocessor error is fatal, it used to kill the WHOLE download even though the media was fine.
# The `<name>.jpg` poster sidecar is instead produced robustly by our own pipeline (worker
# `_download` → `editmod.thumbnail_to_jpg`, with `ensure_poster` cutting a video frame as a
# fallback), so the og:image share preview (Facebook Messenger / MS Teams) keeps working.
# The opt-in EmbedThumbnail below is unaffected: it does its OWN format handling — it embeds a
# jpg/png thumbnail directly and converts anything else to PNG (not the crashing mjpeg2jpeg-to-jpg
# path), so it never needed the convertor and does not reintroduce the crash.
if s["embed_thumbnail"]:
opts["postprocessors"].append(
{"key": "EmbedThumbnail", "already_have_thumbnail": True}
)
opts["postprocessors"].append({"key": "EmbedThumbnail", "already_have_thumbnail": True})
return opts
+73 -35
View File
@@ -15,6 +15,7 @@ Five passes each run:
Runs in the API process (which mounts DOWNLOAD_ROOT). Pure disk/DB work — no YouTube quota.
"""
import logging
import time
from datetime import datetime, timedelta, timezone
@@ -40,9 +41,7 @@ _ORPHAN_MIN_AGE_S = 3600 # only sweep files older than this, so an in-flight do
def _holders(db: Session, asset_id: int) -> list[int]:
return list(
db.execute(
select(DownloadJob.user_id)
.where(DownloadJob.asset_id == asset_id)
.distinct()
select(DownloadJob.user_id).where(DownloadJob.asset_id == asset_id).distinct()
).scalars()
)
@@ -87,11 +86,25 @@ def _sweep_orphan_files(db: Session, download_root: str) -> int:
cutoff = time.time() - _ORPHAN_MIN_AGE_S
removed = 0
# Throttle the whole-tree walk: this runs in the API process, and an unpaced rglob+stat over a
# large download root monopolizes disk IO long enough to make web requests + playback stutter.
# Yield the IO channel after every `batch` files touched so foreground work interleaves.
pause_s = max(0, settings.download_gc_sweep_pause_ms) / 1000.0
batch = max(1, settings.download_gc_sweep_batch)
touched = 0
def _throttle() -> None:
nonlocal touched
touched += 1
if pause_s and touched % batch == 0:
time.sleep(pause_s)
for entry in root.iterdir():
if entry.name.startswith("."): # .edits / .plex-* / .staging / .storyboards + any dotfile
continue
files = [entry] if entry.is_file() else [p for p in entry.rglob("*") if p.is_file()]
for f in files:
_throttle()
try:
if f.resolve() in keep or f.stat().st_mtime > cutoff:
continue
@@ -101,7 +114,9 @@ def _sweep_orphan_files(db: Session, download_root: str) -> int:
pass
if entry.is_dir(): # prune emptied dirs (deepest first), then the channel dir itself
for d in sorted(
(p for p in entry.rglob("*") if p.is_dir()), key=lambda p: len(p.parts), reverse=True
(p for p in entry.rglob("*") if p.is_dir()),
key=lambda p: len(p.parts),
reverse=True,
):
try:
d.rmdir()
@@ -121,15 +136,19 @@ def run_download_gc(db: Session) -> dict:
# 1. Pre-expiry warnings.
warn_before = now + timedelta(days=grace_days)
soon = db.execute(
select(MediaAsset).where(
MediaAsset.status == "ready",
MediaAsset.gc_notified.is_(False),
MediaAsset.expires_at.is_not(None),
MediaAsset.expires_at > now,
MediaAsset.expires_at <= warn_before,
soon = (
db.execute(
select(MediaAsset).where(
MediaAsset.status == "ready",
MediaAsset.gc_notified.is_(False),
MediaAsset.expires_at.is_not(None),
MediaAsset.expires_at > now,
MediaAsset.expires_at <= warn_before,
)
)
).scalars().all()
.scalars()
.all()
)
for asset in soon:
_notify_holders(db, asset, "download_expiring")
asset.gc_notified = True
@@ -138,13 +157,17 @@ def run_download_gc(db: Session) -> dict:
db.commit()
# 2. Expiry deletion (ready assets past TTL, and any errored leftovers with a TTL).
dead = db.execute(
select(MediaAsset).where(
MediaAsset.status.in_(("ready", "error")),
MediaAsset.expires_at.is_not(None),
MediaAsset.expires_at <= now,
dead = (
db.execute(
select(MediaAsset).where(
MediaAsset.status.in_(("ready", "error")),
MediaAsset.expires_at.is_not(None),
MediaAsset.expires_at <= now,
)
)
).scalars().all()
.scalars()
.all()
)
for asset in dead:
_delete_asset(db, asset, "download_expired")
expired += 1
@@ -154,17 +177,24 @@ def run_download_gc(db: Session) -> dict:
# 3. LRU eviction over the total cache cap.
cap = sysconfig.get_int(db, "download_total_max_bytes")
if cap > 0:
total = db.execute(
select(func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(
MediaAsset.status == "ready"
)
).scalar() or 0
total = (
db.execute(
select(func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(
MediaAsset.status == "ready"
)
).scalar()
or 0
)
if total > cap:
candidates = db.execute(
select(MediaAsset)
.where(MediaAsset.status == "ready")
.order_by(MediaAsset.last_access_at.asc().nulls_first())
).scalars().all()
candidates = (
db.execute(
select(MediaAsset)
.where(MediaAsset.status == "ready")
.order_by(MediaAsset.last_access_at.asc().nulls_first())
)
.scalars()
.all()
)
for asset in candidates:
if total <= cap:
break
@@ -176,13 +206,17 @@ def run_download_gc(db: Session) -> dict:
# 4. Reap orphaned errored rows (no file, no holder). get_or_create_asset resets a still-
# referenced errored row to pending on re-enqueue, so these only accumulate once every
# holder is gone; they carry no expires_at, so passes 1-3 never touch them.
orphans = db.execute(
select(MediaAsset).where(
MediaAsset.status == "error",
MediaAsset.ref_count == 0,
MediaAsset.rel_path.is_(None),
orphans = (
db.execute(
select(MediaAsset).where(
MediaAsset.status == "error",
MediaAsset.ref_count == 0,
MediaAsset.rel_path.is_(None),
)
)
).scalars().all()
.scalars()
.all()
)
for asset in orphans:
db.delete(asset) # no file to unlink; FK SET NULL clears any stray job.asset_id
reaped += 1
@@ -194,7 +228,11 @@ def run_download_gc(db: Session) -> dict:
swept = _sweep_orphan_files(db, settings.download_root)
result = {
"warned": warned, "expired": expired, "evicted": evicted, "reaped": reaped, "swept": swept
"warned": warned,
"expired": expired,
"evicted": evicted,
"reaped": reaped,
"swept": swept,
}
log.info("download_gc %s", result)
return result
+75
View File
@@ -0,0 +1,75 @@
"""Short-lived-session DB helpers + staging paths shared by the worker parent and the child pipeline."""
from pathlib import Path
from sqlalchemy import text
from app.config import settings
from app.db import SessionLocal
from app.models import DownloadJob, MediaAsset
# --- small DB helpers (short-lived sessions; the download itself holds no transaction) -------
def _set_job(job_id: int, **fields) -> None:
with SessionLocal() as db:
job = db.get(DownloadJob, job_id)
if job is None:
return
for k, v in fields.items():
setattr(job, k, v)
db.commit()
def _job_status(job_id: int) -> str | None:
with SessionLocal() as db:
return db.execute(
text("SELECT status FROM download_jobs WHERE id = :id"), {"id": job_id}
).scalar()
def _touch_job(job_id: int) -> None:
"""Bump a running job's `updated_at` without changing any value — a stall-watchdog heartbeat for
phases that legitimately write no progress. Extraction writes nothing, and a multi-client-set
fallback re-writes identical values (progress 0 / phase 'downloading'), so ORM `onupdate` would
NOT fire and two slow extraction attempts could otherwise cross the stall timeout and be killed
mid-extraction though they'd have succeeded."""
with SessionLocal() as db:
db.execute(
text("UPDATE download_jobs SET updated_at=now() WHERE id=:id AND status='running'"),
{"id": job_id},
)
db.commit()
def _requeue_if_running(job_id: int, phase: str) -> None:
"""Put the job back to 'queued' for a later retry, but ONLY while it is still 'running'. The
route flips status to paused/cancelled to stop a job; a blind `status='queued'` here would
clobber that and silently revive a job the user just cancelled. The conditional WHERE makes the
requeue a no-op unless we still own a running job."""
with SessionLocal() as db:
db.execute(
text(
"UPDATE download_jobs SET status='queued', phase=:phase, updated_at=now() "
"WHERE id=:id AND status='running'"
),
{"id": job_id, "phase": phase},
)
db.commit()
def _reset_asset_pending(asset_id: int) -> None:
with SessionLocal() as db:
asset = db.get(MediaAsset, asset_id)
if asset and asset.status == "downloading":
asset.status = "pending"
db.commit()
def _staging_dir(asset_id: int) -> Path:
return Path(settings.download_root) / ".staging" / f"asset-{asset_id}"
def _edit_staging_dir(asset_id: int) -> Path:
return Path(settings.download_root) / ".staging" / f"edit-{asset_id}"
+16 -5
View File
@@ -6,6 +6,7 @@ require a short-lived signed **grant** on the file request — because a `<video
header, the password is exchanged once at `/unlock` for an HMAC grant appended to the file URL,
so the raw password never rides in the media URL (or server logs).
"""
import hashlib
import hmac
import secrets
@@ -27,10 +28,9 @@ def is_expired(link: DownloadLink) -> bool:
# --- signed grants (password-protected file access) ----------------------------------------
def _sign(msg: str) -> str:
return hmac.new(
settings.secret_key.encode(), msg.encode(), hashlib.sha256
).hexdigest()[:32]
return hmac.new(settings.secret_key.encode(), msg.encode(), hashlib.sha256).hexdigest()[:32]
def make_grant(token: str, password_version: int, now: float | None = None) -> str:
@@ -58,6 +58,7 @@ def check_grant(token: str, grant: str | None, password_version: int) -> bool:
# --- serialization -------------------------------------------------------------------------
def owner_view(link: DownloadLink, base_url: str = "") -> dict:
"""What the file owner sees when managing their links."""
return {
@@ -72,14 +73,20 @@ def owner_view(link: DownloadLink, base_url: str = "") -> dict:
}
def public_meta(link: DownloadLink, job, asset, file_url: str, poster_url: str | None = None) -> dict:
def public_meta(
link: DownloadLink, job, asset, file_url: str, poster_url: str | None = None
) -> dict:
"""Metadata for the public watch page (only ever returned once access is authorized).
Resolves the owner's per-download display overrides (custom title/channel) over the asset's
auto-extracted values, and surfaces the source + any extra reference links as clickable URLs."""
from app.downloads import service
from app.downloads import mediakind, service
source_url = service.reference_url(job, asset)
# Audio-only assets (mp3 rips, extracted audio) can't play inside a <video> element on WebKit
# (all iOS browsers), so the watch page renders a native <audio> instead. A generic uploaded
# file (media_kind='file') has no player at all — the watch page shows file info + a download.
# Both branches read the stored media_kind (the single source), not a re-derived codec check.
return {
"title": (job and job.display_name) or asset.title,
"uploader": (job and job.display_uploader) or asset.uploader,
@@ -89,6 +96,10 @@ def public_meta(link: DownloadLink, job, asset, file_url: str, poster_url: str |
"duration_s": asset.duration_s,
"width": asset.width,
"height": asset.height,
"audio": asset.media_kind == mediakind.AUDIO,
"media_kind": asset.media_kind,
"size_bytes": asset.size_bytes,
"container": asset.container,
"allow_download": link.allow_download,
"file_url": file_url,
"poster_url": poster_url,
+23
View File
@@ -0,0 +1,23 @@
"""The media-kind discriminator for a library asset — the single home of one invariant.
`MediaAsset.media_kind` is `"video" | "audio" | "file"`. It decides which library actions apply:
media (video/audio) gets play/edit/share; a generic uploaded file gets download + share only, and
is never rendered inline (see the force-download serving in the file/watch routes). Every finalize
path computes it HERE — the download worker + the upload ingest via `derive`, the editor by
carrying the source's kind over — so the rule lives in code, not in a comment that can drift (an
earlier ad-hoc `vcodec is None and acodec is not None` audio check lived only in links.py).
"""
VIDEO = "video"
AUDIO = "audio"
FILE = "file"
def derive(vcodec: str | None, acodec: str | None) -> str:
"""Media kind from the probed codecs: a video stream -> video, audio-only -> audio, neither
(a non-media upload: pdf/zip/docx/…) -> file."""
if vcodec:
return VIDEO
if acodec:
return AUDIO
return FILE
+2 -3
View File
@@ -6,6 +6,7 @@ This injects per-video OG/Twitter tags into the served HTML so a shared link unf
video's title, channel and thumbnail. Real browsers ignore the extra tags and hydrate the SPA as
usual. Password-protected / expired / invalid links fall back to the generic card (no metadata leak).
"""
import html
import re
from pathlib import Path
@@ -71,9 +72,7 @@ def _jpeg_size(path: Path) -> tuple[int, int] | None:
def _watch_tags(token: str, db: Session) -> str | None:
"""Build the OG/Twitter meta block for a watch token, or None to keep the generic card."""
link = db.execute(
select(DownloadLink).where(DownloadLink.token == token)
).scalar_one_or_none()
link = db.execute(select(DownloadLink).where(DownloadLink.token == token)).scalar_one_or_none()
if link is None or linksmod.is_expired(link):
return None
url = f"{settings.app_base}/watch/{token}"

Some files were not shown because too many files have changed in this diff Show More