The demo account is shared, so writing deep_requested onto its subscription polluted
shared state (harmless today — the scheduler's deep paths are human-only — but a
future scheduler path trusting the flag would leak quota). Guard the flag WRITE at the
source (not user.is_demo), which also makes deep_turned_on always False for demo, so
the redundant is_demo check on the immediate backfill is removed.
/code-review (S1, finding 1): sanitize truncated the title at 120 CHARACTERS, but a
path component's hard limit is 255 BYTES and rel_path packs channel + date + id + ext
into that same component — so a long accented/multi-byte title (Hungarian titles are
common here) could already overflow to ENAMETOOLONG, and the new _a{id} suffix shrank
the margin further. sanitize now truncates on a UTF-8 boundary by byte length, so the
title (≤120B) + channel (≤80B) + fixed parts stay comfortably under 255B. Two tests:
byte-cap on accented input, and every rel_path component ≤255B for a pathological
long-accented channel+title+asset_id.
A MediaAsset is unique on (source_kind, source_ref, format_sig), so the same video in
two formats is two assets — but rel_path keyed only on video_id, so they computed the
same path: the second download silently overwrote the first, and deleting either
removed the shared file. rel_path now takes an optional asset_id and appends "_a{id}"
(which Plex ignores — the .nfo sidecar carries the metadata); the worker passes the
asset id. Only affects NEW downloads (existing assets keep their stored rel_path).
Covered by a new test (two ids → two paths; omitted → the plain name, unchanged).
The demo account could spend real YouTube quota where the human-only guard was
missing (the channel_detail enrichment already had it):
- channels PATCH: an immediate run_recent_backfill when deep_requested is turned on;
- discovery: the channel-detail enrich on the Discover tab;
- feed get_video_detail: the off-catalog videos.list lookup for a video we don't
store (the DB-hit path stays free; demo now gets a 404 for unknown videos).
The asset-busy and claim-fail branches wrote status='queued' unconditionally, so a
pause/cancel the route had just written was clobbered back to queued and the job
silently revived. New _requeue_if_running does the requeue as a conditional UPDATE
(...WHERE status='running'), a no-op once the route has moved the job off 'running'.
C-3.5: /auth/request-access had no rate limiter (unlike register/login/reset/demo)
and answered "approved" for an allow-listed email — a public enumeration oracle for
the allow-list, and an unthrottled invite/admin-mail firehose. It now rate-limits per
client IP and returns a uniform "pending" for every outcome (allowed, pending, new,
throttled); an already-allowed user can still just sign in with Google.
C-3.11: _consume_token was a read-check-write, so two concurrent requests could both
consume the same verify/reset token. It's now a single conditional UPDATE
(...WHERE used_at IS NULL AND expires_at >= now RETURNING user_id) — only the request
that flips used_at gets a user back.
Review round 1: the two existing safe_abs_path escape tests both pass even against
the classic buggy `str(path).startswith(str(root))` containment check, so a
regression to that bug would have shipped green. Add the one input that
distinguishes the correct `root not in path.parents` guard: a sibling dir whose name
prefixes the root (/root vs /rootx). Mutation-verified — swapping the guard to
startswith now fails this test. Also corrected the requirements-dev header (installed
by Dockerfile.test, not a nonexistent `dev` stage).
The backend had zero tests. This adds 33, all pure (no DB, no network), plus the
harness to run them in the gate:
- backend/Dockerfile.test: the app's deps + pinned pytest/ruff, code bind-mounted at
run time so it always tests the working tree. Isolated from the prod Dockerfile, so
the published image never carries test tooling.
- requirements-dev.txt: pytest==8.4.2, ruff==0.15.21 (the version the gate already
runs) — the backend lane is now pinned, not just host-global.
- tests: normalize_title (de-shout, hashtag strip, emoji drop, trilingual letters
survive); storage sanitize/rel_path/download_filename and the safe_abs_path
traversal guard (../ and absolute-path escapes rejected — verified by mutation);
links HMAC grants (round-trip, wrong-token, tampered sig/exp, expiry) + is_expired;
a DB-free app-assembly smoke (every router wires up, OpenAPI generates).
DB-backed router smoke (auth/feed/downloads) needs a test Postgres + migrations —
deferred. useCardPager needs hook-test infra (jsdom/renderHook) — deferred.
- Backend: add channel description to the shared _channel_summary projection, so both
the subscribed list and the discovery list carry it (no migration; description is
already stored from the channels.list call)
- New AboutCell: one truncated line in the table + full text in a portalled, always-on
glass hover/focus overlay (wider than Tooltip, flips above the anchor near the bottom,
closes on scroll/resize)
- About column (after Channel) on both the subscribed and discovery tables; hidden in the
mobile card fallback
Lazy enrichment only covers viewed titles, so an actor's count/results were
partial. Add a background backfill: each plex_sync rich-fetches a bounded batch
(60) of not-yet-enriched movies/shows and persists their FULL cast, stamping
people_enriched_at so each is fetched once — the whole library is covered over
successive runs (migration 0057 adds the marker). Extracted the enrichment into a
shared app/plex/people.py (cast_from_meta / enrich_row_people / prune_orphan_people
+ the person-image host/proxy) so the info-page lazy path and the batch share ONE
implementation — no duplication; the info page also stamps the marker so the batch
skips it. Verified: an 8-item batch grows those movies' cast_names 3->24 and fills
plex_people (32->224); the info-page cast is intact after the _rich_meta refactor.
(Plex cast-explore S1b)
Guard the prune's NOT IN against the classic NULL trap: a JSON-null cast array
element would surface as SQL NULL in the referenced set, making name NOT IN
(…, NULL) evaluate NULL for every row → silently prune nothing. Filter NULLs out
of the subquery. (Plex cast-explore S4 review)
The plex_people headshot cache can outlive its subjects — when a title leaves the
Plex library, cast/crew who appear in nothing else are orphaned. prune_orphan_people
deletes rows whose name no longer appears in ANY movie/show cast_names/directors,
run at the end of every plex_sync (best-effort; a jsonb_typeof='array' guard skips
scalar/null JSONB values). The filterable cast_names self-clean with their row;
this cleans the only per-person cache. Verified: an injected orphan is dropped, real
people kept. (Plex cast-explore S4)
The actors currently in the filter now render as cards above the poster grid —
headshot + name + in-scope title count + a remove x — so you SEE who you're
exploring, not just chips in the rail (generalises the old search-only person
cards to the applied filter). New GET /plex/people/cards (photo from the
plex_people cache, count live against cast_names); PlexBrowse fetches + renders
them; i18n en/hu. Verified: John Goodman (11) / Douglas M. Griffin (1) / Bradley
Cooper (14) with photos + counts. (Plex cast-explore S3)
Multiple selected actors now combine as OR (any) by default — click several cast
and see films featuring ANY of them — instead of the old hardcoded AND that
emptied the grid. An AND (all) toggle in the ACTIVE filter section (shown for 2+
actors, mirroring the genre match toggle) narrows to films with ALL of them.
Backend actor_mode param mirrors genre_mode. Verified: OR John Goodman|Douglas
M. Griffin = 11 films; AND = 1 (10 Cloverfield Lane). (Plex cast-explore S2)
The cheap section listing that drives the main sync only stores the top ~3 cast,
so any cast member below that was unfilterable (clicking them AND-emptied the
grid). Now the movie/show info fetch (/item, /show) — which already pulls the full
rich cast — persists it onto cast_names + people_text (so every displayed cast
member becomes filterable) and caches each headshot in the new plex_people table
(migration 0056). Idempotent (only grows a thinner stored set); best-effort (never
500s a read). Verified: 10 Cloverfield Lane 3->8 cast after viewing; plex_people
populated with photos. (Plex cast-explore S1a)
Pick any number of channels; they OR together and every other filter applies
across the lot — "the last week's unwatched from these three".
`channelId`/`channelName` are replaced outright by `channelIds: string[]`, no
legacy reader (standing rule: no back-compat unless asked — a saved view written
against the old field simply loses its channel filter). The wire and the share
URL move with it: repeated `channel_ids` params, `?channel=a,b`. Backend ORs via
`Video.channel_id.in_()`; the count endpoint shares the same params object.
Two things the migration exposed:
- SavedViewsWidget fed DB blobs straight into serializers that now index an
array — the FeedFilters type says nothing about what an older build wrote, so
the blob is coerced where it enters. It white-screened the app before this.
- Chips carried channelName to label themselves. Without it they resolve names
from the channel cache, so they now fall back to the id, not a shared
"This channel" that would render N identical chips while the cache loads.
Only English and Hungarian remain. Drops the de locale files, the LANGUAGES
entry, the Google-locale mapping in auth, and the German welcome-message
template. A legacy "de" preference falls back to English gracefully. Content-
language detection and subtitle languages are untouched (media, not UI).
- apply_channel_details: (branding.get("channel") or {}) so a "channel": null
in brandingSettings can't AttributeError-abort the enrich batch.
- topicLabels: drop empty labels so a malformed topic URL can't render a blank chip.
- fix: reserve the scrollbar gutter (scrollbar-gutter:stable) on the channel
page's scroll container, so switching between the tall Videos tab and the
short About tab no longer shifts the banner/avatar/buttons a few px sideways
(the vertical scrollbar was appearing/disappearing between the two tabs).
- About tab now shows Country (flag + localized name), Language, Topics
(topicCategories → readable chips), and Keywords (brandingSettings keywords
parsed into chips, quoted multi-word tags kept whole). country/language/topics
were already stored; keywords is new (migration 0055_channel_keywords, mapped
in apply_channel_details, returned by channel_detail).
- Discovery → channel page: the Channel-manager "Discover from playlists" tab now
links each channel name to its in-app channel page (ChannelLink onView), so a
discovered channel's About/videos can be inspected BEFORE subscribing (was
subscribe-only, plain text before).
Note: the channel info-card epic itself was already delivered in v0.19.0; this is
the About-tab enrichment follow-up + the header-shift fix. external_links stays
empty by design (YouTube removed the field from the Data API ~2023).
- scheduler _run_changed: count a run as "changed" only on a truthy NUMERIC
value — status-string no-op dicts like {"skipped":"disabled"} (Plex off, the
default) or {"skipped":"no demo account"} were flooding the trail every interval.
- audit.record: truncate target_id to the column width (128) like summary[:255],
so an over-long target (e.g. a 128+ char demo email) can't abort the mutation
the audit row is committed alongside.
- config set/reset: redact URL userinfo (user:pass@) from logged non-secret
values — a proxy setting can embed credentials that shouldn't land in the trail.
- config set: skip the audit row when a non-secret value didn't actually change
(no-op re-save shouldn't add noise); secrets are write-only so always logged.
- audit page title (pageMeta): add the missing "audit" case so the browser tab
reads "Audit log · Siftlode" instead of falling back to "Feed".
- channels CB3: sort discovery rows NULLs-last (title is None) to match the DB
ORDER BY exactly — coercing NULL title to "" floated untitled channels to the top.
- AdminUsers SC1: track in-flight rows in a Set, not a single id — per-row disabling
now lets the admin start concurrent row actions, and a shared id was cleared by
whichever settled first (re-enabling a still-pending row → possible double-submit).
- ChatThread CT4: reset the incoming-count ref when partnerId changes — the Messages
page reuses one ChatThread across conversation switches, so a same-count switch
could skip the open-marks-read badge refresh.
- messages MB3: push a bare {type:"unread"} (drop the now-dead count query — the
client re-fetches on the signal and ignored the number anyway).
- api.ts: extract the shared keepalive `beacon()` helper (saveProgressBeacon +
plexProgressBeacon were near-verbatim copies).
Verified each deferred per-subsystem item against current source, then fixed the
real ones (tradeoffs left as-is, see siftlode-ops/CODE-HYGIENE.md closeout):
- search BUG-3: don't finalise shorts_probed on un-enriched stubs (enrichment
failure/omitted rows) — restores the scheduler's enriched_at guard so a real
Short can't leak into the feed and never get reclassified.
- downloads GC: 4th pass reaps orphaned errored, fileless, unreferenced assets
(they carry no TTL, so the expiry passes never cleared them).
- downloads B6: quota/edit errors now return a structured {code,reason,limit}
the trilingual client localises, instead of hardcoded English + dot-decimal GB.
- channels BB1: reset-backfill re-arms deep sync on every subscriber (bulk update)
instead of 404ing when the admin isn't personally subscribed.
- channels BB2: enrich the stub on BOTH the normal and "already exists" desync
path (nest the insert try inside the client `with`).
- channels CB3: drop the redundant second _discovery_rows read (identity-mapped
rows are already enriched; re-sort in Python for the title tie-break).
- playlists PC1: read the live playlist once in push() and share it with plan_push
+ push_playlist (halves read-quota, closes the second TOCTOU window).
- playlists PC2/PC5: batch the cover thumbnails in one window query (was N+1);
rename combines count+duration into one aggregate + a single cover lookup.
- messages MB2: get_thread only resolves a messageable partner OR one we already
share a thread with (closes the user-enumeration oracle).
- messages MB3: push a live unread-count to the user's other tabs after mark-read.
- messages MC4: list_conversations uses DISTINCT ON + grouped COUNT instead of
loading the whole message history into memory.
- config AB3: per-spec allow_empty so smtp_user/youtube_api_proxy can be blanked
to disable them rather than snapping back to the env default.
purge_user's revoke used `decrypt(refresh) or tok.access_token`, but access_token is
stored ENCRYPTED — so a token row without a refresh token would send ciphertext to
Google's revoke endpoint and silently fail. Decrypt it. (Pre-existing; surfaced by the
review of the OAuth-scope fix.)
Correcting the earlier scope-union-only fix. The real damage from a logout→login isn't
just the stored `scope` field — verified via a live token refresh that the stored refresh
token itself had been REPLACED with a base-only one: a plain sign-in requests only
BASE_SCOPES and Google handed back a fresh base-only access+refresh token, which
_store_token adopted verbatim, destroying the read/write grant (the old refresh token
stays valid on Google's side, so overwriting it is what loses access).
_store_token now detects when an exchange would NARROW our YouTube scopes and, in that
case, keeps the WHOLE existing grant (refresh token, access token, scopes) untouched.
Broader-or-equal exchanges (first grant, read→write upgrade) adopt + union as before.
Unit-verified across base-relogin / upgrade / new-user / base-user cases.
Two user-reported signin bugs:
1) A plain re-login (or logout→login) wiped the user's YouTube grant: login requests
only BASE_SCOPES and Google's returned `scope` can list just those, but _store_token
wrote it verbatim — dropping a previously-granted youtube.readonly/youtube scope, so
can_read flipped to false and the feed demanded a reconnect. The underlying grant
(refresh token) survives such a login, so UNION the scopes instead of narrowing; they
only shrink on full disconnect (purge deletes the token row).
2) Admin 'new access request' email: (a) it named the wrong menu ('Settings → Account'
instead of Users → Access requests); (b) the requester address was invisible so the
'reply reaches them' note looked wrong (Reply-To is in fact set to the requester) —
now spelled out + a deep-link straight to the approve view (?admin=access-requests,
handled in App); (c) it emailed only the static env ADMIN_EMAILS — now notifies the
ACTUAL admins (active role=admin users) unioned with env, so a UI-promoted admin (who
CAN approve — the approve UI is role-gated) is notified too.
resolved_user_id already accesses ws.session unguarded just above (SessionMiddleware
covers the WS scope), so the '"session" in ws.scope' fallback was dead code — read
ws.session directly for consistency (review follow-up).
Loose ends to finish the auth security round:
- register + password-reset-request had an enumeration TIMING oracle: an already-
registered email skipped the create path (hash + row writes + email scheduling) and
responded measurably faster than a new one. Move the whole lookup+create (register)
and lookup+token+email (reset) into a background task with its own DB session, so the
endpoint returns in the same time for any valid email regardless of whether it exists.
Verified: existing vs new now ~equal (was 34ms vs 82ms on register); accounts/tokens
still created off-path.
- messages_ws re-implemented resolved_user_id's per-tab wallet-gated account resolution.
Generalize resolved_user_id to take any HTTPConnection (Request OR WebSocket) and call
it from the WS — one shared, wallet-gated resolution. Behavior-identical (unit-checked).
Post-review hardening (both fail-safe, not bugs):
- _client_ip normalizes IPs via ipaddress and unwraps IPv4-mapped IPv6, so a plain
TRUSTED_PROXY_IPS=10.10.0.1 also matches a peer surfaced as ::ffff:10.10.0.1 (a
Docker/IPv6 self-host footgun that would otherwise silently collapse everyone into
one rate-limit bucket).
- entrypoint.sh + _client_ip docstring: explicit warning never to add uvicorn
--proxy-headers, which would rewrite request.client from the forgeable XFF and defeat
the trust check.
_client_ip trusted the first X-Forwarded-For hop unconditionally, so anyone able to
reach the app port could forge XFF and dodge the login/register/reset/demo rate limits.
Now trust XFF ONLY when the request's socket peer is a configured reverse proxy
(settings.trusted_proxy_ips, e.g. the VPS Caddy's WireGuard peer IP), and take the
RIGHTMOST entry — the client our proxy actually saw and appended, immune to a client
pre-seeding a fake XFF. A request from any other peer (hitting the port directly) is
keyed on its real socket IP, so XFF can't be forged to bypass the limits.
New TRUSTED_PROXY_IPS env (empty default = no proxy, use the direct peer). Documented in
.env.example, docs/self-hosting.md, README. Unit-verified against spoof-through-proxy and
direct-bypass cases.
Adversarial re-review of the session-epoch work surfaced:
- WS auth (messages_ws) skipped the epoch check, so a revoked-but-unexpired cookie
could still open the live push channel after a reset/logout-others. Now mirrors
current_user: loads the user once, rejects a stale-epoch cookie before connecting.
- set_password + logout_others re-stamped the cookie BEFORE db.commit(); a failed
commit would strand the current session at a newer epoch than the DB and wrongly
401 it. Commit first, then re-stamp.
- Welcome verify effect could double-POST the single-use token (StrictMode/remount)
and flip the banner to a false 'invalid'. Fire-once useRef guard.
Left as-is (low value, documented): the Plex image proxy authenticates without a DB
load / epoch check (poster/art fetches only); adding one would cost a DB hit per image.
Signed client-side session cookies had no server-side kill switch: logout + password
reset couldn't evict a stolen/copied cookie (valid until expiry). Add User.session_epoch
(migration 0053), record it in the cookie at every login, and reject in current_user any
cookie whose recorded epoch is behind the account's current one.
- Bump the epoch on: password reset (kills ALL sessions — a reset is a compromise response),
password change + a new 'Log out other sessions' action (both re-stamp the CURRENT cookie
so the caller stays signed in, evicting only the others).
- Per-account epoch map in the session so one account's revocation doesn't evict the other
signed-in accounts in the same browser wallet.
- Missing epoch (pre-SA4 cookie) is treated as 0, so the first bump revokes grandfathered
sessions too.
- New POST /auth/logout-others + a Settings → Account 'Active sessions' button (trilingual).
Secret email tokens now ride the URL fragment (#reset=/#verify=), never the query
(?reset=/?token=): a fragment isn't sent to the server, so the token can't leak into
proxy/access logs or a Referer header.
- Reset: link → /#reset=; the SPA reads the token from location.hash and POSTs it
(unchanged /password-reset/confirm).
- Verify: link → /#verify=; new POST /auth/verify (token in body). The legacy GET
/auth/verify?token= is kept so pre-deploy emails in flight still work until they
expire. The SPA reads the fragment token, POSTs it, shows ok/invalid.
- Welcome: read secret tokens from the fragment, status flags from the query; strip
both after capture so nothing lingers in history.
Closes the Plex #9 backend backlog (frontend was the prior follow-up). Behavior-
neutral cleanup + two two-way-sync bug fixes; PW2 verified a non-bug.
fix(plex): watch-sync PW1 pagination + PW3 lost-update guard
- PW1 watch_history: the history is a GLOBAL cross-account feed read as a single
500-row page, then filtered to the owner — on a busy family server the owner's
recent views can sit past page 1 and be missed. Now paginate (desc) until the
since-cutoff, bounded by max_pages (daily reconcile is the backstop; logs if hit).
- PW3 push_state_to_plex: it flagged synced_to_plex=True unconditionally after a
push, so a local edit landing between the push and the flag-set was marked clean
and never pushed (lost update). Capture the row's freshest timestamp when the
push is scheduled (_push_watch) and only settle the flag if the row is unchanged.
- PW2 was flagged as "accountID hardcoded to 1" but is NOT a bug: on a PMS account 1
is reserved for the owner/admin and an owner link always holds the admin token —
added a clarifying comment so it isn't re-flagged.
chore(plex): backend dedup
- PC2 unified_library + facets shared a ~13-field filter Query signature + p-dict →
one LibraryFilters dataclass injected via Depends(); as_dict() feeds the builders.
- PS-C1 sync.py collection upsert dup (resync_collection ≈ _sync_collections) →
_upsert_collection().
- PS-C2 sync.py 8-field filterable-metadata block dup (_apply_item ≈ _sync_shows) →
_apply_facet_fields().
- PW-C1 _repush_dirty read duration from the mirrored PlexItem.duration_s instead of
a per-row plex.metadata() round-trip.
- PW-C2 rk→id map built inline in two places → _rk_to_id() helper.
- plex.py: delete the no-op _enabled() (never wired as a Depends/called; the real
gate is sysconfig plex_enabled).
- PlexBrowse.tsx: collapse toggleEpisodeWatched into toggleWatched — they were
byte-identical (both take a PlexCard).
- Delete 6 dead plex.json keys (loadMore, playerSoon, filter.library,
playlist.up/down/remove) from en/hu/de — verified 0 refs (the live
playlist.removeShow/removeSeason keys are kept).
Behavior-neutral. tsc green, ruff clean on touched files, localdev boots, Plex renders.
stream.py enforced a hardcoded _MAX_SESSIONS = 4 and never read the
DB-overridable plex_max_transcodes ConfigSpec, so the Configuration → Plex →
"Max concurrent transcodes" knob was dead (same env-vs-DB drift class as the
Downloads/Admin fixes). _enforce_cap(cap) now takes the cap from
sysconfig.get_int(db, "plex_max_transcodes") (>=1 so playback can't be capped to
zero). Bumped the config default 1→4 so the effective default matches the old
hardcoded cap (no behavior change for non-overriders; the knob now works).
From the auth /security-review (contained fixes; architectural SA3 proxy-trust +
SA4 session-revocation deferred to the user):
- SB1: encrypt the OAuth access_token at rest (was plaintext while refresh_token
was encrypted) — a ~1h Google bearer credential. New security.decrypt_optional()
falls back to a refresh for legacy plaintext tokens; column is unbounded String.
E2E-verified: token refreshed → stored as Fernet ciphertext → 319-subscription
YouTube sync succeeded.
- SA5: password_login is no longer a timing/enumeration oracle — it always runs
argon2 (against a decoy hash for unknown/passwordless emails), so response time
can't reveal whether an account exists.
- SB2: Google login only ADOPTS+activates a pre-existing password account when
Google actually attests email_verified (defense-in-depth against takeover); and
the email sync won't overwrite with a value another account owns (avoids a 500).
- demo_login clears the wallet first (demo can't switch to / act as a real account
via the multi-account header); switch_account rejects a suspended target (which
would otherwise clear the whole session on the next request).
Re-review clean; ruff clean; localdev boots; YouTube auth path E2E-verified.
set_user_role (demote) and admin_delete_user counted ALL admins incl. suspended
ones, so with e.g. 1 active + 1 suspended admin you could demote/delete the only
ACTIVE admin — leaving a single suspended admin who can never sign in → permanent
admin lockout needing DB surgery. Now both guards mirror the (already-correct)
suspend guard: block only when the target is an active admin AND it's the last one
(`not target.is_suspended and count_admins(active_only=True) <= 1`) — which also
correctly still allows removing a SUSPENDED admin while one active admin remains.
- Extract runner.token_users(db): the "all users with a stored refresh token"
query was spelled 3× (get_service_user, run_subscription_resync,
sync_all_playlists). get_service_user now reuses it ([0]); sync_all_playlists
imports it from runner (no cycle — runner doesn't import playlists); the now-
unused OAuthToken import is dropped there.
- scheduler.py: _job("rss_poll", lambda db: run_rss_poll(db)) → _job("rss_poll",
run_rss_poll) — the lambda was dead indirection; all sibling jobs pass the
callable directly and _job calls fn(db).
Behavior-neutral. ruff clean, localdev boots, re-review clean.
download_layout and download_retention_days are registered ConfigSpec keys
(admin-editable on the Configuration page), but worker.py read them from
settings.* (env) at download/edit finalize time — so an admin's edit was a
silent no-op (files kept the old layout; TTL stayed at the env default). Read
them via sysconfig (_download_settings), which falls back to the env default
when there's no DB override. Same class as the Downloads B3 fix.
ruff clean, localdev boots, re-review clean.
BUG (PB1): reorder_items only repositioned the items present in the payload,
leaving any omitted item (e.g. one added concurrently in another tab) at its old
position — which then collides with the freshly assigned 0..N-1 range, so
get_playlist's order-by-position returns a nondeterministic/duplicated order.
Now the omitted items get fresh trailing positions (keeping their relative
order), guaranteeing unique contiguous positions.
CLEANUP (PC4): extract _remove_item(db, pl, video_id, mark_dirty=) mirroring the
existing _add_item — remove_watch_later and remove_item duplicated the same
find-by-(playlist,video)/delete/commit block.
Behavior-neutral for the normal full-payload reorder. ruff clean, localdev boots.
- Extract _is_blocked(db, user, channel_id): the BlockedChannel EXISTS query was
copy-pasted in channel_detail, explore_channel, and block_channel.
- Extract _channel_summary(ch): the shared id/title/handle/thumbnail/subscriber/
video_count projection was hand-rolled in list_channels and discover_channels
(the detail dict interleaves these with extras, so it's left as-is).
Behavior-neutral. ruff clean, localdev boots healthy.
BUG-1 (search.py, high): the scrape search source logged one VIDEOS_SEARCH
quota event PER continuation page inside the paging loop, but actions_today
counts events — so a single user search that pages N times consumed N against
search_daily_limit_per_user (its docstring even warns it only works for
once-per-action logging). Log exactly once per request after the loop instead;
the API source already logs once via record_usage (it never auto-pages).
BUG-2 (Feed.tsx, medium): the optimistic-override reset effect keyed on
query.dataUpdatedAt also fired on fetchNextPage (infinite scroll bumps
dataUpdatedAt while page 1 is unchanged), so a just-hidden card flashed back
mid-scroll. Guard with a page-count ref: only reset on a real refetch, not an
append. The two override-reset effects legitimately differ now (resolves the
would-be C-F1 "identical effects" cleanup).
tsc green, ruff clean, localdev boots healthy.
- feed.py _filtered_query: drop the dead 2nd return element (status_expr was
used only internally for WHERE filters; all 3 callers discarded it as _status).
Now returns (query, rank_expr); fixed the stale docstring.
- youtube/client.py: extract _iter_playlist_items() — iter_my_playlist_video_ids
and iter_playlist_items_with_ids were near-identical playlistItems paging loops
(jscpd [173-187]≈[267-281]); both now map over the shared generator.
- sync/videos.py: extract _apply_video_batch() with an on_missing callback —
enrich_pending and refresh_live shared the fetch-map-apply skeleton, differing
only in the query and how they retire rows YouTube omits.
- models.py: add LIVE_OR_UPCOMING = ("live","upcoming"); replace the 4 duplicated
copies (feed.HIDDEN_LIVE, search._LIVE_HIDDEN, videos.py + channels.py inline).
Behavior-neutral. ruff clean on touched files, localdev boots, feed/worker healthy.
- B1 (sticky errored asset): get_or_create_asset now resets a reused status=='error'
asset back to 'pending' (+clears the error), so a fresh enqueue/edit of a once-failed
(source,format) pair actually re-downloads instead of the worker short-circuiting the
new job with the stale error. Errored assets carry no expires_at, so without this the
pair was permanently poisoned for all users until someone hit resume.
- B2 (ref_count leak): _release_asset counts 'error' as a holding state, so deleting an
errored job decrements the ref_count that enqueue always incremented (the worker never
decrements on failure). Errored rows are deliberately NOT deleted here — a concurrent
B1 reuse could otherwise be lost-updated + FK-nulled; the row is fileless and harmless.
- B3: admin storage dashboard reads sysconfig.get_int(db,'download_total_max_bytes')
(the admin-editable DB value GC enforces) instead of the raw env default.
- B4: the single-trim branch of normalize_edit_spec guards its float() coercion like the
crop/segments branches — a malformed trim now yields a 400, not an unhandled 500.
- B5: formats.normalize guards int(max_height) → falls back to "best" instead of 500.
Reviewed (race in an earlier B2 draft caught + fixed); localdev boots, B4/B5 unit-verified.
- C3: `_reference_url` (downloads.py) and public.py's inline source-URL block were
the same rule → extract `service.reference_url(job, asset)`; both surfaces now
share it so the "downloaded from" link can't drift between them.
- C4: Content-Disposition filename derivation (ext pick + doubled-ext strip + join)
was duplicated in download_file and watch_file → extract
`storage.download_filename(display_name, container, path)`.
- C5: inline the one-line `_clean_basename` passthrough (folded into C4).
- C6: the `db.get(MediaAsset, job.asset_id) if job.asset_id else None; _serialize(...)`
resolve-then-serialize dance was repeated across 6 single-job handlers → fold into
`_serialize_job(db, job)`. (File-serving handlers that use the asset for their own
checks keep their explicit resolve.)
Behavior-neutral; ruff/parse clean, localdev boots, downloads routes load.
- C1: remove downloads/formats.py target_ext() — defined but never called (the
worker derives the real extension from the produced file's suffix).
- C2: the download-root containment+existence guard was copy-pasted 6× across the
file-serving endpoints (routes/downloads.py ×3, routes/public.py ×3). Extract
storage.safe_abs_path(root, rel) -> Path|None so this security-sensitive check
lives in one place; behavior identical (same containment test + messages). The
extraction also made `pathlib.Path` unused in both route modules (removed).
peter