- list_channels no longer computes the Normal/Shorts/Live aggregation the
frontend stopped reading when R14 dropped the N/S/L table column; the dead
ManagedChannel.count_* fields are gone too.
- Immediate "get full history" backfill now coordinates with the paced
scheduler via a small in-flight set: capped at 2 concurrent + per-channel
dedup, and run_deep_backfill skips a channel an immediate backfill owns —
killing the cursor race and bounding the quota-drain burst, without losing
the speed win.
- Add test_channel_backfill.py: the channelDetail N/S/L counts + the guard's
no-op / cap / scheduler-skip behaviour.
- Inline the syncedChannels useMemo (its dep is a fresh array each render, so
it never cached); fix the stale "~2-min ticks" comment (it's 10 minutes).
Clicking "get full history" now kicks an immediate background deep
backfill of that one channel (backfill_channel_now: own session, up to
~3000 videos in one pass, quota-guarded) instead of waiting for the
scheduler's paced 10-minute ticks (5 pages/channel/tick). The request
returns instantly and the channel page's live poll shows the count fill
in seconds. A very large channel's tail still falls to the scheduler
(deep_requested stays set).
- DataTable: opt-in `fixedLayout` (colgroup + table-layout:fixed) so the
table fits its container at any zoom/text-size; only the channel manager
opts in — playlists/discovery/Plex keep auto-sizing.
- Remove the N/S/L and Sync columns from the manager table. Fully-synced
status becomes a ✓ charm on the Stored count; the sync filter moves to an
`applySyncFilter` pre-filter still driven by the top chips.
- Channel column is now fluid: ChannelLink truncates with the full name on
hover (title). Data columns get fixed widths so only Channel/About flex.
- N/S/L (normal/short/live) breakdown moves to the channel page: add the
three counts to the channelDetail endpoint (one grouped query, same
filters as the list) + the ChannelDetail type + a meta-line entry.
- Drop the now-dead sync/types i18n keys (en+hu).
1A ChannelPage "Full history" button no longer wraps (whitespace-nowrap).
1B Extract the manager's press-and-hold priority stepper into a shared
PriorityStepper and add it to the channel detail header (subscribed, non-demo);
backend channel_detail now returns the per-subscription `priority`.
2 Unify YouTube + Plex media cards onto one look/behaviour: drop the whole-card
hover LIFT (VideoCard) — which also fixes the clipped-top bug — and instead the
inner thumbnail/poster ZOOMS on hover (both); give the flat Plex posters the same
glass-card 3D depth + accent hover border as the feed cards (shadow-md is a
transparent no-op here, so it's glass-card that provides the depth); clamp Plex
card titles to a fixed 2-line height so 1- vs 2-line titles align.
3 Nav rail is now always a slim icon-only column, flush to the left edge; each icon
reveals a floating name label on hover (Tooltip gains `always` + `openDelay`, and
a right side). Removed the pin/unpin toggle, the expand-on-hover mode and the
navCollapsed pref plumbing.
4 Scope the side-panel peek-open HOVER to the tab handle + the card (not the whole
full-height slot next to the rail), so approaching the nav no longer pops the
filter open; focus stays container-wide. useHoverFocusWithin gains a hover-out
grace period + split hover/focus handler groups.
5 Plex art backdrop fills the whole viewport (a fixed inset-0 layer behind the
transparent app) instead of only the inset <main> box.
6 Plex season binge continues across seasons: the season view's play queue now
spans the whole show (allKeys), so a season's last episode rolls into the next
season's first (seasonKeys stays for "add season to playlist").
7 When a Plex video/episode ends with nothing queued next, the player returns to
the previous page (onEnded → onClose) instead of stranding a black screen.
Bring the same tag control to the channel page, next to Subscribe:
- backend: GET /api/channels/{id} now returns this user's `tag_ids` for the channel
(added to _channel_detail_dict + the ChannelDetail interface), matching the manager
list shape;
- ChannelPage renders the shared ChannelTagsControl, wired to attach/detach/create with
an optimistic update on the channel-detail cache (reverting on failure) and a manager
list invalidation so a tag added here shows there too.
Reuses the R8 popover platform + ChannelTagsControl — no new tag UI.
The catalog only stores the recent-backfill/RSS subset of a channel's uploads (video_count is
YouTube's raw total), so a subscribed channel looked like it was "missing" videos. Now the channel
page header shows "{stored} / {total} videos" when there's a gap, and a "Full history" action queues
the deep backfill (the same deep_requested trigger the channel manager uses) — showing a loading
state until done. Backend: channel_detail now returns deep_requested/backfill_done.
update_channel: payload:dict → ChannelUpdateIn (priority/hidden/deep_requested,
all optional for PATCH semantics). Kills the `int(payload["priority"])` 500 the
prod report hit — a non-numeric/garbage value now returns 422, not 500.
attach_tag: payload:dict → AttachTagIn(tag_id:int) — a non-int tag_id was a 500
in `db.get(Tag, ...)`; the personal-tag ownership check reuses owned_or_404.
Round-1 review finding #4 named only audit.py/plex.py, but the same misplaced
`from app.utils import iso` sat in channels/downloads/feed/notifications too — the
round missed them (ruff enforces no import order here, so the gate was silent).
Move them to alphabetical position for consistency. No behaviour change.
Establish the R6 API-contract infra before the S1b payload:dict migration:
- iso(dt) in utils.py replaces the ~16 hand-rolled `x.isoformat() if x else None`
serializer idioms across the routes.
- get_or_404 / owned_or_404 in routes/_common.py — one 404 + existence-leak policy;
plex's _own_playlist_or_404 and saved_views' _own_view now delegate to it.
- require_write_scope dependency in auth.py collapses the inline has_write_scope→403
checks (channels subscribe/unsubscribe, playlists push/push-plan) that had drifted
into three different messages into one; the conditional delete_playlist guard stays
inline (its message was already canonical).
No response-shape change (iso == isoformat for a present value). Gate: ruff clean,
pytest 163 green (+ test_utils for iso).
The demo account is shared, so writing deep_requested onto its subscription polluted
shared state (harmless today — the scheduler's deep paths are human-only — but a
future scheduler path trusting the flag would leak quota). Guard the flag WRITE at the
source (not user.is_demo), which also makes deep_turned_on always False for demo, so
the redundant is_demo check on the immediate backfill is removed.
The demo account could spend real YouTube quota where the human-only guard was
missing (the channel_detail enrichment already had it):
- channels PATCH: an immediate run_recent_backfill when deep_requested is turned on;
- discovery: the channel-detail enrich on the Discover tab;
- feed get_video_detail: the off-catalog videos.list lookup for a video we don't
store (the DB-hit path stays free; demo now gets a 404 for unknown videos).
- Backend: add channel description to the shared _channel_summary projection, so both
the subscribed list and the discovery list carry it (no migration; description is
already stored from the channels.list call)
- New AboutCell: one truncated line in the table + full text in a portalled, always-on
glass hover/focus overlay (wider than Tooltip, flips above the anchor near the bottom,
closes on scroll/resize)
- About column (after Channel) on both the subscribed and discovery tables; hidden in the
mobile card fallback
- fix: reserve the scrollbar gutter (scrollbar-gutter:stable) on the channel
page's scroll container, so switching between the tall Videos tab and the
short About tab no longer shifts the banner/avatar/buttons a few px sideways
(the vertical scrollbar was appearing/disappearing between the two tabs).
- About tab now shows Country (flag + localized name), Language, Topics
(topicCategories → readable chips), and Keywords (brandingSettings keywords
parsed into chips, quoted multi-word tags kept whole). country/language/topics
were already stored; keywords is new (migration 0055_channel_keywords, mapped
in apply_channel_details, returned by channel_detail).
- Discovery → channel page: the Channel-manager "Discover from playlists" tab now
links each channel name to its in-app channel page (ChannelLink onView), so a
discovered channel's About/videos can be inspected BEFORE subscribing (was
subscribe-only, plain text before).
Note: the channel info-card epic itself was already delivered in v0.19.0; this is
the About-tab enrichment follow-up + the header-shift fix. external_links stays
empty by design (YouTube removed the field from the Data API ~2023).
- channels CB3: sort discovery rows NULLs-last (title is None) to match the DB
ORDER BY exactly — coercing NULL title to "" floated untitled channels to the top.
- AdminUsers SC1: track in-flight rows in a Set, not a single id — per-row disabling
now lets the admin start concurrent row actions, and a shared id was cleared by
whichever settled first (re-enabling a still-pending row → possible double-submit).
- ChatThread CT4: reset the incoming-count ref when partnerId changes — the Messages
page reuses one ChatThread across conversation switches, so a same-count switch
could skip the open-marks-read badge refresh.
- messages MB3: push a bare {type:"unread"} (drop the now-dead count query — the
client re-fetches on the signal and ignored the number anyway).
- api.ts: extract the shared keepalive `beacon()` helper (saveProgressBeacon +
plexProgressBeacon were near-verbatim copies).
Verified each deferred per-subsystem item against current source, then fixed the
real ones (tradeoffs left as-is, see siftlode-ops/CODE-HYGIENE.md closeout):
- search BUG-3: don't finalise shorts_probed on un-enriched stubs (enrichment
failure/omitted rows) — restores the scheduler's enriched_at guard so a real
Short can't leak into the feed and never get reclassified.
- downloads GC: 4th pass reaps orphaned errored, fileless, unreferenced assets
(they carry no TTL, so the expiry passes never cleared them).
- downloads B6: quota/edit errors now return a structured {code,reason,limit}
the trilingual client localises, instead of hardcoded English + dot-decimal GB.
- channels BB1: reset-backfill re-arms deep sync on every subscriber (bulk update)
instead of 404ing when the admin isn't personally subscribed.
- channels BB2: enrich the stub on BOTH the normal and "already exists" desync
path (nest the insert try inside the client `with`).
- channels CB3: drop the redundant second _discovery_rows read (identity-mapped
rows are already enriched; re-sort in Python for the title tie-break).
- playlists PC1: read the live playlist once in push() and share it with plan_push
+ push_playlist (halves read-quota, closes the second TOCTOU window).
- playlists PC2/PC5: batch the cover thumbnails in one window query (was N+1);
rename combines count+duration into one aggregate + a single cover lookup.
- messages MB2: get_thread only resolves a messageable partner OR one we already
share a thread with (closes the user-enumeration oracle).
- messages MB3: push a live unread-count to the user's other tabs after mark-read.
- messages MC4: list_conversations uses DISTINCT ON + grouped COUNT instead of
loading the whole message history into memory.
- config AB3: per-spec allow_empty so smtp_user/youtube_api_proxy can be blanked
to disable them rather than snapping back to the env default.
- Extract _is_blocked(db, user, channel_id): the BlockedChannel EXISTS query was
copy-pasted in channel_detail, explore_channel, and block_channel.
- Extract _channel_summary(ch): the shared id/title/handle/thumbnail/subscriber/
video_count projection was hand-rolled in list_channels and discover_channels
(the detail dict interleaves these with extras, so it's left as-is).
Behavior-neutral. ruff clean, localdev boots healthy.
- feed.py _filtered_query: drop the dead 2nd return element (status_expr was
used only internally for WHERE filters; all 3 callers discarded it as _status).
Now returns (query, rank_expr); fixed the stale docstring.
- youtube/client.py: extract _iter_playlist_items() — iter_my_playlist_video_ids
and iter_playlist_items_with_ids were near-identical playlistItems paging loops
(jscpd [173-187]≈[267-281]); both now map over the shared generator.
- sync/videos.py: extract _apply_video_batch() with an on_missing callback —
enrich_pending and refresh_live shared the fetch-map-apply skeleton, differing
only in the query and how they retire rows YouTube omits.
- models.py: add LIVE_OR_UPCOMING = ("live","upcoming"); replace the 4 duplicated
copies (feed.HIDDEN_LIVE, search._LIVE_HIDDEN, videos.py + channels.py inline).
Behavior-neutral. ruff clean on touched files, localdev boots, feed/worker healthy.
Machine-baseline harvest (ruff + knip), all tsc/parse-green, no runtime change:
- backend: remove 3 unused imports (channels/playlists/youtube) via ruff; drop the
unused `job` binding in unshare_download (keep the _own_job ownership guard call).
- frontend: remove `export` from 23 internally-used-only symbols (knip "unused
exports") to shrink the public surface; delete 2 genuinely-dead declarations
(PlexBrowseResult — leftover from the removed /browse route; WIDGET_TITLES —
hardcoded English titles superseded by i18n).
Held back for a decision (unused here = possibly-unwired, NOT dead — flagged, not
removed): e2ee.lock()/clearDevice() (security primitives never wired to logout / a
"forget device" feature) and loadDefaultViewFilters (App reimplements it inline — a
DRY issue). See siftlode-ops/CODE-HYGIENE.md.
- Live-search view: a results-count selector (20/40/60/100) replaces manual load-more (the free
scrape source pages until that many are gathered); an 'these results are temporary' banner with
a 'Clear now' button that discards them 'as if never added' (api.clearSearch) and returns to
the feed.
- Channel blocklist: a Block/Unblock toggle + 'Blocked' badge on the channel page (blocked
channels don't auto-explore and their videos are hidden), and a 'Blocked channels' section in
the Channel manager with one-click unblock. ChannelDetail.blocked from the backend.
- Admin: a 'Purge discovery' button on the Scheduler page (immediate un-kept search/explore
cleanup). EN/HU/DE throughout.
- Live-search results stay ephemeral: the discovery-cleanup job now also reclaims un-kept
via_search videos (no watch state / not playlisted / channel not subscribed) after a grace
period (search_grace_days), and POST /api/search/clear discards a given result set 'as if
never added' (drops the user's search-finds + deletes the now-orphaned, un-kept videos).
Admin POST /api/admin/purge-discovery runs it on demand (grace 0).
- Count selector: GET /api/search/youtube gains a 'limit' — the free scrape source pages
through continuations until that many results are gathered (no manual load-more); the API
source stays one ≤50 page (cost).
- Per-user channel blocklist (migration 0034 blocked_channels): block/unblock/list endpoints;
blocked channels' videos are dropped from live search before ingest and hidden from the feed
/ Library / explore / channel page; explore refuses a blocked channel.
- New-first ordering: results you already have (subscribed channel, watched/in-progress/saved/
playlisted) sink below genuinely-new discoveries, preserving relevance within each group.
Adds the backend for a dedicated channel page + ephemeral browsing of un-subscribed
channels:
- migration 0033: channels.{total_view_count,published_at,banner_url,external_links,
from_explore}, videos.via_explore, explored_channels (per-user, grace-clocked).
- enrich channels with part=brandingSettings (banner/links) + statistics.viewCount +
snippet.publishedAt — no extra quota.
- GET /api/channels/{id}: About detail + this user's relationship; lazy-enriches the new
About fields (published_at sentinel).
- POST /api/channels/{id}/explore (require_human, quota-reserve guarded): records the
exploration, flags the channel ephemeral unless followed, ingests one page of uploads
(via_explore) + enriches; returns next_page_token for load-more.
- feed: per-explorer gate — a via_explore video is visible only to users who explored or
subscribe to its channel, so exploration never leaks into everyone's catalog.
- subscribe = keep: clears from_explore + via_explore on the channel's videos.
- scheduler explore_cleanup job + explore_grace_days config: hard-delete explored-but-
unkept channels and their untouched ephemeral videos after a grace period.
Replace the leftover 'subfeed' name across logger names + log_config,
frontend localStorage keys, Postgres user/db/volume defaults in the
compose files, .env.example, config.py, backup/restore scripts and the
README. Pure rename; no behavioural change. localStorage keys move from
subfeed.* to siftlode.* (one-time UI-state reset is acceptable).
The quota-attribution action keys were inconsistent (mixed verbs, ad-hoc names,
English-only labels). Introduce a QuotaAction constants holder (one source of
truth), rename every attribution call site to it, and add migration 0020 to
rename historical quota_events.action values. The display label now resolves
from i18n (quotaActions.<key>, EN/HU/DE) instead of a hard-coded English map.
Scheduler job ids and progress phase labels are a separate namespace and are
left untouched.
Subscribing to a channel the user already follows on YouTube (a local/YouTube
desync — e.g. a stale import dropped the local row, so it resurfaced in the
discovery list) made YouTube return 400 "subscription already exists", which
the endpoint reported as a 502 — and the client shows a 502 as a transient
"connection lost" blip, not the real reason. Treat the duplicate as success:
record the subscription locally (the desired end state already holds; the next
resync fills in the resource id). Surface any other YouTube error as 422 with a
clear message, on both subscribe and unsubscribe, so it isn't mistaken for a
connection drop.
Add a "Discover from playlists" tab to the Channel manager that lists
channels appearing in the user's playlists they don't subscribe to, with
a one-click Subscribe.
- GET /api/channels/discovery: local join (playlist_items -> videos ->
channels) minus the user's subscriptions and their own channel. Enriches
stub channels' metadata up front (title/thumbnail/subscriber count via the
API key) so the user can judge a channel before subscribing; videos are
not pulled (the scheduler picks those up).
- POST /api/channels/{id}/subscribe: write-scope gated, subscriptions.insert
+ local Subscription with the returned resource id.
- YouTubeClient.insert_subscription / get_my_channel_id.
- users.yt_channel_id (migration 0019) caches the user's own channel id so
discovery can exclude it.
- Frontend: ChannelDiscovery DataTable, Channels tab toggle (persisted),
api methods, trilingual strings. Subscribe ships a typed notification
payload (ChannelSubscribedMeta) for the inbox to act on.
- Add Last upload, total Length and a Normal/Shorts/Live breakdown, computed on read in the
existing grouped query (~35ms over the full catalog — measured, so no denormalization).
- Subscribers shown compact (919K), since the YouTube API already rounds them.
- Tag cell now shows only the tags actually attached to a channel; a '+' opens a per-channel
picker to attach/detach, instead of listing every user tag on every row.
POST /api/channels/{id}/reset-backfill (admin-only): clear the channel's backfill
markers, re-opt into deep backfill and re-run its recent pull now — a re-fetch-from-
scratch trigger regardless of current sync state. Idempotent (videos upsert by id).
Track who burned how much YouTube API quota. A QuotaEvent audit log (migration
0009) records every spend with the triggering user (NULL = background/system) and
an action label, set via a request/job-scoped contextvar (quota.attribute) so no
call signatures change. User-initiated work (sync subscriptions, unsubscribe,
opt-in recent backfill, manual enrich) attributes to the user; scheduler work to
System, split by action.
- backend: QuotaEvent model + migration 0009; quota.attribute() contextvar;
record_usage logs events; entry points wrapped (routes/sync, routes/channels,
scheduler); GET /api/quota/my-usage + GET /api/quota/admin
- frontend: admin-only Stats page (header nav, page=stats) with daily bars +
per-user breakdown by action and range picker; 'Your API usage' in Settings ->
Sync for every user
Verified: attribution + endpoints compute correctly; events are per-user vs System.
deep_requested is per-subscription, but deep backfill is channel-wide and shared:
once any subscriber requests full history, the whole back-catalog arrives for
everyone. The chip only looked at the current user's flag, so a second subscriber
saw a misleading 'get full history' on a channel already queued by someone else.
Add a channel-level deep_in_queue to /api/channels and show an informational
'full history queued' badge (vs the owner's cancelable button) in that case.
When a user requests full history for a channel whose recent uploads aren't
fetched yet, run a one-channel recent backfill synchronously in the request so
the feed populates at once instead of waiting for the scheduler. Deep paging
still follows on the scheduler (recent-then-deep). The deep-toggle mutation now
also refreshes my-status and the feed.
Per-user opt-in to full-history (deep) backfill so a new user's unique
channels no longer trigger a big shared-quota burst.
- migration 0007: Subscription.deep_requested (default false); seed admins'
existing subscriptions to preserve today's global-backfill behaviour
- run_deep_backfill is now demand-driven: only channels at least one user has
requested are deep-backfilled; recent backfill stays unconditional (cheap)
- estimate_deep_backfill ETA helper (quota-bound) surfaced in /api/sync/my-status
- POST /api/sync/deep-all to opt all my channels in; PATCH channels accepts
deep_requested
- UI: per-channel Full history toggle, Backfill everything action, deep
progress + ETA in Channels header and Settings - Sync
Backend: /api/channels (list + PATCH priority/hidden + attach/detach user tags),
user-tag CRUD on /api/tags, /api/sync/my-status (per-user channel sync counts).
Frontend: feed|channels page nav (URL-synced) from the account menu; a slide-in
tabbed Settings panel (Appearance, Notifications=6b sound+duration, Sync status +
actions + admin pause/resume, Account); a channel manager with priority, hide,
per-channel user tags, sync badges and 'view in feed'. Notifications now honor the
configurable sound + auto-dismiss settings.
peter