Failed downloads stored the raw yt-dlp/ffmpeg text and dumped it as a truncated red
one-liner in the queue. Classify the failure into a category (unavailable / login_required /
geo_blocked / format_unavailable / drm / postprocess / source_missing / network / unknown)
and surface it properly:
- backend: app/downloads/errors.classify maps the raw message to a category code; the worker
stores it in the new download_jobs.error_code (migration 0061) at both failure sites; resume
clears it; the serializer exposes it. Raw text kept as technical detail.
- frontend: the queue row shows a clean localized reason + a "details" link opening a modal
with the explanation, the raw text behind a disclosure, and a Retry only for transient
categories (network/postprocess/unknown).
- tests: classify() unit coverage over verbatim yt-dlp/ffmpeg message shapes.
First real customers of the DB lane:
- test_plex_facets: GET /api/plex/facets?scope=show returns 200 (not the
"cannot extract elements from a scalar" 500) for a show whose genres is a JSONB
scalar `null` — the 0.53.1 read-guard's regression test. Mutation-checked:
removing the jsonb_typeof='array' guard turns it red with that exact error.
- migration 0060 (S3a): one-time UPDATE …=NULL WHERE jsonb_typeof=… 'null' over the
plex_items/plex_shows tag columns, so pre-0.53.1 JSONB-null rows become uniform SQL
NULL and the read-guards go belt-only. test_plex_jsonb_normalize verifies scalar-null
→ SQL NULL while a real array is preserved.
Gate: ruff clean; DB lane 176 passed; pure-logic lane unchanged (skips the DB tests).
- C-3.7: fold a per-link password_version into the signed watch grant (migration
0059 adds the column) and bump it on every password change, so rotating a share
link's password invalidates outstanding grants at once instead of letting them
live out the 6h TTL. make_grant/check_grant take the version; update_link bumps.
- C-3.8: the share recipient picker and share-by-email now reuse the messageable
filter (not-demo AND active AND not-suspended) instead of only excluding demo,
so suspended/deactivated addresses aren't leaked or reachable as targets.
- C-3.9: GET /keys/{user_id} adopts get_thread's MB2 guard — a non-messageable
target with no shared history returns the same "User not found" as a missing id,
so it can't be probed to enumerate users or fetch suspended users' keys.
- C-3.12: _register_account catches IntegrityError on the insert (the select-then-
insert TOCTOU) and treats it as the already-registered no-op.
- C-3.13: trigger_job claims the running flag atomically under _activity_lock
(compare-and-set) instead of a bare check, closing the double-start window.
- Move _messageable/is_messageable_user into a shared app/userscope.py so downloads
doesn't import the messages route module.
Shared /watch links showed a "broken play button" on iPad (any iOS
browser — all forced onto WebKit) while playing fine on desktop Chrome.
Root cause: the stored mp4's video codec was AV1 (confirmed via ffprobe),
which WebKit can't decode. The existing H.264 guard was only a
format_sort *preference*, so a non-YouTube source (Facebook) that ships
AV1 as its best video-only stream slipped through.
- B1: for the mp4 compat profile, prefer H.264 at the SELECTION level
with a progressive fallback (bestvideo[vcodec^=avc1]+bestaudio /
best[vcodec^=avc1] / anything) so an avc1 progressive is chosen over
an AV1-only video-only stream.
- B2: re-encode the rare AV1/VP9-only result to H.264+AAC (+faststart)
in the worker before storing, gated on the compat profile; corrects
the asset codec metadata to match.
- Bump _SIG_VERSION 2->3 so cached AV1 assets re-derive.
- Default download preset is now 720p (migration 0058 re-seeds builtins
720p-first; ProfileEditor BLANK 1080->720).
- Tests for the selector, compat opt-outs, and the transcode predicate.
Lazy enrichment only covers viewed titles, so an actor's count/results were
partial. Add a background backfill: each plex_sync rich-fetches a bounded batch
(60) of not-yet-enriched movies/shows and persists their FULL cast, stamping
people_enriched_at so each is fetched once — the whole library is covered over
successive runs (migration 0057 adds the marker). Extracted the enrichment into a
shared app/plex/people.py (cast_from_meta / enrich_row_people / prune_orphan_people
+ the person-image host/proxy) so the info-page lazy path and the batch share ONE
implementation — no duplication; the info page also stamps the marker so the batch
skips it. Verified: an 8-item batch grows those movies' cast_names 3->24 and fills
plex_people (32->224); the info-page cast is intact after the _rich_meta refactor.
(Plex cast-explore S1b)
The cheap section listing that drives the main sync only stores the top ~3 cast,
so any cast member below that was unfilterable (clicking them AND-emptied the
grid). Now the movie/show info fetch (/item, /show) — which already pulls the full
rich cast — persists it onto cast_names + people_text (so every displayed cast
member becomes filterable) and caches each headshot in the new plex_people table
(migration 0056). Idempotent (only grows a thinner stored set); best-effort (never
500s a read). Verified: 10 Cloverfield Lane 3->8 cast after viewing; plex_people
populated with photos. (Plex cast-explore S1a)
- fix: reserve the scrollbar gutter (scrollbar-gutter:stable) on the channel
page's scroll container, so switching between the tall Videos tab and the
short About tab no longer shifts the banner/avatar/buttons a few px sideways
(the vertical scrollbar was appearing/disappearing between the two tabs).
- About tab now shows Country (flag + localized name), Language, Topics
(topicCategories → readable chips), and Keywords (brandingSettings keywords
parsed into chips, quoted multi-word tags kept whole). country/language/topics
were already stored; keywords is new (migration 0055_channel_keywords, mapped
in apply_channel_details, returned by channel_detail).
- Discovery → channel page: the Channel-manager "Discover from playlists" tab now
links each channel name to its in-app channel page (ChannelLink onView), so a
discovered channel's About/videos can be inspected BEFORE subscribing (was
subscribe-only, plain text before).
Note: the channel info-card epic itself was already delivered in v0.19.0; this is
the About-tab enrichment follow-up + the header-shift fix. external_links stays
empty by design (YouTube removed the field from the Data API ~2023).
Signed client-side session cookies had no server-side kill switch: logout + password
reset couldn't evict a stolen/copied cookie (valid until expiry). Add User.session_epoch
(migration 0053), record it in the cookie at every login, and reject in current_user any
cookie whose recorded epoch is behind the account's current one.
- Bump the epoch on: password reset (kills ALL sessions — a reset is a compromise response),
password change + a new 'Log out other sessions' action (both re-stamp the CURRENT cookie
so the caller stays signed in, evicting only the others).
- Per-account epoch map in the session so one account's revocation doesn't evict the other
signed-in accounts in the same browser wallet.
- Missing epoch (pre-SA4 cookie) is treated as 0, so the first bump revokes grandfathered
sessions too.
- New POST /auth/logout-others + a Settings → Account 'Active sessions' button (trilingual).
Give TV shows the same filterable metadata as movies so the TV grid can be
filtered/sorted, not just library+sort. Backend: migration 0052 adds
rating/content_rating/studio/originally_available_at/genres/directors/cast_names/
people_text to plex_shows (+ GIN/indexes, people_text folded into search_vector);
_sync_shows populates them cheaply from the show section listing (no per-item
calls). /browse show-branch gains the movie filter set (minus duration) plus an
aggregate per-user watch-state (a show rolls up its episodes: all watched=watched,
any progress=in_progress, none=new) with year/rating/release sorts; /facets returns
show facets. Frontend: PlexSidebar renders the metadata filters + watch-state for TV
libraries (duration stays movie-only); show cards show a watched/in-progress badge.
i18n plex.inProgress (en/hu/de). Needs a Plex re-sync to populate the new columns.
Plex records watch state per Plex account; Siftlode per user in plex_states. A new plex_link table
maps a Siftlode user to a Plex account; for the owner (MVP) the row uses the server admin token
(uses_admin=True), so no separate Plex login. app/plex/watch_sync.py reads the owner account's
viewCount/viewOffset/lastViewedAt (already present in the catalog mirror's section listing) and
upserts them into the owner's plex_states — 'Plex is master' on this first import, but only where
Plex has a watch record (Siftlode-only states are preserved; union on the intersection). Idempotent.
Admin routes: GET/POST /api/plex/watch/link (status + enable/disable; first enable runs the import)
and POST /api/plex/watch/import (re-run). Migration 0051_plex_link. Two-way push (Phase B) and the
incremental Plex→Siftlode reconcile (Phase C) build on this in later ships.
Two visual gaps for non-catalog downloads:
- Channel link: YouTube already exposes channel_url; Facebook exposes none but a
numeric uploader_id that resolves at facebook.com/<id>. `_uploader_url` derives
it so the auto-detected channel renders as a real clickable link.
- Poster: a source with no thumbnail (e.g. a direct reddit HLS URL) showed a
blank image box. The worker now cuts a representative frame with ffmpeg
(`ensure_poster`) into the `<base>.jpg` sidecar and records `poster_path`
(migration 0050). The card, the public watch page (<video poster> + og:image),
and link previews fall back to it via new authed + public poster endpoints.
Adds `app.downloads.backfill` (one-off, re-run-safe) to fill uploader_url
(re-extract YouTube/Facebook metadata) and posters for pre-existing downloads.
The edit (pencil) action now edits a download's full display metadata — title,
channel name, channel link and any number of extra reference URLs — instead of
just the name. The channel and links render as clickable links on the library
card, and the channel link is auto-filled from the source (yt-dlp channel_url)
when available. Shared watch pages resolve the same per-download overrides, so a
rename/channel/link edit is reflected on the public /watch page too, with every
link clickable.
Adds migration 0049 (media_assets.uploader_url; download_jobs.display_uploader,
display_uploader_url, extra_links) and generalizes the rename endpoint into a
metadata update with URL validation. EN/HU/DE strings included.
Personal ordered lists of Plex items, kept in Siftlode's own DB (works for users
without a Plex account, like watch-state) — the "your own lists" counterpart to
shared collections. Plex-direction sync is a later phase (plex_rating_key
reserved).
Backend: migration 0048 (plex_playlists + plex_playlist_items with position),
PlexPlaylist/PlexPlaylistItem models, and per-user CRUD endpoints under
/api/plex/playlists (list [+?contains for the add dialog], create [seeded],
detail [ordered cards], rename, delete, add/remove item, reorder). _leaf_card
handles the movie/episode mix. Frontend: "Add to playlist" dialog from the movie
info page (all users), a Playlists section in PlexSidebar (list + create),
PlexPlaylistView (reorder up/down, remove, rename, delete, Play all), and
PlexPlayer gained an optional `queue` so play-through follows the list order
(prev/next + auto-advance). i18n en/hu/de. Verified end-to-end on localdev
(backend CRUD + the create→add→view→play-through UI flow).
Backend (migration 0047_plex_collections): a plex_collections table mirrors every Plex
collection (card metadata + smart flag + an editable flag reserved for Phase 2); membership
is stored as GIN-indexed collection_keys on member movies (plex_items) and shows (plex_shows).
The background sync (plex_sync) now fetches each library's collections + their children and
rebuilds membership — so ALL reads are local (Plex's dual-language collection queries are slow;
this trades a ~4-min background sync for instant reads). New /api/plex/collections endpoint;
/browse gains a combinable filter; item_detail returns the movie's collection
'strips' (sibling titles as playable cards, smallest/most-specific collection first) — all pure
local lookups.
Frontend: PlexSidebar gains a searchable Collection picker + an active-collection chip;
PlexInfo renders the collection strips (playable posters + 'Browse collection' → sets the
filter); the collection is part of PlexFilters (persisted). i18n en/hu/de.
Phase 2 (create/edit collections with write-back to Plex) is separate.
Backend (migration 0046_plex_people_search): a new people_text column (cast +
director names, de-duped) is folded into the generated search_vector at weight B,
so the top search box finds titles by an actor/director name and ranks them above
summary-only mentions. New /api/plex/people endpoint returns the cast/crew matching
the term (name prefix or word-start) with a film count + a headshot (pulled from one
representative film's live metadata; image bytes disk-cached).
Frontend: PlexBrowse shows matching people as virtual cards above the grid; clicking
one adds them to the actor/director filter (multi-value, from the C1 work) and clears
the search box so you land on exactly that person's films. Answers the 'why does
"drew" match Rambo?' confusion — it was matching the word in the synopsis; now names
are searchable. i18n en/hu/de.
⚠️ Prod needs a Plex re-sync after deploy to populate people_text (search_vector
regenerates automatically once the column has data).
Backend (migration 0045_plex_filter_meta): plex_items gains rating (audienceRating
~IMDb), content_rating, studio, originally_available_at, and GIN-indexed genres /
directors / cast_names — all mirrored from the cheap section listing (no per-item API
calls; they also seed a future watch-habit recommender). /browse gains genre / content-
rating / year / rating / duration / added-within / director / actor / studio filters
(@> containment, GIN) + sort by year|rating|duration|release; new /facets endpoint
returns available genres+ratings (with counts) and the year/rating/duration bounds. A
thin on-disk image cache (.plex-img-cache) serves posters/art/cast photos from local
disk after first fetch (~7-14x faster repeat loads).
Frontend: PlexSidebar grows the full filter set (facet-driven genre/age chips, rating
steps, year range inputs, duration buckets, added-within, active people/studio chips,
clear-all); filters persist per-account as one JSON blob. PlexInfo metadata (year,
genre, director, cast, studio, IMDb score) is clickable → sets the matching filter and
returns to the filtered grid (page variant only; the in-player overlay stays read-only
so a stray click can't stop playback). i18n en/hu/de.
Every download now records the clean source page URL and shows it on the
Downloads page (open in a new tab, or copy to clipboard). The worker stores
yt-dlp's canonical webpage_url on the asset (migration 0043 adds
media_assets.source_webpage_url); the serializer prefers it and falls back to a
URL derived from source_kind+source_ref, so YouTube, external YouTube links and
external URLs (e.g. Facebook reels) all get a correct reference, and queued/older
rows work before the worker fills it. Edit clips return null (a clip's source is
the user's own earlier download, not a web page). i18n en/hu/de.
Share a download by a capability URL (/watch/{token}) that anyone can play on a login-free page —
distinct from the registered-user ACL share. Per-link controls: optional expiry, allow-download
toggle (stream-only vs downloadable), optional argon2 password. Revoke = delete.
- migration 0042 + DownloadLink model; app/downloads/links.py (token, HMAC grant sign/verify)
- owner endpoints (POST/GET/PATCH/DELETE links) + /recipients for the internal user picker
- public router (no auth): watch meta / password unlock→signed grant / range-aware file serve
(inline vs attachment); unlock is rate-limited; meta verifies the file exists on disk
Verified end-to-end: 206 range, inline vs attachment, wrong-password 403, tampered-grant 403.
Embedding (thumbnail/chapters) rewrote the whole file — a 10 GB video spent minutes making a
10 GB temp copy — while the Plex .nfo + poster sidecars already carry that metadata. So:
- builtin presets no longer embed (migration 0040 re-seeds them; formats._DEFAULTS embed off);
only the tiny FFmpegThumbnailsConvertor (for poster.jpg) remains, so downloads go
video -> audio -> merge -> done with no full-file rewrite
- default preset is now 1080p (Best stays available, second in the list) so a long video
doesn't silently pull ~10 GB and blow the quota
Users who want self-contained files can enable embedding in a custom profile.
Noisy YouTube titles are cleaned for display + storage, raw kept in videos.original_title:
- app/titles.normalize_title: strip emoji/symbols (keep accents), remove trailing SEO hashtag
clusters (keep numeric #3 episode markers), context-aware de-shout (mostly-ALL-CAPS titles ->
Title Case with an acronym whitelist + function-word lowercasing; otherwise only long all-caps
words), collapse repeated punctuation
- applied at enrichment (sync/videos.py) and in the download worker (ad-hoc yt-dlp titles);
catalog downloads inherit the normalized title automatically
- migration 0039: add original_title, preserve raw, rewrite title (generated search_vector
regenerates); reversible via original_title
Backfill on localdev: 122115/273417 titles normalized in ~2 min. Verified in the feed + on
real messy samples (emoji/de-shout/hashtags), accents + acronyms (PS5/AI/USA/PC) preserved.
Per-user named snapshots of the feed's FeedFilters. New saved_views table
(migration 0035, unique per user+name, position, is_default) + SavedView
model, and a /api/saved-views router (list/create/patch/delete/reorder)
gated behind require_human so the shared demo account can't pollute it.
Setting is_default clears the flag on the user's other views (one default).
- Live-search results stay ephemeral: the discovery-cleanup job now also reclaims un-kept
via_search videos (no watch state / not playlisted / channel not subscribed) after a grace
period (search_grace_days), and POST /api/search/clear discards a given result set 'as if
never added' (drops the user's search-finds + deletes the now-orphaned, un-kept videos).
Admin POST /api/admin/purge-discovery runs it on demand (grace 0).
- Count selector: GET /api/search/youtube gains a 'limit' — the free scrape source pages
through continuations until that many results are gathered (no manual load-more); the API
source stays one ≤50 page (cost).
- Per-user channel blocklist (migration 0034 blocked_channels): block/unblock/list endpoints;
blocked channels' videos are dropped from live search before ingest and hidden from the feed
/ Library / explore / channel page; explore refuses a blocked channel.
- New-first ordering: results you already have (subscribed channel, watched/in-progress/saved/
playlisted) sink below genuinely-new discoveries, preserving relevance within each group.
Adds the backend for a dedicated channel page + ephemeral browsing of un-subscribed
channels:
- migration 0033: channels.{total_view_count,published_at,banner_url,external_links,
from_explore}, videos.via_explore, explored_channels (per-user, grace-clocked).
- enrich channels with part=brandingSettings (banner/links) + statistics.viewCount +
snippet.publishedAt — no extra quota.
- GET /api/channels/{id}: About detail + this user's relationship; lazy-enriches the new
About fields (published_at sentinel).
- POST /api/channels/{id}/explore (require_human, quota-reserve guarded): records the
exploration, flags the channel ephemeral unless followed, ingests one page of uploads
(via_explore) + enriches; returns next_page_token for load-more.
- feed: per-explorer gate — a via_explore video is visible only to users who explored or
subscribe to its channel, so exploration never leaks into everyone's catalog.
- subscribe = keep: clears from_explore + via_explore on the channel's videos.
- scheduler explore_cleanup job + explore_grace_days config: hard-delete explored-but-
unkept channels and their untouched ephemeral videos after a grace period.
Makes the local search behave more like YouTube's — finding videos by the
uploader's own keywords or the query that surfaced them, not only words in the
title. A DB-generated, weighted search_vector (migration 0032) replaces the
title-only FTS index:
- keywords: the creator's snippet.tags (free — already in the snippet we fetch),
stored on enrich.
- search_terms: distinct live-search queries that surfaced the video (across all
users), appended by the search route — folds YouTube's relevance into local
search (a video YT returned for a query becomes findable by it even without a
title match), the user's own idea.
- description (truncated) for broad recall on the existing catalog.
Weighted title(A) > keywords+queries(B) > description(C) so ts_rank keeps title
hits on top. A plain GIN index on the generated column guarantees index use (no
expression/param matching). Verified on localdev: recall 146->213 for one query;
7 'eurovision' hits via the document but not the title; index scan confirmed.
Two related search improvements:
1) Your own live-search results now belong to your Mine feed. A new per-user
search_finds table (migration 0030) records each video you surface via your
YouTube search (the route inserts them idempotently). The Mine feed becomes
'your non-hidden subscriptions OR your search finds', and the Source filter
now applies in Mine too: organic = subscriptions, search = your search finds,
all = both (default stays organic, so the main feed is unchanged). The shared
Library keeps using the global via_search flag.
2) Feed search ranks by relevance instead of a whole-phrase substring. A custom
unaccent_simple text-search config + GIN index (migration 0031) back a
YouTube-like fuzzy match: word-order-independent, multi-word AND, prefix on
the word being typed, accent-insensitive. A new 'relevance' sort orders by
ts_rank; the channel name still matches as a substring. The rank is scaled to
an integer so the keyset cursor pages it exactly (a raw float4 breaks paging).
_filtered_query returns the rank expr so only the feed list uses it.
Feed text search used plain ILIKE, which is case- but not diacritic-insensitive, so 'tiesto'
missed the many titles spelled 'Tiësto' — a search that ingested ~45 results showed only ~12.
Enable the postgres unaccent extension (migration 0029) and wrap both sides of the title/
channel match in unaccent(), so 'tiesto' now matches 'Tiësto'. Applies to feed, count and
facets alike.
Add a live YouTube search that materialises results into the shared catalog so
they render with the normal feed cards + in-app player and gain per-user state.
- YouTubeClient.search_videos(): search.list (100 units), embeddable-only, returns
flat stubs + nextPageToken; surfaces liveBroadcastContent for live filtering.
- routes/search.py GET /api/search/youtube: require_human + per-user daily cap
(search_daily_limit_per_user, default 70) + can_spend pre-check (429 on either);
drops live/upcoming, upserts channel stubs (channels.list) + video stubs, enriches
(videos.list), runs the youtube.com/shorts probe, then excludes Shorts/live and
returns feed cards in relevance order with the YouTube pageToken as the cursor.
- Provenance: videos.via_search / channels.from_search (migration 0028) flag
search-discovered rows; the feed hides them from the Library (scope=all) by default
via exclude_search_discovered, leaving the Mine feed untouched.
- quota.actions_today() counts a user's per-action events today for the cap; only the
search.list call is attributed VIDEOS_SEARCH so the counter is exactly 1 per search.
Private user-to-user messages are end-to-end encrypted: the server only ever
stores ciphertext + iv and acts as a key directory (public keys) plus an opaque
store for each user's private key, wrapped client-side with a passphrase the
server never sees — so not even an admin can read a conversation. A separate
kind=system message (plaintext, no sender) powers a server-authored Siftlode
welcome shown on first open, reusable later for announcements.
- models: rework Message (kind, nullable sender/body, ciphertext+iv) + MessageKey;
migrations 0026 (table) + 0027 (E2EE rework).
- routes/messages.py: key directory/blob endpoints, ciphertext send, conversations
+ threads (system + user), lazy welcome, all gated by require_human.
- realtime.py: in-process WebSocket connection registry; /ws delivers sent
messages to a user's open tabs instantly (sync-callable push, single-process).
- app_state gains configured + setup_token_hash (migration 0025); existing installs (any users)
are marked configured so they never see the wizard, a fresh DB starts in setup mode.
- On first boot the app mints a one-time setup token and logs the wizard URL (only the hash is
stored); the token gates the setup steps (added in 6c).
- A middleware locks all /api + /auth routes (except /api/setup, /api/version, /healthz, static)
to 503 until configured, with a DB-free cached fast-path once setup completes.
- GET /api/setup/status tells the SPA whether to show the wizard.
- New Users page tabs (Users & roles / Access requests / Demo) and a tab-ified Configuration
page, both via a reusable Tabs component (persisted active tab).
- Admin can suspend/unsuspend (migration 0023 adds users.is_suspended) and delete accounts
from the Users & roles tab, with guards (demo / self / last admin).
- Email notifications to the affected user: suspended (reactive, on a blocked sign-in),
reinstated, role changed, and account deleted; the approval email now carries a clickable
app link. The scheduled/manual demo reset also seeds sample channel subscriptions.
- Hide the 'unverified email' chip for Google accounts (Google attests the email).
- Link a Google account to a password account, and adopt the Google identity onto a matching
email account instead of 500ing on a duplicate; set or change a password from Settings.
- Expose has_google/has_password on /api/me for the Sign-in methods UI.
- Mark Google logins email-verified (backfill existing rows, migration 0024); stop a routine
login from clobbering an admin-assigned role (env ADMIN_EMAILS stays the bootstrap admin).
- Suspension login-gates (password + Google callback + current_user) with a rate-limited
'suspended' notice; shared purge_user (cascade delete + access-request cleanup + Google-grant
revoke) behind self- and admin-deletion; single app_base source for user-facing email links.
Add email+password auth alongside Google: argon2id hashing; users gains
password_hash/email_verified/is_active and google_sub becomes nullable (migration
0022); a single-use, hashed auth_tokens table for email verification + password
reset. Registration creates a pending (inactive, unverified) account + an access
request; sign-in needs verified email AND admin approval. Anti-enumeration: uniform
register/login/reset responses (a correct-password owner still gets a specific
pending reason). allow_registration flag (DB-tunable). Admin users-list + role
endpoint (guards: not self, not demo, not the last admin); approving access (or a
manual whitelist add) now activates the matching pending account. current_user
rejects deactivated accounts. Verified end-to-end via curl + DB.
Add a generic admin-editable config layer (epic 4a): system_config KV table
(migration 0021), a sysconfig registry that is the single source of truth for
DB-overridable keys (type/group/default/bounds/secret) + a DB-override-or-env
resolver, and admin endpoints (GET/PATCH/DELETE /api/admin/config + a test-email
probe). Secrets are Fernet-encrypted at rest (TOKEN_ENCRYPTION_KEY); without it
they stay env-only. First group wired end-to-end: Email/SMTP — email.py now
reads host/port/user/from/password via the resolver (own session, since email
is sent from background tasks).
The quota-attribution action keys were inconsistent (mixed verbs, ad-hoc names,
English-only labels). Introduce a QuotaAction constants holder (one source of
truth), rename every attribution call site to it, and add migration 0020 to
rename historical quota_events.action values. The display label now resolves
from i18n (quotaActions.<key>, EN/HU/DE) instead of a hard-coded English map.
Scheduler job ids and progress phase labels are a separate namespace and are
left untouched.
Add a "Discover from playlists" tab to the Channel manager that lists
channels appearing in the user's playlists they don't subscribe to, with
a one-click Subscribe.
- GET /api/channels/discovery: local join (playlist_items -> videos ->
channels) minus the user's subscriptions and their own channel. Enriches
stub channels' metadata up front (title/thumbnail/subscriber count via the
API key) so the user can judge a channel before subscribing; videos are
not pulled (the scheduler picks those up).
- POST /api/channels/{id}/subscribe: write-scope gated, subscriptions.insert
+ local Subscription with the returned resource id.
- YouTubeClient.insert_subscription / get_my_channel_id.
- users.yt_channel_id (migration 0019) caches the user's own channel id so
discovery can exclude it.
- Frontend: ChannelDiscovery DataTable, Channels tab toggle (persisted),
api methods, trilingual strings. Subscribe ships a typed notification
payload (ChannelSubscribedMeta) for the inbox to act on.
Expose the maintenance job's rolling re-validation batch (videos re-checked per
run) as an admin control on the Scheduler dashboard, stored in app_state
(migration 0018; NULL = the env/config default). The job reads the effective
value each run via a state helper, clamped to a sane range. Trilingual.
Add a server-backed notification center that coexists with the client-side
transient bell: a per-user `notifications` table (type/title/body/data JSON/
read/dismissed), a `/api/me/notifications` CRUD API (list, unread_count, read,
read_all, dismiss, clear), and a left-nav inbox module with a live unread badge
polled via useLiveQuery. Known types render trilingual text from type+data
(English stored text is the fallback); read rows are trimmed past a soft cap.
Also adds the schema the maintenance job builds on: videos.list?part=status
columns (embeddable/privacy_status/upload_status) and the validation lifecycle
columns (last_checked_at, unavailable_since, unavailable_reason).
Page-id validation is centralized in one PAGES source of truth (isPage) so the
new page survives reload without a second allowlist to keep in sync.
Add scheduler_settings (per-job interval override, migration 0015) and a
PATCH /api/admin/scheduler/jobs/{id} that persists the override and live-
reschedules the running APScheduler job. Intervals load from the DB (env
defaults as fallback); the snapshot reports the live trigger interval.
Add the shared demo-account plumbing: users.is_demo marks the single
shared demo user, demo_whitelist holds the admin-curated emails that may
enter it without Google sign-in, and a small in-process RateLimiter
(generic groundwork) for throttling the demo-login endpoint per IP.
Store a SHA-256 of each playlist's name + ordered video ids as last synced from /
pushed to YouTube (migration 0013, backfilled for already-clean linked playlists).
'dirty' is now recomputed on every edit by comparing the current fingerprint to that
baseline instead of being stickily set true. So manually restoring the original order
(or undo back to it) clears dirty on its own — the Reset to YouTube button disappears
and no YouTube read is needed. Baseline is set on read-sync, repull (reset), and a
clean push; a missing baseline counts as dirty (unknown YT state). Verified the
migration's fingerprint matches the app's runtime fingerprint exactly.
The old per-video status='saved' becomes membership in a built-in, undeletable
'watch_later' playlist. Migration 0012 moves existing saved videos into each user's
Watch later list and demotes the states to 'new'. The feed/playlist serializers now
expose a 'saved' boolean (EXISTS in watch_later); the card bookmark toggles watch_later
via new /api/playlists/watch-later add/remove endpoints (create-on-demand) instead of
setting a status. Removed the 'saved' show-filter and status. Watch later shows a
localized name and hides rename/delete in the Playlists page and add-to-playlist popover.
Add per-user local playlists: Playlist + PlaylistItem models (with forward-looking
kind/source/yt_playlist_id/dirty columns for the later YouTube-sync phases) and
migration 0011. New /api/playlists routes: list (with optional contains=<video_id>
membership flags for the add-to-playlist popover), create, get detail (items
serialized via the feed serializer, with per-user watch state), rename, delete,
add/remove item, and reorder. Watch later (built-in) is protected from deletion.
Add position_seconds (+progress_updated_at) to video_states so watch progress
survives across devices and can drive a feed filter. New POST
/api/videos/{id}/progress checkpoints the player position (clearing trivially
-early and near-finished positions). Feed serialize exposes position_seconds and
a show=in_progress filter lists started-but-unfinished videos. Un-marking
'watched' now keeps a stored position instead of deleting the row.
Track who burned how much YouTube API quota. A QuotaEvent audit log (migration
0009) records every spend with the triggering user (NULL = background/system) and
an action label, set via a request/job-scoped contextvar (quota.attribute) so no
call signatures change. User-initiated work (sync subscriptions, unsubscribe,
opt-in recent backfill, manual enrich) attributes to the user; scheduler work to
System, split by action.
- backend: QuotaEvent model + migration 0009; quota.attribute() contextvar;
record_usage logs events; entry points wrapped (routes/sync, routes/channels,
scheduler); GET /api/quota/my-usage + GET /api/quota/admin
- frontend: admin-only Stats page (header nav, page=stats) with daily bars +
per-user breakdown by action and range picker; 'Your API usage' in Settings ->
Sync for every user
Verified: attribution + endpoints compute correctly; events are per-user vs System.
Move the access whitelist from the ALLOWED_EMAILS env var into a DB Invite table
(env kept as bootstrap fallback), and add a self-service request + admin approval
flow with fail-soft email.
- models: Invite(email, status pending|approved|denied, requested_at, decided_*)
- migration 0008: invites table; seed env ALLOWED_EMAILS u ADMIN_EMAILS as approved
- auth: is_allowed() (DB-first, env fallback); a denied Google login records a pending
request and bounces to /?access=requested instead of a raw 403; public POST
/auth/request-access; upsert is idempotent so repeats don't re-spam admins
- routes/admin.py (admin-only): list/approve/deny invites + manual add
- email.py: smtplib + Gmail App Password, fail-soft (skips if SMTP unset)
- /api/me exposes pending_invites; config + .env.example gain SMTP_*
- UI: Login 'Request access' form + access=requested/denied handling; Settings ->
Access requests (approve/deny + add); admin nudge toast on pending requests
Verified locally: request-access creates a pending invite and emails the admin;
seed approved npeter83; guinea-pig yt.trash2023 denied until approved.
peter