A season marked watched, a pull from Plex and a YouTube resume point made their first row by a read
then an add, and failed on the unique key when a first checkpoint or the watched mark made it in
between. They go through the same insert-then-read as the checkpoint (the resume point is one
upsert); a pull that meets a local change still waiting for its push leaves it.
Folding duplicate Watch later lists moved their videos but not a Home pin, which the cascade then
deleted; the first pin of a dropped copy moves onto the kept list.
Marking one notice read trimmed before its own flag was flushed, so the trim left one past the cap.
The route test now crosses the cap and reads the result from a session of its own.
The last Plex HTTP files do no database work of their own, so ALLOWED is empty: no route builds
SQL, touches the session or a unit's table. Discover's rails move whole into plex/discover.py; a
user's playlists, watch-state writes (the race-safe finish, an early checkpoint, a status, a
season or show at once) and the watch-sync link go through plex/queries.py and plex/service.py.
The plex package joins FENCED: it reads no other unit's tables, says whose a row is through
userscope (the rollup and the watch sync's 8 inline filters moved to owned_by), and commits only in
its named jobs — the library sync, TMDB enrichment, the watch sync and its background pushes.
A playlist's has-item probe keeps the plain key lookup, so a NUL in it is still the 422 a test pins.
Pins 178 -> 0; basedpyright baseline 118 -> 106.
PlexLibrary, PlexShow, PlexSeason, PlexItem, PlexPerson, PlexCollection, PlexState, PlexLink,
PlexPlaylist and PlexPlaylistItem leave app/models.py for app/plex/models.py; app.tables registers
them. Every import is re-pointed; no schema change.
The code cited its plan instead of saying why: `E28 review J4`, `(R6 S3b)`, `S1 justified…`,
`deep-review round 5`. That plan lives in the private ops repo, so for every reader of this one the
citation was noise, and a sentence whose subject was the plan step ("round 2 showed…") hid its
lesson behind an id. Each one is gone and the "why" kept: a bare citation is deleted, a sentence
carrying a lesson is reworded around it ("a later attempt showed…").
Measured (`scripts/audit_metrics.py`, product code): 652 lines in 270 files carried an epic code by
the definition the next commit holds (507 / 241 by the old, two-digit-only one) -> 0. Across all
code, tests, scripts and configs: ~1100 epic-code lines plus ~320 sprint/round ids.
Behaviour unchanged, checked mechanically: every changed .py file has the same AST once docstrings
are blanked; css/sh/toml/yml/json files are identical once comments are stripped; the only
non-comment changes in TS/JS are test titles, one description string in a contract table, and five
ESLint messages, whose text is all the gates snapshot diff holds. Not touched: Markdown (docs and
the changelog keep their history), Alembic migrations, TV episode notation (`S2 · E5`), DESIGN.md
ids (`C1`, `B4`, `S3h`), tracker ids (`SEC-16`, `DX-17`).
One stale claim fixed on the way: test_hls_reaper's docstring said the player checkpoints "every
10 s whether or not it is playing", which plex/stream.py calls wrong; it now says what the code does.
A mirrored library that answers "0 items" is not believed (its rows stay), but
that verdict stopped the whole reconcile: one library emptied on Plex with its
mirrored items kept made every run incomplete, and un-watches in every other
library never synced again. Each section now settles on its own walk, and only
the settled sections' rows are cleared; the unsettled ones are named in the
warning.
A library left out of the allowlist is never scanned, so its watch states were cleared on
every reconcile; clearing is now limited to the sections walked. A section we mirror items
from that says it holds 0 is not believed complete. The page size is no longer read as the
section total when totalSize is missing (measured: the server always sends totalSize on a
paged request, 0 for an empty one).
E32 review round 1 (B1, B2, B3, B8).
- Plex watch reconcile: a section the server SAYS is empty (totalSize 0, nothing listed) is a
complete listing. It read as "cut short" forever, so one empty library stopped un-watches from
ever syncing. A listing with no total at all stays incomplete (the client now returns None for
it instead of 0), and a whole scan that listed nothing still clears nothing. Tests now cover
completeness across two sections, both orders.
- Plex collections: no members from a collection whose listing counts some (childCount) keeps
the membership we hold, in the full sync and in the targeted resync after a write.
- YouTube playlists: no items for a playlist whose listing counts some (itemCount) keeps the
mirror in the bulk pull, and is refused as retryable in "reset to YouTube" and in the push
plan/merge, where it read as "YouTube deleted them all". The rule sits beside vanished().
- A playlist deleted on YouTube answered "reconnect your account"; it now says it is gone
(errors.youtube.gone, HU+EN).
- The pure 3-way merge moved to sync/playlist_merge.py; playlists.py is back under 800 lines and
off the size list.
The daily watch reconcile reads "absent from Plex's scan" as "Plex un-watched it" and deletes the
row — but it walked the sections without asking _paginate whether the walk reached the server's
total, so a Plex hiccup mid-walk (an empty page early) un-watched every title it failed to reach
until the next complete run. The scan now reports completeness; an incomplete one still pulls in
what it saw, re-pushes dirty rows, and clears nothing (logged as plex.watch.reconcile_incomplete).
The HU overlay had the same shape: clear every row's HU text, then refill from the listing. It now
reads the listing first and clears only after a complete one. The overlay moved to plex/overlay.py
(plex/sync.py drops under the 800-line limit and off the size ratchet's list).
Tests: plex/paths (23% -> 100%: prefix boundary, longest prefix, dead mount), the reconcile's
clear/keep/re-push rules, the cut-short scan and listing (each seen red with the guard removed).
114 exception handlers neither logged nor re-raised. The defect was never that
they are silent — plenty of them should be — it is that their silence said
nothing, so nobody could tell a decision from an oversight. Each one has now been
read. 44 got a line; the other 70 carry `# silent-ok: <why>`.
What the silent ones had been hiding, in the cases that got a line:
- A failed ffprobe sent every HDR file down the 8-bit pipeline, so playback
degraded to a software transcode with nothing saying why; a failed audio probe
collapsed a multi-audio file to one track and the player showed no language
picker; a failed first-PTS probe left subtitles seconds out of sync.
- A subtitle sidecar that existed but would not parse made a PARTIAL track read as
complete, permanently — the viewer loses every cue past the cached window.
- An unreadable crop/trim/cut-list in an edit spec was dropped, so the clip came
out uncropped, untrimmed, or shorter than the one the user built.
- A dropped `max_height` downloaded at best quality against an explicit cap.
- A failed videos.list enrich made every candidate undroppable-from-judgement and
therefore dropped: a working search rendered as "no results".
- An unusable password hash answered as "wrong password", locking the owner out
and telling them they mistyped; a rotated TOKEN_ENCRYPTION_KEY turned every
stored token into "not connected".
- A stored admin setting that would not coerce fell back to the env default while
the settings page went on showing the saved value.
- A one-off InnerTube config failure pinned the whole process to the hard-coded
fallback key, and searches return nothing when that goes stale.
- The HLS sweep reported 0 swept — like a clean run — when it could not list its
own root, leaving every crashed session's segments on disk forever.
Four of them are counted rather than logged per item (dropped cues, dropped cut
list segments, unreclaimed GC files) and reported once per call, because the
alternative is one line per cue over a whole subtitle track.
ruff gains S110 and BLE001. Every blind `except Exception` that remains carries
`# noqa: BLE001 — <why>`, which is the point: catching everything is sometimes
right, and always worth a sentence. plex/sync.py's three best-effort backfill
catches now keep the traceback — a blind catch over a batch that fetches AND
writes could be Plex, the database or our own bug, and `%s` of the exception told
them apart badly. main.py's boot-time log-level read likewise.
logs.event's `code` is positional-only now: `event("x", code=...)` was a TypeError,
and `code` is exactly the word a caller reaches for when the thing that failed has
one of its own.
tests/test_swallowed_exceptions.py is the wider rule ruff cannot see: a handler
that neither logs nor re-raises must say why. Both cases mutation-checked — a
marker removed and a log line removed each fail it, naming the handler.
25 literal names covered 34 modules. plex/client.py, plex/stream.py,
plex/subtitles.py, plex/sync.py and two Plex route modules all called themselves
`siftlode.plex`; downloads/pipeline.py called itself `siftlode.worker`;
routes/channels.py called itself `siftlode.api`. The `logger` column is in every
stored row so E22.6's console can filter on it, and so an operator can pin a level
to one part of the system — neither works when six modules share a name.
`logs.get_logger(__name__)` derives it: app/plex/sync.py -> siftlode.plex.sync.
One name per module, and it cannot drift from the file it sits in.
tests/test_logger_names.py pins the derivation, that everything stays under the
one `siftlode` tree the handler and the tests attach to, and that no module
hand-rolls a name again unless the line says why (`# logger-name-ok: <why>`).
Two do: get_logger's own call, and `siftlode.http`, which is named for the traffic
it carries rather than the module it lives in — it replaced uvicorn's access log
and is what an operator pins when they want request lines and nothing else.
app/scheduler.py called its logger `logger`; every other module calls it `log`.
ruff gains LOG, G, T20, TRY400, TRY401 and S112. Four findings cleared with them:
worker.py logged `str(exc)[:120]` where the traceback was the whole question, and
three prints/f-strings that are deliberate now say so on the line. S110 and BLE001
stay off until S3 — 23 handlers are outstanding and each is a judgement about one
of them; enabling the rules first would just mean 23 blanket noqas.
The incremental watch-sync (run_plex_watch_sync) learned a movie's watched
transition only from the history feed + onDeck. When a movie is finished on
Plex (commonly by scrubbing through the end credits) Plex sets viewCount>0,
clears viewOffset and DROPS it from Continue-Watching, so onDeck no longer
lists it (and onDeck can only ever ADD an in-progress row, never resolve one),
while a scrub-to-end frequently emits no history session row. The stale local
status='new', position>0 row then survived until the DAILY full reconcile —
which is why movies kept showing "in progress".
Add _reconcile_inprogress_movies: for each local in-progress MOVIE that Plex's
onDeck no longer lists, read its current per-account viewCount/viewOffset with
one metadata() call and apply it via the shared _pull_apply, flipping to
watched when Plex says so. Bounded by a probe cap (100), most-recently-touched
first; episodes are left to onDeck/history. No-signal rows are preserved (an
un-watch stays the daily reconcile's job).
Tests: watched-flip, onDeck+episode skip, and no-signal-preserved.
Deep-review of the quickfix batch surfaced two data/preference-loss bugs plus
smaller items; all fixed with tests.
- watch_sync _pull_apply: don't let the Plex-authoritative pull overwrite a DIRTY
local row (a change whose immediate push failed) — that stranded the value
forever since _repush_dirty only retries dirty rows. Leave it dirty for re-push.
- PlexProvider / ChannelsProvider: a partial shared link opened in a fresh tab
persisted mount-time DEFAULTS over the account's stored scope/sort/filters
(base read before the account was pinned). Persist/keep only URL-present dims;
re-hydrate the omitted ones from storage.
- playback resume write-back: guard against out-of-order delivery (a superseded
older push now drops) and evict the throttle map so it can't grow unbounded.
- PlexPlayer: announce mid-playback buffering to assistive tech (role=status +
sr-only label); new plex.player.buffering (HU+EN).
- columnSort: hoist the shared key:dir sort codec used by both URL-state modules.
- Tests: dirty-guard + flag-settle + per-key throttle scoping + superseded-push
(backend); hold-repeat/suppress-click/unmount + partial-link no-clobber +
refetch-doesn't-clobber-undo (frontend). Backend pytest 475, FE vitest 519.
The incremental/reconcile job docstrings still described last-write-wins and a
30-min cadence; update them to match the S2 behavior (Plex wins on divergence,
15-min incremental) so the "why" doesn't drift from the code.
Make Plex the source of truth for watch state:
- Write-back during playback: item_progress now pushes the resume position to
Plex on the periodic checkpoint too (not only the final flush), throttled to
one push per item every ~10s (in-memory, single-process rate limiter). The FE
now also fires an immediate (debounced) push on seek.
- Faster incremental pull: 30 → 15 min (plex_watch_sync_interval_min).
- Plex-authoritative pull: _pull_apply resolves a genuine divergence to Plex's
state instead of last-write-wins; the _same_state + synced_to_plex short-circuit
still prevents a freshly-pushed row from bouncing. Removed the now-unused
_LWW_TOLERANCE_S / _sift_ts.
Tests: 5 new in test_plex_watch_sync.py (Plex-wins resume/watched, no-bounce,
throttle final vs periodic).
Mechanical 1.x->2.0 conversion of the self-contained plex modules, behaviour-
equivalent (the characterization net + existing plex tests stay green, 39
passed): app/plex/{sync(15),watch_sync(9),people(1),imgcache(4)} and
app/routes/plex/{admin(4),playback(8),_shared(1)}. Query builders become
select(); .first()/.all()->execute().scalars(); a bulk .update() ->
execute(update()); a single-column .first() -> db.scalar().
browse.py (61) + rollup.agg_subquery (the atomic unit with unified_library)
remain for wave 2.
Closes the Plex #9 backend backlog (frontend was the prior follow-up). Behavior-
neutral cleanup + two two-way-sync bug fixes; PW2 verified a non-bug.
fix(plex): watch-sync PW1 pagination + PW3 lost-update guard
- PW1 watch_history: the history is a GLOBAL cross-account feed read as a single
500-row page, then filtered to the owner — on a busy family server the owner's
recent views can sit past page 1 and be missed. Now paginate (desc) until the
since-cutoff, bounded by max_pages (daily reconcile is the backstop; logs if hit).
- PW3 push_state_to_plex: it flagged synced_to_plex=True unconditionally after a
push, so a local edit landing between the push and the flag-set was marked clean
and never pushed (lost update). Capture the row's freshest timestamp when the
push is scheduled (_push_watch) and only settle the flag if the row is unchanged.
- PW2 was flagged as "accountID hardcoded to 1" but is NOT a bug: on a PMS account 1
is reserved for the owner/admin and an owner link always holds the admin token —
added a clarifying comment so it isn't re-flagged.
chore(plex): backend dedup
- PC2 unified_library + facets shared a ~13-field filter Query signature + p-dict →
one LibraryFilters dataclass injected via Depends(); as_dict() feeds the builders.
- PS-C1 sync.py collection upsert dup (resync_collection ≈ _sync_collections) →
_upsert_collection().
- PS-C2 sync.py 8-field filterable-metadata block dup (_apply_item ≈ _sync_shows) →
_apply_facet_fields().
- PW-C1 _repush_dirty read duration from the mirrored PlexItem.duration_s instead of
a per-row plex.metadata() round-trip.
- PW-C2 rk→id map built inline in two places → _rk_to_id() helper.
F6: coalesce whole-show/season Plex watch-pushes into ONE background task
(push_bulk_state_to_plex) that reuses a single DB session + keep-alive Plex
client, instead of scheduling one task (own session + HTTP client) per episode.
F8: cache a show's live Plex enrichment (metadata + related) per rating_key for
a short TTL and fetch the two calls in parallel; repeat opens skip the network.
Raw payloads are cached; per-user shaping stays out. An empty related list is not
cached (plex.related swallows a transient failure as [] — don't pin it for the TTL).
F10: remove dead code — the pre-unified /browse route + browse(), the /people
route + people() + _person_photo(), api.plexPeople, interface PlexPerson, and the
orphaned plex.people i18n block. DRY: appendPlexFilters() shared by plexLibrary +
plexFacets; one exported plexDetailUi.Filterable (was Fil + Filterable); PlexInfo
migrated onto the shared plexDetailUi hooks/menu (DetailCustomizeMenu gained
overlay + extra props; useDetailPrefs exposes savePref; PrefToggle exported).
Reviewed (high) → clean. F7 (facet aggregate collapse) deferred: self-exclusion
gives each sub-aggregate a distinct WHERE, and no measurement shows /facets slow.
Completes the two-way watch-state sync with two scheduler jobs:
- plex_watch_sync (default 30m): pull recent Plex-side changes (watch history + on-deck, filtered
to the owner account) into Siftlode under last-write-wins (_pull_apply + a _same_state ping-pong
guard + skew tolerance), then re-push any still-unsynced local states.
- plex_watch_reconcile (default daily): full section rescan; uses synced_to_plex to settle what the
incremental feed can't — notably propagating a Plex-side un-watch (clear a previously-mirrored row
Plex no longer has) — while re-pushing never-synced local states and never touching hidden
(Siftlode-only) rows. Union-preserving.
PlexClient gains accounts/watch_history/on_deck; _scan_plex_states is factored out and shared with
the one-time import. Owner accountID is resolved once and cached on the link. Both jobs are
registered in the scheduler (pause-skip, activity tracking, run-now, admin-tunable intervals) with
trilingual (HU/EN/DE) labels + descriptions. New config default plex_watch_reconcile_interval_min.
Verified live against the real Plex server: read feeds, last-write-wins, dirty re-push, the
incremental job end-to-end, and a full reconcile that cleared exactly the one un-watched item with
zero collateral across 17976 scanned.
Mirror watched/unwatched/resume from Siftlode to a linked (owner) Plex account.
- PlexClient: scrobble / unscrobble / set_timeline (/:/scrobble, /:/unscrobble, /:/timeline).
- watch_sync: link_for_push() gates on owner + sync_enabled + initial_import_done; best-effort
push_state_to_plex() runs as a background task with its own session, never raises (Plex being
down must not break the user's action), and flips synced_to_plex on success so Phase C's pull
won't bounce it back.
- item_state / item_progress schedule the push: watched/unwatched immediately, resume only on a
"final" checkpoint (pause / pagehide / unmount) — not every 10s tick — so one watch doesn't spray
/:/timeline at the server. `hidden` is Siftlode-only and never touches Plex.
- Frontend: plexProgress / plexProgressBeacon carry a `final` flag; the periodic checkpoint is
non-final, pause/pagehide/leaving the player are final.
Verified live against the real Plex server (scrobble/unscrobble/timeline round-trip + restore;
link gating; synced_to_plex flip).
The Plex watch import inserted plex_states via per-row ORM add against a pre-read 'existing' set,
which is not safe when the enable call fires more than once while the multi-second import runs (the
toggle wasn't disabled during it): overlapping imports raced to INSERT the same (user_id, item_id)
rows and all but the first hit a UniqueViolation -> 500 (seen on prod). Rewrite the write path as a
chunked PostgreSQL UPSERT (on_conflict_do_update, rows de-duped by item_id) — idempotent and
concurrency-safe, Plex still wins on the intersection and Siftlode-only states are untouched.
Also disable the Settings toggle (and ignore its onChange) while a sync mutation is pending, so it
can't be fired repeatedly; adds a reusable 'disabled' prop to the Switch primitive.
Plex records watch state per Plex account; Siftlode per user in plex_states. A new plex_link table
maps a Siftlode user to a Plex account; for the owner (MVP) the row uses the server admin token
(uses_admin=True), so no separate Plex login. app/plex/watch_sync.py reads the owner account's
viewCount/viewOffset/lastViewedAt (already present in the catalog mirror's section listing) and
upserts them into the owner's plex_states — 'Plex is master' on this first import, but only where
Plex has a watch record (Siftlode-only states are preserved; union on the intersection). Idempotent.
Admin routes: GET/POST /api/plex/watch/link (status + enable/disable; first enable runs the import)
and POST /api/plex/watch/import (re-run). Migration 0051_plex_link. Two-way push (Phase B) and the
incremental Plex→Siftlode reconcile (Phase C) build on this in later ships.
peter