fix(sync): an empty answer no longer empties a mirror; maintenance checks each video once a run

The catalog prune and the subscription import have long refused to reconcile against an empty
listing that contradicts what we hold (should_prune: a 200 with no items is a blip far more often
than a user who removed everything). Two mirrors did not: the YouTube playlist mirrors (dropped
every mirror) and the Plex collections (dropped every collection with its user-set `editable`
flag, and every member's keys — the guard landed in plex/sync.py with the previous commit). Both
now hold the same rule; test_empty_listing_guards covers each way (seen red).

sync/maintenance (19% covered) gets a lifecycle test: recover, grace, delete + one notification
per user, live stream kept, abandoned premiere. It found the rolling revalidation re-fetching the
same videos until `checked` reached the batch (15000 by default) whenever fewer videos were
available — quota spent on answers it already had; a run now skips what it has checked.
This commit is contained in:
2026-10-03 19:33:38 +02:00
parent 5806adda7e
commit 43e4929b45
4 changed files with 271 additions and 2 deletions
+5 -1
View File
@@ -22,7 +22,7 @@ backfill reserve as the other jobs so interactive syncs never starve.
from datetime import timedelta
from sqlalchemy import select
from sqlalchemy import or_, select
from sqlalchemy.orm import Session
from app import logs, progress, quota, sysconfig
@@ -190,6 +190,10 @@ def _revalidate_rolling(db: Session, yt: YouTubeClient) -> dict:
db.execute(
select(Video)
.where(Video.unavailable_since.is_(None))
# Not one this run already checked: with fewer available videos than the batch, the
# loop used to wrap around and re-fetch the same ones until `checked` reached the
# batch, spending quota on answers it already had (E32 S2).
.where(or_(Video.last_checked_at.is_(None), Video.last_checked_at < now))
# NULLS FIRST = never-checked first, then least-recently-checked; equivalent to the old
# `(last_checked_at IS NULL) DESC, last_checked_at ASC` but in the exact shape of the
# partial ix_videos_recheck_order (migration 0069, PERF-18) so it rides that index.
+8 -1
View File
@@ -22,6 +22,7 @@ from app import logs, progress, quota
from app.auth import has_read_scope
from app.models import Channel, Playlist, PlaylistItem, User, Video
from app.sync.runner import token_users
from app.sync.subscriptions import should_prune
from app.sync.videos import apply_video_details, parse_dt
from app.youtube.client import YouTubeClient, YouTubeError
@@ -342,7 +343,13 @@ def pull_user_playlists(db: Session, user: User) -> dict:
.all()
)
# Drop mirrors whose YouTube playlist no longer exists — locked first, as every item write.
gone = [pl for ytid, pl in existing.items() if ytid not in yt_ids]
# Never on an empty answer that contradicts the mirrors we hold: the same 200-with-no-items
# blip `should_prune` guards the subscriptions against would drop every mirror (E32 S2).
gone = (
[pl for ytid, pl in existing.items() if ytid not in yt_ids]
if should_prune(len(yt_ids), len(existing))
else []
)
if gone:
lock_playlists(db, *(pl.id for pl in gone))
for pl in gone: