fix(reader): the Takeout import holds up under abuse, races and a run it cannot see

- Uploads: one at a time per user and ten an hour, each refused only after its body arrived;
  the history read an entry at a time (32 MiB cap), a Watch later list capped at YouTube's 5000,
  a user's kept videos at 200k; any unreadable shape is a 422, not a 500.
- Quota: Watch later lookups stop at the backfill reserve and at a per-user daily share; what is
  not asked about waits, known videos still go in, and the shorts job retries the rest.
- Staging and settling share one per-user advisory lock (no deadlock); a run settles only the
  rows it read, so a row staged meanwhile is never called gone.
- A failed run's notice names a translated failure, never the upstream's text.
- The status says when an import is queued; the page watches it to its end even when left, then
  reads the feed, its counts and the playlists again.
- Settings: one live region, the zone kept (unavailable) while a run goes, focus handed back,
  a missed drop kept from the browser, an oversize file refused before it is sent.
- Scheduler and run lists show Takeout runs by the importing user.
This commit is contained in:
2026-10-23 20:48:23 +02:00
parent e0b02a0c14
commit 9f580fd8c8
29 changed files with 1235 additions and 213 deletions
+23
View File
@@ -181,6 +181,28 @@ def _seed_account(db, email: str, role: str, display_name: str) -> dict:
saved = svid.id
db.commit()
# 5. Videos the Takeout import marks watched: from the subscribed channels, and already cleared by
# the Shorts probe — one not yet probed would wait for it, and the import spec would count
# nothing. One per browser project: they share this account, and a video imported once is not
# marked again.
known = (
list(
db.scalars(
select(Video.id)
.where(
Video.channel_id.in_(channel_ids),
Video.shorts_probed.is_(True),
Video.is_short.is_(False),
Video.id.not_in(used | {saved} if saved else used),
)
.order_by(Video.published_at.desc())
.limit(5)
)
)
if channel_ids
else []
)
return {
"user_id": user.id,
"email": email,
@@ -188,6 +210,7 @@ def _seed_account(db, email: str, role: str, display_name: str) -> dict:
"in_progress": in_progress,
"watched": watched,
"saved": saved,
"takeout_known": known,
}