fix(reader): the Takeout import holds up under abuse, races and a run it cannot see

- Uploads: one at a time per user and ten an hour, each refused only after its body arrived;
  the history read an entry at a time (32 MiB cap), a Watch later list capped at YouTube's 5000,
  a user's kept videos at 200k; any unreadable shape is a 422, not a 500.
- Quota: Watch later lookups stop at the backfill reserve and at a per-user daily share; what is
  not asked about waits, known videos still go in, and the shorts job retries the rest.
- Staging and settling share one per-user advisory lock (no deadlock); a run settles only the
  rows it read, so a row staged meanwhile is never called gone.
- A failed run's notice names a translated failure, never the upstream's text.
- The status says when an import is queued; the page watches it to its end even when left, then
  reads the feed, its counts and the playlists again.
- Settings: one live region, the zone kept (unavailable) while a run goes, focus handed back,
  a missed drop kept from the browser, an oversize file refused before it is sent.
- Scheduler and run lists show Takeout runs by the importing user.
This commit is contained in:
2026-10-23 20:48:23 +02:00
parent e0b02a0c14
commit 9f580fd8c8
29 changed files with 1235 additions and 213 deletions
+6 -2
View File
@@ -3,7 +3,7 @@
Admin-only and read-only — rows leave the table through the retention job
(`job_run_retention_days`). Filtered on the server and paged by `_keyset`, like the server log and
the audit log beside it on the activity page. A row names what a reader needs without a second
request: the channel a user's ad-hoc run worked on, and how many log lines the run left (only
request: what a user's ad-hoc run worked on (the channel, the importing user), and how many log lines the run left (only
WARNING and up are stored, so most runs have none — the count says so before the click).
"""
@@ -116,8 +116,12 @@ def list_runs(
limit,
lambda r: r.started_at,
)
channels = {r.target for r in runs if r.job in scheduler.AD_HOC_JOBS and r.target}
channels = {r.target for r in runs if r.job == scheduler.CHANNEL_BACKFILL and r.target}
titles = catalog_queries.channel_titles(db, channels)
for target in {r.target for r in runs if r.job == scheduler.TAKEOUT_IMPORT and r.target}:
title = scheduler.ad_hoc_target_title(db, scheduler.TAKEOUT_IMPORT, target)
if title is not None:
titles[target] = title
lines: dict[int, int] = {}
if runs:
counted = db.execute(