fix(reader): the Takeout import holds up under abuse, races and a run it cannot see

- Uploads: one at a time per user and ten an hour, each refused only after its body arrived;
  the history read an entry at a time (32 MiB cap), a Watch later list capped at YouTube's 5000,
  a user's kept videos at 200k; any unreadable shape is a 422, not a 500.
- Quota: Watch later lookups stop at the backfill reserve and at a per-user daily share; what is
  not asked about waits, known videos still go in, and the shorts job retries the rest.
- Staging and settling share one per-user advisory lock (no deadlock); a run settles only the
  rows it read, so a row staged meanwhile is never called gone.
- A failed run's notice names a translated failure, never the upstream's text.
- The status says when an import is queued; the page watches it to its end even when left, then
  reads the feed, its counts and the playlists again.
- Settings: one live region, the zone kept (unavailable) while a run goes, focus handed back,
  a missed drop kept from the browser, an oversize file refused before it is sent.
- Scheduler and run lists show Takeout runs by the importing user.
This commit is contained in:
2026-10-23 20:48:23 +02:00
parent e0b02a0c14
commit 9f580fd8c8
29 changed files with 1235 additions and 213 deletions
+5 -3
View File
@@ -72,9 +72,11 @@ DEFAULT_MAX_BODY_BYTES = 1024 * 1024
# A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE DEFAULT. That is why
# `POST /api/diagnostics/events` is absent: its `capped_body` limit is 512 KiB, already below the
# default, so listing it could only widen what it accepts.
# A Google Takeout file (`reader.takeout`): a real watch history of 9500 views, the 18 months Google
# keeps by default, is 3.7 MB; four times that is still well inside one request.
TAKEOUT_MAX_BYTES = 16 * 1024 * 1024
# A Google Takeout file (`reader.takeout`). A real watch history measured 3.7 MB for 9500 views, ~390
# bytes a view, so 32 MiB holds ~86k views. YouTube keeps the history 36 months by default on an
# account made since mid-2020 and for good on an older one, so a heavy viewer's can pass 16 MiB. The
# file is read an entry at a time (`takeout._json_list`), so it costs its own size, not a tree of it.
TAKEOUT_MAX_BYTES = 32 * 1024 * 1024
ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = {
# Ingesting a local file. The body IS the file now — no multipart envelope to allow for —