fix(reader): the watched-channels list keeps its channels, its shares and its retries

- A channel someone follows: the videos its exploration brings in (the list's
  dialog, the channel page's explore) come in as the catalog's own, not
  flagged via_explore, which subscribing would never clear and which kept
  their watched marks waiting for good.
- The explore cleanup keeps the row of a channel a Takeout history shows
  watched (it still reclaims the videos), so a looked-at channel no longer
  drops off the list.
- The Shorts job takes up only users whose list still has lengths to ask or
  channels to name, by each user's own follows, blocks and dismissals; a
  naming that failed is tried again, and the list says it is not complete.
- The length lookups have a daily share of their own: a large history no
  longer leaves the export's Watch later file waiting a day.
- Only a channel of the user's own history can be put out of the list.
- The dialog's videos have a query root of their own: the list's
  invalidations no longer re-ask (and re-explore) under an open dialog.
This commit is contained in:
2026-10-24 01:10:29 +02:00
parent b5f9cf14db
commit 33dd9ff054
9 changed files with 233 additions and 47 deletions
+9 -5
View File
@@ -322,6 +322,10 @@ def stage(db: Session, user: User, parsed: Parsed) -> None:
# How many unknown Watch later videos one user may have looked up on the shared API budget in a day:
# two full Watch later lists (two units a hundred ids). The rest wait for the next day's runs.
_lookups = RateLimiter(max_events=2 * WATCH_LATER_MAX, window_seconds=24 * 3600)
# How many watched videos' lengths one user may ask about in a day for the "watched but not followed"
# list (a unit a fifty). A share of its own: a large history asks for thousands, and on the Watch
# later share it would leave the export's Watch later file waiting a day behind it.
_lengths = RateLimiter(max_events=2 * WATCH_LATER_MAX, window_seconds=24 * 3600)
def _put_in_watch_later(db: Session, user: User) -> dict:
@@ -356,8 +360,8 @@ def _put_in_watch_later(db: Session, user: User) -> dict:
def _ask_lengths(db: Session, user: User) -> dict:
"""Ask YouTube how long the user's watched videos are that the "watched but not followed" list
still needs, and keep the answer — on the same terms and the same daily share as the Watch
later lookups. What is not asked yet (no YouTube-connected account, the reserve reached, the
still needs, and keep the answer — on the same terms as the Watch later lookups, on a daily
share of its own. What is not asked yet (no YouTube-connected account, the reserve reached, the
share spent) waits for a later run, and the list says it is not complete. A YouTube failure is
logged and left for a later run too: it is no reason to fail the import it follows. The caller
commits."""
@@ -365,7 +369,7 @@ def _ask_lengths(db: Session, user: User) -> dict:
fetcher = get_service_user(db) if ask else None
if fetcher is None or quota.backfill_reserve_reached():
return {}
ask = ask[: int(_lookups.room(str(user.id)))]
ask = ask[: int(_lengths.room(str(user.id)))]
if not ask:
return {}
try:
@@ -379,7 +383,7 @@ def _ask_lengths(db: Session, user: User) -> dict:
extra=logs.event("takeout.lengths.failed", user_id=user.id, reason=exc.reason),
)
return {}
_lookups.allow(str(user.id), cost=len(ask))
_lengths.allow(str(user.id), cost=len(ask))
lengths = {
it["id"]: parse_iso8601_duration(it.get("contentDetails", {}).get("duration"))
for it in items
@@ -453,7 +457,7 @@ def apply_waiting(db: Session) -> dict:
marked = added = asked = failed = 0
ready = set(queries.takeout_users_ready(db))
later = set(queries.takeout_users_watch_later_waiting(db))
lengths = set(queries.takeout_users_with_lengths_to_ask(db))
lengths = set(queries.takeout_users_with_list_work(db))
for user_id in sorted(ready | later | lengths):
user = db.get(User, user_id)
if user is None: