refactor(backfill): centralize the reserve-yield guard and drop the per-page connection hold

Addresses the code-review findings on the immediate full-history change:

- Extract quota.backfill_reserve_reached(): the remaining/reserve compare was
  open-coded at seven sites (recent/deep/immediate backfill, explore, rolling
  revalidation, and the new per-page guard). It reads in its OWN short-lived
  session (like record_usage/log_action), so a per-page guard no longer holds
  the caller's transaction open across the YouTube fetch that follows — the
  400-page immediate run no longer pins a pooled connection idle-in-transaction
  through each network round-trip. Migrated all call sites; dropped the now-dead
  sysconfig imports in channels.py and maintenance.py.
- Simplify the deep-backfill loop: channel.backfill_done = not page_token (done
  is False on entry) plus one break, replacing the doubled `if not page_token`.
- Tests patch the backfill_reserve_reached seam directly instead of mocking
  remaining_today internals, removing the fragile dead-value budget iterator.

Full backend suite green; ruff clean.
This commit is contained in:
2026-08-16 02:41:49 +02:00
parent e3ee1baa94
commit 48285a715b
6 changed files with 30 additions and 18 deletions
+3 -3
View File
@@ -183,7 +183,7 @@ def run_recent_backfill(
videos_added = 0
with YouTubeClient(db, user) as yt:
for channel in channels:
if quota.remaining_today(db) <= sysconfig.get_int(db, "backfill_quota_reserve"):
if quota.backfill_reserve_reached():
break
try:
videos_added += backfill_channel_recent(db, yt, channel)
@@ -229,7 +229,7 @@ def run_deep_backfill(db: Session, max_channels: int = 10, max_pages: int = 5) -
videos_added = 0
with YouTubeClient(db, user) as yt:
for channel in channels:
if quota.remaining_today(db) <= sysconfig.get_int(db, "backfill_quota_reserve"):
if quota.backfill_reserve_reached():
break
# Claim the channel (or skip it if an interactive backfill already owns it) so the two
# never advance the same cursor concurrently — symmetric with backfill_channel_now.
@@ -294,7 +294,7 @@ def backfill_channel_now(channel_id: str, user_id: int, max_pages: int = 400) ->
channel = db.get(Channel, channel_id)
if channel is None or channel.backfill_done or not channel.uploads_playlist_id:
return
if quota.remaining_today(db) <= sysconfig.get_int(db, "backfill_quota_reserve"):
if quota.backfill_reserve_reached():
return
try:
# Attribute the spend to the requesting user under the full-backfill action, so it