fix(reader): the Takeout import keeps a later upload, an un-mark and its memory bound

- An ad-hoc request made while its target's run is under way is the run after it: the scheduler
  dropped a second fire of a running id, so a second Takeout file was never imported and its key
  stayed queued until a restart.
- Un-marking or resetting a video is remembered (video_unmarks): an import of an older YouTube view
  no longer marks it watched again; un-marks take the user's Takeout lock.
- One user's failure in the Shorts job's Takeout pass is theirs: the job keeps its own result, the
  others are settled, and a failed lookup spends none of the daily share.
- The upload reads the user's id before giving the read connection back.
- A watched row waits on a discovery video (search or explore) until the catalog keeps it, so a
  mark never pins it against the cleanup jobs.
- The history is decoded a window at a time, a Watch later file over 2 MiB is refused unread, and
  two uploads at once across all users (takeout_server_busy).
- A Watch later list goes in as one presence read and one insert.
This commit is contained in:
2026-10-23 22:07:11 +02:00
parent 9f580fd8c8
commit 9c5f81fd94
17 changed files with 667 additions and 70 deletions
@@ -0,0 +1,44 @@
"""When a user last took a video back to "new"
Revision ID: 0114_video_unmarks
Revises: 0113_takeout_items
Create Date: 2026-10-23
Un-marking or resetting a video leaves no `video_states` row, so this is what tells an import of the
user's YouTube history a video they un-marked here from one never touched.
"""
import sqlalchemy as sa
from alembic import op
revision: str = "0114_video_unmarks"
down_revision: str | None = "0113_takeout_items"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.create_table(
"video_unmarks",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column(
"user_id",
sa.Integer(),
sa.ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"video_id",
sa.String(16),
sa.ForeignKey("videos.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("at", sa.DateTime(timezone=True), nullable=False),
sa.UniqueConstraint("user_id", "video_id", name="uq_video_unmark"),
)
op.create_index("ix_video_unmarks_video_id", "video_unmarks", ["video_id"])
def downgrade() -> None:
op.drop_index("ix_video_unmarks_video_id", table_name="video_unmarks")
op.drop_table("video_unmarks")