peter
18c11ca437
A failed Facebook download read "Download failed": yt-dlp words the login wall (18+ reel, closed group) as "Cannot parse data", which no category matched. Measured on dev with the owner's cookie before writing a rule — the same text is the login wall without a cookie and a refused cookie with one. - errors.diagnose(msg, url, cookie_attached) tries a host-scoped catalog first (all-of markers, cookie state, link fragment) and falls back to the category. First entries: facebook.login_wall, facebook.cookie_rejected, facebook.group_post_video. The module doc is the recipe for the next entry. - download_jobs.error_detail (0084) + backfill of the stored Facebook parse failures that were unknown. - The pipeline's failure branch passes the link and whether the cookie went with it; resume and redownload clear code and detail (redownload never cleared error_code). - A job meeting an already-failed asset got the text but no code, so it read "Download failed" where the first job said why. It is diagnosed again, with the cookie state from credentials.is_authed_asset (now shared with the cookie gate); the redacted network text classifies back to network. - FE: failCopyKeys gives the row and the failure window one lookup — catalog copy, else the category. Copy in EN and HU; a contract test pins every backend entry to both locales.
42 lines
1.8 KiB
Python
42 lines
1.8 KiB
Python
"""add download_jobs.error_detail — the domain-specific entry of the failure catalog (E11.5 S1)
|
|
|
|
Revision ID: 0084_dl_job_error_detail
|
|
Revises: 0083_comfort_pins
|
|
Create Date: 2026-09-13
|
|
|
|
`error_code` is the generic category (it decides whether a Retry is offered). `error_detail` names
|
|
the host-specific catalog entry `downloads.errors.classify` matched on top of it — `facebook.login_wall`
|
|
and its siblings — so the UI can say what went wrong ON THAT SITE and what to do about it, instead of
|
|
the category's generic sentence. Nullable: most failures have no host-specific entry.
|
|
|
|
Backfill: a job's code is written once, at failure time, so a new rule never reaches the rows that
|
|
failed before it. The one shape measured on prod (2026-09-13, jobs 9/66/75/145/156/157/182/256/257) is
|
|
yt-dlp's `[facebook] <id>: Cannot parse data`, stored as `unknown`. Without a cookie that is Facebook's
|
|
login wall. Whether a cookie was attached at the time is not recorded anywhere, so every old row gets
|
|
the no-cookie entry — on prod no account had a Facebook cookie, so that is what they all were.
|
|
"""
|
|
|
|
import sqlalchemy as sa
|
|
from alembic import op
|
|
|
|
revision: str = "0084_dl_job_error_detail"
|
|
down_revision: str | None = "0083_comfort_pins"
|
|
branch_labels: str | None = None
|
|
depends_on: str | None = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.add_column("download_jobs", sa.Column("error_detail", sa.String(length=48), nullable=True))
|
|
op.execute(
|
|
sa.text(
|
|
"UPDATE download_jobs SET error_code = 'login_required', "
|
|
"error_detail = 'facebook.login_wall' "
|
|
"WHERE (error_code IS NULL OR error_code = 'unknown') "
|
|
"AND error ILIKE '%[facebook]%cannot parse data%'"
|
|
)
|
|
)
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.drop_column("download_jobs", "error_detail")
|