feat(downloads): a per-site failure catalog — Facebook's walls say why and what to do
A failed Facebook download read "Download failed": yt-dlp words the login wall (18+ reel, closed group) as "Cannot parse data", which no category matched. Measured on dev with the owner's cookie before writing a rule — the same text is the login wall without a cookie and a refused cookie with one. - errors.diagnose(msg, url, cookie_attached) tries a host-scoped catalog first (all-of markers, cookie state, link fragment) and falls back to the category. First entries: facebook.login_wall, facebook.cookie_rejected, facebook.group_post_video. The module doc is the recipe for the next entry. - download_jobs.error_detail (0084) + backfill of the stored Facebook parse failures that were unknown. - The pipeline's failure branch passes the link and whether the cookie went with it; resume and redownload clear code and detail (redownload never cleared error_code). - A job meeting an already-failed asset got the text but no code, so it read "Download failed" where the first job said why. It is diagnosed again, with the cookie state from credentials.is_authed_asset (now shared with the cookie gate); the redacted network text classifies back to network. - FE: failCopyKeys gives the row and the failure window one lookup — catalog copy, else the category. Copy in EN and HU; a contract test pins every backend entry to both locales.
This commit is contained in:
+23
-2
@@ -32,7 +32,8 @@ from app import sysconfig
|
||||
from app.config import settings
|
||||
from app.db import SessionLocal
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import quota, service
|
||||
from app.downloads import credentials, formats, quota, service
|
||||
from app.downloads import errors as dlerrors
|
||||
from app.downloads.jobdb import (
|
||||
_edit_staging_dir,
|
||||
_requeue_if_running,
|
||||
@@ -399,6 +400,19 @@ def _process_job(job_id: int) -> None:
|
||||
asset_id = asset.id if asset else None
|
||||
asset_status = asset.status if asset else None
|
||||
asset_error = asset.error if asset else None
|
||||
if asset_status == "error":
|
||||
# A job that meets an already-failed asset gets the SAME explanation the job that failed
|
||||
# it got — not a bare "Download failed". Only the text is on the asset, so it is
|
||||
# diagnosed again, for this link, with the cookie state of that asset: a user's private
|
||||
# (cookie-salted) asset failed with their cookie attached, the shared one without.
|
||||
url = service.source_url(job.source_kind, job.source_ref)
|
||||
diagnosis = dlerrors.diagnose(
|
||||
asset_error,
|
||||
url=url,
|
||||
cookie_attached=credentials.is_authed_asset(
|
||||
asset.format_sig, formats.normalize(job.profile_snapshot), job.user_id, url
|
||||
),
|
||||
)
|
||||
|
||||
if asset is None:
|
||||
_set_job(job_id, status="error", error="No media asset", phase=None)
|
||||
@@ -408,7 +422,14 @@ def _process_job(job_id: int) -> None:
|
||||
_finish_from_cache(job_id, asset_id)
|
||||
return
|
||||
if asset_status == "error":
|
||||
_set_job(job_id, status="error", error=asset_error or "Download failed", phase=None)
|
||||
_set_job(
|
||||
job_id,
|
||||
status="error",
|
||||
error=asset_error or "Download failed",
|
||||
error_code=diagnosis.code,
|
||||
error_detail=diagnosis.detail,
|
||||
phase=None,
|
||||
)
|
||||
return
|
||||
if asset_status == "downloading":
|
||||
# Another worker won this asset between our claim and here (a narrow race — the claim SQL
|
||||
|
||||
Reference in New Issue
Block a user