fix: a YouTube video behind the bot wall is downloadable with cookies

A YouTube link can fail with "Sign in to confirm you're not a bot" while every
other YouTube download works — YouTube gates INDIVIDUAL videos behind a
signed-in session. Measured on prod job 273: all twelve of yt-dlp's player
clients fail on that one video while its neighbours download fine, the bgutil
PO-token sidecar is live and answering, and the newest yt-dlp fails identically.
So the download is not broken by our stack or a stale yt-dlp, and no retry,
client switch or upgrade gets past it. A cookie is the only way in.

The failure copy already sends the user to Settings -> Account -> Download
authentication, but YouTube was not in the cookie host registry, so that
dropdown could not help them: the app asked for something it did not offer.

credentials: YouTube joins SUPPORTED_HOSTS (youtube.com, youtu.be,
youtube-nocookie.com). The Settings dropdown, the SEC-12 host scope and the
private-asset cache salt all derive from the row, so that is the whole wiring.

adapters: a signed-in YouTube download runs only cookie-capable player clients.
Measured against yt-dlp's own INNERTUBE_CLIENTS table: with account cookies
attached yt-dlp REMOVES every client whose SUPPORTS_COOKIES is False, and a set
it empties that way raises "No player clients have been requested" — which is
not a per-client flake it retries past. Our primary set is android_vr, one of
those, so without this the cookie would have failed the download on attempt 1
and never reached the POT-backed web fallback it exists for. A test pins the
client list against the installed yt-dlp so a version bump turns the gate red
instead of silently un-authing every YouTube download, and the empty-set message
joins the retriable markers as the second line of defence.

errors: catalog entries for the wall with and without a cookie, so a user who
has already added cookies is told they were rejected (YouTube rotates exported
ones) rather than to add some. The marker is quote-agnostic: YouTube writes the
apostrophe as U+2019, so "you're not a bot" with a straight quote never matches
the real text. EN + HU copy.
This commit is contained in:
2026-09-16 02:03:00 +02:00
parent 595490cb98
commit 864f673ef5
9 changed files with 182 additions and 6 deletions
+3
View File
@@ -155,6 +155,9 @@ class Settings(BaseSettings):
# per-client responses are flaky, and android_vr can get bot-flagged under heavy load) the
# worker retries with the FALLBACK set — the POT-capable web clients (web needs the token +
# Deno) plus android for audio. Both admin-tunable when YouTube shifts what works.
# A SIGNED-IN download (the user added YouTube cookies) keeps only the cookie-capable clients
# of these two sets — yt-dlp drops the rest, so android_vr / android / ios never run there.
# See adapters.YouTubeAdapter._COOKIE_CAPABLE.
download_player_clients: str = "android_vr"
download_player_clients_fallback: str = "web_safari,web,android"
+32 -3
View File
@@ -93,12 +93,41 @@ class YouTubeAdapter(SourceAdapter):
"Requested format is not available",
"Only images are available",
"Sign in to confirm",
# A set yt-dlp emptied because none of its clients take cookies. `client_sets` filters those
# out below, so this is the safety net for a yt-dlp that renames a client or an admin who
# configures both knobs with cookie-incapable clients only.
"No player clients have been requested",
)
# Measured against yt-dlp's own INNERTUBE_CLIENTS table (2026-09-16, on 2026.07.04 and
# 2026.08.19): with account cookies attached yt-dlp REMOVES every client whose SUPPORTS_COOKIES
# is False, and a set left empty raises "No player clients have been requested". Our primary set
# is android_vr — one of those — so a signed-in download would fail on the first attempt and
# never reach the POT-backed web fallback that the cookie exists for. Pinned by test_adapters
# against the installed yt-dlp, so a version bump that moves the table turns the gate red
# instead of silently un-authing every YouTube download.
_COOKIE_CAPABLE = frozenset(
{
"mweb",
"tv",
"tv_downgraded",
"web",
"web_creator",
"web_embedded",
"web_music",
"web_safari",
}
)
# Where a signed-in download lands when the admin's knobs hold no cookie-capable client at all.
# Both clients in ONE set: yt-dlp queries each and merges their formats, which is the cover for
# the per-client flakiness that the two-tier ladder gives a signed-out download.
_AUTHED_DEFAULT = ("web_safari", "web")
def client_sets(self, authed: bool = False) -> list:
return [
c for c in (settings.player_client_list, settings.player_client_fallback_list) if c
] or [None]
sets = [c for c in (settings.player_client_list, settings.player_client_fallback_list) if c]
if authed:
kept = [[c for c in s if c in self._COOKIE_CAPABLE] for s in sets]
return [s for s in kept if s] or [list(self._AUTHED_DEFAULT)]
return sets or [None]
def ydl_opts_extra(self, clients) -> dict:
# == --extractor-args "youtube:player_client=…;youtubepot-bgutilhttp:base_url=…" (list values).
+5
View File
@@ -55,6 +55,11 @@ class HostSpec:
# all derive from this, so there is nothing else to wire up.
SUPPORTED_HOSTS: tuple[HostSpec, ...] = (
HostSpec("facebook", "Facebook", ("facebook.com", "fb.watch")),
# YouTube gates INDIVIDUAL videos behind "Sign in to confirm you're not a bot" (measured on prod
# job 273: every player client fails on that one video while its neighbours download fine, and
# the newest yt-dlp fails the same way — a signed-in session is the only way through). Without a
# row here the `login_required` copy sent the user to a dropdown that could not help them.
HostSpec("youtube", "YouTube", ("youtube.com", "youtu.be", "youtube-nocookie.com")),
HostSpec("instagram", "Instagram", ("instagram.com",)),
HostSpec("twitter", "X (Twitter)", ("twitter.com", "x.com")),
HostSpec("tiktok", "TikTok", ("tiktok.com",)),
+24
View File
@@ -212,6 +212,30 @@ _CATALOG: tuple[_Entry, ...] = (
cookie=False,
link=facebook_group_post_id,
),
# YouTube gates INDIVIDUAL videos behind "Sign in to confirm you're not a bot" (measured on prod
# job 273, a festival set: all 12 of yt-dlp's player clients fail on that one video while its
# neighbours download fine, the POT sidecar is live and answering, and the newest yt-dlp fails
# identically — so this is the video's own signed-out gate, not a broken stack or a stale yt-dlp).
# The marker is quote-agnostic on purpose: YouTube writes the apostrophe as U+2019, so
# "you're not a bot" with a straight quote never matches the real text.
_Entry(
host="youtube",
markers=("[youtube]", "not a bot"),
code="login_required",
detail="youtube.bot_wall",
cookie=False,
),
# The same wall WITH a cookie. Not yet reproduced with a real YouTube cookie (this user's first
# one is being added now) — but "add cookies" is provably the wrong advice for someone who has,
# and yt-dlp names the likeliest cause itself: "The provided YouTube account cookies are no
# longer valid. They have likely been rotated in the browser as a security measure."
_Entry(
host="youtube",
markers=("[youtube]", "not a bot"),
code="login_required",
detail="youtube.cookie_rejected",
cookie=True,
),
)