fix(downloads): the egress filter keeps what yt-dlp fetches itself, and shows no phase
f4m, ism, a bare m3u8 (native under hls_prefer_native) and the DASH generator fetch through the proxy, but were dropped as if ffmpeg dialled them. The filter also took the job's postprocessor hook, twice, and painted every guarded job "processing" before a byte arrived. Tests now hold an allowed tunnel, the worker's own gate, the single-URL branch, and that the pipeline and the backfill turn the gate on while YouTube does not.
This commit is contained in:
@@ -233,7 +233,7 @@ def open_ydl(opts: dict) -> yt_dlp.YoutubeDL:
|
||||
# assembles, the same every caller handed yt-dlp before.
|
||||
ydl = yt_dlp.YoutubeDL(opts) # type: ignore[arg-type]
|
||||
if opts.get(GUARDED):
|
||||
ydl.add_post_processor(HttpOnlyFormats(ydl), when="pre_process")
|
||||
ydl.add_post_processor(HttpOnlyFormats(), when="pre_process")
|
||||
return ydl
|
||||
|
||||
|
||||
@@ -250,9 +250,23 @@ def proxy_url() -> str:
|
||||
return _gate.url
|
||||
|
||||
|
||||
# The protocols yt-dlp fetches over http(s) — the only ones the gate sees. Everything else (rtmp,
|
||||
# rtsp, mms, f4m's own fetcher, ws) would be dialled by ffmpeg or another tool directly.
|
||||
_HTTP_PROTOCOLS = frozenset({"http", "https", "m3u8_native", "http_dash_segments"})
|
||||
# The protocols fetched over http(s) through the proxy option — the only ones the gate sees: yt-dlp's
|
||||
# own fetchers (http, f4m, ism, the DASH segments and their generator, native HLS — which a bare m3u8
|
||||
# also gets, as a guarded download sets hls_prefer_native) and ffmpeg's http input (a live stream or
|
||||
# a clip), which takes the proxy too. Everything else (rtmp, rtsp, mms, ws) ffmpeg or another tool
|
||||
# would dial directly.
|
||||
_HTTP_PROTOCOLS = frozenset(
|
||||
{
|
||||
"http",
|
||||
"https",
|
||||
"f4m",
|
||||
"ism",
|
||||
"m3u8",
|
||||
"m3u8_native",
|
||||
"http_dash_segments",
|
||||
"http_dash_segments_generator",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
class HttpOnlyFormats(PostProcessor):
|
||||
@@ -260,6 +274,11 @@ class HttpOnlyFormats(PostProcessor):
|
||||
yt-dlp re-reads its formats after ("The pre-processors may have modified the formats"), so the
|
||||
selector never sees them. With nothing left, yt-dlp reports no formats — a refusal, not a leak."""
|
||||
|
||||
def set_downloader(self, downloader):
|
||||
# Not the job's postprocessor hooks: they would report this filter as a "processing" step
|
||||
# before a byte is fetched (yt-dlp attaches them here, once per call).
|
||||
self._downloader = downloader
|
||||
|
||||
def run(self, information):
|
||||
# yt-dlp's stub types the info dict as a TypedDict with optional keys; it is a dict here.
|
||||
info = cast("dict[str, Any]", information)
|
||||
|
||||
Reference in New Issue
Block a user