fix(downloads,sync): bound what one user can make the server fetch, download and spend

SEC-12: the download box accepts an arbitrary http(s) URL and yt-dlp fetches
it from the WORKER, so any approved user could aim the server at the compose
network, the LAN or a cloud metadata endpoint — and read the result off
job.error, where "connection refused" vs "HTTP Error 401" vs "timed out" is a
port scanner. downloads/urlguard.py refuses a URL whose host resolves to a
loopback/private/link-local/reserved address, at enqueue AND again in the
worker (DNS can move in between, and the worker is where the socket opens).
A network-class failure on a `url` job now reports one fixed sentence instead
of the raw transport text; the error CODE is still classified from the
original, so the localized explanation is unchanged. DNS rebinding is
explicitly NOT closed — that needs the connection gated inside yt-dlp — and
the module says so rather than implying more than it does.

SEC-13: `noplaylist` only demotes a video-inside-a-playlist URL to that video,
so a URL that is purely a playlist or channel still expanded into every
entry — one enqueue, unbounded downloads. `playlist_items` pins it to one.
Size was unbounded too: `check_enqueue` compares the CURRENT footprint against
the cap, and a download's size is unknown until it lands, so one job could
overshoot by any amount; the owner's remaining budget is now handed to yt-dlp
as `max_filesize`, applied AFTER the adapter merge so no source adapter can
raise a limit the quota layer set. A join's cut-list is capped at 100: an
accurate join emits two filters per segment into one ffmpeg filter_complex.

SEC-14: the YouTube budget is instance-wide, so one user holding down a
manual sync spent the day for everyone. Per-user (not per-IP — the cost
belongs to the account) 3-per-10-minutes limiters on the two bulk pulls,
subscriptions and playlists, with separate budgets so one does not close the
other. The review also listed /rss, /backfill and /enrich; E3 deleted them,
and search stays unlimited on purpose — it is interactive, not a bulk pull.

Gate: pytest 896 -> 924 (none skipped), vitest 671.
This commit is contained in:
2026-09-05 01:03:24 +02:00
parent 688efb9ed6
commit 0cc450d1c9
15 changed files with 429 additions and 5 deletions
+49
View File
@@ -109,3 +109,52 @@ def test_browser_transcode_flags_universal_playability():
assert F("h264", "aac", False) == (True, False, False)
# A failed probe / missing stream is never re-encoded.
assert F(None, None, True) == (False, False, False)
# --- SEC-13 · one enqueue is one bounded download -------------------------------------------
def test_a_playlist_url_can_only_yield_one_item():
# `noplaylist` alone only demotes a video-inside-a-playlist URL to that video; a URL that is
# purely a playlist or channel has no such video, so without this it expands into every entry.
opts = _opts({})
assert opts["noplaylist"] is True
assert opts["playlist_items"] == "1"
def test_the_remaining_quota_becomes_a_hard_size_ceiling():
opts = formats.build_ydl_opts({}, "/tmp/x", lambda d: None, max_bytes=5_000_000)
assert opts["max_filesize"] == 5_000_000
def test_no_budget_means_no_size_filter():
# Unlimited users pass None; a non-positive remainder is the quota layer's refusal to make at
# enqueue, not an impossible yt-dlp filter to express here.
for budget in (None, 0, -1):
assert "max_filesize" not in formats.build_ydl_opts(
{}, "/tmp/x", lambda d: None, max_bytes=budget
)
def test_a_source_adapter_cannot_raise_the_size_ceiling():
# The adapter merge happens after max_filesize is set, so this pins that no adapter option can
# widen a storage cap — the mirror of the cookiefile rule, in the opposite direction.
opts = formats.build_ydl_opts(
{}, "/tmp/x", lambda d: None, extra={"max_filesize": 999_999_999}, max_bytes=5_000_000
)
assert opts["max_filesize"] == 5_000_000
def test_a_join_cut_list_is_capped():
# An accurate join emits two filters per segment into one ffmpeg filter_complex, so an unbounded
# list is an unbounded command line. The surplus is dropped; the legitimate head survives.
segs = [{"start_s": i, "end_s": i + 1} for i in range(500)]
out = editmod.normalize_edit_spec({"segments": segs})
assert len(out["segments"]) == editmod._MAX_SEGMENTS
assert out["segments"][0] == {"start_s": 0.0, "end_s": 1.0}
def test_a_normal_cut_list_is_untouched():
segs = [{"start_s": 0, "end_s": 5}, {"start_s": 10, "end_s": 12}]
out = editmod.normalize_edit_spec({"segments": segs})
assert len(out["segments"]) == 2