fix(downloads,sync): bound what one user can make the server fetch, download and spend
SEC-12: the download box accepts an arbitrary http(s) URL and yt-dlp fetches it from the WORKER, so any approved user could aim the server at the compose network, the LAN or a cloud metadata endpoint — and read the result off job.error, where "connection refused" vs "HTTP Error 401" vs "timed out" is a port scanner. downloads/urlguard.py refuses a URL whose host resolves to a loopback/private/link-local/reserved address, at enqueue AND again in the worker (DNS can move in between, and the worker is where the socket opens). A network-class failure on a `url` job now reports one fixed sentence instead of the raw transport text; the error CODE is still classified from the original, so the localized explanation is unchanged. DNS rebinding is explicitly NOT closed — that needs the connection gated inside yt-dlp — and the module says so rather than implying more than it does. SEC-13: `noplaylist` only demotes a video-inside-a-playlist URL to that video, so a URL that is purely a playlist or channel still expanded into every entry — one enqueue, unbounded downloads. `playlist_items` pins it to one. Size was unbounded too: `check_enqueue` compares the CURRENT footprint against the cap, and a download's size is unknown until it lands, so one job could overshoot by any amount; the owner's remaining budget is now handed to yt-dlp as `max_filesize`, applied AFTER the adapter merge so no source adapter can raise a limit the quota layer set. A join's cut-list is capped at 100: an accurate join emits two filters per segment into one ffmpeg filter_complex. SEC-14: the YouTube budget is instance-wide, so one user holding down a manual sync spent the day for everyone. Per-user (not per-IP — the cost belongs to the account) 3-per-10-minutes limiters on the two bulk pulls, subscriptions and playlists, with separate budgets so one does not close the other. The review also listed /rss, /backfill and /enrich; E3 deleted them, and search stays unlimited on purpose — it is interactive, not a bulk pull. Gate: pytest 896 -> 924 (none skipped), vitest 671.
This commit is contained in:
@@ -109,3 +109,52 @@ def test_browser_transcode_flags_universal_playability():
|
||||
assert F("h264", "aac", False) == (True, False, False)
|
||||
# A failed probe / missing stream is never re-encoded.
|
||||
assert F(None, None, True) == (False, False, False)
|
||||
|
||||
|
||||
# --- SEC-13 · one enqueue is one bounded download -------------------------------------------
|
||||
|
||||
|
||||
def test_a_playlist_url_can_only_yield_one_item():
|
||||
# `noplaylist` alone only demotes a video-inside-a-playlist URL to that video; a URL that is
|
||||
# purely a playlist or channel has no such video, so without this it expands into every entry.
|
||||
opts = _opts({})
|
||||
assert opts["noplaylist"] is True
|
||||
assert opts["playlist_items"] == "1"
|
||||
|
||||
|
||||
def test_the_remaining_quota_becomes_a_hard_size_ceiling():
|
||||
opts = formats.build_ydl_opts({}, "/tmp/x", lambda d: None, max_bytes=5_000_000)
|
||||
assert opts["max_filesize"] == 5_000_000
|
||||
|
||||
|
||||
def test_no_budget_means_no_size_filter():
|
||||
# Unlimited users pass None; a non-positive remainder is the quota layer's refusal to make at
|
||||
# enqueue, not an impossible yt-dlp filter to express here.
|
||||
for budget in (None, 0, -1):
|
||||
assert "max_filesize" not in formats.build_ydl_opts(
|
||||
{}, "/tmp/x", lambda d: None, max_bytes=budget
|
||||
)
|
||||
|
||||
|
||||
def test_a_source_adapter_cannot_raise_the_size_ceiling():
|
||||
# The adapter merge happens after max_filesize is set, so this pins that no adapter option can
|
||||
# widen a storage cap — the mirror of the cookiefile rule, in the opposite direction.
|
||||
opts = formats.build_ydl_opts(
|
||||
{}, "/tmp/x", lambda d: None, extra={"max_filesize": 999_999_999}, max_bytes=5_000_000
|
||||
)
|
||||
assert opts["max_filesize"] == 5_000_000
|
||||
|
||||
|
||||
def test_a_join_cut_list_is_capped():
|
||||
# An accurate join emits two filters per segment into one ffmpeg filter_complex, so an unbounded
|
||||
# list is an unbounded command line. The surplus is dropped; the legitimate head survives.
|
||||
segs = [{"start_s": i, "end_s": i + 1} for i in range(500)]
|
||||
out = editmod.normalize_edit_spec({"segments": segs})
|
||||
assert len(out["segments"]) == editmod._MAX_SEGMENTS
|
||||
assert out["segments"][0] == {"start_s": 0.0, "end_s": 1.0}
|
||||
|
||||
|
||||
def test_a_normal_cut_list_is_untouched():
|
||||
segs = [{"start_s": 0, "end_s": 5}, {"start_s": 10, "end_s": 12}]
|
||||
out = editmod.normalize_edit_spec({"segments": segs})
|
||||
assert len(out["segments"]) == 2
|
||||
|
||||
Reference in New Issue
Block a user