peter
fe38dfbcca
The deep review of the branch raised 47 findings, 18 of which survived adversarial verification. Grouped by what they actually were: **The size ceiling inverted at its boundary.** `remaining_bytes` clamped to 0 for a user already at their storage cap, and `build_ydl_opts` reads 0 as "no limit" — so the one user over quota was the only one downloading with no ceiling at all. `check_enqueue` could not catch it: it ran while the job was queued and the user was still under the cap. The arithmetic moves into `quota.remaining_bytes` (where every other max_bytes decision lives, and where it can be tested), and the worker now fails the job outright rather than proceeding unbounded — as a non-retryable `quota_exceeded`, since retrying cannot help until space is freed. **Silent failures in the new account panel.** All three actions went through `quiet: true` endpoints, which suppress the global error dialog on the promise that the caller reports inline — with no `onError`, that promise was broken and a refusal rendered as nothing. On the broken-SMTP instance this panel exists for, an admin would read silence as "the mail went out". The co-invalidation set and the email-off predicate move to `lib/adminUsers`, shared with the pending-access queue, so the two copies of these mutations cannot drift apart again — which is how half of this was lost in the first place. **A rollback recipe that destroyed data without working.** `alembic downgrade -1` steps back one migration, but a release routinely ships several (0.80.0 shipped four): it drops the newest one's columns and leaves the schema at a revision the older image also cannot resolve, i.e. the crash loop it was meant to escape, minus the data. The dump restore is now the recommended path, and the alembic route asks the target version for its own head revision instead of counting backwards. Also: redirects named in the urlguard docstring beside the DNS-rebinding residual they share a fix with (SEC-17); a malformed port refused as `bad_port` instead of escaping as a 500; the pull cooldown extended to the sibling route spending the same QuotaAction; restore stops the app before dropping its tables; the PowerShell cleanup can no longer decide the script's verdict on 5.1, nor report the wrong exit code; the documented one-liners hardened to match the scripts; the registry's retention window stated where pinning is advertised; the two banners made mutually exclusive so neither can shadow the other; `hover:bg-hover` (a utility this project never generated) replaced by the `selectable` recipe; the clipboard helper reused; and the sync cooldown's copy swept to the two callers that discarded it. Verified: gate green (vitest 678, pytest 949), and the panel, the row hover and the reset banner UAT'd on localdev.
97 lines
3.8 KiB
Python
97 lines
3.8 KiB
Python
"""SEC-12 · the download URL guard.
|
|
|
|
The Download Center hands a user-supplied URL to yt-dlp on the worker, so the server fetches
|
|
whatever the user typed. These pin that a URL resolving into our own network is refused, that the
|
|
refusal reasons stay distinct (an unresolvable host is not the same message as a blocked one), and
|
|
that a normal public URL still goes through.
|
|
|
|
DNS is stubbed rather than live: the point under test is the address JUDGEMENT, and a test that
|
|
resolves real names would be slow, offline-fragile, and would silently start passing if the guard
|
|
stopped resolving at all.
|
|
"""
|
|
|
|
import pytest
|
|
|
|
from app.downloads import urlguard
|
|
|
|
|
|
def _stub_dns(monkeypatch, addrs):
|
|
"""Make every lookup answer with `addrs` (a list of textual IPs)."""
|
|
|
|
def fake_getaddrinfo(host, port, *a, **kw):
|
|
return [(None, None, None, "", (addr, port or 0)) for addr in addrs]
|
|
|
|
monkeypatch.setattr(urlguard.socket, "getaddrinfo", fake_getaddrinfo)
|
|
|
|
|
|
def _blocked_reason(url):
|
|
with pytest.raises(urlguard.BlockedUrl) as e:
|
|
urlguard.check_url(url)
|
|
return e.value.reason
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"addr",
|
|
[
|
|
"127.0.0.1", # loopback
|
|
"10.1.2.3", # RFC1918
|
|
"192.168.0.5", # RFC1918
|
|
"172.16.9.9", # RFC1918
|
|
"169.254.169.254", # link-local — the cloud metadata endpoint
|
|
"0.0.0.0", # unspecified
|
|
"::1", # IPv6 loopback
|
|
"fe80::1", # IPv6 link-local
|
|
"fc00::1", # IPv6 unique-local
|
|
"::ffff:127.0.0.1", # loopback wearing an IPv6 hat
|
|
],
|
|
)
|
|
def test_addresses_on_our_own_network_are_refused(monkeypatch, addr):
|
|
_stub_dns(monkeypatch, [addr])
|
|
assert _blocked_reason("http://whatever.example/video") == urlguard.REASON_PRIVATE
|
|
|
|
|
|
def test_one_bad_answer_in_a_round_robin_blocks_the_whole_host(monkeypatch):
|
|
# A host answering with both a public and an internal address must not become a coin flip:
|
|
# yt-dlp would pick for itself, so the guard refuses if ANY answer is forbidden.
|
|
_stub_dns(monkeypatch, ["93.184.216.34", "127.0.0.1"])
|
|
assert _blocked_reason("https://mixed.example/v") == urlguard.REASON_PRIVATE
|
|
|
|
|
|
def test_a_public_address_passes(monkeypatch):
|
|
_stub_dns(monkeypatch, ["93.184.216.34"])
|
|
urlguard.check_url("https://example.com/watch?v=abc") # does not raise
|
|
|
|
|
|
def test_unresolvable_is_its_own_reason(monkeypatch):
|
|
import socket as _socket
|
|
|
|
def boom(*a, **kw):
|
|
raise _socket.gaierror("nope")
|
|
|
|
monkeypatch.setattr(urlguard.socket, "getaddrinfo", boom)
|
|
# Distinct from PRIVATE on purpose: "we couldn't look that up" and "we refuse to fetch that"
|
|
# are different things to tell a user, and the frontend renders them differently.
|
|
assert _blocked_reason("https://nx.example/v") == urlguard.REASON_UNRESOLVABLE
|
|
|
|
|
|
@pytest.mark.parametrize("url", ["file:///etc/passwd", "ftp://host/x", "gopher://host/1"])
|
|
def test_non_http_schemes_are_refused_before_any_lookup(monkeypatch, url):
|
|
def boom(*a, **kw): # a scheme rejection must not have resolved anything first
|
|
raise AssertionError("DNS was consulted for a non-http scheme")
|
|
|
|
monkeypatch.setattr(urlguard.socket, "getaddrinfo", boom)
|
|
assert _blocked_reason(url) == urlguard.REASON_SCHEME
|
|
|
|
|
|
def test_a_url_without_a_host_is_refused():
|
|
assert _blocked_reason("http:///just/a/path") == urlguard.REASON_NO_HOST
|
|
|
|
|
|
def test_a_malformed_port_is_a_refusal_not_a_crash(monkeypatch):
|
|
# `urlparse(...).port` raises ValueError on ACCESS, not at parse time, and `.hostname` returns
|
|
# first — so an unguarded read turns a user typo into a 500 that escapes the caller's
|
|
# `except BlockedUrl` and loses the localized detail this module exists to produce.
|
|
_stub_dns(monkeypatch, ["93.184.216.34"])
|
|
assert _blocked_reason("http://example.com:99999/v.mp4") == urlguard.REASON_BAD_PORT
|
|
assert _blocked_reason("http://example.com:abc/v.mp4") == urlguard.REASON_BAD_PORT
|