fix(downloads,sync): bound what one user can make the server fetch, download and spend
SEC-12: the download box accepts an arbitrary http(s) URL and yt-dlp fetches it from the WORKER, so any approved user could aim the server at the compose network, the LAN or a cloud metadata endpoint — and read the result off job.error, where "connection refused" vs "HTTP Error 401" vs "timed out" is a port scanner. downloads/urlguard.py refuses a URL whose host resolves to a loopback/private/link-local/reserved address, at enqueue AND again in the worker (DNS can move in between, and the worker is where the socket opens). A network-class failure on a `url` job now reports one fixed sentence instead of the raw transport text; the error CODE is still classified from the original, so the localized explanation is unchanged. DNS rebinding is explicitly NOT closed — that needs the connection gated inside yt-dlp — and the module says so rather than implying more than it does. SEC-13: `noplaylist` only demotes a video-inside-a-playlist URL to that video, so a URL that is purely a playlist or channel still expanded into every entry — one enqueue, unbounded downloads. `playlist_items` pins it to one. Size was unbounded too: `check_enqueue` compares the CURRENT footprint against the cap, and a download's size is unknown until it lands, so one job could overshoot by any amount; the owner's remaining budget is now handed to yt-dlp as `max_filesize`, applied AFTER the adapter merge so no source adapter can raise a limit the quota layer set. A join's cut-list is capped at 100: an accurate join emits two filters per segment into one ffmpeg filter_complex. SEC-14: the YouTube budget is instance-wide, so one user holding down a manual sync spent the day for everyone. Per-user (not per-IP — the cost belongs to the account) 3-per-10-minutes limiters on the two bulk pulls, subscriptions and playlists, with separate budgets so one does not close the other. The review also listed /rss, /backfill and /enrich; E3 deleted them, and search stays unlimited on purpose — it is interactive, not a bulk pull. Gate: pytest 896 -> 924 (none skipped), vitest 671.
This commit is contained in:
@@ -20,10 +20,17 @@ from app.sync.playlists import (
|
||||
repull_playlist,
|
||||
sync_user_playlists,
|
||||
)
|
||||
from app.ratelimit import RateLimiter
|
||||
from app.youtube.client import YouTubeClient, YouTubeError
|
||||
|
||||
router = APIRouter(prefix="/api/playlists", tags=["playlists"])
|
||||
|
||||
# SEC-14, same reasoning as the sync module's limiter: a playlist pull lists every playlist the
|
||||
# caller owns and then every item in each, against the instance-wide YouTube budget. Its own
|
||||
# limiter rather than a shared one — spending the manual subscription-import allowance must not
|
||||
# also close the playlist button, and vice versa.
|
||||
_playlist_pull_limiter = RateLimiter(max_events=3, window_seconds=600)
|
||||
|
||||
|
||||
def _own_playlist(db: Session, user: User, playlist_id: int) -> Playlist:
|
||||
pl = db.get(Playlist, playlist_id)
|
||||
@@ -284,6 +291,8 @@ def sync_youtube(
|
||||
status_code=403,
|
||||
detail="Connect YouTube (read access) to sync your playlists.",
|
||||
)
|
||||
if not _playlist_pull_limiter.allow(str(user.id)):
|
||||
raise HTTPException(status_code=429, detail={"code": "sync_cooldown"})
|
||||
with quota.attribute(user.id, quota.QuotaAction.PLAYLISTS_PULL):
|
||||
return sync_user_playlists(db, user)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user