fix(uploads): read the body after authenticating, not before it

`POST /api/downloads/upload` declared `file: UploadFile = File(...)`, and
FastAPI parses and spools a declared body before the first dependency runs --
`require_human` included. An unauthenticated caller could therefore make the
process write the whole per-file allowance (500 MiB by default) to temp, once
per request. SEC-20's ceiling bounded that number but could not stop the
writing, which is why E27 parked it rather than closing it.

The route now takes no body parameter. `spool_upload` names `require_human`
first and only then streams `request.stream()` to the staging temp file, so a
stranger is refused before a byte moves -- the shape the screenshot route has
used since E16. Filename and display name travel as percent-encoded headers
(latin-1 on the wire, so a Hungarian filename needs the encoding) instead of
multipart parts.

`capped_stream_to` joins `capped_body` in routes/_common.py: the same counting
rule, written to a file descriptor rather than accumulated, because holding
500 MiB would trade a disk problem for a memory one. Its refusal is the
caller's, so the upload route keeps answering 422 `upload_too_large` with the
admin knob's own number.

With no envelope to allow for, the edge is now exactly that knob, and
MULTIPART_ENVELOPE_ALLOWANCE is gone.

The invariant stops being prose: test_body_limit.py asks FastAPI itself
(`route.dependant.body_params`) and fails the gate if any route with a raised
ceiling declares a body. A comment could not have prevented the defect it
described; that test would have.

Both new tests were seen red for the right reason -- before the fix the
anonymous probe got its 401 only after the server had read 4236 bytes of a
multipart body, and re-declaring the File parameter turns the invariant test
red naming the route and the parameter.
This commit is contained in:
2026-09-22 01:59:03 +02:00
parent b82c22b888
commit f5018bdcbc
10 changed files with 396 additions and 109 deletions
+21 -20
View File
@@ -53,31 +53,32 @@ DEFAULT_MAX_BODY_BYTES = 1024 * 1024
# #
# These are RAISED CEILINGS, NOT EXEMPTIONS, and the difference is the whole point. An exemption # These are RAISED CEILINGS, NOT EXEMPTIONS, and the difference is the whole point. An exemption
# would mean "this route is counted by something else" — which is only true when the route's own # would mean "this route is counted by something else" — which is only true when the route's own
# limit applies to the bytes as they ARRIVE. That holds for the screenshot route (no declared body # limit applies to the bytes as they ARRIVE. That, in turn, is only true of a route with NO DECLARED
# parameter: `capped_body` streams it, and the owner check resolves first) and it does NOT hold for # BODY PARAMETER: FastAPI parses and spools a declared body BEFORE the first dependency runs, so a
# the upload route: `file: UploadFile = File(...)` is a declared body, so FastAPI parses and spools # route that declares one has already stored the bytes by the time its own cap — or its auth — gets
# the whole multipart part to a temp file BEFORE the first dependency runs — `require_human` # a say.
# included. Its in-handler cap therefore measures bytes that have already landed. Left uncounted, #
# an anonymous caller could stream without bound into the container's tmpfs; counted here, the # Both listed routes now satisfy that. The screenshot route always did (`capped_body` streams it,
# process refuses at the same number the route itself enforces, before any of it is stored. # and the owner check resolves first). The upload route did NOT until P1: it declared
# `file: UploadFile = File(...)`, so an anonymous caller could make the process write the whole
# per-file allowance to temp before `require_human` refused them — this middleware bounded that
# number but could not stop the writing. It now reads its own body from a dependency declared after
# the auth one, so its in-handler cap measures bytes as they arrive, like everything else here.
#
# That rule is no longer only written here: `test_body_limit.py` parses the routes and fails the
# gate if any route in this dict declares a body parameter. A comment could not have stopped the
# defect it describes; the test can.
# #
# A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE DEFAULT. That is why # A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE DEFAULT. That is why
# `POST /api/diagnostics/events` is absent: its `capped_body` limit is 512 KiB, already below the # `POST /api/diagnostics/events` is absent: its `capped_body` limit is 512 KiB, already below the
# default, so listing it could only widen what it accepts. # default, so listing it could only widen what it accepts.
# What a multipart envelope costs on top of the file itself: the boundary lines, the part headers
# and the sibling `display_name` field. The edge counts RAW BODY bytes while the upload route counts
# the DECODED file, so without this allowance a file of exactly the configured maximum — the number
# the admin page advertises, and the first thing anyone testing the limit reaches for — is refused
# one layer before the route that would have accepted it. Generous on purpose: its job is to keep
# the two layers from disagreeing at the boundary, not to be a second limit.
MULTIPART_ENVELOPE_ALLOWANCE = 64 * 1024
ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = { ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = {
# Ingesting a local video file. `sysconfig`'s `download_upload_max_bytes` is bounded so it can # Ingesting a local file. The body IS the file since P1 — no multipart envelope to allow for —
# never exceed this (see sysconfig.SPECS), which is what stops an admin from raising the route's # so this is exactly the number the route enforces. `sysconfig`'s `download_upload_max_bytes` is
# own cap past the edge and collecting 413s from a layer the admin page never mentions. # bounded so it can never exceed it (see sysconfig.SPECS), which is what stops an admin from
("POST", "/api/downloads/upload"): settings.download_upload_max_bytes # raising the route's own cap past the edge and collecting 413s from a layer the admin page
+ MULTIPART_ENVELOPE_ALLOWANCE, # never mentions.
("POST", "/api/downloads/upload"): settings.download_upload_max_bytes,
("POST", "/api/debug-sessions/{sid}/screenshots"): debugsession.SCREENSHOT_MAX_BYTES, ("POST", "/api/debug-sessions/{sid}/screenshots"): debugsession.SCREENSHOT_MAX_BYTES,
} }
+25 -1
View File
@@ -6,7 +6,7 @@ and "fetch a row this user owns or 404" idioms have one definition, one status c
one existence-leak policy. one existence-leak policy.
""" """
from typing import Any, TypeVar from typing import IO, Any, TypeVar
from fastapi import HTTPException, Request from fastapi import HTTPException, Request
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -79,6 +79,30 @@ async def capped_body(request: Request, limit: int) -> bytes:
return bytes(body) return bytes(body)
async def capped_stream_to(
request: Request, out: IO[bytes], limit: int, refusal: HTTPException
) -> int:
"""`capped_body`'s sibling for a body too large to hold: the same counting rule, written
straight to `out` instead of accumulated, returning the size.
Two deliberate differences from `capped_body`. It never builds a `bytearray` — the upload route
admits 500 MiB by default, so holding the body would trade a disk problem for a memory one. And
the refusal is the CALLER'S, passed in rather than assumed: `capped_body`'s flat 413 is right
for a screenshot, while the upload route answers 422 `upload_too_large` carrying the admin
knob's own number, which is what its dialog shows the user. One counting rule, two answers,
neither of them a second copy of the loop.
The caller owns `out` and owns cleaning it up: a body refused half-way has already written what
arrived before the limit was passed."""
size = 0
async for chunk in request.stream():
size += len(chunk)
if size > limit:
raise refusal
out.write(chunk)
return size
def issue_refused(exc: gitea.GiteaError) -> HTTPException: def issue_refused(exc: gitea.GiteaError) -> HTTPException:
"""The refusal an issue-tracker failure (E16) answers as — the send, the preview and the """The refusal an issue-tracker failure (E16) answers as — the send, the preview and the
admin's connection test alike. Literal reasons, one per case: the frontend's contract test reads admin's connection test alike. Literal reasons, one per case: the frontend's contract test reads
+95 -41
View File
@@ -9,11 +9,13 @@ serves finished files (range-aware, with the user's custom display name), and sh
import os import os
import re import re
import tempfile import tempfile
from collections.abc import AsyncIterator
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from pathlib import Path from pathlib import Path
from urllib.parse import parse_qs, urlparse from urllib.parse import parse_qs, unquote, urlparse
from fastapi import APIRouter, Depends, File, Form, HTTPException, Request, UploadFile from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import FileResponse from fastapi.responses import FileResponse
from pydantic import BaseModel from pydantic import BaseModel
from sqlalchemy import func, select from sqlalchemy import func, select
@@ -38,7 +40,7 @@ from app.models import (
MediaAsset, MediaAsset,
User, User,
) )
from app.routes._common import get_or_404, not_found, owned_or_404 from app.routes._common import capped_stream_to, get_or_404, not_found, owned_or_404
from app.security import hash_password from app.security import hash_password
from app.userscope import is_messageable_user, messageable_clauses from app.userscope import is_messageable_user, messageable_clauses
from app.utils import iso from app.utils import iso
@@ -406,61 +408,113 @@ def enqueue_download(
return _serialize_job(db, job) return _serialize_job(db, job)
# Read the upload in 1 MiB chunks so a large file never sits fully in memory, and the size cap is # The upload's filename and display name travel as HEADERS, not as multipart parts. That is not a
# enforced on the ACTUAL bytes read (a client-set Content-Length can't be trusted to gate it). # style choice: a DECLARED body parameter (`file: UploadFile = File(...)`) is read and spooled by
_UPLOAD_CHUNK = 1024 * 1024 # FastAPI *before the first dependency runs*, so `require_human` could not gate it and an anonymous
# caller could make the process write the whole per-file allowance to temp, per request (P1; SEC-20
# bounded the number but could not stop the writing). Reading the body ourselves, from a dependency
# declared AFTER the auth one, is the only shape that closes it — the same one
# `POST /api/debug-sessions/{sid}/screenshots` uses, and `test_body_limit.py` now enforces it for
# every route with a raised ceiling rather than leaving it to a comment like this one.
#
# Headers are latin-1 on the wire, so both are percent-encoded by the client and unquoted here; a
# Hungarian filename would otherwise be unsendable.
UPLOAD_FILENAME_HEADER = "x-upload-filename"
UPLOAD_DISPLAY_NAME_HEADER = "x-upload-display-name"
@router.post("/upload") @dataclass(frozen=True)
def upload_download( class SpooledUpload:
file: UploadFile = File(...), """What the body dependency hands the handler: the bytes already on disk, plus the two names
display_name: str | None = Form(None), that came alongside them."""
path: Path
size: int
user_id: int
filename: str
display_name: str | None
def _header_text(request: Request, name: str) -> str:
"""One percent-encoded header, decoded. A header that is not valid UTF-8 once unquoted is the
caller's doing, not a server fault — it degrades to empty rather than 500ing, and the route's
own fallback names the file."""
raw = request.headers.get(name, "")
try:
return unquote(raw, errors="strict").strip()
except (UnicodeDecodeError, ValueError):
# silent-ok: a mis-encoded header is the caller's own malformed request, and this one is
# cosmetic — it names the file. Refusing the upload over it would fail a 500 MiB transfer on
# a label; logging it would hand any client a free log line per request.
return ""
async def spool_upload(
request: Request,
user: User = Depends(require_human), user: User = Depends(require_human),
db: Session = Depends(get_db), db: Session = Depends(get_db),
) -> dict: ) -> AsyncIterator[SpooledUpload]:
"""Ingest a local file into the caller's library (E7 S5). Any file type is accepted; media """Stream the request body to a staging temp file, enforcing the per-file cap as it arrives.
(video/audio) gets the full function set, a generic file is download-only. The file is streamed
to a temp path under the download volume (enforcing the per-file cap as it goes), then handed to
the non-yt-dlp ingest which probes + places it and returns an already-`done` job.
A sync (threadpool) handler on purpose: the ingest does blocking work — an ffprobe of up to 30s `user` is taken but unused: it is the whole point. Dependencies resolve in declaration order, so
plus sync DB writes — so it must NOT run on the event loop. `file.file` is the sync file object naming `require_human` here makes the auth check a precondition of reading the body at all — a
Starlette spools the multipart body into.""" stranger is refused before a byte moves. The `yield` shape owns the temp file's life: ingest
MOVES it into place on success (the unlink is then a no-op), and on any other exit — cap hit,
empty, quota, probe or move error, a client that vanished mid-upload — it is removed."""
cap = sysconfig.get_int(db, "download_upload_max_bytes") cap = sysconfig.get_int(db, "download_upload_max_bytes")
staging = Path(settings.download_root) / ".staging" / "uploads" staging = Path(settings.download_root) / ".staging" / "uploads"
staging.mkdir(parents=True, exist_ok=True) staging.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=str(staging)) fd, tmp = tempfile.mkstemp(dir=str(staging))
tmp_path = Path(tmp) tmp_path = Path(tmp)
try: try:
size = 0
with os.fdopen(fd, "wb") as out: with os.fdopen(fd, "wb") as out:
while chunk := file.file.read(_UPLOAD_CHUNK): size = await capped_stream_to(
size += len(chunk) request,
if cap and size > cap: out,
raise HTTPException( cap,
status_code=422, detail={"code": "upload_too_large", "limit": cap} HTTPException(status_code=422, detail={"code": "upload_too_large", "limit": cap}),
)
out.write(chunk)
if size == 0:
raise HTTPException(status_code=400, detail={"code": "upload_empty"})
try:
job = uploadmod.ingest_upload(
db,
user.id,
filename=file.filename or "upload",
src_path=tmp_path,
size=size,
display_name=(display_name or "").strip() or None,
) )
except quota.QuotaExceeded as e: yield SpooledUpload(
raise HTTPException(status_code=422, detail=_quota_detail(e)) path=tmp_path,
return _serialize_job(db, job) size=size,
user_id=user.id,
filename=_header_text(request, UPLOAD_FILENAME_HEADER) or "upload",
display_name=_header_text(request, UPLOAD_DISPLAY_NAME_HEADER) or None,
)
finally: finally:
# Ingest MOVES the temp file into place on success (unlink is then a no-op); on any early exit
# — cap hit, empty, quota, probe/move error — this removes the abandoned staging file.
tmp_path.unlink(missing_ok=True) tmp_path.unlink(missing_ok=True)
@router.post("/upload")
def upload_download(
upload: SpooledUpload = Depends(spool_upload),
db: Session = Depends(get_db),
) -> dict:
"""Ingest a local file into the caller's library (E7 S5). Any file type is accepted; media
(video/audio) gets the full function set, a generic file is download-only. The body IS the file
(see the header note above); `spool_upload` has already authenticated the caller and streamed it
to a temp path under the download volume, so what is left here is the non-yt-dlp ingest which
probes + places it and returns an already-`done` job.
A sync (threadpool) handler on purpose: the ingest does blocking work — an ffprobe of up to 30s
plus sync DB writes — so it must NOT run on the event loop. The streaming half is async because
reading the body is; keeping them in separate callables is what lets each run where it belongs."""
if upload.size == 0:
raise HTTPException(status_code=400, detail={"code": "upload_empty"})
try:
job = uploadmod.ingest_upload(
db,
upload.user_id,
filename=upload.filename,
src_path=upload.path,
size=upload.size,
display_name=upload.display_name,
)
except quota.QuotaExceeded as e:
raise HTTPException(status_code=422, detail=_quota_detail(e))
return _serialize_job(db, job)
class EnqueueEditIn(BaseModel): class EnqueueEditIn(BaseModel):
source_job_id: int = 0 source_job_id: int = 0
edit_spec: dict edit_spec: dict
+22
View File
@@ -15,6 +15,7 @@ tests that COMMIT (the whole point of this lane — transaction-ownership behavi
import os import os
import shutil import shutil
from urllib.parse import quote
import pytest import pytest
from alembic import command from alembic import command
@@ -248,6 +249,27 @@ def as_user(db, override_dep, _configured):
return _as return _as
@pytest.fixture
def upload_headers():
"""Build the headers `POST /api/downloads/upload` takes since P1.
Shared, because two test modules drive that route and a second copy of this is exactly the drift
that lets the wire shape change in one place only. The body IS the file now; the names ride as
percent-encoded headers (`quote` mirrors the client's `encodeURIComponent`, since headers are
latin-1 and a Hungarian filename would not survive raw)."""
def _headers(filename: str, display_name: str | None = None) -> dict[str, str]:
headers = {
"content-type": "application/octet-stream",
"x-upload-filename": quote(filename),
}
if display_name is not None:
headers["x-upload-display-name"] = quote(display_name)
return headers
return _headers
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def _reset_rate_limiters(): def _reset_rate_limiters():
"""Every RateLimiter is a module-level singleton keyed by client IP, and TestClient always """Every RateLimiter is a module-level singleton keyed by client IP, and TestClient always
+48 -5
View File
@@ -251,14 +251,57 @@ def test_the_upload_ceiling_and_the_admin_knob_cannot_drift_apart():
constant. If the knob could be set above the edge, an admin raising it would silently start constant. If the knob could be set above the edge, an admin raising it would silently start
getting 413s from a layer the admin page never mentions. getting 413s from a layer the admin page never mentions.
The relation is `<=`, not `==`, and the gap is the point: the edge counts RAW body bytes while Since P1 the two count the SAME bytes — the body is the file, with no multipart envelope
the route counts the DECODED file, so the edge has to allow for the multipart envelope or a file wrapped round it — so the old allowance is gone and the edge is the knob's own maximum,
of exactly the advertised maximum is refused one layer before the route that would take it. exactly. A gap reappearing means someone re-wrapped the upload in an envelope without saying so.
""" """
spec = next(s for s in sysconfig.SPECS if s.key == "download_upload_max_bytes") spec = next(s for s in sysconfig.SPECS if s.key == "download_upload_max_bytes")
edge = bodylimit.ROUTE_MAX_BODY_BYTES[("POST", "/api/downloads/upload")] edge = bodylimit.ROUTE_MAX_BODY_BYTES[("POST", "/api/downloads/upload")]
assert spec.max is not None and spec.max <= edge assert spec.max == edge
assert edge - spec.max == bodylimit.MULTIPART_ENVELOPE_ALLOWANCE
def _route_objects(root):
"""`_declared_routes`' sibling, keeping the route OBJECT so its FastAPI dependant can be read."""
found: dict[tuple[str, str], object] = {}
stack = [root]
while stack:
node = stack.pop()
for route in getattr(node, "routes", ()) or ():
inner = getattr(route, "original_router", None)
if inner is not None:
stack.append(inner)
continue
for method in getattr(route, "methods", None) or ():
found[(method, route.path)] = route
return found
@pytest.mark.parametrize(("method", "template"), sorted(bodylimit.ROUTE_MAX_BODY_BYTES))
def test_no_route_with_a_raised_ceiling_declares_a_body_parameter(method, template):
"""P1's invariant, and the reason this is a test rather than a comment.
A raised ceiling is only honest when the route's own cap applies to the bytes AS THEY ARRIVE.
FastAPI reads and spools a DECLARED body (`= File(...)`, `= Form(...)`, a Pydantic model)
BEFORE the first dependency resolves — so on such a route this middleware's ceiling is the only
thing between an anonymous caller and the whole allowance, `require_human` never gets a say, and
the route's in-handler cap measures bytes that have already landed. That is exactly what P1
closed on the upload route, which sat in `ROUTE_MAX_BODY_BYTES` with a 500 MiB ceiling and a
`file: UploadFile = File(...)` body for one release.
Asked of FastAPI itself (`route.dependant.body_params`) rather than parsed or grepped: the
router already knows which parameters it will read the body for, and a shape a regex or an AST
walk guesses wrong is precisely how the next route slips through."""
from app.main import app
route = _route_objects(app).get((method, template))
assert route is not None, f"{method} {template} is not a route any more"
body_params = [p.name for p in getattr(route.dependant, "body_params", ())]
assert not body_params, (
f"{method} {template} declares a body parameter ({', '.join(body_params)}), so FastAPI "
"spools it before the first dependency runs and the raised ceiling is all that stands "
"between an anonymous caller and the whole allowance. Read the body yourself, from a "
"dependency declared AFTER the auth one — see routes/downloads.py:spool_upload."
)
def test_a_stored_override_outside_the_bounds_falls_back_on_READ(db): def test_a_stored_override_outside_the_bounds_falls_back_on_READ(db):
+2 -2
View File
@@ -18,12 +18,12 @@ _PW = "open sesame"
@pytest.fixture @pytest.fixture
def unlock_url(as_user, make_user, tmp_path, monkeypatch): def unlock_url(as_user, make_user, tmp_path, monkeypatch, upload_headers):
monkeypatch.setattr(settings, "download_root", str(tmp_path)) monkeypatch.setattr(settings, "download_root", str(tmp_path))
monkeypatch.setattr(probemod, "probe_media", lambda _p: ("h264", "aac", 320, 240, 5)) monkeypatch.setattr(probemod, "probe_media", lambda _p: ("h264", "aac", 320, 240, 5))
owner = as_user(make_user()) owner = as_user(make_user())
job = owner.post( job = owner.post(
"/api/downloads/upload", files={"file": ("clip.mp4", b"fake-video", "video/mp4")} "/api/downloads/upload", content=b"fake-video", headers=upload_headers("clip.mp4")
) )
assert job.status_code == 200, job.text assert job.status_code == 200, job.text
link = owner.post(f"/api/downloads/{job.json()['id']}/links", json={"password": _PW}) link = owner.post(f"/api/downloads/{job.json()['id']}/links", json={"password": _PW})
+124 -25
View File
@@ -123,15 +123,106 @@ def test_ingest_rejects_over_footprint_cap_before_writing(db, make_user, _root,
assert db.query(MediaAsset).count() == 0 # and no orphan asset row assert db.query(MediaAsset).count() == 0 # and no orphan asset row
# --- route: the auth-first body (P1) --------------------------------------------------------
def _counting_body(chunks: int = 4, size: int = 1024):
"""A request body that records how much of itself the server actually pulled.
httpx's ASGI transport feeds the app lazily, one `http.request` message per yielded chunk, so a
chunk that was never yielded is a chunk the app never asked for. That is the whole assertion
below: it is not "the upload was refused", it is "the refusal happened before the bytes moved".
"""
pulled: list[int] = []
def body():
for _ in range(chunks):
pulled.append(size)
yield b"x" * size
return body(), pulled
_BOUNDARY = "p1boundary"
def _counting_multipart(chunks: int = 4, size: int = 1024):
"""The same counter, wrapped in a real multipart envelope.
The content-type is the load-bearing part. Starlette's `request.form()` returns an EMPTY
FormData without touching the stream when the content-type is not multipart/urlencoded — so a
raw-body probe cannot tell "the route stopped reading strangers" apart from "there was nothing
here to parse". Only a well-formed multipart body makes the old shape actually spool.
"""
pulled: list[int] = []
head = (
f"--{_BOUNDARY}\r\n"
'Content-Disposition: form-data; name="file"; filename="clip.mp4"\r\n'
"Content-Type: application/octet-stream\r\n\r\n"
).encode()
tail = f"\r\n--{_BOUNDARY}--\r\n".encode()
def body():
pulled.append(len(head))
yield head
for _ in range(chunks):
pulled.append(size)
yield b"x" * size
pulled.append(len(tail))
yield tail
return body(), pulled
def test_an_anonymous_upload_is_refused_before_a_byte_of_it_is_read(client):
"""P1 — the defect E27 bounded but did not close.
While the route declared `file: UploadFile = File(...)`, FastAPI parsed and spooled the body
BEFORE the first dependency ran, so `require_human` could not gate it: an unauthenticated
caller could make the server write the whole per-file allowance (500 MiB by default) to temp,
per request. SEC-20's ceiling capped the number; it could not stop the writing.
The route now reads its own body from a dependency declared AFTER the auth one, the shape
`POST /api/debug-sessions/{sid}/screenshots` already uses. The claim under test is the strong
one — a stranger's bytes are never read WHATEVER they claim to send, so the probe sends the
multipart the old route would have spooled, not the raw body the new one expects."""
body, pulled = _counting_multipart()
r = client.post(
"/api/downloads/upload",
content=body,
headers={"content-type": f"multipart/form-data; boundary={_BOUNDARY}"},
)
assert r.status_code == 401
assert pulled == [], f"the server read {sum(pulled)} bytes from an anonymous caller"
def test_an_authenticated_upload_still_has_its_body_read(as_user, make_user, _root, monkeypatch):
"""The mutation check for the test above: a route that read NOBODY's body would pass it just as
well as the fixed one. The same counting body, signed in, must be pulled in full."""
_probe(monkeypatch, (None, None, None, None, None))
body, pulled = _counting_body()
c = as_user(make_user())
r = c.post(
"/api/downloads/upload",
content=body,
headers={"content-type": "application/octet-stream", "x-upload-filename": "clip.bin"},
)
assert r.status_code == 200, r.text
assert sum(pulled) == 4096
# --- route ---------------------------------------------------------------------------------- # --- route ----------------------------------------------------------------------------------
def test_upload_route_generic_forces_download(as_user, make_user, _root, monkeypatch): def test_upload_route_generic_forces_download(
as_user, make_user, _root, monkeypatch, upload_headers
):
_probe(monkeypatch, (None, None, None, None, None)) _probe(monkeypatch, (None, None, None, None, None))
c = as_user(make_user()) c = as_user(make_user())
r = c.post( r = c.post(
"/api/downloads/upload", "/api/downloads/upload",
files={"file": ("report.html", b"<script>alert(1)</script>", "text/html")}, content=b"<script>alert(1)</script>",
headers=upload_headers("report.html"),
) )
assert r.status_code == 200, r.text assert r.status_code == 200, r.text
job = r.json() job = r.json()
@@ -145,7 +236,7 @@ def test_upload_route_generic_forces_download(as_user, make_user, _root, monkeyp
def test_upload_route_media_html_extension_not_served_as_html( def test_upload_route_media_html_extension_not_served_as_html(
as_user, make_user, _root, monkeypatch as_user, make_user, _root, monkeypatch, upload_headers
): ):
"""The XSS guard: a MEDIA-classified upload (real codecs) named with an executable extension must """The XSS guard: a MEDIA-classified upload (real codecs) named with an executable extension must
NOT be served with a guessed text/html type. The served Content-Type comes from the verified NOT be served with a guessed text/html type. The served Content-Type comes from the verified
@@ -156,7 +247,8 @@ def test_upload_route_media_html_extension_not_served_as_html(
c = as_user(make_user()) c = as_user(make_user())
r = c.post( r = c.post(
"/api/downloads/upload", "/api/downloads/upload",
files={"file": ("evil.html", b"<script>alert(1)</script>", "text/html")}, content=b"<script>alert(1)</script>",
headers=upload_headers("evil.html"),
) )
assert r.status_code == 200, r.text assert r.status_code == 200, r.text
job = r.json() job = r.json()
@@ -177,7 +269,9 @@ def test_upload_route_media_html_extension_not_served_as_html(
assert "attachment" in pr.headers["content-disposition"].lower() assert "attachment" in pr.headers["content-disposition"].lower()
def test_public_watch_file_forces_download_for_generic_file(as_user, make_user, _root, monkeypatch): def test_public_watch_file_forces_download_for_generic_file(
as_user, make_user, _root, monkeypatch, upload_headers
):
"""The login-free public serve of a generic upload (media_kind='file') must be a neutral """The login-free public serve of a generic upload (media_kind='file') must be a neutral
force-download attachment — the branch that stops uploaded HTML/SVG executing on the public force-download attachment — the branch that stops uploaded HTML/SVG executing on the public
origin. This is a SEPARATE code path from the authenticated /file route, so it needs its own origin. This is a SEPARATE code path from the authenticated /file route, so it needs its own
@@ -186,7 +280,8 @@ def test_public_watch_file_forces_download_for_generic_file(as_user, make_user,
c = as_user(make_user()) c = as_user(make_user())
job = c.post( job = c.post(
"/api/downloads/upload", "/api/downloads/upload",
files={"file": ("notes.html", b"<script>alert(1)</script>", "text/html")}, content=b"<script>alert(1)</script>",
headers=upload_headers("notes.html"),
).json() ).json()
assert job["asset"]["media_kind"] == "file" assert job["asset"]["media_kind"] == "file"
link = c.post(f"/api/downloads/{job['id']}/links", json={"allow_download": True}).json() link = c.post(f"/api/downloads/{job['id']}/links", json={"allow_download": True}).json()
@@ -197,7 +292,7 @@ def test_public_watch_file_forces_download_for_generic_file(as_user, make_user,
def test_public_watch_file_serves_media_inline_with_real_type( def test_public_watch_file_serves_media_inline_with_real_type(
as_user, make_user, _root, monkeypatch as_user, make_user, _root, monkeypatch, upload_headers
): ):
"""A recognised media container IS still served inline with its real audio/video type (so it """A recognised media container IS still served inline with its real audio/video type (so it
plays in the watch page) — the fix must not break legitimate playback. A stream-only link → plays in the watch page) — the fix must not break legitimate playback. A stream-only link →
@@ -206,7 +301,8 @@ def test_public_watch_file_serves_media_inline_with_real_type(
c = as_user(make_user()) c = as_user(make_user())
job = c.post( job = c.post(
"/api/downloads/upload", "/api/downloads/upload",
files={"file": ("clip.mp4", b"fake-mp4-bytes", "video/mp4")}, content=b"fake-mp4-bytes",
headers=upload_headers("clip.mp4"),
).json() ).json()
link = c.post(f"/api/downloads/{job['id']}/links", json={"allow_download": False}).json() link = c.post(f"/api/downloads/{job['id']}/links", json={"allow_download": False}).json()
pr = c.get(f"/api/public/watch/{link['token']}/file") pr = c.get(f"/api/public/watch/{link['token']}/file")
@@ -215,7 +311,9 @@ def test_public_watch_file_serves_media_inline_with_real_type(
assert "inline" in pr.headers["content-disposition"].lower() assert "inline" in pr.headers["content-disposition"].lower()
def test_upload_route_enforces_per_file_cap(as_user, make_user, db, _root, monkeypatch): def test_upload_route_enforces_per_file_cap(
as_user, make_user, db, _root, monkeypatch, upload_headers
):
from app import sysconfig from app import sysconfig
sysconfig.set_value(db, "download_upload_max_bytes", 5) sysconfig.set_value(db, "download_upload_max_bytes", 5)
@@ -223,47 +321,48 @@ def test_upload_route_enforces_per_file_cap(as_user, make_user, db, _root, monke
c = as_user(make_user()) c = as_user(make_user())
r = c.post( r = c.post(
"/api/downloads/upload", "/api/downloads/upload",
files={"file": ("big.bin", b"way over the cap", "application/octet-stream")}, content=b"way over the cap",
headers=upload_headers("big.bin"),
) )
assert r.status_code == 422 assert r.status_code == 422
assert r.json()["detail"]["code"] == "upload_too_large" assert r.json()["detail"]["code"] == "upload_too_large"
def test_upload_cap_cannot_be_disabled_with_zero(db): def test_upload_cap_cannot_be_disabled_with_zero(db):
"""download_upload_max_bytes floors at 1: a stored 0 would read falsy in the route's """download_upload_max_bytes floors at 1, and set_value must enforce it.
`if cap and size > cap` guard and SILENTLY disable the per-file cap, so set_value must reject it.
Pins the min=1 invariant so a revert to min=0 can't pass the gate.""" The falsy-0 footgun this originally guarded is gone — the route's old `if cap and size > cap`
read a stored 0 as "no cap at all", while `capped_stream_to` compares unconditionally and a 0
would refuse everything instead. Refusing everything is the safe direction, but it is still
wrong, and the knob is also the number the admin page and the 422 `limit` advertise. Pins the
min=1 invariant so a revert to min=0 can't pass the gate."""
from app import sysconfig from app import sysconfig
with pytest.raises(ValueError): with pytest.raises(ValueError):
sysconfig.set_value(db, "download_upload_max_bytes", 0) sysconfig.set_value(db, "download_upload_max_bytes", 0)
def test_upload_route_rejects_empty_file(as_user, make_user, _root, monkeypatch): def test_upload_route_rejects_empty_file(as_user, make_user, _root, monkeypatch, upload_headers):
c = as_user(make_user()) c = as_user(make_user())
r = c.post( r = c.post("/api/downloads/upload", content=b"", headers=upload_headers("empty.bin"))
"/api/downloads/upload", files={"file": ("empty.bin", b"", "application/octet-stream")}
)
assert r.status_code == 400 assert r.status_code == 400
assert r.json()["detail"]["code"] == "upload_empty" assert r.json()["detail"]["code"] == "upload_empty"
def test_redownload_rejects_an_upload(as_user, make_user, _root, monkeypatch): def test_redownload_rejects_an_upload(as_user, make_user, _root, monkeypatch, upload_headers):
_probe(monkeypatch, (None, None, None, None, None)) _probe(monkeypatch, (None, None, None, None, None))
c = as_user(make_user()) c = as_user(make_user())
job = c.post( job = c.post("/api/downloads/upload", content=b"data", headers=upload_headers("a.bin")).json()
"/api/downloads/upload", files={"file": ("a.bin", b"data", "application/octet-stream")}
).json()
r = c.post(f"/api/downloads/{job['id']}/redownload") r = c.post(f"/api/downloads/{job['id']}/redownload")
assert r.status_code == 409 assert r.status_code == 409
def test_edit_rejects_a_generic_upload_source(as_user, make_user, _root, monkeypatch): def test_edit_rejects_a_generic_upload_source(
as_user, make_user, _root, monkeypatch, upload_headers
):
_probe(monkeypatch, (None, None, None, None, None)) _probe(monkeypatch, (None, None, None, None, None))
c = as_user(make_user()) c = as_user(make_user())
job = c.post( job = c.post("/api/downloads/upload", content=b"data", headers=upload_headers("a.bin")).json()
"/api/downloads/upload", files={"file": ("a.bin", b"data", "application/octet-stream")}
).json()
r = c.post( r = c.post(
"/api/downloads/edit", "/api/downloads/edit",
json={"source_job_id": job["id"], "edit_spec": {"trim": {"start_s": 0, "end_s": 1}}}, json={"source_job_id": job["id"], "edit_spec": {"trim": {"start_s": 0, "end_s": 1}}},
+27 -1
View File
@@ -23,6 +23,7 @@ vi.mock("../activeAccount", () => ({
})); }));
import { setDebugSession } from "../log"; import { setDebugSession } from "../log";
import { downloadsApi } from "./downloads";
import { import {
beacon, beacon,
req, req,
@@ -46,6 +47,7 @@ class FakeXHR {
onerror: (() => void) | null = null; onerror: (() => void) | null = null;
onabort: (() => void) | null = null; onabort: (() => void) | null = null;
sent = false; sent = false;
body: unknown = null;
constructor() { constructor() {
FakeXHR.last = this; FakeXHR.last = this;
} }
@@ -53,8 +55,9 @@ class FakeXHR {
setRequestHeader(k: string, v: string) { setRequestHeader(k: string, v: string) {
this.headers[k] = v; this.headers[k] = v;
} }
send() { send(body?: unknown) {
this.sent = true; this.sent = true;
this.body = body ?? null;
} }
abort() { abort() {
this.onabort?.(); this.onabort?.();
@@ -384,3 +387,26 @@ describe("the debug session a tab records into", () => {
expect(sentHeaders().has("X-Debug-Session")).toBe(false); expect(sentHeaders().has("X-Debug-Session")).toBe(false);
}); });
}); });
// P1 — the upload's wire shape is a security boundary, not a detail. A multipart part is a DECLARED
// body, which the server reads and spools BEFORE its auth dependency runs; the raw body is what lets
// `require_human` resolve first. A revert to FormData here would silently re-open that, and the
// backend's own test only sees the shape the client actually sends.
describe("uploadDownload's wire shape", () => {
it("sends the File as the raw body, with the names as percent-encoded headers", () => {
const file = new File(["abc"], "nyaralás videó.mp4", { type: "video/mp4" });
void downloadsApi.uploadDownload(file, { displayName: "Nyaralás 2026" });
const x = FakeXHR.last;
expect(x.body).toBe(file);
expect(x.body).not.toBeInstanceOf(FormData);
expect(x.headers["X-Upload-Filename"]).toBe(encodeURIComponent("nyaralás videó.mp4"));
expect(x.headers["X-Upload-Display-Name"]).toBe(encodeURIComponent("Nyaralás 2026"));
// Content-Type is the browser's to set from the File — an explicit one here would be a guess.
expect(x.headers["Content-Type"]).toBeUndefined();
});
it("omits the display-name header when there is no display name", () => {
void downloadsApi.uploadDownload(new File(["abc"], "clip.mp4"));
expect(FakeXHR.last.headers["X-Upload-Display-Name"]).toBeUndefined();
});
});
+21 -10
View File
@@ -372,16 +372,27 @@ export async function req<T = unknown>(
} }
} }
// Multipart upload with progress. `req` rides `fetch`, which can't report upload progress, so a // Upload with progress. `req` rides `fetch`, which can't report upload progress, so a file upload
// file upload uses XHR instead: the caller gets a 0–100% callback as the body streams and an // uses XHR instead: the caller gets a 0–100% callback as the body streams and an optional
// optional AbortSignal to cancel a large upload. Everything else mirrors `req` — the per-tab // AbortSignal to cancel a large upload. Everything else mirrors `req` — the per-tab account header,
// account header, credentials, the structured-detail localization, the global error modal for // credentials, the structured-detail localization, the global error modal for 4xx/5xx, and an
// 4xx/5xx, and an `HttpError` throw — so callers handle failures exactly as they do a `req` call. // `HttpError` throw — so callers handle failures exactly as they do a `req` call.
// The browser sets the multipart Content-Type (with its boundary), so we deliberately don't. //
// `body` is a `Blob`/`File` for a raw upload or `FormData` for a multipart one. The raw shape is
// what `/api/downloads/upload` uses since P1, and it is not a preference: a multipart part is a
// DECLARED body, which FastAPI spools before the route's auth dependency runs. `cfg.headers` is how
// such a caller sends the metadata that used to be sibling form fields.
//
// Content-Type is never set here. The browser derives it — the multipart boundary for FormData, the
// Blob's own type otherwise — and overriding it would break the boundary.
export function uploadReq<T = unknown>( export function uploadReq<T = unknown>(
url: string, url: string,
form: FormData, body: Blob | FormData,
cfg: { onProgress?: (pct: number) => void; signal?: AbortSignal } = {}, cfg: {
onProgress?: (pct: number) => void;
signal?: AbortSignal;
headers?: Record<string, string>;
} = {},
): Promise<T> { ): Promise<T> {
return new Promise<T>((resolve, reject) => { return new Promise<T>((resolve, reject) => {
const xhr = new XMLHttpRequest(); const xhr = new XMLHttpRequest();
@@ -390,7 +401,7 @@ export function uploadReq<T = unknown>(
const requestId = newRequestId(); const requestId = newRequestId();
const started = performance.now(); const started = performance.now();
xhr.setRequestHeader(REQUEST_ID_HEADER, requestId); xhr.setRequestHeader(REQUEST_ID_HEADER, requestId);
for (const [name, value] of Object.entries(identityHeaders())) for (const [name, value] of Object.entries({ ...identityHeaders(), ...cfg.headers }))
xhr.setRequestHeader(name, value); xhr.setRequestHeader(name, value);
if (cfg.onProgress) { if (cfg.onProgress) {
xhr.upload.onprogress = (e) => { xhr.upload.onprogress = (e) => {
@@ -443,6 +454,6 @@ export function uploadReq<T = unknown>(
logRequest(requestId, "POST", url, started, { failed: "aborted" }); logRequest(requestId, "POST", url, started, { failed: "aborted" });
reject(new HttpError(0, ABORTED)); reject(new HttpError(0, ABORTED));
}; };
xhr.send(form); xhr.send(body);
}); });
} }
+11 -4
View File
@@ -187,16 +187,23 @@ export const downloadsApi = {
// Ingest a local file into the library (E7 S5). Multipart via XHR so the dialog can show upload // Ingest a local file into the library (E7 S5). Multipart via XHR so the dialog can show upload
// progress; any file type is accepted (media gets the full function set, a generic file is // progress; any file type is accepted (media gets the full function set, a generic file is
// download-only). Returns the already-`done` job the ingest creates. // download-only). Returns the already-`done` job the ingest creates.
// The body IS the file, and the two names ride as headers. Not a style choice: a multipart part is
// a DECLARED body, which FastAPI reads and spools BEFORE the route's `require_human` dependency
// runs — so with the old shape an unauthenticated caller could make the server write the whole
// per-file allowance to temp (P1). Headers are latin-1 on the wire, hence `encodeURIComponent`;
// the server percent-decodes both. Content-Type is left to the browser (the File's own type).
uploadDownload: ( uploadDownload: (
file: File, file: File,
opts: { displayName?: string; onProgress?: (pct: number) => void; signal?: AbortSignal } = {}, opts: { displayName?: string; onProgress?: (pct: number) => void; signal?: AbortSignal } = {},
): Promise<DownloadJob> => { ): Promise<DownloadJob> => {
const form = new FormData(); const headers: Record<string, string> = {
form.append("file", file); "X-Upload-Filename": encodeURIComponent(file.name || "upload"),
if (opts.displayName) form.append("display_name", opts.displayName); };
return uploadReq<DownloadJob>("/api/downloads/upload", form, { if (opts.displayName) headers["X-Upload-Display-Name"] = encodeURIComponent(opts.displayName);
return uploadReq<DownloadJob>("/api/downloads/upload", file, {
onProgress: opts.onProgress, onProgress: opts.onProgress,
signal: opts.signal, signal: opts.signal,
headers,
}); });
}, },
enqueueEdit: (body: { enqueueEdit: (body: {