peter/siftlode
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(security): SEC-20's exemptions become raised ceilings, and the 413 is dressed

Browse Source 
The module admitted its two opt-outs under a stated rule — "both enforce their own
limit on the bytes they actually read, which is what makes opting them out safe" —
and that rule is false for one of them. `POST /api/downloads/upload` declares
`file: UploadFile = File(...)`, so FastAPI parses and spools the whole multipart
part to a temp file BEFORE the first dependency runs, `require_human` included:
its in-handler cap measures bytes that have already landed. Measured on the
installed fastapi 0.141.1 / starlette 1.6.0 — starlette's max_part_size guards
non-file parts only, so an anonymous chunked upload was bounded by nothing. In the
shipped self-host compose that lands in a RAM-backed tmpfs.

The exposure predates SEC-20 (no route was capped before it); what this branch
added was the claim. So the list becomes a mapping of per-route CEILINGS instead
of a set of exemptions: every request now gets a finite number, the upload route's
being the same one the route itself enforces. `sysconfig`'s knob is bounded by it
so an admin cannot raise the handler's cap past the edge and start collecting 413s
from a layer the admin page never mentions — held by a test, not by prose.

Two more things the review caught in the same module:

The 413 was not byte-identical to `capped_body`'s after all. `json.dumps` puts
spaces where JSONResponse does not, and the test that promised to hold them
together compared PARSED JSON, so it could never have seen it. Built through the
same JSONResponse now, and compared as raw bytes.

And the middleware sat outside CORS and the security headers, so its 413 carried
neither — on the split-origin dev setup the browser drops it and the user gets a
generic network failure instead of the translated too_large, while the identical
refusal from capped_body one layer down renders fine. It moves to just inside
CORS: still ahead of the session, the setup gate and every auth dependency.

Also corrected: the docblock's "largest non-exempt body is 64 KiB, ~16x headroom"
(it is 512 KiB and 2x — the diagnostics sink, named eleven lines further down),
and CLAUDE.md's Request SIZE row, which forbade the one correct use of
capped_body that is actually in the tree.
This commit is contained in:
peter
2026-09-21 23:32:49 +02:00
parent bf518223e5
commit 05121fbc99
5 changed files with 172 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/bodylimit.py
+65 -42
View File
@@ -19,9 +19,12 @@ simply understate. The bytes have to be counted as they arrive, which is the sam
already applies per-route, and counting needs the receive channel.
**Why the refusal is byte-identical to `capped_body`'s.** Two shapes for one fact is how the
`cancelled`/`canceled` bug happened. The 413 body here and the 413 body from `capped_body` are the
same JSON, and `tests/test_body_limit.py` compares them against each other rather than against a
literal, so they cannot drift apart.
`cancelled`/`canceled` bug happened. The 413 body here is built through the same `JSONResponse`
FastAPI renders `HTTPException(413, {"code": "too_large"})` with — not a hand-rolled `json.dumps`,
whose default separators differ by two bytes — and `tests/test_body_limit.py` compares the RAW BYTES
of the two against each other rather than against a literal, so they cannot drift apart. The same
reasoning decides where the middleware is registered (`main.py`): just inside CORS, so this 413
carries the same headers `capped_body`'s does.
**No admin knob, deliberately.** The ceiling is a constant, not a `sysconfig` value: an attacker-
facing limit that an admin can raise from the UI is a limit that can be raised by whoever gets the
@@ -29,40 +32,50 @@ admin session, and nobody has asked to tune it. If a self-hoster ever needs a di
belongs in `config.py` (env, set before the process starts), not in the database.
"""
import json
import re
from starlette.responses import JSONResponse
from starlette.types import ASGIApp, Message, Receive, Scope, Send
# 1 MiB. The largest body any non-exempt route legitimately takes is 64 KiB (the debug-session form,
# `routes/debug_sessions.py:FORM_MAX_BYTES`), so this leaves ~16x headroom: a ceiling that refuses a
# legitimate request would be a worse bug than the one this module fixes.
from app import debugsession
from app.config import settings
# 1 MiB. The largest body any route on the default ceiling legitimately takes is 512 KiB (the client
# log sink, `routes/diagnostics.py:BODY_MAX_BYTES`), so this leaves 2x headroom — a ceiling that
# refuses a legitimate request would be a worse bug than the one this module fixes. That ratio is
# the load-bearing fact, so `test_body_limit.py` holds it rather than the figure above.
DEFAULT_MAX_BODY_BYTES = 1024 * 1024
# The routes that opt out, as (method, path TEMPLATE) — the template, not a concrete path, so the
# list reads like the route decorators it mirrors and `test_body_limit.py` can assert that each one
# still resolves to a real route. A renamed route therefore fails the gate instead of silently
# losing its exemption in prod.
# The routes that need MORE than the default, as (method, path TEMPLATE) → their own ceiling. The
# template, not a concrete path, so the list reads like the route decorators it mirrors and
# `test_body_limit.py` can assert that each one still resolves to a real route: a renamed route
# fails the gate instead of silently losing its allowance in prod.
#
# A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE GLOBAL ONE. That is the whole
# admission rule, and it is why `POST /api/diagnostics/events` is absent: its `capped_body` limit is
# 512 KiB, already below this ceiling, so the global cap never fires on it and an exemption would
# only widen what it accepts. Both entries below enforce their own limit on the bytes they actually
# read, which is what makes opting them out safe rather than merely convenient:
# /api/downloads/upload streams in 1 MiB chunks, caps on the real byte count (E7 S5);
# ingesting a local video file, so it is legitimately unbounded here
# /api/debug-sessions/{sid}/screenshots capped_body at SCREENSHOT_MAX_BYTES (10 MiB), and the
# owner check runs as a dependency BEFORE the body is read
EXEMPT_ROUTES: frozenset[tuple[str, str]] = frozenset(
{
("POST", "/api/downloads/upload"),
("POST", "/api/debug-sessions/{sid}/screenshots"),
}
)
# These are RAISED CEILINGS, NOT EXEMPTIONS, and the difference is the whole point. An exemption
# would mean "this route is counted by something else" — which is only true when the route's own
# limit applies to the bytes as they ARRIVE. That holds for the screenshot route (no declared body
# parameter: `capped_body` streams it, and the owner check resolves first) and it does NOT hold for
# the upload route: `file: UploadFile = File(...)` is a declared body, so FastAPI parses and spools
# the whole multipart part to a temp file BEFORE the first dependency runs — `require_human`
# included. Its in-handler cap therefore measures bytes that have already landed. Left uncounted,
# an anonymous caller could stream without bound into the container's tmpfs; counted here, the
# process refuses at the same number the route itself enforces, before any of it is stored.
#
# A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE DEFAULT. That is why
# `POST /api/diagnostics/events` is absent: its `capped_body` limit is 512 KiB, already below the
# default, so listing it could only widen what it accepts.
ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = {
# Ingesting a local video file. `sysconfig`'s `download_upload_max_bytes` is bounded by this
# same setting (see sysconfig.SPECS), so the admin knob can never ask for more than the edge
# will pass — the two numbers cannot drift into refusing a legitimate upload.
("POST", "/api/downloads/upload"): settings.download_upload_max_bytes,
("POST", "/api/debug-sessions/{sid}/screenshots"): debugsession.SCREENSHOT_MAX_BYTES,
}
# The refusal, in the exact shape `capped_body` raises — `HTTPException(413, {"code": "too_large"})`
# renders as this. Built once; the test asserts it against a real route's response.
TOO_LARGE_BODY = json.dumps({"detail": {"code": "too_large"}}).encode()
# The refusal, byte-for-byte what `capped_body`'s `HTTPException(413, {"code": "too_large"})` renders
# as — built through the same JSONResponse so the separators match too, and asserted against a real
# route's raw bytes in `test_body_limit.py` rather than against a literal here.
TOO_LARGE_BODY = JSONResponse({"detail": {"code": "too_large"}}, status_code=413).body
_PATH_PARAM = re.compile(r"\{[^/{}]+\}")
@@ -108,11 +121,14 @@ async def _send_too_large(send: Send) -> None:
class BodyLimitMiddleware:
"""Refuses any request body over `limit` bytes, before the application sees it.
"""Refuses any request body over its route's ceiling, before the application sees it.
Registered second-outermost in `main.py` — inside the access log so a refusal is still logged
with its request id, outside everything else so the session, CORS, setup-gate and auth layers
never run for a request that is already too large.
Registered just inside CORS in `main.py` — so the 413 is decorated like every other response
and a browser can actually read it — but outside the session, the setup gate and every auth
dependency, none of which run for a request that is already too large.
Every request gets a finite ceiling: `ROUTE_MAX_BODY_BYTES` raises it for the two routes that
genuinely take bulk bytes, and there is no path that skips counting altogether.
"""
def __init__(
@@ -120,24 +136,31 @@ class BodyLimitMiddleware:
app: ASGIApp,
*,
limit: int = DEFAULT_MAX_BODY_BYTES,
exempt: frozenset[tuple[str, str]] = EXEMPT_ROUTES,
routes: dict[tuple[str, str], int] | None = None,
) -> None:
self.app = app
self.limit = limit
self._exempt = tuple((method, template_to_regex(path)) for method, path in exempt)
self._routes = tuple(
(method, template_to_regex(path), cap)
for (method, path), cap in (ROUTE_MAX_BODY_BYTES if routes is None else routes).items()
)
def is_exempt(self, method: str, path: str) -> bool:
return any(m == method and rx.match(path) for m, rx in self._exempt)
def limit_for(self, method: str, path: str) -> int:
"""This request's ceiling: the route's own if it has one, else the global default. Every
request gets a finite number — there is no path through here that is simply not counted."""
for m, rx, cap in self._routes:
if m == method and rx.match(path):
return cap
return self.limit
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http" or self.is_exempt(
scope.get("method", ""), scope.get("path", "")
):
if scope["type"] != "http":
await self.app(scope, receive, send)
return
limit = self.limit_for(scope.get("method", ""), scope.get("path", ""))
declared = _declared_length(scope)
if declared is not None and declared > self.limit:
if declared is not None and declared > limit:
# Refused on the declaration alone: nothing downstream is called, and not one byte of
# the body is read.
await _send_too_large(send)
@@ -153,7 +176,7 @@ class BodyLimitMiddleware:
if message["type"] != "http.request":
return message
seen += len(message.get("body", b""))
if seen > self.limit:
if seen > limit:
# The app is blocked on this receive, so it has not answered yet. Telling it the
# client went away is the only in-band way to stop it reading: `request.stream()`
# turns an `http.disconnect` into ClientDisconnect, which unwinds the handler. Our