peter/siftlode
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(security): cap every request body before auth runs (SEC-20)

Browse Source 
FastAPI reads a declared body IN FULL before the first dependency, so
`POST /auth/password-login` drained an arbitrary upload from an anonymous
caller and only then answered 401. 70 of the 72 body-declaring routes
behaved the same way: one unauthenticated client could spend the
process's memory and time at will.

A raw ASGI middleware, not `@app.middleware("http")` — BaseHTTPMiddleware
cannot wrap `receive`, so it could only trust `Content-Length`, which a
client may omit (chunked) or understate. This refuses an oversized
declaration outright and counts the bytes of everything else as they
arrive, the same rule `capped_body` already applies per-route. Registered
directly inside the access log: a refusal is still logged with its
request id, but the session, CORS, the setup gate and every auth
dependency never run for a body that is already too large.

Two routes opt out, under one stated rule: an exemption is ONLY for a
route whose own cap is larger than the global one. That is why
`/api/diagnostics/events` is absent — it caps itself at 512 KiB, below
the ceiling, so exempting it could only widen what it accepts. The list
names route templates and a test resolves each against the live route
table, so a rename fails the gate instead of silently dropping an
exemption in prod.

The 413 is byte-identical to `capped_body`'s and the test compares the
two against each other rather than against a literal — two shapes for one
fact is how the `cancelled`/`canceled` bug happened.
This commit is contained in:
peter
2026-09-20 22:51:32 +02:00
parent 09d351a0e7
commit 51daf3e388
4 changed files with 429 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/bodylimit.py
+183
View File
@@ -0,0 +1,183 @@
"""The global request-body cap (SEC-20).
**The hole.** FastAPI reads a declared body IN FULL before auth or any other dependency runs — the
body parameter has to be parsed to be passed in. So `POST /auth/password-login` drains a 32 MiB
upload from an anonymous caller and only then answers 401. 70 of the 72 body-declaring routes behave
the same way: one unauthenticated client can spend the process's memory and time at will.
**Why a middleware and not `capped_body` on every route.** `routes/_common.py:capped_body` is the
right tool where a route genuinely takes bulk bytes, and the two routes that use it use it
correctly. But putting it on all 70 others would (a) fight the canonical "Request body → a Pydantic
model" row, since capping means taking the body as raw bytes and parsing it by hand, and (b) leave
the NEXT route added uncovered, because nothing would remind its author. A default that has to be
opted into is not a default. This module inverts it: the cap applies to everything, and the few
routes that need more opt OUT, in one list, next to the code that reads it.
**Why raw ASGI and not `@app.middleware("http")`.** Starlette's BaseHTTPMiddleware cannot wrap
`receive`, so it can only read `Content-Length` — which a client may omit entirely (chunked) or
simply understate. The bytes have to be counted as they arrive, which is the same rule `capped_body`
already applies per-route, and counting needs the receive channel.
**Why the refusal is byte-identical to `capped_body`'s.** Two shapes for one fact is how the
`cancelled`/`canceled` bug happened. The 413 body here and the 413 body from `capped_body` are the
same JSON, and `tests/test_body_limit.py` compares them against each other rather than against a
literal, so they cannot drift apart.
**No admin knob, deliberately.** The ceiling is a constant, not a `sysconfig` value: an attacker-
facing limit that an admin can raise from the UI is a limit that can be raised by whoever gets the
admin session, and nobody has asked to tune it. If a self-hoster ever needs a different ceiling it
belongs in `config.py` (env, set before the process starts), not in the database.
"""
import json
import re
from starlette.types import ASGIApp, Message, Receive, Scope, Send
# 1 MiB. The largest body any non-exempt route legitimately takes is 64 KiB (the debug-session form,
# `routes/debug_sessions.py:FORM_MAX_BYTES`), so this leaves ~16x headroom: a ceiling that refuses a
# legitimate request would be a worse bug than the one this module fixes.
DEFAULT_MAX_BODY_BYTES = 1024 * 1024
# The routes that opt out, as (method, path TEMPLATE) — the template, not a concrete path, so the
# list reads like the route decorators it mirrors and `test_body_limit.py` can assert that each one
# still resolves to a real route. A renamed route therefore fails the gate instead of silently
# losing its exemption in prod.
#
# A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE GLOBAL ONE. That is the whole
# admission rule, and it is why `POST /api/diagnostics/events` is absent: its `capped_body` limit is
# 512 KiB, already below this ceiling, so the global cap never fires on it and an exemption would
# only widen what it accepts. Both entries below enforce their own limit on the bytes they actually
# read, which is what makes opting them out safe rather than merely convenient:
# /api/downloads/upload streams in 1 MiB chunks, caps on the real byte count (E7 S5);
# ingesting a local video file, so it is legitimately unbounded here
# /api/debug-sessions/{sid}/screenshots capped_body at SCREENSHOT_MAX_BYTES (10 MiB), and the
# owner check runs as a dependency BEFORE the body is read
EXEMPT_ROUTES: frozenset[tuple[str, str]] = frozenset(
{
("POST", "/api/downloads/upload"),
("POST", "/api/debug-sessions/{sid}/screenshots"),
}
)
# The refusal, in the exact shape `capped_body` raises — `HTTPException(413, {"code": "too_large"})`
# renders as this. Built once; the test asserts it against a real route's response.
TOO_LARGE_BODY = json.dumps({"detail": {"code": "too_large"}}).encode()
_PATH_PARAM = re.compile(r"\{[^/{}]+\}")
def template_to_regex(template: str) -> re.Pattern[str]:
"""`/api/debug-sessions/{sid}/screenshots` → a regex matching one concrete path.
A path parameter matches `[^/]+` — one segment, never across a `/`, so `{sid}` cannot swallow a
deeper path and hand an exemption to a route that was never granted one.
"""
literals = [re.escape(part) for part in _PATH_PARAM.split(template)]
return re.compile("^" + "[^/]+".join(literals) + "$")
def _declared_length(scope: Scope) -> int | None:
"""The `Content-Length` header as an int, or None when absent or unparseable (a chunked body has
none, and a malformed one is not a number we may trust — both fall through to counting)."""
for name, value in scope.get("headers", ()):
if name == b"content-length":
try:
return int(value)
except ValueError:
# silent-ok: a malformed Content-Length is the caller's doing, not a fault of ours,
# and it is about to be answered anyway — returning None drops the request into the
# counting path, which refuses on the real bytes. Logging it would hand an anonymous
# client a free line in our log for every request it cares to malform.
return None
return None
async def _send_too_large(send: Send) -> None:
await send(
{
"type": "http.response.start",
"status": 413,
"headers": [
(b"content-type", b"application/json"),
(b"content-length", str(len(TOO_LARGE_BODY)).encode()),
],
}
)
await send({"type": "http.response.body", "body": TOO_LARGE_BODY})
class BodyLimitMiddleware:
"""Refuses any request body over `limit` bytes, before the application sees it.
Registered second-outermost in `main.py` — inside the access log so a refusal is still logged
with its request id, outside everything else so the session, CORS, setup-gate and auth layers
never run for a request that is already too large.
"""
def __init__(
self,
app: ASGIApp,
*,
limit: int = DEFAULT_MAX_BODY_BYTES,
exempt: frozenset[tuple[str, str]] = EXEMPT_ROUTES,
) -> None:
self.app = app
self.limit = limit
self._exempt = tuple((method, template_to_regex(path)) for method, path in exempt)
def is_exempt(self, method: str, path: str) -> bool:
return any(m == method and rx.match(path) for m, rx in self._exempt)
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http" or self.is_exempt(
scope.get("method", ""), scope.get("path", "")
):
await self.app(scope, receive, send)
return
declared = _declared_length(scope)
if declared is not None and declared > self.limit:
# Refused on the declaration alone: nothing downstream is called, and not one byte of
# the body is read.
await _send_too_large(send)
return
seen = 0
refused = False
started = False
async def counting_receive() -> Message:
nonlocal seen, refused
message = await receive()
if message["type"] != "http.request":
return message
seen += len(message.get("body", b""))
if seen > self.limit:
# The app is blocked on this receive, so it has not answered yet. Telling it the
# client went away is the only in-band way to stop it reading: `request.stream()`
# turns an `http.disconnect` into ClientDisconnect, which unwinds the handler. Our
# own 413 goes out below, once the app has finished unwinding.
refused = True
return {"type": "http.disconnect"}
return message
async def guarded_send(message: Message) -> None:
nonlocal started
if refused and not started:
return # discard the app's answer to the disconnect; the 413 below is the response
started = True
await send(message)
try:
await self.app(scope, counting_receive, guarded_send)
except Exception: # noqa: BLE001 — only swallowed when WE caused it, see below
# ClientDisconnect (or whatever a handler raises on a truncated read) is the expected
# consequence of the disconnect we just faked, and the caller must get the 413 rather
# than a 500. Anything raised when we did NOT refuse is a real failure and is re-raised
# untouched.
if not refused:
raise
if refused and not started:
await _send_too_large(send)