siftlode/backend/app/bodylimit.py

"""The global request-body cap (SEC-20).

**The hole.** FastAPI reads a declared body IN FULL before auth or any other dependency runs — the
body parameter has to be parsed to be passed in. So `POST /auth/password-login` drains a 32 MiB
upload from an anonymous caller and only then answers 401. 70 of the 72 body-declaring routes behave
the same way: one unauthenticated client can spend the process's memory and time at will.

**Why a middleware and not `capped_body` on every route.** `routes/_common.py:capped_body` is the
right tool where a route genuinely takes bulk bytes, and the two routes that use it use it
correctly. But putting it on all 70 others would (a) fight the canonical "Request body → a Pydantic
model" row, since capping means taking the body as raw bytes and parsing it by hand, and (b) leave
the NEXT route added uncovered, because nothing would remind its author. A default that has to be
opted into is not a default. This module inverts it: the cap applies to everything, and the few
routes that need more opt OUT, in one list, next to the code that reads it.

**Why raw ASGI and not `@app.middleware("http")`.** Starlette's BaseHTTPMiddleware cannot wrap
`receive`, so it can only read `Content-Length` — which a client may omit entirely (chunked) or
simply understate. The bytes have to be counted as they arrive, which is the same rule `capped_body`
already applies per-route, and counting needs the receive channel.

**Why the refusal is byte-identical to `capped_body`'s.** Two shapes for one fact is how the
`cancelled`/`canceled` bug happened. The 413 body here is built through the same `JSONResponse`
FastAPI renders `HTTPException(413, {"code": "too_large"})` with — not a hand-rolled `json.dumps`,
whose default separators differ by two bytes — and `tests/test_body_limit.py` compares the RAW BYTES
of the two against each other rather than against a literal, so they cannot drift apart. The same
reasoning decides where the middleware is registered (`main.py`): just inside CORS, so this 413
carries the same headers `capped_body`'s does.

**No admin knob, deliberately.** The ceiling is a constant, not a `sysconfig` value: an attacker-
facing limit that an admin can raise from the UI is a limit that can be raised by whoever gets the
admin session, and nobody has asked to tune it. If a self-hoster ever needs a different ceiling it
belongs in `config.py` (env, set before the process starts), not in the database.
"""

import re

from starlette.responses import JSONResponse
from starlette.types import ASGIApp, Message, Receive, Scope, Send

from app import debugsession
from app.config import settings

# 1 MiB. The largest body any route on the default ceiling legitimately takes is 512 KiB (the client
# log sink, `routes/diagnostics.py:BODY_MAX_BYTES`), so this leaves 2x headroom — a ceiling that
# refuses a legitimate request would be a worse bug than the one this module fixes. That ratio is
# the load-bearing fact, so `test_body_limit.py` holds it rather than the figure above.
DEFAULT_MAX_BODY_BYTES = 1024 * 1024

# The routes that need MORE than the default, as (method, path TEMPLATE) → their own ceiling. The
# template, not a concrete path, so the list reads like the route decorators it mirrors and
# `test_body_limit.py` can assert that each one still resolves to a real route: a renamed route
# fails the gate instead of silently losing its allowance in prod.
#
# These are RAISED CEILINGS, NOT EXEMPTIONS, and the difference is the whole point. An exemption
# would mean "this route is counted by something else" — which is only true when the route's own
# limit applies to the bytes as they ARRIVE. That, in turn, is only true of a route with NO DECLARED
# BODY PARAMETER: FastAPI parses and spools a declared body BEFORE the first dependency runs, so a
# route that declares one has already stored the bytes by the time its own cap — or its auth — gets
# a say.
#
# Both listed routes now satisfy that. The screenshot route always did (`capped_body` streams it,
# and the owner check resolves first). The upload route did NOT until P1: it declared
# `file: UploadFile = File(...)`, so an anonymous caller could make the process write the whole
# per-file allowance to temp before `require_human` refused them — this middleware bounded that
# number but could not stop the writing. It now reads its own body from a dependency declared after
# the auth one, so its in-handler cap measures bytes as they arrive, like everything else here.
#
# That rule is no longer only written here: `test_body_limit.py` parses the routes and fails the
# gate if any route in this dict declares a body parameter. A comment could not have stopped the
# defect it describes; the test can.
#
# A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE DEFAULT. That is why
# `POST /api/diagnostics/events` is absent: its `capped_body` limit is 512 KiB, already below the
# default, so listing it could only widen what it accepts.
ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = {
    # Ingesting a local file. The body IS the file since P1 — no multipart envelope to allow for —
    # so this ceiling and the route's own cap count exactly the same bytes. `sysconfig`'s
    # `download_upload_max_bytes` is bounded by this same constant (see sysconfig.SPECS), so an
    # admin can only ever LOWER the route's cap below the edge, never raise it past one.
    #
    # WHICH LAYER ANSWERS, measured rather than assumed (deep-review round 1):
    #   * knob LOWERED (an override below the default) — the route's cap is the smaller number, so
    #     `capped_stream_to` trips first and answers 422 `upload_too_large` carrying the admin's own
    #     figure, after draining the rest of the body so the client can read it.
    #   * knob at the DEFAULT — the two numbers are EQUAL, and the edge always wins: a declared
    #     Content-Length over the cap is refused here before a byte is read, and an undeclared body
    #     trips this counter on the same chunk the route would have tripped on. The route's 422 is
    #     unreachable in that configuration, and no amount of headroom would change it — the
    #     declared-length check fires first for any body big enough to matter, and the route's drain
    #     pushes the count past the edge for the rest.
    #
    # So this 413 is a refusal a real user meets on a default install, not a theoretical edge, and
    # it has to be as readable as the route's own: the client localizes `too_large` (`errors.tooLarge`)
    # exactly as it does `upload_too_large`, and the upload dialog refuses an over-cap file before
    # sending it at all, off the `upload_max_bytes` that `GET /api/downloads/usage` reports.
    ("POST", "/api/downloads/upload"): settings.download_upload_max_bytes,
    ("POST", "/api/debug-sessions/{sid}/screenshots"): debugsession.SCREENSHOT_MAX_BYTES,
}

# The refusal, byte-for-byte what `capped_body`'s `HTTPException(413, {"code": "too_large"})` renders
# as — built through the same JSONResponse so the separators match too, and asserted against a real
# route's raw bytes in `test_body_limit.py` rather than against a literal here.
TOO_LARGE_BODY = JSONResponse({"detail": {"code": "too_large"}}, status_code=413).body

_PATH_PARAM = re.compile(r"\{[^/{}]+\}")


def template_to_regex(template: str) -> re.Pattern[str]:
    """`/api/debug-sessions/{sid}/screenshots` → a regex matching one concrete path.

    A path parameter matches `[^/]+` — one segment, never across a `/`, so `{sid}` cannot swallow a
    deeper path and hand an exemption to a route that was never granted one.
    """
    literals = [re.escape(part) for part in _PATH_PARAM.split(template)]
    return re.compile("^" + "[^/]+".join(literals) + "$")


def _declared_length(scope: Scope) -> int | None:
    """The `Content-Length` header as an int, or None when absent or unparseable (a chunked body has
    none, and a malformed one is not a number we may trust — both fall through to counting)."""
    for name, value in scope.get("headers", ()):
        if name == b"content-length":
            try:
                return int(value)
            except ValueError:
                # silent-ok: a malformed Content-Length is the caller's doing, not a fault of ours,
                # and it is about to be answered anyway — returning None drops the request into the
                # counting path, which refuses on the real bytes. Logging it would hand an anonymous
                # client a free line in our log for every request it cares to malform.
                return None
    return None


async def _send_too_large(send: Send) -> None:
    await send(
        {
            "type": "http.response.start",
            "status": 413,
            "headers": [
                (b"content-type", b"application/json"),
                (b"content-length", str(len(TOO_LARGE_BODY)).encode()),
            ],
        }
    )
    await send({"type": "http.response.body", "body": TOO_LARGE_BODY})


class BodyLimitMiddleware:
    """Refuses any request body over its route's ceiling, before the application sees it.

    Registered just inside CORS in `main.py` — so the 413 is decorated like every other response
    and a browser can actually read it — but outside the session, the setup gate and every auth
    dependency, none of which run for a request that is already too large.

    Every request gets a finite ceiling: `ROUTE_MAX_BODY_BYTES` raises it for the two routes that
    genuinely take bulk bytes, and there is no path that skips counting altogether.
    """

    def __init__(
        self,
        app: ASGIApp,
        *,
        limit: int = DEFAULT_MAX_BODY_BYTES,
        routes: dict[tuple[str, str], int] | None = None,
    ) -> None:
        self.app = app
        self.limit = limit
        self._routes = tuple(
            (method, template_to_regex(path), cap)
            for (method, path), cap in (ROUTE_MAX_BODY_BYTES if routes is None else routes).items()
        )

    def limit_for(self, method: str, path: str) -> int:
        """This request's ceiling: the route's own if it has one, else the global default. Every
        request gets a finite number — there is no path through here that is simply not counted."""
        for m, rx, cap in self._routes:
            if m == method and rx.match(path):
                return cap
        return self.limit

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        if scope["type"] != "http":
            await self.app(scope, receive, send)
            return
        limit = self.limit_for(scope.get("method", ""), scope.get("path", ""))

        declared = _declared_length(scope)
        if declared is not None and declared > limit:
            # Refused on the declaration alone: nothing downstream is called, and not one byte of
            # the body is read.
            await _send_too_large(send)
            return

        seen = 0
        refused = False
        started = False

        async def counting_receive() -> Message:
            nonlocal seen, refused
            message = await receive()
            if message["type"] != "http.request":
                return message
            seen += len(message.get("body", b""))
            if seen > limit:
                # The app is blocked on this receive, so it has not answered yet. Telling it the
                # client went away is the only in-band way to stop it reading: `request.stream()`
                # turns an `http.disconnect` into ClientDisconnect, which unwinds the handler. Our
                # own 413 goes out below, once the app has finished unwinding.
                refused = True
                return {"type": "http.disconnect"}
            return message

        async def guarded_send(message: Message) -> None:
            nonlocal started
            if refused and not started:
                return  # discard the app's answer to the disconnect; the 413 below is the response
            started = True
            await send(message)

        try:
            await self.app(scope, counting_receive, guarded_send)
        except Exception:  # only swallowed when WE caused it, see below
            # ClientDisconnect (or whatever a handler raises on a truncated read) is the expected
            # consequence of the disconnect we just faked, and the caller must get the 413 rather
            # than a 500. Anything raised when we did NOT refuse is a real failure and is re-raised
            # untouched.
            if not refused:
                raise

        if refused and not started:
            await _send_too_large(send)