peter
f5018bdcbc
`POST /api/downloads/upload` declared `file: UploadFile = File(...)`, and FastAPI parses and spools a declared body before the first dependency runs -- `require_human` included. An unauthenticated caller could therefore make the process write the whole per-file allowance (500 MiB by default) to temp, once per request. SEC-20's ceiling bounded that number but could not stop the writing, which is why E27 parked it rather than closing it. The route now takes no body parameter. `spool_upload` names `require_human` first and only then streams `request.stream()` to the staging temp file, so a stranger is refused before a byte moves -- the shape the screenshot route has used since E16. Filename and display name travel as percent-encoded headers (latin-1 on the wire, so a Hungarian filename needs the encoding) instead of multipart parts. `capped_stream_to` joins `capped_body` in routes/_common.py: the same counting rule, written to a file descriptor rather than accumulated, because holding 500 MiB would trade a disk problem for a memory one. Its refusal is the caller's, so the upload route keeps answering 422 `upload_too_large` with the admin knob's own number. With no envelope to allow for, the edge is now exactly that knob, and MULTIPART_ENVELOPE_ALLOWANCE is gone. The invariant stops being prose: test_body_limit.py asks FastAPI itself (`route.dependant.body_params`) and fails the gate if any route with a raised ceiling declares a body. A comment could not have prevented the defect it described; that test would have. Both new tests were seen red for the right reason -- before the fix the anonymous probe got its 401 only after the server had read 4236 bytes of a multipart body, and re-declaring the File parameter turns the invariant test red naming the route and the parameter.
Backend conventions
The rules below are already followed across backend/app — they lived only in a few module
docstrings (demo.py, userscope.py, routes/_common.py) until this file collected them (E3 S3,
B-core-20). When you add code, match these so the next reader doesn't have to guess "which pattern do
I copy".
Module layering
Three layers, by who runs the code — not by feature:
-
routes/…— request-only. A route module holds HTTP concerns: auth dependencies, request/ response models, status-code mapping, serialization. It owns logic that runs only on a request. Business logic that a request happens to trigger does not live here if anything else runs it too. -
plex/,downloads/,sync/,issues/— domain logic shared by more than one caller. Anything the scheduler or the worker runs (not just a request) lives in one of these domain packages, so the route, the worker, and the scheduler share one definition. Examples from this codebase:downloads/service.pyowns enqueue + the job state transitions (routes call it);plex/sync.pyis run by both the admin route and the scheduler;downloads/pipeline.pyis the worker's child pipeline;issues/bundle.pyis the one definition of a bug report's files, for the zip export and the tracker. -
Flat
app/*.pycore modules — cross-cutting primitives.auth,sysconfig,quota,audit,notifications,userscope,realtime,ratelimit,progress,logs(the one logging configuration),logsink(itslog_eventstable),diagnostics(a user's consented log level) anddebugsession(a recording that tags its user's lines, E16), … are single-file primitives many layers import. They depend on nothing above them (a core module must never import aroutes/…module).
Extraction trigger: the SECOND consumer. Keep new logic inline in its first caller. The moment a second caller needs it (a worker as well as a route; a scheduler as well as a route), extract it down a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't pre-abstract for a single caller; don't leave shared logic duplicated once there are two.
Transaction ownership
This follows CLAUDE.md's canonical rule verbatim — "the handler/job-wrapper commits; helpers flush
(only quota owns its session)" — so read it that way, not as an exception to it. The commit belongs
to the operation owner; anything called inside another transaction only stages its change
(db.add / flush) and lets the owner commit, so the whole operation is one transaction.
audit.record() is the reference: it adds the audit row and the caller commits it together with the
mutation it records, so an audit entry can never exist without its change (or vice-versa).
- The operation owner is usually the route handler. A
service.*operation the handler delegates a whole transition to (e.g.service.cancel) is that owner and commits its one transition — it isn't a mid-operation helper. A helper that owns its OWN short-lived session (the worker'sjobdbwrites, the same carve-out CLAUDE.md grantsquota) commits that session. Everything else flushes. SessionLocalisexpire_on_commit=False: after a rawUPDATE+ commit, an ORM object you already loaded is stale. Read the fresh value from the row (RETURNING, or a re-SELECT), not the ORM attribute — seeservice.decr_ref_count.
Shared counters and status: do it atomically at the DB
- Never read-modify-write a shared counter in Python (
x.count = x.count + 1). Concurrent sessions lose an update. Use a DB-atomic statement —col = col + 1, orgreatest(col - 1, 0)withRETURNINGso the caller branches on the post-value under the row lock (service.incr_ref_count/decr_ref_count). - Download job status is a
JobStatusStrEnum (downloads/status.py), not bare strings. TheHOLDINGset defines which states hold aMediaAsset(so release accounting stays correct);STOPPED_BY_ROUTEis the route-initiated stop set. A worker child's terminal write goes throughservice.finalize_if_running(an atomicUPDATE … WHERE status='running') so a finishing child can't resurrect a job a route just stopped.
Config: one home per kind
app.config.settings— boot/env config (DB URL, download root, worker concurrency): read once at startup, not user-tunable at runtime.sysconfig.SPECS— the runtime admin knobs (DB-override-or-default). A new admin-tunable value goes inSPECS; don't invent a fourth store. (state.py'smaintenance_batchpredates this and is slated to fold intosysconfig— do not add more like it.)
The gate
Everything below must be green before a change is done (run via siftlode check):
ruff (backend) · tsc ×3 + eslint + prettier + knip (frontend) · vitest · pytest. The
pytest lane runs in backend/docker-compose.test.yml — an ephemeral Postgres + the backend tree
bind-mounted — so the DB-backed integration tests (the db fixture) actually run; SIFTLODE_TEST_ REQUIRE_DB=1 there turns a would-be silent skip into a hard failure. See docs/development.md.