fix(uploads): make both caps answerable, and stop holding a connection

The epic-end deep review found three defects in P1 and four tests that did not
hold what they claimed. They land together because they share the upload route
and its test module; splitting by hunk would have produced commits that do not
individually run green.

* The edge ceiling and the route's own cap are the SAME number at the default
  configuration, so the edge always answers first: a declared Content-Length
  over the cap is refused before a byte is read, and an undeclared body trips
  the middleware on the same chunk the route would have tripped on. The route's
  numbered 422 -- and the drain written for it last week -- are reachable only
  once an admin LOWERS the knob, which is the one shape the browser UAT tested.
  Measured: headroom does not fix this, because the declared-length check fires
  first for any body big enough to matter and the drain pushes the count back
  over the edge for the rest. So the edge's own refusal is made readable
  instead: `errors.tooLarge` localizes `too_large` the way `upload_too_large`
  was already localized, and the dialog refuses an over-cap file BEFORE sending
  it, off the `upload_max_bytes` that GET /api/downloads/usage now reports.
  Measured in Chrome against the API directly: that 413 arrives as `onload`
  with a readable body in 43 ms.

* Reading the body behind the auth dependency put the whole transfer inside the
  request's database session -- the auth SELECT autobegins a transaction, so one
  of the twenty pooled connections sat `idle in transaction`, with nothing to
  time it out, for as long as the client cared to keep sending. That is the
  invariant db.py sizes the pool on and plex/playback.py records an incident
  for. The read transaction now ends before the await. Measured on the dev
  server: an 1818 ms upload shows no idle-in-transaction sample while the bytes
  move, only afterwards, when the handler actually writes.

* The display name moved into a request header, where nothing bounds its length,
  while the column stayed varchar(255) -- so a long enough name was a DataError
  500 after the whole file had already transferred. One `display_text` helper
  now serves all three writers (the metadata PATCH already truncated, the upload
  ingest did not), and the dialog stops at the same number.

* The gate invariant asked `route.dependant.body_params`, which holds only the
  route's OWN callable's parameters: a body declared in a SUB-DEPENDENCY -- the
  upload route's own idiom -- leaves it empty while FastAPI still reads the body
  before auth. It asks `route.body_field` now, which is what FastAPI itself
  consults. The AST registry of body-reading call sites had also never followed
  `capped_stream_to`, so the one route the whole invariant exists for was the
  single site the drift guard could not see.

Four tests that passed over their own mutation are fixed: the cap boundary (a
`>` loosened to `>=` survived the entire suite), the percent-decoding (every
filename the suite sends is ASCII, where quote() is the identity function), the
display-name header (no test at all), and the staging cleanup (deleting the
unlink left the suite green while every refused upload leaked its partial bytes
onto the volume). Every test added or changed here was mutation-checked.
This commit is contained in:
2026-09-22 22:41:22 +02:00
parent 3d7ef9e945
commit efd7b217ca
14 changed files with 428 additions and 67 deletions
+30 -5
View File
@@ -462,6 +462,18 @@ async def spool_upload(
MOVES it into place on success (the unlink is then a no-op), and on any other exit — cap hit,
empty, quota, probe or move error, a client that vanished mid-upload — it is removed."""
cap = sysconfig.get_int(db, "download_upload_max_bytes")
user_id = user.id
# Hand the pooled connection back BEFORE the transfer starts. Resolving `user` and reading the
# cap were both SELECTs, and SQLAlchemy 2.0 autobegins a transaction on the first one — so
# without this the session sits `idle in transaction` holding one of the twenty pooled
# connections for as long as the client cares to keep sending. That is the invariant `db.py`
# sizes the pool on, and `plex/playback.py` records the production incident that taught it;
# reading the body here instead of letting FastAPI spool it (P1) is what put this route at risk
# of breaking it. Auth only reads, so ending the read transaction loses nothing, and the
# handler's first query autobegins a fresh one — one ROLLBACK round-trip against a minute of
# held connection. Everything this dependency still needs is already an int in hand: touching
# an ORM attribute below would reload `user` and check a connection straight back out.
db.rollback()
staging = Path(settings.download_root) / ".staging" / "uploads"
staging.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=str(staging))
@@ -477,9 +489,9 @@ async def spool_upload(
yield SpooledUpload(
path=tmp_path,
size=size,
user_id=user.id,
user_id=user_id,
filename=_header_text(request, UPLOAD_FILENAME_HEADER) or "upload",
display_name=_header_text(request, UPLOAD_DISPLAY_NAME_HEADER) or None,
display_name=service.display_text(_header_text(request, UPLOAD_DISPLAY_NAME_HEADER)),
)
finally:
tmp_path.unlink(missing_ok=True)
@@ -570,7 +582,20 @@ def list_downloads(
@router.get("/usage")
def my_usage(user: User = Depends(require_human), db: Session = Depends(get_db)) -> dict:
return quota.usage(db, user.id)
"""The caller's download quota, plus the per-file upload cap.
The cap rides here rather than in `quota.usage` because it is not a quota value — it is a
`sysconfig` knob about one route — and this is the route layer, where composing the two is the
job. The upload dialog needs it to refuse an over-cap file BEFORE sending it: both server-side
caps answer too late to be pleasant (the edge ceiling refuses the request outright, see
`bodylimit.ROUTE_MAX_BODY_BYTES`), and making the user push 600 MB up the wire to be told the
limit is 500 is a bad answer even when it is a correct one. It is a convenience, not the
boundary: the caps stay where they are and still refuse a client that ignores this.
"""
return {
**quota.usage(db, user.id),
"upload_max_bytes": sysconfig.get_int(db, "download_upload_max_bytes"),
}
# Priority when a source has several jobs (e.g. a done one plus a fresh queued one).
@@ -665,9 +690,9 @@ def update_download_meta(
job = owned_or_404(db, DownloadJob, job_id, user, what="download")
sent = body.model_fields_set
if "display_name" in sent:
job.display_name = ((body.display_name or "").strip())[:255] or None
job.display_name = service.display_text(body.display_name)
if "display_uploader" in sent:
job.display_uploader = ((body.display_uploader or "").strip())[:255] or None
job.display_uploader = service.display_text(body.display_uploader)
if "display_uploader_url" in sent:
job.display_uploader_url = _clean_url(body.display_uploader_url)
if "extra_links" in sent: