Found in the browser UAT, not by the gate. Moving the cap onto the arriving
bytes meant it fires while the client is still sending, and answering + closing
mid-upload is what XHR reports as a network failure: the upload dialog said
"the server couldn't be reached" while the access log showed a correct 422
carrying the limit. A refusal the client cannot read is an outage as far as the
user is concerned.
`capped_stream_to` now reads the remainder to nowhere before raising, so the
client finishes sending and reads the real answer -- re-measured in the
browser, which now names the limit. Nothing past the cap is written, and the
request is still bounded by bodylimit's ceiling, so the cost is bandwidth an
AUTHENTICATED caller was sending anyway. The anonymous path is untouched:
auth still resolves before a byte is read.
The regression test sits on the helper, not the route, and the reason is worth
keeping: the route-level version was written first and PASSED with the drain
deleted. httpx's ASGI transport pumps the request body whether or not the app
reads it, so "drained" and "abandoned" are indistinguishable through
TestClient. Its own mutation check is what exposed it.
`POST /api/downloads/upload` declared `file: UploadFile = File(...)`, and
FastAPI parses and spools a declared body before the first dependency runs --
`require_human` included. An unauthenticated caller could therefore make the
process write the whole per-file allowance (500 MiB by default) to temp, once
per request. SEC-20's ceiling bounded that number but could not stop the
writing, which is why E27 parked it rather than closing it.
The route now takes no body parameter. `spool_upload` names `require_human`
first and only then streams `request.stream()` to the staging temp file, so a
stranger is refused before a byte moves -- the shape the screenshot route has
used since E16. Filename and display name travel as percent-encoded headers
(latin-1 on the wire, so a Hungarian filename needs the encoding) instead of
multipart parts.
`capped_stream_to` joins `capped_body` in routes/_common.py: the same counting
rule, written to a file descriptor rather than accumulated, because holding
500 MiB would trade a disk problem for a memory one. Its refusal is the
caller's, so the upload route keeps answering 422 `upload_too_large` with the
admin knob's own number.
With no envelope to allow for, the edge is now exactly that knob, and
MULTIPART_ENVELOPE_ALLOWANCE is gone.
The invariant stops being prose: test_body_limit.py asks FastAPI itself
(`route.dependant.body_params`) and fails the gate if any route with a raised
ceiling declares a body. A comment could not have prevented the defect it
described; that test would have.
Both new tests were seen red for the right reason -- before the fix the
anonymous probe got its 401 only after the server had read 4236 bytes of a
multipart body, and re-declaring the File parameter turns the invariant test
red naming the route and the parameter.
A Facebook or Instagram reel extracts with no duration, so the asset kept a
NULL duration_s although the file plays for minutes, and the Library row had
no trim action. The finalize probe now returns the duration with the codecs
(the file wins, info fills a gap), and runs before the sidecars so the .nfo
runtime gets it too.
Deep-review round 1 on bug/download-mediakind-probe found a real HIGH the prior
review missed:
- resolve_media_facts used a per-field `probe or info` fallback, so a probe that
RAN and deliberately found no video track (cover-art-only audio: attached_pic
skipped; a still image: format-guarded) had its None video overridden by
yt-dlp's stale vcodec. An audio-mode download from a progressive source (info
carries the source's video codec, embed-thumbnail on) was then classified VIDEO
-> a Play button + a broken in-app <video> with no video track. Only the
download path regressed (upload calls probe_media directly).
Fix: gate precedence on whether ffprobe RAN, not per-field. If the probe read
the file, its codec verdict wins outright (a None means 'no such track'); only a
full ffprobe failure falls back to info. Dimensions still gap-fill from info.
Refactored the parsing into _facts_from_probe so resolve_media_facts can see the
raw probe-ran signal; probe_media keeps its 5-tuple (upload + TestProbeClassifier
unchanged).
- tests: the cover-art-audio and still-image resurrection cases, a differing
populated-info precedence case (the prior-flagged gap), and the ffprobe-failed
info fallback — now stubbing run_ffprobe_json so the real parsing + precedence
run together.
Backend gate green (ruff + full pytest).
A downloaded video was silently demoted to a download-only "file" (no
in-app play/edit) whenever yt-dlp's info dict omitted vcodec/acodec — which
it does for Facebook and other sources. Finalize derived media_kind from
those nulls (`derive(None, None)` -> file), so a genuine H.264/AAC MP4
became non-playable in the library.
Fix: probe the finished file on disk (the ground truth) at finalize and
derive from that, falling back to the info dict only for a field the probe
couldn't read. This is the same ffprobe the upload ingest already used, now
extracted to a shared downloads/probe.py so the download worker and the
upload path share one classifier (and can't drift). Also backfills the
stored vcodec/acodec/width/height from the real file.
Tests: TestResolveMediaFacts pins probe-wins-over-info (incl. the Facebook
no-codec case and yt-dlp's literal "none"); upload's probe tests repoint to
the shared module.
Deep-review remediation for the E7 upload path, all findings adversarially verified:
- Serve every library file with a Content-Type chosen from the ffprobe-verified media_kind
+ a strict container allow-list (storage.served_media_type), never guessed from the
user-controlled file extension. Closes a stored-XSS vector where a media/HTML polyglot
uploaded as "evil.html" was served inline as text/html on the login-free /watch surface.
Applied to both the authenticated /file and the public watch_file endpoints.
- Classify uploaded still images (jpg/png/gif/webp/…) as media_kind='file' so they are
force-downloaded and don't get a broken in-app video player.
- Roll back the on-disk side of an upload when the ingest transaction fails after place_file
(unlink the placed file + its deterministic poster path), so a failed commit can't orphan
bytes in the GC-skipped .uploads tree.
- Floor download_upload_max_bytes at min=1 so a stored 0 can't silently disable the per-file cap.
- Dedupe the ffprobe-JSON invocation into edit.run_ffprobe_json (shared by probe_codecs and
upload._probe) and the upload extension into storage._clean_upload_ext; drop the now-unused
parse_probe_streams.
Tests: public + authenticated force-download and the polyglot-not-served-as-HTML guard, the
real _probe classifier (image/cover-art/dims/duration), the ingest rollback/cleanup path, the
max_jobs branch, the cap-floor invariant, and run_ffprobe_json error handling.
Add the non-yt-dlp ingest path so a user-uploaded file joins the same download
library and inherits its function set. Any file type is accepted: media
(video/audio) gets the full set (metadata, share, play/edit), a generic file
(pdf/zip/docx/…) is download-only.
- MediaAsset.media_kind ("video"|"audio"|"file") — the stored discriminator that
gates library actions. Set at every finalize (download probe, editor carries the
source's kind, upload probe); consolidates the ad-hoc audio-detect that lived only
in links.py into one invariant (downloads.mediakind.derive). Migration 0076 adds
the column + backfills existing rows.
- downloads/upload.py: ingest_upload probes the file (ffprobe → codecs/dims/duration),
places it in a per-user .uploads tree, and creates an already-ready asset + done job
(the same cache-hit shortcut enqueue uses). Uploads never expire (irreplaceable).
- POST /api/downloads/upload: streaming multipart with the per-file cap
(download_upload_max_bytes, default 500 MiB) enforced on actual bytes read; quota
charges the known size up front (quota.check_upload).
- Security: a generic file is always served force-download (attachment +
application/octet-stream) on both the in-app and public endpoints, so uploaded
HTML/SVG can't execute inline. Edit + redownload reject non-media / upload sources.
- python-multipart added (first multipart endpoint in the app).
peter