fix(uploads): make both caps answerable, and stop holding a connection

The epic-end deep review found three defects in P1 and four tests that did not
hold what they claimed. They land together because they share the upload route
and its test module; splitting by hunk would have produced commits that do not
individually run green.

* The edge ceiling and the route's own cap are the SAME number at the default
  configuration, so the edge always answers first: a declared Content-Length
  over the cap is refused before a byte is read, and an undeclared body trips
  the middleware on the same chunk the route would have tripped on. The route's
  numbered 422 -- and the drain written for it last week -- are reachable only
  once an admin LOWERS the knob, which is the one shape the browser UAT tested.
  Measured: headroom does not fix this, because the declared-length check fires
  first for any body big enough to matter and the drain pushes the count back
  over the edge for the rest. So the edge's own refusal is made readable
  instead: `errors.tooLarge` localizes `too_large` the way `upload_too_large`
  was already localized, and the dialog refuses an over-cap file BEFORE sending
  it, off the `upload_max_bytes` that GET /api/downloads/usage now reports.
  Measured in Chrome against the API directly: that 413 arrives as `onload`
  with a readable body in 43 ms.

* Reading the body behind the auth dependency put the whole transfer inside the
  request's database session -- the auth SELECT autobegins a transaction, so one
  of the twenty pooled connections sat `idle in transaction`, with nothing to
  time it out, for as long as the client cared to keep sending. That is the
  invariant db.py sizes the pool on and plex/playback.py records an incident
  for. The read transaction now ends before the await. Measured on the dev
  server: an 1818 ms upload shows no idle-in-transaction sample while the bytes
  move, only afterwards, when the handler actually writes.

* The display name moved into a request header, where nothing bounds its length,
  while the column stayed varchar(255) -- so a long enough name was a DataError
  500 after the whole file had already transferred. One `display_text` helper
  now serves all three writers (the metadata PATCH already truncated, the upload
  ingest did not), and the dialog stops at the same number.

* The gate invariant asked `route.dependant.body_params`, which holds only the
  route's OWN callable's parameters: a body declared in a SUB-DEPENDENCY -- the
  upload route's own idiom -- leaves it empty while FastAPI still reads the body
  before auth. It asks `route.body_field` now, which is what FastAPI itself
  consults. The AST registry of body-reading call sites had also never followed
  `capped_stream_to`, so the one route the whole invariant exists for was the
  single site the drift guard could not see.

Four tests that passed over their own mutation are fixed: the cap boundary (a
`>` loosened to `>=` survived the entire suite), the percent-decoding (every
filename the suite sends is ASCII, where quote() is the identity function), the
display-name header (no test at all), and the staging cleanup (deleting the
unlink left the suite green while every refused upload leaked its partial bytes
onto the volume). Every test added or changed here was mutation-checked.
This commit is contained in:
2026-09-22 22:41:22 +02:00
parent 3d7ef9e945
commit efd7b217ca
14 changed files with 428 additions and 67 deletions
@@ -11,6 +11,9 @@ import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
const h = vi.hoisted(() => ({
uploadDownload: vi.fn(async (_f: File, _o: unknown) => ({ id: 42 })),
// The dialog reads the per-file upload cap so it can refuse an over-cap file before spending the
// user's bandwidth on it. Generous by default; the cap test below lowers it.
downloadUsage: vi.fn(async () => ({ upload_max_bytes: 1_000_000 })),
notify: vi.fn(),
}));
@@ -18,7 +21,9 @@ vi.mock("react-i18next", async (importOriginal) => {
const actual = await importOriginal<typeof import("react-i18next")>();
return { ...actual, useTranslation: () => ({ t: (key: string) => key }) };
});
vi.mock("../lib/api", () => ({ api: { uploadDownload: h.uploadDownload } }));
vi.mock("../lib/api", () => ({
api: { uploadDownload: h.uploadDownload, downloadUsage: h.downloadUsage },
}));
vi.mock("../lib/downloads", () => ({ invalidateDownloads: vi.fn() }));
// The rest of the module stays real: the dialog is a ui/Modal, which hosts its own toasts.
vi.mock("../lib/notifications", async (importOriginal) => ({
@@ -44,6 +49,8 @@ const fileInput = () => document.querySelector('input[type="file"]') as HTMLInpu
beforeEach(() => {
h.uploadDownload.mockClear();
h.notify.mockClear();
h.downloadUsage.mockClear();
h.downloadUsage.mockResolvedValue({ upload_max_bytes: 1_000_000 });
});
afterEach(cleanup);
@@ -76,4 +83,31 @@ describe("UploadDialog", () => {
const btn = screen.getByText("downloads.upload.submit").closest("button")!;
expect(btn.disabled).toBe(true);
});
it("refuses an over-cap file here, naming the limit, instead of sending it", async () => {
// Both server-side caps answer only after the transfer — and on a default install the one that
// answers is the edge ceiling, whose refusal carries no number at all. So the dialog checks the
// size itself: the user learns the limit before a byte moves.
h.downloadUsage.mockResolvedValue({ upload_max_bytes: 4 });
wrap();
await waitFor(() => expect(h.downloadUsage).toHaveBeenCalled());
fireEvent.change(fileInput(), {
target: { files: [new File(["far too many bytes"], "big.bin")] },
});
await waitFor(() => expect(screen.getByText("downloads.errors.upload.tooLarge")).toBeTruthy());
const btn = screen.getByText("downloads.upload.submit").closest("button")!;
expect(btn.disabled).toBe(true);
fireEvent.click(btn);
expect(h.uploadDownload).not.toHaveBeenCalled();
});
it("sends a file that fits, so the check above is not simply refusing everything", async () => {
h.downloadUsage.mockResolvedValue({ upload_max_bytes: 1_000_000 });
wrap();
await waitFor(() => expect(h.downloadUsage).toHaveBeenCalled());
fireEvent.change(fileInput(), { target: { files: [new File(["small"], "ok.bin")] } });
fireEvent.click(screen.getByText("downloads.upload.submit"));
await waitFor(() => expect(h.uploadDownload).toHaveBeenCalledTimes(1));
});
});
+21 -3
View File
@@ -1,10 +1,11 @@
import { useEffect, useRef, useState } from "react";
import { useTranslation } from "react-i18next";
import { useQueryClient } from "@tanstack/react-query";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { UploadCloud, File as FileIcon } from "lucide-react";
import Modal from "./ui/Modal";
import { Button } from "./ui/Button";
import { api } from "../lib/api";
import { qk } from "../lib/queryKeys";
import { isAborted } from "../lib/api/core";
import { notifyActionError } from "../lib/actionErrors";
import { invalidateDownloads } from "../lib/downloads";
@@ -31,6 +32,14 @@ export default function UploadDialog({ onClose }: { onClose: () => void }) {
// plain boolean flickers. Count enter vs leave and only clear the highlight at depth 0.
const dragDepth = useRef(0);
const busy = pct !== null;
// The per-file cap, so an over-cap file is refused HERE instead of after the user has pushed it up
// the wire. Both server-side caps are still the boundary — this is only the pleasant half of it.
// On a default install the one that answers is the edge ceiling (`bodylimit`), whose refusal
// carries no number, so without this the user would be told "too large" and not what the limit is.
// A cap that has not loaded yet blocks nothing — the server still refuses, honestly.
const cap = useQuery({ queryKey: qk.downloadUsage(), queryFn: api.downloadUsage }).data
?.upload_max_bytes;
const tooBig = !!file && !!cap && file.size > cap;
// Closing the dialog mid-upload (X / ESC / backdrop / browser Back all unmount us) must abort the
// in-flight XHR — otherwise it keeps streaming in the background and still toasts success. Only the
@@ -40,7 +49,7 @@ export default function UploadDialog({ onClose }: { onClose: () => void }) {
const pick = () => inputRef.current?.click();
const submit = async () => {
if (!file || busy) return;
if (!file || busy || tooBig) return;
const ac = new AbortController();
abortRef.current = ac;
setPct(0);
@@ -133,6 +142,12 @@ export default function UploadDialog({ onClose }: { onClose: () => void }) {
}}
/>
{tooBig && (
<p className="text-xs text-danger mt-2" role="alert">
{t("downloads.errors.upload.tooLarge", { size: formatBytes(cap ?? 0) })}
</p>
)}
<label className="block text-sm font-medium mt-4 mb-1">
{t("downloads.upload.nameLabel")}
</label>
@@ -141,6 +156,9 @@ export default function UploadDialog({ onClose }: { onClose: () => void }) {
onChange={(e) => setName(e.target.value)}
placeholder={t("downloads.upload.namePlaceholder")}
disabled={busy}
// The column behind it is varchar(255) and the name rides in a REQUEST HEADER, where nothing
// else bounds it; the server truncates to the same number (downloads.service.display_text).
maxLength={255}
className={`${inputCls} mb-5`}
/>
@@ -175,7 +193,7 @@ export default function UploadDialog({ onClose }: { onClose: () => void }) {
>
{t("downloads.actions.cancel")}
</Button>
<Button variant="primary" size="sm" onClick={submit} disabled={!file || busy}>
<Button variant="primary" size="sm" onClick={submit} disabled={!file || busy || tooBig}>
{t("downloads.upload.submit")}
</Button>
</div>
+1
View File
@@ -3,6 +3,7 @@
"generic": "The server couldn't carry out that action. Please try again.",
"demo": "This isn't available in the demo account.",
"validation": "Some of the values you entered weren't valid. Please check them and try again.",
"tooLarge": "That was too large for the server to accept.",
"server": "Server error",
"syncCooldown": "You've synced a few times just now — give it a few minutes. The YouTube budget is shared with everyone on this instance, and background syncing keeps running either way.",
"notFound": {
+1
View File
@@ -3,6 +3,7 @@
"generic": "A szerver nem tudta végrehajtani a műveletet. Próbáld újra.",
"demo": "Ez a bemutató fiókban nem érhető el.",
"validation": "Néhány megadott érték érvénytelen volt. Ellenőrizd, majd próbáld újra.",
"tooLarge": "Ez túl nagy, a szerver nem fogadta el.",
"server": "Szerverhiba",
"syncCooldown": "Az imént többször is szinkronizáltál — várj pár percet. A YouTube-keret közös az instance összes felhasználójával, a háttérszinkron pedig így is fut tovább.",
"notFound": {
+47 -13
View File
@@ -74,11 +74,14 @@ beforeEach(() => {
vi.stubGlobal("XMLHttpRequest", FakeXHR as unknown as typeof XMLHttpRequest);
});
const form = () => new FormData();
// The upload body is a Blob and only a Blob: `FormData` was accepted until P1 and is gone, because
// a multipart part is a DECLARED body that FastAPI spools before the route's auth dependency runs —
// and nothing on the backend can parse one any more (`python-multipart` was dropped with it).
const body = () => new Blob(["x"]);
describe("uploadReq", () => {
it("resolves with the parsed JSON on a 2xx", async () => {
const p = uploadReq<{ id: number }>("/u", form());
const p = uploadReq<{ id: number }>("/u", body());
const x = FakeXHR.last;
x.status = 200;
x.responseText = '{"id":7}';
@@ -93,14 +96,14 @@ describe("uploadReq", () => {
it("rejects immediately if the signal is already aborted (never sends)", async () => {
const ac = new AbortController();
ac.abort();
const p = uploadReq("/u", form(), { signal: ac.signal });
const p = uploadReq("/u", body(), { signal: ac.signal });
await expect(p).rejects.toSatisfy(isAborted);
expect(FakeXHR.last?.sent).not.toBe(true);
});
it("aborts the XHR and rejects when the signal fires", async () => {
const ac = new AbortController();
const p = uploadReq("/u", form(), { signal: ac.signal });
const p = uploadReq("/u", body(), { signal: ac.signal });
ac.abort(); // → our abort listener calls xhr.abort() → onabort → reject
await expect(p).rejects.toSatisfy(isAborted);
});
@@ -108,7 +111,7 @@ describe("uploadReq", () => {
it("calls the unauthorized handler on 401", async () => {
const onUnauth = vi.fn();
setUnauthorizedHandler(onUnauth);
const p = uploadReq("/u", form());
const p = uploadReq("/u", body());
const x = FakeXHR.last;
x.status = 401;
x.onload!();
@@ -118,7 +121,7 @@ describe("uploadReq", () => {
});
it("localizes a structured 422 upload_too_large and surfaces the error", async () => {
const p = uploadReq("/u", form());
const p = uploadReq("/u", body());
const x = FakeXHR.last;
x.status = 422;
x.responseText = '{"detail":{"code":"upload_too_large","limit":5}}';
@@ -128,8 +131,39 @@ describe("uploadReq", () => {
expect(reportError).toHaveBeenCalledWith("downloads.errors.upload.tooLarge");
});
it("localizes the edge ceiling's 413 too_large, which carries no limit", async () => {
// On a DEFAULT install this — not the route's numbered 422 above — is what an over-cap upload
// gets: the ceiling and the route's cap are the same figure and the edge answers first. It fell
// through to the generic "couldn't carry out that action" until the branch existed.
const p = uploadReq("/u", body());
const x = FakeXHR.last;
x.status = 413;
x.responseText = '{"detail":{"code":"too_large"}}';
x.onload!();
const err = (await p.catch((e: unknown) => e)) as HttpError;
expect(err).toBeInstanceOf(HttpError);
expect(err.detail).toBe("errors.tooLarge");
});
it("lets its own identity headers win over a caller's", () => {
// Who a request is from is this layer's to state, not its caller's. Merged the other way round,
// a caller's header of the same name would silently re-attribute the upload — to another
// account, or (here) into someone else's debug session recording.
setDebugSession("0123456789abcdef");
try {
void uploadReq("/u", body(), {
headers: { "X-Debug-Session": "deadbeefdeadbeef", "X-Upload-Filename": "a.bin" },
});
expect(FakeXHR.last.headers["X-Debug-Session"]).toBe("0123456789abcdef");
// ...while a header that is genuinely the caller's still rides.
expect(FakeXHR.last.headers["X-Upload-Filename"]).toBe("a.bin");
} finally {
setDebugSession(null);
}
});
it("offers no string detail as the reason on the upload path either", async () => {
const p = uploadReq("/u", form());
const p = uploadReq("/u", body());
const x = FakeXHR.last;
x.status = 405;
x.responseText = '{"detail":"Method Not Allowed"}';
@@ -140,7 +174,7 @@ describe("uploadReq", () => {
});
it("reports a connectivity failure on a network error", async () => {
const p = uploadReq("/u", form());
const p = uploadReq("/u", body());
FakeXHR.last.onerror!();
await expect(p).rejects.toMatchObject({ status: 0 });
// The soft self-clearing "connection lost" toast (via notify), not a blocking modal.
@@ -329,14 +363,14 @@ describe("req — what it has already put in front of the user", () => {
});
it("marks what the upload path reported the same way", async () => {
const dead = uploadReq("/u", form());
const dead = uploadReq("/u", body());
FakeXHR.last.onerror!();
expect(alreadyReported(await dead.catch((e: unknown) => e))).toBe(false);
const refused = uploadReq("/u", form());
const refused = uploadReq("/u", body());
FakeXHR.last.status = 500;
FakeXHR.last.onload!();
expect(alreadyReported(await refused.catch((e: unknown) => e))).toBe(true);
const gone = uploadReq("/u", form());
const gone = uploadReq("/u", body());
FakeXHR.last.status = 404;
FakeXHR.last.onload!();
expect(alreadyReported(await gone.catch((e: unknown) => e))).toBe(false);
@@ -350,7 +384,7 @@ describe("req — what it has already put in front of the user", () => {
await req("/api/x", { method: "POST" }).catch(() => {}); // a notice is up
getActiveToasts.mockReturnValue([{ id: 9 }]);
remove.mockClear();
const p = uploadReq("/u", form());
const p = uploadReq("/u", body());
FakeXHR.last.status = 502;
FakeXHR.last.onload!();
await expect(p).rejects.toMatchObject({ status: 502 });
@@ -378,7 +412,7 @@ describe("the debug session a tab records into", () => {
expect(sentHeaders().get("X-Debug-Session")).toBe("0123456789abcdef");
beacon("/api/y", {});
expect(sentHeaders().get("X-Debug-Session")).toBe("0123456789abcdef");
void uploadReq("/u", form());
void uploadReq("/u", body());
expect(FakeXHR.last.headers["X-Debug-Session"]).toBe("0123456789abcdef");
} finally {
setDebugSession(null);
+19 -8
View File
@@ -119,6 +119,13 @@ function localizeDetail(d: unknown): string {
if (o?.code === "upload_too_large")
return t("downloads.errors.upload.tooLarge", { size: formatBytes(o.limit) });
if (o?.code === "upload_empty") return t("downloads.errors.upload.empty");
// The EDGE ceiling's refusal (`bodylimit.TOO_LARGE_BODY`), shared by every route and carrying no
// number. Not a theoretical case: on a DEFAULT install it is what an over-cap upload actually
// gets, because the ceiling and the route's own cap are the same figure and the edge answers
// first — the route's numbered `upload_too_large` above is the answer only once an admin has
// LOWERED the knob (see bodylimit.ROUTE_MAX_BODY_BYTES). Without this branch that refusal fell
// through to the generic "couldn't carry out that action", the one message no retry can fix.
if (o?.code === "too_large") return t("errors.tooLarge");
// SEC-12: a download URL the server refuses to fetch. The reasons are deliberately distinct —
// "we could not look that host up" is a different thing to tell someone than "we will not fetch
// from that address" — so an unknown reason falls back to the generic refusal rather than a key.
@@ -378,16 +385,17 @@ export async function req<T = unknown>(
// credentials, the structured-detail localization, the global error modal for 4xx/5xx, and an
// `HttpError` throw — so callers handle failures exactly as they do a `req` call.
//
// `body` is a `Blob`/`File` for a raw upload or `FormData` for a multipart one. The raw shape is
// what `/api/downloads/upload` uses since P1, and it is not a preference: a multipart part is a
// DECLARED body, which FastAPI spools before the route's auth dependency runs. `cfg.headers` is how
// such a caller sends the metadata that used to be sibling form fields.
// `body` is a `Blob`/`File` — the bytes themselves, and nothing else. `FormData` was the other
// accepted shape until P1 and is deliberately gone: a multipart part is a DECLARED body, which
// FastAPI spools to disk before the route's auth dependency runs, and the backend dropped
// `python-multipart` altogether, so there is no longer anything on the other end that could parse
// one. `cfg.headers` is how a caller sends the metadata that used to ride as sibling form fields.
//
// Content-Type is never set here. The browser derives it — the multipart boundary for FormData, the
// Blob's own type otherwise — and overriding it would break the boundary.
// Content-Type is never set here: the browser derives it from the Blob's own type, and a wrong one
// is worse than none.
export function uploadReq<T = unknown>(
url: string,
body: Blob | FormData,
body: Blob,
cfg: {
onProgress?: (pct: number) => void;
signal?: AbortSignal;
@@ -401,7 +409,10 @@ export function uploadReq<T = unknown>(
const requestId = newRequestId();
const started = performance.now();
xhr.setRequestHeader(REQUEST_ID_HEADER, requestId);
for (const [name, value] of Object.entries({ ...identityHeaders(), ...cfg.headers }))
// `identityHeaders()` LAST: who the request is from is this layer's to state, not the caller's.
// Merged the other way round, a caller's header of the same name would silently re-attribute
// the upload to another account — the one thing in here that must not be overridable.
for (const [name, value] of Object.entries({ ...cfg.headers, ...identityHeaders() }))
xhr.setRequestHeader(name, value);
if (cfg.onProgress) {
xhr.upload.onprogress = (e) => {
+7 -3
View File
@@ -124,6 +124,10 @@ export interface DownloadUsage {
max_concurrent: number;
max_jobs: number;
unlimited: boolean;
// The per-file cap on an upload, so the dialog can refuse an over-cap file before spending the
// user's bandwidth on it. Not the same thing as `max_bytes`, which is the whole library
// footprint: this one is a single request's ceiling and is enforced twice server-side anyway.
upload_max_bytes: number;
}
export interface AdminStorage {
@@ -184,9 +188,9 @@ export const downloadsApi = {
spec?: DownloadSpec;
display_name?: string;
}): Promise<DownloadJob> => req("/api/downloads", { method: "POST", body: JSON.stringify(body) }),
// Ingest a local file into the library (E7 S5). Multipart via XHR so the dialog can show upload
// progress; any file type is accepted (media gets the full function set, a generic file is
// download-only). Returns the already-`done` job the ingest creates.
// Ingest a local file into the library (E7 S5). Sent via XHR rather than `req` so the dialog can
// show upload progress; any file type is accepted (media gets the full function set, a generic
// file is download-only). Returns the already-`done` job the ingest creates.
// The body IS the file, and the two names ride as headers. Not a style choice: a multipart part is
// a DECLARED body, which FastAPI reads and spools BEFORE the route's `require_human` dependency
// runs — so with the old shape an unauthenticated caller could make the server write the whole