Files is the one library: every done file, the ones shared with me, or one
folder's, picked in a side rail (sources, own folders, the storage meter) and
mirrored to ?folder=. One table with every kind of value in its own column;
Copy link, Share and Save in sight, every action on a right click: play,
cut, rename in the row, move to an own folder or back home, keep, details,
re-download, delete.
The Downloads page keeps only the queue until the tray replaces it. The old
/api/downloads library routes are gone: list shared, upload, the details
PATCH, delete, the file and the usage now live under /api/files, and a
listed file says which source it came from. The job's "expired" flag is gone
with the state it described.
/api/files is the library's home: list a user's done files (by folder, searched, sorted, or the
ones shared with them), rename / move / keep / delete a file, make, rename and delete own folders
(a deleted folder's files go back to their source folders), and set how long any folder keeps a
file — done files shift by the difference, never closer than the GC's warning window.
An upload can name its folder (an own folder or Uploads), checked before the body is read and again
before the file lands; it is hashed while it streams, and any file can be hashed on first ask. The
download says the hash (Repr-Digest) once it is known; a new file at an asset's path forgets it.
Uploads, deletes, keeps, moves and folder changes are audited (ids only). The fileless "expired"
rows of the old retention are deleted by migration 0103: an expired file now leaves the library.
The shared route helpers moved to routes/files.py; the old /api/downloads library routes delegate
to the new ones until the client moves over.
The Files library starts here. A user's every download, clip and upload now
lands in a source folder made per origin (YouTube, Plex clips, uploads, one per
other site), decided once in downloads/folders.py at every place a job is made.
How long a file stays moves from the shared asset to the user's job: stamped
from its folder when it is done, never for a file the user keeps. The GC warns
each user once, then takes the expired file out of their library; the cache cap
never evicts a kept file, and an evicted one leaves every holder's library.
The media mount is /files now (FILES_ROOT, FILES_HOST_PATH, volume
siftlode_files); docs/self-hosting.md says how to carry an old volume over.
Migration 0102 places every existing job and carries each done file's expiry
over from its asset.
safe_abs_path takes a missing path and answers None, which drops one baselined
type error.
- The cut file is measured against what its owner has left, for a YouTube range (whose fetch never
reads yt-dlp's size ceiling) and for a Plex clip (which fetches nothing), before the fetch and
again before it is placed; a refusal leaves the asset pending for a Retry.
- A silent Plex file is cut silent; a file the probe cannot read says so.
- A second job on a failed or renamed clip gets the same YouTube diagnosis and the fetched title.
- The "too long" refusal names the server's cap; a contract test holds the client's copies of the
clip limits to clip.py; the shipped player clients are pinned against the installed yt-dlp.
- One HDR transfer set and one rating_key lookup; job status words from JobStatus; the display-name
guard sees subscript writes and the clip route; stale comments and types follow the clip job.
An optional crop in clip mode: a Crop toggle puts a frame on the picture (free, 16:9, 9:16 or 1:1;
the whole picture stays the default). The frame is sent in fractions of the box it was drawn in,
with that box's shape; the worker fits the real picture into the box by its display aspect (sample
aspect and rotation aware), drops the part over the bars, and crops first in the filter chain. A
frame covering the whole box is no crop at all, and one off the picture is refused by name.
The crop frame is a ui primitive (CropFrame) shared with the video editor, keyboard-operable.
YouTube: the wide side panel shuts while clipping and comes back after; maximised, the clip panel
docks in the stage. Plex: the picture sits above the clip panel so the frame lies on it.
POST /api/downloads/clip takes a source (a YouTube id or a Plex rating key), a range of at most
ten minutes and, for Plex, the audio track being heard. The clip becomes a per-user asset in the
library like an editor clip, shareable as soon as it is ready. A YouTube clip fetches only its
range (yt-dlp download_ranges: 3-7 s whatever the video's length); a Plex clip reads the library
file in place. Both then go through one re-encode: frame-exact at the marks, H.264 at most 1080p
with HDR tonemapped to SDR, stereo AAC, no chapter track, .mp4.
Job kinds are a JobKind StrEnum now, and Re-download refuses every kind but a download.
Plex, library and /watch tell the operating system what plays (title, line, picture) and take its
play / pause / skip / seek through one hook (lib/mediaSession); the Plex player drives it with its own
seek and clock, so the lock screen shows the film's position, not the stream's. The YouTube player is
left alone: its iframe already publishes all of it, and the OS routes to the frame that plays.
The library and /watch players get the basic keys (lib/mediaKeys, the keymap's mediaPlayer rows) and
open where the viewer stopped (lib/resume): the library keeps the point on the server, per user and
library item (library_states, POST /api/downloads/{id}/progress, listed back on both listings), the
shared link in this browser. A started library row continues, its menu can start it over, and its
picture carries the cards' progress bar. The resume rule (too early / near the end = from the start)
is one function shared with the YouTube route.
The library dialog opens focused on its panel (Modal initialFocus "player"): on the ✕, Space closed
the film. While the media element itself has the focus, Space and the arrows stay its own controls'
— the browser acts on them whatever a listener does, and one Space played and paused.
- The fast join gives each ffconcat entry its own media-only format whitelist, so a library file
can no longer be opened as a nested concat list (the list's whitelist was handed down to it).
- A file the guarded probe cannot read is a download-only file, not a guess from yt-dlp's info.
- A conversion's or an ingest's commit that raises is re-read in a fresh session before any file
is removed: a commit that landed keeps the file its row now names.
- An original that cannot be removed after conversion is logged; the upload stays ready.
- An upload whose conversion is cancelled or deleted midway is freed once nothing holds it; a
paused one is kept.
- A shared link says "preparing" only while its video's job is queued or running.
- The admin storage view counts every file on disk ("Files on disk"), not only ready ones.
- A clip's failure title, unknown failure and cancel dialog, and a clip's and an upload's delete
dialog speak of what the job made; an upload's quota refusal explains that the original is kept.
- /watch announces "preparing" as an alert and moves focus to the new Retry after an action.
- Every ffmpeg/ffprobe over a library file names the demuxers it may use
(edit.INPUT_GUARD; a downloaded thumbnail edit.THUMB_GUARD): a playlist-type
demuxer would read local paths named inside a user's upload.
- A re-encode ends on an even frame (edit.EVEN_FRAME): an odd 4:4:4/RGB source
failed its conversion; a clip stores the frame its file has.
- An upload counts against its owner's cap from the moment it lands (queued, or
kept after a failed conversion) — the footprint, the GC's cap and the admin
view read one ON_DISK predicate — and a conversion that outgrows the cap is
refused, keeping the original.
- The converted name keeps _a{id} (a dotted bare name once lost it and two
uploads shared a file); a converted file placed but not committed is removed,
and the orphan sweep enters .uploads; the LRU cap never evicts an upload.
- Ingest judges the extension the file is stored under; a failed conversion is
convert_failed / upload_missing, reports the "converting" phase, and notifies
as an upload. Job statuses in routes/downloads use JobStatus.
A share link is opened on whatever the friend holds. Measured on an iPhone and an iPad: AV1,
Matroska, HEVC tagged hev1 and MP3 inside an .mp4 fail there; Chrome and Firefox add HEVC and
10-bit / 4:4:4 H.264. The old check judged the codec NAME only and ran on downloads only, so a
10-bit H.264 passed, an upload went out exactly as it came (an iPhone's HEVC .mov, a webm, an mkv)
and a fast trim kept its source's codecs.
One rule now, `downloads/playable.py`: video is H.264 8-bit 4:2:0 + AAC in .mp4, audio-only is
.mp3 or AAC .m4a; `plan` says what must change, `build_cmd` re-encodes only the failing stream (or
remuxes). The worker applies it to a download, an editor clip and — new — an upload: media that
fails the rule is queued (asset pending, job queued) and the worker converts it, keeping only the
converted file (owner's choice). Until then the original sits at rel_path, so Retry starts from it
and a delete removes it; an upload that lost its last holder mid-conversion is reaped by the GC.
Every libx264 run (edit, concat, conversion) now pins `-pix_fmt yuv420p` — without it a 10-bit
source came out High 10. The download cache signature moves to v5 so a cached 10-bit asset
re-derives. The upload toast says when the file is still converting (HU/EN).
Tests: the plan for every measured class; real ffmpeg turning a 10-bit Matroska + Opus and an
mp3-in-mp4 into files the rule accepts (red without the pixel-format pin); the queued upload's
delete, cancel-then-GC, failed conversion and real worker conversion (the cleanup tests red
without their rules); the toast (red without the branch). Live on localdev: 9 fixture uploads
converted, every served file probed H.264 High yuv420p + AAC, or mp3 / m4a.
A plan's decision ids (`D1`, `D4`, `D6-A`) cited in 11 comments and test titles named a document
no reader of this repo has, and the guard had no pattern for them. It has now (mirrored in
audit_metrics), with cases for each form the citations took and for the look-alikes it must leave
alone (a JPEG marker, "3D"); the 11 lines say their why without the id. Case test seen red without
the pattern, the scan red on a planted `D9:`.
The sprint alternative matched only `(S6)` or `S3e2`, so the commonest purged form (`# S6:`,
`S3's`, `"""S6 —`, `S1 justified`) and every phase code (`P1`) would come back unflagged; the phase
codes had in fact survived the purge in 16 lines (Plex client sections, the upload's header shape),
now reworded around them.
One pattern now covers epic, roadmap, sprint and phase codes; the named exceptions are TV episode
notation, a fixture's season or episode title (`title="S1"`) and ruff's `E4/E7/E9`, and a ruff rule
like `S110` does not match. The case test lists the purged forms and no longer claims every form.
audit_metrics copies the definition (ASCII, as JavaScript's \b is) and reports it as
plan_code_lines / plan_code_files.
The code cited its plan instead of saying why: `E28 review J4`, `(R6 S3b)`, `S1 justified…`,
`deep-review round 5`. That plan lives in the private ops repo, so for every reader of this one the
citation was noise, and a sentence whose subject was the plan step ("round 2 showed…") hid its
lesson behind an id. Each one is gone and the "why" kept: a bare citation is deleted, a sentence
carrying a lesson is reworded around it ("a later attempt showed…").
Measured (`scripts/audit_metrics.py`, product code): 652 lines in 270 files carried an epic code by
the definition the next commit holds (507 / 241 by the old, two-digit-only one) -> 0. Across all
code, tests, scripts and configs: ~1100 epic-code lines plus ~320 sprint/round ids.
Behaviour unchanged, checked mechanically: every changed .py file has the same AST once docstrings
are blanked; css/sh/toml/yml/json files are identical once comments are stripped; the only
non-comment changes in TS/JS are test titles, one description string in a contract table, and five
ESLint messages, whose text is all the gates snapshot diff holds. Not touched: Markdown (docs and
the changelog keep their history), Alembic migrations, TV episode notation (`S2 · E5`), DESIGN.md
ids (`C1`, `B4`, `S3h`), tracker ids (`SEC-16`, `DX-17`).
One stale claim fixed on the way: test_hls_reaper's docstring said the player checkpoints "every
10 s whether or not it is playing", which plex/stream.py calls wrong; it now says what the code does.
The demo account — one shared user that whitelisted emails entered from
the login page, reset on a schedule — goes end to end (owner decision,
E34): demo.py and the reset job, users.is_demo and demo_whitelist
(migration 0098 deletes the demo user, its audit rows, the reset job's
override and notices, then drops the column and table; the downgrade
restores the empty schema), the /auth/demo entry, the admin whitelist and
reset, the Demo tab, the banner, the login page's "Try the demo", and
every is_demo branch in routes, userscope, scheduler and the UI.
require_human only ever refused the demo account, so its 87 uses become
current_user (owner-approved); require_write_scope keeps its own check.
The login page's "or" divider now shows only beside the Google button:
without the demo entry under it, an instance without Google would have
said "or" to nothing.
A contract test (removedFeatures) fails on the word anywhere in the
backend, frontend, e2e or scripts — seen red on a planted hit. Size
ceilings tightened for the modules that shrank; admin.py is back under
the limit.
ruff gains RUF100, PLW1510 and flake8-bugbear, every hit fixed rather than
ignored. B008 stays on with FastAPI's Depends/Query declared immutable.
basedpyright moves to standard mode, pytest to --strict-markers, and a hand
run of pytest now requires the database unless SIFTLODE_TEST_REQUIRE_DB=0.
PLW1510 found a real bug: the three ffprobe probes in plex/stream.py never
looked at the exit status, so a probe that failed was read as the file's
truth (an HDR source sent down the HW-safe path, a multi-audio file collapsed
to one track, a subtitle offset taken from garbage) and the warning each
except branch promises never fired. They now run with check=True.
The Postgres image is Alpine, whose en_US.utf8 is byte order: "Életrajzi" and
"Ötödik" sorted after "Zorró" in the feed, the channel lists, the Plex grid,
its genres and collections. `app/collation.py` names the ICU collation of the
reader's language (hu-x-icu puts "Ö" after "O" and "Cs" after "Cu"), wraps
every text ORDER BY with it - the feed's keyset cursor compares with the same
expression, so paging walks the order it sorted in - and orders the lists built
in Python (genre and studio facets, collection members, the discovery re-sort)
through the same collation. Plex title tiebreaks use the shown (HU) title.
- lib/vocab + the `vocab` locale area: TMDB crew jobs and departments, auto-tags (D1 A: the data
stays English, the catalog is named at its one source, useTags), built-in download profiles,
YouTube topics; held key-for-key to the backend's own lists by backendIds.contract.
- Countries by ISO code: TMDB parse stores {code, name}; rows stored before are coded from TMDB's
own country list in one request (run_enrich); Intl names them, historic codes (YU, SU, XI…)
from the catalog.
- Player tracks named from the language tag, forced/SDH flags and codec tail, not Plex's
server-language displayTitle; no invented "Audio N" on the server.
- Download lifecycle notices (shared / expiring / expired / evicted) named from data; the share
notice carries the title.
- Scheduler: the job result rides along as data and is named per key (dashboard + run notice).
- Invite status named; a render crash's notice no longer shows the engine's English message.
- Fix: YouTube links …/wiki/Humour, the tagger looked for "Humor" — Comedy never came from a topic.
Deep-review round 2, on round 1's own fix commit. It found MORE than round 1
did (28 raw, against 13), and most of it was mine -- a new piece of UI attracts
findings a backend-only diff never does. The pattern in almost all of them is
the same: a principle applied to the site that prompted it and not to its
siblings.
* `display_text` reached three of the FIVE writers of a user-supplied name.
`POST /api/downloads` and `POST /api/downloads/edit` still handed Postgres an
unbounded string, so a 300-character name pasted into the Add-download dialog
or the editor was a bare 500 (main.py re-raises every DataError but the NUL
case). Four of the seven reviewers found it independently. The bound now lives
in `models.py`, the three columns are DECLARED with it, and the guard for the
next writer is parsed rather than remembered: every `display_name` /
`display_uploader` assignment under `app/routes/` must go through the helper.
That guard immediately found two more -- the Google sign-in callback writes
the profile name into the same 255-wide column -- so it is six, not five.
* The connection release reached one of the THREE routes that read their own
body. The screenshot route (10 MiB, from a phone) and the client log sink both
still held a pooled connection idle-in-transaction for the length of the
upload, which is the invariant db.py sizes the pool on. All three now call
`db.release_read_connection`, whose precondition -- nothing pending on the
shared session -- is CHECKED rather than described, because a whole-session
rollback discarding a caller's staged write is a mistake this repo has already
made once. `test_body_limit.py` holds the rule for the next such route.
* The AST drift registry keyed its sites by (module, limit expression), so a
second route copied from the first collapsed onto the same entry and went
uncovered -- the exact hole the registry exists to close. Keyed per call site
now, and it carries the release requirement too.
* The dialog's own refusal was announced once and never again: two over-cap
picks leave an identical node with identical text, and an untouched live
region says nothing. It is keyed per pick now, the drop zone reads as invalid
and points at the sentence, and the reason is reachable from the control
rather than only from a one-shot announcement.
* Two tests rebuilt the blind spot they were written to remove: the dialog's cap
test sat nowhere near the boundary (a `>` loosened to `>=` survived it, which
is the mutation round 1 fixed on the server side), and the header-precedence
test only covered an exact-case duplicate while a differently-cased one would
have been COMBINED onto the wire rather than replaced. Both fixed, and the
merge now dedupes case-insensitively because header names are.
Also: the upload dialog's unstubbed usage query was firing a real request from
jsdom in a neighbouring suite and leaking a delayed "connection lost" toast
across test boundaries; `GET /api/downloads/usage` had no test at all though the
whole client-side check hangs off it; and the comment claiming the edge 413 is
readable now says what was measured -- readable to 200 MiB over loopback, a bare
`onerror` through a proxy -- so the pre-send check is named as the mechanism and
that branch as the backstop it is.
Every test added or changed here was mutation-checked. One mutation SURVIVED and
is recorded rather than papered over: the `tooBig` term inside `submit()` cannot
be reached, because `ui/Button` swallows the activation itself, so it is marked
as belt and braces instead of being described as the guard.
The epic-end deep review found three defects in P1 and four tests that did not
hold what they claimed. They land together because they share the upload route
and its test module; splitting by hunk would have produced commits that do not
individually run green.
* The edge ceiling and the route's own cap are the SAME number at the default
configuration, so the edge always answers first: a declared Content-Length
over the cap is refused before a byte is read, and an undeclared body trips
the middleware on the same chunk the route would have tripped on. The route's
numbered 422 -- and the drain written for it last week -- are reachable only
once an admin LOWERS the knob, which is the one shape the browser UAT tested.
Measured: headroom does not fix this, because the declared-length check fires
first for any body big enough to matter and the drain pushes the count back
over the edge for the rest. So the edge's own refusal is made readable
instead: `errors.tooLarge` localizes `too_large` the way `upload_too_large`
was already localized, and the dialog refuses an over-cap file BEFORE sending
it, off the `upload_max_bytes` that GET /api/downloads/usage now reports.
Measured in Chrome against the API directly: that 413 arrives as `onload`
with a readable body in 43 ms.
* Reading the body behind the auth dependency put the whole transfer inside the
request's database session -- the auth SELECT autobegins a transaction, so one
of the twenty pooled connections sat `idle in transaction`, with nothing to
time it out, for as long as the client cared to keep sending. That is the
invariant db.py sizes the pool on and plex/playback.py records an incident
for. The read transaction now ends before the await. Measured on the dev
server: an 1818 ms upload shows no idle-in-transaction sample while the bytes
move, only afterwards, when the handler actually writes.
* The display name moved into a request header, where nothing bounds its length,
while the column stayed varchar(255) -- so a long enough name was a DataError
500 after the whole file had already transferred. One `display_text` helper
now serves all three writers (the metadata PATCH already truncated, the upload
ingest did not), and the dialog stops at the same number.
* The gate invariant asked `route.dependant.body_params`, which holds only the
route's OWN callable's parameters: a body declared in a SUB-DEPENDENCY -- the
upload route's own idiom -- leaves it empty while FastAPI still reads the body
before auth. It asks `route.body_field` now, which is what FastAPI itself
consults. The AST registry of body-reading call sites had also never followed
`capped_stream_to`, so the one route the whole invariant exists for was the
single site the drift guard could not see.
Four tests that passed over their own mutation are fixed: the cap boundary (a
`>` loosened to `>=` survived the entire suite), the percent-decoding (every
filename the suite sends is ASCII, where quote() is the identity function), the
display-name header (no test at all), and the staging cleanup (deleting the
unlink left the suite green while every refused upload leaked its partial bytes
onto the volume). Every test added or changed here was mutation-checked.
`POST /api/downloads/upload` declared `file: UploadFile = File(...)`, and
FastAPI parses and spools a declared body before the first dependency runs --
`require_human` included. An unauthenticated caller could therefore make the
process write the whole per-file allowance (500 MiB by default) to temp, once
per request. SEC-20's ceiling bounded that number but could not stop the
writing, which is why E27 parked it rather than closing it.
The route now takes no body parameter. `spool_upload` names `require_human`
first and only then streams `request.stream()` to the staging temp file, so a
stranger is refused before a byte moves -- the shape the screenshot route has
used since E16. Filename and display name travel as percent-encoded headers
(latin-1 on the wire, so a Hungarian filename needs the encoding) instead of
multipart parts.
`capped_stream_to` joins `capped_body` in routes/_common.py: the same counting
rule, written to a file descriptor rather than accumulated, because holding
500 MiB would trade a disk problem for a memory one. Its refusal is the
caller's, so the upload route keeps answering 422 `upload_too_large` with the
admin knob's own number.
With no envelope to allow for, the edge is now exactly that knob, and
MULTIPART_ENVELOPE_ALLOWANCE is gone.
The invariant stops being prose: test_body_limit.py asks FastAPI itself
(`route.dependant.body_params`) and fails the gate if any route with a raised
ceiling declares a body. A comment could not have prevented the defect it
described; that test would have.
Both new tests were seen red for the right reason -- before the fix the
anonymous probe got its 401 only after the server had read 4236 bytes of a
multipart body, and re-declaring the File parameter turns the invariant test
red naming the route and the parameter.
Two rows of the canonical-patterns table had drifted, found while closing the roadmap audit:
- 63 mutation responses carried no `ok` — `{"unsubscribed": true}`, `{"status": "ok"}`,
`{"added": 3, …}` and so on. Each now leads with `"ok": true` (additive: every field a client
reads is still there).
- Twelve route helpers committed on their own. Six were just the end of their handler's work and now
flush: the playlist item add/remove and Watch later get-or-create, the Plex series/season bulk
mark, and the invite decision + account activation — which an approval used to land in separate
transactions. The other six commit on purpose (a token durable before its link is mailed, a
single-use token burned as its own unit, the Google callback's tail, a background registration
with its own session, the staged search ingest around YouTube calls), each listed with its reason.
`tests/test_route_conventions.py` holds both rules in code; CLAUDE.md's two rows point at it.
~150 HTTPExceptions carried a hand-written English sentence as their detail,
shown verbatim under the Hungarian UI — in the blocking error dialog and on
every inline surface that printed HttpError.detail (sign-in, setup wizard,
change password, the passphrase gate, live search). Four also quoted an
upstream exception's text back to the user.
- app/refusal.py: `refused(status, reason, **params)` → {code: "refused",
reason, …params}; localizeDetail translates errors.refused.<reason> with the
params interpolated. "Unknown X" 404s use the existing not_found(what).
- sysconfig.set_value raises InvalidSetting (reason + bounds) instead of an
English ValueError; the three writers answer it through setting_refused.
- The Plex connection/write/image failures log what the upstream said and
answer a reason; the search scraper's failure is logged and answered as
youtube/unavailable (it crashed refusal_reason before).
- HttpError.detail now only ever holds a translated reason: a string detail
(only the framework's own can remain) is left out, so reading .detail is
safe everywhere — including destructured, the hole the raw-detail source
scan could not see. That scan and its five-file exception list are gone.
- Enforced: test_http_details.py fails on any HTTPException with a sentence
for a detail; the contract test requires every refused reason and every
not_found "what" in both languages, and no key for one that is gone.
- local/jsx-comment-text: a // inside JSX children renders as text; the rule
is ten lines in eslint.config.js rather than eslint-plugin-react's ~1800
lockfile lines.
move_item checked "already in the target?" and then re-parented without a
lock, so a second tab's add or the scheduler's YouTube merge committing the
same video in between hit uq_playlist_video and answered a raw 500. The
re-parent now runs in a savepoint and a conflict falls back to the
already-present outcome. The target is touched only when the row landed in
it (an untouched list was promoted in the rail's "recently changed" sort),
and _touch stamps the database clock like the mixin's own onupdate.
The canonical-patterns table names owned_or_404 against ad-hoc ownership
helpers; six were left (_own_playlist, _own_playlist_or_404, _own_view,
_own_tag, notifications._owned, downloads._own_job), two of them
hand-rolled with a bare English 404. All call sites now use owned_or_404,
and the download/link/notification 404s are structured like the rest. The
contract test reads every "what is gone" the routes name, and the YouTube
refusal reasons, and requires both languages to name each.
114 exception handlers neither logged nor re-raised. The defect was never that
they are silent — plenty of them should be — it is that their silence said
nothing, so nobody could tell a decision from an oversight. Each one has now been
read. 44 got a line; the other 70 carry `# silent-ok: <why>`.
What the silent ones had been hiding, in the cases that got a line:
- A failed ffprobe sent every HDR file down the 8-bit pipeline, so playback
degraded to a software transcode with nothing saying why; a failed audio probe
collapsed a multi-audio file to one track and the player showed no language
picker; a failed first-PTS probe left subtitles seconds out of sync.
- A subtitle sidecar that existed but would not parse made a PARTIAL track read as
complete, permanently — the viewer loses every cue past the cached window.
- An unreadable crop/trim/cut-list in an edit spec was dropped, so the clip came
out uncropped, untrimmed, or shorter than the one the user built.
- A dropped `max_height` downloaded at best quality against an explicit cap.
- A failed videos.list enrich made every candidate undroppable-from-judgement and
therefore dropped: a working search rendered as "no results".
- An unusable password hash answered as "wrong password", locking the owner out
and telling them they mistyped; a rotated TOKEN_ENCRYPTION_KEY turned every
stored token into "not connected".
- A stored admin setting that would not coerce fell back to the env default while
the settings page went on showing the saved value.
- A one-off InnerTube config failure pinned the whole process to the hard-coded
fallback key, and searches return nothing when that goes stale.
- The HLS sweep reported 0 swept — like a clean run — when it could not list its
own root, leaving every crashed session's segments on disk forever.
Four of them are counted rather than logged per item (dropped cues, dropped cut
list segments, unreclaimed GC files) and reported once per call, because the
alternative is one line per cue over a whole subtitle track.
ruff gains S110 and BLE001. Every blind `except Exception` that remains carries
`# noqa: BLE001 — <why>`, which is the point: catching everything is sometimes
right, and always worth a sentence. plex/sync.py's three best-effort backfill
catches now keep the traceback — a blind catch over a batch that fetches AND
writes could be Plex, the database or our own bug, and `%s` of the exception told
them apart badly. main.py's boot-time log-level read likewise.
logs.event's `code` is positional-only now: `event("x", code=...)` was a TypeError,
and `code` is exactly the word a caller reaches for when the thing that failed has
one of its own.
tests/test_swallowed_exceptions.py is the wider rule ruff cannot see: a handler
that neither logs nor re-raises must say why. Both cases mutation-checked — a
marker removed and a log line removed each fail it, naming the handler.
Pause now re-reads the job under the same asset-then-job row lock as resume,
cancel, delete and redownload. Its unlocked write waited for a concurrent
cancel's lock and then set the released job back to paused, a holding status,
so a later delete released the same hold again and could free a file another
user's job still held (measured: the overlapping-request test ends 'paused'
instead of 'canceled' with the old read). The lock test now calls the route
handlers, so it also pins which helper each route uses.
Comments follow the code: resolve_url's contract names both verdicts and that a
failed page read is raised as itself; the inherited-failure diagnosis asks the
cookie gate as it stands when the job is claimed, so its advice fits the cookie
that job's Retry would use; the undecryptable-cookie warning no longer claims a
download goes ahead, since the worker parent reaches it only to diagnose.
A refused Facebook cookie on a group post is no longer "no video in this post":
measured on dev, Facebook answers with its login page (no post id at all), so the
adapter now raises its own POST_NOT_SHOWN verdict and the catalog explains it as
the refused cookie. The adapter owns both post verdicts and the "cannot parse
data" wall wording, so the Chrome retry and the catalog read one constant. The
group-post "no formats" entry is cookie-scoped (with a cookie yt-dlp only gets the
post's video link) and uses the adapter's own group-post test instead of a
"/groups/" substring.
Job transitions that move an asset's ref count (resume, cancel, delete,
redownload) re-read the job under a row lock, asset first, so two overlapping
requests can no longer release the same hold. get_or_create_asset loses an insert
race inside a SAVEPOINT instead of rolling back the caller's pending writes (Retry
lost its cleared error). Every worker-side job failure goes through
finalize_if_running and writes error_detail, so a cancel that landed first is kept
and a failure never shows an earlier one's copy. The worker asks the download
child's own cookie gate (credentials.usable_cookie) when a job inherits a failed
asset.
UI: the Facebook copy no longer points at a Retry button the failure window does
not have, says "paste" like Settings, and the HU wording is fixed
("Újraletöltés" on the re-download confirm). A category this client has no copy
for falls back to the generic sentence instead of a raw key.
Tests: route serialization of error_detail, the probed duration through a real
_download, the page-read cap and a failed read, a Retry off a pending asset, the
confirm-button labels, the lock and savepoint races.
The login-wall copy tells the user to add their cookies and press Retry, but
Retry kept the job on the shared asset the worker never attaches a cookie to,
so it failed on the same wall. Resume now moves a pending/errored job onto the
asset enqueue would pick with the cookie (never off a file or a download in
flight).
Re-download salted its fresh asset with the job's previous asset id, which the
cookie gate cannot recompute, so even a cookie download rebuilt without the
cookie. The rebuild salt is now the new asset's own id (plus the user's cookie
salt when one applies), and is_authed_asset recognises it.
A failed Facebook download read "Download failed": yt-dlp words the login wall
(18+ reel, closed group) as "Cannot parse data", which no category matched.
Measured on dev with the owner's cookie before writing a rule — the same text is
the login wall without a cookie and a refused cookie with one.
- errors.diagnose(msg, url, cookie_attached) tries a host-scoped catalog first
(all-of markers, cookie state, link fragment) and falls back to the category.
First entries: facebook.login_wall, facebook.cookie_rejected,
facebook.group_post_video. The module doc is the recipe for the next entry.
- download_jobs.error_detail (0084) + backfill of the stored Facebook parse
failures that were unknown.
- The pipeline's failure branch passes the link and whether the cookie went
with it; resume and redownload clear code and detail (redownload never
cleared error_code).
- A job meeting an already-failed asset got the text but no code, so it read
"Download failed" where the first job said why. It is diagnosed again, with
the cookie state from credentials.is_authed_asset (now shared with the
cookie gate); the redacted network text classifies back to network.
- FE: failCopyKeys gives the row and the failure window one lookup — catalog
copy, else the category. Copy in EN and HU; a contract test pins every
backend entry to both locales.
Deep-review round 5, over round 4's own fixes.
- `notifyActionError` preferred the server's `detail` over the caller's translated fallback, and
`reportFailure` deliberately leaves 403/404/429 to the caller — so for exactly those statuses the
dozen call sites round 4 converted started printing whatever English a route hand-wrote into an
otherwise Hungarian UI ("No user with that email.", "Not subscribed to this channel", "Unknown
job"). A STRUCTURED detail is still preferred: `req` has already run it through `localizeDetail`,
so it is both localized and more specific. A plain string is not.
- The other half of the same defect, on the backend: round 4's `get_or_404(what="channel")` sweep
reached exactly ONE of the five identical sites in channels.py and shipped the HU/EN string with
it, so the Block button said it in Hungarian and the four controls beside it in English. All five
are converted, and the subscription / share-recipient / scheduler-job 404s with them (two new
reasons in both locales).
- `PlexCollectionEditor`'s add/remove, take-over and delete were try/finally with no catch — in the
very file round 4 edited to give `createNew` one. 403 and 404 are the caller's to report, so the
tick simply never appeared and a CONFIRMED "delete collection" did nothing, silently.
- `UploadDialog` is the only `uploadReq` caller and kept an empty catch whose comment still claimed
the global error modal had it. It had not since `reportFailure` became shared: a proxy 502 raised
only the self-clearing connection notice and a 413/403/404 nothing at all, so the dialog snapped
back to its pre-upload state with no error named. The user's own Cancel stays silent — and since
a cancel and a network drop both carry status 0, that difference is now a named `isAborted`
rather than a string literal at each end.
Regression tests for each (5 vitest, 2 pytest, plus two existing assertions corrected), all
mutation-checked: 6 mutations, each red on the assertion it is named after. vitest 1007,
pytest 1127.
Deep-review round 4, backend half.
- The periodic HLS reaper (round 3) applied the "finished and unfetched for 30 s" rule that only a
NEW session request may apply. A remux finishes long before the film, and a paused player with a
full buffer fetches nothing, so a one-minute pause killed the session and playback died ~30 s after
resuming. The periodic pass is `reap_abandoned` now (the long idle limit, finished or not).
- A subtitle `publish` that could not keep its output (full / read-only cache volume) returned
quietly, so every poll queued the window and the whole-container pass again. It reports failure
and the pass is backed off like a failed read; a cached VTT that cannot be READ is answered
`unavailable` (`cache_unreadable`) instead of scheduling reads that can never land.
- The subtitle routes bound `rating_key` / `sub_ord` (they become a cache file name: a 250-digit
one was ENAMETOOLONG, which pathlib's exists() re-raises -> a 500), and the cache helpers no longer
throw on such a name.
- One app-wide handler turns psycopg's "text fields cannot contain NUL" DataError into a 422: round 3
guarded only the by-rating_key lookup, and `contains=`, `contains_group=`, reorder/bulk key lists
and the collection search still 500'd.
- The hand-written `db.get(...) is None -> 404 "Unknown ..."` siblings use `get_or_404` with a
`what` (channel / video / user), so the Hungarian UI no longer shows English for a gone row.
Round three, on round two's diff. The trend across the loop is 47 -> 34 -> 15 raw
findings; the ops half kept producing the sharpest ones because the gate cannot run it.
**`docker compose start` ignores a newly pinned IMAGE_TAG.** Round two removed the
automatic restart because it re-ran `alembic upgrade head` — and then told the operator to
run that exact command by hand, in both scripts and the docs. `start` relaunches the
container that already exists, which still carries the tag being rolled back FROM, so
pinning 0.79.0 and following the printed instruction brings 0.80.0 back up and re-applies
its migrations. Verified against a real daemon: after changing the pinned tag, `start` kept
the old image and only `up -d` picked up the new one. Both scripts and the doc now say
`up -d`, with the pin-first caveat where the operator reads it.
**The reset flag outlived its token.** Reverting round one's "each link clears the other"
went one step too far: a `#verify=` link pasted over a standing reset notice replaces the
whole fragment, so the reset token is gone — but `resetPending` stayed true, and being
checked first it hid the confirmation while promising a recovery that signing out could no
longer complete. The flag is now cleared when the fragment it describes is gone, which is
narrower than what was reverted and does not blank the signed-out card.
**A test that could not fail.** The shared-asset regression test seeded the asset as
`pending`, and `_reset_asset_pending` only acts on `downloading` — so the release half was a
no-op and deleting it kept the suite green. Seeded as the worker actually leaves it now.
Also: the malformed-URL guard answers with urlguard's structured detail rather than a bare
English string, so both halves of one typo class get the same localized copy, and the third
site in that module handing user input to `urlparse` was swept with it; the asset is
released after the job is terminal, not before; the Stats 403 toast gets the wizard handle
its own comment claimed it had; the pasted Windows recipe sets `$ErrorActionPreference`
inside the block (a scriptblock inherits the caller's, so a `SilentlyContinue` profile
swallowed every throw) and cleans up the container-side dump on the failure path too; the
restore warning moved to the warning stream so a `> log` redirect cannot file it away; and
two comments narrowed to what the code does.
Gate green: vitest 685, pytest 950.
Round two of the review loop, on round one's own diff. Two of its findings were
regressions the fixes themselves introduced, which is exactly what the loop is for:
**A per-user refusal was writing onto a shared row.** MediaAssets are deduped across
users, and the generic failure handler puts the message on the asset — right for an
upstream failure, wrong for "your storage is full". Two users queuing the same video share
one asset, so the first user hitting their cap failed the second user's job with the first
one's private reason, telling someone with an empty library to free up space. The refusal
now raises `_OwnerRefused`, caught separately: the job fails, the asset goes back to
`pending` for whoever else holds it. Pinned by a test that drives `_download` directly —
the refusal happens before extraction, so nothing needs stubbing.
**The restore scripts restarted the app, undoing the restore.** `docker compose start`
re-runs the container command, and that runs `alembic upgrade head` — so on the rollback
path this documents as recommended (restore the dump, THEN pin the older tag) the newer
image immediately re-applied the migrations the restore had just undone, and on a FAILED
restore it put the app back on a half-restored schema. Both are the opposite of why the
stop was added. Nothing is restarted automatically now; the scripts say what to run and
when, and the docs' snippets say the same.
**Reverted rather than patched:** the OnboardingWizard toast (the wizard already renders
that failure inline, next to a Retry button — three recoveries for one failure), and
`useVerifyEntry`'s "each new link clears the other". The latter blanked the signed-out
card's only banner, where nothing renders the reset flag to replace it. The two states are
independent again; what changes is that `dismiss` peels ONE layer, which is what the
original finding was actually about.
Also: `Stats` uses the YouTube helper rather than the bare one, so the route's English 403
detail no longer beats the translated fallback under a Hungarian UI; `urlparse` guarded
alongside `.port` (an unclosed bracket raised just as late), the reason renamed `malformed`
since "bad port" misdescribes it; three docstrings narrowed to what the code actually does
(the worker's byte check is per-job, not an aggregate ledger, and not on the edit path;
the redirect residual leaks a reachability oracle, which the note had denied; the
invalidation helper does not cover suspend); the pasted Windows recipe wrapped so `throw`
can stop it; the bash one-liner's failure branch no longer exits 0; and the tests that were
missing for the round-one fixes — the account panel's error paths, the invalidation set,
and the shared-asset case above.
Gate green: vitest 685, pytest 950.
SEC-12: the download box accepts an arbitrary http(s) URL and yt-dlp fetches
it from the WORKER, so any approved user could aim the server at the compose
network, the LAN or a cloud metadata endpoint — and read the result off
job.error, where "connection refused" vs "HTTP Error 401" vs "timed out" is a
port scanner. downloads/urlguard.py refuses a URL whose host resolves to a
loopback/private/link-local/reserved address, at enqueue AND again in the
worker (DNS can move in between, and the worker is where the socket opens).
A network-class failure on a `url` job now reports one fixed sentence instead
of the raw transport text; the error CODE is still classified from the
original, so the localized explanation is unchanged. DNS rebinding is
explicitly NOT closed — that needs the connection gated inside yt-dlp — and
the module says so rather than implying more than it does.
SEC-13: `noplaylist` only demotes a video-inside-a-playlist URL to that video,
so a URL that is purely a playlist or channel still expanded into every
entry — one enqueue, unbounded downloads. `playlist_items` pins it to one.
Size was unbounded too: `check_enqueue` compares the CURRENT footprint against
the cap, and a download's size is unknown until it lands, so one job could
overshoot by any amount; the owner's remaining budget is now handed to yt-dlp
as `max_filesize`, applied AFTER the adapter merge so no source adapter can
raise a limit the quota layer set. A join's cut-list is capped at 100: an
accurate join emits two filters per segment into one ffmpeg filter_complex.
SEC-14: the YouTube budget is instance-wide, so one user holding down a
manual sync spent the day for everyone. Per-user (not per-IP — the cost
belongs to the account) 3-per-10-minutes limiters on the two bulk pulls,
subscriptions and playlists, with separate budgets so one does not close the
other. The review also listed /rss, /backfill and /enrich; E3 deleted them,
and search stays unlimited on purpose — it is interactive, not a bulk pull.
Gate: pytest 896 -> 924 (none skipped), vitest 671.
Deep-review remediation for the E7 upload path, all findings adversarially verified:
- Serve every library file with a Content-Type chosen from the ffprobe-verified media_kind
+ a strict container allow-list (storage.served_media_type), never guessed from the
user-controlled file extension. Closes a stored-XSS vector where a media/HTML polyglot
uploaded as "evil.html" was served inline as text/html on the login-free /watch surface.
Applied to both the authenticated /file and the public watch_file endpoints.
- Classify uploaded still images (jpg/png/gif/webp/…) as media_kind='file' so they are
force-downloaded and don't get a broken in-app video player.
- Roll back the on-disk side of an upload when the ingest transaction fails after place_file
(unlink the placed file + its deterministic poster path), so a failed commit can't orphan
bytes in the GC-skipped .uploads tree.
- Floor download_upload_max_bytes at min=1 so a stored 0 can't silently disable the per-file cap.
- Dedupe the ffprobe-JSON invocation into edit.run_ffprobe_json (shared by probe_codecs and
upload._probe) and the upload extension into storage._clean_upload_ext; drop the now-unused
parse_probe_streams.
Tests: public + authenticated force-download and the polyglot-not-served-as-HTML guard, the
real _probe classifier (image/cover-art/dims/duration), the ingest rollback/cleanup path, the
max_jobs branch, the cap-floor invariant, and run_ffprobe_json error handling.
Add the non-yt-dlp ingest path so a user-uploaded file joins the same download
library and inherits its function set. Any file type is accepted: media
(video/audio) gets the full set (metadata, share, play/edit), a generic file
(pdf/zip/docx/…) is download-only.
- MediaAsset.media_kind ("video"|"audio"|"file") — the stored discriminator that
gates library actions. Set at every finalize (download probe, editor carries the
source's kind, upload probe); consolidates the ad-hoc audio-detect that lived only
in links.py into one invariant (downloads.mediakind.derive). Migration 0076 adds
the column + backfills existing rows.
- downloads/upload.py: ingest_upload probes the file (ffprobe → codecs/dims/duration),
places it in a per-user .uploads tree, and creates an already-ready asset + done job
(the same cache-hit shortcut enqueue uses). Uploads never expire (irreplaceable).
- POST /api/downloads/upload: streaming multipart with the per-file cap
(download_upload_max_bytes, default 500 MiB) enforced on actual bytes read; quota
charges the known size up front (quota.check_upload).
- Security: a generic file is always served force-download (attachment +
application/octet-stream) on both the in-app and public endpoints, so uploaded
HTML/SVG can't execute inline. Edit + redownload reject non-media / upload sources.
- python-multipart added (first multipart endpoint in the app).
D2-A: the library row's Share button becomes a single action — quickShareLink()
ensures a public link (minting on demand), copies it to the clipboard, and toasts
with an "Open in new tab" action. A copy failure still toasts (warning) with that
action so the link is never a dead end. Advanced controls (password/expiry/revoke/
user-ACL) stay in the existing ShareDialog, reached via a new "Manage sharing" row
button.
D1 (forward-only): CreateLinkIn.allow_download defaults to true, so a newly minted
link is downloadable by default. The default lives in the API, not a migration —
the download_links column default stays false, so no existing link flips. The
ShareDialog create-default is flipped to match.
Tests: TestCreateLinkDefault (default true; explicit false honored); 4 vitest cases
for quickShareLink (mint-vs-reuse, copy ok/fail, create failure). HU/EN parity.
Each user stores their own login cookies (Netscape cookies.txt) for an
auth-walled download host (Facebook, Instagram, X, TikTok, Reddit) in
Settings; a download of an auth-walled link on that host is retried with
the acting user's cookie.
Two load-bearing security controls, both in downloads.credentials as the
single source of truth so enqueue and the worker can't drift:
- cache privacy: a cookie-authed download salts its MediaAsset format_sig
with the user id, so the authed asset is private and never cache-served
to another user. The worker attaches a cookie ONLY when the asset it is
downloading into IS that private, salted identity.
- host scope (SEC-12): a cookie is resolved only for a URL whose netloc
the credential owns (netloc-scoped _host_is); a look-alike/unrelated
host gets none.
cookies_enc is Fernet-encrypted at rest, only decrypted in-worker into a
0600 file under staging/.auth/ (survives the retry wipe + _produced_file
scan) and unlinked in finally. The blob is never returned by a route,
logged, audited, or written to a job's error/snapshot. cookiefile is set
after the adapter's extra merge so an adapter can't clobber it. Settings
UI is write-only (host + set/updated only, never echoed). New
download_credentials table + migration 0074; per-user routes; HU+EN i18n.
Tests: host-scope, cache-privacy (no cross-user serve), encrypt-at-rest,
write-only API, worker attach-gate; FE dom test for the write-only field.
pause/resume/cancel/delete/redownload each hand-rolled their asset bookkeeping
inline in the route (release, failed-asset reset, the 20-line fresh-asset
rebuild), duplicating logic the worker/GC also touch. Move each transition into a
service.* method (service.pause/resume/cancel/delete/redownload) that owns the
mutation + its commit; the routes keep only auth, the HTTP 409 guards, and
serialization. Drops the now-dead _set_status route helper and two stale imports.
Pure behaviour-preserving code motion (routes -56 lines). Green under the E3 S2
transition net + the full downloads suite (363 pass), ruff clean.
MediaAsset.ref_count (how many jobs hold a shared cache asset) was a Python
read-modify-write at four sites (enqueue, enqueue_edit, release_asset, the
redownload route). Two concurrent enqueues of the same (source,format) both read
the same value and write the same +1 → the count ends up one short → the first
release drives it to zero and deletes the file out from under a live holder.
Replace all four with atomic DB ops: incr_ref_count (`ref_count = ref_count + 1`)
and decr_ref_count (`greatest(ref_count - 1, 0)` RETURNING the new value, so the
release decides whether to free the asset from the post-decrement count while
holding the row's write lock — Postgres serialises concurrent releases on that
lock, closing the double-free). Pure DB-atomicity change, no semantic shift.
Green under the E3 S2 ref_count net (5) + full lane (356 pass), ruff clean.
The cancel route writes status 'canceled' but the supervisor compared against
'cancelled', so its "a route stopped it -> kill the child, free the asset"
branch was dead code. A canceled job's child ran to completion and wrote 'done'
unconditionally over the route's 'canceled'; since 'done' holds the asset but
'canceled' does not, a later delete released the asset a second time, dropping
ref_count below the live holder count and deleting a file still in use.
Add app/downloads/status.py as the single source of truth for the lifecycle
literals (JobStatus StrEnum + the HOLDING / STOPPED_BY_ROUTE sets) and use it at
the safety-critical sites: the supervisor stop check, the route cancel, and
release_asset's holding set. Make the worker's terminal done/error writes
conditional on the row still being 'running' so a child finishing just after a
route cancel/pause can no longer resurrect it. Add the missing 'canceled'
supervisor regression test.
(The full literal sweep across the downloads surface is left for the E3
downloads owner-unification; this is the data-safety subset.)
- Floor the effective max_concurrent at 1 on the read path (quota.resolve)
and in the claim SQL (GREATEST(..., 1)). admin_set_quota already floors on
write and the sysconfig default's min is 1, but a stray stored 0 (direct DB
edit) would diverge the two "at limit" checks — the SQL claim-window
exclusion can't see a user "capped" with zero running jobs, while
at_concurrency_limit can — reopening the starvation the fairness fix closes.
Keeping the effective limit >= 1 makes the two checks provably agree.
- shared_with_me now orders by (created_at DESC, id DESC) to match
list_downloads, so the newly-added 500 cap has a stable boundary among
same-timestamp shares instead of a non-deterministic one.
- 0062 upgrade loop: rename the unused loop var "table" to "_table"
(matching the underscore-marked "_col").
Test: a stored max_concurrent=0 flooder no longer wedges the claim loop.
#2 — _claim_job excluded users at their concurrency limit only in Python,
AFTER a fixed LIMIT-50 fetch window. One user flooding the queue with >50
queued jobs while at their cap filled the whole window, chosen came back None,
and every other user's work past position 50 was never seen. Exclude capped
users IN THE SQL (per-user max_concurrent from the quota row, else the
sysconfig default; unlimited users never capped), keeping the Python re-check
as a race guard that can only reject, never over-claim.
#3 — list_downloads and shared_with_me loaded every job for a user with no
bound. Cap both at 500 newest-first (a shared _MAX_LIST, matching the admin
listing's existing cap); the Download Center renders newest-first and 500 is
far beyond any real manual-download history, so no frontend change is needed.
Tests: DB-backed — a capped user's 60-job flood no longer starves a user
behind the window; oldest-by-queue_pos claim, all-capped->None, unlimited
never excluded; and the list cap bounds the result set.
Layering cleanup, no behaviour change (ruff clean backend+scripts, pytest 193):
- Invert the scheduler->routes dependency: reset_demo_state (+ its demo-seed
helpers) move from routes/admin.py to a new service module app/demo.py.
Both routes/admin.py and scheduler.py now import it from app.demo, so the
scheduler no longer reaches into the routes layer — its lazy import (the
routes<->scheduler cycle workaround) is gone, hoisted to a top-level import.
- Publish the private symbols that were imported across module boundaries (the
underscore convention was lying): feed._serialize -> serialize,
feed._to_tsquery_str -> to_tsquery_str, videos._insert_stubs -> insert_stubs,
videos._stub_from_playlist_item -> stub_from_playlist_item,
worker._uploader_url -> uploader_url, plex._shared._item_or_404 ->
item_or_404. Scoped renames — the 5 other route modules keep their own truly
private _serialize helpers.
- _release_asset moves from routes/downloads.py to app/downloads/service.py as
the public release_asset (asset bookkeeping is a service concern); the routes
call it via service.release_asset.
Failed downloads stored the raw yt-dlp/ffmpeg text and dumped it as a truncated red
one-liner in the queue. Classify the failure into a category (unavailable / login_required /
geo_blocked / format_unavailable / drm / postprocess / source_missing / network / unknown)
and surface it properly:
- backend: app/downloads/errors.classify maps the raw message to a category code; the worker
stores it in the new download_jobs.error_code (migration 0061) at both failure sites; resume
clears it; the serializer exposes it. Raw text kept as technical detail.
- frontend: the queue row shows a clean localized reason + a "details" link opening a modal
with the explanation, the raw text behind a disclosure, and a Retry only for transient
categories (network/postprocess/unknown).
- tests: classify() unit coverage over verbatim yt-dlp/ffmpeg message shapes.
All 11 download endpoints off payload:dict. Highlights:
- admin_set_quota: AdminSetQuotaIn — the `int(payload["max_bytes"])` 500 on a
non-number is now a 422; each field set only when provided (is not None).
- profiles/enqueue/edit: CreateProfileIn / UpdateProfileIn / EnqueueDownloadIn /
EnqueueEditIn; spec/edit_spec typed as dict (non-object → 422). _resolve_spec now
takes (profile_id, spec) instead of the raw payload.
- meta/share/links: UpdateDownloadMetaIn / ShareDownloadIn / CreateLinkIn /
UpdateLinkIn; _link_expiry takes an int|None instead of digging the payload.
PATCH endpoints read model_fields_set so an explicit null still clears a field.
Gate: ruff clean, pytest 164 green.
Round-1 review finding #4 named only audit.py/plex.py, but the same misplaced
`from app.utils import iso` sat in channels/downloads/feed/notifications too — the
round missed them (ruff enforces no import order here, so the gate was silent).
Move them to alphabetical position for consistency. No behaviour change.
Establish the R6 API-contract infra before the S1b payload:dict migration:
- iso(dt) in utils.py replaces the ~16 hand-rolled `x.isoformat() if x else None`
serializer idioms across the routes.
- get_or_404 / owned_or_404 in routes/_common.py — one 404 + existence-leak policy;
plex's _own_playlist_or_404 and saved_views' _own_view now delegate to it.
- require_write_scope dependency in auth.py collapses the inline has_write_scope→403
checks (channels subscribe/unsubscribe, playlists push/push-plan) that had drifted
into three different messages into one; the conditional delete_playlist guard stays
inline (its message was already canonical).
No response-shape change (iso == isoformat for a present value). Gate: ruff clean,
pytest 163 green (+ test_utils for iso).
- C-3.7: fold a per-link password_version into the signed watch grant (migration
0059 adds the column) and bump it on every password change, so rotating a share
link's password invalidates outstanding grants at once instead of letting them
live out the 6h TTL. make_grant/check_grant take the version; update_link bumps.
- C-3.8: the share recipient picker and share-by-email now reuse the messageable
filter (not-demo AND active AND not-suspended) instead of only excluding demo,
so suspended/deactivated addresses aren't leaked or reachable as targets.
- C-3.9: GET /keys/{user_id} adopts get_thread's MB2 guard — a non-messageable
target with no shared history returns the same "User not found" as a missing id,
so it can't be probed to enumerate users or fetch suspended users' keys.
- C-3.12: _register_account catches IntegrityError on the insert (the select-then-
insert TOCTOU) and treats it as the already-registered no-op.
- C-3.13: trigger_job claims the running flag atomically under _activity_lock
(compare-and-set) instead of a bare check, closing the double-start window.
- Move _messageable/is_messageable_user into a shared app/userscope.py so downloads
doesn't import the messages route module.
Follow-up to user prod testing of 0.51.0:
- re-download now clears job.display_name so the card title + Content-Disposition filename are
re-derived from the freshly-cleaned title (previously an auto-name from an older un-cleaned
title — still carrying a "… views · … reactions |" prefix — survived the rebuild).
- on-disk AND served names are now a strict shell-safe slug: only [A-Za-z0-9._-] survive; every
other char (the "!" the user saw, plus ( ) [ ] & $ ' " ; | ? * …) collapses to "_". The id no
longer uses [brackets] (glob metacharacters). Shared _slug helper for sanitize + display_filename.
- GC gains a 5th pass: sweep on-disk files/dirs no ready asset owns (old accented channel dirs, a
poster from a superseded naming scheme left after a re-download), skipping the dot system trees
and anything newer than an hour so an in-flight download is never touched.
- Tests: slug strips !/$/()[]; rel_path is bracket-free; orphan-sweep keeps owned/system/fresh files.
peter