Compare commits

..
106 Commits
Author SHA1 Message Date
peter 891507dc75 release: 0.105.0 (feature/e38-ad-hoc-clips) 2026-10-10 23:38:13 +02:00
peter e6894ac0fc release: 0.105.0 — ad-hoc clips 2026-10-10 23:13:22 +02:00
peter 663a7e8012 fix(player): the Plex player's fills hold on its chrome: an amber head on a white played part
A light theme's accent as a fill on the always-dark chrome was ~2.8:1 (Slate), under the 3:1 a
control needs. A native checkbox and slider tint with `accent-chosen` (the accent elsewhere, the
media amber under `media-marks`), and so does the skip button's countdown. The progress bar plays
white with an amber head: amber there marks a place, as the intro / credits markers do, and an amber
played part made a marker ahead of the head read as already played (owner's choice of three).
2026-10-10 22:56:09 +02:00
peter 93192dadd8 fix(player): a chosen tab's underline and an open or pressed icon read the scope's chosen colour
The underline named `--color-chosen`, which is resolved once on :root, so it stayed the accent under
`media-marks` (seen in the browser: fuchsia on Slate light); it reads `--chosen` where it is used
now. An icon button pressed or with its menu open tinted its glyph with the accent; it takes the
chosen colour — the accent everywhere but the player chrome, where it is the media amber.
2026-10-10 22:26:22 +02:00
peter 5b48018e92 fix(downloads): a produced file is held to the space left by its real size
yt-dlp's `max_filesize` only stops a plain HTTP file that announces its size: YouTube's fragmented
formats never meet it — measured on localdev, a 718 MB download ran to the end with 1 MB left and
put its owner four times over the cap. The download now re-reads what is left once its file is made
(after the phone-safe pass) and refuses one larger than that as the owner's refusal; an editor clip,
which nothing checked, is held the same way, as a player clip already was.
2026-10-10 22:10:56 +02:00
peter 8312c6d592 fix(player): what is chosen on the player chrome is marked amber, not written in the accent
A light theme's accent as text on the always-dark chrome was 2.84:1 (Slate). `--chosen` is the
accent unless a scope names its own; `media-marks` (was media-focus) points it, with the focus
ring, at the media amber. A chosen track stays white with an amber tick (MenuCheck reads
`text-chosen`); a chosen settings tab or toggle is white with an amber underline (MEDIA_CHOSEN).
The theme contract test holds the token, the tick and the player components free of accent text.
2026-10-10 21:56:47 +02:00
peter e1d325b942 fix(downloads): a download larger than the space left says so, not "processing failed"
yt-dlp refuses a file announced over `max_filesize` by logging one line and returning with no file
(measured, and pinned in test_formats against a local server): the worker found no output and
failed the job as `postprocess`. `formats.watching_size_cap` now sees that line through our yt-dlp
logger, and the download (and, should it ever happen, a YouTube clip's range fetch) fails as the
owner's refusal with DOWNLOAD_TOO_BIG_MESSAGE, classified `quota_exceeded`. The download's quota
copy says it does not fit in what is left — true at a full cap too — instead of "Storage full".
A pre-existing type error the change touched is fixed (baseline down by one).
2026-10-10 21:56:47 +02:00
peter 977ba244fa fix(clip): a time's error line sits in the flow, the remove button off the cut, and every player surface rings amber
- useTimecodeField holds a time field's draft for its caller, which places TimecodeError (role
  alert) in its own flow under the fields' row: it no longer lies over the crop row taking its
  clicks, runs out of the editor's card, or joins the field's <label> name.
- The editor's remove-cut buttons sit in a row under the track at their cut (trackLeft), off the
  cut's grab area, where a press drags it; RangeTrack's adornment slot is gone.
- media-focus re-points --focus to the media amber on the Plex player, the YouTube stage and every
  bg-media-chrome surface; a contract test holds each such class string to media-focus or
  media-scope, the field's focus edge to the focus token, and --color-focus to the inline theme.
- An upload's quota headline says the converted file does not fit, not "Storage full".
- DESIGN.md: the tokens paragraph says what is re-pointed where; the decision row measures all
  three colours on the chrome.
2026-10-10 21:29:55 +02:00
peter 89d18b46a6 fix(downloads): a converted upload too big says so, and a suspended owner's waiting rebuild is not "preparing"
- A converted upload over the space left (not zero) fails with CONVERTED_TOO_BIG_MESSAGE, not
  "storage is full"; it classifies as quota_exceeded by the phrase it shares with the clip's.
- The public watch page says "preparing" only for a job a worker will take: a suspended owner's
  queued job waits for as long as the suspension lasts, so the link says the video is gone, as
  for a paused rebuild.
- The worker's claim docstring names the restart case: a running job put back in the queue waits.
2026-10-10 21:29:55 +02:00
peter c27cfdcb9c fix(clip): typed times belong to their value, and every mark on the player chrome can be seen
TimecodeField drops a draft when the value moves under it (another segment selected, a nudge), so
leaving the field never commits it to another cut; it is invalid only once offered, with a line
under it saying how to type a time. On the always-dark chrome media-scope points every focus ring
at --media-mark and errors at the new --media-danger (a light theme's accent and danger were under
3:1 there). A track whose dragged handle unmounts ends the drag on the pointer's release instead of
locking up; a remove refused while clips are queued leaves no focus to take later; each remove-cut
button names its cut and is a 24px target. DESIGN.md records the media tokens; stale comments fixed.
2026-10-10 20:28:52 +02:00
peter a867145bcb fix(downloads): a suspended user's queued jobs wait, and a clip too big for the space left says so
A suspension now blocks the user's downloads as quota.resolve already advised: the worker leaves
their queued jobs unclaimed until it is lifted (the confirm says so). A cut clip larger than the
remaining storage is refused with its own message (not "storage is full"), and a clip's quota
refusal reads as a clip's in both languages. The cancel poll compares to JobStatus.RUNNING.
2026-10-10 20:28:52 +02:00
peter 4d4eea2c54 fix(clip): marks on the player chrome keep their amber in a light theme, and a removed cut hands the focus on
- A media amber (`--media-mark`), the same in both themes, for what marks a place on the always-dark
  chrome or over a video: the clip strip's playhead and focused end, the crop frame's ring, Plex's
  intro / credits markers. A light theme's warn was a dark amber there, under 3:1 (contract-tested).
- The editor's remove-cut button rings inside itself (the track clipped its ring), and once it goes
  with its cut the focus moves to the cut beside it, or to Split here, not to <body>.
2026-10-10 15:57:02 +02:00
peter c489b60c8b fix(clip): clip mode keeps its range, its focus and its keys through the player's own moves
- A mark on the very end seeks a moment short of it; while a clip is marked the end of the item
  neither closes the player nor steps the queue, and Plex's auto-skip waits.
- Plex: Escape and Back peel what is over the picture (info, up-next, the countdown) before the
  clip; Back under a dialog is the dialog's; no range opens while the length is unknown.
- YouTube: the focus goes back to the scissors the user can see, else the player; the panel keeps
  its name, Loop and focus when the view is maximised or restored under it.
- A typed time stays in its field until taken, then shows where it landed; unreadable stays marked
  invalid; a decimal comma is read (ui/TimecodeField, the editor's fields too).
- RangeTrack: a drag is its pointer's, a cancelled drag reports final, an adornment sits beside the
  slider, a caller-chosen focus ring (amber on the clip strip). CropFrame: a drag is its pointer's.
- The editor's remove-cut is a real button at the cut's foot; a segment's keep toggle sits over the
  cut's hit area.
2026-10-10 15:35:21 +02:00
peter b4bf71ea64 fix(clip): a clip is held to its owner's storage cap, and its limits and failures say the server's truth
- The cut file is measured against what its owner has left, for a YouTube range (whose fetch never
  reads yt-dlp's size ceiling) and for a Plex clip (which fetches nothing), before the fetch and
  again before it is placed; a refusal leaves the asset pending for a Retry.
- A silent Plex file is cut silent; a file the probe cannot read says so.
- A second job on a failed or renamed clip gets the same YouTube diagnosis and the fetched title.
- The "too long" refusal names the server's cap; a contract test holds the client's copies of the
  clip limits to clip.py; the shipped player clients are pinned against the installed yt-dlp.
- One HDR transfer set and one rating_key lookup; job status words from JobStatus; the display-name
  guard sees subscript writes and the clip route; stale comments and types follow the clip job.
2026-10-10 15:35:12 +02:00
peter 72df11f2c0 fix(ui): a small crop frame takes smaller handles, so it can still be grabbed to move
The handles reach 10 px into the frame from every side; a frame at its minimum on a small picture
(the Downloads editor at 800 px: 41 x 23) was then nearly all handle - 74 of its 920 px still moved
it. Below 64 px on its shorter side the frame takes 12 px corners and 8 px edges: 262 px move it.
2026-10-10 13:54:04 +02:00
peter 0a16bc40ae fix(ui): a crop frame grabbed with the pointer takes the focus, so the arrows go on from there
The press is held back from the page (no text selection, no play / pause underneath), which also
kept it from focusing: after a drag the arrows still went to whatever had the focus before, and
Escape did not reach the frame. The press now focuses the frame or the corner it grabbed (an edge
focuses its frame).
2026-10-10 13:38:49 +02:00
peter 520b0afb90 feat(clip): crop a clip to a frame drawn on the picture, and give the clip panel the room it needs
An optional crop in clip mode: a Crop toggle puts a frame on the picture (free, 16:9, 9:16 or 1:1;
the whole picture stays the default). The frame is sent in fractions of the box it was drawn in,
with that box's shape; the worker fits the real picture into the box by its display aspect (sample
aspect and rotation aware), drops the part over the bars, and crops first in the filter chain. A
frame covering the whole box is no crop at all, and one off the picture is refused by name.

The crop frame is a ui primitive (CropFrame) shared with the video editor, keyboard-operable.
YouTube: the wide side panel shuts while clipping and comes back after; maximised, the clip panel
docks in the stage. Plex: the picture sits above the clip panel so the frame lies on it.
2026-10-10 12:52:59 +02:00
peter e301632765 fix(clip): a clip refusal speaks the user's language, and a choice in the clip form no longer sends it
The backend refuses a clip with a structured reason, but the client showed every one as the generic
"couldn't make the clip": each reason now has its own message in both languages, held to the
backend's reasons in both directions by the contract test. A segmented toggle's and the priority
stepper's buttons had no type, so inside a form one of them submitted it.
2026-10-10 12:52:12 +02:00
peter 50157cc2fd feat(player): clip mode - mark a range of what is playing and cut it into a download
Both players get a scissors in their control row (Plex: the control row; YouTube: the actions and
the docked bar), `c`, and `[` / `]` to mark at the playhead. The clip panel lives in the player, so
the watched video is the preview: an overview of the whole item and a zoomed strip around the
selection whose ends are sliders (arrows step a tenth, Shift a second), Start / End fields with
nudges and "to the playhead", the length, a looping Play selection, a name, Cut. Escape closes the
panel before the player and the focus goes back to the scissors. A cut is a toast with a Downloads
link plus the usual Ready notification.

- ui/RangeTrack + RulerTicks: the shared, domain-free time track (regions, keyboard-steppable
  handles with a 1.5rem hit area, a playhead, a background slot, over a window of the item). The
  Downloads editor's cuts moved onto it, so they are keyboard-steppable now.
- lib/clipRange holds the marking rules (1 s to 10 min, the server's), lib/timecode the editing
  timecode the editor's private tc/parseTc became.
- keymap: a symbol typed through AltGr answers its bare-symbol row - on the HU layout `[` is
  AltGr+F, which arrives as Ctrl+Alt; letter and Mod rows still never match Ctrl+Alt.
- The YouTube stage sits in a wrapper with the panel, so the maximised stage's blur escape matches
  at any depth.
- media-scope: fields on the always-dark player chrome read white in a light theme too.
2026-10-10 03:58:27 +02:00
peter f221c1962a fix(downloads): a YouTube clip goes with the user's own cookie and is diagnosed like a download
A clip fetched its range signed out, so a video behind YouTube's per-video "not a bot" wall could
never be clipped while its download went through with the user's cookie - and the failure copy told
the user to add a cookie the clip would not use. The clip asset is private by construction
(`clip.asset_ref` names its user), so `usable_clip_cookie` attaches the user's YouTube cookie on that
identity, and a failed range fetch now carries the download catalog's diagnosis
(`youtube.bot_wall` / `youtube.cookie_rejected`) instead of a bare category.
2026-10-10 03:58:27 +02:00
peter d466ae915b fix(downloads): a YouTube clip outside the catalog takes its title from the fetch
It was queued as "Clip (clip)" and stayed so: nothing renamed it once yt-dlp had answered.
2026-10-10 02:43:13 +02:00
peter 14598ec320 feat(downloads): ad-hoc clips cut straight from a YouTube video or a Plex item
POST /api/downloads/clip takes a source (a YouTube id or a Plex rating key), a range of at most
ten minutes and, for Plex, the audio track being heard. The clip becomes a per-user asset in the
library like an editor clip, shareable as soon as it is ready. A YouTube clip fetches only its
range (yt-dlp download_ranges: 3-7 s whatever the video's length); a Plex clip reads the library
file in place. Both then go through one re-encode: frame-exact at the marks, H.264 at most 1080p
with HDR tonemapped to SDR, stereo AAC, no chapter track, .mp4.

Job kinds are a JobKind StrEnum now, and Re-download refuses every kind but a download.
2026-10-10 02:31:38 +02:00
peter b16cf7668c fix(downloads): YouTube downloads reach their profile's height again
The primary player client, android_vr, was still offered 1080p and 2160p, but every fetch of
those formats died with HTTP 403: it now needs a stream PO token the bgutil sidecar cannot mint.
The fallback set then served 360p only, so every download finished at 360p without failing.
tv_simply is offered the full ladder and its fetch completes (measured: 1080p, and a localdev
download through the pipeline reached the default profile's 720p). yt-dlp 2026.8.19 marks
the clients that need such a token, so it stops fetching into the 403.
2026-10-10 02:08:43 +02:00
peter 3438aeb695 fix(player): the player's own focus grabs leave a tabbed-to control alone
Turning ready, the pointer leaving the video, the window regaining focus, a wheel over it and
leaving fullscreen each put the focus back on the card unconditionally, so a viewer who had
tabbed to a control right after opening started over from the first button on the next Tab.
They now take it back only when it sits in the iframe or off the card.
2026-10-10 01:14:31 +02:00
peter c003a48244 release: 0.104.0 (improvement/adhoc-batch-oct) 2026-10-10 00:24:27 +02:00
peter 8a2334e198 release: 0.104.0 notes and version 2026-10-10 00:15:23 +02:00
peter 1f010328df fix(plex): plain text survives a decimal reference too long to parse
int() refuses a decimal run past 4300 digits, so one such reference in a summary raised out of the sync and the text migration. Any number past seven significant digits is beyond U+10FFFF, so it becomes U+FFFD like any out-of-range reference; leading zeros are dropped first. The rails test's comment now says what it guards: both rails agree with the rollup.
2026-10-10 00:03:08 +02:00
peter 77638f7a0b fix(player): a link that opens the cut text is scrolled back into view
Focus had scrolled the clamped box itself; opened, the link dropped by that much, out of a scrolling panel's sight. DESIGN states where Close sits in the stacked layout; a stale test comment goes.
2026-10-09 23:46:40 +02:00
peter 2cff4ad954 fix(feed): an override yields only to a refetch that set out after its write was answered
Send order let a refetch read before an unanswered mark committed and drop it. A reset's position 0 is written again after a refetch that set out before the server took the reset. The player-session refetch also waits for the player's closing checkpoint, which the player now tracks (progressFlushes); refetched first, the list put the older position back.
2026-10-09 23:46:39 +02:00
peter 8776586ca3 fix(plex): a matched episode and a collection member offer Details in their menu
Their titles open the item's page, a pointer-only door; the card menu is the keyboard's.
2026-10-09 23:46:39 +02:00
peter 92a17aba83 fix(plex): a reshuffle survives a refresh landing mid-deal; the ↻ spins for a deal only
The ↻ intent was spent as its fetch set out, so a watched toggle's resync cancelling that fetch re-ran it as a re-read of the old draw. It now lasts until the latest deal answers. The button spun on every re-read of the same titles; it now follows the deal. Comments that said the Surprise draw is never refetched are brought in line.
2026-10-09 23:46:39 +02:00
peter fc53bc03bc fix(plex): the rails' Restart comes from the one rollup; plain text decodes whole entity names only
Both Continue-watching rails built a show's card state inline; they now take it from rollup.show_state, so a show part watched with nothing to resume offers Play alone there too. A named reference counts only when its whole name is an HTML5 entity: html.unescape still decoded the legacy "&not" inside "&notice;". Tests cover the grid's in-progress-but-not-resumable case through its SQL and both rails.
2026-10-09 23:46:39 +02:00
peter 6df257d7df fix(plex): Restart only where an episode has a resume point; the Surprise draw is re-read, not redrawn
Show cards carry resumable (ShowState, from the same rollup) so a part-watched
show whose next episode is untouched offers Play, not a Continue and a Restart
that do the same thing; plexStarted is the one rule for cards and menus. The
Surprise rail's key sits under the discover root: whatever refreshes the rails
re-reads the draw (keep=) with its watch state now, kept past the 5-minute gc,
and only the reshuffle deals. Closing the player refreshes the Discover rails.
A show card's play acts on the last click only, and only where it was made.
2026-10-09 22:01:28 +02:00
peter 5572499b56 fix(player): the shut panel keeps the dialog's name; Tab onto a cut link opens the text
With the side panel shut the title heading was gone, and the dialog it named
went unnamed: the strip carries an sr-only copy. A link below a clamp's cut
stayed tabbable out of sight; reaching it now opens the text, measured from the
text's top since focus scrolls the clamped box. DESIGN describes the modal as
built, with log rows for its chrome and the wheel rule.
2026-10-09 22:01:27 +02:00
peter 76738f8ea9 fix(plex): plain text decodes only well-formed character references
html.unescape also decodes the legacy forms without a semicolon, so prose like
"AT&notice" came out as "AT¬ice". The show and collection sync paths are
now tested too.
2026-10-09 22:01:27 +02:00
peter 568bf93510 fix(feed): an optimistic write outlives only the server answers older than it
A reset wrote a position mask over the cached list, so a replay's saved
position stayed hidden: a reopen resumed at 0 and its first checkpoint wrote
over the progress. The reset now writes 0 into the cache, where the player's
closing patch writes the next one. Overrides carry their write order, and a
landing refetch drops only those written before it set out. The player
session ends after the modal unmounts, so a clip auto-marked watched as it
closes still refetches the list.
2026-10-09 22:01:27 +02:00
peter 942154cf49 fix(plex): a Surprise draw stays put until its reshuffle
The rail's query was always stale, so Back from a card's player or page remounted it into a new
draw, as did a return to the window: the card just looked at was gone. It now stands until the
reshuffle (or another scope, language or filter) asks for a new one.
2026-10-09 01:29:12 +02:00
peter 3eb87e7108 feat(plex): every show card plays the episode it would put on
A show card's picture opened the show page everywhere but Home, whose shelves carried the
resolved `next`. Now every Plex picture plays: a show asks `GET /api/plex/show/{id}/next`
(the in-progress episode, else the first unwatched, else S1·E1 — the rule Home and the show
page's Resume follow) on the click, fresh each time. A started show shows Continue + Restart;
its title, an ⓘ and the menu's "Open show" row open its page; one with no episodes opens there.

The question is asked per show, never carried on every card: three probes a show would cost a
grid page up to 180 queries. The hover preview asks it once it opens and names the episode
("Next: S6·E16 — Title"). Related shows on a show page play the same way.
2026-10-09 01:22:00 +02:00
peter 851b97b09d fix(ui): a card's hover chrome answers keyboard focus, not any focus
Back from a channel page hands the card its focus again after a mouse trip;
revealed on focus-within, its hover row stayed up under no pointer until a
reload (owner). The row, the Play layer, the Continue + Restart pair and the
global .reveal rule now show on hover or on :focus-visible on the card or
inside it, so Tab and the arrow keys still reveal it.
2026-10-09 00:44:46 +02:00
peter ee94d1949a fix(feed): a video marked watched in the player stays out of an Unwatched view
Marked watched in the player (by hand, or by the player at its end), a card
left an Unwatched list at once, then came back as "new" when the player
closed, until a reload. The player's closing position patch writes into the
cached feed; the feed took the bumped dataUpdatedAt for fresh server data,
dropped its optimistic overrides, and nothing refetched. The overrides now
clear only when a refetch lands, and a player session that changed a state
refetches the list and its count by the view's filters on close, once its
writes have landed.
2026-10-09 00:44:46 +02:00
peter 4f076b5d45 feat(feed): a card's menu filters the feed by its channel, and back
Under "Open the channel" a video card's menu (the right-click on its
channel name included) adds the channel to the feed's channel filter, or,
when it is already in, takes it out. Channels OR together, so it widens a
filter that names others, as "Show in feed" does. Not on a channel page,
whose view is that channel already, nor in a live YouTube search or on Home.
2026-10-09 00:44:46 +02:00
peter ece74a42ab feat(player): a roomier YouTube player, its panel shut on demand
- The full-height previous/next strips beside the card are gone: the panel's
  transport row and Shift+arrows step on their own, and the video gains the
  7rem the strips' gutters took.
- Close is one icon in the card's top-right corner (the wide panel's head,
  the stacked card's title row).
- The panel's description is cut to three lines with More / Less
  (ClampedText now takes the linked text as children).
- Wide, the panel shuts to a 2.75rem strip (close, open, previous, next) for
  a bigger video; it stays shut while the queue steps on, and a video opened
  from a card starts it open. Focus moves to the counterpart toggle.
- The wheel has one meaning per place: over a panel (or the stacked card)
  with something to scroll it only scrolls, never falling through to the
  volume at the end; over the video, or a panel that fits, it is the volume.
- A long channel name is cut on the panel's line instead of running the
  20rem panel into a sideways scroll.
2026-10-09 00:27:29 +02:00
peter 600d5c39da feat(ui): a button over a picture answers the pointer with an accent ring
The media variant's hover darkened its body from 60 to 75% black, which no
one could see on a poster; a rail's scroll arrow over a card could not be told
from the card's own Play under it. Its hover is now a 2px accent ring over an
80% body (owner pick B), on every card control, Plex and YouTube, and the rail
arrows. DESIGN.md B3 names it beside the edged pills, with its log row.
2026-10-09 00:27:28 +02:00
peter 4c7f73af9d feat(plex): a title opens its page, the picture plays; an episode links up
A card title lit like a link but played the item. Now the picture plays and
the title opens the page of what it names: a movie's or an episode's info
page, a show's page, and for an episode standing as its show (Recently
watched) the show. Every poster and episode card carries it (grid, rails,
person, collection, season, related, Home), a card with no page keeps a plain
title, and the season page gains the menu's Details row too.

The episode page had no way up: the detail now serves the show's and the
season's rating keys, and on the page the show title opens the show and
"S1·E25" the season (Back returns to where the page was).
2026-10-09 00:06:18 +02:00
peter 5f42ff61c8 feat(plex): a Surprise reshuffle deals the previous draw last
The rail draws three unseen titles per kind under the sidebar filters; on a
small filtered pool every reshuffle dealt the same few again. The redraw now
sends the draw on screen (exclude=), the server deals the rest of the pool
first and tops up from the excluded titles only when the pool is short, so a
small pool still fills the rail. (The two discover.py baseline entries went
with the rewritten draw.)
2026-10-09 00:06:11 +02:00
peter 6dd82ef5e2 fix(plex): summaries and taglines are stored as plain text
A Plex agent can hand a summary over as HTML (<p> paragraphs, &aacute;
entities); the info page and the hover card showed the tags. One cleaner,
app/plex/plaintext.py, now runs on every summary and tagline the sync, the
Hungarian overlay and the rich-detail fetch store, and migration 0101 cleans
the rows written before through the same function. Paragraph breaks survive
as newlines, which the info page and the hover card now keep.
2026-10-09 00:06:02 +02:00
peter 75f4efde98 release: 0.103.0 (bug/yt-fullscreen-button) 2026-10-08 22:50:52 +02:00
peter c0059a0c4b release: 0.103.0 notes within the item length cap 2026-10-08 22:36:15 +02:00
peter a29cadf03e release: 0.103.0 notes and version 2026-10-08 22:30:56 +02:00
peter 83886494ce fix(ui): a membership menu re-homes focus a frame later, once its new rows are on the list
Rows that just mounted join the menu's navigation list a render after the status effect ran: a row
focused that early was unknown to it, and the arrows did nothing from it (seen live after a keyboard
Retry). The retry test now arrows on from the row it lands on.
2026-10-08 22:16:47 +02:00
peter 4eaa55385f fix(plex): a collection kept after a failed refresh holds its seed title
When Plex made the collection and only the read after it failed, the kept row had no members: the
title showed unticked in the collection it had just been added to, the collection empty, and the
name stayed in the field for a retry that could only be refused as taken. The seed's row now names
the collection (child_count 1), and the panel clears the field on that refusal as on success. Also
held: the panel's lines sit beside its list, and a keyboard Retry lands on the first row.
2026-10-08 22:12:43 +02:00
peter 7541b1b016 fix(ui): membership menus keep their lines beside the list and focus in the panel
The loading / failed-load lines and their Retry sat inside role=menu, which may own its items only:
the three surfaces now open with `listInside` and the rows sit in a PopoverList. A keyboard Retry
unmounted the focused button and left focus on <body> with the menu open, the arrows dead: focus is
put back on the first row, else the field, else the new Retry. The channel tag editor said a failed
tag load as "no tags yet"; it reads the catalog query's state now. The playlist panel's name and the
tag editor's error state are held by tests.
2026-10-08 22:12:43 +02:00
peter 24ad17fc7d test: hold the show collection's type, an unknown title's 404 and Brave's fullscreen stand-in 2026-10-08 21:18:48 +02:00
peter 48bd038c5f fix(plex): a collection created on Plex keeps its editable row when the refresh read fails
The rollback dropped the new row: the next sync recreated it read-only, out of every 'add to' list, and the duplicate-name check let a retry make a second same-named collection on Plex.
2026-10-08 21:18:48 +02:00
peter 0fce732497 fix(plex): a playlist row's menu adds its title to a collection too
The row menu called the title's action list without the anchor the panel opens beside, and the optional parameter dropped the row silently. The anchor is now required, so every host menu decides.
2026-10-08 21:18:48 +02:00
peter 7e94da72b9 fix(ui): one membership menu for playlists, collections and tags
Three hand-copied panels had drifted: Add to playlist said a failed load as 'no playlists yet'. ui/MembershipMenu now holds them: a loading line and an error with Retry instead of a blank or an empty list, the up/down arrows step from the name field into rows that arrived after the menu opened, and the panels opened from a menu carry their own accessible name. A write that landed flips its tick at once, so a second tap before the refetch undoes it instead of repeating it; a collection edit also refreshes a show's own page.
2026-10-08 21:18:48 +02:00
peter fadb4f17a9 fix(plex): subtitles draw only the player's own track, lifted in the player's own pixels
A direct-played file's own subtitle stream (WebKit lists an mp4's mov_text) was switched to hidden and drawn under any choice, Off included: only the player's <track> is kept hidden and read, other subtitle tracks are switched off. The lift was measured in screen pixels and applied inside the x1.25 windowed player, and re-measured off a box carrying its own (or a mid-slide) transform: an untransformed outer box is measured, the distance divided by the scale, and a cue that just appeared is placed, not slid from where the last one was left.
2026-10-08 21:18:48 +02:00
peter 57a81c5643 fix(plex): subtitles survive hls.js and a remounted track
hls.js emptied the cues of every text track on the element when a manifest
loaded or the media attached, our own <track> included, so a VTT that loaded
first showed nothing for the session; its subtitle and caption controllers are
no longer built. A remounted default <track> was switched to showing by the
browser after the hide had run, drawing the cues natively under ours; the hide
now follows every added track and mode change.
2026-10-08 19:35:47 +02:00
peter dae0b80f6a feat(plex): add a movie or show to a collection from its card menu
"Add to collection…" opens a panel beside the card: the title's library's
manual collections tick it in or out, and a name + Enter creates a new one with
the title in it. It used to take the Collections page, a new collection, its
name and a title search, then another search per title. A lean endpoint lists
the collections with whether each holds the title. Admin only, as on the
title's page.

Also: a collection seeded with a show is made as a show collection (Plex type
2). Measured: as a movie one it came out empty and refused the show after.
2026-10-08 03:09:55 +02:00
peter 75093458c9 feat(plex): subtitles rise above the control bar while it shows
On an iPad the bar covered the text whenever it was up: the browser places a
native cue by its line, which nothing animates. The track now stays loaded as
hidden and PlexSubtitles draws the active cues, measured against the bar and
lifted above it in 200 ms, back down when it hides. The viewer's size, colour,
background and outline settings style the text directly. Also fixes a test
typing slip from the search pill change.
2026-10-08 02:59:13 +02:00
peter 5db6dc8ee6 docs(design): log the touch header search 2026-10-08 02:52:39 +02:00
peter 45a9ceae25 fix(ui): the header search has room on a tablet
On touch the 44 px key cap and Go filled a 44 px pill edge to edge and the
placeholder was cut. The key cap is left out on touch (⌘K still works), the
pill grows to 52 px so a ring shows round Go, the box's floor counts the
controls beside the text, and Go drops to its icon by the column's width,
not the screen's.
2026-10-08 02:52:32 +02:00
peter 5249c41edc fix(player): a fullscreen button the browser cannot honour is not shown
Brave on iPadOS switches element fullscreen off, patches requestFullscreen with
a video-only stand-in and fakes fullscreenEnabled, so the Plex fullscreen button
and the YouTube docked bar's did nothing. Support is now read off
Element.prototype and the buttons are left out without it (owner). A stand-in
that returns a boolean no longer throws. A test's bare matchMedia stub no
longer outlives its describe.
2026-10-08 02:50:51 +02:00
peter c3664826fe fix(ui): a caption never opens under a finger
On an iPad a tap's synthetic mouseenter opened the Tooltip; WebKit then took the
tap as a hover and swallowed its click, so every rail button needed two taps,
and with no mouseleave from a finger the caption stayed up. Tooltip now opens
for a mouse and a key's focus only (owner: no captions on touch).
2026-10-08 02:45:48 +02:00
peter c9b8fe15a3 fix(ui): a scroller with no edge to fade carries no mask
iPad WebKit kept a scrolled container's mask stuck to its content: at the
bottom of a scrolled page the last strip, a table's pager, sat under a dark
band. With nothing to fade there is no mask; isolation keeps its layer.
2026-10-08 02:43:58 +02:00
peter 1044eaf8a9 fix(plex): the player holds the page still and covers it whole; the row drops the countdown
On an iPad a drag on the video scrolled the page behind the player, and in Brave
dvh did not follow the toolbar, so a strip of the page showed under it. The
page is now locked without overscroll on a black root while the player is open,
and the player is sized from its fixed box. The time row keeps the position,
the length and the end clock; the countdown is gone (owner).
2026-10-08 02:42:31 +02:00
peter ca949d22de fix(player): the channel line wraps instead of squeezing the meta into a column
On a narrow card (a tablet, a zoomed window) the unwrapped row of name, meta and
actions left the meta a one-word column. The row now wraps: the meta keeps at
least 12rem, and the actions move down together.
2026-10-08 02:25:58 +02:00
peter 20178f0ec4 release: 0.102.0 (improvement/e37-player) 2026-10-08 01:58:01 +02:00
peter 49a9319f84 fix(player): the click layer clears only the fullscreen button's column, not a full-width band
Measured: across the full width YouTube needs only its seek bar (up to 96 px
from the bottom); the 140 px is the fullscreen button's, at the right end. A
full-width 140 px band handed clicks on the picture to the iframe, which then
takes the keyboard focus and our shortcuts with it. The layer is now two boxes,
hidden from assistive tech (a mouse surface; the keys are its shortcuts), and
the tests render the big view too, where the docked bar sits under the iframe.
2026-10-08 01:42:39 +02:00
peter 70d56ab939 fix(player): YouTube's fullscreen button stays clickable on a short player
The click layer over the embed left the bottom 22 % of the stage free. YouTube's
new embed keeps its fullscreen button 80-128 px above the iframe's bottom, so on
a stage under ~580 px (a small window, a browser zoomed to 150 %) the layer lay
over it and a click paused instead. The cutouts now have px floors (56 top, 140
bottom) and are measured against the iframe's own box.
2026-10-08 01:36:36 +02:00
peter 3865689c26 chore(release): 0.102.0 2026-10-08 01:33:39 +02:00
peter c32429ba69 fix(plex): an episode with no number is named by its title on every card
The cards, the up-next card and the mark-next label printed a question
mark where Plex has no number; the player's header already showed the
title alone. One helper (episodeNumbers/episodeName) for every place.
2026-10-08 01:16:30 +02:00
peter a504550196 docs: the superseded fallback answers its own session; who uses beacon() 2026-10-08 01:16:30 +02:00
peter 5095047c81 test(player): pin the stop URL's ids, the beacon's settle, the kept point's edges
The stop-URL contract passed with the two ids swapped and read the backend
from the working directory; nothing called a real beacon's onSettled or its
library pass-through; the kept-point rule was held by its constants only.
2026-10-08 01:16:30 +02:00
peter 4e70f168b1 fix(player): a layer that opens by itself takes no history entry
The auto-skip countdown and the up-next card pushed an entry without a
user gesture (Chrome's Back button skips those), and one pushed under the
open shortcuts sheet sat above it, so Back closed the hidden layer.
2026-10-08 01:16:30 +02:00
peter 7f17f488ab fix(nav): a sub-view step closes the overlays standing over it first
Leaving the Plex player while one of its layers was up (the info panel when
the film ended, the countdown under the back arrow) landed as a user Back:
the overlay listener spent it closing the top layer and the player stayed.
useHistorySubview's open/back now go through closeOverlaysThenNavigate,
like setPage. The RECLOSE comment no longer cites the shared Plex entry.
2026-10-08 01:16:30 +02:00
peter d1704a5519 revert: the Modal Escape guard for a fullscreen own-content video
Chrome does not hand the page the Escape it spends on leaving fullscreen (checked with a real
keypress: the first Esc leaves fullscreen, the second closes the player). The guard defended a case
that does not happen, under a comment saying it does. This reverts 121ebeeb.
2026-10-07 23:59:32 +02:00
peter 121ebeebc1 fix(ui): an Escape while a dialog's own video is fullscreen does not close the dialog
The page gets the Escape keydown with the video still the fullscreen element. When that element is
the dialog's own content (the library player), closing the dialog took the player away with the
fullscreen. A dialog opened over something else that is fullscreen still closes.
2026-10-07 23:57:04 +02:00
peter ee5290ba01 refactor(plex): the item progress body extends the shared checkpoint
ItemProgressIn repeated ProgressIn's config, fields and bounds; it now adds only `final`. The
threshold comment points where the constants live now, and plexProgressBeacon's comment is back
on it.
2026-10-07 23:16:38 +02:00
peter dd6a666ad1 test(player): hold the session stop and the kept-point rule to the server
- The client's resume thresholds equal the server's (read from both sources).
- The session stop's URL matches the route the server serves it on.
- The player stops its server session by its item and its own id on an unload and on close,
  and keeps it when the page only goes into the back/forward cache.
2026-10-07 23:16:38 +02:00
peter 392099770a fix(plex): an episode Plex has no number for is named by its title alone
The OS media title printed the missing number as "Enull", the player and info headers as a bare
"E". One helper for the three lines.
2026-10-07 23:16:38 +02:00
peter 2aa3ac6e85 fix(player): the browser Back peels each layer over the Plex video, one press at a time
All the layers shared one history entry, so the first Back spent it on one layer and the second
left the player with the stats readout (or the countdown) still up. Each layer now holds its own
entry; Back closes the one opened last.
2026-10-07 23:16:38 +02:00
peter 891dc628f3 fix(library): a listing poll can no longer put back an older resume point
A poll already out when the player closed answered with the point the server held before the
closing beacon, over the row's patch; Continue then opened there and wrote it back. The close now
cancels the listings' fetches before patching and reads them again once the beacon has landed.
2026-10-07 23:16:25 +02:00
peter 0a35f2a3b1 fix(player): a stale session answer can no longer stop the playback that won
- The engine tracks the request out per item: A, then B, then back to A sent a second request
  for A beside the one still out.
- A hardware transcode superseded during its fallback answered with the NEWER session; the losing
  request's player then stopped it as stale. It now answers with its own (dead) session.
- A replaced hls.js instance's late manifest no longer marks the player ready or plays the
  outgoing media.
- A pause given after the answer stands: the manifest and the first canplay read the live intent.
- The debounced A/V-offset restart is dropped on an item change instead of restarting the
  previous item under the next one's title.
2026-10-07 23:16:25 +02:00
peter d25a132495 fix(ui): a dialog whose own video goes fullscreen no longer crashes the app
The portal host followed the fullscreen element into it. When that element is the dialog's own
content (the library player's <video>, by its f key or its own button), the move is into the host's
own descendant, which the DOM refuses: HierarchyRequestError in a layout effect, and the crash
screen replaced the app. The host now stays where it is then — the content already shows.
2026-10-07 23:16:25 +02:00
peter 61d3a1a189 chore(db): drop plex_items.markers, a column nothing writes or reads
Markers are read live from Plex for the item the player opens; the cached copy was never filled.
2026-10-07 02:23:37 +02:00
peter d482a5e8e2 chore(dev): the dev proxy answers a failed connect with 502, as the prod proxy does
Vite's own answer was a bodiless 500, which the app reads as a real server fault and shows the blocking error dialog; a 502 is the passing connection loss it is.
2026-10-07 02:23:37 +02:00
peter 2064373ac2 fix(history): a step back never leaves the app for an older page
Chrome keeps 50 entries a tab, and when the list is full a new entry drops the
oldest one the page left without a user gesture, which can be the page's own
entry under a sub-view or an overlay. A programmatic back() then left the
document; once it landed on an older entry whose restored Plex player started
playing on its own. Every programmatic back() now asks the Navigation API first
and, with no entry of this document below, settles in place: the sub-view turns
to its root, an overlay's entry drops its mark, the channel page and the YouTube
search clear.
2026-10-07 02:23:37 +02:00
peter 0cda5b6dd4 fix(player): only the credits that end an item move on to the next one
An item can carry several credits blocks with mid- and post-credit scenes between
them, and Plex flags the one that really ends it. The player left at the first
block. Now the Plex-final block moves on (by the auto-skip or the Skip button), an
earlier block is only skipped to its end by the button, and with no final flag the
last block ends the item by the button alone.
2026-10-07 01:48:10 +02:00
peter f3c94641a7 fix(player): a library item played from the start keeps its point
A cached file that autoplays from 0:00 fires its one playing event before the resume hook
subscribes, so the hook never counted it as played and reported nothing - not on pause, not on
close. An element already playing when the hook subscribes now counts as played.
2026-10-07 00:36:28 +02:00
peter 59b27467ab feat(player): the OS media controls, the basic keys and resume for the library and shared-link players
Plex, library and /watch tell the operating system what plays (title, line, picture) and take its
play / pause / skip / seek through one hook (lib/mediaSession); the Plex player drives it with its own
seek and clock, so the lock screen shows the film's position, not the stream's. The YouTube player is
left alone: its iframe already publishes all of it, and the OS routes to the frame that plays.

The library and /watch players get the basic keys (lib/mediaKeys, the keymap's mediaPlayer rows) and
open where the viewer stopped (lib/resume): the library keeps the point on the server, per user and
library item (library_states, POST /api/downloads/{id}/progress, listed back on both listings), the
shared link in this browser. A started library row continues, its menu can start it over, and its
picture carries the cards' progress bar. The resume rule (too early / near the end = from the start)
is one function shared with the YouTube route.

The library dialog opens focused on its panel (Modal initialFocus "player"): on the ✕, Space closed
the film. While the media element itself has the focus, Space and the arrows stay its own controls'
— the browser acts on them whatever a listener does, and one Space played and paused.
2026-10-06 23:55:41 +02:00
peter 02feaa9a1c refactor(player): one panel at a time over the Plex video, and one Back cascade
The ⋯ menu, the tracks menu, the tuning panel, the info overlay and the up-next card were a flag
each, so two could stand on top of each other, and Back/Escape were written out three times. They
are one `panel` value now; the stats readout stays a separate toggle beside it. One `back()` peels
the countdown, the panel, then the stats, then leaves — for the arrow, ⌫, Escape and the mouse
Back alike. The mouse Back now also closes the ⋯ menu instead of leaving the player.
2026-10-06 23:04:35 +02:00
peter c1058f1d5a refactor(player): the Plex session machine is a tested engine, one request per item in flight
PlexPlayer's session logic (generation, pending play intent, clock ownership, session offset, reclaim
budget, listener teardown) moves into lib/hlsEngine.ts, a plain class with hls.js behind lib/hlsPort.ts
and its own unit tests.

- A seek, track change or reclaim asked for while a session request is out replaces the queued
  target; it is sent once the in-flight answer settles. A burst of 20 presses past the buffer was 20
  requests and 20 ffmpeg spawns; it is now 2-3, and plays again ~2 s sooner.
- The player stops its server session when it leaves the item, closes or the page unloads
  (POST /api/plex/stream/{rk}/session/{sid}/stop, identity-aware). A left transcode used to run to
  the end of the film first.
- A restart asked for after a new session attached but before it played no longer reads the emptied
  media as the viewer's pause.
2026-10-06 22:31:48 +02:00
peter 23f16b69df release: 0.101.0 (improvement/e36-delivery) 2026-10-06 21:14:35 +02:00
peter c813814c3c chore(release): 0.101.0 2026-10-06 21:04:19 +02:00
peter ed6facb703 fix(downloads): a conversion's commit, a stopped upload, the fast join and a clip's copy hold up
- The fast join gives each ffconcat entry its own media-only format whitelist, so a library file
  can no longer be opened as a nested concat list (the list's whitelist was handed down to it).
- A file the guarded probe cannot read is a download-only file, not a guess from yt-dlp's info.
- A conversion's or an ingest's commit that raises is re-read in a fresh session before any file
  is removed: a commit that landed keeps the file its row now names.
- An original that cannot be removed after conversion is logged; the upload stays ready.
- An upload whose conversion is cancelled or deleted midway is freed once nothing holds it; a
  paused one is kept.
- A shared link says "preparing" only while its video's job is queued or running.
- The admin storage view counts every file on disk ("Files on disk"), not only ready ones.
- A clip's failure title, unknown failure and cancel dialog, and a clip's and an upload's delete
  dialog speak of what the job made; an upload's quota refusal explains that the original is kept.
- /watch announces "preparing" as an alert and moves focus to the new Retry after an action.
2026-10-06 21:02:07 +02:00
peter d44d8d6dda fix(gate): knip sees the watch entry, the build must emit watch.html, and two hand copies are held
- The raw-source contract tests read files from disk (rawSources.testkit)
  instead of import.meta.glob, which knip counted as importing every src file —
  its unused-files lane could never name an orphan. knip now names
  src/watchMain.tsx as an entry (registered in docs/gates.md).
- vite build fails when index.html or watch.html is not emitted.
- The watch-entry import walk follows dynamic import() too.
- audit_metrics.py's plan-code patterns are held equal to the guard's.
- CrashBoundary is domain-free: it moves into components/ui.
2026-10-06 03:19:33 +02:00
peter b20e9a3947 fix(player): the stacked layout's minimum width no longer clips the wide one
The 36rem floor guards the stacked layout's width/height feedback loop; the
wide side-panel card has no such loop, and on a short wide window the floor
made the video taller than the card, cutting off YouTube's control bar. The
floor now applies to the stacked layout only; e2e covers a 1400x380 window.
2026-10-06 03:19:33 +02:00
peter 6073a006b8 fix(downloads): an upload's cancel, progress, failure and notice speak of an upload
- Cancelling an upload asks "Delete upload?" and says the uploaded file is
  removed from the server with it.
- The phone-safe conversion's real ffmpeg % shows as a bar ("Converting for
  phones"), not an indeterminate pulse.
- A failed upload is titled "The upload isn't ready", with its own reasons
  (convert_failed, retryable; upload_missing) instead of download copy.
- The inbox says "Upload ready to share" for a converted upload.
2026-10-06 03:19:32 +02:00
peter dba4758c59 fix(watch): a link whose video is being made again says so, in the page's language
The public watch route answers video_preparing for an asset that is pending or
downloading, and the page shows "being prepared — try again" with a Retry
instead of "this link is gone". watch.html ships one lang; the page now sets
documentElement.lang to the language its strings were picked in.
2026-10-06 03:19:32 +02:00
peter 68cfc7e25d fix(downloads): an upload's conversion is guarded, counted, named and cleaned up
- Every ffmpeg/ffprobe over a library file names the demuxers it may use
  (edit.INPUT_GUARD; a downloaded thumbnail edit.THUMB_GUARD): a playlist-type
  demuxer would read local paths named inside a user's upload.
- A re-encode ends on an even frame (edit.EVEN_FRAME): an odd 4:4:4/RGB source
  failed its conversion; a clip stores the frame its file has.
- An upload counts against its owner's cap from the moment it lands (queued, or
  kept after a failed conversion) — the footprint, the GC's cap and the admin
  view read one ON_DISK predicate — and a conversion that outgrows the cap is
  refused, keeping the original.
- The converted name keeps _a{id} (a dotted bare name once lost it and two
  uploads shared a file); a converted file placed but not committed is removed,
  and the orphan sweep enters .uploads; the LRU cap never evicts an upload.
- Ingest judges the extension the file is stored under; a failed conversion is
  convert_failed / upload_missing, reports the "converting" phase, and notifies
  as an upload. Job statuses in routes/downloads use JobStatus.
2026-10-06 03:19:32 +02:00
peter 00886d92fe fix(player): the YouTube player keeps its size on a short window and when maximised
Two layout bugs since the player grew a side panel, both reported from prod.

Maximised (F) on a wide window, the video was a thin line across the screen: the side-panel layout
aligns the stage `self-start`, and Chrome applies a flex item's self-alignment to an absolutely
positioned box too, so the fixed stage took its content's height — 0 px. The maximised stage now
stretches between its insets whatever the layout asks of it.

On a short window under 1280 px (a laptop at 100%, a 1080p screen at 175%) only the side strips were
left: the card narrows so the chrome under the video fits, but that chrome grows as the card narrows
(its text wraps), and the loop ran the card down to 2 px. The video now never goes narrower than
36rem to make room; past that the card scrolls.

Measured before/after (localdev, real YouTube iframe): 1097×520 open 0×0 → 610×343; 1920×911 maximised
1920×0 → 1920×911; 1440×760 and 1366×650 maximised fill the window; leaving maximise restores the card.
Held by an e2e test at both sizes (red on the old code: 0 px wide, and 911 ≠ 0), green on Chrome,
WebKit and iPad.
2026-10-05 23:40:13 +02:00
peter e7d2eb8cf6 test(downloads): what a share link serves passes the phone-safe rule
A link serves an asset once it is ready, so the contract is pinned on both sides. The code: every
function in app/ that marks something ready must run the rule (`_ensure_phone_safe`, or the
upload's own `playable.plan`) — an AST scan names any that does not, and fails blind if it stops
finding the four paths known today (download, editor clip, upload conversion, upload kept as is).
The bytes: the files an iPhone + iPad refused (HEVC tagged hev1, AV1, Matroska, MP3 inside an
.mp4) and the ones other browsers cannot be trusted with (VP9 + Opus, 10-bit H.264, Opus in Ogg)
go through the upload route, the worker's conversion and the public file route; what the link
hands out must be served as mp4 / m4a / mp3 and pass the rule.

Seen red: a planted ready-marking function in service.py is named; with the conversion step
short-circuited, all seven files fail (on the rule, or on being served as quicktime / webm / ogg).
2026-10-05 21:05:22 +02:00
peter e886890d6e perf(watch): a shared link loads the player, not the app
A /watch link is opened by whoever it was sent to — signed out, often on a phone on mobile data —
and it booted through main.tsx: every app provider, the app's i18n (every locale of every area),
the flag sprites, ~392 KB gzip before the player could start. It now has its own HTML entry,
watch.html → watchMain.tsx, which imports WatchPage and what it renders; the server hands every
/watch/ path that page (the OG block injected as before), and vite dev does the same.

Measured on the build: /watch JS 290 → 87 KB gzip, CSS 104 → 19 KB; the app's own bytes unchanged.
Two leaks had to go for that: lib/format imported the app's i18n for one "just now" string (the
relative-time formatters now take `t`, like the file's other t-formatters), and the boundary's
crash copy came from the same i18n (CrashBoundary takes its copy; ErrorBoundary is the app's
wrapper, the watch entry gives it the page's own strings). The page's popover tree went too: it
renders no popover.

Held by: a contract test walking the entry's runtime imports, naming the path to anything only the
app may load (red on a planted i18n import in lib/format), and holding watch.html's head equal to
index.html's; a DB-lane test that every /watch/ path gets the player's page with the link's OG tags
and app paths get the app (red on the old routing). Screenshots at 375 px, EN and HU, video / audio /
password / gone / wrong password / unlocked: identical to the old entry's, bar the video's buffered bar.
2026-10-05 20:51:39 +02:00
peter 5561d9a379 feat(downloads): every ready file is phone-safe — uploads and clips too
A share link is opened on whatever the friend holds. Measured on an iPhone and an iPad: AV1,
Matroska, HEVC tagged hev1 and MP3 inside an .mp4 fail there; Chrome and Firefox add HEVC and
10-bit / 4:4:4 H.264. The old check judged the codec NAME only and ran on downloads only, so a
10-bit H.264 passed, an upload went out exactly as it came (an iPhone's HEVC .mov, a webm, an mkv)
and a fast trim kept its source's codecs.

One rule now, `downloads/playable.py`: video is H.264 8-bit 4:2:0 + AAC in .mp4, audio-only is
.mp3 or AAC .m4a; `plan` says what must change, `build_cmd` re-encodes only the failing stream (or
remuxes). The worker applies it to a download, an editor clip and — new — an upload: media that
fails the rule is queued (asset pending, job queued) and the worker converts it, keeping only the
converted file (owner's choice). Until then the original sits at rel_path, so Retry starts from it
and a delete removes it; an upload that lost its last holder mid-conversion is reaped by the GC.
Every libx264 run (edit, concat, conversion) now pins `-pix_fmt yuv420p` — without it a 10-bit
source came out High 10. The download cache signature moves to v5 so a cached 10-bit asset
re-derives. The upload toast says when the file is still converting (HU/EN).

Tests: the plan for every measured class; real ffmpeg turning a 10-bit Matroska + Opus and an
mp3-in-mp4 into files the rule accepts (red without the pixel-format pin); the queued upload's
delete, cancel-then-GC, failed conversion and real worker conversion (the cleanup tests red
without their rules); the toast (red without the branch). Live on localdev: 9 fixture uploads
converted, every served file probed H.264 High yuv420p + AAC, or mp3 / m4a.
2026-10-05 03:16:27 +02:00
peter 2768ab5bdd fix(gate): the plan-code guard catches decision codes, and the purge takes them
A plan's decision ids (`D1`, `D4`, `D6-A`) cited in 11 comments and test titles named a document
no reader of this repo has, and the guard had no pattern for them. It has now (mirrored in
audit_metrics), with cases for each form the citations took and for the look-alikes it must leave
alone (a JPEG marker, "3D"); the 11 lines say their why without the id. Case test seen red without
the pattern, the scan red on a planted `D9:`.
2026-10-05 03:16:13 +02:00
309 changed files with 19201 additions and 3287 deletions
+55
View File
@@ -7,6 +7,61 @@ edit the release notes and re-run the generator.
Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading).
## 0.105.0 — 2026-10-10
### New
- Clips: mark a range of what you're watching — a YouTube video or a Plex item — set its ends to the tenth, crop it to a frame drawn on the picture, and cut it straight into your downloads.
### Fixed
- Downloads: a download, a clip or an editor clip that doesn't fit in the storage you have left says so, and is never stored over your cap; YouTube downloads reach their format's height again.
- Player: focus rings, the chosen track or tab, sliders and the progress bar's head are amber on the dark player in every theme, so they show in a light theme too.
## 0.104.0 — 2026-10-10
### New
- YouTube: a card's menu filters the feed to its channel and back, and the player gives the video more room — its side panel shuts on demand.
- Plex: a card's picture plays it and its title opens its page; a show card puts on the episode it would continue or start, and an episode links up to its show.
### Fixed
- Plex: summaries written as HTML read as clean text, and a Surprise draw stays put until you reshuffle. A video marked watched in the player stays out of an Unwatched feed. Plus smaller keyboard and focus fixes.
## 0.103.0 — 2026-10-08
### New
- Plex: “Add to collection…” in a movie's or show's menu ticks the title into or out of your collections, or names a new one with it inside — one step instead of a trip to the Collections page.
### Fixed
- Plex player: subtitles rise above the control bar while it shows and slide back when it hides, so the bar never covers them — on an iPad too.
- iPad: the header search fits the touch screen, tooltips no longer stick, the Plex player page no longer scrolls or shows a blue band, and the fullscreen button stays hidden where the browser cannot really go fullscreen.
- The playlist, collection and tag menus say when their list is loading or failed to load (with a retry) instead of showing it as empty, and work from the keyboard. Plus smaller fixes in the YouTube player's title line.
## 0.102.0 — 2026-10-08
### New
- The library and shared-link players answer the keyboard's and the system's media keys, and resume where you stopped.
### Fixed
- Plex player: the browser Back closes the panels over the video one at a time, and only the credits at the very end move on to the next episode. Plus smaller fixes around resume points and stopping a stream.
## 0.101.0 — 2026-10-06
### New
- A shared link plays on any phone, iPhone and iPad included: downloads, clips and uploads are kept in a format every phone plays, and an upload in another format is converted after it arrives.
- A shared link opens much faster: its page no longer loads the whole app.
### Fixed
- The YouTube player keeps its size on a short window and when maximised — before, it could shrink to a thin strip. Plus smaller fixes around uploads and clips.
## 0.100.0 — 2026-10-05
### New
+3 -1
View File
@@ -65,7 +65,8 @@ how it gets built. Concretely:
`downloads/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `quota`,
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `ratelimit`, `bodylimit`, `progress`, `collation`,
`security`, `logs`, `logsink`, `diagnostics`, `debugsession`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
Frontend: `main.tsx` provider tree → `App.tsx` shell → `moduleRegistry` route-chunks → module
Frontend: `main.tsx` provider tree → `App.tsx` shell (a shared `/watch` link boots from its own
entry, `watch.html` → `watchMain.tsx`, held app-free by a contract test) → `moduleRegistry` route-chunks → module
components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
its menus, `Modal`/`Overlay`, `Button`/`form`, `DataTable`/`VirtualGrid`, `QueryState`, …). That
boundary is one-way and **lint-enforced**, not just described: a platform component may not import a
@@ -101,6 +102,7 @@ CQRS, a repository/DI framework, a router library, a generic CRUD-router factory
| FE static label / button-like link | `Badge` (flat, no edge) · `LinkButton` / `IconLink` | a glass pill that reads as a chip; an `<a>` hand-painted like a button |
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)` | a bare `catch {}` or `notify()` with no error attached |
| FE player on a native media element | `lib/mediaSession` (`useMediaSession`) + `lib/mediaKeys` + `lib/resume` (`useResume`), the element held as STATE | a private `navigator.mediaSession` / `keydown` / position-saving effect per player |
| FE diagnostics | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts` | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded |
| FE state that MIRRORS a prop/query | derive it, or hold a draft (`draft ?? server`) | a copy an effect re-syncs — that paints the stale value first |
| FE state reset on a key change | adjust it DURING render against a remembered key | `useEffect(() => setX(init), [key])` |
+3 -2
View File
@@ -23,8 +23,9 @@ blocker and SponsorBlock keep working.
- **Playlists with two-way YouTube sync** — build them locally, keep them in sync in both directions.
- **In-app player** with resume, plus keyboard/scroll controls.
- **Download Center** — save videos to the server with yt-dlp in a Plex-friendly layout (format
presets, per-user storage quota), trim / crop / split & join them in a built-in editor, then save
to your device, share with another user, or hand out a public watch link.
presets, per-user storage quota), trim / crop / split & join them in a built-in editor, play them
where you left off, then save to your device, share with another user, or hand out a public watch
link.
- **Plex library** *(optional)* — point Siftlode at your own Plex server to browse and play its
movies and shows alongside your feed, with watch state kept in sync.
- **Multi-user** with per-user private state, a shared catalog, and a fair daily API-quota guard.
+1 -1
View File
@@ -1 +1 @@
0.100.0
0.105.0
-40
View File
@@ -437,14 +437,6 @@
"lineCount": 1
}
},
{
"code": "reportArgumentType",
"range": {
"startColumn": 12,
"endColumn": 13,
"lineCount": 10
}
},
{
"code": "reportAttributeAccessIssue",
"range": {
@@ -743,14 +735,6 @@
"lineCount": 1
}
},
{
"code": "reportArgumentType",
"range": {
"startColumn": 37,
"endColumn": 47,
"lineCount": 1
}
},
{
"code": "reportArgumentType",
"range": {
@@ -1142,22 +1126,6 @@
"endColumn": 48,
"lineCount": 1
}
},
{
"code": "reportArgumentType",
"range": {
"startColumn": 39,
"endColumn": 45,
"lineCount": 1
}
},
{
"code": "reportArgumentType",
"range": {
"startColumn": 38,
"endColumn": 43,
"lineCount": 1
}
}
],
"./app/routes/plex/playback.py": [
@@ -1463,14 +1431,6 @@
"lineCount": 1
}
},
{
"code": "reportArgumentType",
"range": {
"startColumn": 38,
"endColumn": 43,
"lineCount": 1
}
},
{
"code": "reportArgumentType",
"range": {
@@ -0,0 +1,42 @@
"""library: a resume point per user and library item
The library player opened every item at 0:00. `library_states` keeps where a user stopped, per job
(their own download, or one shared with them) — the same shape as `video_states`' position, without a
status: the library has no "watched". It starts empty; the downgrade drops it.
"""
import sqlalchemy as sa
from alembic import op
revision: str = "0099_library_states"
down_revision: str | None = "0098_drop_demo_account"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.create_table(
"library_states",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column(
"user_id",
sa.Integer(),
sa.ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column(
"job_id",
sa.Integer(),
sa.ForeignKey("download_jobs.id", ondelete="CASCADE"),
nullable=False,
),
sa.Column("position_seconds", sa.Integer(), nullable=False),
sa.Column("progress_updated_at", sa.DateTime(timezone=True), nullable=False),
sa.UniqueConstraint("user_id", "job_id", name="uq_library_user_job"),
)
op.create_index("ix_library_states_job_id", "library_states", ["job_id"])
def downgrade() -> None:
op.drop_index("ix_library_states_job_id", table_name="library_states")
op.drop_table("library_states")
@@ -0,0 +1,22 @@
"""plex_items: drop `markers`, a column nothing writes or reads
Intro/credits markers are read live from Plex for the item the player opens (with Plex's `final` flag
on the block that ends it); this cached copy was never filled — every row is NULL. The downgrade
restores the empty column.
"""
import sqlalchemy as sa
from alembic import op
revision: str = "0100_drop_plex_item_markers"
down_revision: str | None = "0099_library_states"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.drop_column("plex_items", "markers")
def downgrade() -> None:
op.add_column("plex_items", sa.Column("markers", sa.JSON(), nullable=True))
@@ -0,0 +1,46 @@
"""Plex summaries and taglines as plain text
Revision ID: 0101_plex_text_plain
Revises: 0100_drop_plex_item_markers
Create Date: 2026-10-08
Some Plex agents write a summary as HTML (`<p>…</p>`, `&aacute;`), and the app showed the tags as
text. The sync now stores every summary and tagline through `plain_text` (app/plex/plaintext.py);
this cleans the rows it wrote before, through the same function, so nothing waits for a full resync.
"""
import sqlalchemy as sa
from alembic import op
from app.plex.plaintext import plain_text
revision: str = "0101_plex_text_plain"
down_revision: str | None = "0100_drop_plex_item_markers"
branch_labels: str | None = None
depends_on: str | None = None
_COLUMNS = {
"plex_items": ("summary", "summary_hu", "tagline", "tagline_hu"),
"plex_shows": ("summary", "summary_hu", "tagline", "tagline_hu"),
"plex_collections": ("summary",),
}
def upgrade() -> None:
conn = op.get_bind()
for table, columns in _COLUMNS.items():
for col in columns:
rows = conn.execute(
sa.text(f"SELECT id, {col} FROM {table} WHERE {col} ~ '[<&]'")
).fetchall()
updates = [
{"id": rid, "v": clean}
for rid, value in rows
if (clean := plain_text(value)) != value
]
if updates:
conn.execute(sa.text(f"UPDATE {table} SET {col} = :v WHERE id = :id"), updates)
def downgrade() -> None:
pass # the markup is not wanted back; the next sync would clean it again
+9 -6
View File
@@ -181,15 +181,18 @@ class Settings(BaseSettings):
# for YouTube Proof-of-Origin tokens (bypasses bot-detection + unlocks high-quality
# formats). Empty disables it (yt-dlp still works, but may hit "confirm you're not a bot").
download_pot_base_url: str = "http://bgutil-pot:4416"
# yt-dlp player clients (comma-separated). PRIMARY = android_vr: high-quality DASH, no PO
# token / JS runtime needed, reliable for normal use. On a bot/DRM/format failure (YouTube's
# per-client responses are flaky, and android_vr can get bot-flagged under heavy load) the
# worker retries with the FALLBACK set — the POT-capable web clients (web needs the token +
# Deno) plus android for audio. Both admin-tunable when YouTube shifts what works.
# yt-dlp player clients (comma-separated). PRIMARY = tv_simply: measured 2026-10-10 as the one
# client that is offered 1080p+ AND finishes the fetch, its stream token minted by the bgutil
# sidecar. android_vr (the primary until then) was still offered 2160p, but every fetch died
# with HTTP 403 — it now needs a stream PO token the sidecar cannot mint — and the fallback
# below served 360p only, so every download came out at 360p without failing. On a
# bot/DRM/format failure the worker retries with the FALLBACK set — the POT-capable web clients
# (web needs the token + Deno) plus android, whose 360p is the last resort. Both admin-tunable
# (env) when YouTube shifts what works.
# A SIGNED-IN download (the user added YouTube cookies) keeps only the cookie-capable clients
# of these two sets — yt-dlp drops the rest, so android_vr / android / ios never run there.
# See adapters.YouTubeAdapter._COOKIE_CAPABLE.
download_player_clients: str = "android_vr"
download_player_clients: str = "tv_simply"
download_player_clients_fallback: str = "web_safari,web,android"
@staticmethod
+3 -3
View File
@@ -24,7 +24,7 @@ from sqlalchemy.orm import Session
from app.discover import cards, queries
from app.models import Channel, ComfortPin, PlexItem, PlexShow, Playlist, User, Video
from app.plex import rollup
from app.plex.cards import leaf_card, show_card
from app.plex.cards import SHOW_NEW, leaf_card, show_card
from app.plex.lookups import by_rating_key
# kind -> the ComfortPin column that holds it. The client's vocabulary and the schema's, in one place.
@@ -203,8 +203,8 @@ def _shelf_card(db: Session, user: User, pin: ComfortPin, batch: _Batch, lang: s
sh = db.get(PlexShow, pin.plex_show_id)
if sh is None:
return None
status = rollup.state_map(db, user.id, [sh.id]).get(sh.id, "new")
card = cards.from_plex(show_card(sh, status, lang=lang))
state = rollup.state_map(db, user.id, [sh.id]).get(sh.id, SHOW_NEW)
card = cards.from_plex(show_card(sh, state, lang=lang))
ep = queries.next_episode(db, user.id, sh.id, lang=lang)
nxt = cards.from_plex(ep) if ep else None
+7 -3
View File
@@ -256,16 +256,20 @@ def plex_in_progress(
if show_libs:
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
sq = apply_show_state_filter(
select(PlexShow, prog.c.resumed_at).join(prog, prog.c.show_id == PlexShow.id),
select(PlexShow, prog.c.resumed_at, prog.c.total, prog.c.watched, prog.c.inprog).join(
prog, prog.c.show_id == PlexShow.id
),
prog.c.total,
prog.c.watched,
prog.c.inprog,
"in_progress",
).where(PlexShow.library_id.in_(show_libs))
for sh, resumed_at in db.execute(
for sh, resumed_at, total, watched, inprog in db.execute(
sq.order_by(prog.c.resumed_at.desc().nullslast()).limit(limit)
).all():
out.append((resumed_at, show_card(sh, "in_progress", lang=lang)))
out.append(
(resumed_at, show_card(sh, rollup.show_state(total, watched, inprog), lang=lang))
)
return out
+2 -1
View File
@@ -106,7 +106,8 @@ class YouTubeAdapter(SourceAdapter):
# `player_client=android_vr` yt-dlp logs `Skipping client "android_vr" since it does not support
# cookies`, drops it, and the request that is left with no client fails with "Requested format is
# not available" — which is already a retriable marker above, so the cost is a wasted attempt
# rather than a hard failure. Our primary set is android_vr ALONE, so without this filter every
# rather than a hard failure. Our primary set is tv_simply ALONE, which cannot carry cookies either
# (measured on yt-dlp 2026.08.19: `SUPPORTS_COOKIES` False), so without this filter every
# signed-in download would burn its first attempt that way before reaching the POT-backed web
# fallback the cookie exists for. The list is yt-dlp's own INNERTUBE_CLIENTS[...]
# ["SUPPORTS_COOKIES"], pinned by test_adapters against the installed yt-dlp, so a version bump
+289
View File
@@ -0,0 +1,289 @@
"""Ad-hoc clips: a marked range of a YouTube video or a Plex item, cut into a new per-user asset.
A clip never needs the whole source first. A YouTube clip fetches only its range (yt-dlp's
`download_ranges`; measured: 3-7 s whether the video runs four minutes or an hour, and the fetched
range starts on the marked frame), and a Plex clip reads the library file in place. Either way the
range then goes through ONE re-encode built here, so every clip has the same shape whatever its
source: frame-exact at both ends, H.264 at most 1080p (an HDR source tonemapped to SDR — kept as
10-bit PQ it played washed out), stereo AAC, in an .mp4 that plays on any phone.
A clip may also crop the picture to a frame the user drew over the playing video. The frame is
given in fractions of the box it was drawn in, with that box's aspect: the YouTube iframe never
tells its picture's size, so the frame there is drawn over the player's 16:9 box (a short's 9:16),
and the cut maps it onto the real picture (`crop_filter`) — what lies over a letterbox bar is dropped.
The spec is stored on the job (`DownloadJob.edit_spec`) and is the clip's whole identity: its
signature is the asset's cache key, so marking the same range again is an instant cache hit, and a
different range or audio track is a different clip.
"""
import hashlib
import json
import math
import re
from pathlib import Path
from app.downloads.edit import AAC_ARGS, EVEN_FRAME, INPUT_GUARD, X264_ARGS
from app.plex.sync import HDR_TRANSFERS
SOURCES = ("youtube", "plex")
# Owner's cap: a clip is a moment, not a way to export a whole film to a public link.
MAX_CLIP_S = 600
MIN_CLIP_S = 1
MAX_HEIGHT = 1080
# What yt-dlp fetches the range in (formats.normalize fills the rest): the clip's own ceiling, so the
# re-encode never upscales and never throws away a better source than it keeps.
YOUTUBE_FETCH_SPEC = {"mode": "av", "max_height": MAX_HEIGHT, "container": "mp4"}
# Bump when the spec → output-bytes mapping changes, so a cached clip re-derives.
_SIG_VERSION = 1
_YOUTUBE_ID = re.compile(r"[A-Za-z0-9_-]{11}")
_RATING_KEY = re.compile(r"[0-9]{1,32}")
_MAX_AUDIO_TRACK = 63
# HDR → SDR as measured on the owner's 4K HDR files (2026-10-10): linearise, tonemap (hable),
# convert the primaries and transfer to BT.709 and back to 8-bit 4:2:0.
_TONEMAP = (
"zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=hable:desat=0,"
"zscale=t=bt709:m=bt709:r=tv,format=yuv420p"
)
_SCALE = f"scale=-2:'min({MAX_HEIGHT},ih)'"
# A crop frame is at least this share of its box on each side (the player's frame holds the same
# floor), and what is left of it on the real picture at least _MIN_VISIBLE of it.
MIN_CROP = 0.1
_MIN_VISIBLE = 0.05
# The box aspects a frame may be drawn in: a 9:16 short to a 21:9 film, with room either side.
_CROP_ASPECTS = (0.25, 4.0)
# Float slack on the frame's far edges (x + w, y + h), which the client rounds to 4 decimals.
_EDGE_SLACK = 1e-3
class ClipError(Exception):
"""A clip request that cannot be cut; `reason` is an i18n-mappable key, `params` the numbers its
message names (the cap itself, so the client never says a stale one)."""
def __init__(self, reason: str, **params: float):
self.reason = reason
self.params = params
super().__init__(reason)
def normalize(
source: str,
ref: str,
start_s: float,
end_s: float,
audio_track: int | None = None,
crop: dict | None = None,
) -> dict:
"""The validated, canonical clip spec. Times are rounded to the millisecond and a crop to four
decimals, so the same marks always give the same cache key. Raises ClipError."""
if source not in SOURCES:
raise ClipError("bad_source")
pattern = _YOUTUBE_ID if source == "youtube" else _RATING_KEY
if not isinstance(ref, str) or not pattern.fullmatch(ref):
raise ClipError("bad_source")
if not (math.isfinite(start_s) and math.isfinite(end_s)) or start_s < 0:
raise ClipError("bad_range")
start, end = round(start_s, 3), round(end_s, 3)
if end - start < MIN_CLIP_S:
raise ClipError("too_short")
if end - start > MAX_CLIP_S:
raise ClipError("too_long", limit_s=MAX_CLIP_S)
spec: dict = {"source": source, "ref": ref, "start_s": start, "end_s": end}
if source == "plex" and audio_track is not None:
if not 0 <= audio_track <= _MAX_AUDIO_TRACK:
raise ClipError("bad_audio")
spec["audio_track"] = audio_track
frame = _crop(crop)
if frame is not None:
spec["crop"] = frame
return spec
def _crop(raw: dict | None) -> dict | None:
"""A crop frame, canonical — or None for no crop, which a frame over the whole box also is (the
same clip as one cut without a frame, so the same cache key)."""
if raw is None:
return None
try:
x, y, w, h, aspect = (float(raw[k]) for k in ("x", "y", "w", "h", "aspect"))
except (KeyError, TypeError, ValueError) as e:
raise ClipError("bad_crop") from e
if not all(math.isfinite(v) for v in (x, y, w, h, aspect)):
raise ClipError("bad_crop")
lo, hi = _CROP_ASPECTS
if not lo <= aspect <= hi or x < 0 or y < 0:
raise ClipError("bad_crop")
if w < MIN_CROP - _EDGE_SLACK or h < MIN_CROP - _EDGE_SLACK:
raise ClipError("bad_crop")
if x + w > 1 + _EDGE_SLACK or y + h > 1 + _EDGE_SLACK:
raise ClipError("bad_crop")
frame = {"x": x, "y": y, "w": min(w, 1 - x), "h": min(h, 1 - y), "aspect": aspect}
frame = {k: round(v, 4) for k, v in frame.items()}
if (frame["x"], frame["y"], frame["w"], frame["h"]) == (0, 0, 1, 1):
return None
return frame
def duration(spec: dict) -> float:
return spec["end_s"] - spec["start_s"]
def sig(spec: dict) -> str:
"""The clip's cache signature: the spec itself, versioned."""
body = json.dumps({**spec, "_v": _SIG_VERSION}, sort_keys=True, separators=(",", ":"))
return hashlib.sha256(body.encode()).hexdigest()[:24]
def asset_ref(user_id: int, spec: dict) -> str:
"""The asset's `source_ref`. The user id makes a clip private to whoever cut it, like an edit:
never dedup-shared, never cache-served to someone else."""
return f"{user_id}:{spec['source']}:{spec['ref']}"
def reference_url(spec: dict) -> str | None:
"""Where a YouTube clip came from: its video at the clip's start. A Plex clip has no public page."""
if spec.get("source") != "youtube" or not spec.get("ref"):
return None
return f"https://www.youtube.com/watch?v={spec['ref']}&t={int(spec.get('start_s') or 0)}s"
def is_hdr(probe: dict) -> bool:
"""Whether the first video stream of an ffprobe `-show_streams` JSON carries an HDR transfer."""
for s in probe.get("streams") or []:
if s.get("codec_type") == "video":
# PQ / HLG: an 8-bit SDR encode must tonemap them (the one set the Plex stream uses).
return s.get("color_transfer") in HDR_TRANSFERS
return False
def display_aspect(probe: dict) -> float | None:
"""The picture's shape as a player shows it, from an ffprobe `-show_streams` JSON: the first video
stream's width × its sample aspect ratio over its height, turned a quarter when the stream says
it is rotated (ffmpeg turns such a stream upright before any filter of ours, so a crop is cut
from the upright picture). None when the stream has no size."""
for s in probe.get("streams") or []:
if s.get("codec_type") != "video":
continue
w, h = s.get("width"), s.get("height")
if not isinstance(w, int) or not isinstance(h, int) or w <= 0 or h <= 0:
return None
dar = w * (_ratio(s.get("sample_aspect_ratio")) or 1.0) / h
return 1 / dar if _quarter_turned(s) else dar
return None
def _ratio(v: object) -> float | None:
"""'32:27' → 1.185…; '0:1', 'N/A', a missing or malformed value → None."""
if not isinstance(v, str) or ":" not in v:
return None
num, _, den = v.partition(":")
try:
n, d = float(num), float(den)
except ValueError: # silent-ok: a malformed ratio reads as none given; the caller takes 1:1
return None
return n / d if n > 0 and d > 0 else None
def _quarter_turned(stream: dict) -> bool:
rotation: object = (stream.get("tags") or {}).get("rotate")
for sd in stream.get("side_data_list") or []:
if "rotation" in sd:
rotation = sd["rotation"]
try:
return round(float(str(rotation))) % 180 == 90
except ValueError: # silent-ok: no (None) or a malformed rotation: the stream is upright
return False
def crop_filter(crop: dict, dar: float) -> str | None:
"""The ffmpeg `crop` for a frame drawn in a box of `crop["aspect"]`, over a picture of display
aspect `dar` fitted into that box as a player fits it (whole, centred, bars on the short sides).
The frame's part over a bar is dropped: "" when what is left is the whole picture, None when too
little of the frame lies on the picture at all.
In fractions of the input's own width and height, so a sample aspect ratio other than 1:1 (an
anamorphic DVD) needs nothing of its own, and floored to even pixels: an even 4:2:0 frame, and a
window that never runs past the picture's far edge."""
box = crop["aspect"]
pic_w, pic_h = (1.0, box / dar) if dar > box else (dar / box, 1.0)
def span(start: float, size: float, extent: float) -> tuple[float, float]:
origin = (1 - extent) / 2
a = min(1.0, max(0.0, (start - origin) / extent))
b = min(1.0, max(0.0, (start + size - origin) / extent))
return a, b - a
x, w = span(crop["x"], crop["w"], pic_w)
y, h = span(crop["y"], crop["h"], pic_h)
if w < _MIN_VISIBLE or h < _MIN_VISIBLE:
return None
if w > 1 - 1e-4 and h > 1 - 1e-4:
return ""
def px(axis: str, f: float) -> str:
return f"trunc({axis}*{f:.6f}/2)*2"
return f"crop={px('iw', w)}:{px('ih', h)}:{px('iw', x)}:{px('ih', y)}"
def audio_track_count(probe: dict) -> int:
return sum(1 for s in probe.get("streams") or [] if s.get("codec_type") == "audio")
def build_cmd(
src: Path,
dest: Path,
start_s: float,
dur_s: float,
*,
audio_track: int | None,
hdr: bool,
threads: int,
crop: str = "",
) -> list[str]:
"""ffmpeg for one clip: an input seek (decodes from the keyframe before `start_s` and drops the
frames up to it, so the clip starts on the marked frame), the first video stream, the chosen audio
track (the first when none was chosen; none at all is fine), no subtitles, data or chapters.
A `crop` (crop_filter) comes first: the tonemap and the scale then work on the frame it keeps."""
vf = ",".join([*([crop] if crop else []), *([_TONEMAP] if hdr else []), _SCALE, EVEN_FRAME])
audio = f"0:a:{audio_track}" if audio_track is not None else "0:a:0?"
return [
"ffmpeg",
"-y",
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-progress",
"pipe:1",
"-nostats",
"-ss",
f"{start_s:.3f}",
*INPUT_GUARD,
"-i",
str(src),
"-t",
f"{dur_s:.3f}",
"-map",
"0:v:0",
"-map",
audio,
"-sn",
"-dn",
# A source's chapters would ride into the .mp4 as a data track of their own (an mkv's
# chapter list came out as `bin_data`); a clip of a moment has no chapters to keep.
"-map_chapters",
"-1",
"-vf",
vf,
*X264_ARGS,
"-threads",
str(threads),
*AAC_ARGS,
"-ac",
"2",
"-movflags",
"+faststart",
str(dest),
]
+31
View File
@@ -30,6 +30,7 @@ from sqlalchemy import select
from sqlalchemy.orm import Session
from app import logs
from app.downloads import clip as clipmod
from app.downloads import formats
from app.downloads.adapters import _host_is
from app.models import DownloadCredential, MediaAsset
@@ -259,6 +260,28 @@ def usable_cookie(
return None
if not is_authed_asset(asset, spec, user_id, url):
return None # shared asset — never attach a cookie to a non-private identity
return _readable(cred, user_id, host_key)
def usable_clip_cookie(
db: Session, user_id: int, url: str | None, asset: MediaAsset, clip_spec: dict
) -> str | None:
"""`usable_cookie` for an ad-hoc clip's range fetch. The second gate is the clip's own identity:
a clip asset is private by construction (`clip.asset_ref` names its user, see downloads/clip),
so a cookie goes with it exactly when that asset is THIS user's clip of `clip_spec`. Without it a
video YouTube gates signed out could never be clipped, while its download goes through."""
host_key = host_key_for_url(url)
if host_key is None:
return None
cred = get_credential(db, user_id, host_key)
if cred is None:
return None
if asset.source_kind != "clip" or asset.source_ref != clipmod.asset_ref(user_id, clip_spec):
return None
return _readable(cred, user_id, host_key)
def _readable(cred: DownloadCredential, user_id: int, host_key: str) -> str | None:
raw = decrypt_optional(cred.cookies_enc)
if not raw:
log.warning(
@@ -284,6 +307,14 @@ def resolve_cookiefile(
return _write_cookiefile(raw, staging) if raw else None
def resolve_clip_cookiefile(
db: Session, user_id: int, url: str | None, asset: MediaAsset, clip_spec: dict, staging: Path
) -> Path | None:
"""`resolve_cookiefile` for an ad-hoc clip (`usable_clip_cookie`)."""
raw = usable_clip_cookie(db, user_id, url, asset, clip_spec)
return _write_cookiefile(raw, staging) if raw else None
def _write_cookiefile(raw: str, staging: Path) -> Path:
"""Write the cookie blob to a private 0600 file under `staging/.auth/` and return its path.
+71 -113
View File
@@ -188,6 +188,51 @@ def clip_duration(spec: dict, source_duration: int | None) -> int | None:
# --- ffmpeg: trim / crop --------------------------------------------------------------------
# Every libx264 / aac re-encode in this package uses these, so a re-encoded file is always the
# phone-safe shape (`playable`): `-pix_fmt yuv420p` pins 8-bit 4:2:0 — without it libx264 keeps a
# 10-bit or 4:4:4 source's pixel format and writes H.264 High 10 / High 4:4:4, which browsers decode
# only patchily (the one safe H.264 is 8-bit 4:2:0).
X264_ARGS = ["-c:v", "libx264", "-preset", "veryfast", "-crf", "20", "-pix_fmt", "yuv420p"]
AAC_ARGS = ["-c:a", "aac", "-b:a", "192k"]
# The last step of every video filter chain a libx264 re-encode runs: 4:2:0 halves the chroma, so
# libx264 refuses an odd width or height ("width not divisible by 2") — measured on an odd source
# that is not 4:2:0 already (a 4:4:4 or RGB screen capture, 1365x767). A crop of a 4:2:0 source is
# rounded to its chroma grid by ffmpeg itself; this holds every other path to the encoder to the same
# even frame. Rounding down to even costs at most one pixel row/column.
EVEN_FRAME = "scale=trunc(iw/2)*2:trunc(ih/2)*2"
# The demuxers ffmpeg/ffprobe may use to open a library file — single-file containers only. Without
# it ffmpeg picks the demuxer from the file's CONTENT and NAME, and an upload's name is the user's:
# a playlist-type demuxer (HLS, DASH, concat) opens whatever local paths its entries name, so a
# crafted upload could copy another user's clip or a Plex file into its own converted asset. A file
# no demuxer here can open fails the probe, and an unprobeable upload is a download-only 'file'.
MEDIA_DEMUXERS = (
"mov", # mp4, m4a, mov, 3gp
"matroska", # mkv, webm
"mpegts",
"mpeg",
"avi",
"asf",
"flv",
"ogg",
"mp3",
"aac",
"ac3",
"eac3",
"flac",
"wav",
"w64",
"aiff",
"caf",
"amr",
)
INPUT_GUARD = ["-format_whitelist", ",".join(MEDIA_DEMUXERS)]
# A downloaded thumbnail is the remote site's bytes under the suffix yt-dlp gave it (`.png`, `.webp`,
# `.jpg` — `pipeline._THUMB_SUFFIXES`): only the single-image demuxers those open, so a "thumbnail"
# that is really a playlist cannot pull a local file into the poster. Anything else fails, and the
# poster falls back to a cut frame (`ensure_poster`).
THUMB_GUARD = ["-format_whitelist", "png_pipe,jpeg_pipe,webp_pipe"]
def build_edit_cmd(src: Path, dest: Path, spec: dict, out_ext: str) -> list[str]:
"""ffmpeg command for a trim/crop edit. `-progress pipe:1` streams machine-readable progress.
@@ -214,7 +259,7 @@ def build_edit_cmd(src: Path, dest: Path, spec: dict, out_ext: str) -> list[str]
]
if start:
cmd += ["-ss", f"{start:.3f}"]
cmd += ["-i", str(src)]
cmd += [*INPUT_GUARD, "-i", str(src)]
if end is not None:
dur = end - start
if dur > 0:
@@ -222,21 +267,9 @@ def build_edit_cmd(src: Path, dest: Path, spec: dict, out_ext: str) -> list[str]
if reencode:
# Input -ss + re-encode is frame-accurate (decodes from the prior keyframe, discards up to
# the exact cut). A crop adds the filter; an accurate trim re-encodes with no filter.
if crop:
cmd += ["-vf", f"crop={crop['w']}:{crop['h']}:{crop['x']}:{crop['y']}"]
cmd += [
"-c:v",
"libx264",
"-preset",
"veryfast",
"-crf",
"20",
"-c:a",
"aac",
"-b:a",
"192k",
]
# the exact cut). A crop adds the filter; an accurate trim only evens the frame.
cropf = f"crop={crop['w']}:{crop['h']}:{crop['x']}:{crop['y']}," if crop else ""
cmd += ["-vf", cropf + EVEN_FRAME, *X264_ARGS, *AAC_ARGS]
else:
cmd += ["-c", "copy", "-avoid_negative_ts", "make_zero"]
@@ -284,11 +317,14 @@ def build_concat_plan(src: Path, dest: Path, spec: dict, out_ext: str, staging:
parts, labels = [], []
for i, s in enumerate(segs):
st, en = s["start_s"], s["end_s"]
parts.append(f"[0:v]trim=start={st}:end={en},setpts=PTS-STARTPTS{cropf}[v{i}]")
parts.append(
f"[0:v]trim=start={st}:end={en},setpts=PTS-STARTPTS{cropf},{EVEN_FRAME}[v{i}]"
)
parts.append(f"[0:a]atrim=start={st}:end={en},asetpts=PTS-STARTPTS[a{i}]")
labels.append(f"[v{i}][a{i}]")
parts.append(f"{''.join(labels)}concat=n={len(segs)}:v=1:a=1[v][a]")
cmd = head + [
*INPUT_GUARD,
"-i",
str(src),
"-filter_complex",
@@ -297,26 +333,23 @@ def build_concat_plan(src: Path, dest: Path, spec: dict, out_ext: str, staging:
"[v]",
"-map",
"[a]",
"-c:v",
"libx264",
"-preset",
"veryfast",
"-crf",
"20",
"-c:a",
"aac",
"-b:a",
"192k",
*X264_ARGS,
*AAC_ARGS,
]
else:
listp = staging / "concat.txt"
lines = []
lines = ["ffconcat version 1.0"]
for s in segs:
lines.append(f"file '{_concat_quote(src)}'")
# ffmpeg hands the list's whitelist (which must name `concat` to read the list) down to
# the file it names, so a library file whose bytes are a concat list would be read as one
# and open the paths inside it. Each entry's own whitelist replaces the inherited one.
lines.append(f"option format_whitelist {INPUT_GUARD[1]}")
lines.append(f"inpoint {s['start_s']}")
lines.append(f"outpoint {s['end_s']}")
prep[listp] = "\n".join(lines) + "\n"
cmd = head + ["-f", "concat", "-safe", "0", "-i", str(listp), "-c", "copy"]
guard = ["-format_whitelist", "concat," + INPUT_GUARD[1]]
cmd = head + ["-f", "concat", "-safe", "0", *guard, "-i", str(listp), "-c", "copy"]
if out_ext in ("mp4", "m4a", "mov"):
cmd += ["-movflags", "+faststart"]
@@ -474,14 +507,7 @@ def run_ffmpeg(
raise StatedFailure(f"ffmpeg failed ({ret}): {''.join(err_tail)[-300:]}")
# --- ffmpeg: universal browser playability --------------------------------------------------
# A downloaded file must play in EVERY browser, whatever the source shipped. Universal = H.264
# video + AAC/MP3 audio in an mp4 container: H.264 is the only video codec all browsers decode
# (VP9/AV1 fail on Safari/iOS; HEVC fails on Chrome/Firefox), and mp4/AAC is the safe pairing.
# Anything else is fixed post-download: re-encode the offending stream(s), or just remux when the
# codecs are already fine but the container isn't.
_BROWSER_SAFE_VCODEC = ("h264",)
_BROWSER_SAFE_ACODEC = ("aac", "mp3")
# --- ffprobe ---------------------------------------------------------------------------------
def run_ffprobe_json(
@@ -493,10 +519,10 @@ def run_ffprobe_json(
) -> dict | None:
"""One ffprobe call → its parsed JSON dict, or None on any failure (timeout, non-zero exit,
missing binary, unparseable output). The SINGLE ffprobe-JSON-invocation point in the downloads
package: the browser-playability codec probe here and the shared media probe (`probe.probe_media`,
used by both the download worker's finalize and the upload ingest) both go through it, so the
package: the shared media probe (`probe.probe_json`: the stored facts and the phone-safe rule) goes
through it, so the
subprocess flags/timeout/error handling live in one place instead of drifting copies."""
args = ["ffprobe", "-v", "error"]
args = ["ffprobe", "-v", "error", *INPUT_GUARD]
if show_streams:
args.append("-show_streams")
if show_format:
@@ -517,35 +543,6 @@ def run_ffprobe_json(
return None
def _codecs_from_streams(streams: list) -> tuple[str | None, str | None]:
"""(video_codec, audio_codec) lowercased from an ffprobe JSON stream list — the first of each kind.
Parsed from the JSON `streams` list, NOT ffprobe's flat `default=nw=1` output: there the fields
arrive in ffprobe's own struct order, which puts `codec_name` BEFORE `codec_type`, so a line-by-line
"remember the last codec_type" parser attributes every name to the PREVIOUS stream. Measured on real
downloads: an ordinary H.264+AAC mp4 probed as `v=aac a=None` — the browser-playability check then
believed the video stream was AAC and re-encoded a perfectly fine file on every download. Reading each
stream dict by its own `codec_type` is immune to that ordering."""
vcodec = acodec = None
for s in streams:
name = (s.get("codec_name") or "").lower() or None
ctype = (s.get("codec_type") or "").lower()
if ctype == "video" and vcodec is None:
vcodec = name
elif ctype == "audio" and acodec is None:
acodec = name
return vcodec, acodec
def probe_codecs(path: Path) -> tuple[str | None, str | None]:
"""Return (video_codec, audio_codec) lowercased via one ffprobe call; None for a missing
stream or if ffprobe fails (a failed probe is treated as 'leave it alone' upstream)."""
data = run_ffprobe_json(path, entries="stream=codec_type,codec_name")
if data is None:
return None, None
return _codecs_from_streams(data.get("streams") or [])
def probe_duration(path: Path) -> int:
"""Media duration in whole seconds via ffprobe, or 0 if unknown — a progress denominator for a
re-encode when the yt-dlp info dict carried no `duration` (some URL/live extractions)."""
@@ -555,6 +552,7 @@ def probe_duration(path: Path) -> int:
"ffprobe",
"-v",
"error",
*INPUT_GUARD,
"-show_entries",
"format=duration",
"-of",
@@ -577,49 +575,6 @@ def probe_duration(path: Path) -> int:
return 0
def browser_transcode_flags(
vcodec: str | None, acodec: str | None, is_mp4: bool
) -> tuple[bool, bool, bool]:
"""Decide what (if anything) makes a file universally browser-playable. Returns
(needs_change, reencode_video, reencode_audio). needs_change=False => already H.264 + AAC/MP3
in mp4. A None codec (no such stream, or a failed probe) is never re-encoded."""
rv = bool(vcodec) and vcodec not in _BROWSER_SAFE_VCODEC
ra = bool(acodec) and acodec not in _BROWSER_SAFE_ACODEC
return (rv or ra or not is_mp4), rv, ra
def build_browser_safe_cmd(
src: Path, dest: Path, reencode_video: bool, reencode_audio: bool, threads: int | None = None
) -> list[str]:
"""ffmpeg command that lands `src` as a browser-safe mp4 (faststart): re-encode only the
stream(s) that need it (libx264 / aac), stream-copy the rest, always into mp4. `-progress
pipe:1` streams machine-readable progress for run_ffmpeg. `threads` caps the encoder so a
re-encode can't claim every core on the shared box (see the worker-child de-prioritization)."""
cmd = [
"ffmpeg",
"-y",
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-progress",
"pipe:1",
"-nostats",
"-i",
str(src),
]
cmd += (
["-c:v", "libx264", "-preset", "veryfast", "-crf", "20"]
if reencode_video
else ["-c:v", "copy"]
)
cmd += ["-c:a", "aac", "-b:a", "192k"] if reencode_audio else ["-c:a", "copy"]
if threads and threads > 0:
cmd += ["-threads", str(threads)]
cmd += ["-movflags", "+faststart", str(dest)]
return cmd
# --- filmstrip (editor scrub track) ---------------------------------------------------------
_SB_COLS = 12
@@ -655,6 +610,7 @@ def build_storyboard_cmd(src: Path, dest: Path, duration_s: int | None, geom: di
"-loglevel",
"error",
"-nostdin",
*INPUT_GUARD,
"-i",
str(src),
"-frames:v",
@@ -678,6 +634,7 @@ def build_poster_cmd(src: Path, dest: Path, at_s: float) -> list[str]:
"-nostdin",
"-ss",
f"{max(0.0, at_s):.3f}",
*INPUT_GUARD,
"-i",
str(src),
"-frames:v",
@@ -699,6 +656,7 @@ def build_thumbnail_jpg_cmd(src: Path, dest: Path) -> list[str]:
"-loglevel",
"error",
"-nostdin",
*THUMB_GUARD,
"-i",
str(src),
"-frames:v",
@@ -734,7 +692,7 @@ def _step_failure(err: BaseException) -> str:
def _is_jpeg(path: Path) -> bool:
"""True if the file starts with the JPEG SOI marker (FF D8 FF)."""
"""True if the file starts with the JPEG SOI marker (0xFF 0xD8 0xFF)."""
try:
with path.open("rb") as f:
return f.read(3) == b"\xff\xd8\xff"
+30 -3
View File
@@ -37,9 +37,10 @@ from app.downloads.credentials import host_key_for_url
# Worker-authored strings that this module owns because each is coupled to `_RULES`.
#
# QUOTA_FULL_MESSAGE is raised by the worker and must classify back to `quota_exceeded`; its rule
# below matches a fragment of it, and test_download_errors pins the pair, because a message and its
# rule drifting apart is exactly how a category silently becomes unreachable.
# QUOTA_FULL_MESSAGE and the three *_TOO_BIG_MESSAGEs are raised by the worker and must classify
# back to `quota_exceeded`; their rules below match a fragment of each, and test_download_errors pins
# each one, because a message and its rule drifting apart is exactly how a
# category silently becomes unreachable.
#
# NETWORK_REDACTED_MESSAGE REPLACES a message after classification (`user_error_text`). It is written
# to the shared asset too, and a second job that inherits that failed asset re-diagnoses the stored
@@ -53,6 +54,20 @@ from app.downloads.credentials import host_key_for_url
# constants, imported above; each has a rule below that gives its category back, and a catalog entry
# that gives the site's sentence.
QUOTA_FULL_MESSAGE = "Your download storage is full — free up space, then try again."
# A download, a cut clip, or an upload's converted file, held to the space left, which is not zero:
# "full" would be untrue. All three classify by the one phrase they share.
DOWNLOAD_TOO_BIG_MESSAGE = (
"The video is larger than the download storage you have left — free up space, then try again."
)
CLIP_TOO_BIG_MESSAGE = (
"The clip is larger than the download storage you have left — "
"free up space or cut a shorter clip, then try again."
)
CONVERTED_TOO_BIG_MESSAGE = (
"The converted file is larger than the download storage you have left — "
"free up space, then try again."
)
UPLOAD_MISSING_MESSAGE = "The uploaded file is missing."
NETWORK_REDACTED_MESSAGE = "The server could not reach that address."
# (marker, category) — first match wins, so order most-specific → most-general. Matching is
@@ -61,6 +76,7 @@ _RULES: tuple[tuple[str, str], ...] = (
# Our own refusal: the worker found the user already at their storage cap (SEC-13). Placed
# first — it is our wording, so it can never be a false match on an upstream message.
("download storage is full", "quota_exceeded"),
("larger than the download storage you have left", "quota_exceeded"),
# Our own redaction of a transport failure (see NETWORK_REDACTED_MESSAGE) — our wording too.
(NETWORK_REDACTED_MESSAGE.lower(), "network"),
# Our own verdicts on a post link (adapters.py) — our wording too: its page holds no single video,
@@ -98,6 +114,8 @@ _RULES: tuple[tuple[str, str], ...] = (
# An edit's source download went away — our own wording, and MORE specific than the generic
# "no longer available" below, so it must be matched first.
("source download is no longer available", "source_missing"),
# An upload's original went away before its conversion ran — our wording (UPLOAD_MISSING_MESSAGE).
(UPLOAD_MISSING_MESSAGE.lower().rstrip("."), "upload_missing"),
("source file is missing", "source_missing"),
# Private / deleted / removed / terminated.
("private video", "unavailable"),
@@ -263,6 +281,15 @@ def classify(msg: str | None) -> str:
return "unknown"
def classify_conversion(msg: str | None) -> str:
"""The category of an upload's failed conversion. The generic ffmpeg categories speak of a
download ("the video downloaded but…"), which an upload never was: a failed encode or an
unrecognised failure of one is `convert_failed`; a category of its own (a full quota, a missing
original) stays as it is."""
code = classify(msg)
return "convert_failed" if code in ("postprocess", "unknown") else code
def diagnose(
msg: str | None, *, url: str | None = None, cookie_attached: bool = False
) -> Diagnosis:
+40 -10
View File
@@ -105,6 +105,28 @@ def log_safe(text: object) -> str:
return _PATH_RE.sub("…", text)
# yt-dlp's HTTP downloader, refusing a file announced larger than `max_filesize`, logs this line and
# RETURNS — no exception and no file (measured; test_formats holds it against the pinned yt-dlp).
_OVER_CAP_LINE = "File is larger than max-filesize"
_OVER_CAP: contextvars.ContextVar[list[bool] | None] = contextvars.ContextVar(
"formats_over_cap", default=None
)
@contextlib.contextmanager
def watching_size_cap() -> Iterator[list[bool]]:
"""While yt-dlp works inside it: a one-slot flag, set when it refused a file over the size cap
(`build_ydl_opts(max_bytes=…)`). It then returns as if it had downloaded, and all the caller
would find is no file — a processing fault, to anyone reading it, for what is the owner's
storage. A context variable, like `hiding`: yt-dlp's logger calls run on the extraction's thread."""
flag = [False]
token = _OVER_CAP.set(flag)
try:
yield flag
finally:
_OVER_CAP.reset(token)
class _YdlLog:
"""yt-dlp's logger protocol, pointed at ours.
@@ -124,6 +146,8 @@ class _YdlLog:
is the one line whose level should say so."""
def _emit(self, level: int, msg: str) -> None:
if _OVER_CAP_LINE in msg and (over := _OVER_CAP.get()) is not None:
over[0] = True
# The line goes as an ARGUMENT, not as the format string. yt-dlp writes progress as
# `[download] 53.2% of 12.00MiB`, and while stdlib logging only applies `%` formatting to
# a record that has args, this way the question never arises — and it is the shape ruff's
@@ -182,7 +206,9 @@ _DEFAULTS = {
# v4: EVERY download is clamped to the mp4/H.264/AAC compat profile (no more mkv/webm/VP9/AV1/Opus
# opt-out), so any asset cached under an old non-compat spec — or a stale non-H.264 mp4 that slipped
# the v3 net — must re-derive through the current browser-safe guarantee.
_SIG_VERSION = 4
# v5: the phone-safe rule (`playable`) also judges the pixel format, so a cached 10-bit / 4:4:4 H.264
# asset (which v4 let through as "H.264") re-derives as 8-bit 4:2:0 instead of being reused.
_SIG_VERSION = 5
# Audio-only downloads version on their own, so a change to the audio bytes doesn't re-download every
# cached video. v5: an extracted .mp3/.m4a is stored as itself instead of remuxed into an .mp4 that
# iPad's WebKit refuses to play, so a cached remuxed audio asset must re-derive.
@@ -219,12 +245,13 @@ def normalize(spec: dict) -> dict:
s["embed_subs"] = False
# Browser-playability guarantee: the Download module exists to download → share → PLAY
# in-browser, so every output must play in EVERY browser, iOS/Safari/WebKit included. The only
# universally-safe combo is mp4 + H.264 + AAC/MP3, so we CLAMP the spec to it HERE — the single
# point every path (picker, stored profile, raw API) flows through — instead of only hiding
# options in the UI. Non-mp4 containers (mkv doesn't play at all; webm/VP9/AV1 fail iOS) and any
# explicit vcodec are coerced to the compat profile; Opus audio (Safari-flaky as a bare file)
# falls back to AAC/m4a. A source that exposes no H.264 is still caught by the worker's
# browser-safe re-encode (`_ensure_browser_playable`), so playback is guaranteed end-to-end.
# universally-safe combo is mp4 + H.264 + AAC (an .mp3/.m4a for audio), so we CLAMP the spec to
# it HERE — the single point every path (picker, stored profile, raw API) flows through — instead
# of only hiding options in the UI. Non-mp4 containers (mkv plays nowhere on iOS; AV1 fails there,
# and webm/VP9 plays only on recent devices) and any explicit vcodec are coerced to the compat
# profile; Opus audio falls back to AAC/m4a. A source that exposes no H.264 (or a 10-bit one) is
# still caught by the worker's phone-safe step (`downloads.playable`), so playback is guaranteed
# end-to-end.
s["container"] = "mp4"
s["vcodec"] = None
if s["audio_format"] not in ("m4a", "mp3"):
@@ -278,8 +305,11 @@ def build_ydl_opts(
after the host-scope + private-asset gate in `downloads.credentials`. It is set AFTER the `extra`
merge so a source adapter can never clobber it.
`max_bytes` (SEC-13) is the caller's remaining storage budget; yt-dlp refuses a format larger
than it rather than filling the disk and only then being caught by the quota accounting."""
`max_bytes` (SEC-13) is the caller's remaining storage budget; yt-dlp's HTTP downloader refuses
a file announced larger than it rather than filling the disk and only then being caught by the
quota accounting. Its ffmpeg downloader (a `download_ranges` fetch) does not read it, nor does
a fragmented download (YouTube's formats: measured, 718 MB fetched with 1 MB left) — so the
pipeline also holds every produced file to what is left by its real size."""
s = normalize(spec)
h = s["max_height"]
opts: dict = {
@@ -351,7 +381,7 @@ def build_ydl_opts(
# inside mp4 — a VP9/AV1-in-mp4 file shows a broken player there — so H.264 is selected at the
# SELECTION level, not merely preferred: a sort key can't conjure an H.264 stream a source
# doesn't expose as video-only. A source that ships only non-H.264 is caught after download by
# the worker's browser-safe re-encode (`_ensure_browser_playable`).
# the worker's phone-safe step (`downloads.playable`).
if s["mode"] == "v":
# Video-only: keep it audio-free. Prefer an H.264 video-only stream, then any video-only,
# and only fall to a progressive (audio-bearing) `best` as a last resort.
+48 -17
View File
@@ -6,12 +6,13 @@ Five passes each run:
2. Expiry deletion — delete assets past their TTL (files + row). The FK SET NULL nulls the
holders' jobs' asset_id, so those jobs surface as "expired" (re-downloadable) in the UI.
3. LRU eviction — if a total cache cap is set and exceeded, delete least-recently-accessed
ready assets until back under it.
ready assets (never an upload) until back under it.
4. Orphan reap — delete errored, fileless, unreferenced asset rows (they carry no TTL, so
passes 1-3 skip them; benign row-count hygiene).
5. Orphan FILE sweep — delete on-disk files/dirs no surviving asset owns (cruft left when a
re-download renames an asset, e.g. old accented channel dirs), skipping system trees and
anything newer than an hour so an in-flight download is never touched.
re-download renames an asset, e.g. old accented channel dirs, or an upload's stranded file).
It skips the dot-prefixed system trees (all but `.uploads`) and anything newer than an hour,
so an in-flight download is never touched.
Runs in the API process (which mounts DOWNLOAD_ROOT). Pure disk/DB work — no YouTube quota.
"""
@@ -20,19 +21,19 @@ import time
from datetime import datetime, timedelta, timezone
from pathlib import Path
from sqlalchemy import func, select
from sqlalchemy import func, or_, select
from sqlalchemy.orm import Session
from app import logs, sysconfig
from app.config import settings
from app.downloads import storage
from app.downloads import quota, storage
from app.models import DownloadJob, MediaAsset
from app.notifications import create_notification
log = logs.get_logger(__name__)
# Dot-prefixed top-level trees that are NOT part of the Plex/flat asset layout — the orphan sweep
# never touches them (user clips, Plex HLS/img caches, in-progress staging, editor storyboards).
_SYSTEM_TREE = {".edits", ".plex-hls", ".plex-img-cache", ".staging", ".storyboards"}
# The one dot-prefixed top-level tree the orphan sweep enters. The others are not part of the asset
# layout (user clips, Plex HLS/img caches, in-progress staging, editor storyboards) and are skipped.
_UPLOAD_TREE = ".uploads"
_ORPHAN_MIN_AGE_S = 3600 # only sweep files older than this, so an in-flight download is never hit
@@ -67,16 +68,20 @@ def _delete_asset(db: Session, asset: MediaAsset, ntype: str) -> None:
def _sweep_orphan_files(db: Session, download_root: str) -> int:
"""Delete media/sidecar files (and now-empty dirs) under DOWNLOAD_ROOT that no `ready` asset
owns — cruft left when a re-download changes an asset's on-disk name (e.g. an old accented
channel dir, or a poster from a superseded naming scheme). Skips the dot-prefixed system trees
and only removes files older than `_ORPHAN_MIN_AGE_S`, so a just-completed or in-flight download
is never deleted, and only files NOT owned by a surviving asset."""
channel dir, or a poster from a superseded naming scheme) — or, under `.uploads`, that no upload
row names at all. Skips the other dot-prefixed system trees and only removes files older than
`_ORPHAN_MIN_AGE_S`, so a just-completed or in-flight download is never deleted, and only files
NOT owned by a surviving asset."""
root = Path(download_root)
if not root.is_dir():
return 0
keep: set[Path] = set()
# An upload's file is on disk from ingest on, whatever its state (a queued conversion's original,
# a failed one kept for Retry), so every upload row keeps its file — not only a ready one.
for rel in db.execute(
select(MediaAsset.rel_path).where(
MediaAsset.status == "ready", MediaAsset.rel_path.is_not(None)
or_(MediaAsset.status == "ready", MediaAsset.source_kind == "upload"),
MediaAsset.rel_path.is_not(None),
)
).scalars():
media = (root / rel).resolve()
@@ -99,7 +104,10 @@ def _sweep_orphan_files(db: Session, download_root: str) -> int:
unreclaimed = 0
for entry in root.iterdir():
if entry.name.startswith("."): # .edits / .plex-* / .staging / .storyboards + any dotfile
# .edits / .plex-* / .staging / .storyboards + any dotfile — but `.uploads` is swept: a
# conversion stopped between placing the new file and the commit that points at it (or the
# unlink of the original after it) leaves a file no row names, and nothing else finds it.
if entry.name.startswith(".") and entry.name != _UPLOAD_TREE:
continue
files = [entry] if entry.is_file() else [p for p in entry.rglob("*") if p.is_file()]
for f in files:
@@ -188,17 +196,17 @@ def run_download_gc(db: Session) -> dict:
if cap > 0:
total = (
db.execute(
select(func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(
MediaAsset.status == "ready"
)
select(func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(quota.ON_DISK)
).scalar()
or 0
)
if total > cap:
# An upload counts toward the total (it is disk like any other) but is never evicted:
# it has no source to fetch again, and it lives until its owner deletes it.
candidates = (
db.execute(
select(MediaAsset)
.where(MediaAsset.status == "ready")
.where(MediaAsset.status == "ready", MediaAsset.source_kind != "upload")
.order_by(MediaAsset.last_access_at.asc().nulls_first())
)
.scalars()
@@ -232,6 +240,29 @@ def run_download_gc(db: Session) -> dict:
if orphans:
db.commit()
# 4b. Reap unheld uploads. An upload carries no expires_at, is never evicted and never re-enqueued,
# so the passes above never free one that lost its last holder outside `release_asset` — a
# conversion that finished after its job was cancelled, or one cancelled mid-run (the asset
# was `downloading` then). Its file, ready or still the unconverted original, is at `rel_path`.
unheld = (
db.execute(
select(MediaAsset).where(
MediaAsset.source_kind == "upload",
MediaAsset.ref_count == 0,
MediaAsset.status != "downloading",
)
)
.scalars()
.all()
)
for asset in unheld:
if asset.rel_path: # no holder is left to tell, so no notice — just the file and the row
storage.delete_asset_files(settings.download_root, asset.rel_path)
db.delete(asset)
reaped += 1
if unheld:
db.commit()
# 5. Orphan FILE sweep — remove on-disk cruft no surviving asset owns (old accented dirs, stale
# posters). Runs last so the keep-set reflects the assets that survived passes 2-4.
swept = _sweep_orphan_files(db, settings.download_root)
+5 -4
View File
@@ -147,13 +147,14 @@ def _watch_tags(token: str, db: Session) -> str | None:
return "\n ".join(tags)
def render_watch_html(token: str, index_html: Path, db: Session) -> str | None:
"""Return index.html with the OG block replaced for this watch token, or None to serve the
generic page unchanged (invalid/expired/not-ready link, or a template without the markers)."""
def render_watch_html(token: str, page_html: Path, db: Session) -> str | None:
"""Return the watch page (watch.html) with the OG block replaced for this token, or None to
serve the generic page unchanged (invalid/expired/not-ready link, or a template without the
markers)."""
tags = _watch_tags(token, db)
if tags is None:
return None
text = index_html.read_text(encoding="utf-8")
text = page_html.read_text(encoding="utf-8")
if "<!--OG:START-->" not in text:
return None
# A function replacement avoids re.sub interpreting backslashes/group refs in `tags`.
+520 -55
View File
@@ -7,13 +7,16 @@ from datetime import date, datetime, timedelta, timezone
from pathlib import Path
import yt_dlp
from yt_dlp.utils import download_range_func
from app import logs, sysconfig
from app.config import settings
from app.db import SessionLocal
from app.downloads import clip as clipmod
from app.downloads import edit as editmod
from app.downloads import errors as dlerrors
from app.downloads import adapters, credentials, formats, mediakind, probe, quota, service, storage
from app.downloads import adapters, credentials, formats, mediakind, playable, probe, quota
from app.downloads import service, storage
from app.downloads import urlguard
from app.downloads.jobdb import (
_edit_staging_dir,
@@ -25,7 +28,9 @@ from app.downloads.jobdb import (
)
from app.downloads.stated import StatedFailure
from app.downloads.status import JobStatus
from app.models import DISPLAY_TEXT_MAX, DownloadJob, MediaAsset
from app.models import DISPLAY_TEXT_MAX, DownloadJob, MediaAsset, PlexItem
from app.plex import lookups as plex_lookups
from app.plex import paths as plex_paths
from app.notifications import create_notification
from app.titles import normalize_title
@@ -36,6 +41,15 @@ class _Aborted(Exception):
"""Raised inside the progress hook when the job was paused/canceled mid-download."""
class _ClipFetchFailed(Exception):
"""A clip's YouTube range fetch failed: its message and the download catalog's diagnosis, so a
clip on YouTube's wall says what a download of the same video would."""
def __init__(self, msg: str, diagnosis: dlerrors.Diagnosis):
super().__init__(msg)
self.diagnosis = diagnosis
class _OwnerRefused(Exception):
"""A refusal that belongs to THIS job's owner, not to the source.
@@ -58,6 +72,10 @@ _DB_WRITE_INTERVAL_S = 1.0
_CANCEL_POLL_INTERVAL_S = 1.0
_THUMB_SUFFIXES = (".jpg", ".jpeg", ".png", ".webp")
# The job phase while a file is brought into the phone-safe shape (`_ensure_phone_safe`) for a
# download or an upload. Its own word, not yt-dlp's "processing": it reports ffmpeg's real progress,
# so the queue row shows a % for it, where yt-dlp's post-steps only get an indeterminate pulse.
CONVERTING = "converting"
# yt-dlp's working files in the staging dir. None of them is ever the finished media, but several
# CAN be the biggest file there (a video-only per-format stream, an abandoned fragment run), so the
# staging fallback must exclude them by name — picking one would store e.g. a silent video-only
@@ -75,7 +93,7 @@ def _notify_download_done(db, user_id: int | None, title: str | None, job_id, *,
"""Post a best-effort 'ready' notice to the owner's inbox — downloads/edits used to finish
silently while scheduler jobs and playlist-push already notified (feedback symmetry). A
notification failure must never mask the completed job, so it is isolated + rolled back (mirrors
scheduler._notify_done). `kind` ("download"/"edit") lets the inbox pick accurate copy; the
scheduler._notify_done). `kind` ("download"/"edit"/"upload") lets the inbox pick accurate copy; the
English title here is only a fallback for an unknown type. `user_id`/`title` are captured by the
caller BEFORE its commit, so reading them here needs no post-commit refresh of an expired row."""
if user_id is None:
@@ -208,7 +226,7 @@ def _cancel_poll(job_id: int, interval_s: float = _CANCEL_POLL_INTERVAL_S, *, no
def cancelled() -> bool:
if not state["cancelled"] and allow():
state["cancelled"] = _job_status(job_id) not in ("running", None)
state["cancelled"] = _job_status(job_id) not in (JobStatus.RUNNING, None)
return state["cancelled"]
return cancelled
@@ -328,7 +346,7 @@ def _media_info(info: dict) -> dict:
def _extract_with_fallback(
job_id, asset_id, url, spec, staging, adapter, cookiefile=None, max_bytes=None
job_id, asset_id, url, spec, staging, adapter, cookiefile=None, max_bytes=None, section=None
):
"""Download via yt-dlp using the source `adapter`'s client sets + options. For YouTube the adapter
supplies the primary player-client set then a POT-backed fallback (retried on a bot/DRM/format
@@ -342,9 +360,16 @@ def _extract_with_fallback(
download as itself.
`cookiefile` is the acting user's per-host cookies.txt, already gated by
`credentials.resolve_cookiefile`; passed to every attempt so the fallback stays authenticated.
`credentials.resolve_cookiefile` (a clip: `resolve_clip_cookiefile`); passed to every attempt so
the fallback stays authenticated.
`max_bytes` (SEC-13) is the owner's remaining storage budget, applied to every attempt."""
`max_bytes` (SEC-13) is the owner's remaining storage budget, applied to every attempt. A
`section` fetch goes through yt-dlp's ffmpeg downloader, which does not read it: a clip holds
its own output to the cap (`_process_clip`).
`section` = (start_s, end_s) fetches only that range (an ad-hoc clip, see downloads/clip): the
range is stream-copied, and its first frame is the marked one (measured), so the clip's own
re-encode starts at 0 instead of re-fetching keyframes at the cut."""
client_sets = adapter.client_sets(authed=cookiefile is not None)
outtmpl = str(staging / "%(id)s.%(ext)s")
# Resolving may read a page first — network time the stall watchdog must not see as a stall.
@@ -370,6 +395,9 @@ def _extract_with_fallback(
max_bytes=max_bytes,
)
opts["postprocessor_hooks"] = [_make_pp_hook(job_id)]
if section is not None:
opts["download_ranges"] = download_range_func([], [section])
opts["force_keyframes_at_cuts"] = False
_touch_job(job_id) # stall-watchdog heartbeat: extraction writes no progress, so a slow /
# multi-client-set fallback would otherwise cross the stall timeout with a stale updated_at.
# (Per-ATTEMPT only; a single extract_info that itself runs past the stall timeout is still
@@ -416,48 +444,50 @@ def _download_settings() -> tuple[str, int]:
)
def _ensure_browser_playable(job_id: int, produced: Path, spec: dict, info: dict) -> Path:
"""Make an mp4-compat-profile download play in every browser: H.264 + AAC/MP3 in mp4. If the
download (whatever the source shipped) isn't already that, re-encode the offending stream(s) —
or just remux when the codecs are fine but the container isn't. **Scope:** only the mp4 compat
profile is normalized; an explicit `vcodec` or a non-mp4 custom profile is the user opting out
of the guarantee and is left as-is (so a `webm` profile can still ship VP9/Opus). Returns the
file to store (converted, or the original) and corrects `info`'s codec fields to match disk.
def _ensure_phone_safe(
job_id: int,
produced: Path,
out_dir: Path,
phase: str,
total_s: int | None = None,
*,
keep_source: bool = False,
) -> Path:
"""Bring a file that is about to become a ready library asset into the phone-safe shape
(`playable`), writing any converted copy into `out_dir`. Returns the file to store — `produced`
itself when it already is that shape, or isn't media at all — and removes `produced` once a
converted copy exists, unless `keep_source` (an upload's original, which must outlive the
conversion until the database points at the new file). The job shows `phase` with real ffmpeg
progress while it converts.
An audio download (mode "a") is never touched: yt-dlp already extracted it to .mp3 / .m4a, which
every browser plays as audio/mpeg / audio/mp4. Remuxing it into .mp4 served an mp3 as video/mp4,
which iPad's WebKit refuses to play (prod job 268)."""
if spec.get("mode") == "a" or spec.get("container") != "mp4" or spec.get("vcodec"):
Every ready path calls this — a download, an upload's worker conversion and an editor clip — so
a share link can only ever serve the one shape every phone and browser plays."""
p = playable.plan(probe.probe_json(produced), produced.suffix)
if p is None or not p.needs_change:
return produced
vcodec, acodec = editmod.probe_codecs(produced)
is_mp4 = produced.suffix.lower() == ".mp4"
needs, reenc_v, reenc_a = editmod.browser_transcode_flags(vcodec, acodec, is_mp4)
if not needs:
return produced
dest = produced.with_name(produced.stem + ".browser.mp4")
action = "re-encode" if (reenc_v or reenc_a) else "remux"
dest = out_dir / f"{produced.stem}.safe.{p.ext}"
action = "re-encode" if (p.reencode_video or p.reencode_audio) else "remux"
log.info(
"job %s: %s (v=%s a=%s mp4=%s) -> browser-safe mp4", job_id, action, vcodec, acodec, is_mp4
"job %s: %s (video=%s audio=%s) -> phone-safe .%s",
job_id,
action,
p.reencode_video,
p.reencode_audio,
p.ext,
extra=logs.event("download.phone_safe", job_id=job_id, action=action, ext=p.ext),
)
_set_job(job_id, phase="processing", progress=0, speed_bps=None, eta_s=None)
# Prefer the info duration; probe the file when the extractor didn't provide one, so a long
# re-encode still shows real progress instead of a frozen 0%.
total_s = int(info["duration"]) if info.get("duration") else editmod.probe_duration(produced)
_set_job(job_id, phase=phase, progress=0, speed_bps=None, eta_s=None)
# A re-encode with no known length would hold a frozen 0%; probe the file for a denominator.
editmod.run_ffmpeg(
editmod.build_browser_safe_cmd(
produced, dest, reenc_v, reenc_a, threads=editmod.encode_thread_cap()
),
total_s=total_s,
on_progress=_ffmpeg_progress_writer(job_id, "processing"),
playable.build_cmd(produced, dest, p, threads=editmod.encode_thread_cap()),
total_s=total_s or editmod.probe_duration(produced),
on_progress=_ffmpeg_progress_writer(job_id, phase),
should_cancel=_cancel_poll(job_id),
)
produced.unlink(missing_ok=True)
# Keep the stored asset metadata truthful about what's now on disk.
if reenc_v:
info["vcodec"] = "h264"
if reenc_a:
info["acodec"] = "aac"
if not dest.exists():
raise StatedFailure("ffmpeg produced no output file")
if not keep_source:
produced.unlink(missing_ok=True)
return dest
@@ -495,8 +525,8 @@ def _download(
# SEC-13: what this user has left. `check_enqueue` only compared the footprint as it
# stood when the job was QUEUED — a download's size is unknown until it finishes, so a
# single job could previously blow through the cap by any amount. Handing the remainder
# to yt-dlp as `max_filesize` makes the format selector refuse an oversized file up
# front instead.
# to yt-dlp as `max_filesize` makes it refuse an oversized file (its HTTP downloader
# checks the announced size) instead.
remaining_bytes = quota.remaining_bytes(cdb, user_id)
# …and zero is a REFUSAL, not "no limit". Earlier jobs of this user may have filled the cap
# while this one sat in the queue, and 0 is not an expressible yt-dlp `max_filesize`, so
@@ -505,8 +535,8 @@ def _download(
# free space, which is what the `quota_exceeded` category tells them.
if remaining_bytes == 0:
raise _OwnerRefused(dlerrors.QUOTA_FULL_MESSAGE)
info = _media_info(
_extract_with_fallback(
with formats.watching_size_cap() as over_cap:
fetched = _extract_with_fallback(
job_id,
asset_id,
url,
@@ -516,13 +546,30 @@ def _download(
cookiefile=str(cookiefile) if cookiefile else None,
max_bytes=remaining_bytes,
)
)
info = _media_info(dict(fetched))
# yt-dlp refused a file larger than what is left: it returns with no file, which would
# otherwise read as a processing fault. The owner's storage, so the owner's refusal.
if over_cap[0]:
raise _OwnerRefused(dlerrors.DOWNLOAD_TOO_BIG_MESSAGE)
# Re-resolve now that yt-dlp's info reveals the true source (a bare/redirected url only
# identifies post-extraction) so the .nfo uniqueid type + uploader_url agree on one source.
adapter = adapters.resolve_adapter(source_kind, source_ref, info)
produced = _produced_file(info, staging)
produced = _ensure_browser_playable(job_id, produced, spec, info)
produced = _ensure_phone_safe(
job_id,
produced,
staging,
CONVERTING,
int(info["duration"]) if info.get("duration") else None,
)
# Held to the cap by its real size, as a clip is: yt-dlp's `max_filesize` only stops a plain
# HTTP file that announces its size, and YouTube's fragmented formats never meet it
# (measured: 718 MB fetched with 1 MB left). Read again — other jobs may have landed.
with SessionLocal() as qdb:
room = quota.remaining_bytes(qdb, user_id)
if room is not None and produced.stat().st_size > room:
raise _OwnerRefused(dlerrors.DOWNLOAD_TOO_BIG_MESSAGE)
ext = produced.suffix.lstrip(".").lower()
# Codecs/dims/duration come from a PROBE of the produced file — the ground truth — NOT yt-dlp's
# `info` dict, whose fields are absent for some sources (Facebook among them): deriving
@@ -722,9 +769,19 @@ def _log_download_failed(job_id: int, msg: str, code: str | None, exc: BaseExcep
)
def _fail_edit(job_id: int, asset_id: int, msg: str, exc: BaseException | None = None) -> None:
"""Fail an edit job, and say so ONCE — every way an edit fails comes through here, so this is
where its line and its event code live. `exc` is the exception when there was one.
def _fail_edit(
job_id: int,
asset_id: int,
msg: str,
exc: BaseException | None = None,
*,
what: str = "edit",
diagnosis: dlerrors.Diagnosis | None = None,
) -> None:
"""Fail a local ffmpeg job — an edit, a clip, or an upload's conversion (`what="upload"`) — and
say so ONCE: every way one fails comes through here, so this is where its line and its event code
(`<what>.job.failed`) live. `exc` is the exception when there was one; `diagnosis` replaces the
plain category when the failure was a fetch the download catalog can name (a clip's).
Its traceback goes with the line unless the message already tells it all (`_failure_stack`).
Logged after the write, as on the download error path."""
@@ -739,19 +796,25 @@ def _fail_edit(job_id: int, asset_id: int, msg: str, exc: BaseException | None =
job_id,
status=JobStatus.ERROR,
error=msg,
error_code=dlerrors.classify(msg),
error_code=diagnosis.code
if diagnosis
else dlerrors.classify_conversion(msg)
if what == "upload"
else dlerrors.classify(msg),
error_detail=diagnosis.detail if diagnosis else None,
phase=None,
)
db.commit()
# `msg` is the user's copy; ffmpeg's stderr tail in it can name the source file, whose path is
# built from the channel and the title.
log.warning(
"edit job %s failed: %s",
"%s job %s failed: %s",
what,
job_id,
formats.log_safe(msg),
extra={
**logs.event(
"edit.job.failed",
f"{what}.job.failed",
job_id=job_id,
asset_id=asset_id,
**({"error_type": type(exc).__name__} if exc is not None else {}),
@@ -811,11 +874,24 @@ def _process_edit(job_id: int, asset_id: int) -> None:
)
if not dest_staging.exists():
raise StatedFailure("ffmpeg produced no output file")
# A fast trim stream-copies the source's codecs and container, so the clip is only as
# playable as its source: hold it to the same rule as every other ready file.
dest_staging = _ensure_phone_safe(job_id, dest_staging, staging, "editing", total_s)
out_ext = dest_staging.suffix.lstrip(".").lower() or out_ext
# New storage on the owner's footprint, held to what is left by its real size, as a
# download and a player clip are.
if user_id is not None:
with SessionLocal() as qdb:
room = quota.remaining_bytes(qdb, user_id)
if room is not None and dest_staging.stat().st_size > room:
raise StatedFailure(dlerrors.CLIP_TOO_BIG_MESSAGE)
rel = storage.edit_rel_path(user_id, asset_id, out_ext)
storage.place_file(dest_staging, settings.download_root, rel)
final = storage.abs_path(settings.download_root, rel)
size = final.stat().st_size if final.exists() else None
# The clip's real frame: a re-encode evens an odd crop or source (`editmod.EVEN_FRAME`).
_, _, out_w, out_h, _ = probe.probe_media(final)
_, retention_days = _download_settings()
expires = datetime.now(timezone.utc) + timedelta(days=retention_days)
@@ -827,7 +903,9 @@ def _process_edit(job_id: int, asset_id: int) -> None:
asset.size_bytes = size
asset.container = out_ext
asset.duration_s = total_s or asset.duration_s
if crop:
if out_w and out_h:
asset.width, asset.height = out_w, out_h
elif crop:
asset.width = edit_spec["crop"]["w"]
asset.height = edit_spec["crop"]["h"]
else:
@@ -867,3 +945,390 @@ def _process_edit(job_id: int, asset_id: int) -> None:
_fail_edit(job_id, asset_id, str(exc)[:500], exc)
finally:
shutil.rmtree(staging, ignore_errors=True)
def _clip_room(user_id: int) -> int | None:
"""What the clip's owner may still store (`quota.remaining_bytes`), refusing at zero — see
`_download`: zero is a refusal, not "no limit"."""
with SessionLocal() as db:
remaining = quota.remaining_bytes(db, user_id)
if remaining == 0:
raise _OwnerRefused(dlerrors.QUOTA_FULL_MESSAGE)
return remaining
def _clip_source(
job_id: int, asset_id: int, spec: dict, staging: Path, remaining: int | None
) -> tuple[Path, float, str | None]:
"""The file a clip is cut from, where in it the clip starts, and the source's title when only
the fetch could tell it. A YouTube clip fetches ONLY its range into `staging` (so it starts at
0 there); a Plex clip reads the library file in place."""
if spec["source"] == "youtube":
url = service.source_url("youtube", spec["ref"])
cookiefile: Path | None = None
with SessionLocal() as db:
job = db.get(DownloadJob, job_id)
asset = db.get(MediaAsset, asset_id)
# Signed in when the user has a YouTube cookie, as their downloads are: YouTube gates
# single videos signed out ("not a bot"), and the clip is theirs alone.
if job and asset:
cookiefile = credentials.resolve_clip_cookiefile(
db, job.user_id, url, asset, spec, staging
)
_set_job(job_id, phase="downloading", progress=0)
try:
with formats.watching_size_cap() as over_cap:
info = _extract_with_fallback(
job_id,
asset_id,
url,
formats.normalize(clipmod.YOUTUBE_FETCH_SPEC),
staging,
adapters.resolve_adapter("youtube", spec["ref"]),
cookiefile=str(cookiefile) if cookiefile else None,
max_bytes=remaining,
section=(spec["start_s"], spec["end_s"]),
)
# Should yt-dlp's own cap ever refuse the range (see `_download`): the owner's storage.
if over_cap[0]:
raise _OwnerRefused(dlerrors.CLIP_TOO_BIG_MESSAGE)
except (_Aborted, _OwnerRefused):
raise
except Exception as exc: # re-raised below with the download catalog's diagnosis
msg = str(exc)[:500]
raise _ClipFetchFailed(
msg, dlerrors.diagnose(msg, url=url, cookie_attached=cookiefile is not None)
) from exc
finally:
# Never leave a plaintext session cookie on disk past the fetch (see `_download`).
if cookiefile is not None:
cookiefile.unlink(missing_ok=True)
info = _media_info(dict(info))
title = normalize_title(adapters.resolve_adapter("youtube", spec["ref"]).title(info))
return _produced_file(info, staging), 0.0, title
with SessionLocal() as db:
item = plex_lookups.by_rating_key(db, PlexItem, spec["ref"])
src = plex_paths.local_media_path(db, item.file_path) if item else None
if src is None:
raise StatedFailure("The library file is not reachable from here.")
return src, spec["start_s"], None
def _clip_crop(spec: dict, data: dict) -> str:
"""The clip's crop filter, from its frame and the picture the file really holds ("" for none)."""
frame = spec.get("crop")
if not frame:
return ""
dar = clipmod.display_aspect(data)
if dar is None:
raise StatedFailure("The picture's size could not be read, so the crop cannot be placed.")
crop = clipmod.crop_filter(frame, dar)
if crop is None:
raise StatedFailure("The crop frame lies outside the picture.")
return crop
def _process_clip(job_id: int, asset_id: int) -> None:
"""Cut an ad-hoc clip (downloads/clip): fetch or open its source, ONE re-encode to the clip
shape, then into the user's library beside their edits."""
with SessionLocal() as db:
job = db.get(DownloadJob, job_id)
if job is None:
return
spec = dict(job.edit_spec or {})
user_id = job.user_id
staging = _edit_staging_dir(asset_id)
try:
if staging.exists():
shutil.rmtree(staging, ignore_errors=True)
staging.mkdir(parents=True, exist_ok=True)
# Refused before the fetch and the encode when the owner's storage is already full.
remaining = _clip_room(user_id)
src, start, fetched_title = _clip_source(job_id, asset_id, spec, staging, remaining)
data = probe.probe_json(src)
if not data:
raise StatedFailure("The source file could not be read.")
audio_track = spec.get("audio_track")
tracks = clipmod.audio_track_count(data)
if tracks == 0:
# A silent source: the player still names its track 0, and the clip is silent too.
audio_track = None
elif audio_track is not None and audio_track >= tracks:
raise StatedFailure("The chosen audio track is not in the file.")
crop = _clip_crop(spec, data)
total_s = clipmod.duration(spec)
out = staging / "clip.mp4"
_set_job(job_id, phase="editing", progress=0, speed_bps=None, eta_s=None)
editmod.run_ffmpeg(
clipmod.build_cmd(
src,
out,
start,
total_s,
audio_track=audio_track,
hdr=clipmod.is_hdr(data),
threads=editmod.encode_thread_cap(),
crop=crop,
),
total_s,
on_progress=_ffmpeg_progress_writer(job_id, "editing"),
should_cancel=_cancel_poll(job_id),
)
if not out.exists():
raise StatedFailure("ffmpeg produced no output file")
# The phone-safe shape by construction; still held to the rule every ready file meets.
out = _ensure_phone_safe(job_id, out, staging, "editing", round(total_s))
# Held to the cap by its real size: a range fetch passes yt-dlp's `max_filesize` to its
# ffmpeg downloader, which never reads it, and a Plex clip fetches nothing at all. Read
# again, since other jobs of this owner may have landed during the encode.
remaining = _clip_room(user_id)
if remaining is not None and out.stat().st_size > remaining:
raise _OwnerRefused(dlerrors.CLIP_TOO_BIG_MESSAGE)
ext = out.suffix.lstrip(".").lower() or "mp4"
rel = storage.edit_rel_path(user_id, asset_id, ext)
storage.place_file(out, settings.download_root, rel)
final = storage.abs_path(settings.download_root, rel)
vcodec, acodec, width, height, _ = probe.probe_media(final)
_, retention_days = _download_settings()
with SessionLocal() as db:
asset = db.get(MediaAsset, asset_id)
if asset:
asset.status = "ready"
asset.rel_path = rel
asset.size_bytes = final.stat().st_size if final.exists() else None
asset.container = ext
asset.vcodec, asset.acodec, asset.width, asset.height = (
vcodec,
acodec,
width,
height,
)
asset.duration_s = round(total_s)
asset.media_kind = mediakind.derive(vcodec, acodec)
# The clip's own frame as its poster: the link preview shows the moment, and a Plex
# item has no public thumbnail at all.
asset.poster_path = editmod.ensure_poster(asset, settings.download_root)
asset.error = None
asset.last_access_at = datetime.now(timezone.utc)
asset.expires_at = datetime.now(timezone.utc) + timedelta(days=retention_days)
fields: dict = {"status": JobStatus.DONE, "progress": 100, "phase": None}
# A video outside the catalog was queued as "Clip (clip)": the fetch has its title now.
placeholder = service.clip_title(None, None)
if asset and fetched_title and asset.title == placeholder:
named = service.clip_title(fetched_title, None)
asset.title = named
if job_row := db.get(DownloadJob, job_id):
if job_row.display_name == placeholder:
fields["display_name"] = named[:DISPLAY_TEXT_MAX]
title = asset.title if asset else None
finalized = service.finalize_if_running(db, job_id, **fields)
db.commit()
if finalized:
_notify_download_done(db, user_id, title, job_id, kind="edit")
log.info(
"clip job %s done: asset %s",
job_id,
asset_id,
extra=logs.event(
"clip.job.done", job_id=job_id, asset_id=asset_id, source=spec["source"]
),
)
except (_Aborted, editmod.EditAborted):
# Paused/canceled mid-fetch or mid-encode: free the asset for a later retry.
_reset_asset_pending(asset_id)
log.info(
"clip job %s aborted (pause/cancel)",
job_id,
extra=logs.event("clip.job.aborted", job_id=job_id),
)
except _OwnerRefused as exc:
# Quota full: the job fails, the asset goes back to pending for a Retry once there is room
# (the download's handling; see `_download`).
msg = str(exc)[:500]
with SessionLocal() as db:
service.finalize_if_running(
db,
job_id,
status=JobStatus.ERROR,
error=msg,
error_code=dlerrors.classify(msg),
error_detail=None,
phase=None,
)
db.commit()
_reset_asset_pending(asset_id)
log.info(
"clip job %s refused for its owner: %s",
job_id,
formats.log_safe(msg),
extra=logs.event("clip.job.refused", job_id=job_id),
)
except _ClipFetchFailed as exc:
# silent-ok: not silent — `_fail_edit` is the report, with the fetch's diagnosis.
_fail_edit(job_id, asset_id, str(exc), exc, what="clip", diagnosis=exc.diagnosis)
except Exception as exc: # noqa: BLE001 — surface the failure, keep the worker alive
# silent-ok: not silent — `_fail_edit` is the report, and it takes `exc` for its stack.
_fail_edit(job_id, asset_id, str(exc)[:500], exc, what="clip")
finally:
shutil.rmtree(staging, ignore_errors=True)
def _check_converted_fits(owner_id: int | None, old_size: int, new_size: int) -> None:
"""Refuse a converted upload that would push its owner over the storage cap: the upload was
charged at its original size, and a re-encode can come out larger. The original stays, counted,
for the owner to delete or keep."""
if owner_id is None:
return
with SessionLocal() as db:
if not quota.fits_after_conversion(db, owner_id, old_size, new_size):
raise StatedFailure(dlerrors.CONVERTED_TOO_BIG_MESSAGE)
def _unplace(placed: Path | None) -> None:
"""Remove a converted upload that was placed at its new rel but never committed: the row still
names the original, so nothing else would ever find this file."""
if placed is not None:
placed.unlink(missing_ok=True)
def _conversion_landed(asset_id: int, new_rel: str) -> bool | None:
"""After the commit that makes a converted upload ready RAISED: did it land anyway? A commit can
reach the server and lose its answer (the connection drops after COMMIT). True/False by what the
row now says; None when the row cannot be read either."""
try:
with SessionLocal() as db:
asset = db.get(MediaAsset, asset_id)
return asset is not None and asset.status == "ready" and asset.rel_path == new_rel
except Exception: # noqa: BLE001 — "cannot tell" is the answer, the caller acts on it
# silent-ok: the caller re-raises the commit's own error, which `_fail_edit` reports.
return None
def _drop_original(job_id: int, src: Path) -> None:
"""Remove an upload's original once the converted file is committed in its place. Best effort:
the job is done and its file is ready, so a failure here must not fail it — the original is
named by no row now, and the GC's `.uploads` sweep takes it."""
try:
src.unlink(missing_ok=True)
except OSError as exc:
log.warning(
"upload job %s: original not removed, left to the GC sweep: %s",
job_id,
exc,
extra=logs.event("upload.original.unlink_failed", job_id=job_id),
)
def _free_unheld_upload(asset_id: int) -> bool:
"""Free an upload no job holds any more — cancelled or deleted while its conversion ran, when
`release_asset` had to leave it because the conversion still read its file. Once that has
stopped, free it as the release would have; left alone, its file would sit on disk counted by no
owner's footprint until the GC's upload pass. False when a job still holds it (paused)."""
with SessionLocal() as db:
asset = db.get(MediaAsset, asset_id)
if asset is None or asset.ref_count != 0:
return False
if asset.rel_path:
storage.delete_asset_files(settings.download_root, asset.rel_path)
db.delete(asset)
db.commit()
return True
def _convert_upload(job_id: int, asset_id: int) -> None:
"""Convert a queued upload into the phone-safe shape (`playable`) and make it ready.
The original sits at the asset's `rel_path` (see `upload`); the converted file replaces it there
under the new extension, and only then is the original removed — so a failed or stopped
conversion leaves the original in place for Retry, and a delete still finds it by `rel_path`.
A converted file placed but never committed is removed again on the way out. When the commit's
own outcome is unknown neither file is removed: the row names one of them, and the GC's `.uploads`
sweep takes the one it does not — as it does an original whose unlink after the commit failed."""
with SessionLocal() as db:
asset = db.get(MediaAsset, asset_id)
old_rel = asset.rel_path if asset else None
old_ext = asset.container if asset else None
old_size = (asset.size_bytes or 0) if asset else 0
duration = asset.duration_s if asset else None
job = db.get(DownloadJob, job_id)
owner_id = job.user_id if job else None
src = storage.safe_abs_path(settings.download_root, old_rel) if old_rel else None
if old_rel is None or src is None or not src.exists():
_fail_edit(job_id, asset_id, dlerrors.UPLOAD_MISSING_MESSAGE, what="upload")
return
staging = _staging_dir(asset_id)
placed: Path | None = None # the converted file while it is at its new rel, uncommitted
try:
shutil.rmtree(staging, ignore_errors=True)
staging.mkdir(parents=True, exist_ok=True)
safe = _ensure_phone_safe(job_id, src, staging, CONVERTING, duration, keep_source=True)
ext = safe.suffix.lstrip(".").lower()
new_rel = storage.with_ext(old_rel, old_ext, ext)
if safe != src:
_check_converted_fits(owner_id, old_size, safe.stat().st_size)
storage.place_file(safe, settings.download_root, new_rel)
if new_rel != old_rel:
placed = storage.abs_path(settings.download_root, new_rel)
final = storage.abs_path(settings.download_root, new_rel)
vcodec, acodec, width, height, dur = probe.probe_media(final)
with SessionLocal() as db:
asset = db.get(MediaAsset, asset_id)
if asset:
asset.status = "ready"
asset.rel_path = new_rel
asset.container = ext
asset.size_bytes = final.stat().st_size
asset.vcodec, asset.acodec = vcodec, acodec
asset.width, asset.height = width or asset.width, height or asset.height
asset.duration_s = dur or asset.duration_s
asset.error = None
asset.last_access_at = datetime.now(timezone.utc)
title = asset.title if asset else None
finalized = service.finalize_if_running(
db, job_id, status=JobStatus.DONE, progress=100, phase=None
)
try:
db.commit()
except Exception:
landed = _conversion_landed(asset_id, new_rel)
if landed is False:
raise # rolled back: the original is still the asset's file — fail as any step
# Landed, or cannot tell: the converted file may be the asset's file now, so it is
# never removed blind (the sweep takes whichever file no row names).
placed = None
if landed is None:
raise
db.rollback()
placed = None
if finalized:
_notify_download_done(db, owner_id, title, job_id, kind="upload")
if new_rel != old_rel:
_drop_original(job_id, src)
if not finalized:
_free_unheld_upload(asset_id) # finished after its job was cancelled or deleted
if finalized:
log.info(
"upload job %s converted: asset %s",
job_id,
asset_id,
extra=logs.event("upload.job.done", job_id=job_id, asset_id=asset_id, ext=ext),
)
except editmod.EditAborted:
# Paused/canceled mid-encode: the original is untouched. A paused job keeps it for its
# resume; one cancelled or deleted holds it no more, so it is freed now.
_unplace(placed)
if not _free_unheld_upload(asset_id):
_reset_asset_pending(asset_id)
log.info(
"upload job %s conversion aborted (pause/cancel)",
job_id,
extra=logs.event("upload.job.aborted", job_id=job_id),
)
except Exception as exc: # noqa: BLE001 — surface the failure, keep the worker alive
# silent-ok: not silent — `_fail_edit` is the report, and it takes `exc` for its stack.
_unplace(placed)
_fail_edit(job_id, asset_id, str(exc)[:500], exc, what="upload")
finally:
shutil.rmtree(staging, ignore_errors=True)
+103
View File
@@ -0,0 +1,103 @@
"""The phone-safe rule: what a library file must be so it plays wherever a share link lands.
A shared `/watch` link is opened by whoever got it, on whatever they hold: an iPhone's WebKit, an
Android Chrome, a desktop Firefox. The one shape every one of them plays:
* video → H.264, 8-bit 4:2:0 (`yuv420p`, or its full-range twin `yuvj420p`), AAC audio, in .mp4
* audio only → MP3 in .mp3, or AAC in .m4a
Measured on the owner's iPhone + iPad (2026-10-05): AV1, Matroska, HEVC tagged `hev1` and MP3
inside an .mp4 all fail there. VP9/WebM, HEVC `hvc1` and 10-bit H.264 played on those devices, but
Chrome and Firefox do not decode HEVC and browsers decode 10-bit / 4:4:4 H.264 only patchily, so the
rule is the shape every browser plays, not the shape one device plays.
Every file that becomes a ready library asset passes through here — a download, an upload (the
worker converts it, see `upload`), and an editor clip — so a link can only ever serve this shape.
`plan` says what (if anything) must change; `build_cmd` is the one ffmpeg command that changes it:
re-encode only the stream that fails, copy the rest, or just remux when only the container is wrong.
"""
from dataclasses import dataclass
from pathlib import Path
from app.downloads import edit as editmod
from app.downloads import probe
_SAFE_VCODEC = "h264"
_SAFE_PIX_FMTS = frozenset({"yuv420p", "yuvj420p"})
_VIDEO_EXT = "mp4"
# An audio-only file: the codec, and the extension it plays in. Anything else becomes AAC in .m4a.
_AUDIO_HOME = {"mp3": "mp3", "aac": "m4a"}
_AUDIO_FALLBACK_EXT = "m4a"
@dataclass(frozen=True)
class Plan:
"""What makes one file phone-safe. `ext` is the extension the safe file has (no dot)."""
ext: str
has_video: bool
reencode_video: bool
reencode_audio: bool
needs_change: bool
def plan(data: dict | None, ext: str) -> Plan | None:
"""The phone-safe plan for a file whose `probe.probe_json` is `data` and whose extension is
`ext`; None when it is not media at all (an unreadable probe, a still image, no audio or video
stream) — such a file is a download-only 'file' and never plays inline, so there is nothing to
make safe."""
if data is None:
return None
video, audio = probe.primary_streams(data)
ext = ext.lower().lstrip(".")
if video is not None:
rv = (video.get("codec_name") or "").lower() != _SAFE_VCODEC or (
video.get("pix_fmt") or ""
).lower() not in _SAFE_PIX_FMTS
ra = audio is not None and (audio.get("codec_name") or "").lower() != "aac"
return Plan(_VIDEO_EXT, True, rv, ra, rv or ra or ext != _VIDEO_EXT)
if audio is not None:
home = _AUDIO_HOME.get((audio.get("codec_name") or "").lower())
target = home or _AUDIO_FALLBACK_EXT
return Plan(target, False, False, home is None, home is None or ext != target)
return None
def build_cmd(src: Path, dest: Path, p: Plan, threads: int | None = None) -> list[str]:
"""The ffmpeg command that writes `src` to `dest` (whose suffix is `p.ext`) in the phone-safe
shape: the failing stream(s) re-encoded with the package's shared encoder settings, the rest
stream-copied. An audio-only result drops any cover-art stream. `-progress pipe:1`
streams machine-readable progress; `threads` caps the encoder so a re-encode cannot claim every
core on the shared box (see the worker-child de-prioritization)."""
cmd = [
"ffmpeg",
"-y",
"-hide_banner",
"-loglevel",
"error",
"-nostdin",
"-progress",
"pipe:1",
"-nostats",
*editmod.INPUT_GUARD,
"-i",
str(src),
]
# Exactly the two streams `plan` judged: `V` is a video stream that is not cover art. Nothing
# else rides along — a subtitle or data track the .mp4 muxer cannot take would fail the run.
if p.has_video:
cmd += ["-map", "0:V:0", "-map", "0:a:0?"]
if p.reencode_video:
cmd += ["-vf", editmod.EVEN_FRAME, *editmod.X264_ARGS]
else:
cmd += ["-c:v", "copy"]
else:
cmd += ["-map", "0:a:0"]
cmd += editmod.AAC_ARGS if p.reencode_audio else ["-c:a", "copy"]
if threads and threads > 0:
cmd += ["-threads", str(threads)]
if p.ext in ("mp4", "m4a"):
cmd += ["-movflags", "+faststart"]
cmd += [str(dest)]
return cmd
+52 -25
View File
@@ -34,6 +34,28 @@ _IMAGE_FORMATS = {
}
def primary_streams(data: dict) -> tuple[dict | None, dict | None]:
"""(video stream, audio stream) — the first of each kind — from a NON-None ffprobe JSON dict, by
the same reading the stored facts use: a still image / image-sequence container has neither (it
is a generic 'file'), and a video stream flagged `attached_pic` (an mp3/m4a's cover art) is not a
video track. The phone-safe rule (`playable`) judges exactly these two streams."""
fmt_names = {
n.strip() for n in ((data.get("format") or {}).get("format_name") or "").split(",")
}
if fmt_names & _IMAGE_FORMATS:
return None, None
video = audio = None
for stream in data.get("streams", []):
kind = stream.get("codec_type")
if kind == "video" and video is None:
if (stream.get("disposition") or {}).get("attached_pic"):
continue # cover art, not a video track
video = stream
elif kind == "audio" and audio is None:
audio = stream
return video, audio
def _facts_from_probe(
data: dict,
) -> tuple[str | None, str | None, int | None, int | None, int | None]:
@@ -45,19 +67,11 @@ def _facts_from_probe(
fmt_names = {n.strip() for n in (fmt.get("format_name") or "").split(",")}
if fmt_names & _IMAGE_FORMATS:
return None, None, None, None, None
vcodec = acodec = None
width = height = None
for stream in data.get("streams", []):
kind = stream.get("codec_type")
if kind == "video" and vcodec is None:
if (stream.get("disposition") or {}).get("attached_pic"):
continue # cover art, not a video track
vcodec = stream.get("codec_name")
width = stream.get("width")
height = stream.get("height")
elif kind == "audio" and acodec is None:
acodec = stream.get("codec_name")
video, audio = primary_streams(data)
vcodec = video.get("codec_name") if video else None
width = video.get("width") if video else None
height = video.get("height") if video else None
acodec = audio.get("codec_name") if audio else None
duration = None
try:
@@ -71,16 +85,29 @@ def _facts_from_probe(
return vcodec, acodec, width, height, duration
def probe_media(path: Path) -> tuple[str | None, str | None, int | None, int | None, int | None]:
"""(vcodec, acodec, width, height, duration_s) from one ffprobe call; all None if ffprobe fails
or the file isn't playable media (an unprobeable file — or a still image — is treated as a generic
non-media file, so it becomes a download-only 'file' rather than a broken 'video')."""
data = editmod.run_ffprobe_json(path, show_streams=True, show_format=True)
def probe_json(path: Path) -> dict | None:
"""The full ffprobe JSON (streams + format) of `path`, or None when ffprobe could not read it —
the one call both the stored facts (`facts`) and the phone-safe rule read from."""
return editmod.run_ffprobe_json(path, show_streams=True, show_format=True)
def facts(
data: dict | None,
) -> tuple[str | None, str | None, int | None, int | None, int | None]:
"""(vcodec, acodec, width, height, duration_s) of a `probe_json` result; all None when it is None
(an unprobeable file is treated as a generic non-media file, so it becomes a download-only 'file'
rather than a broken 'video')."""
if data is None:
return None, None, None, None, None
return _facts_from_probe(data)
def probe_media(path: Path) -> tuple[str | None, str | None, int | None, int | None, int | None]:
"""(vcodec, acodec, width, height, duration_s) from one ffprobe call; all None if ffprobe fails
or the file isn't playable media (a still image is a generic 'file', see `primary_streams`)."""
return facts(probe_json(path))
def resolve_media_facts(
path: Path, info: dict
) -> tuple[str | None, str | None, int | None, int | None, int | None]:
@@ -94,18 +121,18 @@ def resolve_media_facts(
is authoritative — INCLUDING a deliberate None (cover-art-only audio, a still image). A per-field
`probe or info` fallback would let `info`'s stale vcodec resurrect a video track the file doesn't
have (yt-dlp reports a progressive `vcodec` for an audio-mode download whose video it then strips),
mislabelling an audio download as a broken in-app 'video'. Only when ffprobe could NOT read the
file at all do we fall back to `info` (the pre-probe behaviour, a safety net); the cosmetic
dimensions and the duration still gap-fill from `info` in either case."""
mislabelling an audio download as a broken in-app 'video'. When ffprobe could NOT read the file
at all, no codec is reported: the probe runs with the media demuxer whitelist, so a file it
cannot open is one the player and the editor cannot open either — and taking `info`'s codecs
for it would make it an editable 'video' whose bytes are whatever the remote site served. The
cosmetic dimensions and the duration still gap-fill from `info` in either case."""
data = editmod.run_ffprobe_json(path, show_streams=True, show_format=True)
info_v = info.get("vcodec") if info.get("vcodec") not in (None, "none") else None
info_a = info.get("acodec") if info.get("acodec") not in (None, "none") else None
info_w = int(info["width"]) if info.get("width") else None
info_h = int(info["height"]) if info.get("height") else None
info_d = int(info["duration"]) if info.get("duration") else None
if data is None:
# ffprobe could not read the file — fall back to the extractor's info entirely.
return info_v, info_a, info_w, info_h, info_d
# Unreadable to the guarded probe: a download-only 'file' (no codecs → media_kind FILE).
return None, None, info_w, info_h, info_d
pv, pa, pw, ph, pd = _facts_from_probe(data)
# Probe ran → its codecs win outright (a None means "no such track", not "unknown"); info only
# gap-fills the width/height/duration the probe couldn't read.
+39 -10
View File
@@ -1,7 +1,9 @@
"""Per-user download limits + footprint accounting.
A user's *footprint* is the total size of the ready assets their library references (cache hits
count — it's "how much disk you're responsible for", not "how much you personally downloaded").
count — it's "how much disk you're responsible for", not "how much you personally downloaded"). An
upload counts from the moment it lands, ready or not: its original is on disk while its conversion
waits, and stays there when the conversion fails (for Retry) — bytes a ready-only sum never saw.
Limits come from a per-user DownloadQuota row if the admin set one, else the sysconfig defaults.
`unlimited` (e.g. the admin's own account) bypasses the byte cap.
@@ -13,21 +15,30 @@ the footprint as it stood when the job was created and earlier jobs may have fil
That is a per-job ceiling, not an aggregate ledger: concurrent jobs of one user each read the same
remainder (in-flight bytes are not `ready`, so `footprint` cannot see them), so a wave of
max_concurrent downloads can still land over the cap — the deliberate trade, since a download's size
is unknown until it finishes. Edit jobs do not consult it at all (tracked as SEC-19).
is unknown until it finishes. CLIP jobs consult it before their fetch or encode and again with
their output's real size before it is placed. Edit jobs do not consult it at all (tracked as SEC-19).
"""
from dataclasses import dataclass
from sqlalchemy import func, select
from sqlalchemy import and_, func, or_, select
from sqlalchemy.orm import Session
from app import sysconfig
from app.downloads.status import JobKind, JobStatus
from app.models import DownloadJob, DownloadQuota, MediaAsset
# Jobs a user is considered to be "holding" (occupy a job slot / footprint). Terminal failures
# (error, canceled) don't count.
_HOLDING = ("queued", "running", "paused", "done")
_RUNNING = ("queued", "running")
_HOLDING = (JobStatus.QUEUED, JobStatus.RUNNING, JobStatus.PAUSED, JobStatus.DONE)
_RUNNING = (JobStatus.QUEUED, JobStatus.RUNNING)
# An asset whose file is on disk: a ready one, or an upload in any state (its original lands at
# ingest, see the module docstring). Every byte sum — a user's footprint, the GC's total-cap pass,
# the admin's storage view — reads it, so they all count the same disk.
ON_DISK = or_(
MediaAsset.status == "ready",
and_(MediaAsset.source_kind == "upload", MediaAsset.rel_path.is_not(None)),
)
@dataclass
@@ -56,7 +67,8 @@ def resolve(db: Session, user_id: int) -> QuotaLimits:
# "at limit" checks — the worker's claim-window exclusion vs at_concurrency_limit — since a
# 0-limit user is "capped" with zero running jobs, which the SQL exclusion can't see. Keep
# them consistent by never letting the effective limit drop below 1. (To BLOCK a user's
# downloads, suspend them or set max_jobs=0, not max_concurrent=0.)
# downloads, suspend them — the worker leaves a suspended user's queued jobs unclaimed — or
# set max_jobs=0, not max_concurrent=0.)
return QuotaLimits(row.max_bytes, max(1, row.max_concurrent), row.max_jobs, row.unlimited)
return QuotaLimits(
max_bytes=sysconfig.get_int(db, "download_default_max_bytes"),
@@ -67,12 +79,17 @@ def resolve(db: Session, user_id: int) -> QuotaLimits:
def footprint(db: Session, user_id: int) -> int:
"""Total bytes of the distinct ready assets this user's active jobs reference."""
"""Total bytes of the distinct assets this user's jobs hold on disk: the ready ones their active
jobs reference, and every upload of theirs — including one whose conversion failed, which keeps
its original (see the module docstring)."""
held_assets = (
select(DownloadJob.asset_id)
.where(
DownloadJob.user_id == user_id,
DownloadJob.status.in_(_HOLDING),
or_(
DownloadJob.status.in_(_HOLDING),
and_(DownloadJob.job_kind == JobKind.UPLOAD, DownloadJob.status == JobStatus.ERROR),
),
DownloadJob.asset_id.is_not(None),
)
.distinct()
@@ -80,7 +97,7 @@ def footprint(db: Session, user_id: int) -> int:
return int(
db.execute(
select(func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(
MediaAsset.id.in_(held_assets), MediaAsset.status == "ready"
MediaAsset.id.in_(held_assets), ON_DISK
)
).scalar()
or 0
@@ -111,7 +128,7 @@ def at_concurrency_limit(db: Session, user_id: int) -> bool:
lim = resolve(db, user_id)
if lim.unlimited:
return False
running = _count(db, user_id, ("running",))
running = _count(db, user_id, (JobStatus.RUNNING,))
return running >= lim.max_concurrent
@@ -160,6 +177,18 @@ def check_upload(db: Session, user_id: int, add_bytes: int) -> None:
raise QuotaExceeded("max_bytes", lim.max_bytes, projected)
def fits_after_conversion(db: Session, user_id: int, old_bytes: int, new_bytes: int) -> bool:
"""Whether an upload converted from `old_bytes` (what it is charged at now) to `new_bytes` keeps
its owner within the cap. A re-encode can come out larger than the original, and nothing else
re-checks it once the upload was let in."""
if new_bytes <= old_bytes: # a smaller file never breaks a cap the larger one was let in under
return True
lim = resolve(db, user_id)
if lim.unlimited:
return True
return footprint(db, user_id) - old_bytes + new_bytes <= lim.max_bytes
def usage(db: Session, user_id: int) -> dict:
"""Compact usage snapshot for the UI (footprint + counts vs limits)."""
lim = resolve(db, user_id)
+7 -2
View File
@@ -22,6 +22,7 @@ from app.config import settings
from app.db import SessionLocal
from app.downloads import edit as editmod
from app.downloads import formats
from app.downloads.status import JobKind
from app.models import DownloadJob
log = logs.get_logger(__name__)
@@ -95,12 +96,16 @@ def _run(job_id, asset_id, job_kind, source_kind, source_ref, spec, user_id) ->
(`cause=child_exit`)."""
# Imported here (not at module top) so the child pays the yt-dlp import cost only when it has
# real work, and to keep the import graph obvious: runner is a thin entry over the pipeline.
from app.downloads.pipeline import _download, _process_edit
from app.downloads.pipeline import _convert_upload, _download, _process_clip, _process_edit
with formats.hiding(source_ref):
try:
if job_kind == "edit":
if job_kind == JobKind.EDIT:
_process_edit(job_id, asset_id)
elif job_kind == JobKind.UPLOAD:
_convert_upload(job_id, asset_id)
elif job_kind == JobKind.CLIP:
_process_clip(job_id, asset_id)
else:
_download(job_id, asset_id, source_kind, source_ref, spec, user_id)
except Exception as exc: # noqa: BLE001 — whatever it is, it is reported here or raw
+123 -14
View File
@@ -16,11 +16,22 @@ from sqlalchemy.orm import Session
from app.config import settings
from app.downloads import adapters
from app.downloads import clip as clipmod
from app.downloads import credentials
from app.downloads import edit as editmod
from app.downloads import formats, mediakind, quota, storage
from app.downloads.status import HOLDING, JobStatus
from app.models import DISPLAY_TEXT_MAX, Channel, DownloadJob, MediaAsset, Video
from app.downloads.status import HOLDING, JobKind, JobStatus
from app.models import (
DISPLAY_TEXT_MAX,
Channel,
DownloadJob,
MediaAsset,
PlexItem,
PlexShow,
Video,
)
from app.plex import lookups as plex_lookups
from app.plex import paths as plex_paths
class EditError(Exception):
@@ -40,10 +51,13 @@ def source_url(source_kind: str, source_ref: str) -> str:
def reference_url(job, asset) -> str | None:
"""The clean "downloaded from" URL for a job — shared by the private Downloads page and the public
watch page so both surfaces always agree. None for edit derivatives (a clip's source is the user's
own earlier download, not a web page); else the canonical page URL yt-dlp recorded; else one derived
from source_kind+source_ref (covers queued/older rows before the worker fills it)."""
if job is None or job.job_kind == "edit":
own earlier download, not a web page) and for a Plex clip (the library is not a public page); a
YouTube clip links its video at the clip's start; else the canonical page URL yt-dlp recorded; else
one derived from source_kind+source_ref (covers queued/older rows before the worker fills it)."""
if job is None or job.job_kind == JobKind.EDIT:
return None
if job.job_kind == JobKind.CLIP:
return clipmod.reference_url(job.edit_spec or {})
if asset is not None and asset.source_webpage_url:
return asset.source_webpage_url
if job.source_kind in ("youtube", "url"):
@@ -180,7 +194,7 @@ def enqueue(
profile_snapshot=spec,
display_name=display_name
or (asset.title[:DISPLAY_TEXT_MAX] if asset.status == "ready" and asset.title else None),
status="queued",
status=JobStatus.QUEUED,
queue_pos=next_queue_pos(db),
)
db.add(job)
@@ -188,7 +202,7 @@ def enqueue(
# Cache hit: the file already exists — no worker round-trip needed.
if asset.status == "ready":
job.status = "done"
job.status = JobStatus.DONE
job.progress = 100
asset.last_access_at = datetime.now(timezone.utc)
@@ -197,7 +211,7 @@ def enqueue(
return job
def _clip_title(source_title: str | None, display_name: str | None) -> str:
def clip_title(source_title: str | None, display_name: str | None) -> str:
# Clipped to DISPLAY_TEXT_MAX, not to a literal: this feeds `DownloadJob.display_name` through
# `enqueue_edit`, the same column `display_text` guards. A machine-made title, so it is clipped
# rather than run through `display_text` — blank here means "no title", not None.
@@ -239,7 +253,7 @@ def enqueue_edit(
ref = f"{user_id}:{src_asset.id}"[:512]
asset = get_or_create_asset(db, "edit", ref, sig)
if not asset.title:
asset.title = _clip_title(src_asset.title, display_name)
asset.title = clip_title(src_asset.title, display_name)
asset.uploader = src_asset.uploader
asset.thumbnail_url = src_asset.thumbnail_url
asset.upload_date = src_asset.upload_date
@@ -251,7 +265,7 @@ def enqueue_edit(
job = DownloadJob(
user_id=user_id,
asset_id=asset.id,
job_kind="edit",
job_kind=JobKind.EDIT,
source_kind="edit",
source_ref=ref,
source_job_id=source_job.id,
@@ -259,14 +273,101 @@ def enqueue_edit(
edit_spec=spec,
profile_snapshot={},
display_name=display_name or asset.title[:DISPLAY_TEXT_MAX],
status="queued",
status=JobStatus.QUEUED,
queue_pos=next_queue_pos(db),
)
db.add(job)
incr_ref_count(db, asset.id)
if asset.status == "ready": # identical clip already produced for this user → instant
job.status = "done"
job.status = JobStatus.DONE
job.progress = 100
asset.last_access_at = datetime.now(timezone.utc)
db.commit()
db.refresh(job)
return job
def _clip_source_facts(db: Session, spec: dict) -> dict:
"""Title, uploader, thumbnail and length of what a clip is cut from, read from our own catalog.
Raises ClipError when a Plex item is unknown or its file cannot be read here — said at enqueue,
rather than as a job that fails in the queue a minute later."""
if spec["source"] == "youtube":
video = db.get(Video, spec["ref"])
if video is None: # not in the catalog: titled "Clip" until yt-dlp names it (worker)
return {"title": None, "uploader": None, "thumbnail_url": None, "length_s": None}
channel = db.get(Channel, video.channel_id)
return {
"title": video.title,
"uploader": channel.title if channel else None,
"thumbnail_url": video.thumbnail_url,
"length_s": video.duration_seconds,
}
item = plex_lookups.by_rating_key(db, PlexItem, spec["ref"])
if item is None:
raise clipmod.ClipError("not_found")
if plex_paths.local_media_path(db, item.file_path) is None:
raise clipmod.ClipError("media_missing")
title = item.title
show = db.get(PlexShow, item.show_id) if item.show_id else None
if show is not None:
episode = (
f"S{item.season_number}·E{item.episode_number}"
if item.season_number is not None and item.episode_number is not None
else None
)
title = " — ".join(p for p in (show.title, episode, item.title) if p)
return {
"title": title,
"uploader": None,
# Plex art is served through our own authenticated routes, never a public URL; the worker
# cuts the clip's poster from the clip itself.
"thumbnail_url": None,
"length_s": item.duration_s,
}
def enqueue_clip(
db: Session, user_id: int, spec: dict, display_name: str | None = None
) -> DownloadJob:
"""Queue an ad-hoc clip (`clipmod.normalize`d spec) for `user_id`: a NEW per-user asset cut
straight from the source — a YouTube range or a Plex library file (see downloads/clip).
Raises ClipError (unknown or unreachable source, a range past its end) or quota.QuotaExceeded."""
facts = _clip_source_facts(db, spec)
# One second of grace: a mark set on the last frame may round past the catalog's whole seconds.
if facts["length_s"] and spec["end_s"] > facts["length_s"] + 1:
raise clipmod.ClipError("past_end")
quota.check_enqueue(db, user_id)
ref = clipmod.asset_ref(user_id, spec)[:512]
asset = get_or_create_asset(db, "clip", ref, clipmod.sig(spec))
if not asset.title:
asset.title = clip_title(facts["title"], display_name)
asset.uploader = facts["uploader"]
asset.thumbnail_url = facts["thumbnail_url"]
asset.duration_s = round(clipmod.duration(spec))
asset.media_kind = mediakind.VIDEO
job = DownloadJob(
user_id=user_id,
asset_id=asset.id,
job_kind=JobKind.CLIP,
source_kind="clip",
source_ref=ref,
edit_spec=spec,
profile_snapshot={},
display_name=display_name or asset.title[:DISPLAY_TEXT_MAX],
status=JobStatus.QUEUED,
queue_pos=next_queue_pos(db),
)
db.add(job)
incr_ref_count(db, asset.id)
if asset.status == "ready": # the same range was already cut for this user → instant
job.status = JobStatus.DONE
job.progress = 100
asset.last_access_at = datetime.now(timezone.utc)
@@ -318,14 +419,22 @@ def release_asset(db: Session, job: DownloadJob) -> None:
re-enqueuing the same (source,format) — get_or_create_asset would reset that row to `pending` and
+1 it, and deleting it under that would FK-null the new job's asset (a lost update on ref_count).
A ready asset is still freed at ref==0 (its file is the point); an orphaned errored row is cheap
(no file) and gets reused+reset by the next enqueue, or reclaimed by a later GC pass."""
(no file) and gets reused+reset by the next enqueue, or reclaimed by a later GC pass.
An UPLOAD is the exception: it is never shared or re-enqueued (its ref is unique), and while it
waits for or failed its phone-safe conversion the original file already sits at `rel_path` — so
it is freed at ref==0 whatever its state, except mid-conversion (`downloading`: the worker child
still reads it; the GC upload pass takes it once the child has stopped)."""
if not job.asset_id or job.status not in HOLDING:
return
asset = db.get(MediaAsset, job.asset_id)
if asset is None:
return
new_count = decr_ref_count(db, asset.id)
if new_count == 0 and asset.status == "ready":
freeable = asset.status == "ready" or (
asset.source_kind == "upload" and asset.status != "downloading"
)
if new_count == 0 and freeable:
if asset.rel_path:
storage.delete_asset_files(settings.download_root, asset.rel_path)
db.delete(asset)
+17
View File
@@ -33,6 +33,23 @@ HOLDING = frozenset(
{JobStatus.QUEUED, JobStatus.RUNNING, JobStatus.PAUSED, JobStatus.DONE, JobStatus.ERROR}
)
class JobKind(StrEnum):
"""What a job produces, which decides the worker path that runs it and what the library may do
with the result (only a DOWNLOAD has a web source to fetch again)."""
DOWNLOAD = "download" # yt-dlp fetches a source into a shared, deduplicated asset
EDIT = "edit" # a per-user trim/crop of a file already in the library
UPLOAD = "upload" # a file the user sent; the worker only converts it
CLIP = (
"clip" # a per-user range of a YouTube video or a Plex item, cut straight from the source
)
# A job kind with no web source to fetch again: Re-download refuses these.
NOT_REFETCHABLE = frozenset({JobKind.EDIT, JobKind.UPLOAD, JobKind.CLIP})
# States a route sets to STOP a running job. The worker's supervisor must step in (kill the child, free
# the asset, leave the route's status intact) when it observes one of these on a job it is running.
STOPPED_BY_ROUTE = frozenset({JobStatus.PAUSED, JobStatus.CANCELED})
+14 -2
View File
@@ -168,13 +168,25 @@ def upload_rel_path(user_id: int, asset_id: int, filename: str) -> str:
episode sidecars). The name is system-decided and bound to the asset id (`_a{id}` keeps two
same-named uploads apart); the original filename is display metadata only. The extension is
preserved (shell-safe, bounded) so the served file + its Content-Disposition carry a correct
suffix; an extensionless upload stays extensionless."""
stem = sanitize(Path(filename).stem, 80)
suffix; an extensionless upload stays extensionless.
The stem holds no dot, so the last dot of the name is always the extension's: every sibling
derived by swapping it (`with_suffix` — the poster, the converted file) keeps `_a{id}`. A dotted
stem with no extension ('Holiday.2024.ä') once lost it, and two such uploads shared one file."""
stem = sanitize(Path(filename).stem, 80).replace(".", "_")
ext = _clean_upload_ext(filename)
suffix = f".{ext}" if ext else ""
return f".uploads/{user_id}/{stem}_a{asset_id}{suffix}"
def with_ext(rel: str, old_ext: str | None, ext: str) -> str:
"""`rel` with its extension — `old_ext`, the stored `container`, None for a bare name — swapped
for `ext`. Not `Path.with_suffix`: on a bare name whose stem holds a dot (an upload placed before
the stems lost their dots) that would take the stem's tail for the extension, and `_a{id}` with it."""
base = rel[: -len(old_ext) - 1] if old_ext and rel.endswith(f".{old_ext}") else rel
return f"{base}.{ext}"
def upload_ext(filename: str) -> str | None:
"""The stored `container` for an uploaded file: its shell-safe lowercased extension, or None when
the upload has no extension (a bare name). Shares `_clean_upload_ext` with `upload_rel_path` so the
+50 -17
View File
@@ -2,14 +2,23 @@
A user-uploaded file joins the SAME library as a downloaded one, so it inherits the whole function
set already built (metadata chips, share links, and — for media — in-app play/edit). Unlike a
download it bypasses yt-dlp AND the worker entirely: the file is already on the host, so ingest
probes it, places it on disk, and creates an already-`ready` MediaAsset + `done` DownloadJob in one
step — the same "the file exists, mark it done now" shortcut `service.enqueue` takes on a cache hit.
download it bypasses yt-dlp: the file is already on the host, so ingest probes it and places it on
disk. A file that already is the phone-safe shape (`playable`) — or is not media at all — becomes an
already-`ready` MediaAsset + `done` DownloadJob in one step, the same "the file exists, mark it done
now" shortcut `service.enqueue` takes on a cache hit.
Media that is NOT phone-safe (an iPhone's HEVC .mov, a VP9 .webm, an .mkv) is placed as it came and
queued: the asset stays `pending` and the job `queued`, and the download worker converts it
(`pipeline._convert_upload`) — so a share link never hands a friend a file their phone cannot play.
Only the converted file is kept (the owner's choice: one file per upload). A 10-bit HDR original
becomes 8-bit H.264 with its colour tags kept — still tagged HDR (HLG/PQ, BT.2020), not tone-mapped
to SDR (the owner's choice too). Until then the original sits at the asset's `rel_path`, which is
what a failed conversion's Retry starts from again, and what a delete removes
(`service.release_asset`).
Kinds (the invariant lives in `mediakind`): a probe with a real video stream -> video, audio-only
-> audio, neither -> a generic 'file' (pdf/zip/docx/…) that is download-only and never rendered
inline (see the force-download serving in the file/watch routes). This module is the ingest seam a
future sub-function (e.g. convert) plugs a new adapter into, mirroring how it sidesteps the worker.
inline (see the force-download serving in the file/watch routes).
An upload does NOT get an `expires_at`: it's user-provided and may be irreplaceable (there's no
source to re-download), so it doesn't age out on the retention clock the way a re-fetchable download
@@ -24,7 +33,8 @@ from sqlalchemy.orm import Session
from app.config import settings
from app.downloads import edit as editmod
from app.downloads import mediakind, probe, quota, service, storage
from app.downloads import mediakind, playable, probe, quota, service, storage
from app.downloads.status import JobKind, JobStatus
from app.models import DownloadJob, MediaAsset, display_text
@@ -44,15 +54,22 @@ def ingest_upload(
size: int,
display_name: str | None = None,
) -> DownloadJob:
"""Ingest an already-uploaded file at `src_path` into `user_id`'s library and return the `done`
job. `src_path` is MOVED into place on success (the caller stages it to a temp file first).
"""Ingest an already-uploaded file at `src_path` into `user_id`'s library and return its job —
`done`, or `queued` for the worker's phone-safe conversion (see the module docstring). `src_path`
is MOVED into place on success (the caller stages it to a temp file first).
Raises quota.QuotaExceeded (holdings/footprint cap) — checked BEFORE any disk write, so a
rejected upload leaves nothing behind and the caller just cleans up its temp file."""
quota.check_upload(db, user_id, size) # the file's size is known → charge it up front
vcodec, acodec, width, height, duration = probe.probe_media(src_path)
data = probe.probe_json(src_path)
vcodec, acodec, width, height, duration = probe.facts(data)
kind = mediakind.derive(vcodec, acodec)
# Judged by the extension the file is PLACED under (and the worker re-judges), not the raw one:
# 'clip.mp4~' is stored as .mp4, and judging 'mp4~' queued a conversion with nothing to convert.
ext = storage.upload_ext(filename) or ""
safe = playable.plan(data, ext) if kind != mediakind.FILE else None
convert = safe is not None and safe.needs_change
# An upload never dedups (each file is its own asset): a random ref makes the cache key unique,
# so `format_sig` is a constant marker rather than a format hash.
@@ -66,17 +83,18 @@ def ingest_upload(
rel = storage.upload_rel_path(user_id, asset.id, filename)
storage.place_file(src_path, settings.download_root, rel)
# From here the file is on disk under `.uploads/`. If the transaction fails before it commits, the
# asset/job rows roll back but the placed file would be ORPHANED — nothing references it, the
# per-user delete path keys off asset.rel_path (which is gone), and the GC orphan sweep skips the
# dot-prefixed `.uploads/` tree. So roll back the on-disk side too on any failure.
# asset/job rows roll back and nothing names the placed file: the per-user delete path keys off
# asset.rel_path (which is gone), and the GC's `.uploads` sweep reaches it only an hour later. So
# roll back the on-disk side too — unless the commit itself raised after it landed (see below).
asset_id = asset.id
try:
final = storage.abs_path(settings.download_root, rel)
actual_size = final.stat().st_size if final.exists() else size
asset.status = "ready"
asset.status = "pending" if convert else "ready"
asset.rel_path = rel
asset.size_bytes = actual_size
asset.container = storage.upload_ext(filename)
asset.container = ext or None
asset.vcodec = vcodec
asset.acodec = acodec
asset.width = width
@@ -95,13 +113,13 @@ def ingest_upload(
job = DownloadJob(
user_id=user_id,
asset_id=asset.id,
job_kind="upload",
job_kind=JobKind.UPLOAD,
source_kind="upload",
source_ref=ref,
profile_snapshot={},
display_name=display_text(display_name),
status="done",
progress=100,
status=JobStatus.QUEUED if convert else JobStatus.DONE,
progress=0 if convert else 100,
queue_pos=service.next_queue_pos(db),
)
db.add(job)
@@ -109,6 +127,11 @@ def ingest_upload(
db.commit()
except Exception:
db.rollback()
if _row_landed(db, asset_id) is not False:
# The commit reached the server and lost its answer, or the row cannot be read: the file
# may be the asset's now, so it is never removed blind — the sweep takes it if no row
# names it.
raise
# The rows are gone; remove the now-unreferenced file so no bytes leak. Clean the poster by its
# DETERMINISTIC path (rel with a .jpg suffix, the same path ensure_poster writes to) rather than
# ensure_poster's return value: it writes the frame directly to that path and returns None on an
@@ -120,3 +143,13 @@ def ingest_upload(
raise
db.refresh(job)
return job
def _row_landed(db: Session, asset_id: int) -> bool | None:
"""After the ingest transaction raised: is the asset row there anyway? A commit can reach the
server and lose its answer (the connection drops after COMMIT). None when it cannot be read."""
try:
return db.get(MediaAsset, asset_id) is not None
except Exception: # noqa: BLE001 — "cannot tell" is the answer, the caller acts on it
# silent-ok: the caller re-raises the ingest's own error.
return None
+11 -5
View File
@@ -339,6 +339,8 @@ STATIC_DIR = Path(__file__).parent / "static_spa"
# stay cached but forces revalidation (cheap 304 via the FileResponse ETag) on every load. The
# hashed bundles under /assets can cache forever (a new build changes their filename).
INDEX_HTML = STATIC_DIR / "index.html"
# A public share link's own page: the player alone, without the app's bundle (frontend/watch.html).
WATCH_HTML = STATIC_DIR / "watch.html"
INDEX_HEADERS = {"Cache-Control": "no-cache"}
app.mount(
"/assets",
@@ -354,16 +356,16 @@ async def index() -> FileResponse:
@app.get("/watch/{token}")
async def watch_page(token: str):
"""Serve the SPA for a public share link, but with per-video Open Graph tags injected so the
"""Serve the public player page for a share link, with per-video Open Graph tags injected so the
link unfurls richly (title/channel/thumbnail) in Messenger/social crawlers. A real browser
ignores the extra tags and hydrates WatchPage as usual; invalid/expired/password links fall
back to the generic card."""
ignores the extra tags and boots WatchPage; invalid/expired/password links fall back to the
generic card."""
from app.downloads import og
with SessionLocal() as db:
rendered = og.render_watch_html(token, INDEX_HTML, db)
rendered = og.render_watch_html(token, WATCH_HTML, db)
if rendered is None:
return FileResponse(INDEX_HTML, headers=INDEX_HEADERS)
return FileResponse(WATCH_HTML, headers=INDEX_HEADERS)
return HTMLResponse(rendered, headers=INDEX_HEADERS)
@@ -381,4 +383,8 @@ async def spa_fallback(full_path: str) -> FileResponse:
# Real SPA-root assets (welcome images, favicon, robots.txt) rarely change and have
# stable names, so a moderate cache is safe and satisfies the cache-lifetime audit.
return FileResponse(candidate, headers={"Cache-Control": "public, max-age=604800"})
# A /watch/ path the route above does not match (a trailing segment, no token) is still the
# player's: WatchPage reads the token off the path and says when a link is gone.
if full_path.startswith("watch/"):
return FileResponse(WATCH_HTML, headers=INDEX_HEADERS)
return FileResponse(INDEX_HTML, headers=INDEX_HEADERS)
+29 -5
View File
@@ -123,7 +123,8 @@ class User(Base, TimestampMixin, CanonicalEmailMixin):
is_active: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true")
# Admin-imposed access block, distinct from is_active (approval lifecycle). A suspended
# account can't sign in by any method and its live sessions are rejected; the user is told
# why and pointed at the operator's contact.
# why and pointed at the operator's contact. The worker leaves its queued downloads unclaimed
# until the block is lifted (`worker._claim_job`).
is_suspended: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false")
# Server-side session revocation (SA4): a monotonic counter bumped on password reset, password
# change, or "log out everywhere". A signed session cookie records the epoch it was minted under;
@@ -462,6 +463,27 @@ class VideoState(Base, UpdatedAtMixin):
progress_updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
class LibraryState(Base):
"""Where a user stopped in a library item (their own download, or one shared with them) — the
library player's resume point. Keyed by the JOB, not the file: a job is the user's library entry,
a re-download keeps it, and deleting the job deletes the row; the shared file under it may outlive
both. A revoked share leaves its recipient's row behind, unread — only a job the user can still
reach is ever read or written — and it is back if the item is shared again. Rows exist only for a
position worth keeping (see `routes._common.kept_position`); no row means "from the start"."""
__tablename__ = "library_states"
__table_args__ = (UniqueConstraint("user_id", "job_id", name="uq_library_user_job"),)
id: Mapped[int] = mapped_column(primary_key=True)
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"))
job_id: Mapped[int] = mapped_column(
ForeignKey("download_jobs.id", ondelete="CASCADE"), index=True
)
# int4: a checkpoint body must stay under POSITION_MAX_S, or the flush is a DataError (a 500).
position_seconds: Mapped[int] = mapped_column(Integer, nullable=False)
progress_updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
class SearchFind(Base, TimestampMixin):
"""Per-user record of a video the user surfaced via their own live YouTube search.
@@ -912,7 +934,9 @@ class DownloadJob(Base, TimestampMixin, UpdatedAtMixin):
)
source_kind: Mapped[str] = mapped_column(String(16)) # "youtube" | "url"
source_ref: Mapped[str] = mapped_column(String(512))
# "download" (default) or "edit" (a per-user trim/crop derivative of another job).
# "download" (default), "edit" (a per-user trim/crop derivative of another job), "upload" (a
# user's own file, converted by the worker when it is not phone-safe — see `downloads.upload`) or
# "clip" (a range cut straight from a YouTube video or a Plex item — see `downloads.clip`).
job_kind: Mapped[str] = mapped_column(String(16), default="download", server_default="download")
# For edit jobs: the parent download this clip is cut from (SET NULL if the parent is deleted).
source_job_id: Mapped[int | None] = mapped_column(
@@ -921,7 +945,9 @@ class DownloadJob(Base, TimestampMixin, UpdatedAtMixin):
source_asset_id: Mapped[int | None] = mapped_column(
ForeignKey("media_assets.id", ondelete="SET NULL")
)
# For edit jobs: the trim/crop recipe {trim?:{start_s,end_s}, crop?:{x,y,w,h}}.
# For edit jobs: the trim/crop recipe {trim?:{start_s,end_s}, crop?:{x,y,w,h}} in pixels. For clip
# jobs: the clip spec {source, ref, start_s, end_s, audio_track?, crop?:{x,y,w,h,aspect}}, its crop
# in fractions of the box it was drawn in (`downloads.clip.normalize`).
edit_spec: Mapped[dict | None] = mapped_column(JSON)
profile_id: Mapped[int | None] = mapped_column(
ForeignKey("download_profiles.id", ondelete="SET NULL")
@@ -1252,8 +1278,6 @@ class PlexItem(Base, TimestampMixin, UpdatedAtMixin):
# bit-depth/HDR. NULL = never (the backfill's work-list). Only HW-decodable transcode items are
# probed (the rest are SW-only by codec and need no bit-depth); see `_probe_media_batch`.
media_probed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
# Intro/credit markers: [{"type": "intro"|"credits", "start_s": int, "end_s": int}, ...]
markers: Mapped[list | None] = mapped_column(JSON)
added_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), index=True)
# Filterable / orderable metadata mirrored from the Plex section listing (no extra API calls;
# `rating` = Plex audienceRating, which is ~the IMDb score). The arrays back genre/person/studio
+18 -2
View File
@@ -3,9 +3,24 @@ playlists, and detail views render. Extracted from routes/plex/browse.py so the
lives beside the other Plex read-model modules (detail.py, query.py) rather than inside the
router. Pure functions — no DB, no request state."""
from typing import NamedTuple
from app.models import PlexItem, PlexShow, PlexState
class ShowState(NamedTuple):
"""A show's watch state rolled up from its episodes, as its card carries it. `resumable`: an unwatched
episode has a resume point, the one the card's Continue resumes and its Restart replays from 0. A
show part watched with its next episode untouched has none: Continue and Restart would both play
that episode from its start, so the card offers Play alone."""
status: str
resumable: bool
SHOW_NEW = ShowState("new", False)
# A card's wide backdrop, for a rail's landscape tile or a crossfading hero — but ONLY when the item
# actually has art. `thumb` is always emitted (its endpoint placeholders a missing poster), but an
# `art` URL for an item with no `art_key` would resolve to a broken/blank image on every card that
@@ -54,7 +69,7 @@ def movie_card(it: PlexItem, st: PlexState | None, *, lang: str) -> dict:
}
def show_card(sh: PlexShow, status: str = "new", *, lang: str) -> dict:
def show_card(sh: PlexShow, state: ShowState = SHOW_NEW, *, lang: str) -> dict:
return {
"id": sh.rating_key,
"type": "show",
@@ -66,7 +81,8 @@ def show_card(sh: PlexShow, status: str = "new", *, lang: str) -> dict:
"art": _art_url(sh.rating_key, sh.art_key),
"season_count": sh.child_count,
# Aggregate watch-state across the show's episodes (new | in_progress | watched) → grid badge.
"status": status,
"status": state.status,
"resumable": state.resumable,
}
+7 -3
View File
@@ -167,12 +167,16 @@ class PlexClient:
keys = ",".join(str(k) for k in rating_keys)
return f"server://{self.machine_id}/com.plexapp.plugins.library/library/metadata/{keys}"
def create_collection(self, section_key: str, title: str, item_rating_key: str) -> dict:
"""Create a (dumb) collection seeded with one item; returns its Metadata (incl. ratingKey)."""
def create_collection(
self, section_key: str, title: str, item_rating_key: str, item_type: int = 1
) -> dict:
"""Create a (dumb) collection seeded with one item; returns its Metadata (incl. ratingKey).
`item_type` is the Plex metadata type of its members — 1 movie, 2 show. Measured: a collection
made as type 1 in a TV section came out EMPTY with a show seed, and refused the show after."""
uri = quote(self._metadata_uri(item_rating_key), safe="")
mc = self._write(
"POST",
f"/library/collections?type=1&smart=0&sectionId={section_key}"
f"/library/collections?type={item_type}&smart=0&sectionId={section_key}"
f"&title={quote(title, safe='')}&uri={uri}",
)
items = mc.get("Metadata", []) or []
+2 -1
View File
@@ -7,6 +7,7 @@ the two never diverge — the same one-parser rule that keeps `people.py`'s cast
(genres/directors/studio/content_rating already land from the cheap listing at `_apply_facet_fields`)."""
from app.plex import people
from app.plex.plaintext import plain_text
def rich_detail(meta: dict) -> dict:
@@ -42,7 +43,7 @@ def rich_detail(meta: dict) -> dict:
"genres": [g.get("tag") for g in (meta.get("Genre") or []) if g.get("tag")][:8],
"directors": [d.get("tag") for d in (meta.get("Director") or []) if d.get("tag")][:4],
"studio": meta.get("studio"),
"tagline": meta.get("tagline"),
"tagline": plain_text(meta.get("tagline")),
"cast": people.cast_from_meta(meta),
}
+3 -2
View File
@@ -17,6 +17,7 @@ from app.models import PlexItem, PlexLibrary, PlexShow
from app.plex import sync as plex_sync
from app.plex.client import PlexClient
from app.plex.genres import hu_genres
from app.plex.plaintext import plain_text
log = logs.get_logger(__name__)
@@ -109,8 +110,8 @@ def _overlay_pass(
for g, meta in hu_by_guid.items():
targets = by_guid[g]
title_hu = (meta.get("title") or "").strip() or None
summary_hu = meta.get("summary")
tagline_hu = (meta.get("tagline") or "").strip() or None
summary_hu = plain_text(meta.get("summary"))
tagline_hu = plain_text(meta.get("tagline"))
genres_hu = hu_genres(plex_sync._tags(meta, "Genre"))
for row in targets:
row.title_hu = title_hu
+51
View File
@@ -0,0 +1,51 @@
"""Plex text fields as plain text.
Plex hands a summary over as the agent wrote it, and some agents (a Hungarian TV listing, for one)
write HTML: `<p>…</p>` paragraphs and `&aacute;`-style entities. The app renders summaries as text,
so the tags showed literally on the info page and the hover card, and a card's 300-character cut
could land inside one. Every summary and tagline is stored through `plain_text`, and the migration
that cleaned the existing rows calls it too, so the two cannot disagree.
"""
import html
import re
from html.entities import html5
_BREAK = re.compile(r"<br\s*/?>", re.IGNORECASE)
# A paragraph or block boundary: a blank line between the two. Only a tag name ends the match, so a
# bare "<" in prose ("a < b", "<3") is left alone.
_BLOCK = re.compile(r"</?(?:p|div|li|ul|ol|h[1-6]|blockquote)\b[^>]*>", re.IGNORECASE)
_TAG = re.compile(r"</?[a-zA-Z][^>]*>")
_BLANKS = re.compile(r"[ \t]*\n[ \t]*\n\s*")
# A well-formed character reference only. `html.unescape` alone also decodes the legacy forms that
# have no semicolon, so prose like "AT&notice" or "R&copy" came out as "AT¬ice" and "R©". A named
# one counts only when its whole name is an entity: for "&notice;" `html.unescape` still falls back
# to the legacy "&not" inside it.
_REF = re.compile(r"&(?:#[0-9]+|#[xX][0-9a-fA-F]+|[A-Za-z][A-Za-z0-9]*);")
def _decode(m: re.Match[str]) -> str:
ref = m.group()
if ref[1] != "#":
return html.unescape(ref) if ref[1:] in html5 else ref
if ref[2] in "xX":
return html.unescape(ref)
# `int()` refuses a decimal run past 4300 digits, and any number past seven significant digits is
# beyond U+10FFFF, which the parser turns into U+FFFD anyway.
digits = ref[2:-1].lstrip("0") or "0"
return "�" if len(digits) > 7 else html.unescape(f"&#{digits};")
def plain_text(value: str | None) -> str | None:
"""`value` with its markup gone: line breaks kept as newlines, paragraphs as a blank line,
every other tag dropped and entities decoded. None or an all-blank result stays None."""
if value is None:
return None
if "<" in value or "&" in value:
value = _BREAK.sub("\n", value)
value = _BLOCK.sub("\n\n", value)
value = _TAG.sub("", value)
value = _REF.sub(_decode, value)
value = _BLANKS.sub("\n\n", value)
return value.strip() or None
+9 -3
View File
@@ -9,6 +9,7 @@ from sqlalchemy import and_, case, func, or_, select
from sqlalchemy.orm import Session, aliased
from app.models import PlexItem, PlexState
from app.plex.cards import ShowState
# Sort sentinel for a rail row whose timestamp is missing, so `key=lambda r: r[0] or NO_TS` reads the
# same at every rail that merges two type-homogeneous lists in Python.
@@ -32,6 +33,11 @@ def show_status(total: int, watched: int, inprog: int) -> str:
return "new"
def show_state(total: int, watched: int, inprog: int) -> ShowState:
"""`show_status` plus whether an episode holds a resume point (the card's `ShowState`)."""
return ShowState(show_status(total, watched, inprog), (inprog or 0) > 0)
def rollup(cards: list[dict]) -> dict:
"""From ORDERED episode cards → the aggregate watch status, the "on deck" episode to Resume (the
last in-progress one, else the first unwatched), the first episode (for Play from the start), and
@@ -92,16 +98,16 @@ def agg_subquery(db: Session, user_id: int, include_resumed: bool = False):
)
def state_map(db: Session, user_id: int, show_ids: list[int]) -> dict[int, str]:
def state_map(db: Session, user_id: int, show_ids: list[int]) -> dict[int, ShowState]:
if not show_ids:
return {}
rows = db.execute(agg_subquery(db, user_id).where(PlexItem.show_id.in_(show_ids))).all()
return {r.show_id: show_status(r.total, r.watched, r.inprog) for r in rows}
return {r.show_id: show_state(r.total, r.watched, r.inprog) for r in rows}
def item_state_map(db: Session, user_id: int, item_ids: list[int]) -> dict[int, PlexState]:
"""A user's PlexState rows for a set of leaf items (movies or episodes), keyed by item_id — the
leaf analogue of `state_map` (which rolls a show's episodes up to a status string). One batched
leaf analogue of `state_map` (which rolls a show's episodes up to a `ShowState`). One batched
fetch, so the caller serializes cards without a per-card query. Empty in → empty dict (no `IN ()`)."""
if not item_ids:
return {}
+30 -10
View File
@@ -35,7 +35,7 @@ from app.config import settings
from app.downloads.edit import encode_thread_cap
from app.models import PlexItem
from app.plex import paths
from app.plex.sync import HW_DECODABLE_CODECS
from app.plex.sync import HDR_TRANSFERS, HW_DECODABLE_CODECS
log = logs.get_logger(__name__)
_HLS_ROOT = (
@@ -203,7 +203,8 @@ _HW_DECODABLE = HW_DECODABLE_CODECS
# plain scale_vaapi=format=nv12 path (Gen 9.5 VPP refuses it) — it needs the OpenCL tonemap graph
# (_venc_opts tonemap branch) when the OpenCL runtime is present, else software libx264. A 10-bit source
# WITHOUT one of these transfers is SDR, not HDR, and must not be tone-mapped. See _probe_video_kind.
_HDR_TRANSFERS = {"smpte2084", "arib-std-b67"}
# THE single source is plex.sync (a clip's tonemap reads it too).
_HDR_TRANSFERS = HDR_TRANSFERS
# The three source classes that drive the transcode-pipeline choice (see _probe_video_kind).
@@ -744,8 +745,9 @@ def start_session(
# `replace_only` makes the supersede identity-aware for the runtime fallback: the whole
# check-and-replace happens under ONE _lock, so a newer same-key session (a concurrent
# seek-restart) installed between the fallback's decision and here is never clobbered. When the
# registered session is no longer `replace_only`, we abort and return None (the caller yields to
# whoever is registered now). The INITIAL spawn never gets None — see its call site.
# registered session is no longer `replace_only`, we abort and return None, and the caller
# answers with its own dead session — never the newer one, whose id a stale requester would
# stop. The INITIAL spawn never gets None — see its call site.
# Software-fallback encode tuning — read only on the SW transcode path (the VAAPI path and
# remux ignore preset/crf/max_height, and remux must not touch the DB for them).
preset, crf, max_height = "veryfast", 20, 0
@@ -840,12 +842,13 @@ def start_session(
)
respawned = _spawn(False, False, replace_only=s)
if respawned is None:
# Superseded/culled mid-fallback — yield whoever is registered now (a newer seek-restart), or
# our own now-dead session if the key was cap-culled. Never None here: the POST /session route
# turns a None return into a "Media file not found on disk" 404 — dishonest for a file that
# exists. A (dead) session instead degrades to a retryable 404 ("No active playback session" /
# "Not available yet") the client re-requests.
return current_session(user_id, key) or s
# Superseded/culled mid-fallback — answer with our own now-dead session. Never the newer one
# registered meanwhile: this answer goes to the request that LOST, whose player drops it as
# stale and stops the session it names (`stop_session`), which stopped the playback that won.
# Never None either: the POST /session route turns a None return into a "Media file not found
# on disk" 404 — dishonest for a file that exists. A dead session instead degrades to a
# retryable 404 ("No active playback session" / "Not available yet") the client re-requests.
return s
s = respawned
ready = wait_for(_seg(0), timeout=25.0, proc=s.proc)
if ready:
@@ -1117,6 +1120,23 @@ def reap_idle(finished_idle_s: float = _FINISHED_IDLE_S) -> int:
return dropped
def stop_session(user_id: int, key: str, session_id: str) -> bool:
"""Stop this viewer's session for `key` now, because the player that asked for it is gone (left
the item, closed, unloaded the page). Left to `reap_idle`, a transcode ran on to the end of the
film first — measured: ~4 cores for a minute after leaving a resumed 480p film.
Only while `session_id` is still the registered one: the player names the session it played, and
a seek-restart or a second tab may have replaced it meanwhile — that newer session is somebody's
current playback, not this request's to end."""
with _lock:
s = _sessions.get((user_id, key))
if s is None or s.id != session_id:
return False
_kill(s)
_sessions.pop((user_id, key), None)
return True
def touch_session(user_id: int, key: str) -> None:
"""Say this viewer is still there for `key`, without fetching anything. `last_access` used to move
only on an HLS file fetch, which a paused player does not do — so the only way to keep a paused
+10 -7
View File
@@ -26,6 +26,7 @@ from app.plex import detail, overlay, people
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
from app.plex.genres import hu_genres
from app.plex.lookups import by_rating_key
from app.plex.plaintext import plain_text
log = logs.get_logger(__name__)
# Codecs/containers a browser can play in a native <video> without any transcoding.
@@ -63,9 +64,11 @@ def classify_playable(vcodec: str | None, acodec: str | None, container: str | N
# outside it (AV1, MPEG-4 ASP/XviD…) is software-only even on the iGPU (it can't decode it to feed its
# own encoder).
HW_DECODABLE_CODECS = frozenset({"h264", "hevc", "h265", "vp9", "vc1", "mpeg2video"})
# Color-transfer values marking an HDR source (PQ / HLG). A 10-bit/HDR source breaks the VAAPI
# scale=nv12 path on Gen 9.5 (see plex.stream) → software transcode, so it counts as NOT HW-capable.
_HDR_TRANSFERS = frozenset({"smpte2084", "arib-std-b67"})
# Color-transfer values marking an HDR source (PQ / HLG). THE single source of this set: plex.stream
# (the tonemap choice) and downloads.clip (a clip's tonemap) import it here, so a transfer added for
# one is tonemapped by all. A 10-bit/HDR source breaks the VAAPI scale=nv12 path on Gen 9.5 (see
# plex.stream) → software transcode, so it counts as NOT HW-capable.
HDR_TRANSFERS = frozenset({"smpte2084", "arib-std-b67"})
def transcode_hw_capable(
@@ -106,7 +109,7 @@ def _video_stream_props(meta: dict) -> tuple[int | None, bool]:
# whole library would drown the sync's own lines.
except (TypeError, ValueError):
depth = None
hdr = str(s.get("colorTrc") or "").lower() in _HDR_TRANSFERS
hdr = str(s.get("colorTrc") or "").lower() in HDR_TRANSFERS
return depth, hdr
return None, False
@@ -468,7 +471,7 @@ def _apply_item(row: PlexItem, lib_id: int, kind: str, meta: dict) -> None:
# carries none (a server ignoring includeGuids mustn't wipe the overlay's only match key).
row.guid = _canonical_guid(meta) or row.guid
row.title = (meta.get("title") or "").strip() or "Untitled"
row.summary = meta.get("summary")
row.summary = plain_text(meta.get("summary"))
row.year = meta.get("year")
row.duration_s = _dur_s(meta)
row.thumb_key = meta.get("thumb")
@@ -576,7 +579,7 @@ def _sync_shows(db: Session, plex: PlexClient, lib: PlexLibrary, stats: dict) ->
_canonical_guid(meta) or sh.guid
) # EN<->HU pairing key; keep prior if listing has none
sh.title = (meta.get("title") or "").strip() or "Untitled"
sh.summary = meta.get("summary")
sh.summary = plain_text(meta.get("summary"))
sh.year = meta.get("year")
sh.thumb_key = meta.get("thumb")
sh.art_key = meta.get("art")
@@ -659,7 +662,7 @@ def _upsert_collection(
db.add(col)
col.library_id = lib_id
col.title = (meta.get("title") or "").strip() or "Untitled"
col.summary = meta.get("summary")
col.summary = plain_text(meta.get("summary"))
col.thumb_key = meta.get("thumb")
col.child_count = meta.get("childCount")
col.smart = str(meta.get("smart")) == "1"
+37 -1
View File
@@ -9,16 +9,52 @@ one existence-leak policy.
from typing import IO, Any, TypeVar
from fastapi import HTTPException, Request
from pydantic import BaseModel, ConfigDict, Field
from sqlalchemy.orm import Session
from app.db import Base
from app.models import User
from app.models import POSITION_MAX_S, User
from app.issues import gitea
from app.refusal import refused
from app.youtube.client import YouTubeError, refusal_reason
M = TypeVar("M", bound=Base)
# Resume-position thresholds of a player that only keeps a resume point (the YouTube player, the
# library's): positions below this are "didn't really start" and within this of the end are
# "basically finished" — neither is worth storing, so the position is cleared instead (no progress
# bar). The client's local resume (`lib/resume.ts`) keeps the same two numbers, held equal by
# `resume.contract.test.ts` beside it. NB: the Plex player intentionally uses a LARGER finish margin
# (30s) because it marks long-form items watched, not just clears a resume bar — see
# routes/plex/playback.py.
PROGRESS_MIN_SECONDS = 5
FINISH_MARGIN_SECONDS = 10
class ProgressIn(BaseModel):
"""A player's resume checkpoint (sent every few seconds while it plays, and as a beacon when the
page goes away)."""
# Finite only: JSON parsing here accepts `Infinity`/`NaN`, and the handler's `int()` turned
# one into a 500.
model_config = ConfigDict(allow_inf_nan=False)
# Accept a float and truncate (`kept_position`) — the player may post a raw currentTime (a float),
# which the old `int(payload.get(...))` silently truncated. A plain `int` field would 422 that on
# the progress hot path, losing the resume position; `float` keeps the old tolerance.
# Bounded above by the int4 column (see POSITION_MAX_S): past it the flush was a 500.
position_seconds: float = Field(0, le=POSITION_MAX_S)
duration_seconds: float = Field(0, le=POSITION_MAX_S)
def kept_position(body: ProgressIn) -> int:
"""The whole-second resume point worth storing for `body`, or 0 for "clear it" (too early to
count as started, or close enough to the end to count as finished)."""
position = max(0, int(body.position_seconds))
duration = int(body.duration_seconds)
near_end = duration > 0 and position > duration - FINISH_MARGIN_SECONDS
return position if position >= PROGRESS_MIN_SECONDS and not near_end else 0
def not_found(what: str = "generic") -> HTTPException:
"""The 404 for a row that is gone — structured, so the bilingual UI says it in the reader's
+2 -1
View File
@@ -484,7 +484,8 @@ def set_user_suspended(
db: Session = Depends(get_db),
) -> dict:
"""Suspend or un-suspend an account. A suspended user can't sign in by any method and any
live session is dropped on its next request. Guards: can't suspend yourself, and can't suspend the last active admin (that would lock admin out). On
live session is dropped on its next request; their queued downloads wait (the worker skips them)
until they are un-suspended. Guards: can't suspend yourself, and can't suspend the last active admin (that would lock admin out). On
un-suspend the user is emailed they can return (suspend is announced reactively, on a blocked
sign-in)."""
suspended = body.suspended
+140 -24
View File
@@ -8,7 +8,8 @@ state, serves finished files (range-aware, with the user's custom display name),
import os
import re
import tempfile
from collections.abc import AsyncIterator
from typing import Literal
from collections.abc import AsyncIterator, Sequence
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path
@@ -16,8 +17,9 @@ from urllib.parse import parse_qs, unquote, urlparse
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import FileResponse
from pydantic import BaseModel
from sqlalchemy import func, select
from pydantic import BaseModel, Field
from sqlalchemy import delete, func, select
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.orm import Session
from app.mailtext import lang_for_user
@@ -26,23 +28,33 @@ from app import audit, sysconfig
from app.auth import admin_user, current_user
from app.config import settings
from app.db import get_db, release_read_connection
from app.downloads import clip as clipmod
from app.downloads import credentials as dlcreds
from app.downloads import edit as editmod
from app.downloads import links as linksmod
from app.downloads import quota, service, storage
from app.downloads import upload as uploadmod
from app.downloads import urlguard
from app.downloads.status import NOT_REFETCHABLE, JobStatus
from app.models import (
DownloadJob,
DownloadLink,
DownloadProfile,
DownloadQuota,
DownloadShare,
LibraryState,
MediaAsset,
User,
display_text,
)
from app.routes._common import capped_stream_to, get_or_404, not_found, owned_or_404
from app.routes._common import (
ProgressIn,
capped_stream_to,
get_or_404,
kept_position,
not_found,
owned_or_404,
)
from app.security import hash_password
from app.userscope import is_messageable_user, messageable_clauses
from app.utils import iso
@@ -131,8 +143,8 @@ def _serialize(job: DownloadJob, asset: MediaAsset | None) -> dict:
"job_kind": job.job_kind,
"source_job_id": job.source_job_id,
"edit_spec": job.edit_spec,
"can_download": ready and job.status == "done",
"expired": job.asset_id is None and job.status == "done",
"can_download": ready and job.status == JobStatus.DONE,
"expired": job.asset_id is None and job.status == JobStatus.DONE,
"asset": None
if asset is None
else {
@@ -164,6 +176,19 @@ def _assets_for(db: Session, jobs: list[DownloadJob]) -> dict[int, MediaAsset]:
return {a.id: a for a in rows}
def _positions_for(db: Session, user: User, jobs: Sequence[DownloadJob]) -> dict[int, int]:
"""`user`'s resume point in each of `jobs` that has one (the library player's `library_states`)."""
ids = [j.id for j in jobs]
if not ids:
return {}
rows = db.execute(
select(LibraryState.job_id, LibraryState.position_seconds).where(
LibraryState.user_id == user.id, LibraryState.job_id.in_(ids)
)
).all()
return {job_id: pos for job_id, pos in rows}
def _serialize_job(db: Session, job: DownloadJob) -> dict:
"""Resolve a job's cache asset (if any) and serialize the pair — the resolve-then-serialize step
every single-job handler shares."""
@@ -502,7 +527,8 @@ def upload_download(
(video/audio) gets the full function set, a generic file is download-only. The body IS the file
(see the header note above); `spool_upload` has already authenticated the caller and streamed it
to a temp path under the download volume, so what is left here is the non-yt-dlp ingest which
probes + places it and returns an already-`done` job.
probes + places it and returns its job — `done`, or `queued` for the worker's phone-safe
conversion (`downloads.upload`).
A sync (threadpool) handler on purpose: the ingest does blocking work — an ffprobe of up to 30s
plus sync DB writes — so it must NOT run on the event loop. The streaming half is async because
@@ -539,7 +565,7 @@ def enqueue_edit(
clip is a NEW per-user derivative in the editor's own library (counts against their quota); the
source file is only read, never modified, so editing a shared video is safe."""
source_job = _accessible_job(db, user, body.source_job_id)
if source_job.status != "done" or source_job.asset_id is None:
if source_job.status != JobStatus.DONE or source_job.asset_id is None:
raise refused(409, "edit_unfinished")
spec = body.edit_spec
display_name = display_text(body.display_name)
@@ -552,6 +578,49 @@ def enqueue_edit(
return _serialize_job(db, job)
class ClipCropIn(BaseModel):
x: float
y: float
w: float
h: float
aspect: float
class EnqueueClipIn(BaseModel):
source: Literal["youtube", "plex"]
# The YouTube video id or the Plex rating key.
ref: str = Field(max_length=32)
start_s: float
end_s: float
# Plex only: which of the file's audio tracks (0-based, in file order) — the one being heard.
audio_track: int | None = None
# A crop frame (downloads/clip): fractions of the box it was drawn in, and that box's aspect.
crop: ClipCropIn | None = None
display_name: str | None = None
@router.post("/clip")
def enqueue_clip(
body: EnqueueClipIn, user: User = Depends(current_user), db: Session = Depends(get_db)
) -> dict:
"""Queue an ad-hoc clip: the marked range of a YouTube video or a Plex item, cut straight from
the source into a new asset in this user's library (downloads/clip)."""
try:
crop = body.crop.model_dump() if body.crop else None
spec = clipmod.normalize(
body.source, body.ref, body.start_s, body.end_s, body.audio_track, crop
)
display_name = display_text(body.display_name)
job = service.enqueue_clip(db, user.id, spec, display_name=display_name)
except clipmod.ClipError as e:
status = 404 if e.reason == "not_found" else 422
detail = {"code": "clip", "reason": e.reason, **e.params}
raise HTTPException(status_code=status, detail=detail) from e
except quota.QuotaExceeded as e:
raise HTTPException(status_code=422, detail=_quota_detail(e)) from e
return _serialize_job(db, job)
def _quota_detail(e: quota.QuotaExceeded) -> dict:
"""Structured error the trilingual client localizes (reason key + numbers), instead of a
pre-baked English string with a dot-decimal GB value. See lib/api.ts localizeDetail()."""
@@ -571,7 +640,14 @@ def list_downloads(user: User = Depends(current_user), db: Session = Depends(get
.all()
)
assets = _assets_for(db, jobs)
return [_serialize(j, assets.get(j.asset_id)) for j in jobs]
positions = _positions_for(db, user, jobs)
return [
{
**_serialize(j, assets.get(j.asset_id) if j.asset_id is not None else None),
"position_seconds": positions.get(j.id, 0),
}
for j in jobs
]
@router.get("/usage")
@@ -593,7 +669,12 @@ def my_usage(user: User = Depends(current_user), db: Session = Depends(get_db))
# Priority when a source has several jobs (e.g. a done one plus a fresh queued one).
_INDEX_RANK = {"done": 4, "running": 3, "paused": 2, "queued": 1}
_INDEX_RANK: dict[str, int] = {
JobStatus.DONE: 4,
JobStatus.RUNNING: 3,
JobStatus.PAUSED: 2,
JobStatus.QUEUED: 1,
}
@router.get("/index")
@@ -603,7 +684,7 @@ def my_download_index(user: User = Depends(current_user), db: Session = Depends(
rows = db.execute(
select(DownloadJob.source_ref, DownloadJob.status).where(
DownloadJob.user_id == user.id,
DownloadJob.status.in_(("queued", "running", "paused", "done")),
DownloadJob.status.in_(tuple(_INDEX_RANK)),
)
).all()
out: dict[str, str] = {}
@@ -627,10 +708,12 @@ def shared_with_me(user: User = Depends(current_user), db: Session = Depends(get
.all()
)
assets = _assets_for(db, jobs)
positions = _positions_for(db, user, jobs)
out = []
for j in jobs:
data = _serialize(j, assets.get(j.asset_id))
data["shared"] = True
data["position_seconds"] = positions.get(j.id, 0)
out.append(data)
return out
@@ -730,11 +813,11 @@ def redownload_download(
never deletes a file another job/user still shares: the old asset is only removed if THIS job
was its last holder."""
job = _own_job_for_update(db, user, job_id)
if job.job_kind in ("edit", "upload"):
# Neither has a re-fetchable source: an editor clip is derived locally, an upload IS the
# source. Re-running yt-dlp against source_kind='upload'/'edit' would just fail.
if job.job_kind in NOT_REFETCHABLE:
# None has a re-fetchable source: an editor clip is derived locally, an upload IS the
# source, an ad-hoc clip is a range of a source the clip itself does not re-fetch.
raise refused(409, "not_redownloadable")
if job.status in ("queued", "running", "paused"):
if job.status in (JobStatus.QUEUED, JobStatus.RUNNING, JobStatus.PAUSED):
raise refused(409, "still_downloading")
service.redownload(db, job)
return _serialize_job(db, job)
@@ -818,6 +901,39 @@ def download_file(
# --- editor filmstrip (scrub track) --------------------------------------------------------
@router.post("/{job_id}/progress")
def set_library_progress(
job_id: int,
body: ProgressIn,
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
"""Checkpoint the library player's resume point in a download the caller can play (their own, or
one shared with them). A position too early or too close to the end clears it instead."""
_accessible_job(db, user, job_id)
position = kept_position(body)
mine = (LibraryState.user_id == user.id) & (LibraryState.job_id == job_id)
if not position:
# A statement, not a loaded row's delete: the periodic checkpoint and the pagehide beacon of
# one stop may both clear it, and the second then deletes nothing rather than failing.
db.execute(delete(LibraryState).where(mine))
db.commit()
return {"ok": True, "job_id": job_id, "position_seconds": 0}
now = datetime.now(timezone.utc)
# One row per (user, job) even when two checkpoints race (the same pair): an upsert, not a
# read-then-insert that the second would fail on the unique key.
db.execute(
pg_insert(LibraryState)
.values(user_id=user.id, job_id=job_id, position_seconds=position, progress_updated_at=now)
.on_conflict_do_update(
constraint="uq_library_user_job",
set_={"position_seconds": position, "progress_updated_at": now},
)
)
db.commit()
return {"ok": True, "job_id": job_id, "position_seconds": position}
@router.get("/{job_id}/storyboard")
def storyboard_meta(
job_id: int, user: User = Depends(current_user), db: Session = Depends(get_db)
@@ -886,7 +1002,7 @@ def share_download(
db: Session = Depends(get_db),
) -> dict:
job = owned_or_404(db, DownloadJob, job_id, user, what="download")
if job.status != "done" or job.asset_id is None:
if job.status != JobStatus.DONE or job.asset_id is None:
raise refused(409, "share_unfinished")
target = body.email.strip().lower()
if not target:
@@ -1033,7 +1149,7 @@ def list_links(
class CreateLinkIn(BaseModel):
password: str = ""
# D1: a newly shared download is downloadable by default — the common intent is "let the
# A newly shared download is downloadable by default — the common intent is "let the
# recipient have the file", not stream-only. Forward-only: this only sets the default for links
# minted from now on; existing links keep whatever `allow_download` they were created with (the
# column default stays False, so no back-catalogue link silently flips). The owner can still
@@ -1050,7 +1166,7 @@ def create_link(
db: Session = Depends(get_db),
) -> dict:
job = owned_or_404(db, DownloadJob, job_id, user, what="download")
if job.status != "done" or job.asset_id is None:
if job.status != JobStatus.DONE or job.asset_id is None:
raise refused(409, "share_unfinished")
pw = body.password.strip()
link = DownloadLink(
@@ -1137,10 +1253,10 @@ def admin_list_downloads(
@admin_router.get("/storage")
def admin_storage(admin: User = Depends(admin_user), db: Session = Depends(get_db)) -> dict:
ready = db.execute(
select(func.count(), func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(
MediaAsset.status == "ready"
)
# Every file on disk (`quota.ON_DISK`): ready ones, and uploads waiting for or failed in their
# conversion — the same disk the footprint and the GC cap count.
on_disk = db.execute(
select(func.count(), func.coalesce(func.sum(MediaAsset.size_bytes), 0)).where(quota.ON_DISK)
).one()
total_assets = db.execute(select(func.count()).select_from(MediaAsset)).scalar()
per_user = db.execute(
@@ -1153,8 +1269,8 @@ def admin_storage(admin: User = Depends(admin_user), db: Session = Depends(get_d
for u in db.execute(select(User).where(User.id.in_([uid for uid, _ in per_user]))).scalars()
}
return {
"ready_files": ready[0],
"total_bytes": int(ready[1]),
"files_on_disk": on_disk[0],
"total_bytes": int(on_disk[1]),
"total_assets": total_assets,
"total_cap_bytes": sysconfig.get_int(db, "download_total_max_bytes"),
"per_user": [
+11 -34
View File
@@ -1,7 +1,7 @@
from datetime import date, datetime, timezone
from fastapi import APIRouter, Depends, Query
from pydantic import BaseModel, ConfigDict, Field
from pydantic import BaseModel
from sqlalchemy import and_, func, or_, select
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.orm import Session
@@ -26,14 +26,19 @@ from app.feedquery import (
windowed_relevance_rows,
)
from app.models import (
POSITION_MAX_S,
ChannelTag,
Tag,
User,
Video,
VideoState,
)
from app.routes._common import get_or_404, youtube_refusal, not_found
from app.routes._common import (
ProgressIn,
get_or_404,
kept_position,
not_found,
youtube_refusal,
)
from app.sync.videos import parse_iso8601_duration
from app.utils import iso
from app.youtube.client import YouTubeClient, YouTubeError
@@ -44,14 +49,6 @@ router = APIRouter(prefix="/api", tags=["feed"])
# "saved" used to be a status; it's now membership in the built-in Watch later playlist.
VALID_STATES = {"new", "watched", "hidden"}
# Resume-position thresholds (mirror the client): positions below this are "didn't
# really start" and within this of the end are "basically finished" — neither is worth
# storing, so we clear the position instead (the video shows no progress bar).
# NB: the Plex player intentionally uses a LARGER finish margin (30s) because it marks
# long-form items watched, not just clears a resume bar — see routes/plex/playback.py.
PROGRESS_MIN_SECONDS = 5
FINISH_MARGIN_SECONDS = 10
# Shared query parameters for /feed and /feed/count.
def _feed_params(
@@ -326,23 +323,10 @@ def clear_video_state(
return {"ok": True, "video_id": video_id, "status": "new", "position_seconds": 0}
class VideoProgressIn(BaseModel):
# Finite only: JSON parsing here accepts `Infinity`/`NaN`, and the handler's `int()` turned
# one into a 500.
model_config = ConfigDict(allow_inf_nan=False)
# Accept a float and truncate (below) — the player may post a raw currentTime (a float), which
# the old `int(payload.get(...))` silently truncated. A plain `int` field would 422 that on the
# 10s progress hot path, losing the resume position; `float` keeps the old tolerance.
# Bounded above by the int4 column (see POSITION_MAX_S): past it the flush was a 500.
position_seconds: float = Field(0, le=POSITION_MAX_S)
duration_seconds: float = Field(0, le=POSITION_MAX_S)
@router.post("/videos/{video_id}/progress")
def set_video_progress(
video_id: str,
body: VideoProgressIn,
body: ProgressIn,
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
@@ -350,22 +334,15 @@ def set_video_progress(
Stores a per-user position on the video_states row without touching watch status, so a
partially-watched video can render a progress bar and match the "in progress" filter.
Trivially-early and near-finished positions clear the position rather than store it."""
position = int(body.position_seconds)
duration = int(body.duration_seconds)
if position < 0:
position = 0
position = kept_position(body)
get_or_404(db, Video, video_id, what="video")
# Decide whether this position is worth keeping (else clear it).
near_end = duration > 0 and position > duration - FINISH_MARGIN_SECONDS
keep = position >= PROGRESS_MIN_SECONDS and not near_end
row = db.execute(
select(VideoState).where(VideoState.user_id == user.id, VideoState.video_id == video_id)
).scalar_one_or_none()
if not keep:
if not position:
# Nothing meaningful to store: drop a status-less row entirely, otherwise just
# zero the position (a saved/hidden video keeps its status).
if row is not None:
+4
View File
@@ -164,6 +164,7 @@ def unified_library(
cast(null(), Integer).label("season_count"),
func.coalesce(st.position_seconds, 0).label("position_seconds"),
m_status.label("status"),
literal(False).label("resumable"),
# "last watched" sort key: when the movie was last resumed, else when it was marked
# watched. NULL for never-touched movies → nullslast sinks them (see the sort dict).
func.coalesce(st.progress_updated_at, st.watched_at).label("last_watched"),
@@ -211,6 +212,8 @@ def unified_library(
PlexShow.child_count.label("season_count"),
literal(0).label("position_seconds"),
s_status.label("status"),
# An episode holds a resume point (`ShowState.resumable`, the card's Continue + Restart).
(inp > 0).label("resumable"),
# Show leg's "last watched": max(resume/watched time) across the show's episodes (see
# rollup.agg_subquery include_resumed). NULL for a never-watched show → nullslast.
agg.c.resumed_at.label("last_watched"),
@@ -276,6 +279,7 @@ def unified_library(
"thumb": f"/api/plex/image/{r.rk}",
"season_count": r.season_count,
"status": r.status,
"resumable": r.resumable,
}
)
+60 -6
View File
@@ -24,7 +24,7 @@ from app.models import (
)
from app.plex import rollup
from app.plex import sync as plex_sync
from app.plex.cards import display_title, movie_card, norm_lang, show_card
from app.plex.cards import SHOW_NEW, display_title, movie_card, norm_lang, show_card
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
from app.plex.lookups import by_rating_key
from app.plex.query import content_libs, runtime_by_key, title_col, top_art_keys
@@ -176,6 +176,40 @@ def collections(
}
@router.get("/collections/for/{item_rating_key}")
def collections_for_item(
item_rating_key: str,
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
"""What a movie or show card's "Add to collection" lists: the editable collections of the title's
library (a collection lives in one library, and only a manual one can take a title — the same
`can_edit` the editor offers), each saying whether the title is in it already, by name."""
movie = by_rating_key(db, PlexItem, item_rating_key)
title: PlexItem | PlexShow | None = (
movie if movie is not None and movie.kind == "movie" else None
) or by_rating_key(db, PlexShow, item_rating_key)
if title is None:
raise not_found("item")
member_of = set(title.collection_keys or [])
rows = (
db.execute(
select(PlexCollection)
.where(PlexCollection.library_id == title.library_id)
.order_by(collated(PlexCollection.title, lang_for_user(user)))
)
.scalars()
.all()
)
return {
"collections": [
{**_collection_card(c), "contains": c.rating_key in member_of}
for c in rows
if _collection_editable(c)
]
}
class Member(NamedTuple):
"""One collection member flattened to everything the Sort control can order by. The page's two
legs are different tables (movies and shows), so there is no single query to ORDER BY — the
@@ -284,7 +318,7 @@ def collection_detail(
if shows
else []
)
sstate = {r.show_id: rollup.show_status(r.total, r.watched, r.inprog) for r in show_rows}
sstate = {r.show_id: rollup.show_state(r.total, r.watched, r.inprog) for r in show_rows}
sresumed = {r.show_id: r.resumed_at for r in show_rows}
def _movie_last_watched(st: PlexState | None) -> datetime | None:
@@ -325,7 +359,7 @@ def collection_detail(
items = [
movie_card(m.obj, mstate.get(m.obj.id), lang=lang)
if m.kind == "movie"
else show_card(m.obj, sstate.get(m.obj.id, "new"), lang=lang)
else show_card(m.obj, sstate.get(m.obj.id, SHOW_NEW), lang=lang)
for m in members
]
years = [m.year for m in members if m.year is not None]
@@ -407,27 +441,47 @@ def create_collection(
seed = body.item_rating_key
if not title or not seed:
raise refused(400, "collection_fields_required")
seed_show = by_rating_key(db, PlexShow, seed)
if library:
lib = (
db.execute(select(PlexLibrary).where(PlexLibrary.plex_key == library)).scalars().first()
)
else:
# The "New collection" flow only knows the seed title → derive its library.
seed_item = by_rating_key(db, PlexItem, seed) or by_rating_key(db, PlexShow, seed)
seed_item = by_rating_key(db, PlexItem, seed) or seed_show
lib = db.get(PlexLibrary, seed_item.library_id) if seed_item else None
if lib is None:
raise not_found("library")
_reject_duplicate_name(db, lib.id, title)
wrote = False
new_rk = ""
try:
with PlexClient(db) as plex:
new = plex.create_collection(lib.plex_key, title, seed)
# A show collection is made as one (Plex type 2): as a movie one it stayed empty.
new = plex.create_collection(lib.plex_key, title, seed, 2 if seed_show else 1)
wrote = True
col = plex_sync.resync_collection(db, plex, lib, str(new.get("ratingKey")))
new_rk = str(new.get("ratingKey") or "")
col = plex_sync.resync_collection(db, plex, lib, new_rk)
col.editable = True
db.commit()
except (PlexError, PlexNotConfigured) as e:
db.rollback()
if wrote and new_rk:
# It is on Plex now, and only the follow-up read failed. Its row is kept, editable: the
# full sync preserves `editable` but creates a missing row read-only, so the collection
# would drop out of every "add to" list; and the duplicate-name check reads these rows,
# so a retry of the same name would make a second one on Plex. Plex made it with exactly
# the seed in it, so the seed's row says so too — otherwise the title showed unticked in
# the collection it was just added to, and the collection empty, until that sync.
db.add(
PlexCollection(
rating_key=new_rk, library_id=lib.id, title=title, editable=True, child_count=1
)
)
Model = PlexItem if lib.kind == "movie" else PlexShow
if row := by_rating_key(db, Model, seed):
row.collection_keys = list(dict.fromkeys([*(row.collection_keys or []), new_rk]))
db.commit()
raise _write_or_pending_error(wrote, e) from e
return _collection_card(col)
+29 -2
View File
@@ -14,6 +14,7 @@ from app.collation import collated, sorted_words
from app import logs
from app.auth import current_user
from app.db import get_db
from app.discover import queries
from app.models import (
PlexCollection,
PlexItem,
@@ -28,7 +29,7 @@ from app.plex import people as plex_people
from app.plex import rollup, subtitles
from app.plex.query import title_col
from app.plex import tmdb as plex_tmdb
from app.plex.cards import episode_card, movie_card, show_card
from app.plex.cards import SHOW_NEW, episode_card, movie_card, show_card
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
from app.plex.lookups import by_rating_key
from app.routes.plex._shared import _norm_lang, item_or_404
@@ -221,6 +222,24 @@ def show_detail(
}
@router.get("/show/{rating_key}/next")
def show_next(
rating_key: str,
lang: str = "en",
user: User = Depends(current_user),
db: Session = Depends(get_db),
) -> dict:
"""The episode a show's card plays: the one in progress, else the first unwatched, else the first
(`discover.queries.next_episode`, the rule Home's shelves and the show page's Resume follow).
Asked one show at a time, when a card is hovered or played, rather than carried on every card of a
grid: three probes a show would cost a 60-card page 180 queries for the few cards anyone plays.
`next` is null for a show with no episodes."""
sh = by_rating_key(db, PlexShow, rating_key)
if sh is None:
raise not_found("show")
return {"next": queries.next_episode(db, user.id, sh.id, lang=_norm_lang(lang))}
def _season_block(se: PlexSeason, cards: list[dict]) -> dict:
"""A season's card for the show page (with its aggregate rollup) plus its episodes (used by the
season subpage, read from the same cached show-detail payload)."""
@@ -260,7 +279,7 @@ def _relatedshow_cards(
if sh is None or rk in seen:
continue
seen.add(rk)
out.append(show_card(sh, statuses.get(sh.id, "new"), lang=lang))
out.append(show_card(sh, statuses.get(sh.id, SHOW_NEW), lang=lang))
if len(out) >= 20:
break
return out
@@ -404,6 +423,10 @@ def item_detail(
"type": m.get("type"),
"start_s": round((m.get("startTimeOffset") or 0) / 1000, 1),
"end_s": round((m.get("endTimeOffset") or 0) / 1000, 1),
# An item can carry several credits blocks with scenes between them; Plex
# flags the one that really ends it. Which credits may leave the item is the
# player's rule (`lib/plexMarkers`), so the flag goes over as Plex sent it.
"final": bool(m.get("final")),
}
)
# Audio + subtitle tracks (ordinals among their stream type — used to -map the selection).
@@ -500,6 +523,7 @@ def item_detail(
prev_id, next_id = _episode_nav(db, it)
show = db.get(PlexShow, it.show_id) if it.kind == "episode" and it.show_id else None
season = db.get(PlexSeason, it.season_id) if show and it.season_id else None
return {
"id": it.rating_key,
"kind": it.kind,
@@ -533,6 +557,9 @@ def item_detail(
"status": st.status if st else "new",
"position_seconds": st.position_seconds if st else 0,
"show_title": ((show.title_hu or show.title if hu else show.title) if show else None),
# The episode page links up to its show and its season (their rating keys = the view keys).
"show_id": show.rating_key if show else None,
"season_id": season.rating_key if season else None,
"season_number": it.season_number,
"episode_number": it.episode_number,
"prev_id": prev_id,
+104 -37
View File
@@ -23,7 +23,13 @@ from app.auth import current_user
from app.db import get_db
from app.models import PlexItem, PlexShow, PlexState, User
from app.plex import rollup
from app.plex.cards import display_title, leaf_card, movie_card, show_card
from app.plex.cards import (
SHOW_NEW,
display_title,
leaf_card,
movie_card,
show_card,
)
from app.plex.query import (
LibraryFilters,
apply_meta_filters,
@@ -69,7 +75,7 @@ def _movie_cards(db: Session, uid: int, items: list[PlexItem], lang: str) -> lis
def _show_cards(db: Session, uid: int, shows: list[PlexShow], lang: str) -> list[dict]:
statuses = rollup.state_map(db, uid, [s.id for s in shows])
return [show_card(s, statuses.get(s.id, "new"), lang=lang) for s in shows]
return [show_card(s, statuses.get(s.id, SHOW_NEW), lang=lang) for s in shows]
def _recently_added(
@@ -232,7 +238,9 @@ def _continue_watching(
# `apply_show_state_filter` the grid uses, so the rail can't drift from the grid's definition.
prog = rollup.agg_subquery(db, uid, include_resumed=True).subquery()
sq = apply_show_state_filter(
select(PlexShow, prog.c.resumed_at).join(prog, prog.c.show_id == PlexShow.id),
select(PlexShow, prog.c.resumed_at, prog.c.total, prog.c.watched, prog.c.inprog).join(
prog, prog.c.show_id == PlexShow.id
),
prog.c.total,
prog.c.watched,
prog.c.inprog,
@@ -243,8 +251,9 @@ def _continue_watching(
.order_by(prog.c.resumed_at.desc().nullslast())
.limit(RAIL_LIMIT)
).all()
for sh, resumed_at in srows:
rows.append((resumed_at, show_card(sh, "in_progress", lang=lang)))
for sh, resumed_at, total, watched, inprog in srows:
state = rollup.show_state(total, watched, inprog)
rows.append((resumed_at, show_card(sh, state, lang=lang)))
rows.sort(key=lambda r: r[0] or rollup.NO_TS, reverse=True)
return [card for _, card in rows[:RAIL_LIMIT]]
@@ -402,11 +411,71 @@ def discover(
}
def _draw(db: Session, stmt, key_col, per: int, exclude: list[str]) -> list:
"""`per` random rows of `stmt`, the previous draw's (`exclude`) left out while the pool allows: a
reshuffle over a small filtered pool kept dealing the same few titles. What the fresh rows leave
short is topped up from the excluded ones, so a pool smaller than a draw still fills the rail."""
fresh = (
db.execute(stmt.where(key_col.not_in(exclude)).order_by(func.random()).limit(per))
.scalars()
.all()
if exclude
else []
)
if len(fresh) == per:
return list(fresh)
taken = [key_col.not_in([r.rating_key for r in fresh])] if fresh else []
rest = db.execute(stmt.where(*taken).order_by(func.random()).limit(per - len(fresh))).scalars()
return [*fresh, *rest.all()]
def _kept(
db: Session, uid: int, movie_ids: list[int], show_ids: list[int], keep: list[str], lang: str
) -> list[dict]:
"""The `keep` titles' cards in their given order, as they stand now: a title watched since the draw
stays in it, marked watched, and one gone from the scope's libraries drops out."""
movies = (
db.execute(
select(PlexItem).where(
PlexItem.kind == "movie",
PlexItem.library_id.in_(movie_ids),
PlexItem.rating_key.in_(keep),
)
)
.scalars()
.all()
if movie_ids
else []
)
shows = (
db.execute(
select(PlexShow).where(PlexShow.library_id.in_(show_ids), PlexShow.rating_key.in_(keep))
)
.scalars()
.all()
if show_ids
else []
)
cards = {
c["id"]: c
for c in [
*_movie_cards(db, uid, list(movies), lang),
*_show_cards(db, uid, list(shows), lang),
]
}
return [cards[k] for k in keep if k in cards]
@router.get("/surprise")
def surprise(
scope: str = "both",
lang: str = "en",
limit: int = Query(default=2 * SURPRISE_PER_KIND, ge=1, le=40),
# The previous draw's rating keys: a reshuffle deals other titles first.
exclude: list[str] = Query(default=[], max_length=40),
# The draw on screen, read again rather than redrawn: its titles' watch state moves (a play, a
# toggle) while the draw itself stands until the ↻.
keep: list[str] = Query(default=[], max_length=40),
f: LibraryFilters = Depends(),
user: User = Depends(current_user),
db: Session = Depends(get_db),
@@ -419,6 +488,8 @@ def surprise(
lang = _norm_lang(lang)
p = f.as_dict()
movie_ids, show_ids = _scope_libs(db, scope)
if keep:
return {"scope": scope, "items": _kept(db, user.id, movie_ids, show_ids, keep, lang)}
per = min(limit, SURPRISE_PER_KIND)
movies = []
if movie_ids:
@@ -432,23 +503,21 @@ def surprise(
)
.exists()
)
movies = (
db.execute(
_movie_meta(
select(PlexItem).where(
PlexItem.kind == "movie",
PlexItem.library_id.in_(movie_ids),
~watched_movie,
),
p,
lang,
f,
)
.order_by(func.random())
.limit(per)
)
.scalars()
.all()
movies = _draw(
db,
_movie_meta(
select(PlexItem).where(
PlexItem.kind == "movie",
PlexItem.library_id.in_(movie_ids),
~watched_movie,
),
p,
lang,
f,
),
PlexItem.rating_key,
per,
exclude,
)
shows = []
if show_ids:
@@ -458,21 +527,19 @@ def surprise(
not_fully_watched = or_(
prog.c.total.is_(None), prog.c.total == 0, prog.c.watched < prog.c.total
)
shows = (
db.execute(
apply_meta_filters(
select(PlexShow)
.outerjoin(prog, prog.c.show_id == PlexShow.id)
.where(PlexShow.library_id.in_(show_ids), not_fully_watched),
PlexShow,
p,
lang,
)
.order_by(func.random())
.limit(per)
)
.scalars()
.all()
shows = _draw(
db,
apply_meta_filters(
select(PlexShow)
.outerjoin(prog, prog.c.show_id == PlexShow.id)
.where(PlexShow.library_id.in_(show_ids), not_fully_watched),
PlexShow,
p,
lang,
),
PlexShow.rating_key,
per,
exclude,
)
mcards = _movie_cards(db, user.id, movies, lang)
scards = _show_cards(db, user.id, shows, lang)
+3 -3
View File
@@ -19,7 +19,7 @@ from app.models import (
)
from app.plex import rollup
from app.plex import tmdb as plex_tmdb
from app.plex.cards import movie_card, norm_lang, show_card
from app.plex.cards import SHOW_NEW, movie_card, norm_lang, show_card
from app.plex.query import content_libs, split_csv, title_col
router = APIRouter(prefix="/api/plex", tags=["plex"])
@@ -57,7 +57,7 @@ def _resolve_known_for(
# A credit resolves against its OWN media_type's bucket only.
if k.get("media_type") == "tv":
if g in shows:
out.append(show_card(shows[g], show_statuses.get(shows[g].id, "new"), lang=lang))
out.append(show_card(shows[g], show_statuses.get(shows[g].id, SHOW_NEW), lang=lang))
elif g in movies:
out.append(movie_card(movies[g], m_states.get(movies[g].id), lang=lang))
return out[:_KNOWN_FOR_SHOWN]
@@ -245,7 +245,7 @@ def person_detail(
.all()
)
statuses = rollup.state_map(db, user.id, [s.id for s in show_rows])
shows = [show_card(sh, statuses.get(sh.id, "new"), lang=lang) for sh in show_rows]
shows = [show_card(sh, statuses.get(sh.id, SHOW_NEW), lang=lang) for sh in show_rows]
known_for = _resolve_known_for(
db, person.known_for if person else None, user.id, movie_lib_ids, show_lib_ids, lang
+24 -17
View File
@@ -15,7 +15,7 @@ from fastapi import (
Response,
)
from fastapi.responses import FileResponse
from pydantic import BaseModel, ConfigDict, Field
from pydantic import BaseModel
from sqlalchemy import select, update
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.orm import Session
@@ -23,7 +23,6 @@ from sqlalchemy.orm import Session
from app.auth import current_user, resolved_user_id
from app.db import SessionLocal, get_db
from app.models import (
POSITION_MAX_S,
PlexItem,
PlexSeason,
PlexShow,
@@ -39,7 +38,7 @@ from app.plex import watch_sync as plex_watch
from app.plex.client import PlexClient, PlexError, PlexNotConfigured
from app.routes.plex._shared import item_or_404
from app.refusal import refused
from app.routes._common import not_found
from app.routes._common import ProgressIn, not_found
router = APIRouter(prefix="/api/plex", tags=["plex"])
@@ -160,11 +159,11 @@ def subtitle(
return Response(subtitles.shift_vtt(hit, offset), media_type="text/vtt")
# Resume/watched thresholds. _PROGRESS_MIN_S matches the feed's PROGRESS_MIN_SECONDS (5), but
# _FINISH_MARGIN_S is DELIBERATELY larger than the feed's FINISH_MARGIN_SECONDS (30 vs 10): this is
# long-form content (movies/episodes with credits) and crossing it MARKS THE ITEM WATCHED, whereas
# the feed's margin only clears a short video's resume bar — different domains, intentional divergence,
# not drift. Keep them separate on purpose (see routes/feed.py).
# Resume/watched thresholds. _PROGRESS_MIN_S matches PROGRESS_MIN_SECONDS (5), but _FINISH_MARGIN_S is
# DELIBERATELY larger than FINISH_MARGIN_SECONDS (30 vs 10): this is long-form content
# (movies/episodes with credits) and crossing it MARKS THE ITEM WATCHED, whereas the feed's and the
# library's margin only clears a resume bar — different domains, intentional divergence, not drift.
# Keep them separate on purpose (see routes/_common.py).
_PROGRESS_MIN_S = 5
_FINISH_MARGIN_S = 30
@@ -337,16 +336,9 @@ def _mark_finished(db: Session, user_id: int, item_id: int, now: datetime) -> bo
return False
class ItemProgressIn(BaseModel):
# Finite only: JSON parsing here accepts `Infinity`/`NaN`, and the handler's `int()` turned
# one into a 500.
model_config = ConfigDict(allow_inf_nan=False)
class ItemProgressIn(ProgressIn):
"""The shared checkpoint (its bounds included), plus whether the player is leaving the item."""
# float, truncated in the handler — the player may post a raw currentTime (a float); a plain
# `int` field would 422 it on the periodic progress checkpoint (the old `int(...)` truncated).
# Bounded above by the int4 column (see POSITION_MAX_S): past it the flush was a 500.
position_seconds: float = Field(0, le=POSITION_MAX_S)
duration_seconds: float = Field(0, le=POSITION_MAX_S)
final: bool = False
@@ -637,12 +629,27 @@ def stream_session(
# The session id is part of the path (not a query param) so the playlist's RELATIVE segment
# names resolve back into the same session — see `stream_hls_file` for why it has to.
"url": f"/api/plex/stream/{it.rating_key}/hls/{s.id}/{s.entry}",
# What the player names when it no longer wants the session (POST …/session/{id}/stop).
"id": s.id,
"transcode": s.transcode,
"hw": s.hw,
"tonemap": s.tonemap,
}
@router.post("/stream/{rating_key}/session/{session_id}/stop")
def stream_session_stop(
rating_key: str,
session_id: str,
user: User = Depends(current_user),
) -> dict:
"""The player no longer wants this session (it left the item, closed, or the page unloaded):
its ffmpeg stops now instead of at the idle reaper. A POST, so the player can send it as a
keepalive beacon from a page being unloaded. Stops nothing but the caller's own session, and
only while it is still the one registered for the item (see `stop_session`)."""
return {"ok": True, "stopped": plex_stream.stop_session(user.id, rating_key, session_id)}
@router.get("/stream/{rating_key}/file")
def stream_file(
rating_key: str,
+24 -1
View File
@@ -16,7 +16,8 @@ from app.config import settings
from app.db import get_db
from app.downloads import links as linksmod
from app.downloads import storage
from app.models import DownloadJob, DownloadLink, MediaAsset
from app.downloads.status import JobStatus
from app.models import DownloadJob, DownloadLink, MediaAsset, User
from app.ratelimit import RateLimiter, client_key
from app.security import verify_password
from app.refusal import refused
@@ -43,9 +44,31 @@ def _link_or_404(db: Session, token: str) -> DownloadLink:
return link
def _being_made(db: Session, job: DownloadJob) -> bool:
"""The job is running, or queued for a worker that will take it: the worker leaves a suspended
owner's queued jobs where they are (`worker._claim_job`), as long as the suspension lasts."""
if job.status == JobStatus.RUNNING:
return True
if job.status != JobStatus.QUEUED:
return False
owner = db.get(User, job.user_id)
return owner is not None and not owner.is_suspended
def _ready_target(db: Session, link: DownloadLink) -> tuple[DownloadJob, MediaAsset]:
job = db.get(DownloadJob, link.job_id)
asset = db.get(MediaAsset, job.asset_id) if job and job.asset_id else None
if (
job is not None
and asset is not None
and asset.status in ("pending", "downloading")
and _being_made(db, job)
):
# Being (re)made — a re-download, or a conversion — so the link works again once it is ready:
# its own reason, so the page can say "try again shortly" rather than "this link is gone".
# Only while the job is still going: a cancelled or paused rebuild leaves the asset pending
# with nothing working on it, and "try again in a few minutes" would never come true.
raise refused(404, "video_preparing")
if job is None or asset is None or asset.status != "ready" or not asset.rel_path:
raise refused(404, "video_gone")
# Verify the file is actually on disk here (not just flagged ready), so the watch page never
+1 -1
View File
@@ -12,7 +12,7 @@ result (measured on real data: every untagged channel got a suggestion, most on
alone). Topic overlap is what makes "you tagged similar channels this way" meaningful.
Purely read-only and deterministic — the Channel Manager surfaces the result as one-tap-accept
"ghost" chips; nothing is written here (D4 = suggest-and-confirm, no silent auto-apply). The
"ghost" chips; nothing is written here (suggest-and-confirm, never a silent auto-apply). The
thresholds are tuning constants rather than admin knobs on purpose: they mirror how ``autotag.py``
keeps its mappings inline, and sysconfig only stores int/str/bool (no float), so a knob would mean
storing percentages — not worth the surface for values an operator has no reason to tune per-install.
+39 -11
View File
@@ -29,6 +29,7 @@ from datetime import datetime, timezone
from functools import partial
from sqlalchemy import text
from sqlalchemy.orm import Session
from app import logs
from app import logsink
@@ -44,7 +45,7 @@ from app.downloads.jobdb import (
_reset_asset_pending,
_staging_dir,
)
from app.downloads.status import STOPPED_BY_ROUTE, JobStatus
from app.downloads.status import STOPPED_BY_ROUTE, JobKind, JobStatus
from app.models import DISPLAY_TEXT_MAX, DownloadJob, MediaAsset
log = logs.get_logger(__name__)
@@ -162,7 +163,12 @@ def _claim_job() -> int | None:
seen.) The per-user limit is the quota row's `max_concurrent` (else the sysconfig default);
`unlimited` users are never capped. A final Python `at_concurrency_limit` re-check still guards
the tiny race between the exclusion subquery and the claim (it can only ever reject, never
over-claim), so correctness never depends on the two observations agreeing."""
over-claim), so correctness never depends on the two observations agreeing.
A suspended user's queued jobs are left where they are: suspension blocks their downloads (not
only their sign-in), and lifting it lets them run in their old place. A job already running
when the user is suspended finishes on that worker; one a worker restart puts back in the queue
(`_requeue_if_running`, `_recover_orphans`) waits with the rest."""
with SessionLocal() as db:
default_mc = sysconfig.get_int(db, "download_default_max_concurrent")
rows = db.execute(
@@ -170,6 +176,10 @@ def _claim_job() -> int | None:
"""
SELECT id, user_id FROM download_jobs
WHERE status='queued'
AND NOT EXISTS (
SELECT 1 FROM users u
WHERE u.id = download_jobs.user_id AND u.is_suspended
)
AND NOT EXISTS (
SELECT 1 FROM media_assets a
WHERE a.id = download_jobs.asset_id AND a.status = 'downloading'
@@ -437,7 +447,7 @@ def _run_supervised(job_id: int, asset_id: int) -> None:
if status is None or status in STOPPED_BY_ROUTE:
reason = "cancelled" # a route stopped it (or the row vanished)
break
if status == "running":
if status == JobStatus.RUNNING:
if stale is not None and stale > stall_s:
reason = "stalled"
break
@@ -448,7 +458,7 @@ def _run_supervised(job_id: int, asset_id: int) -> None:
# teardown. A still-running job over the cap is failed ("timeout"); a terminal one (already
# done/error) just gets its lingering child killed, its DB status left intact ("abandon").
if time.monotonic() - started > wall_s:
reason = "timeout" if status == "running" else "abandon"
reason = "timeout" if status == JobStatus.RUNNING else "abandon"
break
# else: a terminal status under the cap (done/error, or a re-queued contended asset) means
# the child is exiting on its own — keep polling; NEVER kill a finishing child (that could
@@ -479,7 +489,7 @@ def _run_supervised(job_id: int, asset_id: int) -> None:
_reset_asset_pending(asset_id)
if proc.returncode != _CHILD_OK:
status, _ = _job_liveness(job_id)
if status == "running":
if status == JobStatus.RUNNING:
_cleanup_staging(asset_id)
_fail_running(
job_id,
@@ -560,15 +570,12 @@ def _process_job(job_id: int) -> None:
asset_id = asset.id if asset else None
asset_status = asset.status if asset else None
asset_error = asset.error if asset else None
if asset_status == "error":
if asset is not None and asset_status == "error":
# A job that meets an already-failed asset gets that failure's explanation — not a bare
# "Download failed". Only the text is on the asset, so it is diagnosed again, for this
# link, with whether a cookie goes with it asked of the same gate the download child
# used (`credentials.usable_cookie`) — as it stands now, so the advice fits this job's Retry.
url = service.source_url(job.source_kind, job.source_ref)
cookie = credentials.usable_cookie(
db, job.user_id, url, asset, formats.normalize(job.profile_snapshot)
)
url, cookie = _failed_link_and_cookie(db, job, asset)
diagnosis = dlerrors.diagnose(asset_error, url=url, cookie_attached=cookie is not None)
del cookie # only whether there is one matters here
@@ -605,13 +612,34 @@ def _process_job(job_id: int) -> None:
_run_supervised(job_id, asset_id)
def _failed_link_and_cookie(
db: Session, job: DownloadJob, asset: MediaAsset
) -> tuple[str | None, str | None]:
"""The link a job's failed asset was fetched from, and the cookie that fetch would go with now.
A clip's link is its source's (a `clip` ref names the user's range, no page), asked of the
clip's own cookie gate, as its fetch was (`_clip_source`); a Plex clip fetches nothing."""
if job.job_kind == JobKind.CLIP:
spec = dict(job.edit_spec or {})
if spec.get("source") != "youtube":
return None, None
url = service.source_url("youtube", spec["ref"])
return url, credentials.usable_clip_cookie(db, job.user_id, url, asset, spec)
url = service.source_url(job.source_kind, job.source_ref)
spec = formats.normalize(job.profile_snapshot)
return url, credentials.usable_cookie(db, job.user_id, url, asset, spec)
def _finish_from_cache(job_id: int, asset_id: int) -> None:
with SessionLocal() as db:
job = db.get(DownloadJob, job_id)
asset = db.get(MediaAsset, asset_id)
if job and asset:
fields = {"status": JobStatus.DONE, "progress": 100, "phase": None}
if not job.display_name and asset.title:
# A clip queued under the placeholder takes the title its sibling's fetch found.
unnamed = not job.display_name or (
job.job_kind == JobKind.CLIP and job.display_name == service.clip_title(None, None)
)
if unnamed and asset.title:
# display_name is VARCHAR(255); a title can be longer (e.g. a recipe in the title).
fields["display_name"] = asset.title[:DISPLAY_TEXT_MAX]
asset.last_access_at = datetime.now(timezone.utc)
+5 -4
View File
@@ -2,9 +2,10 @@
# Source of truth is requirements.txt (the floor ranges); regenerate after editing it with:
# docker run --rm -v "$PWD":/b python:3.12-slim sh -c 'pip install -q -r /b/requirements.txt && pip freeze --exclude pip --exclude setuptools --exclude wheel' | sort
# The prod Dockerfile + Dockerfile.test install THIS file; the gate's pip-audit (scripts/check.sh)
# scans it, blocking. A security fix may be ONE pin bumped in place instead of a full
# regeneration, which would move every other package too — then `pip install -r requirements.lock && pip check` must
# pass. In place so far: urllib3 2.7.0 -> 2.8.0 (PYSEC-2026-4175/4176/4177).
# scans it, blocking. A security fix, or a yt-dlp release YouTube's changes force, may be ONE pin
# bumped in place instead of a full regeneration, which would move every other package too — then
# `pip install -r requirements.lock && pip check` must pass. In place so far: urllib3 2.7.0 -> 2.8.0 (PYSEC-2026-4175/4176/4177); yt-dlp 2026.7.4 -> 2026.8.19
# (it marks the clients that need a stream PO token, so the downloader stops fetching into a 403).
APScheduler==3.11.3
Authlib==1.7.2
Mako==1.4.1
@@ -62,4 +63,4 @@ uvloop==0.22.1
watchfiles==1.2.0
websockets==17.0.1
yt-dlp-ejs==0.8.0
yt-dlp==2026.7.4
yt-dlp==2026.8.19
+6 -4
View File
@@ -16,12 +16,14 @@ argon2-cffi>=23.1,<26
# Download center: yt-dlp does the extraction/download; ffmpeg (OS-level in the Dockerfile)
# does the merge + postprocessing. The [default] extra pulls yt-dlp-ejs (the challenge-solver
# scripts Deno runs for YouTube's n-signature). yt-dlp is release-often (YouTube changes), so
# keep the floor loose and bump it when extraction breaks.
yt-dlp[default]>=2025.5.22
# keep the floor loose and bump it when extraction breaks. 2026.8.19 marks the clients that need a
# stream PO token, which the shipped player clients (config.download_player_clients) rely on.
yt-dlp[default]>=2026.8.19
# Browser impersonation (`impersonate`) for Facebook downloads made with the user's cookie: some
# links answer a signed-in plain request with "Cannot parse data" and only parse as Chrome
# (measured). The range is YT-DLP's, not ours — yt-dlp 2026.07.04 imports curl_cffi only in
# 0.10.x–0.15.x and silently reports every target "unavailable" outside it. Move it when bumping
# (measured). The range is YT-DLP's, not ours — yt-dlp 2026.07.04 imported curl_cffi only in
# 0.10.x–0.15.x and silently reported every target "unavailable" outside it (2026.08.19 takes up to
# 0.16.x; the cap stays on the measured 0.15 until a bump re-measures it). Move it when bumping
# yt-dlp; `tests/test_facebook_impersonation.py::test_the_locked_curl_cffi_can_impersonate_chrome`
# fails the gate if the locked pair can no longer impersonate.
curl_cffi>=0.10,<0.16
+15
View File
@@ -175,6 +175,21 @@ def test_the_cookie_capable_clients_match_the_installed_ytdlp():
assert set(YouTubeAdapter._AUTHED_DEFAULT) <= live
def test_the_shipped_player_clients_exist_in_the_installed_ytdlp():
# yt-dlp skips a client it does not know with a warning and falls back to its own defaults, so
# a bump that renames or drops one (tv_simply, the one measured to deliver HD) would change
# what every download is fetched with, silently. Read from the Settings defaults, not from
# `settings`, which an environment may override.
from yt_dlp.extractor.youtube._base import INNERTUBE_CLIENTS
from app.config import Settings
fields = Settings.model_fields
for knob in ("download_player_clients", "download_player_clients_fallback"):
names = Settings._clients(fields[knob].default)
assert names and set(names) <= set(INNERTUBE_CLIENTS), (knob, names)
def test_youtube_ydl_opts_extra_carries_player_client_and_pot():
ea = YouTubeAdapter().ydl_opts_extra(["android_vr"])["extractor_args"]
assert ea["youtube"]["player_client"] == ["android_vr"]
+810
View File
@@ -0,0 +1,810 @@
"""Ad-hoc clips (downloads/clip): the spec, the one ffmpeg command every clip goes through, the route,
and the worker path for both sources.
The ffmpeg command is judged at argv level: where the seek sits, which streams are mapped, what the
video filter does to an HDR source. The worker runs with ffmpeg / yt-dlp replaced by stand-ins that
write a file where they were told to, so what is checked is what the pipeline asks of them and what
it stores afterwards.
"""
from pathlib import Path
import pytest
from app import worker as workermod
from app.config import settings
from app.downloads import clip as clipmod
from app.downloads import errors as dlerrors
from app.downloads import formats, pipeline, quota, runner, service
from app.downloads import probe as probemod
from app.downloads.edit import INPUT_GUARD
from app.models import Channel, DownloadJob, MediaAsset, PlexItem, PlexLibrary, PlexShow, Video
from app.plex import paths as plex_paths
# --- the spec -----------------------------------------------------------------------------------
def test_a_spec_is_canonical_so_the_same_marks_hit_the_same_clip():
a = clipmod.normalize("youtube", "abcdefghijk", 10.00049, 40.0)
b = clipmod.normalize("youtube", "abcdefghijk", 10.0, 40.0004)
assert a == b == {"source": "youtube", "ref": "abcdefghijk", "start_s": 10.0, "end_s": 40.0}
assert clipmod.sig(a) == clipmod.sig(b)
assert clipmod.sig(a) != clipmod.sig({**a, "end_s": 41.0})
@pytest.mark.parametrize(
("args", "reason"),
[
(("vimeo", "abcdefghijk", 0, 10), "bad_source"),
(("youtube", "short", 0, 10), "bad_source"),
(("youtube", "abcdefghij/", 0, 10), "bad_source"),
(("plex", "12a", 0, 10), "bad_source"),
(("youtube", "abcdefghijk", -1, 10), "bad_range"),
(("youtube", "abcdefghijk", 0, float("nan")), "bad_range"),
(("youtube", "abcdefghijk", 5, 5.5), "too_short"),
(("youtube", "abcdefghijk", 30, 10), "too_short"),
(("youtube", "abcdefghijk", 0, 600.001), "too_long"),
],
)
def test_a_range_it_cannot_cut_is_refused_with_its_reason(args, reason):
with pytest.raises(clipmod.ClipError) as e:
clipmod.normalize(*args)
assert e.value.reason == reason
def test_the_ten_minute_cap_itself_is_allowed():
assert clipmod.duration(clipmod.normalize("plex", "42", 0, 600)) == 600
def test_an_audio_track_is_kept_for_plex_only_and_must_be_sane():
assert clipmod.normalize("plex", "42", 0, 10, 2)["audio_track"] == 2
assert "audio_track" not in clipmod.normalize("youtube", "abcdefghijk", 0, 10, 2)
with pytest.raises(clipmod.ClipError, match="bad_audio"):
clipmod.normalize("plex", "42", 0, 10, -1)
def test_a_youtube_clip_links_its_video_at_the_clip_start_and_a_plex_clip_links_nothing():
yt = clipmod.normalize("youtube", "abcdefghijk", 75.9, 90)
assert clipmod.reference_url(yt) == "https://www.youtube.com/watch?v=abcdefghijk&t=75s"
assert clipmod.reference_url(clipmod.normalize("plex", "42", 0, 10)) is None
# --- the ffmpeg command ----------------------------------------------------------------------------
def _cmd(**kw):
args = {"audio_track": None, "hdr": False, "threads": 3, **kw}
return clipmod.build_cmd(Path("/m/in.mkv"), Path("/s/out.mp4"), 61.5, 30.0, **args)
def test_the_seek_is_on_the_input_so_the_clip_starts_on_the_marked_frame():
cmd = _cmd()
i = cmd.index("-i")
assert cmd[cmd.index("-ss") + 1] == "61.500" and cmd.index("-ss") < i
assert cmd[i + 1] == "/m/in.mkv"
assert cmd[cmd.index("-t") + 1] == "30.000" and cmd.index("-t") > i
# The single-file demuxer allowlist goes with every library read.
g = cmd.index(INPUT_GUARD[0])
assert cmd[g : g + len(INPUT_GUARD)] == INPUT_GUARD and g < i
assert cmd[-1] == "/s/out.mp4"
def test_it_maps_the_first_video_and_the_heard_audio_track_and_nothing_else():
cmd = _cmd(audio_track=2)
maps = [cmd[i + 1] for i, a in enumerate(cmd) if a == "-map"]
assert maps == ["0:v:0", "0:a:2"]
assert "-sn" in cmd and "-dn" in cmd
# An mkv's chapters would ride into the .mp4 as a data track of their own (measured on a Blu-ray
# rip: `bin_data`), which -dn does not drop.
assert cmd[cmd.index("-map_chapters") + 1] == "-1"
# No track chosen: the first one, and a source with no audio at all still cuts.
maps = [c for i, c in enumerate(_cmd()) if i and _cmd()[i - 1] == "-map"]
assert maps == ["0:v:0", "0:a:0?"]
def test_the_output_is_phone_safe_stereo_h264_at_most_1080p():
cmd = _cmd()
assert cmd[cmd.index("-c:v") + 1] == "libx264"
assert cmd[cmd.index("-pix_fmt") + 1] == "yuv420p"
assert cmd[cmd.index("-c:a") + 1] == "aac" and cmd[cmd.index("-ac") + 1] == "2"
assert cmd[cmd.index("-threads") + 1] == "3"
vf = cmd[cmd.index("-vf") + 1]
assert "min(1080,ih)" in vf and "tonemap" not in vf
def test_an_hdr_source_is_tonemapped_to_sdr_before_it_is_scaled():
vf = _cmd(hdr=True)[_cmd(hdr=True).index("-vf") + 1]
assert vf.index("tonemap") < vf.index("zscale=t=bt709") < vf.index("min(1080,ih)")
@pytest.mark.parametrize(
("transfer", "hdr"),
[("smpte2084", True), ("arib-std-b67", True), ("bt709", False), (None, False)],
)
def test_hdr_is_read_from_the_first_video_stream(transfer, hdr):
data = {
"streams": [{"codec_type": "audio"}, {"codec_type": "video", "color_transfer": transfer}]
}
assert clipmod.is_hdr(data) is hdr
# --- the crop ----------------------------------------------------------------------------------------
_FRAME = {"x": 0.5, "y": 0.25, "w": 0.25, "h": 0.5, "aspect": 16 / 9}
def test_a_crop_frame_is_canonical_and_a_frame_over_the_whole_box_is_no_crop():
a = clipmod.normalize("plex", "42", 0, 10, crop={**_FRAME, "x": 0.50004})
b = clipmod.normalize("plex", "42", 0, 10, crop=_FRAME)
assert a["crop"] == b["crop"] == {"x": 0.5, "y": 0.25, "w": 0.25, "h": 0.5, "aspect": 1.7778}
assert clipmod.sig(a) == clipmod.sig(b)
whole = clipmod.normalize(
"plex", "42", 0, 10, crop={"x": 0, "y": 0, "w": 1, "h": 1, "aspect": 1}
)
assert whole == clipmod.normalize("plex", "42", 0, 10)
# A far edge a hair past the box (the client's rounding) is the box's edge.
assert clipmod.normalize("plex", "42", 0, 10, crop={**_FRAME, "w": 0.5004})["crop"]["w"] == 0.5
@pytest.mark.parametrize(
"frame",
[
{**_FRAME, "w": 0.05},
{**_FRAME, "h": 0.09},
{**_FRAME, "x": -0.1},
{**_FRAME, "x": 0.8},
{**_FRAME, "y": 0.6},
{**_FRAME, "aspect": 0},
{**_FRAME, "aspect": 9},
{**_FRAME, "x": float("nan")},
{"x": 0.1, "y": 0.1, "w": 0.5},
{**_FRAME, "h": "half"},
],
)
def test_a_crop_frame_it_cannot_cut_is_refused(frame):
with pytest.raises(clipmod.ClipError) as e:
clipmod.normalize("plex", "42", 0, 10, crop=frame)
assert e.value.reason == "bad_crop"
@pytest.mark.parametrize(
("stream", "dar"),
[
({"width": 1920, "height": 1080}, 16 / 9),
# An anamorphic DVD: 720×480 stored, shown 16:9.
({"width": 720, "height": 480, "sample_aspect_ratio": "32:27"}, 16 / 9),
({"width": 640, "height": 480, "sample_aspect_ratio": "0:1"}, 4 / 3),
# A phone video stored landscape, shown upright.
({"width": 1280, "height": 720, "side_data_list": [{"rotation": -90}]}, 9 / 16),
({"width": 1280, "height": 720, "tags": {"rotate": "270"}}, 9 / 16),
({"width": 1280, "height": 720, "side_data_list": [{"rotation": 180}]}, 16 / 9),
({"width": None, "height": 720}, None),
],
)
def test_the_display_aspect_is_the_picture_as_a_player_shows_it(stream, dar):
got = clipmod.display_aspect(
{"streams": [{"codec_type": "audio"}, {"codec_type": "video", **stream}]}
)
assert got == pytest.approx(dar) if dar else got is None
def _crop_args(f: str) -> list[float]:
# crop=trunc(iw*W/2)*2:trunc(ih*H/2)*2:trunc(iw*X/2)*2:trunc(ih*Y/2)*2 → [W, H, X, Y]
assert f.startswith("crop=")
return [float(p.split("*")[1].split("/")[0]) for p in f[5:].split(":")]
def test_a_frame_on_a_picture_that_fills_its_box_is_cut_as_drawn():
assert _crop_args(clipmod.crop_filter(_FRAME, 16 / 9)) == pytest.approx([0.25, 0.5, 0.5, 0.25])
def test_a_frame_drawn_in_a_16_9_box_over_a_4_3_picture_maps_onto_the_picture():
# The 4:3 picture fills 3/4 of the box's width from 1/8 in: a frame over its right half's middle.
frame = {"x": 0.5, "y": 0.25, "w": 0.1875, "h": 0.5, "aspect": 16 / 9}
assert _crop_args(clipmod.crop_filter(frame, 4 / 3)) == pytest.approx([0.25, 0.5, 0.5, 0.25])
def test_the_part_of_a_frame_over_a_bar_is_dropped_and_a_frame_on_a_bar_alone_is_no_cut():
over = {"x": 0.0, "y": 0.0, "w": 0.25, "h": 0.5, "aspect": 16 / 9}
w, h, x, y = _crop_args(clipmod.crop_filter(over, 4 / 3))
assert (w, h, x, y) == pytest.approx([1 / 6, 0.5, 0, 0], abs=1e-6)
assert clipmod.crop_filter({**over, "w": 0.13}, 4 / 3) is None
# A letterboxed film: the frame over the whole box is the whole picture.
whole = {"x": 0.0, "y": 0.0, "w": 1.0, "h": 1.0, "aspect": 16 / 9}
assert clipmod.crop_filter(whole, 2.39) == ""
def test_the_crop_comes_first_so_the_tonemap_and_the_scale_work_on_what_it_keeps():
f = clipmod.crop_filter(_FRAME, 16 / 9)
vf = _cmd(hdr=True, crop=f)[_cmd(hdr=True, crop=f).index("-vf") + 1]
assert vf.startswith(f + ",") and vf.index("tonemap") < vf.index("min(1080,ih)")
assert "crop" not in _cmd()[_cmd().index("-vf") + 1]
# --- the route ---------------------------------------------------------------------------------------
@pytest.fixture
def yt_video(db):
db.add(Channel(id="UC_clip", title="The Channel"))
db.flush()
db.add(
Video(
id="abcdefghijk",
channel_id="UC_clip",
title="A Funny Video",
duration_seconds=300,
thumbnail_url="https://i.ytimg.com/vi/abcdefghijk/hq.jpg",
)
)
db.commit()
return "abcdefghijk"
@pytest.fixture
def plex_episode(db, tmp_path, monkeypatch):
lib = PlexLibrary(plex_key="2", title="Shows", kind="show", enabled=True)
db.add(lib)
db.flush()
show = PlexShow(rating_key="s9", library_id=lib.id, title="The Show")
db.add(show)
db.flush()
db.add(
PlexItem(
rating_key="777",
library_id=lib.id,
kind="episode",
title="Pilot",
show_id=show.id,
season_number=1,
episode_number=2,
duration_s=1800,
file_path="/data/shows/pilot.mkv",
)
)
db.commit()
media = tmp_path / "pilot.mkv"
media.write_bytes(b"m" * 8)
monkeypatch.setattr(
plex_paths,
"local_media_path",
lambda _db, p: media if p == "/data/shows/pilot.mkv" else None,
)
return media
def _clip(client, **body):
return client.post("/api/downloads/clip", json={"start_s": 10, "end_s": 40, **body})
def test_a_youtube_clip_is_queued_as_its_own_job_titled_after_its_video(
db, make_user, as_user, yt_video
):
r = _clip(as_user(make_user()), source="youtube", ref=yt_video)
assert r.status_code == 200, r.text
job = r.json()
assert (job["job_kind"], job["status"], job["display_name"]) == (
"clip",
"queued",
"A Funny Video (clip)",
)
assert job["edit_spec"] == {"source": "youtube", "ref": yt_video, "start_s": 10, "end_s": 40}
assert job["source_url"] == f"https://www.youtube.com/watch?v={yt_video}&t=10s"
asset = db.get(MediaAsset, job["asset"]["id"])
assert (asset.duration_s, asset.uploader, asset.media_kind) == (30, "The Channel", "video")
def test_the_same_range_again_is_the_same_clip_but_never_another_users(
db, make_user, as_user, yt_video
):
a, b = make_user(), make_user()
first = _clip(as_user(a), source="youtube", ref=yt_video).json()["asset"]["id"]
again = _clip(as_user(a), source="youtube", ref=yt_video).json()["asset"]["id"]
other = _clip(as_user(b), source="youtube", ref=yt_video).json()["asset"]["id"]
longer = _clip(as_user(a), source="youtube", ref=yt_video, end_s=41).json()["asset"]["id"]
assert first == again
assert len({first, other, longer}) == 3
def test_a_ready_clip_asked_for_again_is_done_at_once(db, make_user, as_user, yt_video):
u = make_user()
first = _clip(as_user(u), source="youtube", ref=yt_video).json()
asset = db.get(MediaAsset, first["asset"]["id"])
asset.status, asset.rel_path = "ready", ".edits/1/clip_1.mp4"
db.commit()
again = _clip(as_user(u), source="youtube", ref=yt_video).json()
assert (again["status"], again["progress"]) == ("done", 100)
@pytest.mark.parametrize(
("body", "status", "reason", "params"),
[
# The cap travels with its refusal, so the client never states a stale one.
({"end_s": 700}, 422, "too_long", {"limit_s": clipmod.MAX_CLIP_S}),
({"start_s": 299.5, "end_s": 330}, 422, "past_end", {}),
({"ref": "nope"}, 422, "bad_source", {}),
],
)
def test_a_youtube_clip_it_cannot_cut_says_why(
db, make_user, as_user, yt_video, body, status, reason, params
):
r = _clip(as_user(make_user()), **{"source": "youtube", "ref": yt_video, **body})
assert r.status_code == status
assert r.json()["detail"] == {"code": "clip", "reason": reason, **params}
assert db.query(DownloadJob).count() == 0
def test_a_plex_clip_is_titled_show_episode_title_and_keeps_its_audio_track(
db, make_user, as_user, plex_episode
):
r = _clip(as_user(make_user()), source="plex", ref="777", audio_track=1)
assert r.status_code == 200, r.text
job = r.json()
assert job["display_name"] == "The Show — S1·E2 — Pilot (clip)"
assert job["edit_spec"]["audio_track"] == 1
assert job["source_url"] is None
def test_a_plex_clip_of_an_unknown_item_is_a_404_and_of_an_unreachable_file_a_422(
db, make_user, as_user, plex_episode
):
c = as_user(make_user())
assert _clip(c, source="plex", ref="999").json()["detail"]["reason"] == "not_found"
assert _clip(c, source="plex", ref="999").status_code == 404
item = db.query(PlexItem).one()
item.file_path = "/data/elsewhere.mkv"
db.commit()
r = _clip(c, source="plex", ref="777")
assert (r.status_code, r.json()["detail"]["reason"]) == (422, "media_missing")
def test_a_clip_over_quota_is_refused_like_a_download(
db, make_user, as_user, yt_video, monkeypatch
):
def full(_db, _uid):
raise quota.QuotaExceeded("max_jobs", limit=5, current=5)
monkeypatch.setattr(quota, "check_enqueue", full)
r = _clip(as_user(make_user()), source="youtube", ref=yt_video)
assert r.status_code == 422
assert db.query(DownloadJob).count() == 0
def test_a_clip_is_not_redownloadable(db, make_user, as_user, yt_video):
c = as_user(make_user())
job = _clip(c, source="youtube", ref=yt_video).json()
row = db.get(DownloadJob, job["id"])
row.status = "done"
db.commit()
r = c.post(f"/api/downloads/{job['id']}/redownload")
assert (r.status_code, r.json()["detail"]["reason"]) == (409, "not_redownloadable")
# --- the worker ----------------------------------------------------------------------------------------
def test_a_clip_job_reaches_the_clip_entry(monkeypatch):
called = []
for name in ("_convert_upload", "_process_edit", "_download", "_process_clip"):
monkeypatch.setattr(pipeline, name, lambda *a, _n=name: called.append(_n))
runner._run(5, 7, "clip", "clip", "1:youtube:abcdefghijk", {}, 1)
assert called == ["_process_clip"]
@pytest.fixture
def worker(db, tmp_path, monkeypatch):
"""The worker path with its external programs replaced: ffmpeg writes the file it was given, the
probe answers what the test says, the phone-safe step and the poster pass through."""
monkeypatch.setattr(settings, "download_root", str(tmp_path / "root"))
seen: dict = {}
def ffmpeg(cmd, *_a, **_k):
seen["cmd"] = cmd
Path(cmd[-1]).write_bytes(b"clip")
monkeypatch.setattr(pipeline.editmod, "run_ffmpeg", ffmpeg)
monkeypatch.setattr(pipeline, "_ensure_phone_safe", lambda _j, p, *_a, **_k: p)
monkeypatch.setattr(pipeline.editmod, "ensure_poster", lambda *_a, **_k: None)
monkeypatch.setattr(probemod, "probe_media", lambda _p: ("h264", "aac", 1920, 1080, 30))
seen["probe"] = {"streams": [{"codec_type": "video"}, {"codec_type": "audio"}]}
monkeypatch.setattr(probemod, "probe_json", lambda _p: seen["probe"])
def run(job_json):
job = db.get(DownloadJob, job_json["id"])
asset = db.get(MediaAsset, job.asset_id)
job.status, asset.status = "running", "downloading"
db.commit()
pipeline._process_clip(job.id, asset.id)
db.expire_all()
return db.get(DownloadJob, job.id), db.get(MediaAsset, asset.id)
seen["run"] = run
return seen
def test_a_plex_clip_is_cut_from_the_library_file_with_its_audio_track(
db, make_user, as_user, plex_episode, worker
):
worker["probe"] = {
"streams": [
{"codec_type": "video", "color_transfer": "smpte2084"},
{"codec_type": "audio"},
{"codec_type": "audio"},
]
}
job = _clip(as_user(make_user()), source="plex", ref="777", audio_track=1).json()
job, asset = worker["run"](job)
assert (job.status, asset.status, asset.container, asset.height) == (
"done",
"ready",
"mp4",
1080,
)
cmd = worker["cmd"]
assert cmd[cmd.index("-i") + 1] == str(plex_episode)
assert cmd[cmd.index("-ss") + 1] == "10.000"
assert "0:a:1" in cmd and "tonemap" in cmd[cmd.index("-vf") + 1]
assert asset.rel_path.startswith(".edits/") and asset.rel_path.endswith(".mp4")
def test_a_cropped_plex_clip_cuts_the_frame_from_the_picture_and_is_its_own_clip(
db, make_user, as_user, plex_episode, worker
):
worker["probe"] = {
"streams": [
{"codec_type": "video", "width": 720, "height": 480, "sample_aspect_ratio": "32:27"}
]
}
c = as_user(make_user())
plain = _clip(c, source="plex", ref="777").json()
job = _clip(c, source="plex", ref="777", crop=_FRAME).json()
assert job["asset"]["id"] != plain["asset"]["id"]
assert job["edit_spec"]["crop"]["w"] == 0.25
job, asset = worker["run"](job)
assert (job.status, asset.status) == ("done", "ready")
vf = worker["cmd"][worker["cmd"].index("-vf") + 1]
# Stored to four decimals, the box's 16:9 is a hair off the file's: well under a pixel.
assert _crop_args(vf.split(",")[0]) == pytest.approx([0.25, 0.5, 0.5, 0.25], abs=1e-4)
def test_a_crop_frame_that_misses_the_picture_fails_and_says_so(
db, make_user, as_user, plex_episode, worker
):
# A 4:3 file, a frame drawn over its pillar bar in a 16:9 box.
worker["probe"] = {"streams": [{"codec_type": "video", "width": 640, "height": 480}]}
frame = {"x": 0.0, "y": 0.0, "w": 0.12, "h": 0.5, "aspect": 16 / 9}
job = _clip(as_user(make_user()), source="plex", ref="777", crop=frame).json()
job, asset = worker["run"](job)
assert (job.status, asset.status) == ("error", "error")
assert "crop" in job.error
assert "cmd" not in worker
def test_a_crop_frame_the_route_cannot_cut_says_why(db, make_user, as_user, yt_video):
r = _clip(as_user(make_user()), source="youtube", ref=yt_video, crop={**_FRAME, "w": 0.01})
assert r.status_code == 422
assert r.json()["detail"] == {"code": "clip", "reason": "bad_crop"}
def test_a_plex_clip_of_a_track_the_file_does_not_have_fails_and_says_so(
db, make_user, as_user, plex_episode, worker
):
job = _clip(as_user(make_user()), source="plex", ref="777", audio_track=3).json()
job, asset = worker["run"](job)
assert (job.status, asset.status) == ("error", "error")
assert "audio track" in job.error
assert "cmd" not in worker
def test_a_youtube_clip_fetches_only_its_range_and_cuts_it_from_zero(
db, make_user, as_user, yt_video, worker, monkeypatch
):
asked: dict = {}
def extract(job_id, asset_id, url, spec, staging, adapter, **kw):
asked.update(url=url, spec=spec, **kw)
f = Path(staging) / "abcdefghijk.mp4"
f.write_bytes(b"range")
return {"requested_downloads": [{"filepath": str(f)}]}
monkeypatch.setattr(pipeline, "_extract_with_fallback", extract)
job = _clip(as_user(make_user()), source="youtube", ref=yt_video).json()
job, asset = worker["run"](job)
assert (job.status, asset.status) == ("done", "ready")
assert asked["section"] == (10, 40)
assert asked["spec"]["max_height"] == 1080
assert asked["url"].endswith(yt_video)
cmd = worker["cmd"]
assert cmd[cmd.index("-ss") + 1] == "0.000" and cmd[cmd.index("-t") + 1] == "30.000"
assert cmd[cmd.index("-i") + 1].endswith("abcdefghijk.mp4")
def test_a_youtube_clip_hands_the_range_to_yt_dlp(monkeypatch, tmp_path):
seen = {}
class FakeYDL:
def __init__(self, opts):
seen.update(opts)
def __enter__(self):
return self
def __exit__(self, *_a):
return False
def extract_info(self, *_a, **_k):
return {}
monkeypatch.setattr(pipeline.yt_dlp, "YoutubeDL", FakeYDL)
monkeypatch.setattr(pipeline, "_touch_job", lambda *_a: None)
adapter = pipeline.adapters.resolve_adapter("youtube", "abcdefghijk")
pipeline._extract_with_fallback(
1,
2,
"https://www.youtube.com/watch?v=abcdefghijk",
pipeline.formats.normalize({}),
tmp_path,
adapter,
section=(10.0, 40.0),
)
ranges = list(seen["download_ranges"]({}, None))
assert [(r["start_time"], r["end_time"]) for r in ranges] == [(10.0, 40.0)]
assert seen["force_keyframes_at_cuts"] is False
def test_service_reference_url_follows_the_clip_spec(db, make_user, as_user, yt_video):
job = _clip(as_user(make_user()), source="youtube", ref=yt_video, start_s=65, end_s=90).json()
row = db.get(DownloadJob, job["id"])
assert service.reference_url(row, None) == f"https://www.youtube.com/watch?v={yt_video}&t=65s"
def test_a_youtube_clip_outside_the_catalog_takes_its_title_from_the_fetch(
db, make_user, as_user, worker, monkeypatch
):
def extract(job_id, asset_id, url, spec, staging, adapter, **kw):
f = Path(staging) / "zyxwvutsrqp.mp4"
f.write_bytes(b"range")
return {"requested_downloads": [{"filepath": str(f)}], "title": "Found On YouTube"}
monkeypatch.setattr(pipeline, "_extract_with_fallback", extract)
job = _clip(as_user(make_user()), source="youtube", ref="zyxwvutsrqp").json()
assert job["display_name"] == "Clip (clip)"
job, asset = worker["run"](job)
assert (job.status, asset.title, job.display_name) == (
"done",
"Found On YouTube (clip)",
"Found On YouTube (clip)",
)
# --- a YouTube clip goes with the user's own cookie, and its failure is diagnosed -----------------
_YT_COOKIES = "# Netscape HTTP Cookie File\n.youtube.com\tTRUE\t/\tTRUE\t0\tSID\tabc\n"
@pytest.fixture
def fernet(monkeypatch):
from cryptography.fernet import Fernet
from app import security
monkeypatch.setattr(security, "_fernet", Fernet(Fernet.generate_key()))
def _capture_extract(monkeypatch, asked: dict):
def extract(job_id, asset_id, url, spec, staging, adapter, **kw):
cookiefile = kw.get("cookiefile")
asked["cookiefile"] = cookiefile
asked["cookie_text"] = Path(cookiefile).read_text() if cookiefile else None
f = Path(staging) / "abcdefghijk.mp4"
f.write_bytes(b"range")
return {"requested_downloads": [{"filepath": str(f)}]}
monkeypatch.setattr(pipeline, "_extract_with_fallback", extract)
def test_a_youtube_clip_goes_signed_in_with_the_users_own_cookie_and_leaves_none_behind(
db, make_user, as_user, yt_video, worker, monkeypatch, fernet
):
from app.downloads import credentials
user = make_user()
credentials.upsert_credential(db, user.id, "youtube", _YT_COOKIES)
db.commit()
asked: dict = {}
_capture_extract(monkeypatch, asked)
job, _asset = worker["run"](_clip(as_user(user), source="youtube", ref=yt_video).json())
assert job.status == "done"
assert asked["cookie_text"] == _YT_COOKIES
assert not Path(asked["cookiefile"]).exists()
def test_a_youtube_clip_never_goes_with_another_users_cookie(
db, make_user, as_user, yt_video, worker, monkeypatch, fernet
):
from app.downloads import credentials
other = make_user()
credentials.upsert_credential(db, other.id, "youtube", _YT_COOKIES)
db.commit()
asked: dict = {}
_capture_extract(monkeypatch, asked)
job, _asset = worker["run"](_clip(as_user(make_user()), source="youtube", ref=yt_video).json())
assert job.status == "done" and asked["cookiefile"] is None
@pytest.mark.parametrize(
("with_cookie", "detail"), [(False, "youtube.bot_wall"), (True, "youtube.cookie_rejected")]
)
def test_a_youtube_clip_on_the_bot_wall_is_diagnosed_like_a_download(
db, make_user, as_user, yt_video, worker, monkeypatch, fernet, with_cookie, detail
):
from app.downloads import credentials
user = make_user()
if with_cookie:
credentials.upsert_credential(db, user.id, "youtube", _YT_COOKIES)
db.commit()
def walled(*_a, **_k):
raise RuntimeError(f"ERROR: [youtube] {yt_video}: Sign in to confirm you’re not a bot.")
monkeypatch.setattr(pipeline, "_extract_with_fallback", walled)
job, asset = worker["run"](_clip(as_user(user), source="youtube", ref=yt_video).json())
assert (job.status, asset.status) == ("error", "error")
assert (job.error_code, job.error_detail) == ("login_required", detail)
# --- held to the owner's storage cap, and a clip that cannot or need not go on ---------------------
def _fetch_range(monkeypatch):
def extract(job_id, asset_id, url, spec, staging, adapter, **kw):
f = Path(staging) / "abcdefghijk.mp4"
f.write_bytes(b"range")
return {"requested_downloads": [{"filepath": str(f)}]}
monkeypatch.setattr(pipeline, "_extract_with_fallback", extract)
@pytest.mark.parametrize("source", ["youtube", "plex"])
def test_a_clip_for_an_owner_whose_storage_is_full_is_refused_before_it_is_cut(
db, make_user, as_user, yt_video, plex_episode, worker, monkeypatch, source
):
_fetch_range(monkeypatch)
ref = yt_video if source == "youtube" else "777"
job = _clip(as_user(make_user()), source=source, ref=ref).json()
monkeypatch.setattr(quota, "remaining_bytes", lambda _db, _uid: 0)
job, asset = worker["run"](job)
# Refused like a download at its cap: the job says why, the asset waits for a Retry.
assert (job.status, job.error, asset.status) == (
"error",
dlerrors.QUOTA_FULL_MESSAGE,
"pending",
)
assert "cmd" not in worker
@pytest.mark.parametrize("source", ["youtube", "plex"])
def test_a_clip_larger_than_what_its_owner_has_left_is_refused_and_not_placed(
db, make_user, as_user, yt_video, plex_episode, worker, monkeypatch, source
):
# The range fetch's own ceiling never reaches yt-dlp's ffmpeg downloader, and a Plex clip
# fetches nothing: the cut file itself is held to the cap.
_fetch_range(monkeypatch)
ref = yt_video if source == "youtube" else "777"
job = _clip(as_user(make_user()), source=source, ref=ref).json()
monkeypatch.setattr(quota, "remaining_bytes", lambda _db, _uid: 3) # the cut is 4 bytes
job, asset = worker["run"](job)
# Not "full": there is room left, just not this much.
assert (job.status, job.error, asset.status, asset.rel_path) == (
"error",
dlerrors.CLIP_TOO_BIG_MESSAGE,
"pending",
None,
)
def test_a_youtube_clip_range_yt_dlp_refuses_for_its_cap_says_it_is_too_big(
db, make_user, as_user, yt_video, worker, monkeypatch
):
# Should yt-dlp's own `max_filesize` refuse the range, it returns with no file (test_formats):
# the owner's storage, not a processing fault, and nothing is cut.
def refused(*_a, max_bytes=None, **_k):
formats._YDL_LOG.debug(
f"[download] File is larger than max-filesize (9000 bytes > {max_bytes} bytes). Aborting."
)
return {"requested_downloads": [{}]}
monkeypatch.setattr(pipeline, "_extract_with_fallback", refused)
job = _clip(as_user(make_user()), source="youtube", ref=yt_video).json()
monkeypatch.setattr(quota, "remaining_bytes", lambda _db, _uid: 500)
job, asset = worker["run"](job)
assert (job.status, job.error_code, job.error, asset.status) == (
"error",
"quota_exceeded",
dlerrors.CLIP_TOO_BIG_MESSAGE,
"pending",
)
assert "cmd" not in worker
def test_a_plex_clip_of_a_silent_file_is_cut_silent(db, make_user, as_user, plex_episode, worker):
# The player names the track it plays 0 even when the file has none.
worker["probe"] = {"streams": [{"codec_type": "video"}]}
job = _clip(as_user(make_user()), source="plex", ref="777", audio_track=0).json()
job, asset = worker["run"](job)
assert (job.status, asset.status) == ("done", "ready")
assert "0:a:0?" in worker["cmd"]
def test_a_plex_clip_of_a_file_the_probe_cannot_read_says_so(
db, make_user, as_user, plex_episode, worker
):
worker["probe"] = None
job = _clip(as_user(make_user()), source="plex", ref="777", audio_track=0).json()
job, asset = worker["run"](job)
assert (job.status, asset.status) == ("error", "error")
assert job.error == "The source file could not be read."
assert "cmd" not in worker
@pytest.mark.parametrize("where", ["fetch", "encode"])
def test_a_paused_clip_frees_its_asset_for_a_later_retry(
db, make_user, as_user, yt_video, worker, monkeypatch, where
):
def stopped(*_a, **_k):
raise pipeline._Aborted() if where == "fetch" else pipeline.editmod.EditAborted()
if where == "fetch":
monkeypatch.setattr(pipeline, "_extract_with_fallback", stopped)
else:
_fetch_range(monkeypatch)
monkeypatch.setattr(pipeline.editmod, "run_ffmpeg", stopped)
job = _clip(as_user(make_user()), source="youtube", ref=yt_video).json()
job, asset = worker["run"](job)
# The route already wrote the job's own state (paused / canceled); the asset is free again.
assert (job.status, job.error, asset.status) == ("running", None, "pending")
# --- a second job on the same clip -----------------------------------------------------------------
def _second_job(db, client, **body):
"""Two jobs for one range: the second shares the first's asset."""
first = _clip(client, **body).json()
second = _clip(client, **body).json()
assert first["asset"]["id"] == second["asset"]["id"]
return db.get(DownloadJob, first["id"]), db.get(DownloadJob, second["id"])
def test_a_second_clip_job_on_a_failed_clip_gets_the_same_youtube_diagnosis(
db, make_user, as_user, yt_video
):
_first, second = _second_job(db, as_user(make_user()), source="youtube", ref=yt_video)
asset = db.get(MediaAsset, second.asset_id)
asset.status = "error"
asset.error = f"ERROR: [youtube] {yt_video}: Sign in to confirm you’re not a bot."
second.status = "running"
db.commit()
workermod._process_job(second.id)
db.expire_all()
job = db.get(DownloadJob, second.id)
assert (job.status, job.error_code, job.error_detail) == (
"error",
"login_required",
"youtube.bot_wall",
)
def test_a_second_clip_job_takes_the_title_its_sibling_found(db, make_user, as_user):
_first, second = _second_job(db, as_user(make_user()), source="youtube", ref="zyxwvutsrqp")
assert second.display_name == "Clip (clip)"
asset = db.get(MediaAsset, second.asset_id)
asset.status, asset.title = "ready", "Found On YouTube (clip)"
second.status = "running"
db.commit()
workermod._finish_from_cache(second.id, asset.id)
db.expire_all()
job = db.get(DownloadJob, second.id)
assert (job.status, job.display_name) == ("done", "Found On YouTube (clip)")
+7 -7
View File
@@ -8,6 +8,7 @@ import pytest
from app.config import settings
from app.downloads import edit as editmod
from app.downloads import playable
from app.downloads import runner
@@ -32,11 +33,12 @@ class TestEncodeThreadCap:
assert editmod.encode_thread_cap() == 1 # max(1, 2 - 1)
class TestBrowserSafeThreads:
_REENCODE_VIDEO = playable.Plan("mp4", True, True, False, True)
class TestPhoneSafeThreads:
def test_threads_flag_added_when_capped(self):
cmd = editmod.build_browser_safe_cmd(
"/in.mkv", "/out.mp4", reencode_video=True, reencode_audio=False, threads=2
)
cmd = playable.build_cmd("/in.mkv", "/out.mp4", _REENCODE_VIDEO, threads=2)
assert "-threads" in cmd
assert cmd[cmd.index("-threads") + 1] == "2"
# The cap precedes the output file, so it binds the encoder (not input decoding).
@@ -44,9 +46,7 @@ class TestBrowserSafeThreads:
@pytest.mark.parametrize("threads", [0, None])
def test_no_threads_flag_when_uncapped(self, threads):
cmd = editmod.build_browser_safe_cmd(
"/in.mkv", "/out.mp4", reencode_video=True, reencode_audio=False, threads=threads
)
cmd = playable.build_cmd("/in.mkv", "/out.mp4", _REENCODE_VIDEO, threads=threads)
assert "-threads" not in cmd
+3
View File
@@ -63,6 +63,9 @@ def test_our_own_worker_refusals_classify_as_themselves():
# fragment of the text. Pinning message and rule together is what stops an edit to the wording
# from silently orphaning the category — the UI would then fall back to "Download failed".
assert classify(dlerrors.QUOTA_FULL_MESSAGE) == "quota_exceeded"
assert classify(dlerrors.CLIP_TOO_BIG_MESSAGE) == "quota_exceeded"
assert classify(dlerrors.CONVERTED_TOO_BIG_MESSAGE) == "quota_exceeded"
assert classify(dlerrors.DOWNLOAD_TOO_BIG_MESSAGE) == "quota_exceeded"
def test_the_redacted_network_text_classifies_back_to_network():
+115 -1
View File
@@ -11,7 +11,8 @@ its OWN session, so every test COMMITS its setup before calling it.
"""
from app.config import settings
from app.downloads import pipeline, quota
from app.downloads import errors as dlerrors
from app.downloads import formats, pipeline, quota
from app.models import DownloadJob, DownloadQuota, MediaAsset, User
from app.worker import _claim_job
@@ -155,6 +156,25 @@ def test_unlimited_user_is_never_excluded(db):
assert _claim_job() == q.id
def test_a_suspended_users_queued_jobs_wait(db):
# Suspending a user blocks their downloads (quota.resolve's advice): a job they queued before
# stays queued, unclaimed, and runs once the suspension is lifted.
s = User(email="suspended@example.com", is_suspended=True)
o = User(email="other@example.com")
db.add_all([s, o])
db.flush()
waits = _job(db, s.id, queue_pos=1)
served = _job(db, o.id, queue_pos=2)
db.commit()
assert _claim_job() == served.id
assert _claim_job() is None
db.expire_all()
assert db.get(DownloadJob, waits.id).status == "queued"
db.get(User, s.id).is_suspended = False
db.commit()
assert _claim_job() == waits.id
# --- #3 · list cap --------------------------------------------------------------------------
@@ -278,3 +298,97 @@ def test_a_refusal_for_one_user_does_not_poison_the_shared_asset(db, monkeypatch
assert db.get(MediaAsset, shared.id).status == "pending"
assert db.get(MediaAsset, shared.id).error is None
assert db.get(DownloadJob, theirs.id).status == "queued"
def test_a_download_over_the_space_left_says_so_and_releases_the_shared_asset(
db, monkeypatch, tmp_path
):
# yt-dlp refuses a file over `max_filesize` by saying so and returning with no file (pinned in
# test_formats). That is the owner's storage, not a processing fault: their refusal, in words
# true of a cap with space still left, and the shared asset goes back for the next job.
monkeypatch.setattr(settings, "download_root", str(tmp_path))
u = User(email="nearcap@example.com")
db.add(u)
db.flush()
db.add(
DownloadQuota(user_id=u.id, max_bytes=1000, max_concurrent=2, max_jobs=100, unlimited=False)
)
held = _sized_asset(db, 400, "near-held")
_job(db, u.id, queue_pos=0, status="done", asset_id=held.id)
shared = MediaAsset(
source_kind="youtube",
source_ref="y" * 11,
format_sig="s",
status="downloading",
ref_count=1,
)
db.add(shared)
db.flush()
mine = _job(db, u.id, queue_pos=1, status="running", asset_id=shared.id)
db.commit()
caps = []
def refuses_over_the_cap(*args, max_bytes=None, **kwargs):
caps.append(max_bytes)
formats._YDL_LOG.debug(
f"[download] File is larger than max-filesize (2000 bytes > {max_bytes} bytes). Aborting."
)
return {"id": "y" * 11, "requested_downloads": [{}]}
monkeypatch.setattr(pipeline, "_extract_with_fallback", refuses_over_the_cap)
pipeline._download(mine.id, shared.id, "youtube", "y" * 11, {}, u.id)
db.expire_all()
job = db.get(DownloadJob, mine.id)
assert caps == [600]
assert (job.status, job.error_code) == ("error", "quota_exceeded")
assert job.error == dlerrors.DOWNLOAD_TOO_BIG_MESSAGE
assert db.get(MediaAsset, shared.id).status == "pending"
def test_a_download_that_outgrows_the_space_left_unannounced_is_refused_and_not_placed(
db, monkeypatch, tmp_path
):
# yt-dlp's `max_filesize` only stops a plain HTTP file that announces its size: YouTube's
# fragmented formats never meet it (measured on localdev: 718 MB fetched with 1 MB left). The
# file it produced is held to what is left by its real size, as a clip is.
monkeypatch.setattr(settings, "download_root", str(tmp_path))
u = User(email="unannounced@example.com")
db.add(u)
db.flush()
db.add(
DownloadQuota(user_id=u.id, max_bytes=1000, max_concurrent=2, max_jobs=100, unlimited=False)
)
held = _sized_asset(db, 400, "unannounced-held")
_job(db, u.id, queue_pos=0, status="done", asset_id=held.id)
shared = MediaAsset(
source_kind="youtube",
source_ref="z" * 11,
format_sig="s",
status="downloading",
ref_count=1,
)
db.add(shared)
db.flush()
mine = _job(db, u.id, queue_pos=1, status="running", asset_id=shared.id)
db.commit()
def fetched_past_the_cap(job_id, asset_id, url, spec, staging, adapter, **kw):
f = staging / "zzzzzzzzzzz.mp4"
f.write_bytes(b"v" * 2000) # 600 left
return {"id": "z" * 11, "requested_downloads": [{"filepath": str(f)}]}
monkeypatch.setattr(pipeline, "_extract_with_fallback", fetched_past_the_cap)
monkeypatch.setattr(pipeline, "_ensure_phone_safe", lambda _j, produced, *_a: produced)
pipeline._download(mine.id, shared.id, "youtube", "z" * 11, {}, u.id)
db.expire_all()
job = db.get(DownloadJob, mine.id)
assert (job.status, job.error_code, job.error) == (
"error",
"quota_exceeded",
dlerrors.DOWNLOAD_TOO_BIG_MESSAGE,
)
asset = db.get(MediaAsset, shared.id)
assert (asset.status, asset.rel_path) == ("pending", None)
assert not any(p.suffix == ".mp4" for p in tmp_path.rglob("*") if "staging" not in p.parts)
+4 -41
View File
@@ -6,7 +6,7 @@ with "PosixPath('.') has an empty name" instead of downloading."""
import pytest
from app.downloads.edit import _codecs_from_streams, _concat_quote, run_ffprobe_json
from app.downloads.edit import _concat_quote, run_ffprobe_json
from app.downloads.pipeline import _media_info, _produced_file
from pathlib import Path, PurePosixPath
@@ -105,46 +105,9 @@ class TestMediaInfo:
assert _media_info(info)["id"] == "vid"
class TestCodecsFromStreams:
# ffprobe emits codec_name BEFORE codec_type, so the old line parser shifted every name onto
# the previous stream and an H.264+AAC mp4 came back as (aac, None) → a needless re-encode.
# _codecs_from_streams reads each stream dict by its own codec_type, immune to that ordering.
def test_reads_each_stream_by_its_own_type(self):
streams = [
{"codec_name": "h264", "codec_type": "video"},
{"codec_name": "aac", "codec_type": "audio"},
]
assert _codecs_from_streams(streams) == ("h264", "aac")
def test_audio_first_ordering_is_handled(self):
streams = [
{"codec_name": "AAC", "codec_type": "audio"},
{"codec_name": "H264", "codec_type": "video"},
]
assert _codecs_from_streams(streams) == ("h264", "aac")
def test_extra_streams_do_not_win_over_the_first_of_their_kind(self):
streams = [
{"codec_name": "h264", "codec_type": "video"},
{"codec_name": "vp9", "codec_type": "video"},
{"codec_name": "mov_text", "codec_type": "subtitle"},
{"codec_name": "opus", "codec_type": "audio"},
]
assert _codecs_from_streams(streams) == ("h264", "opus")
def test_audio_only_and_video_only(self):
assert _codecs_from_streams([{"codec_name": "opus", "codec_type": "audio"}]) == (
None,
"opus",
)
assert _codecs_from_streams([{"codec_name": "av1", "codec_type": "video"}]) == ("av1", None)
def test_empty_stream_list(self):
assert _codecs_from_streams([]) == (None, None)
class TestRunFfprobeJson:
"""The single ffprobe-JSON invocation point (shared by probe_codecs + probe.probe_media). A failed
"""The single ffprobe-JSON invocation point (`probe.probe_json`, which the stored facts and the
phone-safe rule both read). A failed
probe or unparseable output must degrade to None, not raise — the malformed-input safety the old
parse_probe_streams string parser used to own now lives here."""
@@ -174,7 +137,7 @@ class TestRunFfprobeJson:
monkeypatch.setattr(subprocess, "run", lambda *a, **k: _R())
data = run_ffprobe_json(Path("x"), entries="stream=codec_type,codec_name")
assert _codecs_from_streams(data["streams"]) == ("h264", None)
assert data["streams"] == [{"codec_name": "h264", "codec_type": "video"}]
class TestConcatQuote:
+63
View File
@@ -0,0 +1,63 @@
"""An editor clip is held to its owner's storage cap by its real size, as a download and a player
clip are: its file is new storage on the owner's footprint, and nothing else checked it.
Real ffmpeg — the size the check reads is the encoder's.
"""
import shutil
import subprocess
import pytest
from app.config import settings
from app.downloads import errors as dlerrors
from app.downloads import pipeline, quota, service
from app.downloads import upload as uploadmod
from app.models import DownloadJob, DownloadQuota, MediaAsset
pytestmark = pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not available")
def test_an_editor_clip_larger_than_what_is_left_is_refused_and_not_placed(
db, make_user, tmp_path, monkeypatch
):
monkeypatch.setattr(settings, "download_root", str(tmp_path))
src = tmp_path / "in.mp4"
subprocess.run(
[
"ffmpeg", "-y", "-loglevel", "error",
"-f", "lavfi", "-i", "testsrc2=size=160x120:rate=10:duration=2",
"-c:v", "libx264", "-pix_fmt", "yuv420p", str(src),
],
check=True,
) # fmt: skip
u = make_user()
source = uploadmod.ingest_upload(
db, u.id, filename="clip.mp4", src_path=src, size=src.stat().st_size
)
job = service.enqueue_edit(db, u.id, source, {"trim": {"start_s": 0, "end_s": 1.5}})
asset = db.get(MediaAsset, job.asset_id)
job.status, asset.status = "running", "downloading" # the worker's claim
# Ten bytes left: no clip of a real video fits.
db.add(
DownloadQuota(
user_id=u.id,
max_bytes=quota.footprint(db, u.id) + 10,
max_concurrent=2,
max_jobs=100,
unlimited=False,
)
)
db.commit()
pipeline._process_edit(job.id, asset.id)
db.expire_all()
job, asset = db.get(DownloadJob, job.id), db.get(MediaAsset, job.asset_id)
assert (job.status, job.error_code, job.error) == (
"error",
"quota_exceeded",
dlerrors.CLIP_TOO_BIG_MESSAGE,
)
assert asset.rel_path is None
assert not list(tmp_path.glob(".edits/**/*.mp4"))
+198
View File
@@ -0,0 +1,198 @@
"""Every ffmpeg/ffprobe the downloads package runs over a library file names the demuxers it may use
(`edit.INPUT_GUARD`, a downloaded thumbnail's `edit.THUMB_GUARD`).
Without the list, ffmpeg picks a demuxer from the file's content and name, and both are the user's
for an upload (the remote site's for a thumbnail): a playlist-type demuxer opens whatever local paths
its entries name. These tests read the argv each builder hands to ffmpeg — the guard is a flag, so
the flag is what is held — and scan the package for an ffmpeg/ffprobe call written without one.
Not in scope: `app/plex`, whose ffmpeg reads the admin's own Plex library, not user-supplied bytes.
"""
import ast
import shutil
import subprocess
from pathlib import Path
import pytest
from app.downloads import edit, playable
DOWNLOADS = Path(__file__).resolve().parents[1] / "app" / "downloads"
# The demuxers that read paths named inside a file. `concat` is allowed only where the list is ours.
PLAYLIST_DEMUXERS = {"hls", "dash", "concat", "image2", "ffconcat", "tee"}
MEDIA = set(edit.MEDIA_DEMUXERS)
def _whitelists(cmd: list[str]) -> list[set[str] | None]:
"""For each `-i` in `cmd`, the demuxers the `-format_whitelist` given since the previous input
allows — None when that input has none."""
out, current = [], None
for i, tok in enumerate(cmd):
if tok == "-format_whitelist":
current = set(cmd[i + 1].split(","))
elif tok == "-i":
out.append(current)
current = None
return out
def _assert_media_guarded(cmd: list[str], extra: set[str] = frozenset()) -> None:
lists = _whitelists(cmd)
assert lists, cmd # a builder that opens no input is not what this test is about
for allowed in lists:
assert allowed is not None, f"an input with no -format_whitelist: {cmd}"
assert allowed <= MEDIA | extra, allowed - MEDIA - extra
assert not (allowed - extra) & PLAYLIST_DEMUXERS
def test_the_media_list_names_no_demuxer_that_reads_other_paths():
assert not MEDIA & PLAYLIST_DEMUXERS
assert edit.INPUT_GUARD == ["-format_whitelist", ",".join(edit.MEDIA_DEMUXERS)]
def test_the_phone_safe_conversion_is_guarded():
p = playable.Plan("mp4", True, True, True, True)
_assert_media_guarded(playable.build_cmd(Path("in.mov"), Path("out.mp4"), p))
@pytest.mark.parametrize(
"spec",
[
{"trim": {"start_s": 1, "end_s": 3}, "accurate": False},
{"trim": {"start_s": 1, "end_s": 3}, "accurate": True},
{"crop": {"x": 0, "y": 0, "w": 101, "h": 51}},
],
ids=["fast trim", "accurate trim", "crop"],
)
def test_a_clip_is_guarded(spec):
spec = edit.normalize_edit_spec(spec)
_assert_media_guarded(edit.build_edit_cmd(Path("in.mp4"), Path("out.mp4"), spec, "mp4"))
@pytest.mark.parametrize("accurate", [True, False], ids=["accurate join", "fast join"])
def test_a_join_is_guarded(accurate, tmp_path):
segs = [{"start_s": 0, "end_s": 1}, {"start_s": 2, "end_s": 3}]
spec = edit.normalize_edit_spec({"segments": segs, "accurate": accurate})
cmd, _ = edit.build_concat_plan(Path("in.mp4"), Path("out.mp4"), spec, "mp4", tmp_path)
# The fast join reads a concat list WE write, so the list itself may be opened as concat.
_assert_media_guarded(cmd, extra=set() if accurate else {"concat"})
def test_the_fast_join_list_gives_each_entry_the_media_list_without_concat(tmp_path):
# ffmpeg hands the list's whitelist (with `concat`) down to the library file each entry names;
# the entry's own `option format_whitelist` must replace it, or that file may open as a list.
segs = [{"start_s": 0, "end_s": 1}, {"start_s": 2, "end_s": 3}]
spec = edit.normalize_edit_spec({"segments": segs, "accurate": False})
_, prep = edit.build_concat_plan(Path("in.mp4"), Path("out.mp4"), spec, "mp4", tmp_path)
(text,) = prep.values()
lines = text.splitlines()
entries = [i for i, ln in enumerate(lines) if ln.startswith("file ")]
assert len(entries) == 2
for i in entries:
key, _, allowed = lines[i + 1].partition(" format_whitelist ")
assert key == "option", lines[i + 1]
assert set(allowed.split(",")) == MEDIA
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not available")
def test_the_fast_join_still_joins_a_real_file(tmp_path):
src = tmp_path / "in.mp4"
subprocess.run(
["ffmpeg", "-y", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc=size=64x48:rate=10",
"-f", "lavfi", "-i", "sine=frequency=440", "-t", "4", "-c:v", "libx264",
"-pix_fmt", "yuv420p", "-c:a", "aac", str(src)],
check=True,
) # fmt: skip
segs = [{"start_s": 0, "end_s": 1}, {"start_s": 2, "end_s": 3}]
spec = edit.normalize_edit_spec({"segments": segs, "accurate": False})
out = tmp_path / "out.mp4"
cmd, prep = edit.build_concat_plan(src, out, spec, "mp4", tmp_path)
for p, content in prep.items():
p.write_text(content, encoding="utf-8")
subprocess.run(cmd, check=True, capture_output=True)
assert (edit.probe_duration(out) or 0) > 0
def test_the_poster_and_the_filmstrip_are_guarded():
_assert_media_guarded(edit.build_poster_cmd(Path("in.mp4"), Path("p.jpg"), 2.0))
geom = edit.storyboard_geometry(60)
_assert_media_guarded(edit.build_storyboard_cmd(Path("in.mp4"), Path("s.jpg"), 60, geom))
def test_a_downloaded_thumbnail_opens_as_a_single_image_only():
(allowed,) = _whitelists(edit.build_thumbnail_jpg_cmd(Path("t.webp"), Path("t.jpg")))
assert allowed == {"png_pipe", "jpeg_pipe", "webp_pipe"}
def test_every_ffprobe_is_guarded(monkeypatch):
seen = []
class _Done:
stdout = "{}"
def _run(args, **_k):
seen.append(args)
return _Done()
monkeypatch.setattr(edit.subprocess, "run", _run)
edit.run_ffprobe_json(Path("x.mp4"), show_streams=True)
edit.probe_duration(Path("x.mp4"))
assert len(seen) == 2
for args in seen:
i = args.index("-format_whitelist")
assert set(args[i + 1].split(",")) == MEDIA
def _unguarded_calls(source: str) -> list[int]:
"""Lines of a list literal that starts an ffmpeg/ffprobe command or hands one an input (`-i`)
and carries no `*…GUARD` / `*guard` splat and no "-format_whitelist" of its own. A builder that
adds its input in a later `+=` must guard that list — that is where the input is."""
bad = []
for node in ast.walk(ast.parse(source)):
if not isinstance(node, ast.List):
continue
consts = {e.value for e in node.elts if isinstance(e, ast.Constant)}
if not consts & {"-i", "ffprobe"}:
continue
guarded = "-format_whitelist" in consts or any(
isinstance(e, ast.Starred)
and isinstance(v := e.value, ast.Name | ast.Attribute)
and (v.id if isinstance(v, ast.Name) else v.attr).upper().endswith("GUARD")
for e in node.elts
)
if not guarded:
bad.append(node.lineno)
return bad
def test_no_ffmpeg_call_in_the_package_opens_an_input_unguarded():
found = {
f"{p.name}:{line}"
for p in sorted(DOWNLOADS.rglob("*.py"))
for line in _unguarded_calls(p.read_text(encoding="utf-8"))
}
assert found == set()
def test_the_scan_sees_an_unguarded_call():
# The scan above is green over nothing if it cannot see one: both shapes it must catch.
assert _unguarded_calls('cmd = ["ffmpeg", "-y", "-i", src]') == [1]
assert _unguarded_calls('a = ["ffprobe", "-v", "error", str(p)]') == [1]
assert _unguarded_calls('cmd = [*INPUT_GUARD, "-i", src]') == []
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not available")
@pytest.mark.parametrize("ext", ["png", "webp", "jpg"])
def test_the_thumbnail_guard_still_opens_every_thumbnail_a_download_keeps(ext, tmp_path):
# `pipeline._THUMB_SUFFIXES`: what the guard must still let through (a .jpg that is really a
# PNG goes the ffmpeg route too — `thumbnail_to_jpg` copies only true JPEG bytes).
src = tmp_path / f"t.{ext}"
fmt = ["-f", "image2", "-c:v", "png"] if ext == "jpg" else []
subprocess.run(
["ffmpeg", "-y", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc2=size=64x48",
"-frames:v", "1", *fmt, str(src)],
check=True,
) # fmt: skip
out = edit.thumbnail_to_jpg(src, tmp_path / "poster.jpg")
assert out is not None and edit._is_jpeg(out)
+44 -17
View File
@@ -3,6 +3,13 @@ playability guarantee) and must SELECT H.264 (not merely prefer it), so shared d
every browser incl. iOS/Safari. Non-mp4 containers and non-H.264 vcodecs are coerced, not honoured.
See the iPad AV1 broken-player fix + the format narrowing."""
import contextlib
import functools
import http.server
import threading
import yt_dlp
from app.downloads import edit as editmod
from app.downloads import formats
@@ -94,23 +101,6 @@ def test_audio_only_unaffected():
assert _opts({"mode": "a", "audio_format": "m4a"})["format"] == "bestaudio/best"
def test_browser_transcode_flags_universal_playability():
F = editmod.browser_transcode_flags
# Already universal: H.264 + AAC in mp4 → no change.
assert F("h264", "aac", True) == (False, False, False)
assert F("h264", "mp3", True) == (False, False, False)
# Non-H.264 video → full video re-encode (AV1/VP9 fail iOS; HEVC fails Chrome/FF).
assert F("av1", "aac", True) == (True, True, False)
assert F("vp9", "opus", True) == (True, True, True)
assert F("hevc", "aac", True) == (True, True, False)
# H.264 but bad audio → audio-only transcode (video copied).
assert F("h264", "opus", True) == (True, False, True)
# Codecs fine but wrong container → remux only (no re-encode).
assert F("h264", "aac", False) == (True, False, False)
# A failed probe / missing stream is never re-encoded.
assert F(None, None, True) == (False, False, False)
# --- SEC-13 · one enqueue is one bounded download -------------------------------------------
@@ -179,3 +169,40 @@ def test_the_mp3_fix_moved_the_audio_cache_identity(monkeypatch):
current = formats.format_sig(audio)
monkeypatch.setattr(formats, "_AUDIO_SIG_VERSION", 4)
assert formats.format_sig(audio) != current
class _QuietFiles(http.server.SimpleHTTPRequestHandler):
def log_message(self, *args): # the test's output is yt-dlp's, not the server's
pass
def handle(self):
# yt-dlp hangs up on a file over the cap as soon as it reads its size.
with contextlib.suppress(ConnectionError):
super().handle()
def test_yt_dlp_refuses_a_file_over_the_cap_without_raising_and_the_watch_sees_it(tmp_path):
# The boundary `watching_size_cap` reads, held against the yt-dlp this image pins: over
# `max_filesize` its HTTP downloader logs one line and RETURNS — no exception, no file — so
# without the watch the pipeline would only find "no output file" and call it a processing fault.
served = tmp_path / "served"
served.mkdir()
(served / "v.mp4").write_bytes(b"\0" * 200_000)
srv = http.server.ThreadingHTTPServer(
("127.0.0.1", 0), functools.partial(_QuietFiles, directory=str(served))
)
threading.Thread(target=srv.serve_forever, daemon=True).start()
url = f"http://127.0.0.1:{srv.server_port}/v.mp4"
try:
for cap, refused in ((1_000, True), (10_000_000, False)):
out = tmp_path / f"out-{cap}"
opts = formats.build_ydl_opts(
{}, str(out / "%(id)s.%(ext)s"), lambda d: None, max_bytes=cap
)
with formats.watching_size_cap() as over, yt_dlp.YoutubeDL(opts) as ydl:
ydl.extract_info(url, download=True)
assert over[0] is refused, cap
files = [p for p in out.glob("*") if p.suffix == ".mp4"] if out.exists() else []
assert bool(files) is not refused, cap
finally:
srv.shutdown()
+135
View File
@@ -0,0 +1,135 @@
"""The library player's resume point (`library_states`).
A checkpoint stores where the caller stopped in a library item they can play — their own download,
or one shared with them — and the two library listings hand it back per item, so the player opens
there. Too early or too close to the end clears it. Nobody else's checkpoint, and no item the caller
cannot reach, is ever read or written. DB-backed (real Postgres).
"""
import pytest
from sqlalchemy import select
from app.models import DownloadJob, DownloadShare, LibraryState
from app.routes._common import ProgressIn, kept_position
def _done_job(db, user) -> DownloadJob:
job = DownloadJob(
user_id=user.id,
source_kind="url",
source_ref="https://example.com/a.mp4",
status="done",
profile_snapshot={},
)
db.add(job)
db.commit()
return job
def _checkpoint(client, job_id: int, position: float, duration: float = 3000) -> int:
"""Send a checkpoint the caller may make; the point the server kept (0 = cleared)."""
r = client.post(
f"/api/downloads/{job_id}/progress",
json={"position_seconds": position, "duration_seconds": duration},
)
assert r.status_code == 200, r.text
return r.json()["position_seconds"]
def _states(db) -> list[tuple[int, int, int]]:
db.expire_all()
rows = db.execute(
select(LibraryState.user_id, LibraryState.job_id, LibraryState.position_seconds)
).all()
return [tuple(r) for r in rows]
def test_a_checkpoint_is_listed_back_on_the_item(db, make_user, as_user):
me = make_user()
job = _done_job(db, me)
c = as_user(me)
r = c.post(
f"/api/downloads/{job.id}/progress",
json={"position_seconds": 754.9, "duration_seconds": 3000},
)
assert r.status_code == 200 and r.json() == {
"ok": True,
"job_id": job.id,
"position_seconds": 754,
}
[item] = c.get("/api/downloads").json()
assert item["position_seconds"] == 754
# A second checkpoint moves it rather than adding a row.
assert _checkpoint(c, job.id, 900) == 900
assert _states(db) == [(me.id, job.id, 900)]
def test_an_item_never_played_lists_zero(db, make_user, as_user):
me = make_user()
_done_job(db, me)
[item] = as_user(me).get("/api/downloads").json()
assert item["position_seconds"] == 0
def test_too_early_or_near_the_end_clears_it(db, make_user, as_user):
me = make_user()
job = _done_job(db, me)
c = as_user(me)
assert _checkpoint(c, job.id, 600) == 600
assert _checkpoint(c, job.id, 2995) == 0
assert _states(db) == []
assert _checkpoint(c, job.id, 600) == 600
assert _checkpoint(c, job.id, 3) == 0
assert _states(db) == []
# Clearing what is not there is not an error (the beacon and the last checkpoint of one stop).
assert _checkpoint(c, job.id, 0) == 0
# The edges themselves, one second either side — the same rows as the client's
# frontend/src/lib/resume.dom.test.ts: the rule must be applied alike on both sides, and the shared
# constants alone (resume.contract.test.ts) would not see a `>=` that became `>` on one side.
@pytest.mark.parametrize(
("position", "duration", "kept"),
[(4, 3000, 0), (5, 3000, 5), (2990, 3000, 2990), (2991, 3000, 0)],
)
def test_the_kept_point_at_its_exact_edges(position, duration, kept):
body = ProgressIn(position_seconds=position, duration_seconds=duration)
assert kept_position(body) == kept
def test_a_recipient_keeps_their_own_place_in_a_shared_item(db, make_user, as_user):
owner, friend = make_user(), make_user()
job = _done_job(db, owner)
db.add(DownloadShare(job_id=job.id, from_user_id=owner.id, to_user_id=friend.id))
db.commit()
assert _checkpoint(as_user(owner), job.id, 100) == 100
assert _checkpoint(as_user(friend), job.id, 2000) == 2000
assert as_user(owner).get("/api/downloads").json()[0]["position_seconds"] == 100
assert as_user(friend).get("/api/downloads/shared").json()[0]["position_seconds"] == 2000
def test_an_item_the_caller_cannot_play_is_a_404_and_writes_nothing(db, make_user, as_user):
owner, stranger = make_user(), make_user()
job = _done_job(db, owner)
r = as_user(stranger).post(
f"/api/downloads/{job.id}/progress",
json={"position_seconds": 600, "duration_seconds": 3000},
)
assert r.status_code == 404
assert _states(db) == []
def test_deleting_the_job_deletes_its_places(db, make_user, as_user):
me = make_user()
job = _done_job(db, me)
assert _checkpoint(as_user(me), job.id, 600) == 600
db.delete(db.get(DownloadJob, job.id))
db.commit()
assert _states(db) == []
+1 -1
View File
@@ -99,7 +99,7 @@ class TestPublicMetaAudioFlag:
class TestCreateLinkDefault:
"""D1: a newly minted share link is downloadable by default. The one-click Share relies on
"""A newly minted share link is downloadable by default. The one-click Share relies on
this — it POSTs no `allow_download`, so the API default decides. Forward-only lives here (the API
default), NOT in a migration: the DownloadLink column default stays False, so no existing back-
catalogue link silently flips to downloadable."""
+73
View File
@@ -0,0 +1,73 @@
"""A libx264 re-encode of an odd-sized frame: 4:2:0 halves the chroma, so libx264 refuses an odd
width or height ("width not divisible by 2"). Measured: a source that is odd AND not 4:2:0 already
(a 4:4:4 or RGB screen capture, 1365x767) failed its conversion outright; a 4:2:0 source never
reaches the encoder odd (ffmpeg's crop rounds to its chroma grid), but the clip then stored the
size the user dragged, not the one the file has. Every re-encode now ends its filter chain on
`edit.EVEN_FRAME` (at most one pixel row or column), and a clip stores its probed frame.
Real ffmpeg on both paths — the refusal is the encoder's, so only the encoder can show it is gone.
"""
import shutil
import subprocess
from pathlib import Path
import pytest
from app.config import settings
from app.downloads import pipeline, playable, probe, service, storage
from app.downloads import upload as uploadmod
from app.models import DownloadJob, MediaAsset
pytestmark = pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not available")
def _ffmpeg(*args: str) -> None:
subprocess.run(["ffmpeg", "-y", "-loglevel", "error", *args], check=True)
def _frame(path: Path) -> tuple[int | None, int | None]:
_, _, w, h, _ = probe.probe_media(path)
return w, h
def test_an_odd_sized_4_4_4_video_converts_to_the_even_frame_below_it(tmp_path):
src = tmp_path / "in.mkv"
_ffmpeg(
"-f", "lavfi", "-i", "testsrc=size=161x121:rate=10:duration=1",
"-c:v", "libx264", "-pix_fmt", "yuv444p", str(src),
) # fmt: skip
assert _frame(src) == (161, 121) # odd for real (`testsrc2` rounds its own size)
p = playable.plan(probe.probe_json(src), "mkv")
assert p is not None and p.reencode_video
dest = tmp_path / "out.mp4"
subprocess.run(playable.build_cmd(src, dest, p), check=True, capture_output=True)
assert _frame(dest) == (160, 120)
def test_an_odd_crop_makes_a_clip_and_stores_the_frame_it_has(db, make_user, tmp_path, monkeypatch):
monkeypatch.setattr(settings, "download_root", str(tmp_path))
src = tmp_path / "in.mp4"
_ffmpeg(
"-f", "lavfi", "-i", "testsrc2=size=160x120:rate=10:duration=2",
"-f", "lavfi", "-i", "sine=duration=2",
"-c:v", "libx264", "-pix_fmt", "yuv420p", "-c:a", "aac", str(src),
) # fmt: skip
u = make_user()
source = uploadmod.ingest_upload(
db, u.id, filename="clip.mp4", src_path=src, size=src.stat().st_size
)
assert source.status == "done" # already phone-safe: the clip's source as it came
job = service.enqueue_edit(db, u.id, source, {"crop": {"x": 3, "y": 5, "w": 101, "h": 51}})
asset = db.get(MediaAsset, job.asset_id)
job.status, asset.status = "running", "downloading" # the worker's claim
db.commit()
pipeline._process_edit(job.id, asset.id)
db.expire_all()
job, asset = db.get(DownloadJob, job.id), db.get(MediaAsset, job.asset_id)
assert (job.status, job.error, asset.status) == ("done", None, "ready")
final = storage.abs_path(str(tmp_path), asset.rel_path)
assert _frame(final) == (100, 50)
assert (asset.width, asset.height) == (100, 50) # what the file is, not what was asked
+170
View File
@@ -0,0 +1,170 @@
"""The share-link contract: every media file a /watch link can serve passes the phone-safe rule
(`downloads.playable`).
Two halves. The structural one reads the code: a link serves an asset once it is `ready`, so every
function that marks one ready must run the rule — a new path to `ready` that skips it is named here,
before a friend's phone meets the file. The end-to-end one sends the files measured broken on an
iPhone + iPad (and the ones browsers other than Safari cannot be trusted with) through upload, the
worker's conversion and the public file route, and judges the bytes the link actually hands out.
"""
import ast
import shutil
import subprocess
from pathlib import Path
import pytest
from app.config import settings
from app.downloads import pipeline, playable, probe
from app.models import DownloadJob, MediaAsset
APP = Path(__file__).resolve().parents[1] / "app"
# The calls that ARE the rule: the worker's step (download, clip, upload conversion) and the upload's
# judgement of a file it keeps as it came.
_RULE_CALLS = {"_ensure_phone_safe", "plan"}
def _marks_ready(node: ast.AST) -> bool:
"""`x.status = "ready"`, `status="ready"`, or `"…" if … else "ready"` handed to either."""
def is_ready(v: ast.AST | None) -> bool:
if isinstance(v, ast.Constant):
return v.value == "ready"
if isinstance(v, ast.IfExp):
return is_ready(v.body) or is_ready(v.orelse)
return False
if isinstance(node, ast.Assign):
return is_ready(node.value) and any(
isinstance(t, ast.Attribute) and t.attr == "status" for t in node.targets
)
return isinstance(node, ast.keyword) and node.arg == "status" and is_ready(node.value)
def _calls(fn: ast.AST) -> set[str]:
names = set()
for n in ast.walk(fn):
if isinstance(n, ast.Call):
f = n.func
names.add(f.attr if isinstance(f, ast.Attribute) else getattr(f, "id", ""))
return names
def ready_sites(source: str) -> list[tuple[str, int, bool]]:
"""(function, line, runs the rule) for every place `source` marks something ready."""
sites = []
for fn in ast.walk(ast.parse(source)):
if not isinstance(fn, ast.FunctionDef | ast.AsyncFunctionDef):
continue
runs_rule = bool(_calls(fn) & _RULE_CALLS)
for n in ast.walk(fn):
if _marks_ready(n):
sites.append((fn.name, n.lineno, runs_rule))
return sites
def test_every_path_to_ready_runs_the_rule():
found, skipped = [], []
for path in sorted(APP.rglob("*.py")):
for name, line, runs_rule in ready_sites(path.read_text(encoding="utf-8")):
where = f"{path.relative_to(APP.parent).as_posix()}:{line} {name}"
found.append(where)
if not runs_rule:
skipped.append(where)
# The paths known today: a download, an editor clip, an ad-hoc clip, an upload's conversion, an
# upload kept as is.
# Fewer means the scan went blind (a rename, a new status spelling) — not that a path is gone.
names = {w.rsplit(" ", 1)[1] for w in found}
assert {
"_download",
"_process_edit",
"_process_clip",
"_convert_upload",
"ingest_upload",
} <= names, found
assert skipped == [], f"marks an asset ready without the phone-safe rule: {skipped}"
def test_the_scan_names_a_path_that_skips_the_rule():
planted = 'def restore(asset):\n asset.status = "ready"\n'
guarded = (
'def restore(asset, p):\n _ensure_phone_safe(1, p, p, "x")\n asset.status = "ready"\n'
)
assert ready_sites(planted) == [("restore", 2, False)]
assert ready_sites(guarded) == [("restore", 3, True)]
assert (
ready_sites('def f(a, x):\n a.update(status="pending" if x else "ready")\n')[0][2]
is False
)
# --- end to end: upload → worker → the bytes a link serves ---------------------------------------
_V = ["-f", "lavfi", "-i", "testsrc2=size=160x120:rate=10:duration=1"]
_A = ["-f", "lavfi", "-i", "sine=duration=1"]
# name -> (ffmpeg args after the inputs, filename the user uploads)
_FILES = {
"hevc tagged hev1 (iOS: broken)": (
[*_V, *_A, "-c:v", "libx265", "-tag:v", "hev1", "-c:a", "aac"],
"phone.mov",
),
"av1 (iOS: broken)": ([*_V, *_A, "-c:v", "libsvtav1", "-c:a", "aac"], "clip.mp4"),
"h264 in matroska (iOS: broken)": ([*_V, *_A, "-c:v", "libx264", "-c:a", "aac"], "clip.mkv"),
"mp3 inside an mp4 (iOS: broken)": ([*_A, "-c:a", "libmp3lame", "-f", "mp4"], "song.mp4"),
"vp9 + opus webm": ([*_V, *_A, "-c:v", "libvpx-vp9", "-c:a", "libopus"], "clip.webm"),
"10-bit h264": (
[*_V, *_A, "-c:v", "libx264", "-pix_fmt", "yuv420p10le", "-c:a", "aac"],
"clip.mp4",
),
"opus in ogg": ([*_A, "-c:a", "libopus"], "voice.ogg"),
}
@pytest.fixture
def media_root(tmp_path, monkeypatch):
root = tmp_path / "media"
root.mkdir()
monkeypatch.setattr(settings, "download_root", str(root))
return root
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not available")
@pytest.mark.parametrize("name", _FILES)
def test_what_a_link_serves_passes_the_rule(
name, db, as_user, make_user, client, media_root, tmp_path, upload_headers
):
args, filename = _FILES[name]
src = tmp_path / f"src{Path(filename).suffix}"
subprocess.run(["ffmpeg", "-y", "-loglevel", "error", *args, str(src)], check=True)
owner = as_user(make_user())
r = owner.post(
"/api/downloads/upload", content=src.read_bytes(), headers=upload_headers(filename)
)
assert r.status_code == 200, r.text
job_id = r.json()["id"]
assert r.json()["status"] == "queued" # every one of these fails the rule as it came
job = db.get(DownloadJob, job_id)
asset = db.get(MediaAsset, job.asset_id)
job.status, asset.status = "running", "downloading" # what the worker's claim does
db.commit()
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
assert db.get(DownloadJob, job_id).status == "done"
link = owner.post(f"/api/downloads/{job_id}/links", json={})
assert link.status_code == 200, link.text
served = client.get(f"/api/public/watch/{link.json()['token']}/file")
assert served.status_code == 200
got = tmp_path / "served.bin"
got.write_bytes(served.content)
data = probe.probe_json(got)
ext = {"video/mp4": "mp4", "audio/mp4": "m4a", "audio/mpeg": "mp3"}.get(
served.headers["content-type"].split(";")[0]
)
assert ext is not None, served.headers["content-type"] # served as a type every phone plays
verdict = playable.plan(data, ext)
assert verdict is not None and not verdict.needs_change, (name, data["streams"])
+157
View File
@@ -0,0 +1,157 @@
"""The phone-safe rule (`downloads.playable`): what a library file must be so a share link plays
on any phone or browser, and the one ffmpeg command that makes it so.
The plan table is the measured device verdict, class by class: what failed on a real iPhone + iPad
(AV1, Matroska, HEVC, MP3 inside an .mp4) must change, and so must what browsers other than Safari
cannot be trusted with (10-bit / 4:4:4 H.264, VP9, Opus). The ffmpeg cases run the real encoder and
probe its output, because the rule is about the bytes, not the flags.
"""
import shutil
import subprocess
from pathlib import Path
import pytest
from app.downloads import playable, probe
# conftest turns a missing ffmpeg into a hard error in the gate (SIFTLODE_TEST_REQUIRE_DB), so this
# only ever skips on an explicit pure-logic run.
needs_ffmpeg = pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not available")
def _data(video=None, audio=None, fmt="mov,mp4,m4a,3gp,3g2,mj2", cover=False):
streams = []
if cover:
streams.append(
{"codec_type": "video", "codec_name": "mjpeg", "disposition": {"attached_pic": 1}}
)
if video:
codec, pix = video
streams.append({"codec_type": "video", "codec_name": codec, "pix_fmt": pix})
if audio:
streams.append({"codec_type": "audio", "codec_name": audio})
return {"streams": streams, "format": {"format_name": fmt}}
# (probe, extension) -> (ext, reencode_video, reencode_audio, needs_change)
CASES = {
"h264 8-bit + aac in mp4": (_data(("h264", "yuv420p"), "aac"), "mp4", ("mp4", 0, 0, 0)),
"full-range h264 is still 8-bit 4:2:0": (
_data(("h264", "yuvj420p"), "aac"),
"mp4",
("mp4", 0, 0, 0),
),
"10-bit h264": (_data(("h264", "yuv420p10le"), "aac"), "mp4", ("mp4", 1, 0, 1)),
"4:4:4 h264": (_data(("h264", "yuv444p"), "aac"), "mp4", ("mp4", 1, 0, 1)),
"hevc (an iPhone's .mov)": (_data(("hevc", "yuv420p10le"), "aac"), "mov", ("mp4", 1, 0, 1)),
"vp9 + opus webm": (_data(("vp9", "yuv420p"), "opus"), "webm", ("mp4", 1, 1, 1)),
"av1": (_data(("av1", "yuv420p"), "aac"), "mp4", ("mp4", 1, 0, 1)),
"h264 + aac in mkv: remux only": (_data(("h264", "yuv420p"), "aac"), "mkv", ("mp4", 0, 0, 1)),
"h264 + opus in mp4": (_data(("h264", "yuv420p"), "opus"), "mp4", ("mp4", 0, 1, 1)),
"mp3 beside video in mp4": (_data(("h264", "yuv420p"), "mp3"), "mp4", ("mp4", 0, 1, 1)),
"a silent h264 video": (_data(("h264", "yuv420p")), "mp4", ("mp4", 0, 0, 0)),
"mp3": (_data(audio="mp3", fmt="mp3"), "mp3", ("mp3", 0, 0, 0)),
"mp3 with cover art": (_data(audio="mp3", fmt="mp3", cover=True), "mp3", ("mp3", 0, 0, 0)),
"mp3 inside an mp4: remux to .mp3": (_data(audio="mp3"), "mp4", ("mp3", 0, 0, 1)),
"aac in m4a": (_data(audio="aac"), "m4a", ("m4a", 0, 0, 0)),
"aac audio-only in mp4: remux to .m4a": (_data(audio="aac"), "mp4", ("m4a", 0, 0, 1)),
"opus in ogg": (_data(audio="opus", fmt="ogg"), "ogg", ("m4a", 0, 1, 1)),
}
@pytest.mark.parametrize("name", CASES)
def test_the_plan_for_each_class(name):
data, ext, (want_ext, rv, ra, needs) = CASES[name]
p = playable.plan(data, ext)
assert p is not None
assert (p.ext, p.reencode_video, p.reencode_audio, p.needs_change) == (
want_ext,
bool(rv),
bool(ra),
bool(needs),
)
@pytest.mark.parametrize(
"data",
[
None, # ffprobe could not read it
{"streams": [], "format": {"format_name": "pdf"}},
_data(("png", "rgb24"), fmt="png_pipe"), # a still image is a download-only file
],
)
def test_a_file_that_is_not_media_has_no_plan(data):
assert playable.plan(data, "bin") is None
def test_the_extension_is_read_with_or_without_its_dot():
data = _data(("h264", "yuv420p"), "aac")
assert playable.plan(data, ".MP4") == playable.plan(data, "mp4")
assert not playable.plan(data, ".MP4").needs_change
def test_a_video_re_encode_pins_8_bit_420_and_maps_only_the_judged_streams():
p = playable.Plan("mp4", True, True, False, True)
cmd = playable.build_cmd(Path("/in.mov"), Path("/out.mp4"), p)
assert cmd[cmd.index("-pix_fmt") + 1] == "yuv420p"
assert cmd[cmd.index("-c:v") + 1] == "libx264"
assert cmd[cmd.index("-c:a") + 1] == "copy"
maps = [cmd[i + 1] for i, a in enumerate(cmd) if a == "-map"]
assert maps == ["0:V:0", "0:a:0?"] # never cover art, never a subtitle the muxer refuses
assert cmd[-3:] == ["-movflags", "+faststart", "/out.mp4"]
def test_an_audio_remux_copies_the_audio_alone():
p = playable.Plan("mp3", False, False, False, True)
cmd = playable.build_cmd(Path("/in.mp4"), Path("/out.mp3"), p)
assert [cmd[i + 1] for i, a in enumerate(cmd) if a == "-map"] == ["0:a:0"]
assert "-c:v" not in cmd and "-pix_fmt" not in cmd
assert cmd[cmd.index("-c:a") + 1] == "copy"
assert "-movflags" not in cmd # an .mp3 has no moov atom to move
# --- the real encoder ------------------------------------------------------------------------
def _ffmpeg(*args: str) -> None:
subprocess.run(["ffmpeg", "-y", "-loglevel", "error", *args], check=True)
def _convert(src: Path, tmp_path: Path) -> Path:
p = playable.plan(probe.probe_json(src), src.suffix)
assert p is not None and p.needs_change
dest = tmp_path / f"out.{p.ext}"
subprocess.run(playable.build_cmd(src, dest, p), check=True, capture_output=True)
return dest
@needs_ffmpeg
def test_a_10_bit_matroska_with_opus_comes_out_phone_safe(tmp_path):
src = tmp_path / "in.mkv"
_ffmpeg(
"-f", "lavfi", "-i", "testsrc2=size=160x120:rate=10:duration=1",
"-f", "lavfi", "-i", "sine=duration=1",
"-c:v", "libx264", "-pix_fmt", "yuv420p10le", "-c:a", "libopus",
str(src),
) # fmt: skip
out = _convert(src, tmp_path)
data = probe.probe_json(out)
video, audio = probe.primary_streams(data)
assert (video["codec_name"], video["pix_fmt"], video["profile"]) == ("h264", "yuv420p", "High")
assert audio["codec_name"] == "aac"
assert out.suffix == ".mp4"
assert not playable.plan(data, out.suffix).needs_change # the rule accepts its own output
@needs_ffmpeg
def test_mp3_inside_an_mp4_comes_out_as_an_mp3(tmp_path):
# The one unsafe file prod held: an audio download remuxed into .mp4, which iPhone and iPad
# both refused to play.
src = tmp_path / "in.mp4"
_ffmpeg("-f", "lavfi", "-i", "sine=duration=1", "-c:a", "libmp3lame", "-f", "mp4", str(src))
out = _convert(src, tmp_path)
data = probe.probe_json(out)
assert out.suffix == ".mp3"
assert probe.primary_streams(data)[1]["codec_name"] == "mp3"
assert not playable.plan(data, out.suffix).needs_change
+4
View File
@@ -159,6 +159,10 @@ def test_a_collection_is_created_with_a_well_formed_uri(plex):
assert plex.seen[-1].method == "POST"
assert params["title"] == "Best of: 90s" and params["sectionId"] == "3"
assert params["uri"] == "server://abc/com.plexapp.plugins.library/library/metadata/42"
assert params["type"] == "1" # a movie collection unless told otherwise
# A show collection is made as one: made as a movie one, it came out empty in a TV section.
plex.create_collection("3", "Shows", "42", 2)
assert plex.seen[-1].url.params["type"] == "2"
plex.routes["/library/collections"] = {}
with pytest.raises(PlexError):
+33 -1
View File
@@ -5,6 +5,7 @@ DB lane (`db` fixture) — skipped without Postgres.
from datetime import datetime, timezone
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import select
@@ -14,6 +15,7 @@ from app.db import get_db
from app.main import app
from app.models import PlexCollection, PlexItem, PlexLibrary, PlexShow, User
from app.plex.client import PlexError
from app.plex.lookups import by_rating_key
from app.routes.plex import collections as coll_routes
@@ -224,8 +226,9 @@ class _CollStub:
if self.fail_on == name:
raise PlexError("boom")
def create_collection(self, section_key, title, item_rating_key):
def create_collection(self, section_key, title, item_rating_key, item_type=1):
self._maybe("create_collection")
self.created_type = item_type
return {"ratingKey": "NEW"}
def add_collection_item(self, rk, item):
@@ -320,6 +323,22 @@ def test_create_writes_through_to_plex(db, monkeypatch, override_dep):
assert body["editable"] is True # create marks the new collection editable
@pytest.mark.parametrize(("seed", "plex_type"), [("s1", 2), ("m1", 1)])
def test_create_makes_a_show_collection_as_one(db, monkeypatch, override_dep, seed, plex_type):
# Measured on a real Plex: a collection made as a movie one (type 1) in a TV section came out
# empty with a show seed, and refused the show when it was added after.
lib, user = _seed_editable(db)
db.add(PlexShow(rating_key="s1", library_id=lib.id, title="Series"))
db.commit()
stub = _CollStub()
client = _admin_client(db, user, monkeypatch, override_dep, stub, resync=_resync_create)
r = client.post(
"/api/plex/collections", json={"library": "", "title": "New", "item_rating_key": seed}
)
assert r.status_code == 200, r.text
assert stub.created_type == plex_type
def test_create_resync_failure_is_422(db, monkeypatch, override_dep):
# The collection was created in Plex (wrote=True) but the follow-up resync read failed → 422 with
# the honest "reconcile on the next sync" detail, not a false "write failed".
@@ -337,6 +356,19 @@ def test_create_resync_failure_is_422(db, monkeypatch, override_dep):
assert r.status_code == 422, r.text
assert r.json()["detail"] == {"code": "refused", "reason": "collection_pending_sync"}
assert "create_collection" in stub.calls
# It exists on Plex: its row stays, editable, until the next sync fills it in — so it is offered
# where collections are added to, and the same name is not made twice.
row = by_rating_key(db, PlexCollection, "NEW")
assert row is not None and row.editable and row.title == "Fresh Coll"
# Plex made it with the seed in it: the seed's row says so, so the title shows ticked in it.
assert row.child_count == 1
assert "NEW" in (by_rating_key(db, PlexItem, "m1").collection_keys or [])
again = client.post(
"/api/plex/collections",
json={"library": "1", "title": "Fresh Coll", "item_rating_key": "m1"},
)
assert again.status_code == 409, again.text
assert stub.calls.count("create_collection") == 1
def test_mutation_failure_returns_422_write_failed(db, monkeypatch, override_dep):
@@ -0,0 +1,86 @@
"""A movie or show card's "Add to collection": the collections it can go into, each saying whether the
title is in it already. DB lane (`db` fixture)."""
from datetime import datetime, timezone
from fastapi.testclient import TestClient
import app.state as state
from app.auth import current_user
from app.db import get_db
from app.main import app
from app.models import PlexCollection, PlexItem, PlexLibrary, PlexShow, User
def _seed(db):
movies = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
shows = PlexLibrary(plex_key="2", title="Shows", kind="show", enabled=True)
db.add_all([movies, shows])
db.flush()
enriched = datetime(2024, 1, 1, tzinfo=timezone.utc)
db.add_all(
[
PlexItem(
rating_key="m1",
title="Film",
kind="movie",
library_id=movies.id,
collection_keys=["cA"],
people_enriched_at=enriched,
),
PlexItem(
rating_key="e1",
title="An episode",
kind="episode",
library_id=shows.id,
people_enriched_at=enriched,
),
PlexShow(rating_key="s1", title="Series", library_id=shows.id, collection_keys=[]),
# The movie library's: two manual ones it can go into, and three it cannot.
PlexCollection(rating_key="cA", library_id=movies.id, title="Alpha", editable=True),
PlexCollection(rating_key="cB", library_id=movies.id, title="Beta", editable=True),
PlexCollection(rating_key="cP", library_id=movies.id, title="Plain", editable=False),
PlexCollection(
rating_key="cS", library_id=movies.id, title="Smart", editable=True, smart=True
),
PlexCollection(
rating_key="cI", library_id=movies.id, title="IMDb Top 250", editable=True
),
# The show library's own.
PlexCollection(rating_key="cT", library_id=shows.id, title="Sci-fi", editable=True),
]
)
user = User(email="collfor@example.com")
db.add(user)
db.commit()
return user
def _get(db, monkeypatch, override_dep, rk):
user = _seed(db)
monkeypatch.setattr(state, "_configured_cache", True)
override_dep(get_db, db.info["request"])
override_dep(current_user, lambda: user)
return TestClient(app).get(f"/api/plex/collections/for/{rk}")
def test_a_movie_lists_its_librarys_manual_collections_and_which_hold_it(
db, monkeypatch, override_dep
):
resp = _get(db, monkeypatch, override_dep, "m1")
assert resp.status_code == 200
got = [(c["id"], c["contains"]) for c in resp.json()["collections"]]
assert got == [("cA", True), ("cB", False)]
def test_a_show_lists_its_own_librarys(db, monkeypatch, override_dep):
resp = _get(db, monkeypatch, override_dep, "s1")
assert [(c["id"], c["contains"]) for c in resp.json()["collections"]] == [("cT", False)]
def test_an_episode_is_not_a_collection_member(db, monkeypatch, override_dep):
assert _get(db, monkeypatch, override_dep, "e1").status_code == 404
def test_an_unknown_key_is_not_found(db, monkeypatch, override_dep):
assert _get(db, monkeypatch, override_dep, "nope").status_code == 404
+107 -2
View File
@@ -19,7 +19,7 @@ import app.state as state
from app.auth import current_user
from app.db import get_db
from app.main import app
from app.models import PlexItem, PlexLibrary, PlexShow, User
from app.models import PlexItem, PlexLibrary, PlexSeason, PlexShow, PlexState, User
from app.plex import detail
from app.plex.sync import _apply_facet_fields, _enrich_people_batch
@@ -276,8 +276,16 @@ def test_item_detail_episode_serves_rich_from_live(db, monkeypatch, override_dep
show = PlexShow(rating_key="s1", library_id=lib.id, title="A Show")
db.add(show)
db.flush()
season = PlexSeason(rating_key="se1", show_id=show.id, season_number=1)
db.add(season)
db.flush()
ep = PlexItem(
rating_key="e1", library_id=lib.id, kind="episode", title="Pilot", show_id=show.id
rating_key="e1",
library_id=lib.id,
kind="episode",
title="Pilot",
show_id=show.id,
season_id=season.id,
)
db.add(ep)
db.commit()
@@ -289,6 +297,8 @@ def test_item_detail_episode_serves_rich_from_live(db, monkeypatch, override_dep
assert resp.status_code == 200, resp.text
body = resp.json()
assert body["imdb_id"] == "tt0068646" # from the live meta, not the (NULL) row
# The page's links up to the show and the season.
assert (body["show_id"], body["season_id"]) == ("s1", "se1")
assert [c["name"] for c in body["cast"]] == ["Marlon Brando", "Al Pacino"]
db.refresh(ep)
assert ep.full_cast is None # episodes are not persisted/enriched
@@ -401,3 +411,98 @@ def test_item_detail_names_tracks_from_their_language_not_plexs_title(
("magyar Forced", "hu", True, False, None),
("English SDH", "en", False, True, None),
]
def test_item_detail_markers_carry_plex_final_flag(db, monkeypatch, override_dep):
"""An item can carry several credits markers — mid- and post-credit scenes sit between them —
and only the one Plex flags `final` is the real end; the player needs that flag to know which
credits may move on to the next item."""
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
db.add(PlexItem(rating_key="m2", library_id=lib.id, kind="movie", title="Iron Man"))
db.commit()
user = _user(db, "markers@example.com")
db.commit()
meta = {
"Marker": [
{"type": "intro", "startTimeOffset": 1000, "endTimeOffset": 60000},
{"type": "credits", "startTimeOffset": 6811000, "endTimeOffset": 6831000},
{
"type": "credits",
"startTimeOffset": 7549000,
"endTimeOffset": 7559000,
"final": True,
},
{"type": "commercial", "startTimeOffset": 1, "endTimeOffset": 2},
]
}
monkeypatch.setattr(detail_routes, "PlexClient", lambda db: _StubClient(meta))
resp = _get(db, user, monkeypatch, override_dep, "/api/plex/item/m2")
assert resp.status_code == 200, resp.text
assert resp.json()["markers"] == [
{"type": "intro", "start_s": 1.0, "end_s": 60.0, "final": False},
{"type": "credits", "start_s": 6811.0, "end_s": 6831.0, "final": False},
{"type": "credits", "start_s": 7549.0, "end_s": 7559.0, "final": True},
]
def _show_with_episodes(db, n=3):
lib = PlexLibrary(plex_key="3", title="Shows", kind="show", enabled=True)
db.add(lib)
db.flush()
show = PlexShow(rating_key="sn1", library_id=lib.id, title="Next Show")
db.add(show)
db.flush()
eps = [
PlexItem(
rating_key=f"sn1e{i}",
library_id=lib.id,
kind="episode",
title=f"Episode {i}",
show_id=show.id,
season_number=1,
episode_number=i,
)
for i in range(1, n + 1)
]
db.add_all(eps)
db.flush()
return eps
def test_show_next_plays_the_first_episode_of_an_untouched_show(db, monkeypatch, override_dep):
"""A show card's picture plays: never started → its first episode."""
_show_with_episodes(db)
user = _user(db, "next1@example.com")
db.commit()
resp = _get(db, user, monkeypatch, override_dep, "/api/plex/show/sn1/next")
assert resp.status_code == 200, resp.text
nxt = resp.json()["next"]
assert nxt["id"] == "sn1e1"
assert nxt["show_title"] == "Next Show"
def test_show_next_continues_the_episode_in_progress(db, monkeypatch, override_dep):
"""Started → the episode in progress (the show page's Resume), not the first unwatched."""
eps = _show_with_episodes(db)
user = _user(db, "next2@example.com")
db.add(PlexState(user_id=user.id, item_id=eps[0].id, status="watched"))
db.add(PlexState(user_id=user.id, item_id=eps[2].id, status="new", position_seconds=120))
db.commit()
resp = _get(db, user, monkeypatch, override_dep, "/api/plex/show/sn1/next")
assert resp.json()["next"]["id"] == "sn1e3"
assert resp.json()["next"]["position_seconds"] == 120
def test_show_next_is_null_for_an_empty_show_and_404_for_an_unknown_one(
db, monkeypatch, override_dep
):
_show_with_episodes(db, n=0)
user = _user(db, "next3@example.com")
db.commit()
assert _get(db, user, monkeypatch, override_dep, "/api/plex/show/sn1/next").json() == {
"next": None
}
assert _get(db, user, monkeypatch, override_dep, "/api/plex/show/nope/next").status_code == 404
+53
View File
@@ -165,6 +165,30 @@ def test_continue_watching_movies_and_shows(db, monkeypatch, override_dep):
assert "m1" not in ids
def test_a_continue_watching_show_says_whether_it_has_an_episode_to_resume(
db, monkeypatch, override_dep
):
# Both in-progress rails (Discover's and Home's) agree with the one rollup: a show part watched
# with nothing to resume offers Play alone, not Continue + Restart.
from app.discover.queries import plex_in_progress
user = _seed(db)
def rails():
d = _get(db, user, monkeypatch, override_dep, "/api/plex/discover?scope=both").json()
home = {c["id"]: c for _, c in plex_in_progress(db, user.id, lang="en")}
return {c["id"]: c for c in d["continue_watching"]}["sh1"], home["sh1"]
assert [c["resumable"] for c in rails()] == [True, True]
ep1 = db.execute(select(PlexItem).where(PlexItem.rating_key == "sh1e1")).scalar_one()
st = db.execute(
select(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == ep1.id)
).scalar_one()
st.status, st.position_seconds = "watched", 0
db.commit()
assert [(c["status"], c["resumable"]) for c in rails()] == [("in_progress", False)] * 2
def test_continue_watching_null_status_in_hidden_out(db, monkeypatch, override_dep):
# A started movie whose PlexState.status is NULL is still "in progress": `NULL NOT IN (...)` is NULL
# (not TRUE), so a bare notin_ would wrongly drop it — the or_(is_(None), notin_) branch keeps it.
@@ -316,6 +340,35 @@ def test_surprise_excludes_watched_and_caps_per_kind(db, monkeypatch, override_d
assert len([c for c in items if c["type"] == "show"]) <= 3
def test_a_reshuffle_deals_other_titles_first(db, monkeypatch, override_dep):
# The ↻ sends the previous draw: what is left of the pool comes first, and a pool smaller than a
# draw is topped up from the excluded titles rather than leaving the rail short.
user = _seed(db)
def movies(url: str) -> list[str]:
items = _get(db, user, monkeypatch, override_dep, url).json()["items"]
return [c["id"] for c in items if c["type"] == "movie"]
for _ in range(5): # random order: one lucky pass would prove nothing
got = movies("/api/plex/surprise?scope=movie&exclude=m2")
assert got[0] == "m3" and set(got) == {"m2", "m3"}
assert set(movies("/api/plex/surprise?scope=movie&exclude=m2&exclude=m3")) == {"m2", "m3"}
def test_a_kept_draw_is_read_again_not_redrawn(db, monkeypatch, override_dep):
# The rail's draw stands until the ↻, but its cards' watch state moves: `keep` answers the titles on
# screen, in their order and as they stand now, a watched one included; the scope still bounds it.
user = _seed(db)
url = "/api/plex/surprise?scope=both&keep=m3&keep=sh1&keep=m1&keep=nope"
items = _get(db, user, monkeypatch, override_dep, url).json()["items"]
assert _ids(items) == ["m3", "sh1", "m1"]
by_id = {c["id"]: c for c in items}
assert by_id["m1"]["status"] == "watched"
assert by_id["m3"]["position_seconds"] > 0
movie_only = "/api/plex/surprise?scope=movie&keep=sh1&keep=m2"
assert _ids(_get(db, user, monkeypatch, override_dep, movie_only).json()["items"]) == ["m2"]
def test_discover_scope_movie_excludes_shows(db, monkeypatch, override_dep):
user = _seed(db)
d = _get(db, user, monkeypatch, override_dep, "/api/plex/discover?scope=movie").json()
+159
View File
@@ -0,0 +1,159 @@
"""Plex summaries and taglines are stored as plain text.
A Hungarian TV agent hands summaries over as HTML (`<p>A műsorban … p&aacute;rosnak …</p>`), and
the info page and the hover card showed the tags. `plain_text` is the one cleaner: the sync's every
summary path and the HU overlay store through it, and migration 0101 cleaned the rows written before.
"""
import pytest
from alembic import command
from alembic.config import Config
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.db import engine
from app.models import PlexCollection, PlexItem, PlexLibrary, PlexShow
from app.plex.detail import rich_detail
from app.plex.overlay import sync_overlay
from app.plex.plaintext import plain_text
from app.plex import sync
from app.plex.sync import _apply_item, _upsert_collection
HU_SUMMARY = "<p>A műsorban minden p&aacute;rosnak stoppal kell eljutnia.</p>"
@pytest.mark.parametrize(
"raw, clean",
[
(HU_SUMMARY, "A műsorban minden párosnak stoppal kell eljutnia."),
("<p>One.</p><p>Two.</p>", "One.\n\nTwo."),
("Line one<br>line two<BR/>three", "Line one\nline two\nthree"),
("<b>Bold</b> &amp; <i>more</i>", "Bold & more"),
# Prose that only looks like markup stays as written.
("a < b and 1 <3 you", "a < b and 1 <3 you"),
# Only a reference with its semicolon is one: an ampersand in prose stays as written.
("AT&notice, R&copy and H&M", "AT&notice, R&copy and H&M"),
# ...and only a whole entity name: a legacy one at its start ("&not", "&amp") is not.
("AT&notice; and &ampx;", "AT&notice; and &ampx;"),
("&#233;t&#xE9; &copy;", "été ©"),
# A decimal reference too long for int() to parse is out of range, like &#x110000;.
("a &#" + "9" * 5000 + "; b", "a � b"),
("&#" + "0" * 5000 + "65;", "A"),
("Plain summary.", "Plain summary."),
(" padded ", "padded"),
("<p></p>", None),
("", None),
(None, None),
],
)
def test_plain_text(raw, clean):
assert plain_text(raw) == clean
def test_the_item_sync_stores_a_summary_as_plain_text():
row = PlexItem()
_apply_item(row, lib_id=1, kind="episode", meta={"title": "Ep", "summary": HU_SUMMARY})
assert row.summary == "A műsorban minden párosnak stoppal kell eljutnia."
def test_the_show_sync_stores_a_summary_as_plain_text(db, monkeypatch):
lib = PlexLibrary(plex_key="2", title="Shows", kind="show", enabled=True)
db.add(lib)
db.commit()
meta = {"ratingKey": "s1", "title": "Show", "summary": HU_SUMMARY}
def listing(_plex, _key, typ, progress=None):
if progress is not None:
progress["complete"] = True
return [meta] if typ == sync._TYPE_SHOW else []
monkeypatch.setattr(sync, "_paginate", listing)
sync._sync_shows(db, object(), lib, {"shows": 0, "seasons": 0, "episodes": 0})
db.commit()
show = db.execute(select(PlexShow).where(PlexShow.rating_key == "s1")).scalar_one()
assert show.summary == "A műsorban minden párosnak stoppal kell eljutnia."
def test_a_collection_stores_its_summary_as_plain_text(db):
lib = PlexLibrary(plex_key="1", title="Movies", kind="movie", enabled=True)
db.add(lib)
db.flush()
col = _upsert_collection(db, None, "c1", lib.id, {"title": "C", "summary": "<p>A &amp; B</p>"})
assert col.summary == "A & B"
def test_a_fetched_tagline_is_plain_text():
assert rich_detail({"tagline": "<i>An offer</i> &amp; more"})["tagline"] == "An offer & more"
class _StubPlex:
def __init__(self, items: list[dict]):
self._items = items
def section_items(self, key, item_type, start, size):
return self._items[start : start + size], len(self._items)
def test_the_hungarian_overlay_stores_plain_text(db):
en_lib = PlexLibrary(plex_key="4", title="Movies", kind="movie")
hu_lib = PlexLibrary(plex_key="9", title="Filmek", kind="movie", metadata_lang="hu")
db.add_all([en_lib, hu_lib])
db.flush()
hu_lib.overlay_of = en_lib.id
row = PlexItem(rating_key="m1", library_id=en_lib.id, kind="movie", title="Heat", guid="tmdb:1")
db.add(row)
db.commit()
meta = {
"ratingKey": "h1",
"title": "Szemtől szemben",
"summary": HU_SUMMARY,
"tagline": "<i>Egy &eacute;let</i>",
"Guid": [{"id": "tmdb://1"}],
}
sync_overlay(db, _StubPlex([meta]), hu_lib, en_lib.id, {"overlaid": 0})
db.commit()
db.refresh(row)
assert row.summary_hu == "A műsorban minden párosnak stoppal kell eljutnia."
assert row.tagline_hu == "Egy élet"
def test_0101_cleans_the_stored_rows(db):
cfg = Config("alembic.ini")
command.downgrade(cfg, "0100_drop_plex_item_markers")
upgraded = False
try:
with Session(engine) as s:
lib = PlexLibrary(plex_key="1", title="Shows", kind="show", enabled=True)
s.add(lib)
s.flush()
ep = PlexItem(
rating_key="e1",
library_id=lib.id,
kind="episode",
title="Ep",
summary=HU_SUMMARY,
tagline_hu="<i>x</i>",
)
plain = PlexItem(
rating_key="m1", library_id=lib.id, kind="movie", title="M", summary="a < b"
)
show = PlexShow(rating_key="s1", library_id=lib.id, title="S", summary_hu="<p>Hu</p>")
col = PlexCollection(rating_key="c1", library_id=lib.id, title="C", summary="A &amp; B")
s.add_all([ep, plain, show, col])
s.commit()
ids = (ep.id, plain.id, show.id, col.id)
command.upgrade(cfg, "head")
upgraded = True
with Session(engine) as s:
ep2, plain2 = s.get(PlexItem, ids[0]), s.get(PlexItem, ids[1])
show2, col2 = s.get(PlexShow, ids[2]), s.get(PlexCollection, ids[3])
assert ep2 and plain2 and show2 and col2
assert ep2.summary == "A műsorban minden párosnak stoppal kell eljutnia."
assert ep2.tagline_hu == "x"
assert plain2.summary == "a < b"
assert show2.summary_hu == "Hu"
assert col2.summary == "A & B"
finally:
if not upgraded:
command.upgrade(cfg, "head")
@@ -96,6 +96,8 @@ def test_remux_item_stays_a_stream_copy_session(db, monkeypatch, override_dep):
assert body["mode"] == "hls"
assert body["transcode"] is False
assert captured["transcode"] is False
# The id the player names when it stops the session (POST …/session/{id}/stop).
assert body["id"] == "sid"
def test_direct_item_never_opens_a_session(db, monkeypatch, override_dep):
+40 -7
View File
@@ -18,6 +18,7 @@ from app.db import get_db
from app.main import app
from app.models import PlexItem, PlexLibrary, PlexSeason, PlexShow, PlexState, User
from app.plex import rollup, sync
from app.plex.cards import ShowState
def _dt(day: int) -> datetime:
@@ -184,21 +185,53 @@ class TestRollup:
def test_state_map_new_inprogress_watched(self, db):
s = _seed(db)
show, user = s["show"], s["user"]
# e1 in-progress, e2 new -> in_progress
assert rollup.state_map(db, user.id, [show.id]) == {show.id: "in_progress"}
# e1 in-progress, e2 new -> in_progress, and e1's resume point makes it resumable
assert rollup.state_map(db, user.id, [show.id]) == {show.id: ShowState("in_progress", True)}
# empty input short-circuits
assert rollup.state_map(db, user.id, []) == {}
# a fresh user with no state -> new
other = User(email="rollup-other@example.com")
db.add(other)
db.commit()
assert rollup.state_map(db, other.id, [show.id]) == {show.id: "new"}
# mark every episode watched for `user` -> watched
assert rollup.state_map(db, other.id, [show.id]) == {show.id: ShowState("new", False)}
# e1 finished, e2 untouched: part watched, but no episode to resume — Continue and Restart
# would both play e2 from its start
db.execute(delete(PlexState).where(PlexState.user_id == user.id))
for e in db.execute(select(PlexItem).where(PlexItem.kind == "episode")).scalars():
db.add(PlexState(user_id=user.id, item_id=e.id, status="watched"))
e1, e2 = db.execute(
select(PlexItem).where(PlexItem.kind == "episode").order_by(PlexItem.episode_number)
).scalars()
db.add(PlexState(user_id=user.id, item_id=e1.id, status="watched"))
db.commit()
assert rollup.state_map(db, user.id, [show.id]) == {show.id: "watched"}
assert rollup.state_map(db, user.id, [show.id]) == {
show.id: ShowState("in_progress", False)
}
# mark every episode watched for `user` -> watched
db.add(PlexState(user_id=user.id, item_id=e2.id, status="watched"))
db.commit()
assert rollup.state_map(db, user.id, [show.id]) == {show.id: ShowState("watched", False)}
def test_the_grid_card_says_whether_a_show_has_an_episode_to_resume(
self, db, monkeypatch, override_dep
):
s = _seed(db)
user = s["user"]
cards = {
i["id"]: i
for i in _get(db, user, monkeypatch, override_dep, "/api/plex/library").json()["items"]
}
assert cards["s1"]["resumable"] is True
# e1 finished, e2 untouched: in progress, but nothing to resume
e1 = db.execute(select(PlexItem).where(PlexItem.rating_key == "e1")).scalar_one()
db.execute(
delete(PlexState).where(PlexState.user_id == user.id, PlexState.item_id == e1.id)
)
db.add(PlexState(user_id=user.id, item_id=e1.id, status="watched"))
db.commit()
cards = {
i["id"]: i
for i in _get(db, user, monkeypatch, override_dep, "/api/plex/library").json()["items"]
}
assert (cards["s1"]["status"], cards["s1"]["resumable"]) == ("in_progress", False)
def _fake_paginate(by_type):
+101
View File
@@ -0,0 +1,101 @@
"""A player that no longer wants its HLS session stops it, instead of leaving ffmpeg to run on.
Left to the idle reaper, a transcode ran to the end of the film after the viewer had gone (measured on
localdev: ~4 cores for a minute after leaving a resumed 480p film). The player names the session it
played; the server stops it only while it is still that viewer's registered session for the item, so
a stop that arrives after a seek-restart (or from a second tab) cannot end the newer playback.
No `db` fixture: the route reads nothing from the database (`current_user` is overridden wholesale).
"""
import time
import pytest
from fastapi.testclient import TestClient
import app.state as state
from app.auth import current_user
from app.main import app
from app.models import User
from app.plex import stream as plex_stream
VIEWER = 41
class _Proc:
def __init__(self):
self.terminated = False
def poll(self):
return None
def terminate(self):
self.terminated = True
def wait(self, timeout=None):
return 0
class _Session:
def __init__(self, session_id: str, directory):
self.id = session_id
self.dir = directory
self.proc = _Proc()
self.last_access = time.time()
@pytest.fixture
def sessions(monkeypatch):
live: dict = {}
monkeypatch.setattr(plex_stream, "_sessions", live)
return live
@pytest.fixture
def client(monkeypatch, override_dep):
monkeypatch.setattr(state, "_configured_cache", True)
user = User(id=VIEWER, email="stop@example.com")
override_dep(current_user, lambda: user)
return TestClient(app)
def _session(tmp_path, sid: str) -> _Session:
d = tmp_path / sid
d.mkdir()
return _Session(sid, d)
def test_the_named_session_is_stopped_and_its_directory_goes(client, tmp_path, sessions):
s = _session(tmp_path, "sid-a")
sessions[(VIEWER, "m1")] = s
r = client.post("/api/plex/stream/m1/session/sid-a/stop", json={})
assert r.status_code == 200, r.text
assert r.json() == {"ok": True, "stopped": True}
assert s.proc.terminated
assert sessions == {}
assert not s.dir.exists()
def test_a_session_that_replaced_the_named_one_plays_on(client, tmp_path, sessions):
"""The stop of the session a seek-restart replaced arrives after the new one registered."""
newer = _session(tmp_path, "sid-b")
sessions[(VIEWER, "m1")] = newer
r = client.post("/api/plex/stream/m1/session/sid-a/stop", json={})
assert r.json() == {"ok": True, "stopped": False}
assert not newer.proc.terminated
assert sessions == {(VIEWER, "m1"): newer}
def test_another_viewers_session_is_not_this_viewers_to_stop(client, tmp_path, sessions):
theirs = _session(tmp_path, "sid-a")
sessions[(7, "m1")] = theirs
r = client.post("/api/plex/stream/m1/session/sid-a/stop", json={})
assert r.json() == {"ok": True, "stopped": False}
assert not theirs.proc.terminated
def test_a_stop_with_no_session_left_is_not_an_error(client, sessions):
"""The reaper or a cap may have dropped it first; the beacon must not fail loudly for that."""
r = client.post("/api/plex/stream/m1/session/sid-a/stop", json={})
assert r.status_code == 200, r.text
assert r.json() == {"ok": True, "stopped": False}
+7 -2
View File
@@ -695,7 +695,9 @@ class TestTranscodeStartSession:
# F2/race: start_session waits on seg_0 OUTSIDE _lock. If a concurrent same-key start (a rapid
# seek-restart) or _enforce_cap replaces _sessions[key] while we wait, our proc dying must NOT
# trigger the SW fallback — that would tear down the NEWER session and breach the transcode cap.
# The identity guard must detect it's been superseded and yield the currently-registered session.
# The identity guard must detect it's been superseded. It answers with its OWN (dead) session,
# never the newer one: the answer goes to the request that lost, whose player drops it as stale
# and stops the session it names — naming the newer one stopped the playback that won.
monkeypatch.setattr(
stream, "_select_encode", lambda db, item, transcode, src: (True, False)
)
@@ -708,9 +710,12 @@ class TestTranscodeStartSession:
newer = _Newer()
captured = {}
def wait_then_supersede(path, timeout=20.0, proc=None):
# Simulate the concurrent replacement happening during our out-of-lock wait, then report the
# (now-killed) HW proc produced nothing.
captured["s"] = stream._sessions[(7, "12345")]
stream._sessions[(7, "12345")] = newer
return False
@@ -720,7 +725,7 @@ class TestTranscodeStartSession:
)
# No SW respawn — only the initial HW _spawn ran; the newer session was left intact.
assert len(_CapturingProc.calls) == 1
assert s is newer # yielded the currently-registered session, did not clobber it
assert s is captured["s"] # its own session, not the newer one
assert stream._sessions[(7, "12345")] is newer
def test_a_cap_culled_hw_session_does_not_respawn_and_is_not_reported_missing(
+8 -2
View File
@@ -23,7 +23,10 @@ def client(monkeypatch, override_dep) -> TestClient:
return TestClient(app)
@pytest.mark.parametrize("path", ["/api/videos/v1/progress", "/api/plex/item/98614/progress"])
@pytest.mark.parametrize(
"path",
["/api/videos/v1/progress", "/api/plex/item/98614/progress", "/api/downloads/1/progress"],
)
@pytest.mark.parametrize("raw", ["Infinity", "-Infinity", "NaN"])
def test_a_non_finite_position_is_refused(client, path, raw):
r = client.post(
@@ -34,7 +37,10 @@ def test_a_non_finite_position_is_refused(client, path, raw):
assert r.status_code == 422, r.text
@pytest.mark.parametrize("path", ["/api/videos/v1/progress", "/api/plex/item/98614/progress"])
@pytest.mark.parametrize(
"path",
["/api/videos/v1/progress", "/api/plex/item/98614/progress", "/api/downloads/1/progress"],
)
@pytest.mark.parametrize("field", ["position_seconds", "duration_seconds"])
def test_a_position_past_the_int4_column_is_refused_not_a_500(client, path, field):
"""Finite but past the int4 column: the flush was a DataError, i.e. a 500 with a traceback."""
+4 -1
View File
@@ -20,7 +20,10 @@ _PW = "open sesame"
@pytest.fixture
def unlock_url(as_user, make_user, tmp_path, monkeypatch, upload_headers):
monkeypatch.setattr(settings, "download_root", str(tmp_path))
monkeypatch.setattr(probemod, "probe_media", lambda _p: ("h264", "aac", 320, 240, 5))
video = {"codec_type": "video", "codec_name": "h264", "pix_fmt": "yuv420p"}
audio = {"codec_type": "audio", "codec_name": "aac"}
data = {"streams": [video, audio], "format": {"format_name": "mov,mp4", "duration": "5"}}
monkeypatch.setattr(probemod, "probe_json", lambda _p: data)
owner = as_user(make_user())
job = owner.post(
"/api/downloads/upload", content=b"fake-video", headers=upload_headers("clip.mp4")
+4 -4
View File
@@ -7,7 +7,7 @@ from pydantic import ValidationError
from app.routes.channels import AttachTagIn, ChannelUpdateIn
from app.routes.downloads import AdminSetQuotaIn
from app.routes.feed import VideoProgressIn
from app.routes._common import ProgressIn
from app.routes.setup import SetupAdminIn
from app.routes.tags import TagUpdateIn
@@ -52,10 +52,10 @@ def test_admin_quota_rejects_non_numeric_limit():
def test_progress_accepts_fractional_seconds():
# The player may post a raw currentTime (a float). The field must accept it (truncated in the
# handler) — a plain `int` field would 422 on the progress hot path.
assert VideoProgressIn(position_seconds=12.7).position_seconds == 12.7
assert int(VideoProgressIn(position_seconds=12.7).position_seconds) == 12
assert ProgressIn(position_seconds=12.7).position_seconds == 12.7
assert int(ProgressIn(position_seconds=12.7).position_seconds) == 12
with pytest.raises(ValidationError):
VideoProgressIn(position_seconds="not-a-number")
ProgressIn(position_seconds="not-a-number")
# --- PATCH models: model_fields_set separates "absent" from "explicit null". ---
+67
View File
@@ -0,0 +1,67 @@
"""Which HTML page the server hands a path (main.py), DB-lane.
The frontend builds two entries: the app (index.html) and a shared link's player alone (watch.html), so
the friend a /watch link was sent to never downloads the app. Every /watch/ path must get the player's
page — with the link's Open Graph tags when the token names a playable link — and nothing else may.
"""
import pytest
from app import main
from app.config import settings
from app.downloads import probe as probemod
_PAGE = """<!doctype html><html><head>
<!--OG:START--><meta property="og:title" content="Siftlode" /><!--OG:END-->
</head><body><script src="/assets/{entry}.js"></script></body></html>"""
@pytest.fixture
def pages(tmp_path, monkeypatch):
index, watch = tmp_path / "index.html", tmp_path / "watch.html"
index.write_text(_PAGE.format(entry="app"), encoding="utf-8")
watch.write_text(_PAGE.format(entry="watch"), encoding="utf-8")
monkeypatch.setattr(main, "INDEX_HTML", index)
monkeypatch.setattr(main, "WATCH_HTML", watch)
@pytest.fixture
def link_token(as_user, make_user, tmp_path, monkeypatch, upload_headers):
monkeypatch.setattr(settings, "download_root", str(tmp_path / "media"))
video = {"codec_type": "video", "codec_name": "h264", "pix_fmt": "yuv420p"}
audio = {"codec_type": "audio", "codec_name": "aac"}
data = {"streams": [video, audio], "format": {"format_name": "mov,mp4", "duration": "5"}}
monkeypatch.setattr(probemod, "probe_json", lambda _p: data)
owner = as_user(make_user())
job = owner.post(
"/api/downloads/upload",
content=b"fake-video",
headers=upload_headers("clip.mp4", display_name="Holiday clip"),
)
assert job.status_code == 200, job.text
link = owner.post(f"/api/downloads/{job.json()['id']}/links", json={})
assert link.status_code == 200, link.text
return link.json()["token"]
@pytest.mark.parametrize("path", ["/", "/feed", "/downloads/42"])
def test_an_app_path_gets_the_app(pages, client, path):
r = client.get(path)
assert r.status_code == 200
assert "/assets/app.js" in r.text
@pytest.mark.parametrize("path", ["/watch/no-such-token", "/watch/no-such-token/extra"])
def test_every_watch_path_gets_the_player_alone(pages, client, path):
r = client.get(path)
assert r.status_code == 200
assert "/assets/watch.js" in r.text and "/assets/app.js" not in r.text
assert r.headers["cache-control"] == "no-cache"
def test_a_live_link_unfurls_from_the_players_page(pages, client, link_token):
r = client.get(f"/watch/{link_token}")
assert r.status_code == 200
assert "/assets/watch.js" in r.text
assert 'content="Holiday clip"' in r.text
assert '<meta property="og:title" content="Siftlode" />' not in r.text # replaced, not appended
+218 -27
View File
@@ -7,15 +7,18 @@ guards that keep an upload out of paths that assume a re-fetchable source (edit
DB-backed (real Postgres) — skipped when no database is reachable. `settings.download_root` is
pointed at a per-test tmp dir so ingest's on-disk moves are isolated. The ingest/route tests
monkeypatch the shared `probe.probe_media` so they don't shell out to ffprobe; the classifier's OWN
monkeypatch the shared `probe.probe_json` so they don't shell out to ffprobe; the classifier's OWN
parsing (image → file, cover-art skip, dims, duration) is covered directly by `TestProbeClassifier`
below, which stubs one level down (`run_ffprobe_json`) and exercises the real `probe_media`.
below, which stubs one level down (`run_ffprobe_json`) and exercises the real `probe_media`. The
phone-safe conversion of an upload runs the real ffmpeg (`TestPhoneSafeUpload`).
"""
import ast
import asyncio
import io
import pathlib
import shutil
import subprocess
from pathlib import Path
import pytest
@@ -25,9 +28,11 @@ from app.config import settings
from app.routes._common import capped_stream_to
from sqlalchemy import select
from app.downloads import mediakind, quota, storage
from app.downloads import gc as gcmod
from app.downloads import mediakind, pipeline, playable, quota, service, storage
from app.downloads import probe as probemod
from app.downloads import upload as uploadmod
from app.downloads.stated import StatedFailure
from app.models import DISPLAY_TEXT_MAX, DownloadJob, DownloadQuota, MediaAsset
@@ -51,9 +56,20 @@ def _root(tmp_path, monkeypatch):
def _probe(monkeypatch, result):
# upload ingest now calls the shared `probe.probe_media` (same ground-truth ffprobe the download
# worker uses); patch it at that shared home.
monkeypatch.setattr(probemod, "probe_media", lambda _p: result)
"""Stub the shared ffprobe call (`probe.probe_json`, the same ground truth the download worker
reads) with the JSON a file of these (vcodec, acodec, width, height, duration) would give — its
video 8-bit 4:2:0, so an H.264 + AAC .mp4 is already phone-safe and ingests straight to `done`.
All-None is a file ffprobe could not read."""
v, a, w, h, d = result
streams = []
if v:
streams.append(
{"codec_type": "video", "codec_name": v, "pix_fmt": "yuv420p", "width": w, "height": h}
)
if a:
streams.append({"codec_type": "audio", "codec_name": a})
data = {"streams": streams, "format": {"format_name": "mov,mp4", "duration": d}}
monkeypatch.setattr(probemod, "probe_json", lambda _p: data if streams else None)
def test_ingest_generic_file_is_download_only(db, make_user, _root, monkeypatch):
@@ -281,7 +297,7 @@ def test_upload_route_generic_forces_download(
def test_upload_route_media_html_extension_not_served_as_html(
as_user, make_user, _root, monkeypatch, upload_headers
db, as_user, make_user, _root, monkeypatch, upload_headers
):
"""The XSS guard: a MEDIA-classified upload (real codecs) named with an executable extension must
NOT be served with a guessed text/html type. The served Content-Type comes from the verified
@@ -298,6 +314,14 @@ def test_upload_route_media_html_extension_not_served_as_html(
assert r.status_code == 200, r.text
job = r.json()
assert job["asset"]["media_kind"] == "video" and job["asset"]["container"] == "html"
# Media in no phone-safe container is queued for conversion to .mp4, so today it is never served
# as .html at all. The serving guard below is the second line: a row from before that rule (or
# one whose conversion is skipped some day) must still not run as HTML.
assert job["status"] == "queued"
row = db.get(DownloadJob, job["id"])
db.get(MediaAsset, row.asset_id).status = "ready"
row.status = "done"
db.commit()
# Authenticated endpoint: never text/html.
fr = c.get(f"/api/downloads/{job['id']}/file")
@@ -427,7 +451,7 @@ def test_a_non_ascii_filename_survives_the_header_round_trip(
def test_a_long_display_name_is_truncated_on_EVERY_route_that_takes_one(
db, as_user, make_user, _root, monkeypatch, upload_headers
):
"""The sweep, held end to end rather than at one site.
"""The sweep, held end to end rather than at one site (the clip route joined it later).
`display_text` was introduced for the upload path and applied to three of the five writers of
this varchar(255) column; `POST /api/downloads` and `POST /api/downloads/edit` kept the old
@@ -479,15 +503,32 @@ def test_a_long_display_name_is_truncated_on_EVERY_route_that_takes_one(
assert patched.json()["display_name"] == bound
assert patched.json()["display_uploader"] == bound
# 6. the clip route — the clip panel's name field.
clip = c.post(
"/api/downloads/clip",
json={
"source": "youtube",
"ref": "dQw4w9WgXcQ",
"start_s": 0,
"end_s": 5,
"display_name": long_name,
},
)
assert clip.status_code == 200, clip.text
assert clip.json()["display_name"] == bound
def _display_write_sites() -> dict[tuple[str, str], set[str]]:
"""Every place under `backend/app/` that puts a value into a display field, as
(module relative to `app/`, enclosing function) -> the set of value expressions written there.
TWO SHAPES, because the first version of this guard walked only `ast.Assign` and was therefore
THREE SHAPES, because the first version of this guard walked only `ast.Assign` and was therefore
blind to the shape the upload route itself uses — `SpooledUpload(..., display_name=...)` is an
`ast.keyword`, not an assignment, so the one site with an unbounded REQUEST HEADER behind it was
the site the guard could not see.
the site the guard could not see. The third, `fields["display_name"] = …`, is a dict a finalize
call is spread from (the worker's), which the first two missed as well. A value passed
POSITIONALLY is still unseen: a writer binds it to a name or a keyword first, as every route here
does.
ANNOTATED assignments are excluded on purpose: `display_name: Mapped[str | None] =
mapped_column(...)` in `models.py` and `display_name: str | None = None` in a Pydantic model are
@@ -513,6 +554,13 @@ def _display_write_sites() -> dict[tuple[str, str], set[str]]:
t.id if isinstance(t, ast.Name) else t.attr
for t in node.targets
if isinstance(t, (ast.Name, ast.Attribute))
} | {
# `fields["display_name"] = …` — a dict the write is then spread from.
t.slice.value
for t in node.targets
if isinstance(t, ast.Subscript)
and isinstance(t.slice, ast.Constant)
and isinstance(t.slice.value, str)
}
value = node.value
elif isinstance(node, ast.keyword):
@@ -534,9 +582,12 @@ def _display_write_sites() -> dict[tuple[str, str], set[str]]:
# Bounded by `display_text` unless noted:
# service.redownload CLEARS the field so the fresh metadata wins — `None` is the
# one value that needs no bound, and a clear is still a write.
# service.enqueue / enqueue_edit `display_name` is already bounded by its caller; the fallback
# slices the YouTube title to the same width.
# service.enqueue / enqueue_edit / `display_name` is already bounded by its caller; the fallback
# enqueue_clip slices the source's title to the same width.
# downloads.upload_download passes through `spool_upload`'s value, bounded there.
# routes enqueue_clip the `display_name=` keyword passes on the name bound above it.
# pipeline._download / _process_clip / worker._finish_from_cache
# slice a fetched or stored title to the column's width.
#
# Held as EXACT EQUALITY rather than as a list of offenders. The offender form could only fail
# upward: refactoring the writers into a shape the walker did not recognise left it asserting
@@ -549,11 +600,16 @@ DISPLAY_WRITE_SITES = {
"if asset.status == 'ready' and asset.title else None)"
},
("downloads/service.py", "enqueue_edit"): {"display_name or asset.title[:DISPLAY_TEXT_MAX]"},
("downloads/service.py", "enqueue_clip"): {"display_name or asset.title[:DISPLAY_TEXT_MAX]"},
("downloads/pipeline.py", "_download"): {"meta.title[:DISPLAY_TEXT_MAX]"},
("downloads/pipeline.py", "_process_clip"): {"named[:DISPLAY_TEXT_MAX]"},
("worker.py", "_finish_from_cache"): {"asset.title[:DISPLAY_TEXT_MAX]"},
("downloads/upload.py", "ingest_upload"): {"display_text(display_name)"},
("routes/auth.py", "callback"): {"display_text(userinfo.get('name'))"},
("routes/auth.py", "_complete_link"): {"display_text(userinfo.get('name'))"},
("routes/downloads.py", "enqueue_download"): {"display_text(body.display_name)"},
("routes/downloads.py", "enqueue_edit"): {"display_text(body.display_name)"},
("routes/downloads.py", "enqueue_clip"): {"display_text(body.display_name)", "display_name"},
("routes/downloads.py", "update_download_meta"): {
"display_text(body.display_name)",
"display_text(body.display_uploader)",
@@ -779,10 +835,10 @@ def test_ingest_rejects_over_max_jobs(db, make_user, _root, monkeypatch):
def test_ingest_cleans_up_placed_files_on_commit_failure(db, make_user, _root, monkeypatch):
"""A failure AFTER storage.place_file has moved the file onto disk must roll back AND unlink the
placed file + any (even partial) poster — otherwise the orphan is permanent (the per-user delete
keys off the rolled-back asset.rel_path, and the GC orphan sweep skips the dot-prefixed .uploads/
tree). Exercises the rollback/cleanup branch that IS the leak remediation, incl. the poster case
where ensure_poster left a partial file but returned None."""
placed file + any (even partial) poster — otherwise nothing names it (the per-user delete keys
off the rolled-back asset.rel_path) and it sits on disk until the GC's `.uploads` sweep. Exercises
the rollback/cleanup branch, incl. the poster case where ensure_poster left a partial file but
returned None."""
_probe(monkeypatch, ("h264", "aac", 320, 240, 5)) # video → the poster path runs
u = make_user()
@@ -813,6 +869,28 @@ def test_ingest_cleans_up_placed_files_on_commit_failure(db, make_user, _root, m
assert db.query(MediaAsset).count() == 0 # the flushed asset rolled back — no dangling row
def test_ingest_keeps_the_file_when_its_commit_landed_but_raised(db, make_user, _root, monkeypatch):
# The commit reached the server and lost its answer: the row names the file now, so removing it
# as "rolled back" would leave a ready upload pointing at nothing.
_probe(monkeypatch, ("h264", "aac", 320, 240, 5))
monkeypatch.setattr(uploadmod.editmod, "ensure_poster", lambda *_a: None)
u = make_user()
real_commit = db.commit
def _landed_then_lost():
real_commit()
raise RuntimeError("connection lost after COMMIT")
monkeypatch.setattr(db, "commit", _landed_then_lost)
src = _root / "clip.mp4"
src.write_bytes(b"fake-video-bytes")
with pytest.raises(RuntimeError):
uploadmod.ingest_upload(db, u.id, filename="clip.mp4", src_path=src, size=16)
(asset,) = db.query(MediaAsset).all()
assert storage.abs_path(str(_root), asset.rel_path).exists()
# --- classifier: the REAL probe_media parsing (stubs ffprobe one level down) ------------------
# The ingest/route tests above stub probe_media wholesale; these exercise its OWN logic — image
# detection, the cover-art skip, dimension/duration parsing — by faking only run_ffprobe_json, so a
@@ -998,19 +1076,132 @@ class TestResolveMediaFacts:
)
assert v is None and mediakind.derive(v, a) == mediakind.FILE
def test_info_fills_in_only_when_ffprobe_could_not_read_the_file(self, monkeypatch):
# ffprobe failed entirely (None) → fall back to info so we never regress the working path.
def test_a_file_the_guarded_probe_cannot_read_is_a_download_only_file(self, monkeypatch):
# ffprobe (media demuxers only) failed entirely: the player and the editor cannot open it
# either, so info's codecs must not make it an editable 'video'. Dimensions still gap-fill.
self._ffprobe(monkeypatch, None)
v, a, w, h, _d = probemod.resolve_media_facts(
Path("x"), {"vcodec": "vp9", "acodec": "opus", "width": 1920, "height": 1080}
Path("x"),
{"vcodec": "avc1.64001f", "acodec": "mp4a.40.2", "width": 1920, "height": 1080},
)
assert (v, a, w, h) == ("vp9", "opus", 1920, 1080)
assert (v, a, w, h) == (None, None, 1920, 1080)
assert mediakind.derive(v, a) == mediakind.FILE
def test_info_none_codec_string_is_treated_as_absent(self, monkeypatch):
# yt-dlp writes the literal "none" for a missing stream — it must not become a codec name.
self._ffprobe(monkeypatch, None) # probe failed → info path, where the "none" scrub applies
v, a, w, h, _d = probemod.resolve_media_facts(
Path("x"), {"vcodec": "none", "acodec": "aac"}
# --- the phone-safe conversion (an iPhone's HEVC, a webm, an mkv) ------------------------------
def _queued_upload(db, make_user, root, monkeypatch, name="clip.mov"):
"""An upload no phone could play as it is: probed as HEVC, so ingest queues it."""
_probe(monkeypatch, ("hevc", "aac", 640, 360, 6))
src = root / "src.bin"
src.write_bytes(b"hevc bytes")
job = uploadmod.ingest_upload(db, make_user().id, filename=name, src_path=src, size=10)
return job, db.get(MediaAsset, job.asset_id)
class TestPhoneSafeUpload:
def test_an_unsafe_video_is_queued_for_the_worker_not_done(
self, db, make_user, _root, monkeypatch
):
job, asset = _queued_upload(db, make_user, _root, monkeypatch)
assert (job.status, job.progress, asset.status) == ("queued", 0, "pending")
assert asset.rel_path.endswith(".mov") # the original waits at rel_path for the worker
assert storage.abs_path(str(_root), asset.rel_path).exists()
def test_a_safe_video_still_lands_done(self, db, make_user, _root, monkeypatch):
_probe(monkeypatch, ("h264", "aac", 640, 360, 6))
src = _root / "src.bin"
src.write_bytes(b"h264 bytes")
job = uploadmod.ingest_upload(
db, make_user().id, filename="clip.mp4", src_path=src, size=10
)
assert (v, a) == (None, "aac")
assert mediakind.derive(v, a) == mediakind.AUDIO
assert job.status == "done" and db.get(MediaAsset, job.asset_id).status == "ready"
def test_deleting_a_queued_upload_removes_its_original(self, db, make_user, _root, monkeypatch):
job, asset = _queued_upload(db, make_user, _root, monkeypatch)
path, asset_id = storage.abs_path(str(_root), asset.rel_path), asset.id
service.delete(db, job)
assert not path.exists()
assert db.get(MediaAsset, asset_id) is None
def test_one_cancelled_mid_conversion_is_reaped_once_the_worker_let_go(
self, db, make_user, _root, monkeypatch
):
job, asset = _queued_upload(db, make_user, _root, monkeypatch)
path, asset_id = storage.abs_path(str(_root), asset.rel_path), asset.id
asset.status = "downloading" # the worker child is reading the original
job.status = "running"
db.commit()
service.cancel(db, job)
assert path.exists() # never pulled out from under a running ffmpeg
db.get(MediaAsset, asset_id).status = "pending" # the worker stopped the child and let go
db.commit()
gcmod.run_download_gc(db)
db.expire_all()
assert not path.exists()
assert db.get(MediaAsset, asset_id) is None
def test_a_failed_conversion_keeps_the_original_for_retry(
self, db, make_user, _root, monkeypatch
):
job, asset = _queued_upload(db, make_user, _root, monkeypatch)
path = storage.abs_path(str(_root), asset.rel_path)
job.status, asset.status = "running", "downloading"
db.commit()
def _ffmpeg_fails(*_a, **_k):
raise StatedFailure("ffmpeg failed (1): unsupported")
monkeypatch.setattr(pipeline.editmod, "run_ffmpeg", _ffmpeg_fails)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
assert (db.get(DownloadJob, job.id).status, db.get(MediaAsset, asset.id).status) == (
"error",
"error",
)
assert path.exists()
service.resume(db, db.get(DownloadJob, job.id))
assert db.get(MediaAsset, asset.id).status == "pending" # Retry converts from it again
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="ffmpeg not available")
def test_the_worker_converts_it_and_keeps_only_the_safe_file(
self, db, make_user, _root, monkeypatch
):
# Real bytes, real probe, real ffmpeg: a 10-bit H.264 + Opus Matroska, the kind of file
# that reached the share page as-is before this rule.
src = _root / "src.mkv"
subprocess.run(
[
"ffmpeg", "-y", "-loglevel", "error",
"-f", "lavfi", "-i", "testsrc2=size=160x120:rate=10:duration=1",
"-f", "lavfi", "-i", "sine=duration=1",
"-c:v", "libx264", "-pix_fmt", "yuv420p10le", "-c:a", "libopus", str(src),
],
check=True,
) # fmt: skip
job = uploadmod.ingest_upload(
db, make_user().id, filename="holiday.mkv", src_path=src, size=src.stat().st_size
)
asset = db.get(MediaAsset, job.asset_id)
original = storage.abs_path(str(_root), asset.rel_path)
assert job.status == "queued"
job.status, asset.status = "running", "downloading" # what the worker's claim does
db.commit()
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
asset = db.get(MediaAsset, job.asset_id)
assert db.get(DownloadJob, job.id).status == "done"
assert (asset.status, asset.container, asset.vcodec, asset.acodec) == (
"ready",
"mp4",
"h264",
"aac",
)
final = storage.abs_path(str(_root), asset.rel_path)
assert final.suffix == ".mp4" and final.exists()
assert asset.size_bytes == final.stat().st_size
assert not original.exists() # one file per upload
assert not playable.plan(probemod.probe_json(final), final.suffix).needs_change
+622
View File
@@ -0,0 +1,622 @@
"""An upload after ingest: what it costs its owner from the moment it lands, how the worker's
conversion names, checks and cleans up the file it makes, and what the owner and a share link see.
DB-backed. The conversion's ffmpeg step (`pipeline._ensure_phone_safe`) is stubbed to write a file of
a chosen size — these tests are about what happens around the encode; `test_upload.py` runs the real
one, and `test_playable.py` judges its bytes.
"""
import os
import time
from datetime import datetime, timedelta, timezone
from pathlib import Path
import pytest
from app import sysconfig
from app.config import settings
from app.downloads import errors as dlerrors
from app.downloads import gc as gcmod
from app.downloads import pipeline, quota, storage
from app.downloads import probe as probemod
from app.downloads import upload as uploadmod
from app.downloads.stated import StatedFailure
from app.models import DownloadJob, DownloadQuota, MediaAsset, Notification
from sqlalchemy import select, update
from sqlalchemy.exc import OperationalError
from sqlalchemy.orm import Session
@pytest.fixture
def _root(tmp_path, monkeypatch):
monkeypatch.setattr(settings, "download_root", str(tmp_path))
return tmp_path
def _probe(monkeypatch, vcodec="hevc"):
"""ffprobe's view of every file here: `vcodec` + AAC, 8-bit 4:2:0 — HEVC is queued for the
worker, H.264 lands as it came."""
data = {
"streams": [
{"codec_type": "video", "codec_name": vcodec, "pix_fmt": "yuv420p",
"width": 640, "height": 360},
{"codec_type": "audio", "codec_name": "aac"},
],
"format": {"format_name": "mov,mp4", "duration": "6"},
} # fmt: skip
monkeypatch.setattr(probemod, "probe_json", lambda _p: data)
def _ingest(db, root, user_id, name="clip.mov", size=10):
src = root / "incoming.bin"
src.write_bytes(b"o" * size)
job = uploadmod.ingest_upload(db, user_id, filename=name, src_path=src, size=size)
return job, db.get(MediaAsset, job.asset_id)
def _claim(db, job, asset):
"""What the worker's claim does before it converts."""
job.status, asset.status = "running", "downloading"
db.commit()
def _converts_to(monkeypatch, size, phases=None):
"""The encode writes a `size`-byte .mp4 into its staging dir (and notes the phase it reports)."""
def _encode(job_id, produced, out_dir, phase, total_s=None, **_k):
if phases is not None:
phases.append(phase)
out = Path(out_dir) / "converted.mp4"
out.write_bytes(b"c" * size)
return out
monkeypatch.setattr(pipeline, "_ensure_phone_safe", _encode)
def _cap(db, user_id, max_bytes):
db.add(
DownloadQuota(
user_id=user_id, max_bytes=max_bytes, max_concurrent=2, max_jobs=50, unlimited=False
)
)
db.commit()
def _uploads_dir(root, user_id) -> Path:
return root / ".uploads" / str(user_id)
# --- what it costs ---------------------------------------------------------------------------
def test_a_queued_upload_counts_against_its_owner_from_the_moment_it_lands(
db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id, size=10)
assert (job.status, asset.status) == ("queued", "pending")
assert quota.footprint(db, u.id) == 10 # its original is on disk already
def test_a_failed_conversion_still_counts_its_kept_original(db, make_user, _root, monkeypatch):
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id, size=10)
job.status, asset.status = "error", "error" # kept for Retry
db.commit()
assert storage.abs_path(str(_root), asset.rel_path).exists()
assert quota.footprint(db, u.id) == 10
def test_a_conversion_that_outgrows_the_cap_is_refused_and_keeps_the_original(
db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
u = make_user()
_cap(db, u.id, max_bytes=100)
job, asset = _ingest(db, _root, u.id, size=10)
original = storage.abs_path(str(_root), asset.rel_path)
_claim(db, job, asset)
_converts_to(monkeypatch, size=500)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
job = db.get(DownloadJob, job.id)
assert (job.status, job.error_code) == ("error", "quota_exceeded")
# 90 of the 100 are still free: the converted file is too big, the storage is not full.
assert job.error == dlerrors.CONVERTED_TOO_BIG_MESSAGE
assert original.exists()
assert [p.name for p in _uploads_dir(_root, u.id).iterdir()] == [original.name]
def test_a_conversion_that_shrinks_passes_an_owner_at_the_cap(db, make_user, _root, monkeypatch):
_probe(monkeypatch)
u = make_user()
_cap(db, u.id, max_bytes=10)
job, asset = _ingest(db, _root, u.id, size=10)
_claim(db, job, asset)
_converts_to(monkeypatch, size=5)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
assert db.get(DownloadJob, job.id).status == "done"
assert db.get(MediaAsset, asset.id).size_bytes == 5
# --- the converted file's name and its leftovers -----------------------------------------------
def test_two_dotted_bare_names_keep_a_file_each(db, make_user, _root, monkeypatch):
# 'Holiday.2024.ä': its suffix is no extension (nothing shell-safe is left of it), so the stem's
# own dot once read as one — and the converted name lost `_a{id}`, the one thing keeping two apart.
_probe(monkeypatch)
u = make_user()
_converts_to(monkeypatch, size=5)
finals = []
for _ in range(2):
job, asset = _ingest(db, _root, u.id, name="Holiday.2024.ä")
_claim(db, job, asset)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
asset = db.get(MediaAsset, asset.id)
assert asset.status == "ready"
assert asset.rel_path.endswith(f"_a{asset.id}.mp4"), asset.rel_path
finals.append(asset.rel_path)
assert finals[0] != finals[1]
assert all(storage.abs_path(str(_root), r).exists() for r in finals)
def test_a_bare_names_stem_holds_no_dot_so_a_swapped_suffix_keeps_its_id():
# Each of the two name rules is held on its own: a new name's stem has no dot, so a sibling
# derived by `with_suffix` (the poster, the converted file) keeps `_a{id}`.
rel = storage.upload_rel_path(1, 5, "Holiday.2024.ä")
assert rel == ".uploads/1/Holiday_2024_a5"
assert Path(rel).with_suffix(".jpg").name == "Holiday_2024_a5.jpg"
def test_an_upload_placed_with_a_dotted_bare_name_keeps_its_id_when_converted(
db, make_user, _root, monkeypatch
):
# A row from before the stems lost their dots: its stem's tail is no extension.
_probe(monkeypatch)
u = make_user()
_converts_to(monkeypatch, size=5)
job, asset = _ingest(db, _root, u.id, name="Holiday.2024.ä")
legacy = f".uploads/{u.id}/Holiday.2024_a{asset.id}"
storage.abs_path(str(_root), asset.rel_path).rename(storage.abs_path(str(_root), legacy))
asset.rel_path, asset.container = legacy, None
_claim(db, job, asset)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
assert db.get(MediaAsset, asset.id).rel_path == f"{legacy}.mp4"
def test_a_conversion_failing_after_it_placed_its_file_removes_it(
db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id)
original = storage.abs_path(str(_root), asset.rel_path)
_claim(db, job, asset)
_converts_to(monkeypatch, size=5)
def _probe_fails(_path):
raise RuntimeError("probe crashed")
monkeypatch.setattr(pipeline.probe, "probe_media", _probe_fails)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
assert db.get(DownloadJob, job.id).status == "error"
assert original.exists() # Retry starts from it
# The row still names the original: a converted file left beside it is one nothing would find.
assert [p.name for p in _uploads_dir(_root, u.id).iterdir()] == [original.name]
def _finishing_commit_raises(monkeypatch, *, lands: bool) -> None:
"""The commit that makes the converted upload ready raises — after it reached the server
(`lands`: the connection dropped before its answer) or before (it rolled back)."""
real_finalize, real_commit = pipeline.service.finalize_if_running, Session.commit
state = {"session": None, "fired": False}
def _finalize(db, *a, **k):
if not state["fired"] and state["session"] is None:
state["session"] = db
return real_finalize(db, *a, **k)
def _commit(self):
if self is state["session"] and not state["fired"]:
state["fired"] = True
if lands:
real_commit(self)
raise OperationalError("COMMIT", {}, Exception("connection lost"))
return real_commit(self)
monkeypatch.setattr(pipeline.service, "finalize_if_running", _finalize)
monkeypatch.setattr(Session, "commit", _commit)
def test_a_finishing_commit_that_landed_but_raised_keeps_the_converted_file(
db, make_user, _root, monkeypatch
):
# The row names the converted file now: removing it as "never committed" would leave the asset
# pointing at nothing and its original named by no row, for the sweep to take — both copies gone.
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id)
_claim(db, job, asset)
_converts_to(monkeypatch, size=5)
_finishing_commit_raises(monkeypatch, lands=True)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
asset = db.get(MediaAsset, asset.id)
assert (db.get(DownloadJob, job.id).status, asset.status) == ("done", "ready")
assert storage.abs_path(str(_root), asset.rel_path).exists()
def test_a_finishing_commit_that_rolled_back_fails_and_keeps_the_original(
db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id)
original = storage.abs_path(str(_root), asset.rel_path)
_claim(db, job, asset)
_converts_to(monkeypatch, size=5)
_finishing_commit_raises(monkeypatch, lands=False)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
assert db.get(DownloadJob, job.id).status == "error"
assert [p.name for p in _uploads_dir(_root, u.id).iterdir()] == [original.name]
def test_an_original_that_cannot_be_removed_leaves_the_upload_ready(
db, make_user, _root, monkeypatch
):
# The converted file is committed and the job done: the original's unlink failing must not flip
# a working asset to an error no Retry is offered for. The sweep takes the original later.
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id)
original = storage.abs_path(str(_root), asset.rel_path)
_claim(db, job, asset)
_converts_to(monkeypatch, size=5)
real_unlink = Path.unlink
def _unlink(self, missing_ok=False):
if self == original:
raise PermissionError("read-only")
return real_unlink(self, missing_ok=missing_ok)
monkeypatch.setattr(Path, "unlink", _unlink)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
asset = db.get(MediaAsset, asset.id)
assert (db.get(DownloadJob, job.id).status, asset.status) == ("done", "ready")
assert original.exists()
# --- what the owner is told ------------------------------------------------------------------
def test_a_failed_encode_is_a_failed_conversion_not_a_download_step(
db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
job, asset = _ingest(db, _root, make_user().id)
_claim(db, job, asset)
def _ffmpeg_fails(*_a, **_k):
raise StatedFailure("ffmpeg failed (1): unsupported")
monkeypatch.setattr(pipeline.editmod, "run_ffmpeg", _ffmpeg_fails)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
assert db.get(DownloadJob, job.id).error_code == "convert_failed"
def test_a_missing_original_says_so(db, make_user, _root, monkeypatch):
_probe(monkeypatch)
job, asset = _ingest(db, _root, make_user().id)
_claim(db, job, asset)
storage.abs_path(str(_root), asset.rel_path).unlink()
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
assert db.get(DownloadJob, job.id).error_code == "upload_missing"
def test_the_conversion_reports_its_own_phase(db, make_user, _root, monkeypatch):
_probe(monkeypatch)
job, asset = _ingest(db, _root, make_user().id)
_claim(db, job, asset)
phases = []
_converts_to(monkeypatch, size=5, phases=phases)
pipeline._convert_upload(job.id, asset.id)
assert phases == ["converting"] # not yt-dlp's "processing": the queue row shows ffmpeg's %
def test_the_owner_is_told_their_upload_is_ready(db, make_user, _root, monkeypatch):
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id)
_claim(db, job, asset)
_converts_to(monkeypatch, size=5)
pipeline._convert_upload(job.id, asset.id)
notes = db.execute(select(Notification).where(Notification.user_id == u.id)).scalars().all()
assert [n.data.get("kind") for n in notes] == ["upload"]
def test_an_extension_is_judged_as_it_is_stored(db, make_user, _root, monkeypatch):
# 'clip.mp4~' is stored as .mp4; judged by the raw 'mp4~' it queued a conversion of a file that
# was phone-safe already.
_probe(monkeypatch, vcodec="h264")
job, asset = _ingest(db, _root, make_user().id, name="clip.mp4~")
assert (job.status, asset.status, asset.container) == ("done", "ready", "mp4")
# --- the garbage collector ---------------------------------------------------------------------
def _aged(path: Path, hours: float = 5) -> None:
old = time.time() - hours * 3600
os.utime(path, (old, old))
def test_the_sweep_takes_a_stranded_upload_file_and_keeps_a_waiting_original(
db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
u = make_user()
_job, asset = _ingest(db, _root, u.id)
original = storage.abs_path(str(_root), asset.rel_path)
stranded = original.with_name("left_by_a_stopped_conversion_a999.mp4")
stranded.write_bytes(b"x")
_aged(original)
_aged(stranded)
gcmod._sweep_orphan_files(db, str(_root))
assert original.exists() # queued for conversion: its row names it, ready or not
assert not stranded.exists() # no row names it
@pytest.mark.parametrize(
("job_status", "asset_status"),
[
("queued", "pending"), # waiting for the worker
("running", "downloading"), # being converted, for longer than the sweep's age floor
("error", "error"), # failed, kept for Retry
("done", "ready"), # converted
],
)
def test_the_sweep_keeps_an_aged_upload_file_its_row_names_in_any_state(
job_status, asset_status, db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
job, asset = _ingest(db, _root, make_user().id)
path = storage.abs_path(str(_root), asset.rel_path)
job.status, asset.status = job_status, asset_status
db.commit()
_aged(path)
gcmod._sweep_orphan_files(db, str(_root))
assert path.exists()
def _row(db, asset_id: int) -> MediaAsset | None:
"""The asset as the database has it now (a `get` would refresh the session's own copy, and
raise for one deleted under it)."""
db.expire_all()
return db.execute(select(MediaAsset).where(MediaAsset.id == asset_id)).scalar_one_or_none()
def _route_moved(db, job, asset, job_status: str, ref_count: int) -> None:
"""What the cancel/pause/delete route leaves behind, written as SQL: the session's copy of
`ref_count` is stale (the routes count it in SQL), so an ORM assignment of the same value would
write nothing."""
db.execute(update(DownloadJob).where(DownloadJob.id == job.id).values(status=job_status))
db.execute(update(MediaAsset).where(MediaAsset.id == asset.id).values(ref_count=ref_count))
db.commit()
def _stopped_by(monkeypatch, db, job, asset, job_status: str, ref_count: int) -> None:
"""The conversion is stopped mid-encode after the route moved the job to `job_status` and its
hold on the asset to `ref_count` (cancel and delete release it; pause keeps it)."""
def _encode(*_a, **_k):
_route_moved(db, job, asset, job_status, ref_count)
raise pipeline.editmod.EditAborted()
monkeypatch.setattr(pipeline, "_ensure_phone_safe", _encode)
def test_an_upload_cancelled_mid_conversion_is_freed_once_the_conversion_stops(
db, make_user, _root, monkeypatch
):
# `release_asset` leaves it while the conversion reads the file; once it stopped, nothing holds it
# — kept, it would sit on disk outside every footprint until the GC's upload pass.
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id, size=10)
original = storage.abs_path(str(_root), asset.rel_path)
_claim(db, job, asset)
_stopped_by(monkeypatch, db, job, asset, "canceled", 0)
pipeline._convert_upload(job.id, asset.id)
assert _row(db, asset.id) is None
assert not original.exists()
assert quota.footprint(db, u.id) == 0
def test_an_upload_paused_mid_conversion_keeps_its_original_for_the_resume(
db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id, size=10)
original = storage.abs_path(str(_root), asset.rel_path)
_claim(db, job, asset)
_stopped_by(monkeypatch, db, job, asset, "paused", 1)
pipeline._convert_upload(job.id, asset.id)
db.expire_all()
assert db.get(MediaAsset, asset.id).status == "pending"
assert original.exists()
def test_an_upload_whose_conversion_finished_after_its_job_was_cancelled_is_freed(
db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
u = make_user()
job, asset = _ingest(db, _root, u.id, size=10)
_claim(db, job, asset)
_converts_to(monkeypatch, size=5)
real_encode = pipeline._ensure_phone_safe
def _encode_then_cancelled(*a, **k):
out = real_encode(*a, **k)
_route_moved(db, job, asset, "canceled", 0)
return out
monkeypatch.setattr(pipeline, "_ensure_phone_safe", _encode_then_cancelled)
pipeline._convert_upload(job.id, asset.id)
assert _row(db, asset.id) is None
left = _uploads_dir(_root, u.id)
assert not left.exists() or list(left.iterdir()) == []
def test_the_cache_cap_never_evicts_an_upload(db, make_user, _root, monkeypatch):
_probe(monkeypatch, vcodec="h264")
u = make_user()
_job, upload = _ingest(db, _root, u.id, name="mine.mp4", size=10)
db.add(
MediaAsset(
source_kind="youtube", source_ref="vid1", format_sig="best", status="ready",
rel_path="Chan/clip.mp4", size_bytes=10, title="downloaded",
last_access_at=datetime.now(timezone.utc),
)
) # fmt: skip
# The upload is the least recently used: an LRU pass that may take it takes it first.
upload.last_access_at = datetime.now(timezone.utc) - timedelta(days=30)
db.commit()
(_root / "Chan").mkdir()
(_root / "Chan" / "clip.mp4").write_bytes(b"d" * 10)
sysconfig.set_value(db, "download_total_max_bytes", 15) # 20 on disk
db.commit()
gcmod.run_download_gc(db)
db.expire_all()
assert db.get(MediaAsset, upload.id).status == "ready"
assert storage.abs_path(str(_root), upload.rel_path).exists()
# --- a share link ----------------------------------------------------------------------------
@pytest.mark.parametrize(
("job_status", "asset_status", "reason"),
[
("queued", "pending", "video_preparing"), # a re-download waiting for the worker
("running", "downloading", "video_preparing"), # being made now
("canceled", "pending", "video_gone"), # a rebuild cancelled: nothing will finish it
("paused", "pending", "video_gone"), # held until its owner resumes it, maybe never
],
)
def test_a_link_says_preparing_only_while_its_video_is_being_made(
job_status, asset_status, reason, db, as_user, make_user, _root, monkeypatch, upload_headers
):
# A link is made on a finished job; its asset can be (re)made under it later — a shared cache row
# re-fetched, a conversion retried. While that runs the page says "try again shortly"; once
# nothing is making it, it says the video is gone rather than promising it forever.
_probe(monkeypatch, vcodec="h264")
c = as_user(make_user())
job = c.post(
"/api/downloads/upload", content=b"h264 bytes", headers=upload_headers("clip.mp4")
).json()
link = c.post(f"/api/downloads/{job['id']}/links", json={"allow_download": False}).json()
assert c.get(f"/api/public/watch/{link['token']}").status_code == 200
db.get(MediaAsset, job["asset"]["id"]).status = asset_status
db.get(DownloadJob, job["id"]).status = job_status
db.commit()
r = c.get(f"/api/public/watch/{link['token']}")
assert r.status_code == 404
assert r.json()["detail"]["reason"] == reason
def test_a_link_to_a_suspended_owners_waiting_rebuild_says_it_is_gone(
db, as_user, make_user, _root, monkeypatch, upload_headers
):
# The worker leaves a suspended user's queued jobs where they are, maybe for good: like a paused
# rebuild, nothing is making the video, so "try again shortly" would never come true.
_probe(monkeypatch, vcodec="h264")
owner = make_user()
c = as_user(owner)
job = c.post(
"/api/downloads/upload", content=b"h264 bytes", headers=upload_headers("clip.mp4")
).json()
link = c.post(f"/api/downloads/{job['id']}/links", json={"allow_download": False}).json()
db.get(MediaAsset, job["asset"]["id"]).status = "pending"
db.get(DownloadJob, job["id"]).status = "queued"
owner.is_suspended = True
db.commit()
r = c.get(f"/api/public/watch/{link['token']}")
assert r.status_code == 404
assert r.json()["detail"]["reason"] == "video_gone"
# A rebuild already running when the owner was suspended still finishes.
db.get(DownloadJob, job["id"]).status = "running"
db.commit()
assert c.get(f"/api/public/watch/{link['token']}").json()["detail"]["reason"] == (
"video_preparing"
)
def test_the_admin_storage_view_counts_a_waiting_upload(db, as_user, make_user, _root, monkeypatch):
# The same disk the owner's footprint and the GC's cap count (`quota.ON_DISK`).
_probe(monkeypatch)
u = make_user()
_ingest(db, _root, u.id, size=10)
r = as_user(make_user(role="admin")).get("/api/admin/downloads/storage").json()
assert (r["files_on_disk"], r["total_bytes"]) == (1, 10)
assert {p["user_id"]: p["footprint_bytes"] for p in r["per_user"]} == {u.id: 10}
@pytest.mark.parametrize(
("job_status", "asset_status", "ref_count"),
[
("done", "ready", 1), # a finished upload its owner keeps
("queued", "pending", 1), # one waiting for the worker
("canceled", "downloading", 0), # cancelled mid-conversion: the worker still reads it
],
)
def test_the_upload_reap_leaves_one_still_held_or_still_read(
job_status, asset_status, ref_count, db, make_user, _root, monkeypatch
):
_probe(monkeypatch)
job, asset = _ingest(db, _root, make_user().id)
path = storage.abs_path(str(_root), asset.rel_path)
asset.status = asset_status
db.commit()
_route_moved(db, job, asset, job_status, ref_count)
gcmod.run_download_gc(db)
assert _row(db, asset.id) is not None
assert path.exists()
+74
View File
@@ -0,0 +1,74 @@
"""The worker child's dispatch (`runner._run`) and the edit path's last step before it stores a clip.
`runner._run` picks the pipeline entry by the job's kind; the tests elsewhere call those entries
directly, so a kind routed to the wrong one (an upload sent to yt-dlp) would pass all of them.
"""
from pathlib import Path
import pytest
from app.config import settings
from app.downloads import pipeline, runner, service, storage
from app.downloads import probe as probemod
from app.downloads import upload as uploadmod
from app.models import DownloadJob, MediaAsset
@pytest.mark.parametrize(
("kind", "entry"),
[("upload", "_convert_upload"), ("edit", "_process_edit"), ("download", "_download")],
)
def test_each_job_kind_reaches_its_own_pipeline_entry(kind, entry, monkeypatch):
called = []
for name in ("_convert_upload", "_process_edit", "_download"):
monkeypatch.setattr(pipeline, name, lambda *a, _n=name: called.append(_n))
runner._run(5, 7, kind, "url", "https://example.org/v", {}, 1)
assert called == [entry]
def test_a_clip_the_phone_safe_step_changed_is_stored_under_its_new_extension(
db, make_user, tmp_path, monkeypatch
):
# A fast trim stream-copies its source's container; when the phone-safe step remuxes it (an
# audio clip out of an .mp4 becomes an .m4a), the stored path and container follow the new file.
monkeypatch.setattr(settings, "download_root", str(tmp_path))
data = {
"streams": [{"codec_type": "audio", "codec_name": "aac"}],
"format": {"format_name": "mov,mp4", "duration": "10"},
}
monkeypatch.setattr(probemod, "probe_json", lambda _p: data)
src = tmp_path / "in.bin"
src.write_bytes(b"a" * 10)
u = make_user()
source = uploadmod.ingest_upload(db, u.id, filename="song.m4a", src_path=src, size=10)
# Stored as an .mp4, the way an audio file was kept before every ready file met the rule.
kept = db.get(MediaAsset, source.asset_id)
old = storage.abs_path(str(tmp_path), kept.rel_path)
kept.rel_path, kept.container = kept.rel_path[: -len(".m4a")] + ".mp4", "mp4"
old.rename(storage.abs_path(str(tmp_path), kept.rel_path))
db.commit()
job = service.enqueue_edit(
db, u.id, source, {"trim": {"start_s": 1, "end_s": 3}, "accurate": False}
)
asset = db.get(MediaAsset, job.asset_id)
job.status, asset.status = "running", "downloading"
db.commit()
def _trim(cmd, *_a, **_k):
Path(cmd[-1]).write_bytes(b"t" * 4) # the stream-copied clip, at the path ffmpeg was given
def _remux(job_id, produced, out_dir, phase, total_s=None, **_k):
out = Path(out_dir) / "remuxed.m4a"
out.write_bytes(b"r" * 3)
return out
monkeypatch.setattr(pipeline.editmod, "run_ffmpeg", _trim)
monkeypatch.setattr(pipeline, "_ensure_phone_safe", _remux)
pipeline._process_edit(job.id, asset.id)
db.expire_all()
asset = db.get(MediaAsset, asset.id)
assert (db.get(DownloadJob, job.id).status, asset.container) == ("done", "m4a")
assert asset.rel_path.endswith(".m4a")
assert storage.abs_path(str(tmp_path), asset.rel_path).read_bytes() == b"rrr"
+180 -112
View File
@@ -112,7 +112,18 @@ style guide.
own chip), never `--warn`: a score is not a warning. What sits on a picture looks the same in both themes,
because the picture does: the broadcast red `--live` under `--live-fg` (never `--danger`: "live" is not an
error), `--rating-media` on a poster's scrim, and `--media-chrome` for the always-dark player's menus
(owner decision F16). Black and white are the media scrims and their text. Five schemes × dark
(owner decision F16). On that chrome `--media-mark` is a marked place (the playhead, the crop frame, a
Plex marker), the focus ring and what is chosen, and `--media-danger` an error, as a light theme's
accent and danger are dark on the chrome: `media-marks` re-points `--focus` and `--chosen` on the
player (the Plex player, the YouTube stage) and on every surface painted `--media-chrome` (its menus,
panels and bars), and a form there (`media-scope`) re-points `--danger` too. A chosen item there is
white like the rest, marked by an amber tick (a track) or underline (a tab, `ui/mediaChrome`), never
the accent as text; a choice filled with the accent keeps it, its own text holds on it. A fill that
must be seen there — a native checkbox or slider (`accent-chosen`), the skip button's countdown — is
the amber too; the Plex progress bar plays white with an amber head, so an amber marker ahead of
the head never reads as already played.
`theme.contract.test.ts` measures the red on the chrome and holds every class string that paints it
to name one of the two. Black and white are the media scrims and their text. Five schemes × dark
and light (B2): **Midnight** (blue), **Forest** (cyan), **Slate** (fuchsia), **Matrix** (moss) each earn their
place with their own hue family; **Fjord** (Midnight's navy with a calm steel-blue accent) is the one kept
exception, the calm twin of the vivid Midnight. A new scheme still needs its own hue family. Normal text ≥ 4.5:1 and a control's edge ≥ 3:1 in **every** scheme and mode.
@@ -135,9 +146,11 @@ style guide.
- **State layers (B3):** hover and pressed are a layer, not a colour swap: the text colour mixed at **10%**
(hover) / **16%** (pressed) over whatever the control sits on, so it reads on the page, a card and glass
alike. A filled control mixes its own text colour into its fill at 14% / 24%. Opacity dimming and
`bg-card` hovers are violations (invisible on a light card). Selected stays the solid accent. The one
exception is an edged pill (a chip, a toolbar or form trigger, a segment): its hover is the accent edge
(`.pill:hover`, `glass-hover`), which reads on every surface too (owner decision F14).
`bg-card` hovers are violations (invisible on a light card). Selected stays the solid accent. The
exceptions are an edged pill (a chip, a toolbar or form trigger, a segment): its hover is the accent edge
(`.pill:hover`, `glass-hover`), which reads on every surface too (owner decision F14); and a dark
translucent button on a picture (the `media` variant: a card's hover row, a rail's scroll arrows, Restart),
whose hover is a 2px accent ring over a darker body, since a layer is lost in the picture.
A card that is one row with actions answers as a whole: the layer is the card's, under its badges and
buttons, and the row's open button spans the card beneath the actions (a hover that stopped at the first
badge read as broken; owner, E28 — `AdminUsers` user rows).
@@ -398,7 +411,9 @@ pick a density step; it never changes the anatomy or loses an action. [Nielsen #
`mediaTitleClass`, `mediaPictureBox`).
- **Play only where the click plays.** The hover layer dims the picture and shows `MediaPlayGlyph`, the
one round media Play (the button family's `media` variant), only on a card whose click starts
playback; a show without a next episode opens instead and shows none. The hover row (watched, save or
playback. Every Plex picture plays, a show too: the episode in progress (Continue + Restart, offered only
when an episode holds a resume point), else the first unwatched, else the first, which the click asks the server for (`/show/{id}/next`); its title
and an ⓘ open the show's page, and one with no episodes opens there. The hover row (watched, save or
add, "…") docks at the picture's bottom-right; the length badge steps aside while it shows.
- **Density.** A grid card: a two-line title and the full meta (Plex: year · ★ score; a video: the
channel, then views · age). On a shelf (any horizontal strip) the title is one line, cut with an
@@ -411,7 +426,8 @@ pick a density step; it never changes the anatomy or loses an action. [Nielsen #
had become the strip's.
- **The preview (the owner's pick C).** Resting the mouse on a Plex card for ~0.9 s opens a panel beside
it: the whole title, year · length · ★, and the synopsis in short lines (`ui/MediaPreview` +
`useMediaPreview`). Mouse only (a keyboard walk or a tap never opens it; "…" → Media info has the same),
`useMediaPreview`); a show's adds the episode it would play ("Next: S6·E16 — Title"), asked as the
panel opens. Mouse only (a keyboard walk or a tap never opens it; "…" → Media info has the same),
passive (it never takes a click), and any press on the card closes it. Long text never goes in a
native `title` tooltip.
- **Home's hero is a card** (the owner's pick A): the same hover row and Play on its picture, the same
@@ -528,10 +544,48 @@ of a full-width panel and nothing on the right.
**Restart**, the video card's own pair and words (`MediaResumeButtons`, one component for every card and
Home's hero). A poster stacks the pair; a landscape picture sets it side by side, icon-only on a small one.
On touch a tap continues and the card's "…" has "Play from the beginning" under its Play row. The restart
applies to the item the player opened on (`fromStart`); every item it moves on to resumes.
- **The YouTube modal:** its chrome uses the `Button` rungs (44 on touch), the side arrows stay inside the
screen, and the description opens from an ⓘ button, because a hover popover never opens on a tap (measured
live). [WCAG 1.4.13]
applies to the item the player opened on (`fromStart`); every item it moves on to resumes. A library
row is not a card and keeps its one play button: started, that button continues, its "…" has Continue +
"Play from the beginning", and its picture carries the cards' progress bar.
- **The YouTube modal:** its chrome uses the `Button` rungs (44 on touch); Previous / Next live in the panel
(its transport row, or the shut panel's strip) and on Shift+← / →, with no strips beside the card. Close is
an icon at the top right of the panel (wide: its head, or its strip) and at the end of the title row
under the video (stacked). Wide, the panel shows the description outright, clamped to three
lines with a More (`ClampedText`; a Tab onto a link below the cut opens it), and collapses to a 2.75rem
strip of close, expand and Previous / Next, kept shut as the queue steps on and open again on a new
open. Stacked, the description opens from an ⓘ, because a hover popover never opens on a tap (measured
live). The wheel: over the video it sets the volume; over a panel with something to scroll it scrolls
only, never the volume, even at its end; over one with nothing to scroll it sets the volume. [WCAG 1.4.13]
- **Clip mode (both players, owner's picks A + P1 + a):** a scissors in the control row (Plex: the C5 row,
before the ⋯; YouTube: the actions beside Download, and the docked bar) or `C` opens a clip panel IN the
player; `[` / `]` mark the start / end at the playhead and open it when shut. The video being watched is the
preview — no second stream. Plex: the panel takes the seek bar's and the row's place, and the picture stands
above it. YouTube: it sits under the video (the card's fit counts it, so the video shrinks to make room)
and the wide side panel shuts to its strip while the clip panel is open (owner; it opens again after, unless
the user opened it meanwhile); maximised, the panel takes the docked bar's place; the browser's fullscreen
steps back to maximised. The panel: an overview of the whole item (the selection and the zoomed window on it), a zoomed strip
around the selection with the two ends (`ui/RangeTrack`: each end a slider — arrows step a tenth,
Shift a second; drawn 4px, hit 1.5rem), Start / End fields with a tenth back / forward and "to the
playhead", the length, Play selection (loops), a name, Cancel / Cut clip. On the always-dark chrome the
ends are white, every focus ring in the panel and the playhead the media amber `--media-mark` (the same
in both themes; a light theme's accent and warn all but vanished there). A typed time that cannot be
read stays in its field once offered (Enter, leaving it), marked invalid with a line under the fields'
row saying how to type one (in the flow: the row below moves down rather than being covered); a time typed over a mark that then moves (a nudge, another segment) is dropped; one
past the end shows where it landed (decimal point or comma).
The strip's window follows the selection when it settles, never under a dragging pointer. Plex seeks once a drag is
released (a seek may restart the transcode); YouTube follows the drag; a mark on the very end seeks a
moment short of it, and while a clip is marked the end of the item never closes the player or steps the
queue, and Plex's auto-skip waits. Escape closes the panel before the player, but after anything over
the picture (Plex: the info, the up-next card, the countdown); ⌫ under a dialog is the dialog's. The
scissors the user can see gets the focus back (maximised: the docked bar's), else the player. The panel
keeps its name, Loop and focus when the view is maximised or restored under it. **Crop (owner's picks K1 + a):** a Crop toggle in the panel puts a
frame over the picture (`ui/CropFrame`: on the picture, not its bars; the outside dimmed; drag to move, edges
and corners to resize, thirds while dragging; the arrows move it, a focused corner's resize it, Shift
further; focus ring amber) with a shape — Free, 16:9, 9:16, 1:1 (the largest of it the picture holds) — and
Whole picture; off, the clip is the whole picture. Escape turns the crop off before it closes the panel. A
press outside the frame is still the video's. A cut is a toast with a Downloads link and the usual
"Ready to watch" notification. Limits are the server's: 1 s to 10 min (`lib/clipRange`, held to
`downloads/clip.py` by `clipLimits.contract.test.ts`; the "too long" message names the server's cap).
## Dialogs — _Decided_ (C4, built in S3f)
@@ -541,7 +595,9 @@ of a full-width panel and nothing on the right.
fields (the issue composer's Edit / Preview switch); one that loads after opening moves the focus
there when it arrives, as long as the focus still rests on the ✕;
`action` — a confirm, where the least destructive button takes `autoFocus`; `close` — anything to read,
browse or manage. [WAI-ARIA APG]
browse or manage; `player` — a dialog whose body is a player (the library's): the focus rests on the
panel itself, so Space and the arrows reach the player's keys and no button takes them — Space on a
focused ✕ closed the film. [WAI-ARIA APG]
- **The header and the footer are fixed; only the body scrolls,** with the B9 fade at each edge, so the
primary action stays on screen when the on-screen keyboard takes half of it (the viewport's
`interactive-widget=resizes-content` makes the keyboard shrink the layout, where Chrome would otherwise
@@ -605,104 +661,116 @@ of a full-width panel and nothing on the right.
## Decision log
| Date | Decision | Chosen | Over | Why |
| ---------- | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 2026-09-24 | Floating button | Back to top: a round 48 glass FAB; "round" = floats over something (media or the page) | A Material 3 rounded square; a labelled pill; the old 64 circle | The common web convention for back to top; round tells it apart from every toolbar button; 64 was oversized |
| 2026-09-24 | Device contract | Tablet portrait + landscape and up; phones out | A 1024px floor with a "too small" message | Portrait tablet use is real, and the app already renders at 820 portrait |
| 2026-09-24 | Card actions on touch | Fewer visible, the rest in a "…" menu | 44px buttons that no longer fit the card | Six 44px targets need more width than a card has |
| 2026-09-24 | Action placement | One menu with several triggers, never gesture-only, a desktop background menu that mirrors the toolbar | Visible buttons for everything | Fewer non-critical buttons on screen, without hiding any function |
| 2026-09-24 | Text selection | Text stays selectable; a click that ends a selection does nothing | Click-to-play on titles | The user must be able to copy any text the app shows |
| 2026-09-24 | A1 Button shape | r8 + a rung per context; glass is a Button variant | r12; a pill | Nested corners (r8 in r12 in r16); the pill already means "a filter or a choice" |
| 2026-09-24 | A2 Icon buttons | Two sizes, r8; a round button only on media | One size; all round | A lone toolbar icon at the row size looks lost; round rows read as tags |
| 2026-09-24 | A3 Touch size | 44 for every control primitive | 36 + gaps; Button only | One rule (HIG 44) instead of two sizes nobody remembers |
| 2026-09-24 | A4 Input | Bordered, a 55% fg-into-bg edge (≥ 3:1) | Filled; today's look as a primitive | The only option that clears WCAG 1.4.11 without a heavy fill on glass |
| 2026-09-24 | A5 Dropdown trigger | Pill in a toolbar, r8 in a form | Pill everywhere; r8 everywhere | Matches the neighbours in each context |
| 2026-09-24 | A6 Checkbox, switch, slider | Native, accent-tinted, the row is the target | Drawn primitives | Free keyboard / screen-reader / forced-colours support; only 3 sites |
| 2026-09-24 | B1 Density | Compact 32 / 28 / 24 by explicit height | Standard 38 / 32 / 26 | The owner's read of the lab; touch is covered separately (A3), and one toolbar row lines up at 28 |
| 2026-09-24 | B2 Scheme set | Midnight, Forest→cyan, Slate→fuchsia, Matrix→lime | Today's six | The accent collided with danger (YouTube), warn (Slate) and success (Forest, Matrix); Midnight and Starship were one scheme twice; 4 of 6 light accents failed 4.5:1 |
| 2026-09-24 | B3 State layers | Text colour at 10% / 16% over anything | Material's 8 / 12%; today's opacity + bg-card | Visible in every scheme and mode (≥ 1.2:1); today's row hover was 1.00:1 on a card |
| 2026-09-24 | B4 Page chrome | One sticky band + fade; filters fold when short | A floating glass bar; everything scrolls | Tabs and filters always in reach, in one place; a keyboard-up tablet keeps its content |
| 2026-09-24 | B5 Surfaces | One class + radius per role | Today's per-site mix | Nested corners and no per-site alpha drift |
| 2026-09-24 | B6 /watch | The app's surfaces, native video controls | Flat and minimal | The page strangers see should look like the product; native controls work on any device |
| 2026-09-24 | B7 Nav rail | Soft tint + edge bar, optional labels (a user setting), slim | Solid accent (recommended); a Material rail | The owner's combination: calmer than a solid fill, labels for who wants them, no width cost |
| 2026-09-24 | B8 Loading | Skeleton by shape, spinner in a control | A spinner everywhere | Nothing jumps when data lands; a long wait doesn't look stuck |
| 2026-09-24 | B9 Fade | Toolbars / headers, panel tabs, dialog edges, chip rows | Thumbnails; buttons | The north-star depth where content scrolls under chrome; not on content or small controls |
| 2026-09-24 | C1 Action classification | The table (lab C1): state + primary at rest, everyday toggles on hover, "…" on touch, destructive into the menu | Visible buttons per site, as today | One action list per object, nothing gesture-only, fewer buttons without losing a function |
| 2026-09-24 | C2 Side panels | A tap opens an overlay drawer; pin is a separate button ≥ 1024 | A tap pins and the content reflows | Below 768 the filters were unreachable; on touch a tap pinned and pushed the grid |
| 2026-09-24 | C3 Players on touch | Tap = show the controls; 44px seek hit area; 44 buttons; the YT description behind ⓘ | Today (tap also pauses; 7.5px seek bar) | Measured live: you couldn't reach the seek bar without pausing, and the hover-only description never opened on a tap |
| 2026-09-24 | C4 Dialogs | Focus by purpose; a pinned footer | Focus on Close; the footer scrolls | A form starts in its first field, and Save stays on screen with the keyboard up |
| 2026-09-24 | C5 Plex player controls | The owner's lean set: no Stop / Restart / ±10 s buttons, Back by position, Next only with a next item, Resume + From the start on the card, touch double-tap ±10 s | Every control in one row (it ran off-screen below 1024); Back by a press counter; ±10 s keyboard-only | The row fits a portrait tablet; the decision to restart is made on the card; a tablet keeps the 10 s skip |
| 2026-09-24 | Glass tuner | Retired; the schemes are the only colour choice | Keep it with ranges clamped to the rules | Free tuning is the drift the constitution exists to stop (owner) |
| 2026-09-24 | Matrix toned down | Moss #86a05a at Midnight's lightness, near-neutral greys; every white-on-accent part → `--accent-fg` | The B2 lime #a3e635 | The owner found the lime neon and tiring; it was the loudest accent (L 0.85, C 0.21), and fixed white parts sat on it at 1.5:1 |
| 2026-09-24 | Field primitives | `Input` / `Textarea` / `InnerInput` + `fieldClass`, `SelectTrigger`, the native controls; the lint rule "no raw `<input>`" | `inputCls` and ~40 hand-styled boxes | Six edges, four radii and two focus looks for one control; the ≥ 3:1 edge held nowhere |
| 2026-09-25 | Surface roles | `SURFACE` roles in `ui/surfaces.ts`; the lint rule "no bare glass class"; menus p-1 with r8 rows | Three tiers paired ad hoc with five radii and three menu paddings | ~140 sites drifted; nested corners only hold if the radius comes with the role |
| 2026-09-25 | Static labels | Flat tonal `Badge`, no edge; a filtering tag is an edged chip | A glass pill with an edge, like the chips | Edged = pressable at a glance; a static tag that looks like a chip gets clicked and does nothing |
| 2026-09-25 | Header search | A 40 glass pill with the field edge (`lg`); what sits inside it is round | 44, r16, a glass box with a faint edge | The one rounded box among pills, and its edge under 3:1; the hero search is the YouTube/Gmail pill |
| 2026-09-25 | Switch row (F12) | The whole settings row toggles (`SwitchRow`; a hand-laid row points a label at the switch) | Only the 36×24 switch toggles | The iOS / Material settings row; the text was a dead click beside the control it names |
| 2026-09-25 | Edges and hovers (F14) | The ≥ 3:1 edge on the field family (fields, the form trigger); labelled pills keep the faint edge and the accent-edge hover | The ≥ 3:1 edge and a state-layer hover on every control | 1.4.11 asks an edge only where nothing else identifies the control; a heavy edge on every chip and tab weighs the whole UI down |
| 2026-09-25 | Rating and live (F16) | Own tokens: `--rating` (theme-scoped gold), `--live`/`--live-fg`, `--rating-media`, `--media-chrome`; palette classes lint-refused | Map them to `--warn` / `--danger` | A gold star and a red LIVE are what the eye expects; as warn/danger they would read as a status. The old amber score was 2:1 on light |
| 2026-09-25 | Fjord trial (B2 exception) | A fifth scheme, temporary: Midnight's neutrals, Playnite's steel accent (#337599 lifted to #5aa1cb) and steel rules; exit = fold into Midnight or remove | Retune Midnight straight away; borrow only the qualities | The owner wants to live with the calmer palette app-wide before it replaces the default; it breaks "own hue family" on purpose, and only until that call |
| 2026-09-25 | Whole-card row (owner) | A card row's B3 layer is the card's `::before` under its content; the open button's hit layer spans the card, the actions sit above it | The name button's own hover, ending at the first badge or button | A row that half-answers the pointer reads as broken; a badge click did nothing |
| 2026-09-25 | Badge tones (F29) | `Badge tone` neutral / accent / warn at 6%; light `--warn` #b45309 → #a14a07; edged label spans lint-refused | Keep edged status pills (a private Badge in Users, Settings "Granted", the suggestions count) | Edged = pressable at a glance; amber-700 was 4.58:1 as text and fell under 4.5 on its own tint |
| 2026-09-26 | Page band built (B4) | Controls only in the band (intros scroll); the fold shows the same row in place; one `h1` per page (`PageHeading` steps the hidden one aside); `SectionLabel` headings; Skip to content | A popover of the folded filters; the page title kept visible | A folded row is still the same controls, nothing duplicated; the rail and the tab title already name the page |
| 2026-09-26 | Fade built (B9) | `fade-below` on sticky toolbars and table headers; the scroller mask where the band sits on the backdrop | A painted band over the backdrop | The backdrop is the colour under the band, so fading the content into it is the same edge |
| 2026-09-26 | Viewport units | `dvh` everywhere (the Plex player stage with S3f) | `vh` / `h-screen` | `100vh` hides the last rows under a tablet browser's toolbar |
| 2026-09-26 | Video actions built (C1) | One list per video (menu, right-click, long-press, "…"); the card keeps watched / save / add to playlist on its picture on hover; download, details, copy, hide, reset in the menu | The six-button row under the card and the hover ⓘ | The menu is the one door every device has; the card loses a line of height and the touch card its row of 28px buttons |
| 2026-09-26 | Plex actions built (C1) | One list per Plex title (play/open, watched, add to playlist, media info, copy title, pin); the poster keeps watched / add / "…" on its picture on hover; seasons get open / watched / add / copy | The hover ⓘ, the pin-only menu, an episode's add button at rest, a collection's "Remove" X on the card | The same door as a video's; the collection grid goes to two columns on a phone so the touch "…" leaves the title its line |
| 2026-09-26 | Playlist actions built (C1) | A playlist row's "…" (play, pin, copy title / link, move, order, remove), Delete + an undo toast; one list per playlist (rename, pin, YouTube open / push / revert, delete) from its rail row and its header's "…" | The row's X column and its confirm; the header's pencil, YouTube link, push, revert and trash buttons | Remove is reversible, so it needs no confirm; the header keeps Play all; a narrow table drops "#" and Channel so the title keeps a width |
| 2026-09-26 | Channel + Plex list actions (C1) | A channel row's "…" (open, pin, YouTube, copy, full history, hide + undo, unsubscribe); a Plex playlist's title / show / season "…", Delete + undo; its header's "…" (rename, delete); an action toast lasts ≥ 8 s | The channel row's full-history, hide and unsubscribe buttons; the Plex list's red X on every row, show and season; the header's pencil and trash | Hiding and removing are reversible, so they need no confirm and the undo needs time; unsubscribe and deleting a playlist keep theirs |
| 2026-09-26 | Download / notification / Scheduler actions (C1) | A download row's "…" (play, share, save, edit, details, re-download, manage sharing, copies, cancel / delete / remove); a library row's Share stays in sight, ahead of save; a notification row's "…" and hover dismiss; the inbox band's and the Scheduler strip's "…" | The download row's five rare buttons and its cancel / delete / remove; the source link's copy button; an activity notice's Find / Undo links; the band's Clear everything; the strip's Purge discovery | The destructive rows keep their confirm, and Clear everything gains one; a row link keeps the browser's own menu; Find in feed drops the tag / search / age filters that could hide its video |
| 2026-09-26 | Channel menu: Tags + Priority (C1) | Tags A–Z as in-place checks; Priority as fixed steps 10 · 5 · 3 · 2 · 1 · 0, the current one ticked; a pick re-sorts at once and the page + focus follow the row; one drawn `MenuCheck` for every checkable row that cannot be an `<input>` | Relative steps (+1 / +5); straight to the top; the stepper only; seven hand-drawn check marks (three box sizes, a tinted box, a × for "on", a trailing tick) | Bubbling a channel up is one pick; fixed levels keep the scale the reader already uses; the list moving where the reader can see it is the feedback |
| 2026-09-26 | Reorder without a drag (WCAG 2.5.7) | Row menus: Top / Up / Down / Bottom (YouTube + Plex playlists); saved views: four buttons with the place beside the selected view (owner: A); side panels in edit mode: an Up / Down pair per group (owner: A), the name wrapping there | A "…" on every saved-view row; a "…" on every panel group; drag only | A visible, one-click path where the list is edited; the edge moves shown off so the buttons never shift |
| 2026-09-26 | Titles in click surfaces | The surface stays a button, flagged `selectable`: its text selects, and the click that ends a selection does nothing; every plain `Pressable` pinned with its reason | Rebuilding each card and row so the title sits outside the button; exceptions for master-detail lists | Chrome selects inside a button once it is `user-select: text` (measured), so one flag on the primitive keeps the keyboard, focus and list walking every surface already has |
| 2026-09-26 | Right-click on a selection | The browser's own menu opens on selected words; beside them the object's menu | A "Copy selection" row in the app menu | The reader selected the words to do something with them; the native menu copies, searches and translates, and changes nothing else |
| 2026-09-26 | Keyboard ring on a flush row | Inside the row (a table row by its tag, a log line, a show header in its clipping card, with the card's corners) | The ring outside, as on a card | Outside it ran under the table head and past the table, and a clipping card cut it away entirely (invisible focus) |
| 2026-09-26 | Fade at rest | The sticky header's fade appears with the first 1rem of scroll (a scroll-driven animation; no fade where unsupported) | A fade that is always drawn | At rest it dimmed the first row and hid its ring's top edge; nothing is under the header yet |
| 2026-09-26 | Narrow playlist table | "#" and Channel hide by the table's own width (under 42rem), not the window's | Hiding below the `sm` window width | Beside the playlist rail a desktop window left the table 432 px and the title 0 px |
| 2026-09-26 | Background menu built | Five list pages (feed, Plex grid, channels, playlist, notifications); the toolbar's sort / view / filters and the band's actions; no destructive row | Every page with a toolbar; the "…" menus copied whole, delete included | The tab-only pages have nothing a menu would add; a blank-area right-click must not be one click from delete |
| 2026-09-26 | Media card anatomy (C) | One frame; the meta line pinned to its bottom; the length a badge on the picture; the Plex card on the same primitive as the video card | A: the year as an overline over the title; B: the year inline after it; a fixed two-line title slot | A one-line title left a hole the eye reads as a missing line; C puts nothing under a variable-height title |
| 2026-09-26 | One Play | `MediaPlayGlyph` (the media button variant) and only where the click plays | The Plex card's hand-painted accent circle; a playable dot on every card | One Play for every card; a dot that only a colour and a tooltip explain says nothing |
| 2026-09-26 | Filtering values on the info page | A value that filters (year, certificate, studio, director) is an edged chip; a fact is flat text | Everything flat, the clickable ones only hover-tinted | The S3d rule: edged = pressable, so the reader sees which values do something |
| 2026-09-26 | Shelf density | On any strip: one title line over one meta line | The grid's two lines and full meta on a strip | The tallest card set the strip's height and left holes in the cards with less to say |
| 2026-09-26 | Compact list row (B) | The title leads; a faint "…" at the end; the toggles float over the date on hover | A leading 128px action column | That column sat empty at rest, a hole at the start of every row |
| 2026-09-26 | Media-info layout (B) | A main column (collections first, crew, cast) beside a facts sidebar; an empty column is not drawn | Every section full width, stacked | The collection is the most used; the keywords' column left a void beside a short details box |
| 2026-09-26 | Mixed Home shelf (A) | Plex titles as wide art beside videos | Posters, with the videos not stretched | Stretching a video card to a poster's height left an ~80px gap |
| 2026-09-27 | Home hero | A card (hover row + Play on the picture, the whole menu, a touch "…"; owner: A); three quarters wide and centred on a wide window; one fact line; the summary at the measure | The hero's pin row alone; full width; an 18rem picture column | The one answer stands in focus, and it offers what the same card offers anywhere |
| 2026-09-27 | Home shelf tiles | A channel / playlist tile: Play, ↗ open, "…" with Open; a show acts on its next episode, ⓘ opens it; the shelf slot 232px | The tile's pin row alone; 200px | A tile without its object's actions read as broken; at 200 the Play touched the hover row |
| 2026-09-27 | Synopsis preview (C) | A ~0.9 s mouse rest opens a passive panel beside the card | The synopsis as the picture's `title` tooltip; an ⓘ to hover; none | One long tooltip line fired anywhere on the art and clashed with the cut title's own tooltip |
| 2026-09-27 | Reading measure | Prose capped at ~65 characters a line (`measure`) | The container's full width | At 110–130 characters a line the eye loses the next line's start |
| 2026-09-24 | Chip family | One `Chip` (filter / token / tag / suggest / ghost) + `DismissibleChip`; nothing under 24; the lint rule "no raw `<button>` outside ui/" | Five chip geometries in five files | The applied-filter row, the tag cells, the year picker and the saved-view editor each drew their own chip; a one-letter tag was 21px wide |
| 2026-09-24 | Button families | `Button` / `IconButton` / `Pressable` / chips / `DragHandle`; `lg` 40 for a hero CTA; pills and a 48 Play on media (from C5) | Every button hand-painted | 268 hand-rolled buttons in 79 files drifted into as many looks; the ladder only holds if every button stands on it |
| 2026-09-27 | C4 as built | `initialFocus` required on every `Modal`; the commit row in `ModalFooter`, portalled under the body from any depth; the fades driven by the body's scroll timeline; `interactive-widget=resizes-content` | An optional prop defaulting to the ✕; a `footer` prop (the issue composer's buttons live three components down); always-on fades | A default is how 21 dialogs came to open on the ✕; lifting the composer's state to reach a prop is the larger change; a resting fade dimmed the first line |
| 2026-09-27 | C3 + C5 as built | A tap toggles the bar through `pointerType` from `pointerdown`; a double-tap takes back its first tap's toggle; the card's pair is the video card's `MediaResumeButtons` (Continue / Restart), a sibling over the picture; the YT description is an ⓘ click popover | A delayed single tap (wait out the double-tap window); new "Resume / From the start" words for Plex; the pair inside the picture's button | A delayed tap feels dead; one media card says one pair of words; a button inside a button is invalid and the hover row already sits beside the picture |
| 2026-09-27 | C2 as built | The side panel's drawer is `ui/Modal` `drawer-start`; drawer = a coarse pointer or below 768; the pin offered in its header ≥ 1024 | A drawer of its own inside `SidePanel` | Esc, Back, the scrim, the trap and the scroll lock are the dialog's; a second implementation is how two shells drift |
| 2026-09-30 | Composition and the reader (E30 S6) | A chapter and a Conformance step: the space is used, one focal point, worst real data, where am I, a click answers at once, zoom and text size first-class; the person page as two columns (facts and bio left, "Known for" right) | Rules for pieces only; the bio in a full-width panel; a narrower panel | The person page passed every piece rule with half its panel empty; the owner reads zoomed in, so zoom is a requirement, not a nicety |
| 2026-09-30 | Hungarian words (E30 S5) | The D3 list: Feed, Shorts, Admin, Token, Port stay; stream, Prio, Job, Audit, Demo, Auto, Info, Frontend/Backend, Backfill, transcode get the Hungarian word; one name per thing (kérelem, behúzás, one "Test connection") | Keep every loanword; translate Feed/Shorts too | The owner chose the list (D3); the final sweep found the same feature under two names in three places |
| 2026-09-29 | Numbers and order in the UI language (E30 S3) | Every number through `lib/format.ts` in the UI language (decimal mark, grouping, compact, units); A→Z by the shown name in the UI language's alphabet, client and server alike; the language comes off `useTranslation`, never the singleton | `toFixed`, hand-built "1.2K", `localeCompare` in the browser's locale, byte-order server sorts | A Hungarian reader saw "8.3", "1.2K", "Életrajzi" after "Z"; the singleton read also left values in the old language after a switch (React Compiler) |
| 2026-09-27 | Narrow rows by container | The download row, the playlist table and the Channels tag row stack by a container query; a fact line never wraps | Breakpoints on the window | The iPad zoomed in, split screen and a column beside a rail are narrow while the window is not; the zoomed download row broke "18.9 / MB" apart |
| 2026-09-27 | Phones deferred | A phone-only layout (the hamburger nav, a folded feed band) waits for its own epic; E28 stays desktop + tablet | Fix the phone views inside E28 | Principle 2; the first phone attempts made the feed worse, and the epic's review grows with every sprint |
| 2026-09-27 | Conformance check (S3g) | A four-step check before a UI change is done: primitive at a named rung, the chapter re-read, the rendered page measured by kind and look, every state seen | A reviewer's eye per sprint | The S1 probe found 69 button looks; after S3 every control kind is drawn by its primitive, and the check keeps a new look from arriving unnoticed |
| 2026-09-27 | Segmented rung | `SegmentedToggle` defaults to the toolbar rung (28, `text-xs`); 24 only for a panel's captioned match toggle | The small default | Two toolbars had inherited 10px type from the default beside 28px buttons |
| 2026-09-27 | Copy rule (S3g) | One name per feature (the rail may shorten it); four verbs for four meanings; sentence case with the app's places capitalised; "…" when a command asks first; one-sentence messages without a stop; curly quotes; dates relative in lists with the exact minute on hover, three absolute forms; units from Intl, short | Word-for-word names everywhere; the verbs as they were; all lower case; no "…"; always a stop; always absolute dates; long or compact units | The copy survey found 7 EN and 6 HU double names, 22 Delete against 27 Remove, five date shapes and three spellings of a minute; the owner picked every recommended option |
| 2026-09-27 | One glyph, one meaning (S4) | Play only for playback or resuming a paused transfer; Retry = `RefreshCw`; reset / revert = `RotateCcw` | Retry drawn with Play | The first UI-review run (Downloads) found a failed row whose Retry showed ▷ in the row and its menu |
| 2026-09-27 | Fjord kept | Fjord stays a fifth scheme beside Midnight; the trial wording goes | Fold it into Midnight; remove it | The owner lived with it and likes it; the vivid Midnight stays for those who want it. B2's own-hue rule still binds any new scheme |
| 2026-09-27 | Tablet columns | A table hides `tabletHidden` columns below 64rem of its own width; Channels drops Prio, About, Length | Hide About only; give the name a bigger share | At 1024 the channel name had 60 px; the owner chose to drop all three there |
| 2026-09-27 | Glyph pager | First / previous / next / last as glyph buttons round the page box, names as tooltips | Words beside the arrows (hidden only on a phone) | The owner's call: the words widened the row; four arrows are the shared convention |
| 2026-09-27 | Match switch on the panel | `ui/MatchToggle` above the picker from two picks, everywhere; one wording (Any / All) | The picker's popover footer (Feed, Plex collection) | A closed picker hid an active "all"; six pickers had drawn it two ways |
| 2026-09-27 | Missing picture, bare link | A glyph of the kind for no picture; a download with no name is titled by its short link, once | An empty box; the URL as title, channel and link | The owner's picks 4A and 3A: an empty box read as loading, the URL showed three times |
| 2026-09-27 | Copy on a selection | Right-click on selected words keeps the browser's menu; no app Copy row | An app menu with Copy first | A real browser's menu has copy, search and translate; only the desktop pane's is thin |
| 2026-10-01 | Zoom target 125–133% | Screens are held at 125% and 133% browser zoom (1536 × 729, 1444 × 685 on a 1080p window) and the largest text size | 200% browser zoom | The owner: 200% was a guess; 125–133% is what they read Siftlode and most sites at, so 200% would be an upper bound nobody uses |
| 2026-10-01 | E31 Nav rail at short heights | One "Admin" rail item opening a panel of the admin pages | A denser rail when short; a visible "+4" hint | The rail fits at 133–150% without a hidden scroll; the admin pages are rare and admin-only |
| 2026-10-01 | E31 Configuration sections | A vertical, grouped section list, as Settings has | Group tabs + sub-tabs; one long page with a contents list | Sixteen tabs wrapped to three rows at 133%; one pattern for both settings pages |
| 2026-10-01 | E31 Statistics overview | Number tiles, and the per-operation counts as horizontal bars | Three narrow cards; a 30-day chart | The label sits by its value; what spends the quota is seen at a glance |
| 2026-10-01 | E31 Channel page header | The full banner, folding to one row once the grid scrolls | Always compact; no banner | The header is fixed and took 40% of a 685px window; the channel keeps its face at first sight |
| 2026-10-01 | E31 Channel page watch state | A fresh channel page shows the unwatched videos; a picked state is remembered | Show all, forgotten on reload | The owner: open a channel to see what is new; a choice made there should stick |
| 2026-10-01 | E31 Audit log runs | A job's successful runs fold into one row that opens, whatever ran between; only that job's failure (never folded, danger colour) breaks it | A filter defaulting to admin actions; both | Nothing hidden, the noise collapsed, the failure seen |
| 2026-10-01 | E31 Users row | Role and state as badges, the actions in the row's one menu | An inline role toggle; icon-only buttons | Action placement: the row shows state, rare and destructive actions live in the menu |
| 2026-10-01 | E31 YouTube player | Video beside an info panel at 1280px and wider (the video as large as the height allows); stacked below | Today's stack with a one-line bar; full-window like Plex | The owner's eyesight: about 1460px of video on 1080p instead of 1008, the actions still visible |
| 2026-10-01 | E31 Plex info and show pages | A second column for the facts, keywords and crew, as the person page; a collection's hero sized to its content | Full-width text; a larger poster and fanart | The right ~40% was empty; one composition for every media page |
| 2026-10-01 | E31 Nav rail rows are links | Each page row is a real link: a middle-click or Ctrl-click opens the page in a new tab | A button | The owner: open a page in a tab of its own, as on any site |
| 2026-10-02 | E31 Plex detail second column (refined) | The facts over the key crew beside the text; the keywords, chips that filter like the genres, under the text; the show page gets all three too | All three in the column, as the mockup drew it | At 20 keywords and 12 crew the column ran to twice the text's height (849 vs ~400 px at 133%): a hole under the text instead of beside it |
| 2026-10-02 | E31 Plex detail poster and crew | The movie and show poster keeps its 2:3 and grows a rung in a wide hero (w-56 at a 64rem panel); the crew shows its first 6 names and "Show N more" (a list of 8 or fewer shows whole) | Poster stretched to the row; a "More" on the keywords | The row is as tall as the crew column, so the poster drew ~175×590 and cropped; the crew, not the keywords, set the hero's height |
| 2026-10-04 | E32 `field` focus skips a mode switch | A form opens on its first field; a control above it that is not a field (the composer's Edit / Preview) is passed over by marking the field `data-initial-focus` | The first text field everywhere | That moved the focus in the download dialog off its profile choice and in the upload dialog off its drop zone; the mark changes only the form that needs it |
| 2026-10-04 | E35 Lint for z-levels and edged labels | Both rules stay; the two eslint selectors (`Z_RAW`, `EDGED_LABEL`) that refused them go | Keep the selectors | 0 hits since the E28 sweep that fixed the sites: a check that never fires costs a probe test and a snapshot each, and the UI review holds both rules |
| Date | Decision | Chosen | Over | Why |
| ---------- | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| 2026-09-24 | Floating button | Back to top: a round 48 glass FAB; "round" = floats over something (media or the page) | A Material 3 rounded square; a labelled pill; the old 64 circle | The common web convention for back to top; round tells it apart from every toolbar button; 64 was oversized |
| 2026-09-24 | Device contract | Tablet portrait + landscape and up; phones out | A 1024px floor with a "too small" message | Portrait tablet use is real, and the app already renders at 820 portrait |
| 2026-09-24 | Card actions on touch | Fewer visible, the rest in a "…" menu | 44px buttons that no longer fit the card | Six 44px targets need more width than a card has |
| 2026-09-24 | Action placement | One menu with several triggers, never gesture-only, a desktop background menu that mirrors the toolbar | Visible buttons for everything | Fewer non-critical buttons on screen, without hiding any function |
| 2026-09-24 | Text selection | Text stays selectable; a click that ends a selection does nothing | Click-to-play on titles | The user must be able to copy any text the app shows |
| 2026-09-24 | A1 Button shape | r8 + a rung per context; glass is a Button variant | r12; a pill | Nested corners (r8 in r12 in r16); the pill already means "a filter or a choice" |
| 2026-09-24 | A2 Icon buttons | Two sizes, r8; a round button only on media | One size; all round | A lone toolbar icon at the row size looks lost; round rows read as tags |
| 2026-09-24 | A3 Touch size | 44 for every control primitive | 36 + gaps; Button only | One rule (HIG 44) instead of two sizes nobody remembers |
| 2026-09-24 | A4 Input | Bordered, a 55% fg-into-bg edge (≥ 3:1) | Filled; today's look as a primitive | The only option that clears WCAG 1.4.11 without a heavy fill on glass |
| 2026-09-24 | A5 Dropdown trigger | Pill in a toolbar, r8 in a form | Pill everywhere; r8 everywhere | Matches the neighbours in each context |
| 2026-09-24 | A6 Checkbox, switch, slider | Native, accent-tinted, the row is the target | Drawn primitives | Free keyboard / screen-reader / forced-colours support; only 3 sites |
| 2026-09-24 | B1 Density | Compact 32 / 28 / 24 by explicit height | Standard 38 / 32 / 26 | The owner's read of the lab; touch is covered separately (A3), and one toolbar row lines up at 28 |
| 2026-09-24 | B2 Scheme set | Midnight, Forest→cyan, Slate→fuchsia, Matrix→lime | Today's six | The accent collided with danger (YouTube), warn (Slate) and success (Forest, Matrix); Midnight and Starship were one scheme twice; 4 of 6 light accents failed 4.5:1 |
| 2026-09-24 | B3 State layers | Text colour at 10% / 16% over anything | Material's 8 / 12%; today's opacity + bg-card | Visible in every scheme and mode (≥ 1.2:1); today's row hover was 1.00:1 on a card |
| 2026-09-24 | B4 Page chrome | One sticky band + fade; filters fold when short | A floating glass bar; everything scrolls | Tabs and filters always in reach, in one place; a keyboard-up tablet keeps its content |
| 2026-09-24 | B5 Surfaces | One class + radius per role | Today's per-site mix | Nested corners and no per-site alpha drift |
| 2026-09-24 | B6 /watch | The app's surfaces, native video controls | Flat and minimal | The page strangers see should look like the product; native controls work on any device |
| 2026-09-24 | B7 Nav rail | Soft tint + edge bar, optional labels (a user setting), slim | Solid accent (recommended); a Material rail | The owner's combination: calmer than a solid fill, labels for who wants them, no width cost |
| 2026-09-24 | B8 Loading | Skeleton by shape, spinner in a control | A spinner everywhere | Nothing jumps when data lands; a long wait doesn't look stuck |
| 2026-09-24 | B9 Fade | Toolbars / headers, panel tabs, dialog edges, chip rows | Thumbnails; buttons | The north-star depth where content scrolls under chrome; not on content or small controls |
| 2026-09-24 | C1 Action classification | The table (lab C1): state + primary at rest, everyday toggles on hover, "…" on touch, destructive into the menu | Visible buttons per site, as today | One action list per object, nothing gesture-only, fewer buttons without losing a function |
| 2026-09-24 | C2 Side panels | A tap opens an overlay drawer; pin is a separate button ≥ 1024 | A tap pins and the content reflows | Below 768 the filters were unreachable; on touch a tap pinned and pushed the grid |
| 2026-09-24 | C3 Players on touch | Tap = show the controls; 44px seek hit area; 44 buttons; the YT description behind ⓘ | Today (tap also pauses; 7.5px seek bar) | Measured live: you couldn't reach the seek bar without pausing, and the hover-only description never opened on a tap |
| 2026-09-24 | C4 Dialogs | Focus by purpose; a pinned footer | Focus on Close; the footer scrolls | A form starts in its first field, and Save stays on screen with the keyboard up |
| 2026-09-24 | C5 Plex player controls | The owner's lean set: no Stop / Restart / ±10 s buttons, Back by position, Next only with a next item, Resume + From the start on the card, touch double-tap ±10 s | Every control in one row (it ran off-screen below 1024); Back by a press counter; ±10 s keyboard-only | The row fits a portrait tablet; the decision to restart is made on the card; a tablet keeps the 10 s skip |
| 2026-09-24 | Glass tuner | Retired; the schemes are the only colour choice | Keep it with ranges clamped to the rules | Free tuning is the drift the constitution exists to stop (owner) |
| 2026-09-24 | Matrix toned down | Moss #86a05a at Midnight's lightness, near-neutral greys; every white-on-accent part → `--accent-fg` | The B2 lime #a3e635 | The owner found the lime neon and tiring; it was the loudest accent (L 0.85, C 0.21), and fixed white parts sat on it at 1.5:1 |
| 2026-09-24 | Field primitives | `Input` / `Textarea` / `InnerInput` + `fieldClass`, `SelectTrigger`, the native controls; the lint rule "no raw `<input>`" | `inputCls` and ~40 hand-styled boxes | Six edges, four radii and two focus looks for one control; the ≥ 3:1 edge held nowhere |
| 2026-09-25 | Surface roles | `SURFACE` roles in `ui/surfaces.ts`; the lint rule "no bare glass class"; menus p-1 with r8 rows | Three tiers paired ad hoc with five radii and three menu paddings | ~140 sites drifted; nested corners only hold if the radius comes with the role |
| 2026-09-25 | Static labels | Flat tonal `Badge`, no edge; a filtering tag is an edged chip | A glass pill with an edge, like the chips | Edged = pressable at a glance; a static tag that looks like a chip gets clicked and does nothing |
| 2026-09-25 | Header search | A 40 glass pill with the field edge (`lg`); what sits inside it is round | 44, r16, a glass box with a faint edge | The one rounded box among pills, and its edge under 3:1; the hero search is the YouTube/Gmail pill |
| 2026-09-25 | Switch row (F12) | The whole settings row toggles (`SwitchRow`; a hand-laid row points a label at the switch) | Only the 36×24 switch toggles | The iOS / Material settings row; the text was a dead click beside the control it names |
| 2026-09-25 | Edges and hovers (F14) | The ≥ 3:1 edge on the field family (fields, the form trigger); labelled pills keep the faint edge and the accent-edge hover | The ≥ 3:1 edge and a state-layer hover on every control | 1.4.11 asks an edge only where nothing else identifies the control; a heavy edge on every chip and tab weighs the whole UI down |
| 2026-09-25 | Rating and live (F16) | Own tokens: `--rating` (theme-scoped gold), `--live`/`--live-fg`, `--rating-media`, `--media-chrome`; palette classes lint-refused | Map them to `--warn` / `--danger` | A gold star and a red LIVE are what the eye expects; as warn/danger they would read as a status. The old amber score was 2:1 on light |
| 2026-09-25 | Fjord trial (B2 exception) | A fifth scheme, temporary: Midnight's neutrals, Playnite's steel accent (#337599 lifted to #5aa1cb) and steel rules; exit = fold into Midnight or remove | Retune Midnight straight away; borrow only the qualities | The owner wants to live with the calmer palette app-wide before it replaces the default; it breaks "own hue family" on purpose, and only until that call |
| 2026-09-25 | Whole-card row (owner) | A card row's B3 layer is the card's `::before` under its content; the open button's hit layer spans the card, the actions sit above it | The name button's own hover, ending at the first badge or button | A row that half-answers the pointer reads as broken; a badge click did nothing |
| 2026-09-25 | Badge tones (F29) | `Badge tone` neutral / accent / warn at 6%; light `--warn` #b45309 → #a14a07; edged label spans lint-refused | Keep edged status pills (a private Badge in Users, Settings "Granted", the suggestions count) | Edged = pressable at a glance; amber-700 was 4.58:1 as text and fell under 4.5 on its own tint |
| 2026-09-26 | Page band built (B4) | Controls only in the band (intros scroll); the fold shows the same row in place; one `h1` per page (`PageHeading` steps the hidden one aside); `SectionLabel` headings; Skip to content | A popover of the folded filters; the page title kept visible | A folded row is still the same controls, nothing duplicated; the rail and the tab title already name the page |
| 2026-09-26 | Fade built (B9) | `fade-below` on sticky toolbars and table headers; the scroller mask where the band sits on the backdrop | A painted band over the backdrop | The backdrop is the colour under the band, so fading the content into it is the same edge |
| 2026-09-26 | Viewport units | `dvh` everywhere (the Plex player stage with S3f) | `vh` / `h-screen` | `100vh` hides the last rows under a tablet browser's toolbar |
| 2026-09-26 | Video actions built (C1) | One list per video (menu, right-click, long-press, "…"); the card keeps watched / save / add to playlist on its picture on hover; download, details, copy, hide, reset in the menu | The six-button row under the card and the hover ⓘ | The menu is the one door every device has; the card loses a line of height and the touch card its row of 28px buttons |
| 2026-09-26 | Plex actions built (C1) | One list per Plex title (play/open, watched, add to playlist, media info, copy title, pin); the poster keeps watched / add / "…" on its picture on hover; seasons get open / watched / add / copy | The hover ⓘ, the pin-only menu, an episode's add button at rest, a collection's "Remove" X on the card | The same door as a video's; the collection grid goes to two columns on a phone so the touch "…" leaves the title its line |
| 2026-09-26 | Playlist actions built (C1) | A playlist row's "…" (play, pin, copy title / link, move, order, remove), Delete + an undo toast; one list per playlist (rename, pin, YouTube open / push / revert, delete) from its rail row and its header's "…" | The row's X column and its confirm; the header's pencil, YouTube link, push, revert and trash buttons | Remove is reversible, so it needs no confirm; the header keeps Play all; a narrow table drops "#" and Channel so the title keeps a width |
| 2026-09-26 | Channel + Plex list actions (C1) | A channel row's "…" (open, pin, YouTube, copy, full history, hide + undo, unsubscribe); a Plex playlist's title / show / season "…", Delete + undo; its header's "…" (rename, delete); an action toast lasts ≥ 8 s | The channel row's full-history, hide and unsubscribe buttons; the Plex list's red X on every row, show and season; the header's pencil and trash | Hiding and removing are reversible, so they need no confirm and the undo needs time; unsubscribe and deleting a playlist keep theirs |
| 2026-09-26 | Download / notification / Scheduler actions (C1) | A download row's "…" (play, share, save, edit, details, re-download, manage sharing, copies, cancel / delete / remove); a library row's Share stays in sight, ahead of save; a notification row's "…" and hover dismiss; the inbox band's and the Scheduler strip's "…" | The download row's five rare buttons and its cancel / delete / remove; the source link's copy button; an activity notice's Find / Undo links; the band's Clear everything; the strip's Purge discovery | The destructive rows keep their confirm, and Clear everything gains one; a row link keeps the browser's own menu; Find in feed drops the tag / search / age filters that could hide its video |
| 2026-09-26 | Channel menu: Tags + Priority (C1) | Tags A–Z as in-place checks; Priority as fixed steps 10 · 5 · 3 · 2 · 1 · 0, the current one ticked; a pick re-sorts at once and the page + focus follow the row; one drawn `MenuCheck` for every checkable row that cannot be an `<input>` | Relative steps (+1 / +5); straight to the top; the stepper only; seven hand-drawn check marks (three box sizes, a tinted box, a × for "on", a trailing tick) | Bubbling a channel up is one pick; fixed levels keep the scale the reader already uses; the list moving where the reader can see it is the feedback |
| 2026-09-26 | Reorder without a drag (WCAG 2.5.7) | Row menus: Top / Up / Down / Bottom (YouTube + Plex playlists); saved views: four buttons with the place beside the selected view (owner: A); side panels in edit mode: an Up / Down pair per group (owner: A), the name wrapping there | A "…" on every saved-view row; a "…" on every panel group; drag only | A visible, one-click path where the list is edited; the edge moves shown off so the buttons never shift |
| 2026-09-26 | Titles in click surfaces | The surface stays a button, flagged `selectable`: its text selects, and the click that ends a selection does nothing; every plain `Pressable` pinned with its reason | Rebuilding each card and row so the title sits outside the button; exceptions for master-detail lists | Chrome selects inside a button once it is `user-select: text` (measured), so one flag on the primitive keeps the keyboard, focus and list walking every surface already has |
| 2026-09-26 | Right-click on a selection | The browser's own menu opens on selected words; beside them the object's menu | A "Copy selection" row in the app menu | The reader selected the words to do something with them; the native menu copies, searches and translates, and changes nothing else |
| 2026-09-26 | Keyboard ring on a flush row | Inside the row (a table row by its tag, a log line, a show header in its clipping card, with the card's corners) | The ring outside, as on a card | Outside it ran under the table head and past the table, and a clipping card cut it away entirely (invisible focus) |
| 2026-09-26 | Fade at rest | The sticky header's fade appears with the first 1rem of scroll (a scroll-driven animation; no fade where unsupported) | A fade that is always drawn | At rest it dimmed the first row and hid its ring's top edge; nothing is under the header yet |
| 2026-09-26 | Narrow playlist table | "#" and Channel hide by the table's own width (under 42rem), not the window's | Hiding below the `sm` window width | Beside the playlist rail a desktop window left the table 432 px and the title 0 px |
| 2026-09-26 | Background menu built | Five list pages (feed, Plex grid, channels, playlist, notifications); the toolbar's sort / view / filters and the band's actions; no destructive row | Every page with a toolbar; the "…" menus copied whole, delete included | The tab-only pages have nothing a menu would add; a blank-area right-click must not be one click from delete |
| 2026-09-26 | Media card anatomy (C) | One frame; the meta line pinned to its bottom; the length a badge on the picture; the Plex card on the same primitive as the video card | A: the year as an overline over the title; B: the year inline after it; a fixed two-line title slot | A one-line title left a hole the eye reads as a missing line; C puts nothing under a variable-height title |
| 2026-09-26 | One Play | `MediaPlayGlyph` (the media button variant) and only where the click plays | The Plex card's hand-painted accent circle; a playable dot on every card | One Play for every card; a dot that only a colour and a tooltip explain says nothing |
| 2026-09-26 | Filtering values on the info page | A value that filters (year, certificate, studio, director) is an edged chip; a fact is flat text | Everything flat, the clickable ones only hover-tinted | The S3d rule: edged = pressable, so the reader sees which values do something |
| 2026-09-26 | Shelf density | On any strip: one title line over one meta line | The grid's two lines and full meta on a strip | The tallest card set the strip's height and left holes in the cards with less to say |
| 2026-09-26 | Compact list row (B) | The title leads; a faint "…" at the end; the toggles float over the date on hover | A leading 128px action column | That column sat empty at rest, a hole at the start of every row |
| 2026-09-26 | Media-info layout (B) | A main column (collections first, crew, cast) beside a facts sidebar; an empty column is not drawn | Every section full width, stacked | The collection is the most used; the keywords' column left a void beside a short details box |
| 2026-09-26 | Mixed Home shelf (A) | Plex titles as wide art beside videos | Posters, with the videos not stretched | Stretching a video card to a poster's height left an ~80px gap |
| 2026-09-27 | Home hero | A card (hover row + Play on the picture, the whole menu, a touch "…"; owner: A); three quarters wide and centred on a wide window; one fact line; the summary at the measure | The hero's pin row alone; full width; an 18rem picture column | The one answer stands in focus, and it offers what the same card offers anywhere |
| 2026-09-27 | Home shelf tiles | A channel / playlist tile: Play, ↗ open, "…" with Open; a show acts on its next episode, ⓘ opens it; the shelf slot 232px | The tile's pin row alone; 200px | A tile without its object's actions read as broken; at 200 the Play touched the hover row |
| 2026-09-27 | Synopsis preview (C) | A ~0.9 s mouse rest opens a passive panel beside the card | The synopsis as the picture's `title` tooltip; an ⓘ to hover; none | One long tooltip line fired anywhere on the art and clashed with the cut title's own tooltip |
| 2026-09-27 | Reading measure | Prose capped at ~65 characters a line (`measure`) | The container's full width | At 110–130 characters a line the eye loses the next line's start |
| 2026-09-24 | Chip family | One `Chip` (filter / token / tag / suggest / ghost) + `DismissibleChip`; nothing under 24; the lint rule "no raw `<button>` outside ui/" | Five chip geometries in five files | The applied-filter row, the tag cells, the year picker and the saved-view editor each drew their own chip; a one-letter tag was 21px wide |
| 2026-09-24 | Button families | `Button` / `IconButton` / `Pressable` / chips / `DragHandle`; `lg` 40 for a hero CTA; pills and a 48 Play on media (from C5) | Every button hand-painted | 268 hand-rolled buttons in 79 files drifted into as many looks; the ladder only holds if every button stands on it |
| 2026-09-27 | C4 as built | `initialFocus` required on every `Modal`; the commit row in `ModalFooter`, portalled under the body from any depth; the fades driven by the body's scroll timeline; `interactive-widget=resizes-content` | An optional prop defaulting to the ✕; a `footer` prop (the issue composer's buttons live three components down); always-on fades | A default is how 21 dialogs came to open on the ✕; lifting the composer's state to reach a prop is the larger change; a resting fade dimmed the first line |
| 2026-09-27 | C3 + C5 as built | A tap toggles the bar through `pointerType` from `pointerdown`; a double-tap takes back its first tap's toggle; the card's pair is the video card's `MediaResumeButtons` (Continue / Restart), a sibling over the picture; the YT description is an ⓘ click popover | A delayed single tap (wait out the double-tap window); new "Resume / From the start" words for Plex; the pair inside the picture's button | A delayed tap feels dead; one media card says one pair of words; a button inside a button is invalid and the hover row already sits beside the picture |
| 2026-09-27 | C2 as built | The side panel's drawer is `ui/Modal` `drawer-start`; drawer = a coarse pointer or below 768; the pin offered in its header ≥ 1024 | A drawer of its own inside `SidePanel` | Esc, Back, the scrim, the trap and the scroll lock are the dialog's; a second implementation is how two shells drift |
| 2026-09-30 | Composition and the reader (E30 S6) | A chapter and a Conformance step: the space is used, one focal point, worst real data, where am I, a click answers at once, zoom and text size first-class; the person page as two columns (facts and bio left, "Known for" right) | Rules for pieces only; the bio in a full-width panel; a narrower panel | The person page passed every piece rule with half its panel empty; the owner reads zoomed in, so zoom is a requirement, not a nicety |
| 2026-09-30 | Hungarian words (E30 S5) | The D3 list: Feed, Shorts, Admin, Token, Port stay; stream, Prio, Job, Audit, Demo, Auto, Info, Frontend/Backend, Backfill, transcode get the Hungarian word; one name per thing (kérelem, behúzás, one "Test connection") | Keep every loanword; translate Feed/Shorts too | The owner chose the list (D3); the final sweep found the same feature under two names in three places |
| 2026-09-29 | Numbers and order in the UI language (E30 S3) | Every number through `lib/format.ts` in the UI language (decimal mark, grouping, compact, units); A→Z by the shown name in the UI language's alphabet, client and server alike; the language comes off `useTranslation`, never the singleton | `toFixed`, hand-built "1.2K", `localeCompare` in the browser's locale, byte-order server sorts | A Hungarian reader saw "8.3", "1.2K", "Életrajzi" after "Z"; the singleton read also left values in the old language after a switch (React Compiler) |
| 2026-09-27 | Narrow rows by container | The download row, the playlist table and the Channels tag row stack by a container query; a fact line never wraps | Breakpoints on the window | The iPad zoomed in, split screen and a column beside a rail are narrow while the window is not; the zoomed download row broke "18.9 / MB" apart |
| 2026-09-27 | Phones deferred | A phone-only layout (the hamburger nav, a folded feed band) waits for its own epic; E28 stays desktop + tablet | Fix the phone views inside E28 | Principle 2; the first phone attempts made the feed worse, and the epic's review grows with every sprint |
| 2026-09-27 | Conformance check (S3g) | A four-step check before a UI change is done: primitive at a named rung, the chapter re-read, the rendered page measured by kind and look, every state seen | A reviewer's eye per sprint | The S1 probe found 69 button looks; after S3 every control kind is drawn by its primitive, and the check keeps a new look from arriving unnoticed |
| 2026-09-27 | Segmented rung | `SegmentedToggle` defaults to the toolbar rung (28, `text-xs`); 24 only for a panel's captioned match toggle | The small default | Two toolbars had inherited 10px type from the default beside 28px buttons |
| 2026-09-27 | Copy rule (S3g) | One name per feature (the rail may shorten it); four verbs for four meanings; sentence case with the app's places capitalised; "…" when a command asks first; one-sentence messages without a stop; curly quotes; dates relative in lists with the exact minute on hover, three absolute forms; units from Intl, short | Word-for-word names everywhere; the verbs as they were; all lower case; no "…"; always a stop; always absolute dates; long or compact units | The copy survey found 7 EN and 6 HU double names, 22 Delete against 27 Remove, five date shapes and three spellings of a minute; the owner picked every recommended option |
| 2026-09-27 | One glyph, one meaning (S4) | Play only for playback or resuming a paused transfer; Retry = `RefreshCw`; reset / revert = `RotateCcw` | Retry drawn with Play | The first UI-review run (Downloads) found a failed row whose Retry showed ▷ in the row and its menu |
| 2026-09-27 | Fjord kept | Fjord stays a fifth scheme beside Midnight; the trial wording goes | Fold it into Midnight; remove it | The owner lived with it and likes it; the vivid Midnight stays for those who want it. B2's own-hue rule still binds any new scheme |
| 2026-09-27 | Tablet columns | A table hides `tabletHidden` columns below 64rem of its own width; Channels drops Prio, About, Length | Hide About only; give the name a bigger share | At 1024 the channel name had 60 px; the owner chose to drop all three there |
| 2026-09-27 | Glyph pager | First / previous / next / last as glyph buttons round the page box, names as tooltips | Words beside the arrows (hidden only on a phone) | The owner's call: the words widened the row; four arrows are the shared convention |
| 2026-09-27 | Match switch on the panel | `ui/MatchToggle` above the picker from two picks, everywhere; one wording (Any / All) | The picker's popover footer (Feed, Plex collection) | A closed picker hid an active "all"; six pickers had drawn it two ways |
| 2026-09-27 | Missing picture, bare link | A glyph of the kind for no picture; a download with no name is titled by its short link, once | An empty box; the URL as title, channel and link | The owner's picks 4A and 3A: an empty box read as loading, the URL showed three times |
| 2026-09-27 | Copy on a selection | Right-click on selected words keeps the browser's menu; no app Copy row | An app menu with Copy first | A real browser's menu has copy, search and translate; only the desktop pane's is thin |
| 2026-10-01 | Zoom target 125–133% | Screens are held at 125% and 133% browser zoom (1536 × 729, 1444 × 685 on a 1080p window) and the largest text size | 200% browser zoom | The owner: 200% was a guess; 125–133% is what they read Siftlode and most sites at, so 200% would be an upper bound nobody uses |
| 2026-10-01 | E31 Nav rail at short heights | One "Admin" rail item opening a panel of the admin pages | A denser rail when short; a visible "+4" hint | The rail fits at 133–150% without a hidden scroll; the admin pages are rare and admin-only |
| 2026-10-01 | E31 Configuration sections | A vertical, grouped section list, as Settings has | Group tabs + sub-tabs; one long page with a contents list | Sixteen tabs wrapped to three rows at 133%; one pattern for both settings pages |
| 2026-10-01 | E31 Statistics overview | Number tiles, and the per-operation counts as horizontal bars | Three narrow cards; a 30-day chart | The label sits by its value; what spends the quota is seen at a glance |
| 2026-10-01 | E31 Channel page header | The full banner, folding to one row once the grid scrolls | Always compact; no banner | The header is fixed and took 40% of a 685px window; the channel keeps its face at first sight |
| 2026-10-01 | E31 Channel page watch state | A fresh channel page shows the unwatched videos; a picked state is remembered | Show all, forgotten on reload | The owner: open a channel to see what is new; a choice made there should stick |
| 2026-10-01 | E31 Audit log runs | A job's successful runs fold into one row that opens, whatever ran between; only that job's failure (never folded, danger colour) breaks it | A filter defaulting to admin actions; both | Nothing hidden, the noise collapsed, the failure seen |
| 2026-10-01 | E31 Users row | Role and state as badges, the actions in the row's one menu | An inline role toggle; icon-only buttons | Action placement: the row shows state, rare and destructive actions live in the menu |
| 2026-10-01 | E31 YouTube player | Video beside an info panel at 1280px and wider (the video as large as the height allows); stacked below | Today's stack with a one-line bar; full-window like Plex | The owner's eyesight: about 1460px of video on 1080p instead of 1008, the actions still visible |
| 2026-10-01 | E31 Plex info and show pages | A second column for the facts, keywords and crew, as the person page; a collection's hero sized to its content | Full-width text; a larger poster and fanart | The right ~40% was empty; one composition for every media page |
| 2026-10-01 | E31 Nav rail rows are links | Each page row is a real link: a middle-click or Ctrl-click opens the page in a new tab | A button | The owner: open a page in a tab of its own, as on any site |
| 2026-10-02 | E31 Plex detail second column (refined) | The facts over the key crew beside the text; the keywords, chips that filter like the genres, under the text; the show page gets all three too | All three in the column, as the mockup drew it | At 20 keywords and 12 crew the column ran to twice the text's height (849 vs ~400 px at 133%): a hole under the text instead of beside it |
| 2026-10-02 | E31 Plex detail poster and crew | The movie and show poster keeps its 2:3 and grows a rung in a wide hero (w-56 at a 64rem panel); the crew shows its first 6 names and "Show N more" (a list of 8 or fewer shows whole) | Poster stretched to the row; a "More" on the keywords | The row is as tall as the crew column, so the poster drew ~175×590 and cropped; the crew, not the keywords, set the hero's height |
| 2026-10-04 | E32 `field` focus skips a mode switch | A form opens on its first field; a control above it that is not a field (the composer's Edit / Preview) is passed over by marking the field `data-initial-focus` | The first text field everywhere | That moved the focus in the download dialog off its profile choice and in the upload dialog off its drop zone; the mark changes only the form that needs it |
| 2026-10-04 | E35 Lint for z-levels and edged labels | Both rules stay; the two eslint selectors (`Z_RAW`, `EDGED_LABEL`) that refused them go | Keep the selectors | 0 hits since the E28 sweep that fixed the sites: a check that never fires costs a probe test and a snapshot each, and the UI review holds both rules |
| 2026-10-06 | `player` focus | A dialog that is a player opens focused on its panel (the YouTube player's rule, now `Modal`'s too); on the media element itself, Space and the arrows stay the element's own | `close`, or the media element | On the ✕ Space closed the player; on the element the browser's own controls acted on the same key as the player's (one Space played and paused, measured in Chrome) |
| 2026-10-08 | Captions on touch | A `Tooltip` opens for a mouse and a key's focus only, never for a finger | A caption on tap; a long-press caption | On an iPad the tap that showed the caption was taken as a hover and lost its click (every rail button took two taps), and no finger sends the mouseleave that hides it; the rail's labels setting names the buttons on touch |
| 2026-10-08 | Header search on touch | No key cap on a touch screen; the pill grows to 52 px so a ring shows round the 44 px Go; Go drops to its icon by the column's width | Keep the key cap; a 44 px Go overhanging a 40 px pill | On a tablet the 44 px key cap and Go sat on the pill's edge and cut the placeholder; an overhanging Go breaks the nested radii |
| 2026-10-08 | Plex subtitles and the control bar | The player draws the cues itself; while the bar shows they rise above it (200 ms), then settle back | The browser's own cues, moved by `line` | On an iPad the bar covered the text; a native cue cannot be animated or measured, and our layer behaves the same on every engine |
| 2026-10-08 | Hover on a button over a picture | The `media` variant's hover is a 2px accent ring over a darker body (Plex + YT cards, rail arrows; option B) | An accent fill (reads as "selected"); a white wash (weak on a bright poster) | The 60 → 75% black step was invisible, and over a card a rail arrow could not be told from the card's own Play under it |
| 2026-10-08 | Plex "Add to collection" | A movie or show card's menu row opens a panel beside the card: its library's manual collections tick in place, a name + Enter creates one with the title in it (admin; option A) | An Add submenu with a "New…" dialog; the per-title editor modal from the menu | A collection took a trip to the Collections page, a name, a title search, then a second search per title; the panel is the video card's "Add to playlist" |
| 2026-10-08 | YouTube modal chrome | No prev/next strips beside the card; Close an icon (the panel's top right; stacked, the title row's end); the description clamped to three lines + More; the panel shuts to a 2.75rem strip of close, expand, prev/next (option D1) | Keep the strips; a "Close" text button; the full description in a scroller; a panel that hides outright | The strips cost the video 7rem of width for a second copy of the panel's buttons; a long description pushed the channel and the actions out of sight; a hidden panel left no way to step the queue without the keyboard |
| 2026-10-08 | The wheel over the YouTube modal | Over the video: volume. Over a panel with something to scroll: scroll only, never volume, not even at its end. Over a panel with nothing to scroll: volume (owner) | Volume everywhere; a scroll that hands over to the volume at its end | Reading a long description turned the volume down once it hit the end |
| 2026-10-09 | A Plex show card plays | Its picture plays the show's next episode everywhere (an episode with a resume point → Continue + Restart, else the first unwatched, else S1·E1), resolved on the click; the preview names it once up; title and ⓘ open the show (option A + hover ask) | The picture opens the show page (as before); every card carrying `next` | A show card on Discover had no Play while Home's had one; carrying `next` costs a grid page up to 180 queries for the few cards anyone plays |
| 2026-10-10 | Clip mode in the players (E38) | Scissors in the control row + `C`, `[` `]`; the panel in the player with the watched video as preview; overview + zoomed strip on `ui/RangeTrack` (ends are sliders); a toast + the Ready notification after Cut | A ⋯ menu entry; permanent In / Out buttons; a big dialog with its own preview (a second Plex transcode / YouTube iframe) | Owner's picks A, P1, a; a film is hours and a clip seconds, so only a zoomed strip gives a hand the precision. The Downloads editor's cuts moved onto the same track (keyboard-steppable now) |
| 2026-10-10 | Crop in clip mode (E38) | A Crop toggle in the clip panel: a frame over the playing picture (`ui/CropFrame`, the editor's crop too), Free / 16:9 / 9:16 / 1:1, Whole picture; maximised YouTube keeps the panel in the bar's place; the wide side panel shuts while clipping | A frame always on the picture in clip mode (K2); a free shape only | Owner's K1 + a: a crop is rare, so it never covers the picture unasked; a phone or square clip needs a held shape. The iframe hides its picture's size: drawn on the 16:9 box, the cut fits it to the real picture |
| 2026-10-10 | Marks on the player chrome | `--media-mark` (amber) and `--media-danger` (light red), the same in both themes; `media-marks` points the focus ring (`--focus`) and the chosen mark (`--chosen`: a track's tick, a tab's underline; the label stays white) at the amber on the player and every chrome surface; `media-scope` (a form) also the errors at the red | The theme's accent ring, chosen text, warn and danger | On the chrome (#171717) a light theme's accent ring (Slate 2.84:1), warn (2.99:1) and danger (2.77:1) all fall under 3:1; the accent as chosen text under 4.5:1 |
+1
View File
@@ -112,6 +112,7 @@ project checks needs a row.
| coverage | not in the gate; no threshold | Owner decision E32 D4: never enforced; "every regression test seen red" is the bar, and a percentage invites tests written to move it. Out of the lanes since E35: read once, by hand, it found 5 bugs; as a lane it was a number nobody acted on, and the slowest vitest run (the one that flaked). On demand: docs/development.md. | frontend 80% / backend 78% of lines (E32 S1) |
| CVE gate | HIGH/CRITICAL with a fix available only | `--ignore-unfixed` + `--severity HIGH,CRITICAL`: a finding with no fix cannot be acted on by an upgrade; it shows in the scan output. A fixable one fails the publish, and the only way past it is `.trivyignore.yaml`. | 0 open (E32 S1) |
| npm audit | `--omit=dev --audit-level=high` | What ships to the browser, at high or above. The dev toolchain's advisories were burned down by hand in E32 S1 (18 → 0). | 0 |
| knip | `entry` — one file added | Not a narrowing: knip's Vite plugin reads entries from index.html only, so the shared link's page (`watch.html` → `src/watchMain.tsx`) is named here, or the lane reports that whole entry unused. It adds files to the lane, never takes any out. | 1 entry |
| React Compiler | the functions it skips, pinned | `components/compilerCoverage.contract.test.ts` — the list may only shrink. End state: only VirtualGrid (TanStack Virtual, an IncompatibleLibrary) remains. | 93 (E32 S1) → 29 (E32 S3) → 1, VirtualGrid (E32 S4) |
## Accepted findings
+51 -1
View File
@@ -1,4 +1,4 @@
import { test, expect } from "@playwright/test";
import { test, expect, type Page } from "@playwright/test";
import { openApp, openFirstVideo } from "./helpers";
// Locks the in-app player's open/close contract. The player is a portaled overlay whose
@@ -26,3 +26,53 @@ test.describe("player modal", () => {
await expect(page.getByTestId("player-modal")).toHaveCount(0);
});
});
// The video's size, in the windows the owner meets: layout bugs no DOM test can see. Each viewport
// is a real one — 1920×911 is a 1080p screen at 100% (the side-panel layout), 1097×520 the same
// screen at 175% or a small laptop at 100% (the stacked layout, on a short window), 1400×380 a wide
// window made short by docked devtools (the side-panel layout, whose card does not scroll).
test.describe("player modal sizing", () => {
test.beforeEach(async ({ page }) => {
await page.route(/youtube\.com/, (route) => route.abort());
});
const box = async (page: Page, selector: string) =>
(await page.locator(selector).first().boundingBox())!;
for (const [w, h] of [
[1920, 911],
[1097, 520],
[1400, 380],
] as const) {
test(`the video keeps a real size at ${w}×${h}, and maximised fills the window`, async ({
page,
}) => {
await page.setViewportSize({ width: w, height: h });
await openApp(page);
await openFirstVideo(page);
// A short window narrows the card so the chrome under the video fits — but the chrome grows
// as the card narrows (its text wraps), and that loop once ran the card down to 2 px.
const stage = await box(page, ".player-stage");
expect(stage.width).toBeGreaterThanOrEqual(Math.min(w * 0.5, 560));
expect(stage.height).toBeGreaterThan(stage.width * 0.5);
// …and all of it is in sight: its bottom edge is the YouTube control bar, and the side-panel
// card clips what overflows it (it does not scroll), the stacked card scrolls to it.
const overflow = await page
.locator(".player-stage")
.first()
.evaluate((el) => {
const card = el.parentElement!;
return el.getBoundingClientRect().bottom - card.getBoundingClientRect().bottom;
});
expect(overflow).toBeLessThanOrEqual(0.5);
// Maximise (F): the stage is fixed to the viewport. In the side-panel layout its `self-start`
// once aligned the fixed box too, and it shrank to its content's height — 0 px, a thin line.
await page.getByTestId("player-modal").press("f");
await expect(page.locator(".player-stage--max")).toHaveCount(1);
const max = await box(page, ".player-stage--max");
expect([Math.round(max.width), Math.round(max.height)]).toEqual([w, h]);
});
}
});
+2
View File
@@ -1,5 +1,7 @@
{
"$schema": "https://unpkg.com/knip@6/schema.json",
// The shared link's page (watch.html): the Vite plugin reads only index.html for entries.
"entry": ["src/watchMain.tsx"],
"project": ["**/*.{ts,tsx,js}"],
"playwright": {
"config": ["playwright.config.ts"],
+49 -77
View File
@@ -2,23 +2,19 @@ import { useLayoutEffect, useState } from "react";
import { useTranslation } from "react-i18next";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { qk } from "../lib/queryKeys";
import { ListPlus, Plus } from "lucide-react";
import { ListPlus } from "lucide-react";
import { YouTubeIcon } from "../lib/brandIcons";
import { MenuCheck } from "./ui/MenuCheck";
import { api, type Playlist } from "../lib/api";
import { notifyActionError } from "../lib/actionErrors";
import { useScrollFade } from "../lib/useScrollFade";
import { playlistName } from "../lib/playlistName";
import { PopoverItem, PopoverSurface } from "./ui/AnchoredPopover";
import { PopoverSurface } from "./ui/AnchoredPopover";
import { useAnchoredPopover } from "./ui/anchoredPopoverHooks";
import { IconButton } from "./ui/IconButton";
import { Input } from "./ui/Input";
import { MembershipMenu } from "./ui/MembershipMenu";
import { SURFACE } from "./ui/surfaces";
// A small anchored menu for toggling a video's membership in the user's playlists, with inline
// "new playlist" creation. Same shape as the channel tag editor (the popover pilot): radio-style
// toggles plus a free-text creator, so it follows that file's two conventions — typeahead OFF and
// the input swallowing non-Escape keys, since character and arrow keys belong to the text field.
// "new playlist" creation. Its body is the shared MembershipMenu (as the channel tag editor's).
export default function AddToPlaylist({
videoId,
variant = "ghost",
@@ -94,9 +90,7 @@ function MembershipPanel({
}) {
const { t } = useTranslation();
const qc = useQueryClient();
const { ref: fadeRef, style: fadeStyle } = useScrollFade();
const open = pop.open;
const [newName, setNewName] = useState("");
const [busy, setBusy] = useState(false);
const membership = useQuery({
@@ -115,13 +109,17 @@ function MembershipPanel({
// A failure is said, and the list refreshed either way. With no catch, a playlist deleted on
// another device (a 404 `not_found/playlist` — still offered, from a cache up to 30 s old) was an
// unhandled rejection: the tick did not change, nothing was said, the gone row stayed.
// unhandled rejection: the tick did not change, nothing was said, the gone row stayed. A write that
// landed flips the tick at once: a second tap before the refetch returned would otherwise repeat it.
async function toggle(pl: Playlist) {
if (busy) return;
setBusy(true);
try {
if (pl.has_video) await api.removeFromPlaylist(pl.id, videoId);
else await api.addToPlaylist(pl.id, videoId);
qc.setQueryData<Playlist[]>(qk.playlistMembership(videoId), (old) =>
old?.map((p) => (p.id === pl.id ? { ...p, has_video: !p.has_video } : p)),
);
} catch (e) {
notifyActionError(e, t(pl.has_video ? "playlists.removeFailed" : "playlists.addFailed"));
}
@@ -129,19 +127,20 @@ function MembershipPanel({
setBusy(false);
}
async function createAndAdd() {
const name = newName.trim();
if (!name || busy) return;
async function createAndAdd(name: string) {
if (busy) return false;
setBusy(true);
let made = false;
try {
const pl = await api.createPlaylist(name);
await api.addToPlaylist(pl.id, videoId);
setNewName("");
made = true;
} catch (e) {
notifyActionError(e, t("playlists.addFailed"));
}
refresh();
setBusy(false);
return made;
}
// All of the user's playlists are add targets, including YouTube-linked ones — adding
@@ -149,67 +148,40 @@ function MembershipPanel({
const lists = membership.data ?? [];
return (
<>
<PopoverSurface popover={pop} className={`${SURFACE.popover} w-60 p-2`}>
{/* A React portal still propagates events up the REACT tree, so a click inside this panel
would otherwise reach the feed card that hosts the trigger and start playback. */}
<div onClick={(e) => e.stopPropagation()}>
<div className="text-xs text-muted px-1.5 pb-1.5">
{t("playlists.addToPlaylistTitle")}
</div>
<div
ref={fadeRef}
style={fadeStyle}
aria-busy={busy}
className="max-h-56 overflow-y-auto no-scrollbar"
>
{lists.length === 0 && !membership.isLoading && (
<div className="text-xs text-muted px-1.5 py-2">{t("playlists.noneYet")}</div>
)}
{lists.map((pl) => (
<PopoverItem
key={pl.id}
// A video can sit in several playlists at once — checkboxes, not one choice.
role="menuitemcheckbox"
checked={pl.has_video}
onSelect={() => toggle(pl)}
// NOT `disabled={busy}`: the native disabled attribute would blur the focused item to
// <body> mid-request and permanently kill roving focus (useListNavigation listens on
// the floating element only). `toggle()` already no-ops while busy, so re-entrancy is
// guarded without touching focus; `aria-busy` on the list conveys the pending state.
label={playlistName(pl, t)}
className="w-full flex items-center gap-2 text-sm px-1.5 py-1.5 rounded-lg text-fg outline-hidden transition"
>
<MenuCheck kind="many" on={!!pl.has_video} />
<span className="truncate flex-1 text-left">{playlistName(pl, t)}</span>
{(pl.source === "youtube" || pl.yt_playlist_id) && (
<YouTubeIcon className="w-3 h-3 shrink-0 text-muted" />
)}
</PopoverItem>
))}
</div>
<div className="flex items-center gap-1.5 border-t border-border mt-1.5 pt-1.5">
<Plus className="w-3.5 h-3.5 text-muted shrink-0" />
<Input
className="flex-1"
size="sm"
value={newName}
onChange={(e) => setNewName(e.target.value)}
onKeyDown={(e) => {
// Escape bubbles to the popover's dismiss; every other key is text editing here and
// must not reach the surrounding menu's list navigation.
if (e.key === "Escape") return;
e.stopPropagation();
if (e.key === "Enter") {
e.preventDefault();
void createAndAdd();
}
}}
placeholder={t("playlists.newPlaylist")}
/>
</div>
</div>
</PopoverSurface>
</>
<PopoverSurface
popover={pop}
className={`${SURFACE.popover} w-60 p-2`}
ariaLabel={t("playlists.addToPlaylistTitle")}
listInside
>
<MembershipMenu
heading={t("playlists.addToPlaylistTitle")}
rows={lists.map((pl) => ({
id: pl.id,
label: playlistName(pl, t),
checked: !!pl.has_video,
extra:
pl.source === "youtube" || pl.yt_playlist_id ? (
<YouTubeIcon className="w-3 h-3 shrink-0 text-muted" />
) : undefined,
}))}
onToggle={(row) => {
const pl = lists.find((p) => p.id === row.id);
if (pl) void toggle(pl);
}}
status={
membership.isError && !membership.data
? "error"
: membership.isLoading
? "loading"
: "ready"
}
onRetry={() => void membership.refetch()}
emptyText={t("playlists.noneYet")}
errorText={t("common.loadError")}
busy={busy}
create={{ placeholder: t("playlists.newPlaylist"), onCreate: createAndAdd }}
/>
</PopoverSurface>
);
}
@@ -91,7 +91,7 @@ describe("the last sign-in column", () => {
</QueryClientProvider>,
);
expect(await screen.findByText(relativeTime(at, "en"))).toBeTruthy();
expect(await screen.findByText(relativeTime(at, "en", i18n.t))).toBeTruthy();
expect(screen.getAllByText("—")).toHaveLength(1);
});
});
+1 -1
View File
@@ -316,7 +316,7 @@ function UsersRoles({ me }: { me: Me }) {
}
>
<span className="relative hidden sm:inline-block w-20 shrink-0 text-right text-2xs text-muted tabular-nums">
{u.last_login_at ? relativeTime(u.last_login_at, lang) : "—"}
{u.last_login_at ? relativeTime(u.last_login_at, lang, t) : "—"}
</span>
</Tooltip>
<MoreButton
@@ -0,0 +1,61 @@
// @vitest-environment jsdom
//
// The channel tag editor says a failed load of the tag catalog as one, with a retry — not as "no tags
// yet, create one below", which invited a duplicate of a tag that exists. i18n is key-passthrough.
import { describe, it, expect, afterEach, beforeAll, vi } from "vitest";
import { render, screen, fireEvent, cleanup } from "@testing-library/react";
import { FloatingTree } from "@floating-ui/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
vi.mock("react-i18next", async (importOriginal) => {
const actual = await importOriginal<typeof import("react-i18next")>();
return {
...actual,
useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }),
};
});
import { ChannelTagsControl } from "./ChannelTags";
import { api } from "../lib/api";
beforeAll(() => {
class Stub {
observe() {}
unobserve() {}
disconnect() {}
takeRecords() {
return [];
}
}
globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver;
});
afterEach(() => {
cleanup();
vi.restoreAllMocks();
});
describe("the channel tag editor", () => {
it("says a failed load of the tags as one, with a retry — not as no tags yet", async () => {
vi.spyOn(api, "tags").mockRejectedValue(new Error("down"));
const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
render(
<QueryClientProvider client={qc}>
<FloatingTree>
<ChannelTagsControl
userTags={[]}
attachedIds={[]}
onToggle={() => {}}
onCreate={() => {}}
/>
</FloatingTree>
</QueryClientProvider>,
);
fireEvent.click(screen.getByLabelText("channels.row.editTags"));
const alert = await screen.findByRole("alert", { hidden: true });
expect(alert.textContent).toContain("common.loadError");
expect(screen.queryByText("channels.tags.noneYet")).toBeNull();
expect(alert.closest("[role=menu]")).toBeNull();
expect(await screen.findByText("common.retry")).toBeTruthy();
});
});
+26 -70
View File
@@ -1,17 +1,15 @@
import { useRef, useState } from "react";
import { Plus, Sparkles } from "lucide-react";
import { MenuCheck } from "./ui/MenuCheck";
import { useTranslation } from "react-i18next";
import { type Tag } from "../lib/api";
import { PopoverSurface, PopoverItem } from "./ui/AnchoredPopover";
import { PopoverSurface } from "./ui/AnchoredPopover";
import { useAnchoredPopover } from "./ui/anchoredPopoverHooks";
import { useScrollFade } from "../lib/useScrollFade";
import { fitChipsWithPill } from "../lib/chipOverflow";
import { chipLaneClass, useChipOverflowFit } from "../lib/useChipOverflowFit";
import { IconButton } from "./ui/IconButton";
import { Chip, DismissibleChip } from "./ui/Chip";
import { chipClass } from "./ui/chipStyles";
import { Input } from "./ui/Input";
import { MembershipMenu } from "./ui/MembershipMenu";
import { useTags } from "../lib/useTags";
import { SURFACE } from "./ui/surfaces";
// The channel tag surface, shared by the channel manager (table + cards) and the channel detail
@@ -194,8 +192,9 @@ function SuggestedChipsLane({
}
/** The editor body: toggle the user's existing tags on/off for this channel, plus an inline
* "create + attach" input. Rendered inside an anchored `menu` popover. (Exported for the channel
* detail page in Part 2; internal for now.) */
* "create + attach" input. Rendered inside an anchored `menu` popover. The rows are the caller's
* `userTags`; whether that list has loaded is read from the catalog query itself (mounted only while
* the editor is open), so a failed or pending load is said as such, not as "no tags yet". */
function TagEditorMenu({
userTags,
attachedIds,
@@ -208,71 +207,27 @@ function TagEditorMenu({
onCreate: (name: string) => void;
}) {
const { t } = useTranslation();
const [draft, setDraft] = useState("");
const { ref: fadeRef, style: fadeStyle } = useScrollFade();
const tags = useTags();
const attached = new Set(attachedIds);
const submit = () => {
const name = draft.trim();
if (!name) return;
onCreate(name);
setDraft("");
};
return (
<div className="flex flex-col">
<div ref={fadeRef} style={fadeStyle} className="max-h-60 overflow-y-auto no-scrollbar">
{userTags.map((tg) => {
const on = attached.has(tg.id);
return (
<PopoverItem
key={tg.id}
// Several tags can be on at once — a checkbox set, not one exclusive choice.
role="menuitemcheckbox"
checked={on}
onSelect={() => onToggle(tg.id)}
label={tg.name}
className={`w-full flex items-center gap-2 text-left text-xs px-2 py-1.5 rounded-lg outline-hidden transition ${
on ? "text-fg" : "text-muted"
}`}
>
<MenuCheck kind="many" on={on} />
<span className="truncate">{tg.name}</span>
</PopoverItem>
);
})}
{userTags.length === 0 && (
<p className="text-2xs text-muted px-2 py-1.5">{t("channels.tags.noneYet")}</p>
)}
</div>
<div className="mt-1 pt-1.5 border-t border-border flex items-center gap-1">
<Input
className="flex-1"
size="sm"
value={draft}
onChange={(e) => setDraft(e.target.value)}
onKeyDown={(e) => {
// Escape bubbles to the popover's dismiss (close + refocus the trigger); every other key —
// arrows, Home/End, typed characters — is text editing here, so keep it out of the menu's
// list-navigation/typeahead handlers on the surrounding surface.
if (e.key === "Escape") return;
e.stopPropagation();
if (e.key === "Enter") {
e.preventDefault();
submit();
}
}}
maxLength={64}
placeholder={t("channels.tags.createPlaceholder")}
/>
<IconButton
variant="outline"
onClick={submit}
disabled={!draft.trim()}
label={t("channels.tags.create")}
>
<Plus />
</IconButton>
</div>
</div>
<MembershipMenu
dense
rows={userTags.map((tg) => ({ id: tg.id, label: tg.name, checked: attached.has(tg.id) }))}
onToggle={(row) => onToggle(row.id as number)}
status={tags.isError && !tags.data ? "error" : tags.isPending ? "loading" : "ready"}
onRetry={() => void tags.refetch()}
emptyText={t("channels.tags.noneYet")}
errorText={t("common.loadError")}
create={{
placeholder: t("channels.tags.createPlaceholder"),
maxLength: 64,
submitLabel: t("channels.tags.create"),
onCreate: (name) => {
onCreate(name);
return true;
},
}}
/>
);
}
@@ -473,6 +428,7 @@ export function ChannelTagsControl({
<PopoverSurface
popover={editPop}
className={`${SURFACE.menu} w-56 animate-[popIn_0.16s_ease]`}
listInside
>
<TagEditorMenu
userTags={userTags}
@@ -0,0 +1,265 @@
// @vitest-environment jsdom
//
// The clip panel: it opens on its In handle, every way of marking keeps a range the server takes,
// Cut sends the range (and a Plex clip's audio track) and closes, a refused Cut stays open and says
// so, and "Play selection" loops back to In. The range is the caller's state, so the test holds it.
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { act, cleanup, fireEvent, render, screen, waitFor } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { useState } from "react";
import type { ClipRequest } from "../lib/api";
import type { ClipRange } from "../lib/clipRange";
vi.mock("react-i18next", async (importOriginal) => {
const actual = await importOriginal<typeof import("react-i18next")>();
return { ...actual, useTranslation: () => ({ t: (k: string) => k }) };
});
const enqueueClip = vi.fn<(body: ClipRequest) => Promise<unknown>>();
vi.mock("../lib/api", async (importOriginal) => {
const actual = await importOriginal<typeof import("../lib/api")>();
return { ...actual, api: { ...actual.api, enqueueClip: (b: ClipRequest) => enqueueClip(b) } };
});
const notify = vi.fn();
vi.mock("../lib/notifications", async (importOriginal) => {
const actual = await importOriginal<typeof import("../lib/notifications")>();
return { ...actual, notify: (n: unknown) => notify(n) };
});
const notifyActionError = vi.fn();
vi.mock("../lib/actionErrors", async (importOriginal) => {
const actual = await importOriginal<typeof import("../lib/actionErrors")>();
return { ...actual, notifyActionError: (e: unknown, t: string) => notifyActionError(e, t) };
});
import ClipPanel, { type ClipCrop, type ClipSource } from "./ClipPanel";
afterEach(() => {
cleanup();
vi.clearAllMocks();
});
beforeEach(() => {
// A block, not an expression: a function returned from beforeEach is run as its teardown.
enqueueClip.mockResolvedValue({ id: 1 });
});
function Harness({
src = { source: "plex", ref: "777", audioTrack: 1 },
start = { start: 100, end: 130 },
current = 100,
onClose = () => {},
onPlayFrom = () => {},
onSeek = () => {},
duration = 3600,
}: {
src?: ClipSource;
start?: ClipRange;
current?: number;
onClose?: () => void;
onPlayFrom?: (t: number) => void;
onSeek?: (t: number, final: boolean) => void;
duration?: number;
}) {
const [clip, setClip] = useState(start);
const [crop, setCrop] = useState<ClipCrop | null>(null);
return (
<QueryClientProvider client={new QueryClient()}>
<output data-testid="range">{`${clip.start}-${clip.end}`}</output>
<output data-testid="crop">{crop ? JSON.stringify(crop) : "none"}</output>
<ClipPanel
src={src}
clip={clip}
onChange={setClip}
duration={duration}
current={current}
picture={16 / 9}
crop={crop}
onCrop={setCrop}
onSeek={onSeek}
onPlayFrom={onPlayFrom}
onClose={onClose}
/>
</QueryClientProvider>
);
}
const range = () => screen.getByTestId("range").textContent;
describe("ClipPanel", () => {
it("opens with the focus on its In handle, so the arrows move the mark at once", () => {
render(<Harness />);
const handle = screen.getByRole("slider", { name: "clip.in" });
expect(document.activeElement).toBe(handle);
fireEvent.keyDown(handle, { key: "ArrowLeft", shiftKey: true });
expect(range()).toBe("99-130");
});
it("takes a typed time on Enter and carries the other end along past it", () => {
render(<Harness />);
const [inField] = screen.getAllByRole("textbox", { name: "clip.in" });
fireEvent.change(inField!, { target: { value: "2:20" } });
fireEvent.keyDown(inField!, { key: "Enter" });
expect(range()).toBe("140-170");
});
it("keeps the focus in a time field that took a typed time on Enter", () => {
render(<Harness />);
const field = () => screen.getAllByRole("textbox", { name: "clip.in" })[0]!;
field().focus();
fireEvent.change(field(), { target: { value: "2:20" } });
fireEvent.keyDown(field(), { key: "Enter" });
expect(range()).toBe("140-170");
expect(document.activeElement).toBe(field());
expect((field() as HTMLInputElement).value).toBe("0:02:20.0");
});
it("shows the mark it holds, not a time it could not take", () => {
render(<Harness start={{ start: 3500, end: 3600 }} />);
const out = () => screen.getAllByRole("textbox", { name: "clip.out" })[0] as HTMLInputElement;
// Past the end: clamped back to the end, and the field says so.
fireEvent.change(out(), { target: { value: "1:15:00" } });
fireEvent.blur(out());
expect(range()).toBe("3500-3600");
expect(out().value).toBe("1:00:00.0");
expect(out().getAttribute("aria-invalid")).toBeNull();
// Not a time at all: kept for fixing, and marked as not taken.
fireEvent.change(out(), { target: { value: "abc" } });
fireEvent.blur(out());
expect(out().value).toBe("abc");
expect(out().getAttribute("aria-invalid")).toBe("true");
expect(range()).toBe("3500-3600");
});
it("says an unreadable time on a line of its own under the fields, read out, over nothing", () => {
render(<Harness />);
const out = () => screen.getAllByRole("textbox", { name: "clip.out" })[0] as HTMLInputElement;
fireEvent.change(out(), { target: { value: "abc" } });
fireEvent.blur(out());
const line = screen.getByRole("alert");
expect(line.textContent).toBe("common.timecodeInvalidHours");
expect(out().getAttribute("aria-describedby")).toBe(line.id);
// In the panel's flow, after the fields' row: it pushes the crop row down rather than lying over
// its buttons and taking their clicks.
expect(line.closest('[role="group"]')).toBeNull();
expect(line.className).not.toMatch(/\babsolute\b/);
expect(line.parentElement?.className).toMatch(/\bmedia-scope\b/);
});
it("takes a decimal comma, as a Hungarian hand types it", () => {
render(<Harness />);
const field = screen.getAllByRole("textbox", { name: "clip.in" })[0]!;
fireEvent.change(field, { target: { value: "1:42,5" } });
fireEvent.keyDown(field, { key: "Enter" });
expect(range()).toBe("102.5-130");
});
it("never seeks the player onto the item's very end (that would finish the item)", () => {
const onSeek = vi.fn();
render(<Harness onSeek={onSeek} duration={600} start={{ start: 590, end: 595 }} />);
fireEvent.keyDown(screen.getByRole("slider", { name: "clip.out" }), { key: "End" });
expect(range()).toBe("590-600");
const [time, final] = onSeek.mock.lastCall!;
expect(final).toBe(true);
expect(time).toBeLessThan(600);
expect(time).toBeGreaterThan(599);
});
it("marks an end at the playhead", () => {
render(<Harness current={115.5} />);
fireEvent.click(screen.getByRole("button", { name: "clip.outHere" }));
expect(range()).toBe("100-115.5");
});
it("cuts the range with the Plex audio track and closes", async () => {
const onClose = vi.fn();
render(<Harness onClose={onClose} start={{ start: 100.12345, end: 130 }} />);
fireEvent.change(screen.getByRole("textbox", { name: "clip.name" }), {
target: { value: " The bit " },
});
fireEvent.click(screen.getByRole("button", { name: "clip.cut" }));
await waitFor(() => expect(onClose).toHaveBeenCalled());
expect(enqueueClip).toHaveBeenCalledWith({
source: "plex",
ref: "777",
start_s: 100.123,
end_s: 130,
audio_track: 1,
display_name: "The bit",
});
expect(notify).toHaveBeenCalledWith(
expect.objectContaining({ level: "success", message: "clip.queued" }),
);
});
it("sends no audio track for a YouTube clip", async () => {
render(<Harness src={{ source: "youtube", ref: "abcdefghijk" }} />);
fireEvent.click(screen.getByRole("button", { name: "clip.cut" }));
await waitFor(() => expect(enqueueClip).toHaveBeenCalled());
const body = enqueueClip.mock.calls[0]![0];
expect(body.source).toBe("youtube");
expect(Object.keys(body)).not.toContain("audio_track");
});
it("stays open and says so when the cut is refused", async () => {
const onClose = vi.fn();
const err = new Error("boom");
enqueueClip.mockRejectedValue(err);
render(<Harness onClose={onClose} />);
fireEvent.click(screen.getByRole("button", { name: "clip.cut" }));
await waitFor(() => expect(notifyActionError).toHaveBeenCalledTimes(1));
expect(notifyActionError.mock.calls[0]![0]).toBe(err);
expect(notifyActionError.mock.calls[0]![1]).toBe("clip.failed");
expect(onClose).not.toHaveBeenCalled();
expect((screen.getByRole("button", { name: "clip.cut" }) as HTMLButtonElement).disabled).toBe(
false,
);
});
it("loops the selection: playing past End goes back to Start", () => {
const onPlayFrom = vi.fn();
const { rerender } = render(<Harness onPlayFrom={onPlayFrom} current={110} />);
fireEvent.click(screen.getByRole("button", { name: "clip.loop" }));
expect(onPlayFrom).toHaveBeenLastCalledWith(100);
onPlayFrom.mockClear();
act(() => rerender(<Harness onPlayFrom={onPlayFrom} current={125} />));
expect(onPlayFrom).not.toHaveBeenCalled();
act(() => rerender(<Harness onPlayFrom={onPlayFrom} current={130.2} />));
expect(onPlayFrom).toHaveBeenLastCalledWith(100);
});
it("cuts the whole picture until the crop is turned on, then the frame with its box's shape", async () => {
render(<Harness />);
expect(screen.queryByRole("group", { name: "clip.shape" })).toBeNull();
fireEvent.click(screen.getByRole("button", { name: "clip.crop" }));
expect(screen.getByRole("button", { name: "clip.crop" }).getAttribute("aria-pressed")).toBe(
"true",
);
fireEvent.click(screen.getByRole("button", { name: "clip.cut" }));
await waitFor(() => expect(enqueueClip).toHaveBeenCalled());
expect(enqueueClip.mock.calls[0]![0].crop).toEqual({
x: 0.1,
y: 0.1,
w: 0.8,
h: 0.8,
aspect: 1.7778,
});
});
it("holds the frame to a shape without cutting, and goes back to the whole picture", () => {
render(<Harness />);
fireEvent.click(screen.getByRole("button", { name: "clip.crop" }));
// A segmented choice inside the panel's form: never a submit.
fireEvent.click(screen.getByRole("button", { name: "9:16" }));
expect(enqueueClip).not.toHaveBeenCalled();
const tall = JSON.parse(screen.getByTestId("crop").textContent!) as ClipCrop;
expect(tall.shape).toBe("9:16");
expect(tall.rect.h).toBeCloseTo(1, 6);
expect((tall.rect.w * 16) / 9 / tall.rect.h).toBeCloseTo(9 / 16, 6);
fireEvent.click(screen.getByRole("button", { name: "clip.whole" }));
expect(JSON.parse(screen.getByTestId("crop").textContent!)).toEqual({
rect: { x: 0, y: 0, w: 1, h: 1 },
shape: "free",
});
fireEvent.click(screen.getByRole("button", { name: "clip.crop" }));
expect(screen.getByTestId("crop").textContent).toBe("none");
});
});
+468
View File
@@ -0,0 +1,468 @@
import { useEffect, useEffectEvent, useId, useRef, useState, type SubmitEvent } from "react";
import { useTranslation } from "react-i18next";
import { useQueryClient } from "@tanstack/react-query";
import { Crop, Crosshair, Maximize, Minus, Plus, Repeat, Scissors, X } from "lucide-react";
import { api, type ClipRequest } from "../lib/api";
import { DISPLAY_NAME_MAX } from "../lib/api/downloads";
import { invalidateDownloads } from "../lib/downloads";
import { notify } from "../lib/notifications";
import { notifyActionError } from "../lib/actionErrors";
import { navigateTo } from "../lib/nav";
import { log } from "../lib/log";
import { formatTimecode } from "../lib/timecode";
import { handleBounds, markIn, markOut, zoomWindow, type ClipRange } from "../lib/clipRange";
import { fitRatio, fracRatio, WHOLE, type Frac } from "../lib/cropGeometry";
import { RangeTrack, RulerTicks, type RangeWindow } from "./ui/RangeTrack";
import { CropFrame } from "./ui/CropFrame";
import SegmentedToggle from "./ui/SegmentedToggle";
import { Button } from "./ui/Button";
import { IconButton } from "./ui/IconButton";
import { Input } from "./ui/Input";
import { TimecodeError, TimecodeField } from "./ui/TimecodeField";
import { useTimecodeField, type TimecodeDraft } from "./ui/useTimecodeField";
// The clip panel: marks a range of what is playing and cuts it into a download of its own. It lives
// IN the player (the YouTube card under the video, the Plex player's control band), so the video
// being watched is the preview: dragging a handle or pressing a timeline moves the player there.
//
// Two tracks: the whole item, with the selection and the zoomed window on it, and a zoomed strip
// around the selection where the handles are. The strip's window follows the selection when it
// settles, not while a handle is being dragged — a window that moved under the pointer would run
// away from the drag. A film is hours long and a clip seconds: only the strip gives a hand the
// precision, and the fields and their tenth-of-a-second steps give the rest.
//
// The range itself is the caller's state (`clip`), so the player's `[` / `]` keys and this panel
// mark the same range by the same rules (lib/clipRange).
//
// A clip is the whole picture unless the user turns the crop on: then a frame over the video
// (`ClipCropFrame`, which the player places over its picture) marks the part to keep, free or held
// to an output shape. The crop is the caller's state too, so the player's Escape can turn it off
// before it closes the panel.
/** The output shapes a crop frame can be held to (width / height of the clip), or a free frame. */
type ClipShape = "free" | "16:9" | "9:16" | "1:1";
const SHAPES: Record<Exclude<ClipShape, "free">, number> = {
"16:9": 16 / 9,
"9:16": 9 / 16,
"1:1": 1,
};
/** A clip's crop: the frame, in fractions of the picture, and the shape it is held to. */
export interface ClipCrop {
rect: Frac;
shape: ClipShape;
}
const CROP_START: ClipCrop = { rect: { x: 0.1, y: 0.1, w: 0.8, h: 0.8 }, shape: "free" };
const round4 = (v: number) => Math.round(v * 10_000) / 10_000;
const lockOf = (crop: ClipCrop, picture: number) =>
crop.shape === "free" ? null : fracRatio(SHAPES[crop.shape], picture);
export interface ClipSource {
source: ClipRequest["source"];
ref: string;
/** Plex: the audio track that is playing, which the clip carries. */
audioTrack?: number | null;
}
interface Props {
src: ClipSource;
clip: ClipRange;
onChange: (r: ClipRange) => void;
duration: number;
/** The player's position, live. */
current: number;
/** The shape (width / height) of the picture the crop frame is drawn on. */
picture: number;
crop: ClipCrop | null;
onCrop: (c: ClipCrop | null) => void;
/** Move the player; `final` false while a pointer still drags (a player whose seek is costly waits). */
onSeek: (time: number, final: boolean) => void;
/** Play from `time` (the loop's restart, and "Play selection"). */
onPlayFrom: (time: number) => void;
onClose: () => void;
}
const NUDGE_S = 0.1;
// How far short of the item's end a seek stops: a mark on the very end moves the player to its last
// moment, not past it. A player put at its end ENDS, and the item counts as finished — YouTube marks
// it watched and shows its end screen, a Plex position reads as the whole — while a mark there is only
// a place. (The players keep themselves and the clip open at an end while one is marked; this keeps
// the item unfinished.)
const LAST_MOMENT_S = 0.25;
// On the always-dark chrome the ends are white (a light theme's accent is a dark blue there, which
// all but vanished on the bar) and the playhead amber, so an end and the playhead are never confused.
// The media amber (`--media-mark`), not the theme's warn: a light theme's is dark on this bar. Every
// focus ring in the panel is that amber as well (media-scope re-points --focus), as the crop frame's
// is: the theme accent's ring was under 3:1 on the chrome.
const PLAYHEAD = "bg-media-mark";
export default function ClipPanel({
src,
clip,
onChange,
duration,
current,
picture,
crop,
onCrop,
onSeek,
onPlayFrom,
onClose,
}: Props) {
const { t } = useTranslation();
const qc = useQueryClient();
const rootRef = useRef<HTMLFormElement>(null);
const [name, setName] = useState("");
const [busy, setBusy] = useState(false);
const [looping, setLooping] = useState(false);
// The strip's window while a handle drags (null at rest: it follows the selection).
const [frozen, setFrozen] = useState<RangeWindow | null>(null);
const hours = duration >= 3600;
const tc = (s: number) => formatTimecode(s, hours);
const view = frozen ?? zoomWindow(clip, duration);
const bounds = handleBounds(clip, duration);
const length = clip.end - clip.start;
// Opened from a button or a key, the panel takes the focus to its In handle: the button that
// opened it is gone (its row is this panel now), and the arrows then move the mark at once. Without
// scrolling there: the player's card is still fitting itself around the new panel at this point,
// and a card that has to scroll (a short window) scrolled the top of the video out of sight.
useEffect(() => {
rootRef.current
?.querySelector<HTMLElement>('[data-handle="in"]')
?.focus({ preventScroll: true });
}, []);
// The crop turned off from its frame (Escape there): the frame took the focus with it, so it goes to
// the Crop toggle rather than to <body>, where the player's dialog no longer holds it.
const cropButtonRef = useRef<HTMLButtonElement>(null);
const cropOn = crop != null;
useEffect(() => {
if (cropOn) return;
const lost = document.activeElement == null || document.activeElement === document.body;
if (lost) cropButtonRef.current?.focus();
}, [cropOn]);
// "Play selection" loops: past Out, back to In.
const restart = useEffectEvent(() => onPlayFrom(clip.start));
useEffect(() => {
if (looping && current >= clip.end) restart();
}, [looping, current, clip.end]);
const seek = (time: number, final: boolean) =>
onSeek(Math.max(0, Math.min(time, duration - LAST_MOMENT_S)), final);
const moveIn = (time: number, final: boolean) => {
setFrozen(final ? null : (frozen ?? view));
const next = markIn(clip, time, duration);
onChange(next);
seek(next.start, final);
};
const moveOut = (time: number, final: boolean) => {
setFrozen(final ? null : (frozen ?? view));
const next = markOut(clip, time, duration);
onChange(next);
seek(next.end, final);
};
const inField = useTimecodeField(clip.start, hours, (v) => moveIn(v, true));
const outField = useTimecodeField(clip.end, hours, (v) => moveOut(v, true));
const timeErrorId = useId();
const body: ClipRequest = {
source: src.source,
ref: src.ref,
start_s: Math.round(clip.start * 1000) / 1000,
end_s: Math.round(clip.end * 1000) / 1000,
...(src.audioTrack != null ? { audio_track: src.audioTrack } : {}),
...(crop
? {
crop: {
x: round4(crop.rect.x),
y: round4(crop.rect.y),
w: round4(crop.rect.w),
h: round4(crop.rect.h),
aspect: round4(picture),
},
}
: {}),
...(name.trim() ? { display_name: name.trim() } : {}),
};
const submit = async (e: SubmitEvent<HTMLFormElement>) => {
e.preventDefault();
if (busy) return;
setBusy(true);
try {
await api.enqueueClip(body);
invalidateDownloads(qc);
log.info("clip.queued", "clip queued", { source: src.source, seconds: Math.round(length) });
notify({
level: "success",
message: t("clip.queued"),
action: { label: t("clip.view"), onClick: () => navigateTo("downloads") },
});
onClose();
} catch (err) {
// A refusal the server explains (too long, quota) is already in the error dialog; a lost
// connection or a 404 is not, and must not leave the panel looking as if nothing happened.
notifyActionError(err, t("clip.failed"));
setBusy(false);
}
};
return (
<form
ref={rootRef}
onSubmit={submit}
aria-label={t("clip.title")}
data-testid="clip-panel"
className="media-scope flex flex-col gap-2 bg-media-chrome/95 px-3 py-2.5 text-sm"
// The panel's own presses are not the video's (play/pause) nor the backdrop's (close).
onClick={(e) => e.stopPropagation()}
>
<div className="flex items-center gap-3">
<span className="flex shrink-0 items-center gap-1.5 font-medium">
<Scissors className="h-4 w-4" aria-hidden="true" />
{t("clip.title")}
</span>
<RangeTrack
label={t("clip.overview")}
duration={duration}
playhead={current}
playheadClassName={PLAYHEAD}
onSeek={(time, final) => final && seek(time, true)}
className="h-5 flex-1 cursor-pointer rounded-sm bg-white/15"
regions={[
{
key: "window",
start: view.from,
end: view.to,
className: "border-x border-white/50",
},
{ key: "sel", start: clip.start, end: clip.end, className: "bg-accent/70" },
]}
/>
<IconButton variant="overlay" size="sm" onClick={onClose} label={t("clip.close")}>
<X />
</IconButton>
</div>
<RangeTrack
label={t("clip.strip")}
duration={duration}
view={view}
playhead={current}
playheadClassName={PLAYHEAD}
onSeek={seek}
className="h-14 cursor-pointer rounded-md bg-white/10 text-white/45"
background={
<RulerTicks view={view} format={(s) => formatTimecode(s, hours).replace(/\.\d$/, "")} />
}
regions={[
{
key: "sel",
start: clip.start,
end: clip.end,
className: "bg-accent/25 border-y-2 border-accent",
},
]}
handles={[
{
id: "in",
label: t("clip.in"),
time: clip.start,
...bounds.in,
valueText: tc(clip.start),
step: NUDGE_S,
bigStep: 1,
onChange: moveIn,
barClassName: "bg-white",
},
{
id: "out",
label: t("clip.out"),
time: clip.end,
...bounds.out,
valueText: tc(clip.end),
step: NUDGE_S,
bigStep: 1,
onChange: moveOut,
barClassName: "bg-white",
},
]}
/>
<div className="flex flex-wrap items-center gap-x-4 gap-y-2">
<MarkField
label={t("clip.in")}
value={clip.start}
field={inField}
errorId={timeErrorId}
onSet={(v) => moveIn(v, true)}
nudge={NUDGE_S}
hereLabel={t("clip.inHere")}
onHere={() => moveIn(current, true)}
/>
<MarkField
label={t("clip.out")}
value={clip.end}
field={outField}
errorId={timeErrorId}
onSet={(v) => moveOut(v, true)}
nudge={NUDGE_S}
hereLabel={t("clip.outHere")}
onHere={() => moveOut(current, true)}
/>
<span className="tabular-nums text-muted" aria-live="polite">
{t("clip.length", { length: formatTimecode(length) })}
</span>
<Button
variant="overlay"
size="sm"
pressed={looping}
icon={<Repeat />}
onClick={() => {
if (!looping) onPlayFrom(clip.start);
setLooping(!looping);
}}
>
{t("clip.loop")}
</Button>
</div>
{(inField.invalid || outField.invalid) && <TimecodeError id={timeErrorId} hours={hours} />}
<div className="flex flex-wrap items-center gap-2">
<Button
ref={cropButtonRef}
variant="overlay"
size="sm"
pressed={crop != null}
icon={<Crop />}
onClick={() => onCrop(crop ? null : CROP_START)}
>
{t("clip.crop")}
</Button>
{crop && (
<>
<SegmentedToggle
ariaLabel={t("clip.shape")}
value={crop.shape}
onChange={(shape) => {
const next = { ...crop, shape };
const lock = lockOf(next, picture);
onCrop(lock == null ? next : { ...next, rect: fitRatio(crop.rect, lock) });
}}
options={[
{ value: "free", label: t("clip.shapeFree") },
{ value: "16:9", label: "16:9" },
{ value: "9:16", label: "9:16" },
{ value: "1:1", label: "1:1" },
]}
/>
<Button
variant="overlay"
size="sm"
icon={<Maximize />}
onClick={() => onCrop({ rect: WHOLE, shape: "free" })}
>
{t("clip.whole")}
</Button>
</>
)}
</div>
<div className="flex flex-wrap items-center gap-2">
<Input
size="sm"
className="min-w-48 flex-1"
value={name}
maxLength={DISPLAY_NAME_MAX}
onChange={(e) => setName(e.target.value)}
placeholder={t("clip.namePlaceholder")}
aria-label={t("clip.name")}
/>
<Button variant="overlay" size="sm" onClick={onClose}>
{t("common.cancel")}
</Button>
<Button type="submit" variant="primary" size="sm" loading={busy} icon={<Scissors />}>
{t("clip.cut")}
</Button>
</div>
</form>
);
}
/** One end's field: the time typed (Enter or leaving the field sets it), a tenth back and forward,
* and "to the playhead". */
function MarkField({
label,
value,
field,
errorId,
onSet,
nudge,
hereLabel,
onHere,
}: {
label: string;
value: number;
field: TimecodeDraft;
errorId: string;
onSet: (v: number) => void;
nudge: number;
hereLabel: string;
onHere: () => void;
}) {
const { t } = useTranslation();
return (
<div className="flex items-center gap-1" role="group" aria-label={label}>
<span className="mr-1 text-muted">{label}</span>
<IconButton
variant="overlay"
size="sm"
onClick={() => onSet(value - nudge)}
label={t("clip.earlier", { label })}
>
<Minus />
</IconButton>
<TimecodeField
size="sm"
className="w-24 text-center tabular-nums"
field={field}
errorId={errorId}
aria-label={label}
/>
<IconButton
variant="overlay"
size="sm"
onClick={() => onSet(value + nudge)}
label={t("clip.later", { label })}
>
<Plus />
</IconButton>
<IconButton variant="overlay" size="sm" onClick={onHere} label={hereLabel}>
<Crosshair />
</IconButton>
</div>
);
}
/** The clip's crop frame, over the player's picture box (the caller positions this layer over it). */
export function ClipCropFrame({
picture,
crop,
onCrop,
}: {
picture: number;
crop: ClipCrop;
onCrop: (c: ClipCrop) => void;
}) {
return (
<CropFrame
aspect={picture}
rect={crop.rect}
ratio={lockOf(crop, picture)}
onChange={(rect) => onCrop({ ...crop, rect })}
className="z-base"
/>
);
}
@@ -10,7 +10,9 @@ import type { DownloadJob } from "../lib/api";
const mocks = vi.hoisted(() => ({
jobs: [] as unknown[],
confirm: vi.fn(async (_opts: { title: string; confirmLabel?: string }) => false),
confirm: vi.fn(
async (_opts: { title: string; message?: string; confirmLabel?: string }) => false,
),
}));
vi.mock("react-i18next", async (importOriginal) => {
@@ -124,4 +126,58 @@ describe("DownloadCenter confirm dialogs", () => {
["downloads.confirm.deleteTitle", "downloads.actions.delete"],
]);
});
it("cancelling an upload says the uploaded file goes with it", async () => {
// Nothing to fetch again: the cancel deletes the original from the server.
mocks.jobs = [{ ...job(3, "queued"), job_kind: "upload" }];
render(
<QueryClientProvider client={new QueryClient()}>
<DownloadCenter />
</QueryClientProvider>,
);
await pick("Clip 3", "downloads.actions.cancel");
await waitFor(() => expect(mocks.confirm).toHaveBeenCalledTimes(1));
const [opts] = mocks.confirm.mock.calls[0]!;
expect([opts.title, opts.confirmLabel]).toEqual([
"downloads.confirm.cancelUploadTitle",
"downloads.confirm.cancelUploadAction",
]);
});
it("cancelling a clip speaks of the clip", async () => {
mocks.jobs = [{ ...job(4, "running"), job_kind: "edit" }];
render(
<QueryClientProvider client={new QueryClient()}>
<DownloadCenter />
</QueryClientProvider>,
);
await pick("Clip 4", "downloads.actions.cancel");
await waitFor(() => expect(mocks.confirm).toHaveBeenCalledTimes(1));
const [opts] = mocks.confirm.mock.calls[0]!;
expect([opts.title, opts.message, opts.confirmLabel]).toEqual([
"downloads.confirm.cancelEditTitle",
"downloads.confirm.cancelEditBody",
"downloads.confirm.cancelEditAction",
]);
});
it.each([
["upload", "downloads.confirm.deleteUploadTitle", "downloads.confirm.deleteUploadBody"],
["edit", "downloads.confirm.deleteEditTitle", "downloads.confirm.deleteEditBody"],
["download", "downloads.confirm.deleteTitle", "downloads.confirm.deleteBody"],
] as const)("deleting a finished %s says what happens to its file", async (kind, title, body) => {
// Only a download's file may stay cached for other users; an upload's or a clip's is theirs alone.
mocks.jobs = [{ ...job(5, "done"), job_kind: kind }];
render(
<QueryClientProvider client={new QueryClient()}>
<DownloadCenter />
</QueryClientProvider>,
);
fireEvent.click(screen.getByRole("tab", { name: /downloads\.tabs\.done/ }));
await screen.findByText("Clip 5");
await pick("Clip 5", "downloads.actions.delete");
await waitFor(() => expect(mocks.confirm).toHaveBeenCalledTimes(1));
const [opts] = mocks.confirm.mock.calls[0]!;
expect([opts.title, opts.message]).toEqual([title, body]);
});
});
@@ -235,6 +235,46 @@ describe("DownloadCenter · the library row", () => {
]);
});
it("a started item continues — its bar shows how far — and its menu can start it over", async () => {
mocks.jobs = [job(2, "done", { position_seconds: 30 })];
mount("done");
await screen.findByText("Clip 2");
expect(rowButtons("Clip 2")[0]).toBe("card.continueTitle");
// The media cards' bar along the picture's bottom edge: 30 s of the 60 s item.
expect(rowOf("Clip 2").querySelector<HTMLElement>(".bg-accent")!.style.width).toBe("50%");
await openMenu(() => fireEvent.contextMenu(rowOf("Clip 2")));
expect(menuRows().slice(0, 2)).toEqual(["card.continue", "card.restartTitle"]);
// Starting over opens the player at 0:00, not at the kept point.
fireEvent.click(
screen
.getAllByRole("menuitem", { hidden: true })
.find((b) => b.textContent === "card.restartTitle")!,
);
const video = await waitFor(() => {
const v = document.querySelector("video");
expect(v).not.toBeNull();
return v!;
});
let t = 0;
Object.defineProperties(video, {
currentTime: { get: () => t, set: (v: number) => void (t = v), configurable: true },
duration: { get: () => 60, configurable: true },
readyState: { get: () => 1, configurable: true },
});
video.dispatchEvent(new Event("loadedmetadata"));
expect(video.currentTime).toBe(0);
});
it("an item never played offers Play only, with no bar", async () => {
mocks.jobs = [job(2, "done", { position_seconds: 0 })];
mount("done");
await screen.findByText("Clip 2");
expect(rowOf("Clip 2").querySelector(".bg-accent")).toBeNull();
await openMenu(() => fireEvent.contextMenu(rowOf("Clip 2")));
expect(menuRows()).not.toContain("card.restartTitle");
});
it("copies the share link from the row's own Share button", async () => {
mocks.jobs = [job(2, "done")];
vi.spyOn(api, "downloadLinks").mockResolvedValue([] as never);
+16 -4
View File
@@ -68,7 +68,9 @@ export default function DownloadCenter() {
const [manage, setManage] = useState(false);
const [metaJob, setMetaJob] = useState<DownloadJob | null>(null);
const [shareJob, setShareJob] = useState<DownloadJob | null>(null);
const [playJob, setPlayJob] = useState<DownloadJob | null>(null);
// What the player opens, and whether over the kept point ("Play from the beginning").
const [playing, setPlaying] = useState<{ job: DownloadJob; fromStart: boolean } | null>(null);
const setPlayJob = (job: DownloadJob, fromStart = false) => setPlaying({ job, fromStart });
const [editJob, setEditJob] = useState<DownloadJob | null>(null);
const [errorJob, setErrorJob] = useState<DownloadJob | null>(null);
@@ -276,7 +278,9 @@ export default function DownloadCenter() {
<IconBtn
open
onClick={() => setPlayJob(job)}
title={t("downloads.actions.play")}
title={
job.position_seconds ? t("card.continueTitle") : t("downloads.actions.play")
}
>
<PlayCircle className="w-4 h-4" />
</IconBtn>
@@ -315,7 +319,9 @@ export default function DownloadCenter() {
<IconBtn
open
onClick={() => setPlayJob(job)}
title={t("downloads.actions.play")}
title={
job.position_seconds ? t("card.continueTitle") : t("downloads.actions.play")
}
>
<PlayCircle className="w-4 h-4" />
</IconBtn>
@@ -357,7 +363,13 @@ export default function DownloadCenter() {
{manage && <ProfileEditor onClose={() => setManage(false)} />}
{metaJob && <MetaEditModal job={metaJob} onClose={() => setMetaJob(null)} />}
{shareJob && <ShareDialog job={shareJob} onClose={() => setShareJob(null)} />}
{playJob && <LibraryPlayer job={playJob} onClose={() => setPlayJob(null)} />}
{playing && (
<LibraryPlayer
job={playing.job}
fromStart={playing.fromStart}
onClose={() => setPlaying(null)}
/>
)}
{editJob && <VideoEditor job={editJob} onClose={() => setEditJob(null)} />}
{errorJob && (
<ErrorModal
+24 -55
View File
@@ -1,61 +1,30 @@
import { Component, type ErrorInfo, type ReactNode } from "react";
import type { ReactNode } from "react";
import i18n from "../i18n";
import { notify } from "../lib/notifications";
import { errorAttrs, log } from "../lib/log";
import { reloadingForNewBuild } from "../lib/staleBuild";
import { Button } from "./ui/Button";
import { CrashBoundary, type CrashCopy } from "./ui/CrashBoundary";
interface Props {
children: ReactNode;
}
interface State {
crashed: boolean;
}
const copy = (): CrashCopy => ({
title: i18n.t("errors.boundary.title"),
subtitle: i18n.t("errors.boundary.subtitle"),
reload: i18n.t("errors.boundary.reload"),
});
// Catches render-time crashes anywhere in the tree, logs a fatal notification (kept
// in the center across the reload) and offers a recovery action.
export default class ErrorBoundary extends Component<Props, State> {
state: State = { crashed: false };
// A fatal notification, kept in the center across the reload.
const onCrash = () =>
notify({
level: "fatal",
title: i18n.t("errors.boundary.notifTitle"),
// Never the raw `error.message`: a JS engine's English ("Cannot read properties of undefined…")
// is no message for the reader — the `ui.crash` log carries it for diagnosis.
message: i18n.t("errors.boundary.notifMessage"),
requiresInteraction: true,
});
static getDerivedStateFromError(): State {
return { crashed: true };
}
componentDidCatch(error: Error, info: ErrorInfo) {
// The tab is already reloading onto a newer build (lib/staleBuild): this is the chunk it could not
// load, not a crash — no fatal notice, which would outlive the reload.
if (reloadingForNewBuild()) return;
log.fatal("ui.crash", "the app crashed while rendering", {
...errorAttrs(error),
...(info.componentStack ? { component_stack: info.componentStack } : {}),
});
notify({
level: "fatal",
title: i18n.t("errors.boundary.notifTitle"),
// Never the raw `error.message`: a JS engine's English ("Cannot read properties of undefined…")
// is no message for the reader — the `ui.crash` log above carries it for diagnosis.
message: i18n.t("errors.boundary.notifMessage"),
requiresInteraction: true,
});
}
render() {
if (this.state.crashed) {
if (reloadingForNewBuild()) return null;
return (
<div className="min-h-dvh grid place-items-center text-muted p-6 text-center">
<div>
<div className="text-lg font-semibold text-fg mb-1">
{i18n.t("errors.boundary.title")}
</div>
<div className="text-sm mb-3">{i18n.t("errors.boundary.subtitle")}</div>
<Button variant="link" onClick={() => location.reload()} className="text-sm">
{i18n.t("errors.boundary.reload")}
</Button>
</div>
</div>
);
}
return this.props.children;
}
// The app's crash boundary: `CrashBoundary` in the app's language, plus the notice in the center.
export default function ErrorBoundary({ children }: { children: ReactNode }) {
return (
<CrashBoundary copy={copy} onCrash={onCrash}>
{children}
</CrashBoundary>
);
}
@@ -0,0 +1,217 @@
// @vitest-environment jsdom
//
// The feed's optimistic layer: a card shows what was just written to it until the server's own answer
// lands. Which answers count is the rule pinned here: a refetch that LANDS (for what was written
// before it set out), a filter change; never an append of the next page. A mark made while a refetch
// was in flight is newer than its answer, and dropping it put a card marked watched back into an
// Unwatched view. The grid is a stub that shows what Feed hands it.
import { describe, it, expect, vi, beforeAll, afterEach } from "vitest";
import { render, screen, fireEvent, waitFor, cleanup, act } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
vi.mock("react-i18next", async (importOriginal) => ({
...(await importOriginal<typeof import("react-i18next")>()),
useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }),
}));
vi.mock("./wizardContext", () => ({ useWizard: () => ({ openWizard: vi.fn() }) }));
vi.mock("./VirtualFeed", () => ({
default: ({
items,
onState,
fetchNextPage,
}: {
items: { id: string; status: string }[];
onState: (id: string, status: string) => void;
fetchNextPage: (o: object) => unknown;
}) => (
<ol aria-label="results">
{items.map((v) => (
<li key={v.id}>
<span>{`${v.id}:${v.status}`}</span>
<button onClick={() => onState(v.id, "watched")}>watch {v.id}</button>
<button onClick={() => onState(v.id, "hidden")}>hide {v.id}</button>
</li>
))}
<button onClick={() => void fetchNextPage({})}>more</button>
</ol>
),
}));
import Feed from "./Feed";
import { api, DEFAULT_TAG_MODES, type FeedFilters, type Video } from "../lib/api";
import { qk } from "../lib/queryKeys";
beforeAll(() => {
class Stub {
observe() {}
unobserve() {}
disconnect() {}
takeRecords() {
return [];
}
}
globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver;
globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver;
});
afterEach(() => {
cleanup();
vi.restoreAllMocks();
});
const FILTERS: FeedFilters = {
tags: [],
tagModes: DEFAULT_TAG_MODES,
channelIds: [],
q: "",
sort: "newest",
scope: "my",
includeNormal: true,
includeShorts: false,
includeLive: false,
show: "all",
};
const video = (id: string) =>
({
id,
title: id,
channel_id: "UCx",
channel_title: "Channel",
status: "new",
position_seconds: 0,
duration_seconds: 600,
published_at: "2026-01-01T00:00:00Z",
watch_url: `https://www.youtube.com/watch?v=${id}`,
}) as unknown as Video;
// The server never takes the mark in these tests: whatever a list shows as "watched" is the override.
const page = (ids: string[], next: string | null = null) =>
({ items: ids.map(video), next_cursor: next }) as never;
function mount(feed: ReturnType<typeof vi.fn>, setState?: () => Promise<unknown>) {
vi.spyOn(api, "feed").mockImplementation(feed as never);
vi.spyOn(api, "feedSummary").mockResolvedValue({ count: 1, facets: {} } as never);
if (setState) vi.spyOn(api, "setState").mockImplementation(setState as never);
else vi.spyOn(api, "setState").mockResolvedValue({ ok: true } as never);
const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
qc.setQueryData(qk.me(), { id: 1, can_write: true, prefs: {} });
const ui = (filters: FeedFilters) => (
<QueryClientProvider client={qc}>
<Feed
filters={filters}
setFilters={() => {}}
view="cards"
setView={() => {}}
canRead
ytSearch={null}
onYtSearch={() => {}}
onExitYtSearch={() => {}}
/>
</QueryClientProvider>
);
const r = render(ui(FILTERS));
return { qc, rerender: (f: FeedFilters) => r.rerender(ui(f)) };
}
const settle = () => act(() => new Promise((r) => setTimeout(r, 20)));
describe("Feed — optimistic overrides", () => {
it("give way to a refetch that lands after them", async () => {
let answer: (v: unknown) => void = () => {};
const feed = vi
.fn()
.mockResolvedValueOnce(page(["v1"]))
.mockImplementationOnce(() => new Promise((r) => (answer = r)));
const { qc } = mount(feed);
fireEvent.click(await screen.findByText("watch v1"));
await screen.findByText("v1:watched");
act(() => void qc.invalidateQueries({ queryKey: qk.feed() }));
await waitFor(() => expect(feed).toHaveBeenCalledTimes(2));
await settle();
await act(async () => answer(page(["v1"])));
await waitFor(() => expect(screen.getByText("v1:new")).toBeTruthy());
});
it("keep a mark made while the refetch was in flight", async () => {
let answer: (v: unknown) => void = () => {};
const feed = vi
.fn()
.mockResolvedValueOnce(page(["v1"]))
.mockImplementationOnce(() => new Promise((r) => (answer = r)));
const { qc } = mount(feed);
await screen.findByText("v1:new");
act(() => void qc.invalidateQueries({ queryKey: qk.feed() }));
await waitFor(() => expect(feed).toHaveBeenCalledTimes(2));
await settle();
fireEvent.click(screen.getByText("watch v1"));
await screen.findByText("v1:watched");
// The answer the server gave before it heard of the mark.
await act(async () => answer(page(["v1"])));
await settle();
expect(screen.getByText("v1:watched")).toBeTruthy();
});
it("keep a mark sent before the refetch set out but not answered by then", async () => {
// The server may read the list before it commits the mark: that answer is older than the mark.
let answer: (v: unknown) => void = () => {};
let commit: (v: unknown) => void = () => {};
const feed = vi
.fn()
.mockResolvedValueOnce(page(["v1"]))
.mockImplementationOnce(() => new Promise((r) => (answer = r)));
const { qc } = mount(feed, () => new Promise((r) => (commit = r)));
fireEvent.click(await screen.findByText("watch v1"));
await screen.findByText("v1:watched");
act(() => void qc.invalidateQueries({ queryKey: qk.feed() }));
await waitFor(() => expect(feed).toHaveBeenCalledTimes(2));
await settle();
await act(async () => answer(page(["v1"])));
await settle();
expect(screen.getByText("v1:watched")).toBeTruthy();
await act(async () => commit({ ok: true }));
});
it("are taken back by a failed write only while no later write stands", async () => {
// Marked watched, then hidden before the first answer came back: the refused mark must not take
// the hide with it and put the card back in the view.
const writes: { resolve: (v: unknown) => void; reject: (e: unknown) => void }[] = [];
mount(
vi.fn().mockResolvedValue(page(["v1", "v2"])),
() => new Promise((resolve, reject) => writes.push({ resolve, reject })),
);
fireEvent.click(await screen.findByText("watch v1"));
fireEvent.click(await screen.findByText("hide v1"));
await waitFor(() => expect(screen.queryByText(/^v1:/)).toBeNull());
await act(async () => writes[0]?.reject(new Error("refused")));
await settle();
expect(screen.queryByText(/^v1:/)).toBeNull();
await act(async () => writes[1]?.reject(new Error("refused")));
await waitFor(() => expect(screen.getByText("v1:new")).toBeTruthy());
});
it("survive an append of the next page", async () => {
let answer: (v: unknown) => void = () => {};
const feed = vi
.fn()
.mockResolvedValueOnce(page(["v1"], "c1"))
.mockImplementationOnce(() => new Promise((r) => (answer = r)));
mount(feed);
fireEvent.click(await screen.findByText("watch v1"));
await screen.findByText("v1:watched");
fireEvent.click(screen.getByText("more"));
await waitFor(() => expect(feed).toHaveBeenCalledTimes(2));
await settle();
await act(async () => answer(page(["v2"])));
await screen.findByText("v2:new");
await settle();
expect(screen.getByText("v1:watched")).toBeTruthy();
});
it("are dropped by a change of the filters", async () => {
const { rerender } = mount(vi.fn().mockResolvedValue(page(["v1"])));
fireEvent.click(await screen.findByText("watch v1"));
await screen.findByText("v1:watched");
rerender({ ...FILTERS });
await waitFor(() => expect(screen.getByText("v1:new")).toBeTruthy());
});
});

Some files were not shown because too many files have changed in this diff Show More