Compare commits

..
54 Commits
Author SHA1 Message Date
peter e804ff3b8e release: 0.95.0 (improvement/p1-upload-auth-first)
gate / frontend (push) Successful in 4m35s
gate / backend (push) Successful in 6m18s
2026-09-23 01:32:45 +02:00
peter cd6f119883 chore(release): 0.95.0
The P1 epic: the upload route reads its own body after `require_human`
resolves, so an anonymous caller can no longer make the server spool 500 MiB
before the auth check runs. Three deep-review fix rounds on top of it.
2026-09-23 01:32:32 +02:00
peter 321776480e fix(uploads): make the drift guards fail for the reasons they claim
Deep-review round 3, targeted at the surfaces round 2's fix commit changed.
Five of the eight findings are the same shape: the guards written to stop
half-done sweeps were themselves half-done.

test_upload.py's display-field guard walked only `ast.Assign`, so the one
writer with an unbounded REQUEST HEADER behind it — `spool_upload`'s
`display_name=display_text(...)` keyword — was invisible to the guard meant
to protect it. It also stopped at `app/routes/`, missing the four writers in
`downloads/` that actually construct the row, and asserted `not offenders`,
which enforces nothing once a refactor moves the writers out of its view.
It now walks assignments AND keywords across `app/`, and holds exact set
equality against a registry of all twelve write sites with each one's value.

test_body_limit.py's release guard used `any(... ast.walk(fn))`, which
accepts a `release_read_connection` that runs AFTER the await it exists to
precede — satisfying the letter of the rule while pinning a connection for
exactly the transfer it was supposed to free. Position is checked now, and a
late release is its own, louder failure. The registry keyed sites by file
basename while scanning recursively, and `admin.py`, `discover.py` and
`playlists.py` each exist both at the top level and under `plex/` — the same
entry-collapse the registry was widened to close. Keyed by relative path now.
And all three guards recognised a body read only by the `capped_` naming
convention, so a route spelling `await request.body()` inline declared no
ceiling, released no connection, and was invisible to every one of them.

`release_read_connection`'s precondition claimed the caller "must have
NOTHING PENDING, and that is checked rather than described" while checking
only UNFLUSHED ORM work. Measured: after `add`+`flush` all three collections
are empty and the rollback loses the row, and Core DML never enters them at
all — both of them this codebase's canonical write shapes. It now also asks
the transaction directly via `pg_current_xact_id_if_assigned()`.

`errors.tooLarge` was reworded to "try a smaller file", but that key answers
every route's body cap: the issue composer's over-size JSON draft got advice
about a file it does not have. Back to payload-neutral; the file-specific
sentence keeps its own key.

The upload dialog's new danger border outranked `dragOver`, so a user
replacing an over-cap file got no sign the drop zone was live — and with a
file picked the label shows its name rather than "drop it here", so the
border was the only feedback left.

Four further findings were refuted 3/3 and are not fixed: `aria-invalid` on
the drop zone (measured in Chromium — the AX node does carry it on a native
button), a bidi-override spoof through `display_text` (the served filename
already strips those categories, and the proposed filter would have broken
every ZWJ emoji), and `capped_body` not draining (the screenshot route's edge
ceiling equals its own cap, so that refusal is unreachable, and the log sink
is sent by beacon with nobody reading the answer).
2026-09-23 01:20:22 +02:00
peter 7c0647f87a fix(uploads): finish the sweeps round 1 left half-done
Deep-review round 2, on round 1's own fix commit. It found MORE than round 1
did (28 raw, against 13), and most of it was mine -- a new piece of UI attracts
findings a backend-only diff never does. The pattern in almost all of them is
the same: a principle applied to the site that prompted it and not to its
siblings.

* `display_text` reached three of the FIVE writers of a user-supplied name.
  `POST /api/downloads` and `POST /api/downloads/edit` still handed Postgres an
  unbounded string, so a 300-character name pasted into the Add-download dialog
  or the editor was a bare 500 (main.py re-raises every DataError but the NUL
  case). Four of the seven reviewers found it independently. The bound now lives
  in `models.py`, the three columns are DECLARED with it, and the guard for the
  next writer is parsed rather than remembered: every `display_name` /
  `display_uploader` assignment under `app/routes/` must go through the helper.
  That guard immediately found two more -- the Google sign-in callback writes
  the profile name into the same 255-wide column -- so it is six, not five.

* The connection release reached one of the THREE routes that read their own
  body. The screenshot route (10 MiB, from a phone) and the client log sink both
  still held a pooled connection idle-in-transaction for the length of the
  upload, which is the invariant db.py sizes the pool on. All three now call
  `db.release_read_connection`, whose precondition -- nothing pending on the
  shared session -- is CHECKED rather than described, because a whole-session
  rollback discarding a caller's staged write is a mistake this repo has already
  made once. `test_body_limit.py` holds the rule for the next such route.

* The AST drift registry keyed its sites by (module, limit expression), so a
  second route copied from the first collapsed onto the same entry and went
  uncovered -- the exact hole the registry exists to close. Keyed per call site
  now, and it carries the release requirement too.

* The dialog's own refusal was announced once and never again: two over-cap
  picks leave an identical node with identical text, and an untouched live
  region says nothing. It is keyed per pick now, the drop zone reads as invalid
  and points at the sentence, and the reason is reachable from the control
  rather than only from a one-shot announcement.

* Two tests rebuilt the blind spot they were written to remove: the dialog's cap
  test sat nowhere near the boundary (a `>` loosened to `>=` survived it, which
  is the mutation round 1 fixed on the server side), and the header-precedence
  test only covered an exact-case duplicate while a differently-cased one would
  have been COMBINED onto the wire rather than replaced. Both fixed, and the
  merge now dedupes case-insensitively because header names are.

Also: the upload dialog's unstubbed usage query was firing a real request from
jsdom in a neighbouring suite and leaking a delayed "connection lost" toast
across test boundaries; `GET /api/downloads/usage` had no test at all though the
whole client-side check hangs off it; and the comment claiming the edge 413 is
readable now says what was measured -- readable to 200 MiB over loopback, a bare
`onerror` through a proxy -- so the pre-send check is named as the mechanism and
that branch as the backstop it is.

Every test added or changed here was mutation-checked. One mutation SURVIVED and
is recorded rather than papered over: the `tooBig` term inside `submit()` cannot
be reached, because `ui/Button` swallows the activation itself, so it is marked
as belt and braces instead of being described as the guard.
2026-09-22 23:31:55 +02:00
peter efd7b217ca fix(uploads): make both caps answerable, and stop holding a connection
The epic-end deep review found three defects in P1 and four tests that did not
hold what they claimed. They land together because they share the upload route
and its test module; splitting by hunk would have produced commits that do not
individually run green.

* The edge ceiling and the route's own cap are the SAME number at the default
  configuration, so the edge always answers first: a declared Content-Length
  over the cap is refused before a byte is read, and an undeclared body trips
  the middleware on the same chunk the route would have tripped on. The route's
  numbered 422 -- and the drain written for it last week -- are reachable only
  once an admin LOWERS the knob, which is the one shape the browser UAT tested.
  Measured: headroom does not fix this, because the declared-length check fires
  first for any body big enough to matter and the drain pushes the count back
  over the edge for the rest. So the edge's own refusal is made readable
  instead: `errors.tooLarge` localizes `too_large` the way `upload_too_large`
  was already localized, and the dialog refuses an over-cap file BEFORE sending
  it, off the `upload_max_bytes` that GET /api/downloads/usage now reports.
  Measured in Chrome against the API directly: that 413 arrives as `onload`
  with a readable body in 43 ms.

* Reading the body behind the auth dependency put the whole transfer inside the
  request's database session -- the auth SELECT autobegins a transaction, so one
  of the twenty pooled connections sat `idle in transaction`, with nothing to
  time it out, for as long as the client cared to keep sending. That is the
  invariant db.py sizes the pool on and plex/playback.py records an incident
  for. The read transaction now ends before the await. Measured on the dev
  server: an 1818 ms upload shows no idle-in-transaction sample while the bytes
  move, only afterwards, when the handler actually writes.

* The display name moved into a request header, where nothing bounds its length,
  while the column stayed varchar(255) -- so a long enough name was a DataError
  500 after the whole file had already transferred. One `display_text` helper
  now serves all three writers (the metadata PATCH already truncated, the upload
  ingest did not), and the dialog stops at the same number.

* The gate invariant asked `route.dependant.body_params`, which holds only the
  route's OWN callable's parameters: a body declared in a SUB-DEPENDENCY -- the
  upload route's own idiom -- leaves it empty while FastAPI still reads the body
  before auth. It asks `route.body_field` now, which is what FastAPI itself
  consults. The AST registry of body-reading call sites had also never followed
  `capped_stream_to`, so the one route the whole invariant exists for was the
  single site the drift guard could not see.

Four tests that passed over their own mutation are fixed: the cap boundary (a
`>` loosened to `>=` survived the entire suite), the percent-decoding (every
filename the suite sends is ASCII, where quote() is the identity function), the
display-name header (no test at all), and the staging cleanup (deleting the
unlink left the suite green while every refused upload leaked its partial bytes
onto the volume). Every test added or changed here was mutation-checked.
2026-09-22 22:41:22 +02:00
peter 3d7ef9e945 fix(uploads): drain an over-cap body before refusing it
Found in the browser UAT, not by the gate. Moving the cap onto the arriving
bytes meant it fires while the client is still sending, and answering + closing
mid-upload is what XHR reports as a network failure: the upload dialog said
"the server couldn't be reached" while the access log showed a correct 422
carrying the limit. A refusal the client cannot read is an outage as far as the
user is concerned.

`capped_stream_to` now reads the remainder to nowhere before raising, so the
client finishes sending and reads the real answer -- re-measured in the
browser, which now names the limit. Nothing past the cap is written, and the
request is still bounded by bodylimit's ceiling, so the cost is bandwidth an
AUTHENTICATED caller was sending anyway. The anonymous path is untouched:
auth still resolves before a byte is read.

The regression test sits on the helper, not the route, and the reason is worth
keeping: the route-level version was written first and PASSED with the drain
deleted. httpx's ASGI transport pumps the request body whether or not the app
reads it, so "drained" and "abandoned" are indistinguishable through
TestClient. Its own mutation check is what exposed it.
2026-09-22 02:11:21 +02:00
peter 7fce7a7ce5 chore(deps): drop python-multipart -- nothing parses a multipart body now
Its own entry in requirements.txt said the download center's local-file upload
was "the only multipart endpoint in the app". Since that route reads a raw
body, the claim is false and the package is dead weight: no route declares
File/Form, and nothing calls `request.form()`.

Removed from requirements.txt and from its single pinned line in
requirements.lock. The lock is generated and normally regenerated wholesale,
but a full regeneration would bump every transitive dependency, which has no
business riding inside a security fix -- the next legitimate regeneration
reproduces exactly this state, since the package is gone from the source.

The `multipart` entry in logs.py's third-party level map goes with it: a level
for a logger that can no longer exist.
2026-09-22 01:59:09 +02:00
peter f5018bdcbc fix(uploads): read the body after authenticating, not before it
`POST /api/downloads/upload` declared `file: UploadFile = File(...)`, and
FastAPI parses and spools a declared body before the first dependency runs --
`require_human` included. An unauthenticated caller could therefore make the
process write the whole per-file allowance (500 MiB by default) to temp, once
per request. SEC-20's ceiling bounded that number but could not stop the
writing, which is why E27 parked it rather than closing it.

The route now takes no body parameter. `spool_upload` names `require_human`
first and only then streams `request.stream()` to the staging temp file, so a
stranger is refused before a byte moves -- the shape the screenshot route has
used since E16. Filename and display name travel as percent-encoded headers
(latin-1 on the wire, so a Hungarian filename needs the encoding) instead of
multipart parts.

`capped_stream_to` joins `capped_body` in routes/_common.py: the same counting
rule, written to a file descriptor rather than accumulated, because holding
500 MiB would trade a disk problem for a memory one. Its refusal is the
caller's, so the upload route keeps answering 422 `upload_too_large` with the
admin knob's own number.

With no envelope to allow for, the edge is now exactly that knob, and
MULTIPART_ENVELOPE_ALLOWANCE is gone.

The invariant stops being prose: test_body_limit.py asks FastAPI itself
(`route.dependant.body_params`) and fails the gate if any route with a raised
ceiling declares a body. A comment could not have prevented the defect it
described; that test would have.

Both new tests were seen red for the right reason -- before the fix the
anonymous probe got its 401 only after the server had read 4236 bytes of a
multipart body, and re-declaring the File parameter turns the invariant test
red naming the route and the parameter.
2026-09-22 01:59:03 +02:00
peter b82c22b888 release: 0.94.0 (improvement/e27-contract-hardening)
gate / frontend (push) Successful in 4m32s
gate / backend (push) Successful in 6m8s
2026-09-22 00:34:12 +02:00
peter f839a461e6 chore(release): 0.94.0 — framework & contract hardening 2026-09-22 00:34:06 +02:00
peter 8525666913 test: round 4 — an assertion that could not fail, and a comment that did not add up
Two findings, both in the review's own work; nothing in the product code. That is
the shift that ends the loop: 22 → 13 → 6 → 2, and the last two are a test and a
comment rather than a defect.

The "an override that is fine is still honoured" leg of the bounds-fallback test
stored `spec.max` — which, for this key, IS the env default, because the spec's
max is derived from it. So the assertion was satisfied by the fallback it existed
to rule out: mutating `_coerce` to reject EVERY int override left all three legs
green. It stores `spec.max - 1` now and asserts the value differs from the
default, so the leg can actually fail.

And the shrink test's comment claimed `3 - 5 = -2` for an expression that computes
`chips.length - visible` = `2 - 3 = -1` — naming neither operand nor the result,
in a comment that was rewritten one round earlier precisely for miscounting.
2026-09-22 00:22:30 +02:00
peter 3f24248e01 fix: round 3 — a vacuous ordering test, a clamp that was a silent rewrite, a regex that missed
Round 3 over round 2's fixes. The trend is 22 → 13 → 6, and the character has
shifted: these are mostly defects in the REVIEW's own work, not in the epic.

The ordering test I added to hold the middleware's position could not fail for
either arrangement it names. All four `@app.middleware("http")` layers — the
access log, the two header decorators AND the setup gate that must be inside —
register as the same `BaseHTTPMiddleware`, so asserting on the class passes for
the pre-move stack and for a cap moved back outside CORS. It names the layers by
their dispatch function now, which is the only thing that distinguishes them.

The `sysconfig` bounds check CLAMPED, and clamping is a silent rewrite — the thing
this module refuses to do everywhere else, where an unusable override falls back
to the env default and logs once. On the max side the difference is cosmetic; on
the min side it is a live bug. `download_upload_max_bytes` had `min=0` before E7,
and a stored 0 means "no per-file cap" to the route that reads it. Clamped to 1 it
becomes a ONE-BYTE cap: uploads dead, nothing in the log, the admin page showing
"1". It raises `OutOfRange` now and takes the existing fallback, with its own
event code.

The `capped_body` sweep grepped for an identifier as the limit argument, so a site
written `capped_body(request, 4 * 1024 * 1024)` — or with the parameter not named
`request` — was MISSED, and a miss reads as "no new sites". Parsed with `ast`
instead: every call is seen whatever it looks like, and an unfamiliar argument
shape fails the equality instead of vanishing.

Frontend: the shrink test drove the mock in a way the real hook cannot produce
(it moved `visible` in the same step as the shrink, and the comment miscounted the
tags), so it reproduced the right END STATE by accident rather than the sequence.
`visible` is now left alone across the shrink, which IS the precondition.

Ops: the baseline guard answered "did this run rewrite it?" and therefore went
green on the second run over an already-tightened, uncommitted baseline — the
exact state its own comment says marks the localdev stack stale ever after. It
asks both questions now. And `.dockerignore` joins BAKED_PATHS: it decides what
the COPY lines actually take, so it is an input to the build context exactly as
the copied trees are.
2026-09-22 00:12:30 +02:00
peter 6b86673af4 chore: untrack tsconfig.tsbuildinfo, and hold the CI baseline the way the local gate does
`frontend/tsconfig.tsbuildinfo` was swept into the previous commit by a `git add
-A`. It is a machine-local incremental artifact carrying absolute Windows paths,
no lane reads or regenerates it, and it was already stale in the commit that added
it — it does not list the contract test that same commit created. Untracked, and
`*.tsbuildinfo` ignored, because the day anything does start rewriting it a green
gate would leave a tracked file dirty: the exact hazard the basedpyright baseline
just got a guard for.

And CI ran bare basedpyright while the local lane gained that guard, so a
burn-down would have been green on the server and red on the developer's machine
over the same tree. Two gates, one tree, one answer.
2026-09-21 23:59:59 +02:00
peter d924c665f8 fix(frontend): round 2 — the popover latch reset read a stale, negative count
The reset I added guards on `hidden === 0`, and `hidden` can be NEGATIVE.
`useChipOverflowFit` keeps `visible` in state and re-decides in a layout effect,
so the render in which the chip set SHRINKS still carries the previous
measurement — delete three tags under an open "+N" and `chips.length - visible`
goes to -2, the equality never matches, and the flag survives the one window the
reset exists to close. The next re-fit brings the menu back on its own, which is
the original defect with an extra step. Clamped at all three lanes, with a test
that drives the shrink through the stale value.

Also: the Scheduler characterization banner said "the four tests below" and now
sits above six, the last two of which are genuine regression guards for the
range-refusal fix. A future mutation check would have read it, seen them go red,
and concluded the harness was broken rather than that the guard worked.
2026-09-21 23:59:59 +02:00
peter aa51644f5f fix(security): round 2 — the SEC-20 fix had four defects of its own
A second review round over the first round's fixes, which is where the remaining
real bugs usually are. All four are the same failure as the round it reviewed: a
claim stated in a comment that the code does not hold.

The middleware moved inside CORS, but the setup gate is registered LATER and is
therefore OUTSIDE it — so the comment's "the setup gate never runs for a body that
is already too large" was false the moment it was written. On an unconfigured
instance that gate opens a DB session per request, which is exactly the anonymous
pre-auth work SEC-20 exists to remove. The cap now sits between them, and the
order is asserted against `app.user_middleware` rather than described: prose about
a stack that `add_middleware` builds inside-out is how this got backwards.

The edge ceiling and the route's own cap were pinned EQUAL, but they measure
different things — the edge counts raw body bytes, the route counts the decoded
file. A file of exactly the advertised maximum was refused one layer before the
route that would have taken it. The edge now allows for the multipart envelope,
and the test asserts the relation and the gap instead of equality.

`sysconfig`'s new `max` was enforced on write only. That bound did not exist
before this epic, so a value that was legal when it was written can be sitting in
the database — served unclamped it produces exactly the drift the comment says is
impossible, with the admin page showing a number outside its own range. Int
overrides are clamped on read now, which also survives an env default being
lowered later.

And `DOWNLOAD_UPLOAD_MAX_BYTES=0` meant "no per-file cap" to the route but a hard
zero to the edge: every upload refused, including the admin page's own way back,
recoverable only by editing env and restarting. The setting takes `ge=1`.

Two test gaps from the same round: the header assertion that is the ONLY check of
the middleware's position was DB-gated and skipped on a host without Postgres, and
the headroom "ratio" the docblock calls load-bearing was held as `<=`, i.e. no
ratio at all. Both fixed, plus a sweep that every `capped_body` call site in the
routers is known and fits under its ceiling — the point of that list is the third
entry, not the two that exist today.
2026-09-21 23:59:45 +02:00
peter 378b7559e4 build(frontend): finish the React 19 move, and hold the versions that were left in prose
react-i18next was still on ^14.1.2. The E27 S1 plan named it as "the one peer that
actually needs moving — React 19 support is the 15 line", and then it did not
move. The 14 line's <Trans> reads `element.ref`, removed in React 19, and its
withTranslation types are built on the GLOBAL JSX namespace @types/react 19 no
longer declares — both latent only because this app imports neither. 15.7.4 fixes
both at the source (`c.props.ref ?? c.ref`, `React.JSX.LibraryManagedAttributes`).

`reactEngine.contract.test.ts` joins its Node and Python twins and holds what was
only a comment: the compiler's `target` against the installed React major, react
and react-dom together, the types against the runtime, and react-i18next's floor.
A stale target is not a corrupt bundle — the plugin validates the option — but it
is a silent one, since tsc never sees Babel's output.

pythonEngine gains the seventh spelling nobody held: `requirements.lock`'s own
regeneration recipe names a python image, and wheels resolve against THAT
interpreter, so a recipe left behind yields a lock that does not match the image
installing it.

The type lane now covers `alembic` too, for the reason the ruff lane was widened
to it — migrations run on every container start, so one that raises crash-loops
prod. It came back at zero new errors, so the baseline is untouched. `tests/` is
deliberately still out, and the config says why.

vitest.config's docblock claimed the React transform is "a no-op for the plain .ts
node tests". It is not: the compiler rewrites src/lib's hooks in that lane too,
which is the point of sharing one factory with the build — and worth knowing
before ruling the compiler out of a divergence.

Tests: useCardPager had none, in either the hook or its two callers, though its
page clamp moved to render in S2 — the transformation where a non-convergent
condition is a crash rather than a glitch. Four cases now, including termination
on an empty list. Combobox's and Avatar's close/failed-URL cases are relabelled
CHARACTERIZATION: both pass against the pre-sweep code, and the tests beside them
are the ones that discriminate.
2026-09-21 23:33:02 +02:00
peter 05121fbc99 fix(security): SEC-20's exemptions become raised ceilings, and the 413 is dressed
The module admitted its two opt-outs under a stated rule — "both enforce their own
limit on the bytes they actually read, which is what makes opting them out safe" —
and that rule is false for one of them. `POST /api/downloads/upload` declares
`file: UploadFile = File(...)`, so FastAPI parses and spools the whole multipart
part to a temp file BEFORE the first dependency runs, `require_human` included:
its in-handler cap measures bytes that have already landed. Measured on the
installed fastapi 0.141.1 / starlette 1.6.0 — starlette's max_part_size guards
non-file parts only, so an anonymous chunked upload was bounded by nothing. In the
shipped self-host compose that lands in a RAM-backed tmpfs.

The exposure predates SEC-20 (no route was capped before it); what this branch
added was the claim. So the list becomes a mapping of per-route CEILINGS instead
of a set of exemptions: every request now gets a finite number, the upload route's
being the same one the route itself enforces. `sysconfig`'s knob is bounded by it
so an admin cannot raise the handler's cap past the edge and start collecting 413s
from a layer the admin page never mentions — held by a test, not by prose.

Two more things the review caught in the same module:

The 413 was not byte-identical to `capped_body`'s after all. `json.dumps` puts
spaces where JSONResponse does not, and the test that promised to hold them
together compared PARSED JSON, so it could never have seen it. Built through the
same JSONResponse now, and compared as raw bytes.

And the middleware sat outside CORS and the security headers, so its 413 carried
neither — on the split-origin dev setup the browser drops it and the user gets a
generic network failure instead of the translated too_large, while the identical
refusal from capped_body one layer down renders fine. It moves to just inside
CORS: still ahead of the session, the setup gate and every auth dependency.

Also corrected: the docblock's "largest non-exempt body is 64 KiB, ~16x headroom"
(it is 512 KiB and 2x — the diagnostics sink, named eleven lines further down),
and CLAUDE.md's Request SIZE row, which forbade the one correct use of
capped_body that is actually in the tree.
2026-09-21 23:32:49 +02:00
peter bf518223e5 fix(frontend): the scheduler's two number editors stop swallowing what you typed
Both refused an out-of-range value by discarding it: the entry vanished, the box
snapped back to the server's number, and nothing said whether the save had
happened, been rejected, or been ignored. That is the house rule about honest
states, applied to a write instead of a read. They now keep the draft, mark the
field aria-invalid and name the range (HU+EN), and the Save control is disabled
rather than silently inert.

The interval editor also dropped a keyboard user's focus on every exit — Escape,
Cancel and Save all unmount the focused input — so it hands focus back to the
pencil that opened it, the pattern IssueComposer already uses.

Tests: the four interval-editor cases added in S2 are relabelled CHARACTERIZATION,
because they pass against the pre-sweep code too (the old effect was already
guarded by `editing`). The editor that genuinely lost typing to a poll is the
maintenance batch box, whose old effect had no guard at all, and it had no such
test — it does now, anchored on a value that follows the server so it cannot pass
on a page that simply has not polled yet.
2026-09-21 23:32:32 +02:00
peter 332d24a646 fix(frontend): four state defects the deep-review found in the E27 S2 sweep
Every one of them is the same shape: a value derived correctly, and a WRITE to
the thing it derives from left unattended. Deriving hides a stale value; it does
not consume it.

PlayerModal — the watch toggle inherited the previous video's auto-watch. Moving
the queue-advance sync into render was right for `currentVideoId`, but the same
block also synced `status`, and that one had an ordering constraint of its own:
useYtProgress's cleanup for the OUTGOING item can still call setWatched(true)
near the end of a clip, and cleanups run after the commit. As an effect the sync
landed last and corrected it; from render it lands first and the stray write
wins. Press Next in the final ten seconds and the NEXT video showed as watched —
one click on the toggle then sent onState(next, "new") for an item already new.
Fixed by keying the mirror to the id it describes, so a write for another item is
ignored rather than displayed, and no ordering has to be argued about at all.

ChannelsProvider — a focus-channel intent was masked by the page rather than
consumed, and nothing ever cleared it. The manager is a page chunk: it unmounts
on every navigation, and its seed effect runs again on each mount, so returning
to it re-applied a channel name from earlier in the session over whatever the
user had typed since. The consumed marker now lives in the provider, keyed on the
token — which also closes the case the old clear-on-leave effect never handled,
opening a channel page (that does not change `page` at all).

TagFilterChips + ChannelTags x2 — `open = openRaw && hidden > 0` hides the
overflow popover when the lane stops overflowing, but floating-ui only reports a
close for a real event on a mounted surface, so the raw flag stayed true and the
menu came back by itself on the next re-fit — a window resize or the text-size
slider is enough, with nobody near it. The flag is now reset in render against
the same condition that hides it.

Avatar + downloadRow — remembering a failed image BY URL fixed the recycled-row
frame but never forgot it, so a transient 429 pinned a channel to its initial for
the life of the instance, never re-requesting. The memory is now dropped once
`src` moves off it, which keeps both properties.

And playerModalHooks' last exhaustive-deps suppression claimed "nothing in the
body reads a render value" while reading five. The invariant that is actually
load-bearing is that each of them is a per-item snapshot — written down instead.

All four fixes are mutation-checked: each new test was seen red, against the
fixed line only, naming the real defect.
2026-09-21 23:32:22 +02:00
peter c5e022f056 refactor(frontend): the long tail — and eslint-suppressions.json is gone (E27 S2)
Closes the burn-down. Hook suppressions 113 -> 34, `exhaustive-deps` 45 -> 2 (the target was
<10), and every one of the 34 survivors is an inline directive NAMING the invariant it
protects. The suppressions file is deleted, because a bulk file hides the next violation
instead of answering it — and it cannot hold a warning-level rule at all.

The sprint plan's three verdicts missed the answer that turned out to be the most common one:
adjust the state DURING RENDER. React re-runs the component and discards that pass rather than
committing it, so the stale frame an effect-based reset paints does not exist. Applied to the
Plex player's up-next card, the playlist view reset, the feed's optimistic overrides, the
channel banner's retry count, the download center's tab guard, the Plex search highlight, the
comfort-pin latch, Welcome's per-view outcome and the card pager's page clamp. It is legal only
for a component's own state — PlexBrowse's language commit resets PlexProvider's filters, so
that one stays an effect and says why.

Mirrors of a prop or query became derived values or `draft ?? server`:
- `downloadRow` had ui/Avatar's defect exactly — a boolean "poster failed" that an effect reset,
  in rows recycled down a virtualized list, so one job's missing poster painted over the next
  one's for a frame. Keyed by URL now.
- Scheduler's two number editors held a mirror an effect re-synced, on a page that POLLS — the
  shape a poll can overwrite mid-edit. A draft while editing, the server's value otherwise; six
  tests where there were none, the one behavioural change (the save button stops being dirty at
  commit rather than at the echo) mutation-checked.
- SetupWizard's "no token" and WatchPage's "gone" are facts about the URL, so they are seeded and
  derived rather than pushed into state by an effect that had to commit first.

Elsewhere: PlexInfo's rating chip was a component declared during render — a new component type
every render, so React remounted the subtree and dropped the focus on its filter button. Hoisted.
Feed's undo action and PlexPlayer's session reclaim both re-entered their own useCallback, which
the checker reads as a forward reference; both go through the ref idiom this file already uses.
BootGate's dep list was load-bearing for correctness — a re-run meant `location.reload()` — so
the "already adopted THIS identity" comparison now enforces it in code, with a test that fails
without it (the existing reload tests all install their spy too late to catch it).

One correction to the batch before this: keying ChatThread's and Messages' decrypt passes on
`q.data` instead of `q.dataUpdatedAt` was wrong. react-query's structural sharing makes it safe
in production, but the component must not melt down when something hands back a fresh object per
render — and Messages.keyboard.dom.test does exactly that, so the effect span the suite. Both are
back on the fetch as the trigger with the payload behind a ref, which is honest AND loop-proof;
the replaced comment had warned about precisely this.

CLAUDE.md gains four canonical-patterns rows for the above, and its "React 18" line is now true.
2026-09-21 01:44:21 +02:00
peter 4065ac7e6a refactor(frontend): the two players — one real ordering bug avoided, five suppressions gone (E27 S2)
The interesting part of this batch is a change that was WRONG and the test suite said so.

PlayerModal synced its local watch-status mirror and `currentVideoId` from an effect on
`active.id`. Moving that into render is right — as an effect there was a frame per queue advance
showing the new episode's chrome over the previous item's watch state, with `navigated` still
true so the "you have navigated away" affordance flashed. But the same effect also advanced
`currentIdRef`, and that line must NOT move: useYtProgress's per-item cleanup persists the
OUTGOING item's position and guards it on `currentIdRef.current === id`. React runs every
cleanup before any setup, so in an effect the ref still names the item being left. Advanced to
render time the flip happens first, the checkpoint is skipped, and stepping next→prev back to an
item resumes from the frozen launch queue's stale position. "resumes an item played earlier in
the SAME session" caught it immediately (748 instead of 830); the ref write stays in its own
effect with the ordering written down.

The rest:
- PlayerModal's keyboard handler is a useEffectEvent, so the "bound once, reads live state"
  contract is enforced instead of asserted. `focusModal` is a useCallback now — it reads a ref
  and nothing else, and three hooks take it as a parameter where a fresh identity would spread.
- PlexPlayer's session reclaim called `loadSession` from inside loadSession's own useCallback,
  which the checker reads as a forward reference. Routed through a ref, the file's existing
  answer for exactly this (detachSessionRef, cancelAutoSkipRef).
- PlexPlayer's item-load effect excluded `prefs` by hand so a track pick from the player's own
  menu would not double-load the session. The two language prefs go through a ref instead —
  "latest value, not a trigger", said in code rather than in a comment.
- PlexPlayer closes its up-next card during the render that changes `id`, not a commit later.
- The one PlexPlayer directive that stays is the live session counter, whose cleanup must bump
  the CURRENT value: the rule's advice to copy it into a variable is precisely wrong there, and
  the comment now says why rather than pointing at another comment.

Suppressions 75 -> 70; exhaustive-deps 20 -> 17.
2026-09-21 00:27:15 +02:00
peter ce08fc2aec refactor(frontend): a stable useHistorySubview, and PlexBrowse can stop disabling the check (E27 S2)
useHistorySubview returned a fresh object with a fresh `open` on every render. Correct, but it
made "re-run when the sub-view moves" unwriteable: PlexBrowse had three effects that wanted
exactly that and had to disable the dependency check to get it, and a fourth that listed `sub`
honestly and therefore re-ran on EVERY parent render, saved only by a ref guard.

Fixed at the source, which is where the house rule says a principle belongs: `open` and `back`
are useCallbacks and the returned object is memoised. `url` is a module constant at every call
site (PLEX_SUB_URL, MODE_URL) and the body otherwise touches only module functions, so there is
nothing left to churn on. All three PlexBrowse dep lists are now the honest ones, and the
comment that claimed `open` was "behaviourally stable" is replaced by it actually being stable.

PlexBrowse's remaining four are escape hatches, and two of them are now ONE:
- `infoScrollRef` is written from four places, and the compiler objects to a ref being modified
  outside the effect that reads it. That objection is a conservative false positive — a scroll
  memory needs exactly that arrangement — but four bare mutations were worth concentrating
  anyway. One `rememberInfoScroll` writer now holds the "stash on the way out, reset on a fresh
  page" rule, and answers the checker once instead of three times.
- The language-commit effect cannot move into render: it resets PlexProvider's filters, and a
  render-phase setState may only touch the component's own state. Said so at the site.

Suppressions 78 -> 75; exhaustive-deps 23 -> 20.
2026-09-21 00:20:05 +02:00
peter ea6d829c29 refactor(frontend): the context providers adjust state in render, not after it (E27 S2)
Eleven suppressions across the five providers. The recurring shape here was "an effect that
undoes something when you navigate away", and every one of them ran a commit LATE — the new
page painted once while the old module's state was still live.

Adjusted during render instead (React discards that pass rather than committing it, so there
is no in-between frame):
- SavedViewsProvider reset the sticky view and closed its modal from an effect on `source`, so
  the first Plex render carried the feed's sticky id — long enough for a stale view chip. It
  also followed the active view into stickyId from a second effect; that one converges by
  construction, since preferSticky returns the sticky match when there is one.
- NavigationProvider ended a live YouTube search from an effect on `page`. Keyed on the page
  CHANGE rather than derived from it, because "ended" has to be permanent — deriving would have
  quietly brought the search back on return to the feed.
- ChannelsProvider's manager focus is now simply DERIVED from the page: it means nothing off
  the Channels page, and focusChannel sets it and navigates in one handler.

Honest deps by freezing what is genuinely mount-time:
- FeedFiltersProvider persisted a share link's filters with an empty dep list; listing `filters`
  honestly would have re-persisted on every edit and spun. The mount value is frozen into its
  own state, so all three deps are stable and the effect says what it does.
- ChannelsProvider's URL-seed layout effect and NavigationProvider's initial history stamp get
  the same treatment. The three slice setters they call were already stable useCallbacks
  (useFilterState / useSortState / useAccountPersistedState all return one).
- SavedViewsProvider reads `cfg` through a ref: it re-memoizes on every filter edit, and the
  per-source seed guard already enforces once. That also made the set-state finding go away —
  with the render value gone, the checker can see the seed for what it is.

WizardProvider keeps its one escape hatch, now inline with the invariant: auto-opening is a
one-way decision taken from server data as it arrives, and deriving it would reopen a wizard
the user had just closed.

Suppressions 89 -> 78; exhaustive-deps 27 -> 23.
2026-09-21 00:12:44 +02:00
peter 6a7ff330f6 refactor(frontend): the shared hooks in lib/ stop lying about their deps (E27 S2)
Ten more suppressions, and React 19 turns out to supply the missing primitive.

useEffectEvent (stable since React 19.2, which S1 brought in) is the canonical answer to the
"bind the listener once, but read the latest values" pattern these hooks were disabling the
checker for. Three of playerModalHooks' four escape hatches become honest dep lists: the
wheel-volume handler, the native-controls handover, and the fullscreen-change sync. The
contract is no longer prose — the compiler enforces it.

The fourth stays, deliberately, and now says why at the site: the YouTube player's lifecycle
effect must run once per video (re-running destroys and rebuilds the iframe), and YT.Player
takes its callbacks as an object literal, which an effect event may not be passed into.

Fixed, not suppressed:
- useDismiss captured onClose when the popover opened, so a caller that re-rendered with a
  different handler while it stayed open kept getting the OLD one called. The disabled dep list
  asserted the opposite ("read fresh from this run's closure"). Both the handler and the ref
  list are read through refs now; three tests, the two that matter mutation-checked.
- usePref wrapped its seed in a useCallback whose dep list had to be excused. useState calls a
  lazy initializer exactly once, so the memo was saving nothing; inlined.
- useCardPager clamped an out-of-range page from an effect. Adjusted during render instead —
  React discards that pass rather than committing it, so no out-of-range page is ever painted.
- plexPlayerHooks' auto-skip countdown reads its marker from the ref that already existed, so
  the dep list names its real inputs. That also made the set-state-in-effect finding go away:
  the progress is genuinely timer-driven, and the checker can now see that.
- useUrlMirror and useChipOverflowFit only needed their stable refs listed.

Two test-harness suppressions (a render-phase probe ref, a module-level setState handle) moved
inline with the reason they exist: both exist to observe a hook from outside without changing
what is being measured.

Suppressions 99 -> 89; exhaustive-deps 35 -> 27.
2026-09-21 00:07:12 +02:00
peter 7e1aeeb9fe refactor(frontend): honest effect deps in the app shell and the ui layer (E27 S2)
Burns the first 14 of the 113 hook suppressions. Three verdicts, applied per site:

A useEffect that should not exist (6):
- Avatar remembered "the image failed" as a boolean an effect reset on src change. The
  component is recycled across items in the virtualized feed, so after one avatar 429s the
  next item's slot committed the fallback initial over a perfectly good image and only the
  post-paint effect cleared it. The failure is now keyed BY URL; derived state cannot be stale.
- Combobox cleared the typed query from an effect on `open`. Moved into setOpenAnd, the single
  funnel every close goes through. That also stops a spurious onQueryChange("") AT MOUNT, which
  told a controlled caller its own initial query had been cleared before the user touched it.
- CommandPalette reset query + picked row from an effect. Closed now means UNMOUNTED, not
  hidden, so a fresh open starts blank because mounting already means that -- and the ranking
  memo over every channel, playlist and saved view stops running while the palette is shut.
- Pager's page box held a copy of the page that an effect re-synced; it now derives it and
  keeps a draft only while being edited.
- App's ?link= / ?oauth= / ?admin= one-shots asserted "once per mount" with an empty dep list.
  The ref guard the codebase already uses in 22 places ENFORCES it, so the honest deps can
  include `t` -- which a language switch legitimately changes, and which would otherwise have
  re-fired the toast.

Honest deps that also fix something (5):
- App's module preload was keyed on the account id, so a permission change that alters the
  module set without moving the id never re-preloaded. `preload()` is idempotent, so depending
  on `me` costs nothing.
- App's access-request nudge now re-issues on a language switch; it used to stay in the old
  language until the pending count changed.
- App's diagnostics effect: applyDiagnostics already no-ops on an unchanged effective level,
  so picking four fields out by hand was guarding nothing and missed any field added later.
- DataTable's two filter effects: a ref holds the latest onFiltersChange so commitFilters can
  be stable, which is what lets the effects name it instead of switching the check off.

Genuine escape hatches (7) moved OUT of eslint-suppressions.json into inline directives that
name the invariant at the site -- the toast handover's render-phase global, the live region's
deliberate arming tick, the docked one-slot component, the popover engine's floating-ui list
state, dnd-kit's per-drag `moved` flag, BackToTop's DOM read and the palette's debounced
announcement. A reader of the code now sees why; a JSON file no one opens does not.

Tests: Avatar (3, new file) and Pager (3) pin the derived-state frames with a layout-phase
probe -- layout effects run before any passive effect, so they read the frame the browser would
paint. Combobox (2) pins the close-clears-query contract and the no-callback-at-mount rule.
All three sets mutation-checked against the effect-based originals.

Suppressions 113 -> 99; exhaustive-deps 45 -> 35.
2026-09-20 23:57:08 +02:00
peter 7d682c90d5 build(frontend): React 18 -> 19 with the React Compiler (E27 S1)
The compiler auto-memoises, which is the point: it infers what the app's
112 `useCallback` and 48 `useMemo` call sites were hand-written for and
applies the equivalent itself. It is on repo-wide with no opt-out list —
it bails on anything it cannot prove safe, and says so through the lint
rules. Verified in the built bundle rather than assumed: 369
`react.memo_cache_sentinel` markers across 35 of the 76 chunks.

ONE transform, shared by vite.config.ts and vitest.config.ts through
reactPlugin.ts. Not two `react()` calls: if the compiler ran only in the
production build, every vitest run would exercise code the user never
receives and a compiler-induced change would ship green — the same
gate-vs-image split 0.93.2 was released to fix. The plugin is imported
rather than named by the usual string, because a string is invisible to
knip and, worse, a typo in it is a silent no-op that ships an uncompiled
bundle, while a bad import is a tsc error.

eslint-plugin-react-hooks 5 -> 7 brings the whole compiler rule set: 283
findings, triaged rather than bulk-suppressed, because the two halves
needed different answers.

  react-hooks/refs (217) is OFF. 153 of those 217 are this app's popover
  architecture, not its history: 79 are @floating-ui/react's documented
  `pop.refs.setReference` / `getReferenceProps` callback-ref setters, and
  74 more are passing the popover handle down as a prop, which is what
  components/ui's popover engine is for. Suppressing them would not help
  the next popover anyone writes. The remaining ~64 are genuine
  render-phase `ref.current = x` writes and are real, but the rule cannot
  tell them apart here; they are E27 S2's work.

  The other 66 go into eslint-suppressions.json as a ratchet — new
  violations fail the gate, and `--prune-suppressions` makes a fixed one
  fail too until it is removed. 45 of them are `set-state-in-effect`,
  which IS E27 S2's subject, so that sprint now has a measured worklist.

react-i18next deliberately stays on 14. It never declared a React-19
peer — every version from 14 to 17 asks only for `react >= 16.8.0` — and
what the newer lines actually require is i18next 25/26 against our 23.
That is an i18n runtime migration wearing a React hat, and it buys this
sprint nothing.

Two React 19 type changes: `useRef<T>(null)` is now `RefObject<T | null>`
(the honest shape — the element is null until it mounts), so
PlexPlayerSettings' prop follows; and reactPlugin.ts joins
tsconfig.node.json's include.

The compiler skips memoising VirtualGrid and reports it: TanStack
Virtual's `useVirtualizer()` returns functions that cannot be memoised
without going stale. That is the right trade for a component whose cost
is its row window, and it is the one such site in the app, so it carries
a targeted disable with the reason — a second incompatible library
anywhere else must still stop the gate.

Nine lanes green, vitest 1558. Browser UAT on the dev server against the
still-running React 18 build as a control: boot, feed (virtualized grid,
popover menu, a filter change that moved the list 195,187 -> 64 items
with the URL mirrored, so no stale memoised UI), command palette, player
modal — no console errors on any of it.
2026-09-20 23:21:08 +02:00
peter cc9fc3b544 build: pin one Python version across the app, the images and the gate
0.93.2 shipped a week's worth of confusion about which Node this repo
runs on; the Python side had the identical shape and nobody had looked.
Prod (Dockerfile), the test image (Dockerfile.test) and CI (check.yml)
each spelled 3.12 as an independent literal, there was no
`.python-version`, no `requires-python`, and ruff had no target-version
so it judged our syntax against some other interpreter. The dev machine
had no 3.12 installed at all — only 3.13 and 3.14 — so every ad-hoc host
script ran two majors ahead of production.

`.python-version` is now the single source of truth, and the contract is
held by tests rather than by prose. It is split in two because no single
runner can see every link: the pytest suite runs inside the test image,
which mounts backend/ alone and cannot read the repo-root Dockerfile or
the workflow.

  test_python_engine_contract.py : running interpreter == pyrightconfig
                                   == ruff.toml
  pythonEngine.contract.test.ts  : .python-version == pyrightconfig ==
                                   ruff.toml == Dockerfile ==
                                   Dockerfile.test == check.yml

Together they close the loop, and the backend half carries the assertion
that pays — that the interpreter running the suite is the pinned one. A
green suite on another minor does not mean the shipped image is green;
it means they have drifted and nobody has noticed yet.
2026-09-20 23:02:11 +02:00
peter c9101e3450 ci: type-check the backend — the gate's ninth lane (E27 S3)
Three tsc projects held the frontend and NOTHING held the backend: ~36k
lines of `app/`, annotated in 113 of 128 files, that no lane had ever
read. Asymmetric by accident, not by decision.

basedpyright, in basic mode. Measured against mypy on this tree with the
dependencies actually installed — without them basedpyright errors on
every import and mypy's --ignore-missing-imports quietly turns them into
Any and UNDER-reports. Both land ~190-200 errors with the same
distribution, so the choice fell to what differs: basedpyright is ~2x
faster (4.4s vs 7.4s, and this runs on every `siftlode lint`), resolves
our `app.*` imports from its own config with no MYPYPATH, and has a
first-class `--writebaseline`. That last one is the ratchet itself;
mypy would have needed a third-party tool for it.

198 pre-existing errors are baselined, so the lane fails on NEW errors
only and the list burns down epic by epic. A green lane means no new
damage, not a clean codebase.

It runs in the pytest test image rather than on the host, and that is
forced rather than chosen: `requirements.lock` cannot be installed on
Windows at all — uvloop refuses to build there — so a host venv could
only ever be the lock minus a package, and a type checker without the
real third-party packages reports nonsense rather than nothing. The
image installs the pin from requirements-dev.txt, so the lane's version
IS the pin and there is no host copy to drift. The cost is that
`siftlode lint` now needs Docker, which it did not before.
2026-09-20 23:02:02 +02:00
peter 51daf3e388 feat(security): cap every request body before auth runs (SEC-20)
FastAPI reads a declared body IN FULL before the first dependency, so
`POST /auth/password-login` drained an arbitrary upload from an anonymous
caller and only then answered 401. 70 of the 72 body-declaring routes
behaved the same way: one unauthenticated client could spend the
process's memory and time at will.

A raw ASGI middleware, not `@app.middleware("http")` — BaseHTTPMiddleware
cannot wrap `receive`, so it could only trust `Content-Length`, which a
client may omit (chunked) or understate. This refuses an oversized
declaration outright and counts the bytes of everything else as they
arrive, the same rule `capped_body` already applies per-route. Registered
directly inside the access log: a refusal is still logged with its
request id, but the session, CORS, the setup gate and every auth
dependency never run for a body that is already too large.

Two routes opt out, under one stated rule: an exemption is ONLY for a
route whose own cap is larger than the global one. That is why
`/api/diagnostics/events` is absent — it caps itself at 512 KiB, below
the ceiling, so exempting it could only widen what it accepts. The list
names route templates and a test resolves each against the live route
table, so a rename fails the gate instead of silently dropping an
exemption in prod.

The 413 is byte-identical to `capped_body`'s and the test compares the
two against each other rather than against a literal — two shapes for one
fact is how the `cancelled`/`canceled` bug happened.
2026-09-20 22:51:32 +02:00
peter 09d351a0e7 docs(ci): the two gate jobs queue, they do not run in parallel
Declared side by side, so they read as parallel; the lab runs a single
act_runner with one slot, so the backend job starts at the exact second the
frontend one ends. Measured across runs #86-#91: ~4 + ~6 = ~10 minutes from the
push, not ~6.

Written down because believing the comment cost something real — the ops
release read-back budgeted the max instead of the sum and timed out on its
first live run, reporting a still-running gate as a failing one.
2026-09-20 22:05:56 +02:00
peter b5078003ce release: 0.93.2 (bug/ci-node-version-split)
gate / frontend (push) Successful in 3m51s
gate / backend (push) Successful in 6m7s
2026-09-20 21:49:56 +02:00
peter f2c32cfb8e chore(release): 0.93.2 — a Plex amount reads the same in every browser 2026-09-20 21:49:36 +02:00
peter 1453fa4a33 ci: run the gate on branch pushes only, not on release tags
A release pushes main and then the v<ver> tag, and a bare `on: [push]` treated
the tag as a second push of the identical commit — every release ran the whole
gate twice to the same verdict (runs #81-#90 are five such pairs).

It has to be `branches`, not the `tags-ignore` this obviously wants to be:
Gitea's matchPushEvent requires every named filter to match and each one bails
on the wrong ref kind, so `tags-ignore` rejects every BRANCH push and silently
disables the gate. Measured in go-gitea v1.27.3, where it differs from GitHub.

A trigger filter that stops firing is invisible, so the ops `release` now reads
the verdict back for the sha it pushed and says so when there is no run for it.
2026-09-20 19:47:19 +02:00
peter efe106dfda build: pin one Node major across the dev, CI and build machines
The server gate ran node:22 while the dev gate ran node:24, and the two disagree about
Intl.NumberFormat's compact currency padding and about which realm's Blob a Response hands
back — so the frontend job had been red for 73 of its last 90 runs while the local gate was
green, and nothing said so.

Align everything on 24, with `.nvmrc` as the single source of truth: the CI container, the
prod build stage, `engines`, and `@types/node` (which was two majors ahead of the runtime, so
tsc was green over APIs the shipped engine does not have). The backend job's node only runs
the checkout action, so it tracks the pin as hygiene — node 20 is EOL — but deliberately
stays outside the contract.

The pin is held by a contract test that reads every one of those places AND the engine
running it, so the next drift is a red lane on its first run instead of a CI log nobody reads.

Verified on node:24-bookworm (the CI lane: 228 files / 1552 tests, typecheck, lint,
format:check, knip all exit 0) and on node:24-alpine (the prod build stage, v24.21.0, SPA
built). Each assertion of the new test was mutation-checked red.
2026-09-20 18:06:24 +02:00
peter ec63056bd7 test(api): assert the export body's brand, not which realm built its Blob
`expect(x).toBeInstanceOf(Blob)` asks which realm constructed the body, not
whether there is one. Under jsdom the ambient `Blob` is jsdom's class, while
`Response.blob()` hands back node's own on Node 22 and jsdom's on Node 24 — so
both assertions passed locally and failed on the server gate, for a reason that
says nothing about the code under test. `.text()` is the mirror image: jsdom's
Blob does not implement it, so reading the bytes back is green on 22 and red on
24.

Both now assert the brand (`[object Blob]`, agreed by both realms) plus size and
type. Mutation-checked in node:22-bookworm: dropping `{ raw: true }` from
`debugExport` still turns the test red, so the repaired assertion still catches
the defect it was written for.

core.test.ts also stops building its Response from a jsdom `Blob`: on Node 22
undici does not recognise it as a body and stringifies it, so the fixture's body
was the 13 bytes of "[object Blob]" rather than the zip header. A string body is
what the sibling test already used.
2026-09-20 16:33:26 +02:00
peter 09becb3d00 fix(plex): pin the compact currency format instead of inheriting the engine's
`{ notation: "compact", maximumFractionDigits: 1 }` leaves the MINIMUM to the
engine, and engines disagree: the same options render "$160.0M" on Node 22 and
"$160M" on Node 24 (measured 2026-09-20 in node:22-bookworm, the image the
server gate runs, against the local Node 24.17). Browsers split along the same
line, so a viewer on an older engine saw the padded form. `minimumFractionDigits:
0` makes both render "$160M"; the plain `compact` helper (vote counts) was
measured too and already agrees on both.

This is why the server gate's frontend job was red on PlexInfo since 09-17 while
the local gate was green: the two run different Node versions.
2026-09-20 16:33:26 +02:00
peter 8d2ec5e867 release: 0.93.1 (bug/preview-crashes-the-tracker)
gate / frontend (push) Failing after 4m10s
gate / backend (push) Successful in 5m50s
2026-09-20 07:55:37 +02:00
peter 0e9839e73d chore(release): 0.93.1 — the report preview works again 2026-09-20 07:51:16 +02:00
peter 05ab963951 fix(issues): the preview must not ask the tracker for a repository context
The composer's preview did not fail a call — it killed the Gitea process, and
with it the git server and the container registry. `POST /api/v1/markdown` with
`{"Mode": "gfm", "Context": "<repo>"}` makes Gitea resolve every hex-shaped word
as a commit link, and on the API's render path the commit checker it asks is nil:

  panic: runtime error: invalid memory address or nil pointer dereference
  renderhelper.(*commitChecker).IsCommitIDExisting(0x0)
   <- markup.hashCurrentPatternProcessor (html_commit.go:217)

Every report carries hex — a tab id, a request id — so this was not an edge case
but the only case: the preview answered "the tracker isn't answering" every time,
because the tracker had just died.

Measured on throwaway instances, not argued. Same body, same call:
  gfm + Context + "5645b1c0ae01"  -> 1.26.4 CRASH, 1.27.1 CRASH, 1.27.3 200
  gfm + Context, no hex           -> 200
  gfm, no Context                 -> 200, and the HTML keeps its tables, task
                                     lists and chroma-highlighted code blocks
Upstream fixed it in 1.27.2 ("fix: avoid markup render panic", #38698), but a
self-hoster runs whatever version they run, so the payload is what gets fixed.

Sending was never affected: the created issue's own page renders the same body
fine (measured) — a real repo context has a real commit checker. The only loss is
that a hash or a `#123` is not yet a link in the preview; the issue page links it.

The test pins the payload dict, because the defect is in what we send.
2026-09-20 07:51:08 +02:00
peter 8142484347 release: 0.93.0 (feature/e16-debug-issue)
gate / frontend (push) Failing after 4m11s
gate / backend (push) Successful in 5m58s
2026-09-20 05:54:26 +02:00
peter 0d91b82f86 chore(release): 0.93.0 — report a fault from inside the app
E16: a recorded debug session becomes a drafted issue with its own logs attached, sent to the private Gitea tracker or downloaded as a zip. Includes the four epic-end review rounds and the Plex player's focus trap.
2026-09-20 05:54:03 +02:00
peter 90fc86d437 fix(diagnostics): deep-review round 4 — a claim only its owner may write, and a player that keeps the Tab
Round 4 was targeted at the surfaces rounds 2 and 3 changed. 29 findings raised,
26 survived deduplication, 12 survived three adversarial readings each.

`set_delivery` renewed the claim but never checked it was still ours, while its
sibling `release_send` always did. `expire_on_commit` is off, so the token it
re-stamped came from the caller's own memory, not the row: a send that outlived
its 600 s lease came back, overwrote the take-over send's committed links with a
full-column write, and put its own dead token back. It now takes the token, reads
the row's claim with a targeted scalar (not a refresh, which would discard the
`issue_url` assigned one line earlier) and raises `SendClaimLost` rather than
writing. An issue created by a send that had already lost its claim is logged with
its number — that record is the only one that will exist.

The tracker's settings are read above the pre-call checkpoint instead of inside
`_client`. Reading them there was a SELECT one line after the commit that was
meant to end the transaction, and held it — idle, snapshot pinned — across the
repository check and the create. The comment and the module docstring said
otherwise; measuring it in pg_stat_activity is what settled it. Two more comments
were saying things the code had stopped doing: the `**state` spread does not carry
the claim (`set_delivery` strips and re-stamps it), and react-query does not
debounce the preview — the editor being unmounted is what paces it.

The composer stopped stranding the focus on `<body>`: a send that creates the
issue and then fails on an attachment unmounts the button that was just pressed,
and only the success path handed the focus on. Export no longer saves into a
refusal it does not need — the zip is the way out of a stuck send, and its
pre-save was the one step that could not survive one. The delivery banner
announces through a role instead of an `aria-live` created with its own message.

`historyState` returns a union, so `chipTone` matching on the same keys is checked
by tsc rather than by hope.

PlexPlayer was the only full-screen overlay in the tree with no focus containment
at all — Tab walked out of an opaque player into a page nobody can see. It now
carries the shared trap. Closing that hole exposed the one underneath it:
`useDismiss` registered as a layer that traps the Tab while installing no trap,
so the player stood its own down for its own menus. Measured in a real browser:
30 Tabs with the settings panel open, 9 escapes into the nav rail, and 0 after.
Escape ordering is unchanged — the panel first, then the player.

Three tests that did not hold their line: the advisory lock's call site (deleting
it left the suite green), the trail assertion whose `all()` was vacuous over an
empty filter, and a `layerStack` assertion the stand-in dialog already forced.
All eleven executable fixes mutation-checked red for the right reason; the two
comment corrections have nothing to pin.
2026-09-20 04:30:42 +02:00
peter 2fe58073e4 fix(diagnostics): deep-review round 3 — a keyboard owner that is not a Tab trap
Round 2 made the step prompt a blocking layer so that `j` and the `g` chords
would not reach the page behind it. The dialog it sits in gave up its Tab trap
because of that, while the prompt's own replacement trap covered only its own
two controls — so Tab from anywhere else in the dialog (the bar's Screenshot or
Stop, the dialog's own buttons) walked out behind the scrim. `layerStack` now
separates the two questions it had folded into one flag: a layer can own the
keyboard and still leave the Tab trap to the dialog below it. The mirror case,
a dialog opening over the prompt, falls out of the same split.

A send's claim is now a lease with an owner. Every progress report re-stamps
it, so a send slower than the lease cannot lose it while it is still running,
and only the send that took a claim can give it back. The issue_create audit
row is staged inside the send and rides the same checkpoint as `issue_url`, so
the trail cannot be missing for an issue that exists. The attachment-name
prefix has one definition, shared by the producer and both readers.

A running send is reported to the UI. A reload or a second tab used to render
it exactly like a send that STOPPED, and offered a retry the server was certain
to refuse; the composer now says the report is being sent, holds the draft, and
puts both the retry and Create issue out. The past-sessions list says the same
for the same reason — it is the list a user opens to check on a send, and both
"draft" and "half sent" describe one that died. `/preview` and the retry path
got a pace of their own, and the per-user byte cap took the user-scoped lock
that makes it a cap instead of a guess.

Verified in a real browser on localdev: Tab cycles the whole dialog with the
prompt open, the prompt still swallows the page's keys, and the issue link is
focusable where the composer puts it. All 24 regression tests mutation-checked.
2026-09-20 02:43:56 +02:00
peter bc03e82687 fix(issues): a send keeps its claim through every step, not just up to the issue
Creating the issue rebuilt the delivery state from scratch and dropped the
claim with it, so for a new issue the claim covered only the creation — a
draft, a screenshot or a second send could change the report under the uploads
and the body edit that follow, which is exactly what the claim is for. The
state is now built on what was read, and a test walks the send's checkpoints
to see the claim still held at each one.

Found by mutation-checking the round-2 tests: removing the claim's release
left them green, because for a new issue there was nothing left to release.
2026-09-20 01:03:23 +02:00
peter 3f983e6d76 fix(diagnostics): deep-review round 2 — a send that claims, not one that blocks
The send no longer holds the session's row lock while it talks to the tracker:
it claims the session, gives the lock back, and checkpoints each step. A Stop
is never held behind a send, while a draft, a screenshot or a second send are
refused at once instead of queueing. The claim ages out, so a request that
died mid-send does not strand the session.

What the composer says is what happened: an attachment that went is no longer
named as one that did not (the missing body edit is its own line), and the
issue's screenshot count comes from the server when the list cannot be read,
so a failed read no longer blocks saving the draft. The zip is fetched as a
request, so its pace is a message and not a saved error page. Screenshots of a
finished report can be removed again; only one still being sent is held.

A recording's log elevation can no longer outlive it. The step prompt keeps
the keyboard (the page's keys stay quiet) and traps Tab itself. An image
pasted into a dialog opened over the composer stays that dialog's.
2026-09-20 00:51:12 +02:00
peter 134ec9a68e fix(diagnostics): deep-review round 1 — a send that cannot race, lose or repeat itself
The send, the draft and the screenshots take the session's row lock, so an
autosave or a paste during a send waits for it and then sees it, and two sends
cannot open two issues. A save keeps the delivery's progress; a failed edit
that shows the screenshots is finished by the retry; an answer that is not the
tracker's API (a redirect, a proxy page) is refused, not a crash that loses the
issue's record. Only an attempt that opened an issue counts toward the pace.

Screenshots: deleting one after the send is refused, a user's storage is
capped across sessions, new sessions and zip exports are paced, images are
stored uncompressed in the zip. One liveness rule, one session view, one image
type table.

Logging: a session's elevation is tracked apart from the user's own, so the
browser's own requests (no header) no longer lower the floor a session needs;
Stop ends it at once.

Composer: an image pasted anywhere is taken; what is still unsaved is saved on
close, and "saved" is shown only when it is; the draft is held while it is
sent; a failed screenshot read says so instead of reading as none; focus moves
to what stays; the preview follows the text size. Pill: the step prompt and a
running Stop survive the pill moving between hosts, the prompt keeps the
dialog's Tab trap, the toast region lets clicks through, other tabs hear of a
Start or Stop, the last seconds are sent before the cap, a capture with no
frame is an error.
2026-09-20 00:09:52 +02:00
peter 9cf177b198 feat(diagnostics): send a bug report to a private Gitea issue tracker (E16 S4)
- admin config group "Bug reports": Gitea address, repository, token (secret),
  and "Check the connection" — the repository must answer, take issues and
  be PRIVATE (a report carries its reporter's log); checked on every send too
- POST /api/debug-sessions/{id}/issue sends the SAVED draft: create the issue,
  upload each file on its own request (the measured API), then show the
  screenshots inline; progress is kept in the draft, so a send that stops
  half-way keeps its issue and the retry sends only what is missing; 5 new
  issues an hour per user; audited as issue_create
- the composer: Create issue (only where a tracker is set up, naming the
  repository), a partial-send banner with retry, and a Preview — the tracker's
  own rendering in a sandboxed frame, or the zip's markdown without one
- the recorder's refusals answer in the localized family (debug_session_*,
  screenshot_*, issue(s)_*), not bare codes the UI could only call generic
- a tracker refusal logs the status only: its message can quote the title back
2026-09-19 22:48:20 +02:00
peter a5fae489e0 test: a request that raises is rolled back, as production rolls it back
The shared test session kept whatever a failing handler flushed and never
committed, so a later read in the test saw a write production loses when
get_db closes the session — and a handler whose failure path must commit
(E16: the tracker's issue link) looked right without the commit.
2026-09-19 22:48:20 +02:00
peter 8d0083d4e4 feat(diagnostics): the issue composer — draft, review, screenshots, zip export (E16 S3)
After Stop (or at the cap, or from Settings → Diagnosztika's past sessions)
one dialog drafts the issue: title, description, the steps drafted from the
session's markers and page changes (a marker without a label is named by its
time), the environment (version, browser, system, window, language). Beside
it, what goes with it: the browser and server logs — each reviewable line by
line, an unticked line is not sent — and the screenshots, removable, with
paste / drop / pick to add more.

- it saves as it goes (debug_sessions.draft, migration 0090): the rendered
  markdown in the user's language, the excluded rows, and its own fields to
  reopen as it was; a sent session is read-only
- the zip export (issue.md, client.log, server.log, screenshots) is built
  from the saved draft by app/issues/bundle.py — the one definition the Gitea
  delivery (S4) will share; .log, not .jsonl, which Gitea refuses
- the log line's level tags and clock move to components/logLineParts so the
  review list draws a level the way both log viewers do
2026-09-19 22:26:49 +02:00
peter 9f177557d2 docs(ui): the toast dock in the primitives catalog 2026-09-19 22:08:21 +02:00
peter 46fce4a767 feat(diagnostics): the debug recorder — start, the pill, steps, screenshots, stop (E16 S2)
- the start dialog is the consent: level (Részletes/Teljes), what is recorded,
  retention and the cap (now served as diagnostics.session_minutes)
- the pill docks under the toasts: elapsed time, Step (an optional label,
  its own Escape layer), Screenshot (tab capture where the browser has it),
  Stop; compact on a phone
- while recording, lib/log tags every event and every request path (req,
  beacon, uploadReq via one identityHeaders) names the session; applyDiagnostics
  raises the level for the session's run and never lowers a raised one, and
  lapses at whichever ends first
- Stop sends what is waiting before the server closes the session; the cap
  says so and opens the composer
- Settings → Diagnosztika and the command palette start/step/stop it
- the evidence leaves the log upload's own request lines out
- the step shortcut is deliberately palette-only: the keymap's D1 rule bars a
  new modifier chord, and a bare letter collides with the players'
2026-09-19 22:08:10 +02:00
peter 2d9595b0e5 feat(ui): a control can dock under the toasts and ride every toast host
The toast stack already has a host in the page and in every overlay (dialogs,
both players, the fullscreen player), the topmost one shows it, and inside an
overlay it sits above the player's control band. lib/toastDock lets one control
use exactly that placement; it renders outside the live region, so a ticking
control is not read out every second.
2026-09-19 22:08:10 +02:00
peter a7d7456279 feat(diagnostics): debug sessions — start, stop, the tag on every line, screenshots (E16 S1)
A session is one recording, capped at 60 minutes by the server. While it is
open, a request naming the caller's own session in X-Debug-Session is logged
at the session's level with every line tagged; a header naming anything else
is ignored. The browser's batches count a session as consent to its level and
keep the tag only on its own events. Screenshots are judged by their bytes,
capped in size and count, and served back to their owner only.

- debug_sessions / debug_screenshots tables, one open session per user
  (partial unique index), log_events.debug_session_id indexed
- /api/debug-sessions: start (joins a running one), stop, list, get, events,
  screenshots; /api/me reports the live session
- the hourly log job closes expired sessions and drops them with the logs
- logsink.serialize and routes._common.capped_body shared by their readers
2026-09-19 21:38:53 +02:00
peter 44a792c34f test(i18n): a 404 target the scan cannot read fails the backend-id contract
what=SOME_CONSTANT reached no locale check at all; only _common.py's own
pass-through may name the target by variable.
2026-09-19 21:38:53 +02:00
186 changed files with 13663 additions and 946 deletions
+54 -6
View File
@@ -1,22 +1,50 @@
# Server-side mirror of the local `siftlode check` gate (E1 S3 / DX-1), so `main` is always green
# independent of any one machine. Two parallel jobs match the two local lanes:
# independent of any one machine. Two jobs match the two local lanes:
# frontend — tsc (3 projects) + eslint + prettier --check + knip + vitest
# backend — ruff + the full pytest suite (pure-logic + the DB-backed lane) against a real Postgres
#
# They are declared side by side but they DO NOT run in parallel: the lab has a single act_runner
# with one slot, so they queue. Measured across runs #86-#91 — the backend job starts at the exact
# second the frontend one ends, every time, and the pair costs ~4 + ~6 = **~10 minutes from the
# push**. Anything that waits on this gate has to budget for the sum, not the max; the ops
# `release` read-back timed out on its first real run for believing otherwise.
#
# Runner: the lab act_runner (LXC 115) mounts the Docker socket, so service containers work. The backend
# runs pytest NATIVELY against a `postgres` service (not the local docker-compose.test.yml, whose
# bind-mount is unreliable under a mounted-socket runner); fidelity is kept by installing the SAME pinned
# requirements, Postgres 16, and ffmpeg, and by setting SIFTLODE_TEST_REQUIRE_DB=1 so a DB/ffmpeg that
# failed to come up is a hard error, never a silent skip. node 22 matches the prod build image
# (Dockerfile `node:22-alpine`).
# failed to come up is a hard error, never a silent skip.
#
# The frontend container's node major is NOT free: it must equal the prod build image's and the dev
# machine's, because vitest/tsc read the engine's own Intl and web-API implementations and those split
# between majors. `.nvmrc` is the single source of truth and
# `frontend/src/lib/nodeEngine.contract.test.ts` fails this gate if any of them drift apart.
name: gate
on: [push, pull_request]
# Branch pushes only. A release pushes `main` and then the `v<ver>` tag, and a bare `on: [push]`
# treated the tag as a second push of the IDENTICAL commit — every release ran this whole gate
# twice, to the same verdict, for nothing (runs #81-#90 are five such pairs). `branches: ['**']`
# keeps every branch covered and drops tag refs.
#
# It has to be `branches`, NOT the `tags-ignore` this obviously wants to be. Gitea's matchPushEvent
# requires EVERY named filter to match, and each one bails on the wrong ref kind: `tags-ignore`
# starts with `if !refName.IsTag() { break }`, so a filter meant to skip tags rejects every BRANCH
# push and the gate stops running entirely. `branches` bails the same way on a tag, which is the
# direction we want. (Measured in go-gitea v1.27.3 modules/actions/workflows.go, not assumed — it
# differs from GitHub, where `tags-ignore` alone leaves branches unfiltered.)
#
# If that filter were ever wrong the gate would not run at all — which is why `siftlode release`
# now ends by reading the verdict back for the sha it pushed, and says so out loud when there is
# no run for it. A gate that silently stops running is the failure this repo already lived through.
on:
push:
branches: ["**"]
pull_request:
jobs:
frontend:
runs-on: ubuntu-latest
container: node:22-bookworm
container: node:24-bookworm
defaults:
run:
working-directory: frontend
@@ -34,7 +62,11 @@ jobs:
# A combined python+node image: python 3.12 (matches the prod Dockerfile) for the tests, and node
# so the JS actions (checkout) run. actions/setup-python is NOT usable here — the runner can't reach
# GitHub's python-versions download, so it must be baked into the image.
container: nikolaik/python-nodejs:python3.12-nodejs20
#
# This node runs the checkout action, never our code, so it is deliberately OUTSIDE the engine
# contract the frontend job is held to — it tracks `.nvmrc` as hygiene (node 20 went EOL in April
# 2026), but a missing upstream tag here must never be able to block the gate.
container: nikolaik/python-nodejs:python3.12-nodejs24
services:
postgres:
image: postgres:16-alpine
@@ -61,6 +93,22 @@ jobs:
run: python -m ruff check --config backend/ruff.toml backend scripts
- name: ruff format --check
run: python -m ruff format --check --config backend/ruff.toml backend scripts
# The ninth lane (E27 S3). Run from backend/ so it resolves exactly as the local lane does —
# pyrightconfig.json's include/extraPaths are relative to the config, but the baseline is
# found relative to the working directory. New errors fail; the ~200 pre-existing ones are in
# .basedpyright/baseline.json and burn down epic by epic. Matching is per (file, rule, column
# span) plus a count, so a new error of the SAME shape in a file where a sibling was just
# fixed is absorbed — burn the hot files down first.
#
# The second step is not redundant. basedpyright TIGHTENS the baseline in place when the count
# drops, with no --writebaseline, and CI throws its workspace away — so without this a
# burn-down is green here and red on the developer's `siftlode check`, which has the same
# guard. Two gates over one tree must not answer differently.
- name: basedpyright
run: python -m basedpyright --project pyrightconfig.json
working-directory: backend
- name: basedpyright baseline is committed
run: git diff --exit-code -- backend/.basedpyright/baseline.json
- name: pytest
run: python -m pytest
working-directory: backend
+4
View File
@@ -37,3 +37,7 @@ backups/
Thumbs.db
.idea/
.vscode/
# TypeScript incremental build artifacts — machine-local, absolute-path-bearing, and no gate lane
# reads or regenerates them, so a committed one is stale from the commit that adds it.
*.tsbuildinfo
+1
View File
@@ -0,0 +1 @@
24
+1
View File
@@ -0,0 +1 @@
3.12
+80
View File
@@ -7,6 +7,86 @@ edit the release notes and re-run the generator.
Upgrading, pinning a version and rolling back: [docs/self-hosting.md](docs/self-hosting.md#upgrading).
## 0.95.0 — 2026-09-23
A file that is too big to upload says so before it is sent, instead of failing as if the server were down - and the server no longer writes half a gigabyte to disk for someone who never signed in.
### Fixes
- Picking a file over the size limit used to end in "the server couldn't be reached" - after the whole file had been sent. The dialog now checks the size the moment you choose the file, tells you what the limit is, and does not start. If a refusal does come from the server, it reads as a sentence about the size rather than as a network failure.
- A long name on a download no longer breaks it. Anything typed past the length of the field used to fail the whole operation with a server error - on an upload, after the entire file had already transferred. Names are simply shortened now, on every screen where one can be set.
- While the file you picked is too large, the drop zone still lights up when you drag a replacement onto it. It used to stay red, which read as though drag-and-drop had stopped working - in exactly the moment you were trying to fix the problem.
### Internal
- An upload is now read only after the server knows who is sending it. Until now anyone at all could make the server write up to 500 MB to disk and only then be told they were not allowed to upload, because the file arrived before the password was ever checked. It is read a piece at a time, straight to disk, and stops the moment it passes the limit.
- An upload no longer holds a database connection open while the file is transferring. One slow upload used to occupy a connection for its entire duration, and enough of them at once would have starved everything else on the server.
- The parser for the old upload format is gone from the app entirely, along with the size allowance it required.
- Three adversarial review rounds closed the work: 13 findings, then 28, then 8. The last round found almost nothing in the app itself - its findings were in the tests the round before had written, which is why it was the last. Four of its claims were disproved rather than fixed, one of them a proposed change that would have broken emoji in names. Every regression test was proven to fail with its own fix taken out.
## 0.94.0 — 2026-09-22
No new features, on purpose: a release spent entirely on the machinery underneath, so that whole classes of fault stop being possible rather than being fixed one at a time.
### Fixes
- Typing an impossible number into a schedule - 0 minutes, 5000 minutes - used to close the editor and quietly keep the old value, as if it had been saved. The editor now says what the allowed range is, refuses to save until the number is inside it, and hands the keyboard back to the button you opened it from when you leave with Escape.
### Internal
- Every request is now measured against a size limit before anything else reads it - before a password is checked, before an address is even resolved. Until now a stranger could make the app swallow a 32 MB upload and only then be told the password was wrong. Each address carries its own ceiling, including the ones that legitimately receive a film.
- The server's Python is type-checked by the release gate for the first time: about 36 000 lines carried type annotations that nothing had ever verified. The mistakes already there are written down so they can be worked off release by release, and a new one now fails the build.
- The app moved to React 19 and its compiler. The compiler now does, automatically and correctly, the bookkeeping that was written by hand in about 150 places - and hand-written bookkeeping is where most of this year's stale-value and stale-click bugs came from.
- The 45 places where the old code told that same checker to stop looking at a piece of work are gone. Each was either a real bug, work that should never have been in that part of the code, or a genuine exception that now states the rule it depends on in code rather than in a comment.
- The engine versions the app is built and shipped with are now pinned and checked on both sides, the way the JavaScript side was fixed in the previous release.
- Four adversarial review rounds closed the work: 22 findings, then 13, then 6, then 2 - and the last two were in the review's own tests, not in the app. Everything found was fixed here, and every regression test was proven to fail with its own fix taken out.
## 0.93.2 — 2026-09-20
An amount on a Plex film reads the same in every browser.
### Fixes
- A film's budget and revenue could read "$160.0M" in one browser and "$160M" in another, from the same number: asked only for a limit of one decimal, a browser is free to pad up to it or not. The amount is now written the same way everywhere.
### Internal
- The machine that builds a release, the one that checks it and the one it is written on had drifted onto three different versions of the JavaScript engine — the reason those checks quietly disagreed for a month. They are held to one version now, and a check fails the moment any of them moves away from the others.
- Those checks no longer run twice over the same code on every release, and their verdict is read back as part of releasing instead of waiting on a page nobody opens.
## 0.93.1 — 2026-09-20
The report composer's preview works again.
### Fixes
- Looking at a report before sending it said "the issue tracker isn't answering" — every time, for everyone. The preview asked the tracker to resolve links against the repository, and a report always carries something that looks like a commit hash: an id of the tab, of a request. On a tracker older than Gitea 1.27.2 that request does not fail, it stops the tracker — so the preview now asks for the rendering without the repository, which shows the same tables, lists and code, and can no longer take a tracker down. Sending a report was never affected.
## 0.93.0 — 2026-09-20
A way to report a fault from inside the app: record what you did, mark the steps, attach screenshots — and the app writes the report for you, with the logs of exactly that session attached.
### Features
- Start a recording (the command palette, or Settings → Diagnostics) and a pill follows you: a timer, “Step” to mark what you just did, a screenshot button, and Stop. Everything you and the server do while it runs is collected under that one session, and nothing else is.
- Stop opens the composer: a drafted title and description, the steps you marked interleaved with the pages you visited, a table of your browser and the app's version, and the attachments — each one reviewable, each one removable.
- The log lines that would be sent can be read before they go, line by line, and any of them left out.
- Screenshots come from a paste, a drop, a file, or capturing the tab — up to eight, and any of them can be dropped again before sending.
- Send opens an issue on the project's tracker with the report and its logs attached, and the preview shows it exactly as the tracker will render it. Five reports an hour.
- Without a tracker configured — or whenever you would rather keep it — the same report downloads as a zip instead.
- An administrator sets the tracker up in Settings → System (address, repository, token) and can test the connection there. The server refuses to send to a public repository: a report carries its reporter's log.
### Fixes
- The Plex player keeps the keyboard inside itself: Tab no longer walks out of a playing film into the page behind it, not even while its own settings panel is open.
- A report cannot be sent twice at once, from any tab: while one send runs the others say so, and a send that fails half-way retries only what is missing — never a second issue.
- A recording ends by itself after an hour, and both Stop and that hour return your diagnostics level to whatever it was before you started.
### Internal
- A session's logs and screenshots are purged with the session after 7 days, like every other diagnostic line, and another account's session id is ignored rather than refused.
- Four adversarial review rounds at the epic's end: 80 findings fixed across them, every regression test proven to fail with its own fix removed, and the keyboard and focus work measured in a real browser rather than argued.
## 0.92.0 — 2026-09-19
A debt burn: every item the roadmap had deferred is done or dropped — Plex playback per viewer, refusals said in your language, controls that keep the keyboard focus, and a screen reader that hears Hungarian.
+35 -29
View File
@@ -7,17 +7,18 @@ worse than no rule — if you change how the code works, change this file in the
Siftlode is a self-hosted YouTube reader (+ a local Plex browser/player and a download center). It is a
**modular monolith**: one Postgres, one FastAPI process (uvicorn + APScheduler), one worker process
(yt-dlp/ffmpeg), one PO-token sidecar. React 18 + react-query frontend with a router-free navigation
provider.
(yt-dlp/ffmpeg), one PO-token sidecar. React 19 (with the React Compiler) + react-query frontend, with a
router-free navigation provider.
## The gate — never merge or ship red
The local gate is **ruff (`check` + `format --check`) + typecheck (3 tsc projects) + eslint + prettier
The local gate is **ruff (`check` + `format --check`) + basedpyright (backend) + typecheck (3 tsc projects) + eslint + prettier
--check + knip + vitest + pytest**. It runs on every `publish`, which **refuses to build on failure — there is deliberately no
`--skip` flag.** Run it and get it green **before** any merge, release, or handing work to review.
- Full gate (agents with the private ops dispatcher): `siftlode check`.
- Directly, without the dispatcher: `cd frontend && npm run typecheck && npm run lint && npm run knip && npm run test`, plus backend `ruff check backend` + `ruff format --check backend` and `pytest` (the DB-lane needs Postgres).
- **The backend type lane runs in the test image, not on the host** — `requirements.lock` cannot be installed on Windows (uvloop), and a type checker without the real packages reports nonsense. It fails on NEW errors only; the pre-existing ones sit in `backend/.basedpyright/baseline.json` and burn down epic by epic. Fixed some? Regenerate with `--writebaseline`.
- New DB-backed test? It must actually run, not silently skip — assume `SIFTLODE_TEST_REQUIRE_DB=1`.
## Workflow
@@ -56,9 +57,9 @@ how it gets built. Concretely:
## Architecture shape (don't fight it)
`routes/*` (HTTP only: validation, auth-dependency, serialization) → domain packages `sync/`, `plex/`,
`downloads/`, `youtube/` (shared by routes + scheduler + worker) → flat core (`auth`, `quota`,
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `realtime`, `ratelimit`, `progress`,
`security`, `logs`, `logsink`, `diagnostics`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
`downloads/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `quota`,
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `realtime`, `ratelimit`, `bodylimit`, `progress`,
`security`, `logs`, `logsink`, `diagnostics`, `debugsession`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
Frontend: `main.tsx` provider tree → `App.tsx` shell → `moduleRegistry` route-chunks → module
components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
its menus, `Modal`/`Overlay`, `Button`/`form`, `DataTable`/`VirtualGrid`, `QueryState`, …). That
@@ -74,29 +75,34 @@ CQRS, a repository/DI framework, a router library, a generic CRUD-router factory
## Canonical patterns — when you have a choice, pick THIS
| Area | Use | Not |
| ----------------------- | ------------------------------------------------------------ | ------------------------------------------------------ |
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
| Request body | a Pydantic model | `payload: dict` |
| Ownership check | `owned_or_404` / `get_or_404` | ad-hoc `_own_playlist`/`_owned`/`_own_job` |
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`), test-held | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok` |
| Transaction commit | the handler commits; helpers `flush` (test-held list) | helpers that commit (only `quota` owns its session) |
| Admin knob storage | `sysconfig` (values) | env / `app_state` / `scheduler_settings` scatter |
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
| FE per-account state | `usePref` | raw `readAccount`/`writeAccount` or bespoke hooks |
| FE action button | `ui/Button` — pick a `SIZES` rung | a hand-rolled `<button className="px-3 py-2 …">` |
| FE keyboard shortcut | a row in `lib/keymap.ts` + `dispatch()` | a private `keydown` listener with its own key literals |
| FE list/grid keys | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open` | a private arrow-key / roving-tabindex handler |
| FE view push (history) | `rememberNavPosition()` right before `history.pushState` | a push that forgets: Back strands focus on `<body>` |
| FE control | `SegmentedToggle`; a long or user's list: `FilterSelect` | a native `<select>` (lint-enforced) |
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)` | a bare `catch {}` or `notify()` with no error attached |
| FE network call | `req`/`uploadReq`/`beacon`, or `lib/http.ts` `send` (public pages) | a bare `fetch`/`XMLHttpRequest`/`WebSocket` (lint-enforced) |
| FE diagnostics | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts` | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded |
| Area | Use | Not |
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
| Request body | a Pydantic model | `payload: dict` |
| Request SIZE | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES` | `capped_body` on a route that does not read raw bytes itself |
| Ownership check | `owned_or_404` / `get_or_404` | ad-hoc `_own_playlist`/`_owned`/`_own_job` |
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`), test-held | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok` |
| Transaction commit | the handler commits; helpers `flush` (test-held list) | helpers that commit (only `quota` owns its session) |
| Admin knob storage | `sysconfig` (values) | env / `app_state` / `scheduler_settings` scatter |
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
| FE per-account state | `usePref` | raw `readAccount`/`writeAccount` or bespoke hooks |
| FE action button | `ui/Button` — pick a `SIZES` rung | a hand-rolled `<button className="px-3 py-2 …">` |
| FE keyboard shortcut | a row in `lib/keymap.ts` + `dispatch()` | a private `keydown` listener with its own key literals |
| FE list/grid keys | `NavGrid`/`keyboardNav` + `navItemAttrs` + `data-nav-open` | a private arrow-key / roving-tabindex handler |
| FE view push (history) | `rememberNavPosition()` right before `history.pushState` | a push that forgets: Back strands focus on `<body>` |
| FE control | `SegmentedToggle`; a long or user's list: `FilterSelect` | a native `<select>` (lint-enforced) |
| FE loading/empty/error | `QueryState` / `StateMessage` | ad-hoc conditionals |
| FE failed write | `meta.failTitle`, else `notifyActionError(err, title)` | a bare `catch {}` or `notify()` with no error attached |
| FE network call | `req`/`uploadReq`/`beacon`, or `lib/http.ts` `send` (public pages) | a bare `fetch`/`XMLHttpRequest`/`WebSocket` (lint-enforced) |
| FE diagnostics | `lib/log.ts` (`log.info("event.code", msg, attrs)`); a filter change → `lib/filterLog.ts`, a player → `lib/mediaLog.ts` | `console.*` (lint-enforced); a value only a higher level may carry, logged unguarded |
| FE state that MIRRORS a prop/query | derive it, or hold a draft (`draft ?? server`) | a copy an effect re-syncs — that paints the stale value first |
| FE state reset on a key change | adjust it DURING render against a remembered key | `useEffect(() => setX(init), [key])` |
| FE effect that must not re-subscribe | `useEffectEvent` (React 19.2+), else a latest-value ref | an excused dep list asserting the same thing in prose |
| FE hook suppression | an inline `eslint-disable-next-line` NAMING the invariant | `eslint-suppressions.json` (deliberately absent — E27 S2) |
## House rules that bite (learned from real bugs)
+3 -1
View File
@@ -10,7 +10,9 @@ ARG GIT_SHA=unknown
ARG BUILD_DATE=
# Stage 1: build the frontend SPA
FROM node:22-alpine AS frontend
# The major is pinned to `.nvmrc` and held there by frontend/src/lib/nodeEngine.contract.test.ts:
# this image, the CI gate's image and the dev machine must be ONE engine (see that test for why).
FROM node:24-alpine AS frontend
WORKDIR /fe
# Deps first, keyed on the lockfile, so editing app source never re-resolves node_modules.
COPY frontend/package.json frontend/package-lock.json ./
+1 -1
View File
@@ -1 +1 @@
0.92.0
0.95.0
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,81 @@
"""add debug_sessions + debug_screenshots, and index log_events.debug_session_id (E16)
Revision ID: 0089_debug_sessions
Revises: 0088_channels_feed_fallback_at
Create Date: 2026-09-19
A debug session is one recording: Start → reproduce → Stop (app/debugsession.py). Every log line
of that user's requests while it runs carries its id in `log_events.debug_session_id` — the column
E22 created and left NULL — and the issue it ends in is built from exactly those rows, so the column
gets its index now. Partial: nearly every row has no session, and an index of NULLs would be most
of the table for nothing.
At most one OPEN session per user, held by a partial unique index rather than by the start handler
alone: two tabs pressing Start at once would otherwise both pass a "none open" check.
Screenshots are rows, not files: bounded (size and count per session, app/debugsession.py), gone
with their session by cascade, and a self-host has no file volume to configure for them.
"""
import sqlalchemy as sa
from alembic import op
revision: str = "0089_debug_sessions"
down_revision: str | None = "0088_channels_feed_fallback_at"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.create_table(
"debug_sessions",
sa.Column("id", sa.String(16), primary_key=True),
sa.Column(
"user_id",
sa.Integer,
sa.ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
index=True,
),
sa.Column("level", sa.String(8), nullable=False),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("until", sa.DateTime(timezone=True), nullable=False),
sa.Column("ended_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("ended_reason", sa.String(16), nullable=True),
sa.Column("issue_url", sa.String(512), nullable=True),
)
op.create_index(
"uq_debug_sessions_open_per_user",
"debug_sessions",
["user_id"],
unique=True,
postgresql_where=sa.text("ended_at IS NULL"),
)
op.create_table(
"debug_screenshots",
sa.Column("id", sa.Integer, primary_key=True),
sa.Column(
"session_id",
sa.String(16),
sa.ForeignKey("debug_sessions.id", ondelete="CASCADE"),
nullable=False,
index=True,
),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("mime", sa.String(16), nullable=False),
sa.Column("size", sa.Integer, nullable=False),
sa.Column("data", sa.LargeBinary, nullable=False),
)
op.create_index(
"ix_log_events_debug_session",
"log_events",
["debug_session_id"],
postgresql_where=sa.text("debug_session_id IS NOT NULL"),
)
def downgrade() -> None:
op.drop_index("ix_log_events_debug_session", table_name="log_events")
op.drop_table("debug_screenshots")
op.drop_index("uq_debug_sessions_open_per_user", table_name="debug_sessions")
op.drop_table("debug_sessions")
@@ -0,0 +1,27 @@
"""add debug_sessions.draft — the issue being written from a session (E16 S3)
Revision ID: 0090_debug_session_draft
Revises: 0089_debug_sessions
Create Date: 2026-09-19
The composer saves as it goes: the title, the rendered markdown body that an export or an issue
carries, the log rows the user took out, and the form's own fields so reopening it later (Settings →
Diagnosztika lists past sessions) shows what was written. NULL until the composer first saves.
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects.postgresql import JSONB
revision: str = "0090_debug_session_draft"
down_revision: str | None = "0089_debug_sessions"
branch_labels: str | None = None
depends_on: str | None = None
def upgrade() -> None:
op.add_column("debug_sessions", sa.Column("draft", JSONB, nullable=True))
def downgrade() -> None:
op.drop_column("debug_sessions", "draft")
+5 -4
View File
@@ -13,16 +13,17 @@ Three layers, by *who* runs the code — not by feature:
response models, status-code mapping, serialization. It owns logic that runs *only* on a request.
Business logic that a request happens to trigger does **not** live here if anything else runs it too.
2. **`plex/`, `downloads/`, `sync/` — domain logic shared by more than one caller.** Anything the
2. **`plex/`, `downloads/`, `sync/`, `issues/` — domain logic shared by more than one caller.** Anything the
**scheduler or the worker** runs (not just a request) lives in one of these domain packages, so the
route, the worker, and the scheduler share one definition. Examples from this codebase:
`downloads/service.py` owns enqueue + the job state transitions (routes call it); `plex/sync.py` is
run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline.
run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline;
`issues/bundle.py` is the one definition of a bug report's files, for the zip export and the tracker.
3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth`, `sysconfig`, `quota`, `audit`,
`notifications`, `userscope`, `realtime`, `ratelimit`, `progress`, `logs` (the one logging
configuration), `logsink` (its `log_events` table) and `diagnostics` (a user's consented log
level), … are single-file primitives many layers import. They depend on nothing above them (a core
configuration), `logsink` (its `log_events` table), `diagnostics` (a user's consented log
level) and `debugsession` (a recording that tags its user's lines, E16), … are single-file primitives many layers import. They depend on nothing above them (a core
module must never import a `routes/…` module).
**Extraction trigger: the SECOND consumer.** Keep new logic inline in its first caller. The moment a
+3
View File
@@ -58,6 +58,9 @@ class AuditAction:
DOWNLOAD_CREDENTIAL_DELETE = "download_credential_delete"
# The log itself was cleared (a single row survives the clear as tamper-evidence)
AUDIT_CLEAR = "audit_clear"
# E16: a user sent a debug session's issue to the tracker — the one act that takes their log
# off this instance, so it is recorded (target = the session, summary = the issue's URL).
ISSUE_CREATE = "issue_create"
def record(
+6 -2
View File
@@ -4,7 +4,7 @@ from sqlalchemy import delete, func, select
from sqlalchemy.orm import Session
from starlette.requests import HTTPConnection
from app import diagnostics
from app import debugsession
from app import email as email_mod
from app import logs
from app import mailtext
@@ -367,7 +367,11 @@ def _session_user(request: Request, db: Session, user_id: int | None, is_default
# session middleware and this task's context. `request.state` is shared across middleware layers
# (it lives on the ASGI scope), so the access-log line can read it back. The user's consented
# diagnostics level rides along: their requests are logged — and stored — in that detail (S5).
logs.set_user(user.id, diagnostics.effective_level(user), user.diagnostics_until)
# So does a debug session the request names (E16): only the caller's own open one counts, and a
# header naming anything else is ignored rather than refused — a stale tab must keep working.
session = debugsession.from_header(db, user, request.headers.get(debugsession.HEADER))
level, until = debugsession.level_for(user, session)
logs.set_user(user.id, level, until, session.id if session else None)
request.state.user_id = user.id
return user
+231
View File
@@ -0,0 +1,231 @@
"""The global request-body cap (SEC-20).
**The hole.** FastAPI reads a declared body IN FULL before auth or any other dependency runs — the
body parameter has to be parsed to be passed in. So `POST /auth/password-login` drains a 32 MiB
upload from an anonymous caller and only then answers 401. 70 of the 72 body-declaring routes behave
the same way: one unauthenticated client can spend the process's memory and time at will.
**Why a middleware and not `capped_body` on every route.** `routes/_common.py:capped_body` is the
right tool where a route genuinely takes bulk bytes, and the two routes that use it use it
correctly. But putting it on all 70 others would (a) fight the canonical "Request body → a Pydantic
model" row, since capping means taking the body as raw bytes and parsing it by hand, and (b) leave
the NEXT route added uncovered, because nothing would remind its author. A default that has to be
opted into is not a default. This module inverts it: the cap applies to everything, and the few
routes that need more opt OUT, in one list, next to the code that reads it.
**Why raw ASGI and not `@app.middleware("http")`.** Starlette's BaseHTTPMiddleware cannot wrap
`receive`, so it can only read `Content-Length` — which a client may omit entirely (chunked) or
simply understate. The bytes have to be counted as they arrive, which is the same rule `capped_body`
already applies per-route, and counting needs the receive channel.
**Why the refusal is byte-identical to `capped_body`'s.** Two shapes for one fact is how the
`cancelled`/`canceled` bug happened. The 413 body here is built through the same `JSONResponse`
FastAPI renders `HTTPException(413, {"code": "too_large"})` with — not a hand-rolled `json.dumps`,
whose default separators differ by two bytes — and `tests/test_body_limit.py` compares the RAW BYTES
of the two against each other rather than against a literal, so they cannot drift apart. The same
reasoning decides where the middleware is registered (`main.py`): just inside CORS, so this 413
carries the same headers `capped_body`'s does.
**No admin knob, deliberately.** The ceiling is a constant, not a `sysconfig` value: an attacker-
facing limit that an admin can raise from the UI is a limit that can be raised by whoever gets the
admin session, and nobody has asked to tune it. If a self-hoster ever needs a different ceiling it
belongs in `config.py` (env, set before the process starts), not in the database.
"""
import re
from starlette.responses import JSONResponse
from starlette.types import ASGIApp, Message, Receive, Scope, Send
from app import debugsession
from app.config import settings
# 1 MiB. The largest body any route on the default ceiling legitimately takes is 512 KiB (the client
# log sink, `routes/diagnostics.py:BODY_MAX_BYTES`), so this leaves 2x headroom — a ceiling that
# refuses a legitimate request would be a worse bug than the one this module fixes. That ratio is
# the load-bearing fact, so `test_body_limit.py` holds it rather than the figure above.
DEFAULT_MAX_BODY_BYTES = 1024 * 1024
# The routes that need MORE than the default, as (method, path TEMPLATE) → their own ceiling. The
# template, not a concrete path, so the list reads like the route decorators it mirrors and
# `test_body_limit.py` can assert that each one still resolves to a real route: a renamed route
# fails the gate instead of silently losing its allowance in prod.
#
# These are RAISED CEILINGS, NOT EXEMPTIONS, and the difference is the whole point. An exemption
# would mean "this route is counted by something else" — which is only true when the route's own
# limit applies to the bytes as they ARRIVE. That, in turn, is only true of a route with NO DECLARED
# BODY PARAMETER: FastAPI parses and spools a declared body BEFORE the first dependency runs, so a
# route that declares one has already stored the bytes by the time its own cap — or its auth — gets
# a say.
#
# Both listed routes now satisfy that. The screenshot route always did (`capped_body` streams it,
# and the owner check resolves first). The upload route did NOT until P1: it declared
# `file: UploadFile = File(...)`, so an anonymous caller could make the process write the whole
# per-file allowance to temp before `require_human` refused them — this middleware bounded that
# number but could not stop the writing. It now reads its own body from a dependency declared after
# the auth one, so its in-handler cap measures bytes as they arrive, like everything else here.
#
# That rule is no longer only written here: `test_body_limit.py` parses the routes and fails the
# gate if any route in this dict declares a body parameter. A comment could not have stopped the
# defect it describes; the test can.
#
# A route belongs here ONLY IF ITS OWN CAP IS LARGER THAN THE DEFAULT. That is why
# `POST /api/diagnostics/events` is absent: its `capped_body` limit is 512 KiB, already below the
# default, so listing it could only widen what it accepts.
ROUTE_MAX_BODY_BYTES: dict[tuple[str, str], int] = {
# Ingesting a local file. The body IS the file since P1 — no multipart envelope to allow for —
# so this ceiling and the route's own cap count exactly the same bytes. `sysconfig`'s
# `download_upload_max_bytes` is bounded by this same constant (see sysconfig.SPECS), so an
# admin can only ever LOWER the route's cap below the edge, never raise it past one.
#
# WHICH LAYER ANSWERS, measured rather than assumed (deep-review round 1):
# * knob LOWERED (an override below the default) — the route's cap is the smaller number, so
# `capped_stream_to` trips first and answers 422 `upload_too_large` carrying the admin's own
# figure, after draining the rest of the body so the client can read it.
# * knob at the DEFAULT — the two numbers are EQUAL, and the edge always wins: a declared
# Content-Length over the cap is refused here before a byte is read, and an undeclared body
# trips this counter on the same chunk the route would have tripped on. The route's 422 is
# unreachable in that configuration, and no amount of headroom would change it — the
# declared-length check fires first for any body big enough to matter, and the route's drain
# pushes the count past the edge for the rest.
#
# So this 413 is a refusal a real user meets on a default install, not a theoretical edge, and
# it has to be as readable as the route's own: the client localizes `too_large` (`errors.tooLarge`)
# exactly as it does `upload_too_large`, and the upload dialog refuses an over-cap file before
# sending it at all, off the `upload_max_bytes` that `GET /api/downloads/usage` reports.
("POST", "/api/downloads/upload"): settings.download_upload_max_bytes,
("POST", "/api/debug-sessions/{sid}/screenshots"): debugsession.SCREENSHOT_MAX_BYTES,
}
# The refusal, byte-for-byte what `capped_body`'s `HTTPException(413, {"code": "too_large"})` renders
# as — built through the same JSONResponse so the separators match too, and asserted against a real
# route's raw bytes in `test_body_limit.py` rather than against a literal here.
TOO_LARGE_BODY = JSONResponse({"detail": {"code": "too_large"}}, status_code=413).body
_PATH_PARAM = re.compile(r"\{[^/{}]+\}")
def template_to_regex(template: str) -> re.Pattern[str]:
"""`/api/debug-sessions/{sid}/screenshots` → a regex matching one concrete path.
A path parameter matches `[^/]+` — one segment, never across a `/`, so `{sid}` cannot swallow a
deeper path and hand an exemption to a route that was never granted one.
"""
literals = [re.escape(part) for part in _PATH_PARAM.split(template)]
return re.compile("^" + "[^/]+".join(literals) + "$")
def _declared_length(scope: Scope) -> int | None:
"""The `Content-Length` header as an int, or None when absent or unparseable (a chunked body has
none, and a malformed one is not a number we may trust — both fall through to counting)."""
for name, value in scope.get("headers", ()):
if name == b"content-length":
try:
return int(value)
except ValueError:
# silent-ok: a malformed Content-Length is the caller's doing, not a fault of ours,
# and it is about to be answered anyway — returning None drops the request into the
# counting path, which refuses on the real bytes. Logging it would hand an anonymous
# client a free line in our log for every request it cares to malform.
return None
return None
async def _send_too_large(send: Send) -> None:
await send(
{
"type": "http.response.start",
"status": 413,
"headers": [
(b"content-type", b"application/json"),
(b"content-length", str(len(TOO_LARGE_BODY)).encode()),
],
}
)
await send({"type": "http.response.body", "body": TOO_LARGE_BODY})
class BodyLimitMiddleware:
"""Refuses any request body over its route's ceiling, before the application sees it.
Registered just inside CORS in `main.py` — so the 413 is decorated like every other response
and a browser can actually read it — but outside the session, the setup gate and every auth
dependency, none of which run for a request that is already too large.
Every request gets a finite ceiling: `ROUTE_MAX_BODY_BYTES` raises it for the two routes that
genuinely take bulk bytes, and there is no path that skips counting altogether.
"""
def __init__(
self,
app: ASGIApp,
*,
limit: int = DEFAULT_MAX_BODY_BYTES,
routes: dict[tuple[str, str], int] | None = None,
) -> None:
self.app = app
self.limit = limit
self._routes = tuple(
(method, template_to_regex(path), cap)
for (method, path), cap in (ROUTE_MAX_BODY_BYTES if routes is None else routes).items()
)
def limit_for(self, method: str, path: str) -> int:
"""This request's ceiling: the route's own if it has one, else the global default. Every
request gets a finite number — there is no path through here that is simply not counted."""
for m, rx, cap in self._routes:
if m == method and rx.match(path):
return cap
return self.limit
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http":
await self.app(scope, receive, send)
return
limit = self.limit_for(scope.get("method", ""), scope.get("path", ""))
declared = _declared_length(scope)
if declared is not None and declared > limit:
# Refused on the declaration alone: nothing downstream is called, and not one byte of
# the body is read.
await _send_too_large(send)
return
seen = 0
refused = False
started = False
async def counting_receive() -> Message:
nonlocal seen, refused
message = await receive()
if message["type"] != "http.request":
return message
seen += len(message.get("body", b""))
if seen > limit:
# The app is blocked on this receive, so it has not answered yet. Telling it the
# client went away is the only in-band way to stop it reading: `request.stream()`
# turns an `http.disconnect` into ClientDisconnect, which unwinds the handler. Our
# own 413 goes out below, once the app has finished unwinding.
refused = True
return {"type": "http.disconnect"}
return message
async def guarded_send(message: Message) -> None:
nonlocal started
if refused and not started:
return # discard the app's answer to the disconnect; the 413 below is the response
started = True
await send(message)
try:
await self.app(scope, counting_receive, guarded_send)
except Exception: # noqa: BLE001 — only swallowed when WE caused it, see below
# ClientDisconnect (or whatever a handler raises on a truncated read) is the expected
# consequence of the disconnect we just faked, and the caller must get the 413 rather
# than a 500. Anything raised when we did NOT refuse is a real failure and is re-raised
# untouched.
if not refused:
raise
if refused and not started:
await _send_too_large(send)
+13 -2
View File
@@ -1,7 +1,7 @@
from pathlib import Path
from typing import Literal, get_args, get_origin
from pydantic import ValidationInfo, field_validator, model_validator
from pydantic import Field, ValidationInfo, field_validator, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
# Shipped placeholder; production must override it. Used by the startup guard below.
@@ -247,7 +247,11 @@ class Settings(BaseSettings):
# Per-file cap for local uploads (E7). Unlike a download (whose size is unknown until the worker
# finishes), an upload's size is known at ingest, so this bounds a single file directly; the
# user's overall byte footprint is still governed by download_default_max_bytes.
download_upload_max_bytes: int = 524_288_000 # 500 MiB
# ge=1: this number is also the EDGE ceiling (`bodylimit.ROUTE_MAX_BODY_BYTES`), where 0 is a
# real limit of zero rather than the route's "no per-file cap" sentinel — an instance set to 0
# would refuse every upload, including the admin page's own way back. To lift the cap in
# practice, set a large number.
download_upload_max_bytes: int = Field(default=524_288_000, ge=1) # 500 MiB
# Retention: an asset expires this many days after its last access; the GC job then
# deletes it (once no job references it) after an additional grace window.
download_retention_days: int = 30
@@ -351,6 +355,13 @@ class Settings(BaseSettings):
tmdb_api_base: str = "https://api.themoviedb.org/3"
# Metadata language for TMDB fetches (keyword/overview locale). Stored EN by default.
tmdb_language: str = "en-US"
# --- Bug reports (E16): where a debug session's issue goes. All three empty = no tracker; the
# composer still exports a zip. The repository must be PRIVATE — an issue carries the reporter's
# log — and app/issues/gitea.py refuses a public one. Runtime knobs (sysconfig, admin UI). ---
issues_gitea_url: str = ""
issues_gitea_repo: str = ""
issues_gitea_token: str = ""
# How often the TMDB enrichment job fetches a batch of not-yet-enriched Plex titles, in minutes.
tmdb_enrich_interval_min: int = 120
# Titles fetched per TMDB enrichment run (TMDB is fast + generously rate-limited).
+48 -1
View File
@@ -1,6 +1,6 @@
from collections.abc import Iterator
from sqlalchemy import create_engine
from sqlalchemy import create_engine, text
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
from app.config import settings
@@ -34,3 +34,50 @@ def get_db() -> Iterator[Session]:
yield db
finally:
db.close()
def release_read_connection(db: Session) -> None:
"""End a request's READ transaction so its pooled connection goes back BEFORE a long await.
SQLAlchemy 2.0 autobegins on the first SELECT, so a request that has merely authenticated is
already holding one of the connections above — `idle in transaction` to Postgres, and nothing
times it out. A route that then awaits something slow (a request body still arriving, an
external fetch) pins it for the whole wait, which is precisely the invariant the pool is sized
on in the comment above. Measured: the next checkout after this call reuses the very backend
that was released, and a 1.8 s upload goes from one sustained `idle in transaction` to none.
Every route that reads its own body from a dependency needs this, and `test_body_limit.py`
holds that — the three that do are the ones with a raised (or self-imposed) body ceiling, and
they got here one at a time, which is how the upload route ended up the only one fixed.
The caller must have NOTHING PENDING, and that is checked rather than described. The session is
shared with every dependency that ran before, so a rollback here would silently discard a write
one of them staged — `service.enqueue` already learned that lesson the expensive way and uses a
SAVEPOINT for it. CLAUDE.md: a rule stated in a comment must also be enforced in code.
TWO checks, because the obvious one is blind to the shapes this codebase actually writes in.
`db.new`/`deleted`/`dirty` hold only UNFLUSHED unit-of-work: the moment a helper calls
`db.flush()` — CLAUDE.md's canonical split is "the handler commits; helpers flush" — the object
is persistent with no unflushed history and all three sets are empty again, and Core DML
(`db.execute(update(...))`) never enters them at all. Measured: after `add`+`flush` the sets are
empty, the old check passed, and the rollback lost the row. So the transaction is asked
directly. Postgres assigns a transaction id on the first WRITE and never for a read, so
`pg_current_xact_id_if_assigned()` is NULL for exactly the state this function is safe in — and
it sees a raw `text("UPDATE …")` too, which no session-level bookkeeping would.
"""
if db.new or db.deleted or db.dirty:
raise RuntimeError(
"release_read_connection() with pending work: something ahead of this dependency "
"staged a write, which ending the transaction would discard. Commit it first, or do "
"it after the await."
)
if (
db.in_transaction()
and db.execute(text("SELECT pg_current_xact_id_if_assigned()")).scalar() is not None
):
raise RuntimeError(
"release_read_connection() on a transaction that has already WRITTEN: something ahead "
"of this dependency flushed a row or ran an UPDATE/INSERT/DELETE, which ending the "
"transaction would discard. Commit it first, or do it after the await."
)
db.rollback()
+485
View File
@@ -0,0 +1,485 @@
"""Debug sessions (E16): Start → reproduce → Stop, and the evidence a bug report is built from.
While a session is open its user's browser names it in `X-Debug-Session` on every request. The auth
dependency accepts the header only for the caller's OWN open session (anything else is ignored, never
an error — a stale tab must not break the app) and then logs that request at the session's level with
every line tagged by its id; the browser tags its own events the same way. The issue is later built
from exactly the rows carrying the tag, so what the review list shows is what gets sent.
Starting a session IS the consent for its level, for its duration: the start dialog carries the
consent text. It needs no change to the user's own diagnostics level, and none is made — when the
session ends the level in force is simply the user's own again.
A session is capped at `MAX_MINUTES`, set by the server at Start. An open one past its cap is over
whether or not anything has closed it yet: it reads as over at once, a new Start closes it, and the
hourly log job sweeps the rest. Sessions go with the logs they point at, after `log_retention_days`.
Core layer: imports `models`, `diagnostics`.
"""
import secrets
from datetime import datetime, timedelta, timezone
from enum import StrEnum
from typing import Literal, get_args
from uuid import uuid4
from sqlalchemy import and_, delete, func, not_, select, update
from sqlalchemy.orm import Session, defer
from app import diagnostics
from app.models import DebugScreenshot, DebugSession, LogEvent, User
HEADER = "x-debug-session"
MAX_MINUTES = diagnostics.SESSION_MINUTES
SCREENSHOT_MAX_BYTES = 10 * 1024 * 1024
SCREENSHOTS_MAX = 8
# What one user may keep in screenshots across ALL their sessions (until retention takes them): the
# per-session cap alone would let Start → 8 images → Stop repeat without end.
SCREENSHOT_USER_MAX_BYTES = 100 * 1024 * 1024
# The screenshot types kept, and the extension each is exported and sent under — one table, so an
# accepted type always has a name.
IMAGE_TYPES = {"image/png": "png", "image/jpeg": "jpg", "image/webp": "webp"}
# What a screenshot is called in the zip and on the issue. `issues/bundle.py` builds the names from
# this and `IMAGE_TYPES`; this module and `issues/gitea.py` read them back to tell a screenshot's
# link from a log's. One definition, because three copies of a name is how the next rename breaks
# `images_pending` into silence.
SCREENSHOT_PREFIX = "screenshot-"
# Rows an issue can carry. A 60-minute session at Teljes is a few thousand rows; past this the
# bundle says how many it left out rather than growing without bound.
BUNDLE_MAX = 5_000
SessionLevel = Literal["DEBUG", "TRACE"]
SESSION_LEVELS: tuple[str, ...] = get_args(SessionLevel)
# Detail order of the diagnostics levels: a session raises the level in force, never lowers it.
_DETAIL = {"INFO": 0, "DEBUG": 1, "TRACE": 2}
_ID_LEN = 16
class EndReason(StrEnum):
STOPPED = "stopped"
EXPIRED = "expired"
def is_live(session: DebugSession, now: datetime | None = None) -> bool:
"""Open and within its cap — THE liveness rule: an open session past its cap is over."""
return session.ended_at is None and session.until > (now or datetime.now(timezone.utc))
def open_session(db: Session, user: User) -> DebugSession | None:
"""The user's live session, or None. Read-only: one past its cap is over and reads as None;
`start` and the hourly sweep are what close it."""
session = _unclosed(db, user)
if session is None or not is_live(session, datetime.now(timezone.utc)):
return None
return session
def _unclosed(db: Session, user: User) -> DebugSession | None:
return db.scalar(
select(DebugSession).where(DebugSession.user_id == user.id, DebugSession.ended_at.is_(None))
)
def start(db: Session, user: User, level: SessionLevel) -> DebugSession:
"""Open a session, or return the live one — a second tab pressing Start joins the recording
rather than ending it. One past its cap is closed first: the one-open-per-user index would
refuse the new row beside it. Flushes; the caller commits. Two concurrent Starts can still both
get here: the index refuses the second, and the caller joins the one that won."""
now = datetime.now(timezone.utc)
current = _unclosed(db, user)
if current is not None:
if is_live(current, now):
return current
end(current, EndReason.EXPIRED)
db.flush()
session = DebugSession(
id=secrets.token_hex(_ID_LEN // 2),
user_id=user.id,
level=level,
started_at=now,
until=now + timedelta(minutes=MAX_MINUTES),
)
db.add(session)
db.flush()
return session
def stop(session: DebugSession) -> None:
"""The user's Stop. Past its cap the session had already ended on its own, and a late Stop must
not rename that."""
end(session, EndReason.STOPPED if is_live(session) else EndReason.EXPIRED)
def end(session: DebugSession, reason: EndReason) -> None:
"""Close a session. An expired one is dated at its cap, not at whenever it was noticed."""
if session.ended_at is not None:
return
session.ended_at = (
session.until
if reason is EndReason.EXPIRED
else min(datetime.now(timezone.utc), session.until)
)
session.ended_reason = reason
def from_header(db: Session, user: User, value: str | None) -> DebugSession | None:
"""The session a request names, if it is the caller's own and still live; else None."""
if not value or len(value) != _ID_LEN or not all(c in "0123456789abcdef" for c in value):
return None
session = db.get(DebugSession, value)
if (
session is None
or session.user_id != user.id
or not is_live(session, datetime.now(timezone.utc))
):
return None
return session
def level_for(user: User, session: DebugSession | None) -> tuple[str, datetime | None]:
"""The diagnostics level (and its expiry) a request is logged at: the user's own, raised to the
session's while one is named — whichever records more.
A named session also caps the expiry at its own: this pair is what the process tracks under the
session (app/logs.py), and an expiry outliving the recording would hold the root logger down
after it ended."""
own = diagnostics.effective_level(user)
until = user.diagnostics_until if own != diagnostics.BASE else None
if session is None:
return own, until
if _DETAIL[session.level] <= _DETAIL[own]:
return own, min(until, session.until) if until else session.until
return session.level, session.until
def lock(db: Session, session: DebugSession) -> None:
"""Re-read the session under a row lock, held until the caller commits. Everything that changes
what an issue is made of — the draft, the screenshots, the send itself — takes it first: a send
in flight sees none of them change under it, and each of them sees a send that committed."""
db.refresh(session, with_for_update=True)
def save_draft(session: DebugSession, draft: dict) -> None:
"""Keep the composer's draft (the caller validated, locked and commits). Shape: `title`, `body`
(the rendered markdown an export or an issue carries), `excluded` (log row ids taken out of the
evidence) and `form` (the composer's own fields, to reopen it as it was). A delivery's progress
is not the composer's to replace, and is kept."""
progress = (session.draft or {}).get("delivery")
session.draft = {**draft, "delivery": progress} if progress else draft
# --- a delivery's progress -----------------------------------------------------------------
# Written by app/issues/gitea.py into the draft; read HERE, so the send's decision and what the
# composer shows are one reading of it.
def delivery(session: DebugSession) -> dict:
return dict((session.draft or {}).get("delivery") or {})
def not_uploaded(session: DebugSession) -> list[str]:
"""The files a send that stopped half-way has still to upload — by name, as the issue carries
them. What the composer names to the user."""
return list(delivery(session).get("pending") or [])
def images_pending(session: DebugSession) -> bool:
"""Whether the issue's body still has to be edited to show the screenshots it already carries
(every upload went, the edit that shows them did not)."""
state = delivery(session)
shown = state.get("images_shown")
return not shown and any(n.startswith(SCREENSHOT_PREFIX) for n in state.get("links") or {})
def unfinished(session: DebugSession) -> bool:
"""Whether a send owes the issue anything — the retry's question."""
return bool(not_uploaded(session)) or images_pending(session)
# A send claims the session while it talks to the tracker, instead of holding the row's lock across
# those calls: a Stop, an autosave or a screenshot would queue behind a lock for the whole send.
# They see the claim and are refused at once instead. A claim not renewed for this long was left by
# a request that died mid-send (the process restarted), and the next send takes it over — which is
# why a live send RENEWS it every time it reports progress (`set_delivery`). A claim is a lease, and
# a send slower than the lease must not lose it while it is still running.
SEND_CLAIM_SECONDS = 600
def _claim(session: DebugSession) -> dict:
claim = delivery(session).get("sending")
return claim if isinstance(claim, dict) else {}
def _stamp(token: str) -> dict:
return {"at": datetime.now(timezone.utc).isoformat(), "by": token}
def sending(session: DebugSession) -> bool:
"""Whether a send is under way right now (a live claim)."""
at = _claim(session).get("at")
if not at:
return False
return (datetime.now(timezone.utc) - datetime.fromisoformat(at)).total_seconds() < (
SEND_CLAIM_SECONDS
)
def claim_send(session: DebugSession) -> str:
"""Mark the session as being sent, and return the token that OWNS this claim. The caller holds
the lock and commits, which is what makes the claim visible to everyone else."""
token = uuid4().hex
_write_delivery(session, {**delivery(session), "sending": _stamp(token)})
return token
def release_send(session: DebugSession, token: str) -> None:
"""Give back our OWN claim, and only ours: one that aged out and was taken over belongs to the
send holding it now, and handing that one back would let a third send in beside it."""
if _claim(session).get("by") != token:
return
_write_delivery(session, {k: v for k, v in delivery(session).items() if k != "sending"})
class SendClaimLost(Exception):
"""This send's claim is no longer on the row: it aged out and another send took it over. The
send must stop — what it uploads from here it can no longer record, so a retry would send it
again — and it must not write, because the row now holds the other send's progress."""
def claim_owner(db: Session, session: DebugSession) -> str | None:
"""Who holds the row's claim RIGHT NOW, read from the ROW and not from the object.
`expire_on_commit` is off (app/db.py), so a checkpoint does not re-read: a send that stalled
past its lease still carries its own dead token in memory long after another send took over.
A targeted scalar, not `lock`/`db.refresh` — a refresh here would discard the `issue_url` that
`gitea.deliver` assigns one line before its first progress report, and that link is the one
thing the send must not lose."""
return db.scalar(
select(DebugSession.draft["delivery"]["sending"]["by"].as_string()).where(
DebugSession.id == session.id
)
)
def set_delivery(db: Session, session: DebugSession, state: dict, token: str) -> None:
"""Write a delivery's progress (app/issues/gitea.py keeps it; this module reads it), and renew
the claim while doing it — a send still reporting progress is still alive, so one slower than
the lease cannot expire while it is still running.
The claim is not the caller's to carry: `gitea.deliver` threads ONE snapshot of this dict
through a whole send, so a progress report that carried the claim wrote a stale one back at
every step. It is stamped from `token` here instead, and only after the ROW confirms the claim
is still ours. Without that check the write is a full-column overwrite with no WHERE: a send
that lost its lease came back and stomped the take-over send's committed links, then put its
own dead token back on an idle session (deep-review round 4). `release_send` has always made
the same check; this is the other half of it."""
if claim_owner(db, session) != token:
raise SendClaimLost(session.id)
state = {k: v for k, v in state.items() if k != "sending"}
state["sending"] = _stamp(token)
_write_delivery(session, state)
def _write_delivery(session: DebugSession, state: dict) -> None:
# A new dict, so the JSONB column sees the change.
session.draft = {**(session.draft or {}), "delivery": state}
def serialize(session: DebugSession | None, screenshots: int = 0) -> dict | None:
if session is None:
return None
return {
"id": session.id,
"level": session.level,
"started_at": session.started_at.isoformat(),
"until": session.until.isoformat(),
"ended_at": session.ended_at.isoformat() if session.ended_at else None,
"ended_reason": session.ended_reason,
"issue_url": session.issue_url,
# A send that stopped half-way: the files it has still to upload, and whether the issue's
# body has still to be edited to show the screenshots it already carries. Both are true of a
# send that is RUNNING too — every step commits its progress — so `sending` goes with them:
# without it a reload or a second tab renders a live send as a failed one, and offers a
# retry that is certain to be refused.
"pending_files": not_uploaded(session),
"images_pending": images_pending(session),
"sending": sending(session),
"screenshots": screenshots,
"draft": session.draft,
}
def view(db: Session, session: DebugSession) -> dict:
"""The session as the API serves it — one shape wherever it is served."""
return serialize(session, screenshot_count(db, session)) or {}
def screenshot_count(db: Session, session: DebugSession) -> int:
return (
db.scalar(
select(func.count())
.select_from(DebugScreenshot)
.where(DebugScreenshot.session_id == session.id)
)
or 0
)
# --- screenshots ---------------------------------------------------------------------------
# By content, not by the declared type: what is stored is served back to a browser as an image.
_SIGNATURES = (
(b"\x89PNG\r\n\x1a\n", "image/png"),
(b"\xff\xd8\xff", "image/jpeg"),
)
def sniff(data: bytes) -> str | None:
"""The image type `data` really is — one of IMAGE_TYPES — or None."""
for magic, mime in _SIGNATURES:
if data.startswith(magic):
return mime
if len(data) >= 12 and data[:4] == b"RIFF" and data[8:12] == b"WEBP":
return "image/webp"
return None
def lock_user(db: Session, user_id: int) -> None:
"""Hold this user's own lock for the rest of the transaction. A session's row lock bounds what
THAT session may keep; the byte cap below is the user's across all of them, and a lock on one
session cannot serialize a check that reads them all — two uploads to two sessions both read the
total before either commits, and both pass."""
db.execute(select(func.pg_advisory_xact_lock(user_id)))
def stored_screenshot_bytes(db: Session, user_id: int) -> int:
return (
db.scalar(
select(func.coalesce(func.sum(DebugScreenshot.size), 0))
.join(DebugSession, DebugSession.id == DebugScreenshot.session_id)
.where(DebugSession.user_id == user_id)
)
or 0
)
class ScreenshotRefused(Exception):
"""A screenshot that cannot be kept; `code` is the refusal's machine word."""
def __init__(self, code: str) -> None:
super().__init__(code)
self.code = code
def add_screenshot(db: Session, session: DebugSession, data: bytes) -> DebugScreenshot:
"""Keep one screenshot. Locks the user and the session, and flushes; the caller commits.
Refused while a send is under way or once the issue was sent (it can no longer carry it), past
the per-session count or the user's storage, or when it is not an image we can show."""
if not data:
raise ScreenshotRefused("empty")
mime = sniff(data)
if mime is None:
raise ScreenshotRefused("not_an_image")
# Under both locks: two uploads at once cannot both pass the count (the session's), nor both
# pass the user's byte cap from two different sessions (the user's). Always in this order.
lock_user(db, session.user_id)
lock(db, session)
if sending(session):
raise ScreenshotRefused("sending")
if session.issue_url is not None:
raise ScreenshotRefused("already_sent")
if screenshot_count(db, session) >= SCREENSHOTS_MAX:
raise ScreenshotRefused("too_many")
if stored_screenshot_bytes(db, session.user_id) + len(data) > SCREENSHOT_USER_MAX_BYTES:
raise ScreenshotRefused("storage_full")
shot = DebugScreenshot(
session_id=session.id,
created_at=datetime.now(timezone.utc),
mime=mime,
size=len(data),
data=data,
)
db.add(shot)
db.flush()
return shot
def remove_screenshot(db: Session, session: DebugSession, shot: DebugScreenshot) -> None:
"""Delete one screenshot, under the lock. Refused while a send is under way or unfinished: a
delivery names the screenshots by their order, and removing one would renumber what it still
has to send. Once the issue has everything, this only drops OUR copy — the issue keeps its
own. The caller commits."""
lock(db, session)
if sending(session):
raise ScreenshotRefused("sending")
if unfinished(session):
raise ScreenshotRefused("already_sent")
db.delete(shot)
db.flush()
def screenshots(db: Session, session: DebugSession) -> list[DebugScreenshot]:
"""The session's screenshots, oldest first, WITHOUT their bytes loaded."""
return list(
db.scalars(
select(DebugScreenshot)
.where(DebugScreenshot.session_id == session.id)
.options(defer(DebugScreenshot.data))
.order_by(DebugScreenshot.id)
)
)
# --- the evidence --------------------------------------------------------------------------
# The browser's own log upload (`POST` here, every few seconds while recording) is how the evidence
# arrives, not part of it: its server lines would fill the bundle with the recording's plumbing. The
# browser leaves its copies out for the same reason (frontend lib/diagnostics `isOwnRequest`).
INGEST_PATH = "/api/diagnostics/events"
def _evidence(session: DebugSession):
upload_line = and_(
LogEvent.event == "http.request", LogEvent.attrs["path"].astext == INGEST_PATH
)
# `IS TRUE`, not the bare condition: for a row with no event or no attrs it is NULL, and
# `NOT NULL` would drop exactly the rows that are evidence.
return (LogEvent.debug_session_id == session.id, not_(upload_line.is_(True)))
def bundle(
db: Session, session: DebugSession, *, drop_excluded: bool = False
) -> tuple[list[LogEvent], int]:
"""The session's log rows, browser and server interleaved in time, capped at BUNDLE_MAX — and
how many rows it has in all, so a cut bundle can say so. `drop_excluded` leaves out the rows
the user took out in the composer: what an export or an issue carries."""
where = list(_evidence(session))
excluded = (session.draft or {}).get("excluded") or [] if drop_excluded else []
if excluded:
where.append(LogEvent.id.not_in(excluded))
rows = list(
db.scalars(
select(LogEvent).where(*where).order_by(LogEvent.ts, LogEvent.id).limit(BUNDLE_MAX)
)
)
total = len(rows)
if total == BUNDLE_MAX:
total = db.scalar(select(func.count()).select_from(LogEvent).where(*where)) or total
return rows, total
# --- housekeeping (the hourly log job) -----------------------------------------------------
def sweep(db: Session) -> int:
"""Stage closing every open session past its cap; returns how many."""
now = datetime.now(timezone.utc)
result = db.execute(
update(DebugSession)
.where(DebugSession.ended_at.is_(None), DebugSession.until <= now)
.values(ended_at=DebugSession.until, ended_reason=EndReason.EXPIRED.value)
.execution_options(synchronize_session=False)
)
return result.rowcount or 0
def gc(db: Session, retention_days: int) -> int:
"""Stage deleting sessions (and by cascade their screenshots) started before the log retention
window — the rows they would bundle are gone by then too. Returns how many."""
cutoff = datetime.now(timezone.utc) - timedelta(days=max(1, retention_days))
result = db.execute(delete(DebugSession).where(DebugSession.started_at < cutoff))
return result.rowcount or 0
+4
View File
@@ -27,6 +27,9 @@ DiagnosticsLevel = Literal["INFO", "DEBUG", "TRACE"]
LEVELS: tuple[str, ...] = get_args(DiagnosticsLevel)
BASE: DiagnosticsLevel = "INFO"
ELEVATION_HOURS = 24
# How long a debug session records at most (E16, app/debugsession.py). Here, beside the other
# consent window, because the start dialog's consent text states it and reads it from `state`.
SESSION_MINUTES = 60
def effective_level(user: User, now: datetime | None = None) -> DiagnosticsLevel:
@@ -47,6 +50,7 @@ def state(db: Session, user: User) -> dict:
"level": level,
"until": user.diagnostics_until.isoformat() if level != BASE else None,
"retention_days": sysconfig.get_int(db, "log_retention_days"),
"session_minutes": SESSION_MINUTES,
}
+2 -2
View File
@@ -25,7 +25,7 @@ from app.downloads.jobdb import (
)
from app.downloads.stated import StatedFailure
from app.downloads.status import JobStatus
from app.models import DownloadJob, MediaAsset
from app.models import DISPLAY_TEXT_MAX, DownloadJob, MediaAsset
from app.notifications import create_notification
from app.titles import normalize_title
@@ -597,7 +597,7 @@ def _download(
}
if job and not job.display_name and meta.title:
# display_name is VARCHAR(255) — clip a long title (see _fill_asset_meta_early).
fields["display_name"] = meta.title[:255]
fields["display_name"] = meta.title[:DISPLAY_TEXT_MAX]
# Atomic terminal write: finalizes ONLY while the row is still running, so a child that
# finishes just after a route cancel/pause can't resurrect it (see finalize_if_running).
finalized = service.finalize_if_running(db, job_id, **fields)
+8 -5
View File
@@ -20,7 +20,7 @@ from app.downloads import credentials
from app.downloads import edit as editmod
from app.downloads import formats, mediakind, quota, storage
from app.downloads.status import HOLDING, JobStatus
from app.models import Channel, DownloadJob, MediaAsset, Video
from app.models import DISPLAY_TEXT_MAX, Channel, DownloadJob, MediaAsset, Video
class EditError(Exception):
@@ -179,7 +179,7 @@ def enqueue(
profile_id=profile_id,
profile_snapshot=spec,
display_name=display_name
or (asset.title[:255] if asset.status == "ready" and asset.title else None),
or (asset.title[:DISPLAY_TEXT_MAX] if asset.status == "ready" and asset.title else None),
status="queued",
queue_pos=next_queue_pos(db),
)
@@ -198,10 +198,13 @@ def enqueue(
def _clip_title(source_title: str | None, display_name: str | None) -> str:
# Clipped to DISPLAY_TEXT_MAX, not to a literal: this feeds `DownloadJob.display_name` through
# `enqueue_edit`, the same column `display_text` guards. A machine-made title, so it is clipped
# rather than run through `display_text` — blank here means "no title", not None.
if display_name:
return display_name[:255]
return display_name[:DISPLAY_TEXT_MAX]
base = source_title or "Clip"
return f"{base} (clip)"[:255]
return f"{base} (clip)"[:DISPLAY_TEXT_MAX]
def enqueue_edit(
@@ -255,7 +258,7 @@ def enqueue_edit(
source_asset_id=src_asset.id,
edit_spec=spec,
profile_snapshot={},
display_name=display_name or asset.title,
display_name=display_name or asset.title[:DISPLAY_TEXT_MAX],
status="queued",
queue_pos=next_queue_pos(db),
)
+2 -2
View File
@@ -25,7 +25,7 @@ from sqlalchemy.orm import Session
from app.config import settings
from app.downloads import edit as editmod
from app.downloads import mediakind, probe, quota, service, storage
from app.models import DownloadJob, MediaAsset
from app.models import DownloadJob, MediaAsset, display_text
def _title_from_filename(filename: str) -> str:
@@ -99,7 +99,7 @@ def ingest_upload(
source_kind="upload",
source_ref=ref,
profile_snapshot={},
display_name=(display_name or "").strip() or None,
display_name=display_text(display_name),
status="done",
progress=100,
queue_pos=service.next_queue_pos(db),
+3
View File
@@ -0,0 +1,3 @@
"""Bug reports built from debug sessions (E16): the files an issue carries, and where they go —
downloaded as a zip, or (S4) sent to a Gitea issue tracker. The session itself — what is recorded
and tagged — is the core module `app/debugsession.py`."""
+79
View File
@@ -0,0 +1,79 @@
"""What an issue from a debug session carries, as files — one definition for the zip export and the
Gitea upload, so the two cannot drift apart.
* `issue.md` — the title and the body the user wrote and reviewed in the composer (rendered there,
in their language; the server does not rewrite it);
* `client.log` / `server.log` — the session's evidence (`debugsession.bundle`) minus the rows the
user took out, one JSON object per line, browser rows and server rows apart. `.log`, not
`.jsonl`: Gitea refuses that extension (measured, plans/e16 S0);
* the screenshots, in the order they were taken.
Every row is already redacted twice (at capture, and again when stored), and nothing here adds a
field the review list did not show.
"""
import io
import json
import zipfile
from dataclasses import dataclass
from sqlalchemy import select
from sqlalchemy.orm import Session
from app import debugsession, logsink
from app.models import DebugScreenshot, DebugSession
@dataclass(frozen=True)
class IssueFile:
name: str
data: bytes
mime: str
def title_of(session: DebugSession) -> str:
# Not a sentence of ours: the composer always saves a title in the user's language, and a
# session exported before it ever saved is named by its date alone, in no language at all.
title = ((session.draft or {}).get("title") or "").strip()
return title or f"{session.started_at:%Y-%m-%d %H:%M} · {session.id}"
def body_of(session: DebugSession) -> str:
return ((session.draft or {}).get("body") or "").strip()
def _lines(rows) -> bytes:
return "".join(
json.dumps(logsink.serialize(r), ensure_ascii=False, default=str) + "\n" for r in rows
).encode()
def files(db: Session, session: DebugSession) -> list[IssueFile]:
"""The session's issue as files: the text, the two logs, the screenshots."""
rows, _ = debugsession.bundle(db, session, drop_excluded=True)
text = f"# {title_of(session)}\n\n{body_of(session)}\n"
out = [
IssueFile("issue.md", text.encode(), "text/markdown"),
IssueFile("client.log", _lines(r for r in rows if r.source == "web"), "text/plain"),
IssueFile("server.log", _lines(r for r in rows if r.source != "web"), "text/plain"),
]
shots = db.scalars(
select(DebugScreenshot)
.where(DebugScreenshot.session_id == session.id)
.order_by(DebugScreenshot.id)
)
for n, shot in enumerate(shots, start=1):
ext = debugsession.IMAGE_TYPES[shot.mime]
name = f"{debugsession.SCREENSHOT_PREFIX}{n}.{ext}"
out.append(IssueFile(name, shot.data, shot.mime))
return out
def zipped(items: list[IssueFile]) -> bytes:
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as z:
for item in items:
# The images are compressed already: deflating them again costs time and wins nothing.
method = zipfile.ZIP_STORED if item.mime.startswith("image/") else zipfile.ZIP_DEFLATED
z.writestr(item.name, item.data, compress_type=method)
return buf.getvalue()
+288
View File
@@ -0,0 +1,288 @@
"""The Gitea issue tracker a debug session's issue goes to (E16 S4) — measured against Gitea 1.27.1
before a line of this was written (plans/e16-debug-issue.md, S0).
Configured by an admin (sysconfig `issues_gitea_url` / `_repo` / `_token`, the token encrypted at
rest). The repository must be PRIVATE: an issue carries its reporter's log — page titles, filter
values, at Teljes (TRACE) what they typed into search — and a public repository would publish it. That is
checked on every send, not only in the admin's "test connection", because a repository can be made
public after the fact.
What an issue is made of comes from `bundle.files` (the same files the zip export carries). Sending
is three kinds of call, measured:
* `POST /repos/{repo}/issues` — the title and the body the user wrote;
* `POST /repos/{repo}/issues/{n}/assets?name=…` — ONE file per request (field `attachment`);
* `PATCH /repos/{repo}/issues/{n}` — the body again, with the screenshots shown inline once their
links exist.
A private repository's attachment answers 404 to a browser not signed in to Gitea, which is why the
composer's own preview shows our copies, never these links.
Delivery keeps its progress in the session's draft (`delivery`: the issue number, the files still to
upload, the links of those uploaded, whether the body shows the screenshots yet), so a failure
half-way keeps the issue that exists and a retry does only what is missing — never a second issue.
Every step is checkpointed (the caller's `db.commit`), so what did happen is kept whatever fails
next — and the first checkpoint comes before the first call, with the tracker's settings read ABOVE
it, so neither a row lock nor a read snapshot nor a pooled connection is held across the calls. The
caller claims the session for the send first (`debugsession.claim_send`) and passes the claim's
token in, which is what keeps a second send, an autosave or a screenshot out of the way; the claim
is renewed by every progress report, and a report whose claim is gone raises rather than writing
over the send that took it. `debugsession.not_uploaded` / `images_pending` read that progress.
"""
from collections.abc import Callable, Iterator
from contextlib import contextmanager
from typing import Any, Literal
import httpx
from sqlalchemy.orm import Session
from app import debugsession, logs, sysconfig
from app.issues import bundle
from app.models import DebugSession
log = logs.get_logger(__name__)
TIMEOUT_SECONDS = 20.0
Reason = Literal[
"not_configured", # url, repository or token missing
"unreachable", # no answer, or a 5xx
"bad_token", # 401/403: the token is wrong or lacks write:issue / read:repository
"repo_not_found", # 404 on the repository
"public_repo", # the repository is public — an issue would publish the reporter's log
"issues_disabled", # the repository has its issue tracker switched off
"rejected", # any other 4xx (a file type the instance does not allow, …), or an answer that is
# not the API's (a redirect — an http:// address behind https —, a page that is not JSON)
]
class GiteaError(Exception):
def __init__(self, reason: Reason) -> None:
super().__init__(reason)
self.reason: Reason = reason
def _settings(db: Session) -> tuple[str, str, str]:
return (
sysconfig.get_str(db, "issues_gitea_url").strip().rstrip("/"),
sysconfig.get_str(db, "issues_gitea_repo").strip().strip("/"),
sysconfig.get_str(db, "issues_gitea_token").strip(),
)
def target(db: Session) -> dict | None:
"""Where a report goes, for the composer to say so — the repository — or None when no tracker is
configured (the report can then only be downloaded)."""
url, repo, token = _settings(db)
return {"repo": repo} if url and repo and token else None
@contextmanager
def _client(settings: tuple[str, str, str]) -> Iterator[tuple[httpx.Client, str]]:
# Takes the settings already read, not the session: reading them HERE opened a transaction one
# line after `deliver` had ended one, and held it across the calls below (deep-review round 4).
url, repo, token = settings
if not (url and repo and token):
raise GiteaError("not_configured")
with httpx.Client(
base_url=f"{url}/api/v1",
headers={"Authorization": f"token {token}", "Accept": "application/json"},
timeout=TIMEOUT_SECONDS,
) as client:
yield client, repo
def _call(client: httpx.Client, method: str, path: str, **kwargs) -> httpx.Response:
try:
r = client.request(method, path, **kwargs)
except httpx.HTTPError as exc:
log.warning(
"the issue tracker did not answer",
extra=logs.event("issues.gitea.unreachable", path=path, error=type(exc).__name__),
)
raise GiteaError("unreachable") from None
if r.status_code < 300:
return r
# The status only: Gitea's message can quote back what was sent — the user's own title.
log.warning(
"the issue tracker refused a call",
extra=logs.event("issues.gitea.refused", path=path, status=r.status_code),
)
if r.status_code in (401, 403):
raise GiteaError("bad_token")
if r.status_code == 404:
raise GiteaError("repo_not_found")
if r.status_code >= 500:
raise GiteaError("unreachable")
raise GiteaError("rejected")
def _json(r: httpx.Response, *keys: str) -> dict[str, Any]:
"""The answer as the API's JSON, holding `keys` — anything else is refused, not a crash, so a
step after the issue was created cannot take down the request that holds its record.
The CREATE call is the one place this cannot keep that record: its answer IS the record. An
answer it cannot read there leaves the session unsent, and a retry opens another issue — the
price of not being able to tell a refused create from an unreadable one."""
try:
data = r.json()
# silent-ok: not JSON is one of the answers refused (and logged) just below.
except ValueError:
data = None
if not isinstance(data, dict) or any(k not in data for k in keys):
log.warning(
"the issue tracker answered something that is not its API",
extra=logs.event("issues.gitea.unexpected", status=r.status_code),
)
raise GiteaError("rejected")
return data
def _private_repo(client: httpx.Client, repo: str) -> dict:
data = _json(_call(client, "GET", f"/repos/{repo}"))
if not data.get("private"):
raise GiteaError("public_repo")
if not data.get("has_issues"):
raise GiteaError("issues_disabled")
return data
def check(db: Session) -> dict:
"""The admin's "test connection": the repository answers, is private and takes issues."""
with _client(_settings(db)) as (client, repo):
data = _private_repo(client, repo)
return {"repo": data.get("full_name") or repo, "url": data.get("html_url")}
def preview(db: Session, text: str) -> str:
"""The body as Gitea itself will render it (its markdown flavour, its sanitiser).
**No `Context`, deliberately — a repository context here does not fail the call, it kills the
tracker.** With one, Gitea's post-processor turns every hex-shaped word into a commit link, and
on the API's render path the commit checker it asks is nil: the whole Gitea process segfaults
and restarts, taking the git server (and, here, the container registry) down with it. Every
report carries hex — a tab id, a request id — so this was not an edge case but the only case.
Measured on throwaway instances (2026-09-20): `{"Mode": "gfm", "Context": "<repo>"}` with
`5645b1c0ae01` in the text crashes 1.26.4 and 1.27.1 and is fixed in 1.27.3 — but a self-hoster
runs whatever version they run, so the payload stays safe here rather than relying on theirs.
Dropping `Context` is not a downgrade of the preview: measured against the same body, the HTML
keeps its tables, its task lists and its highlighted code blocks. It only stops resolving
repo-relative links (a hash, a `#123`), which the issue page itself still resolves.
"""
with _client(_settings(db)) as (client, _repo):
r = _call(client, "POST", "/markdown", json={"Text": text, "Mode": "gfm"})
return r.text
def deliver(
db: Session,
session: DebugSession,
*,
token: str,
checkpoint: Callable[[], None],
before_create: Callable[[], None] | None = None,
on_created: Callable[[], None] | None = None,
) -> str:
"""Send the session's issue, or finish sending it; returns its URL.
`token` is the caller's claim on this session (`debugsession.claim_send`). Every progress report
renews it and checks the row still holds it, so a send that outlived its lease stops here rather
than writing over the one that took over (`debugsession.SendClaimLost`).
`checkpoint` persists what has happened so far and is called after every step (the caller's
commit — this module does not decide when a transaction ends). `before_create` runs right before
the tracker is asked to open a NEW issue, after the checks that can refuse it without asking,
and may raise to stop it; `on_created` runs right after one was opened, in time to be committed
by the SAME checkpoint that first writes `issue_url` — anything the caller must not be able to
lose separately from the link belongs there.
"""
state = debugsession.delivery(session)
files = bundle.files(db, session)
attachments = [f for f in files if f.name != "issue.md"]
settings = _settings(db)
# Reading the bundle and the tracker's settings opened a transaction; end it before the first
# call, so nothing holds a snapshot (and an idle connection) across minutes of tracker I/O.
# The settings are read HERE and handed to `_client`, not read inside it: reading them there
# was a SELECT one line after this commit, which reopened the transaction and held it — idle,
# with its snapshot pinned — across the repository check and the create (deep-review round 4
# measured it in pg_stat_activity). A comment saying otherwise sat right here.
checkpoint()
with _client(settings) as (client, repo):
_private_repo(client, repo)
if "number" not in state:
if before_create is not None:
before_create()
created = _json(
_call(
client,
"POST",
f"/repos/{repo}/issues",
json={"title": bundle.title_of(session), "body": bundle.body_of(session)},
),
"number",
"html_url",
)
# Built ON the state read at the start so nothing already in it is dropped. The claim
# is NOT among those things, whatever this comment used to say: `set_delivery` strips
# any `sending` passed to it and stamps the row's own from `token` (round 4 — removing
# the spread left the whole suite green, because the claim never travelled here).
state = {
**state,
"number": created["number"],
"pending": [f.name for f in attachments],
"links": {},
}
session.issue_url = created["html_url"]
try:
debugsession.set_delivery(db, session, state, token)
except debugsession.SendClaimLost:
# The one step where losing the claim costs something irreversible: the issue is
# already open on the tracker and this is what would have recorded its link. We
# still must not write — another send owns the row and is opening its own issue —
# so say which issue was left behind, because nothing else will.
# noqa TRY400: the traceback is one SendClaimLost carrying the session id and
# nothing else; the issue number below is the whole news.
log.error( # noqa: TRY400
"an issue was created by a send that had already lost its claim",
extra=logs.event(
"issues.gitea.orphaned",
number=created["number"],
session_id=session.id,
),
)
raise
if on_created is not None:
on_created()
checkpoint()
log.info(
"issue created",
extra=logs.event("issues.gitea.created", number=created["number"]),
)
number = state["number"]
for item in attachments:
if item.name not in state["pending"]:
continue
asset = _json(
_call(
client,
"POST",
f"/repos/{repo}/issues/{number}/assets",
params={"name": item.name},
files={"attachment": (item.name, item.data, item.mime)},
),
"browser_download_url",
)
state = {
**state,
"pending": [n for n in state["pending"] if n != item.name],
"links": {**state["links"], item.name: asset["browser_download_url"]},
}
debugsession.set_delivery(db, session, state, token)
checkpoint()
shots = [n for n in state["links"] if n.startswith(debugsession.SCREENSHOT_PREFIX)]
if shots and not state.get("images_shown"):
images = "\n".join(f"![{n}]({state['links'][n]})" for n in sorted(shots))
body = f"{bundle.body_of(session)}\n\n---\n\n{images}"
_call(client, "PATCH", f"/repos/{repo}/issues/{number}", json={"body": body})
debugsession.set_delivery(db, session, {**state, "images_shown": True}, token)
checkpoint()
return session.issue_url or ""
+48 -13
View File
@@ -100,12 +100,14 @@ class _RequestScope:
request; every copied context shares the same object, so what the dependency writes is seen by
the handler and by the request line itself."""
__slots__ = ("user_id", "level")
__slots__ = ("user_id", "level", "debug_session_id")
def __init__(self) -> None:
self.user_id: int | None = None
# The user's consented diagnostics level as a logging number; None at the base level.
self.level: int | None = None
# The user's own open debug session this request named (E16), else None.
self.debug_session_id: str | None = None
_scope: contextvars.ContextVar[_RequestScope | None] = contextvars.ContextVar(
@@ -113,11 +115,23 @@ _scope: contextvars.ContextVar[_RequestScope | None] = contextvars.ContextVar(
)
def set_user(user_id: int | None, level: str = "INFO", until: datetime | None = None) -> None:
def set_user(
user_id: int | None,
level: str = "INFO",
until: datetime | None = None,
debug_session: str | None = None,
) -> None:
"""Attribute the rest of this request's log lines to `user_id`, and log them at the user's
consented diagnostics `level` until `until` (E22 S5; "INFO" = no elevation). Called from the
auth dependency — the one place the session is actually resolved into an account — rather than
from the request middleware, which runs OUTSIDE the session middleware."""
from the request middleware, which runs OUTSIDE the session middleware.
`debug_session` tags every line with the user's open debug session (E16). The caller has
already checked it is theirs and open; a session's level arrives here as `level`/`until`.
The root logger's floor is tracked per elevation: a request without the session's header (an
`<img>`, a `<video>` — the browser sends those itself) carries only the user's own level, and
must not lower the floor a concurrent request of the session still needs."""
_user_id.set(user_id)
scope = _scope.get()
if scope is None: # outside log_requests (a script, a unit test): this context only
@@ -125,9 +139,11 @@ def set_user(user_id: int | None, level: str = "INFO", until: datetime | None =
_scope.set(scope)
elevated = level != "INFO" and until is not None and until > datetime.now(timezone.utc)
scope.user_id = user_id
scope.debug_session_id = debug_session
scope.level = resolve_level(level) if elevated else None
if user_id is not None:
_track_elevation(user_id, scope.level, until)
key = user_id if debug_session is None else (user_id, debug_session)
_track_elevation(key, scope.level, until)
def set_job(job_id: int | None) -> None:
@@ -328,6 +344,12 @@ class ContextFilter(logging.Filter):
scope = _scope.get()
if scope is not None and scope.user_id is not None and not hasattr(record, "user_id"):
record.user_id = scope.user_id
if (
scope is not None
and scope.debug_session_id is not None
and not hasattr(record, "debug_session_id")
):
record.debug_session_id = scope.debug_session_id
return True
@@ -444,7 +466,6 @@ _THIRD_PARTY_LEVELS = {
"httpx": "WARNING",
"httpcore": "WARNING",
"asyncio": "WARNING",
"multipart": "WARNING",
"PIL": "WARNING",
"alembic": "INFO",
# uvicorn's own startup/shutdown/WebSocket lines are worth keeping; its ACCESS log is not —
@@ -545,18 +566,21 @@ def current_level() -> str:
# The base level (`LOG_LEVEL`, then the admin knob). NOTSET until configure_logging runs — a unit
# test, a script — so the gate lets everything through there, as it did before the gate existed.
_base_level = logging.NOTSET
# user id → (level number, expiry) for every elevation this process has seen and not yet expired.
_elevations: dict[int, tuple[int, datetime]] = {}
# → (level number, expiry) for every elevation this process has seen and not yet expired: a user's
# own, by user id, and a debug session's (E16), by (user id, session id) — its requests alone carry
# its level, so it is tracked apart from the user's own requests.
_ElevationKey = int | tuple[int, str]
_elevations: dict[_ElevationKey, tuple[int, datetime]] = {}
_elevations_lock = threading.Lock()
_root_applied: int | None = None
def _track_elevation(user_id: int, level: int | None, until: datetime | None) -> None:
def _track_elevation(key: _ElevationKey, level: int | None, until: datetime | None) -> None:
with _elevations_lock:
if level is None or until is None:
_elevations.pop(user_id, None)
_elevations.pop(key, None)
else:
_elevations[user_id] = (level, until)
_elevations[key] = (level, until)
active = bool(_elevations)
if active or _root_applied != _base_level:
_apply_root_level()
@@ -573,8 +597,8 @@ def _apply_root_level(force: bool = False) -> None:
# different answer would otherwise apply them in either order, and the older one could win —
# the root back at the base while an elevation is still active.
with _elevations_lock:
for user_id in [u for u, (_, until) in _elevations.items() if until <= now]:
del _elevations[user_id]
for key in [k for k, (_, until) in _elevations.items() if until <= now]:
del _elevations[key]
lowest = min([_base_level, *(lvl for lvl, _ in _elevations.values())])
if force or lowest != _root_applied:
logging.getLogger().setLevel(lowest)
@@ -611,13 +635,24 @@ def forget_user(user_id: int) -> None:
del _erased[stale]
_erased[user_id] = now
with _elevations_lock:
_elevations.pop(user_id, None)
for key in [
k for k in _elevations if k == user_id or (isinstance(k, tuple) and k[0] == user_id)
]:
del _elevations[key]
# That pop can have removed the lowest elevation, and the root level is the lowest of the base
# and every unexpired one. Recomputed here rather than left to whichever later request happens
# to call `_track_elevation` — the invariant `_apply_root_level` documents holds at every pop.
_apply_root_level()
def end_session_elevation(session_id: str) -> None:
"""A debug session was stopped: its elevation ends now, not at its cap."""
with _elevations_lock:
for key in [k for k in _elevations if isinstance(k, tuple) and k[1] == session_id]:
del _elevations[key]
_apply_root_level()
def is_erased(user_id: int | None) -> bool:
if user_id is None:
return False
+22
View File
@@ -44,6 +44,7 @@ from app import logs
from app.config import settings
from app.db import engine
from app.models import LogEvent
from app.utils import iso
QUEUE_MAX = 10_000
BATCH_MAX = 500
@@ -343,3 +344,24 @@ def gc(db: Session, retention_days: int) -> int:
cutoff = datetime.now(timezone.utc) - timedelta(days=max(1, retention_days))
result = db.execute(delete(LogEvent).where(LogEvent.ts < cutoff))
return result.rowcount or 0
def serialize(row: LogEvent) -> dict:
"""A stored row as the API sends it — one shape for every reader: the admin console (E22.6) and
a debug session's evidence list (E16)."""
return {
"id": row.id,
"ts": iso(row.ts),
"level": row.level,
"source": row.source,
"logger": row.logger,
"event": row.event,
"msg": row.msg,
"attrs": row.attrs,
"exc": row.exc,
"request_id": row.request_id,
"user_id": row.user_id,
"job_id": row.job_id,
"debug_session_id": row.debug_session_id,
"version": row.version,
}
+33 -10
View File
@@ -23,6 +23,7 @@ from fastapi.staticfiles import StaticFiles
from sqlalchemy.exc import DataError
from starlette.middleware.sessions import SessionMiddleware
from app import bodylimit
from app import logsink
from app import state
from app import sysconfig
@@ -37,6 +38,7 @@ from app.routes import (
auth as auth_routes,
channels,
config as config_routes,
debug_sessions as debug_sessions_routes,
diagnostics as diagnostics_routes,
discover as discover_routes,
downloads,
@@ -188,15 +190,6 @@ app.add_middleware(
https_only=settings.session_https_only, # Secure flag when served over HTTPS (prod)
)
if settings.frontend_origin:
app.add_middleware(
CORSMiddleware,
allow_origins=[settings.frontend_origin],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Paths reachable while the instance is unconfigured (setup mode). Everything else under /api or
# /auth is locked until the wizard finishes; static/SPA always loads so the wizard page can render.
_SETUP_OPEN_PREFIXES = ("/api/setup", "/api/version")
@@ -216,6 +209,34 @@ async def setup_gate(request, call_next):
return await call_next(request)
# SEC-20 — the global request-body cap (app/bodylimit.py). FastAPI reads a declared body IN FULL
# before the first dependency, so without this the anonymous login route is a free memory/time sink.
#
# WHERE it sits is the whole design, and it is pinned by `test_body_limit.py` rather than by this
# comment, because `add_middleware` inverts the reading order (last registered = OUTERMOST) and a
# stack described in prose drifts. Registered here it lands directly INSIDE CORS and directly
# OUTSIDE the setup gate, which is the narrowest position that satisfies both constraints:
#
# * inside CORS and the security headers, so the 413 is decorated like every other response. A
# refusal the browser drops for a missing Access-Control-Allow-Origin is no better than a hang,
# and `capped_body`'s 413 one layer down IS decorated — two shapes for one fact is exactly what
# bodylimit.py exists to avoid.
# * outside the setup gate, the session and every auth dependency under the router — none of which
# run for a body that is already too large. The setup gate matters specifically: on an instance
# that is not yet configured it opens a DB session per request, which is precisely the anonymous
# pre-auth work SEC-20 exists to remove.
app.add_middleware(bodylimit.BodyLimitMiddleware)
if settings.frontend_origin:
app.add_middleware(
CORSMiddleware,
allow_origins=[settings.frontend_origin],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.middleware("http")
async def static_cache_headers(request, call_next):
"""Long-cache the content-hashed SPA bundles. Vite hashes their filename, so a given
@@ -258,7 +279,8 @@ async def security_headers(request, call_next):
# The access log (app/logs.py): request id, route template, status, duration — one line, never a
# query string. Registered LAST so it is the OUTERMOST http middleware and therefore times the whole
# request, including what the three above do. Being outermost also puts it OUTSIDE SessionMiddleware,
# request, including what every layer above does — the body cap included, so a 413 is still logged
# with its request id and duration. Being outermost also puts it OUTSIDE SessionMiddleware,
# which is why the account comes from `request.state.user_id` (set by the auth dependency, the one
# place a session is actually resolved into an account) rather than from `request.session`.
app.middleware("http")(log_requests)
@@ -273,6 +295,7 @@ app.include_router(discover_routes.router)
app.include_router(search_routes.router)
app.include_router(me.router)
app.include_router(diagnostics_routes.router)
app.include_router(debug_sessions_routes.router)
app.include_router(notifications.router)
app.include_router(messages.router)
app.include_router(channels.router)
+87 -4
View File
@@ -12,6 +12,7 @@ from sqlalchemy import (
Index,
Integer,
JSON,
LargeBinary,
SmallInteger,
String,
Text,
@@ -22,6 +23,34 @@ from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
from sqlalchemy.orm import Mapped, mapped_column, relationship, validates
from app.db import Base
# Every user-supplied display string in this schema shares one width, and it lives HERE so the
# columns below are declared with it rather than with a literal: a name a person typed goes into
# `User.display_name` (their Google profile name), `DownloadJob.display_name` and
# `DownloadJob.display_uploader`. A value wider than the column is a psycopg DataError, which
# `main.py` re-raises for everything but the NUL case — i.e. a bare 500 for a name somebody pasted.
DISPLAY_TEXT_MAX = 255
def display_text(raw: str | None) -> str | None:
"""A user-supplied display string on its way into one of those columns: stripped, truncated,
blank -> None.
One definition because there are SIX writers and they had drifted: `POST /api/downloads`,
`POST /api/downloads/edit`, `POST /api/downloads/upload`, the metadata PATCH's two fields, and
the Google sign-in callback. Only the PATCH truncated. Truncating is the right answer rather
than refusing: the name is a label on work the user has already done — a file already
transferred, an account already authenticated — and losing its tail is not worth failing the
request over.
`test_upload.py` holds both halves: that every route assigning one of these fields goes through
this helper (parsed, not grepped), and that each of those routes truncates rather than 500s. A
sweep is exactly the kind of thing that gets done five-sixths of the way, which is how this
ended up being written twice.
"""
return (raw or "").strip()[:DISPLAY_TEXT_MAX] or None
from app.utils import canonical_email
@@ -113,7 +142,7 @@ class User(Base, TimestampMixin, CanonicalEmailMixin):
# not cleared eagerly — it simply stops counting).
diagnostics_level: Mapped[str | None] = mapped_column(String(8))
diagnostics_until: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
display_name: Mapped[str | None] = mapped_column(String(255))
display_name: Mapped[str | None] = mapped_column(String(DISPLAY_TEXT_MAX))
avatar_url: Mapped[str | None] = mapped_column(String(1024))
role: Mapped[str] = mapped_column(String(16), default="user", server_default="user")
# The user's own YouTube channel id, resolved lazily (channels.list?mine=true) and
@@ -565,7 +594,15 @@ class LogEvent(Base):
the retention job (`log_retention_days`) bounds whatever another process writes late."""
__tablename__ = "log_events"
__table_args__ = (Index("ix_log_events_user_ts", "user_id", "ts"),)
__table_args__ = (
Index("ix_log_events_user_ts", "user_id", "ts"),
# E16: an issue's bundle is `WHERE debug_session_id = …`; nearly every row has none.
Index(
"ix_log_events_debug_session",
"debug_session_id",
postgresql_where="debug_session_id IS NOT NULL",
),
)
id: Mapped[int] = mapped_column(BigInteger, primary_key=True)
ts: Mapped[datetime] = mapped_column(DateTime(timezone=True), index=True)
@@ -584,6 +621,52 @@ class LogEvent(Base):
version: Mapped[str | None] = mapped_column(String(64))
class DebugSession(Base):
"""One recording: Start → reproduce → Stop (E16, app/debugsession.py). While it is open, every
request of its user that names it (`X-Debug-Session`) is logged at `level` and tagged with `id`,
and the issue it ends in is built from the rows carrying that tag. `until` is the hard cap the
server set at Start; an open session past it is over whether or not the sweep has closed it yet.
One OPEN session per user (`uq_debug_sessions_open_per_user`, partial on `ended_at IS NULL`)."""
__tablename__ = "debug_sessions"
__table_args__ = (
Index(
"uq_debug_sessions_open_per_user",
"user_id",
unique=True,
postgresql_where="ended_at IS NULL",
),
)
id: Mapped[str] = mapped_column(String(16), primary_key=True)
user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE"), index=True)
level: Mapped[str] = mapped_column(String(8))
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
until: Mapped[datetime] = mapped_column(DateTime(timezone=True))
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
# app/debugsession.py `EndReason`: stopped by the user, or past `until`.
ended_reason: Mapped[str | None] = mapped_column(String(16))
issue_url: Mapped[str | None] = mapped_column(String(512))
# The issue being written from it (E16 S3; app/debugsession.py `save_draft` holds its shape).
draft: Mapped[dict | None] = mapped_column(JSONB)
class DebugScreenshot(Base):
"""A screenshot attached to a debug session (E16) — pasted, dropped, or captured from the pill.
Stored in the row: bounded per session (app/debugsession.py), and gone with it by cascade."""
__tablename__ = "debug_screenshots"
id: Mapped[int] = mapped_column(primary_key=True)
session_id: Mapped[str] = mapped_column(
ForeignKey("debug_sessions.id", ondelete="CASCADE"), index=True
)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True))
mime: Mapped[str] = mapped_column(String(16))
size: Mapped[int] = mapped_column(Integer)
data: Mapped[bytes] = mapped_column(LargeBinary)
class AppState(Base):
"""Single-row global app state (admin-controlled)."""
@@ -898,11 +981,11 @@ class DownloadJob(Base, TimestampMixin, UpdatedAtMixin):
ForeignKey("download_profiles.id", ondelete="SET NULL")
)
profile_snapshot: Mapped[dict] = mapped_column(JSON)
display_name: Mapped[str | None] = mapped_column(String(255))
display_name: Mapped[str | None] = mapped_column(String(DISPLAY_TEXT_MAX))
# Per-user display overrides + extras (like display_name — metadata only, never the file).
# `display_uploader`/`display_uploader_url` override the asset's auto channel name/link;
# `extra_links` is an optional list of extra reference URLs shown on the card + watch page.
display_uploader: Mapped[str | None] = mapped_column(String(255))
display_uploader: Mapped[str | None] = mapped_column(String(DISPLAY_TEXT_MAX))
display_uploader_url: Mapped[str | None] = mapped_column(Text)
extra_links: Mapped[list | None] = mapped_column(JSON)
status: Mapped[str] = mapped_column(
+78 -2
View File
@@ -6,13 +6,15 @@ and "fetch a row this user owns or 404" idioms have one definition, one status c
one existence-leak policy.
"""
from typing import Any, TypeVar
from typing import IO, Any, TypeVar
from fastapi import HTTPException
from fastapi import HTTPException, Request
from sqlalchemy.orm import Session
from app.db import Base
from app.models import User
from app.issues import gitea
from app.refusal import refused
from app.youtube.client import YouTubeError, refusal_reason
M = TypeVar("M", bound=Base)
@@ -63,3 +65,77 @@ def youtube_refusal(exc: Exception) -> HTTPException:
YouTube being unavailable to us — the retryable answer."""
reason = refusal_reason(exc) if isinstance(exc, YouTubeError) else "unavailable"
return HTTPException(status_code=422, detail={"code": "youtube", "reason": reason})
async def capped_body(request: Request, limit: int) -> bytes:
"""The request body, refused (413 `too_large`) the moment it passes `limit` bytes while still
arriving. Counted, not taken from Content-Length: a declared length can be absent (chunked) or
wrong, and counting stops reading at the same point a declared-length check would have."""
body = bytearray()
async for chunk in request.stream():
body += chunk
if len(body) > limit:
raise HTTPException(status_code=413, detail={"code": "too_large"})
return bytes(body)
async def capped_stream_to(
request: Request, out: IO[bytes], limit: int, refusal: HTTPException
) -> int:
"""`capped_body`'s sibling for a body too large to hold: the same counting rule, written
straight to `out` instead of accumulated, returning the size.
Two deliberate differences from `capped_body`. It never builds a `bytearray` — the upload route
admits 500 MiB by default, so holding the body would trade a disk problem for a memory one. And
the refusal is the CALLER'S, passed in rather than assumed: `capped_body`'s flat 413 is right
for a screenshot, while the upload route answers 422 `upload_too_large` carrying the admin
knob's own number, which is what its dialog shows the user. One counting rule, two answers,
neither of them a second copy of the loop.
The caller owns `out` and owns cleaning it up: a body refused half-way has already written what
arrived before the limit was passed.
**The refusal DRAINS the rest of the body before it is raised, and that is not politeness.**
Answering mid-upload and closing the connection is what the client sees as a network failure,
not as a refusal: the browser's XHR fires `onerror` before it can read the response, so a file
over the limit reported itself as "the server couldn't be reached" instead of naming the limit —
measured in a real browser, not reasoned about. Reading the remainder to nowhere lets the client
finish sending, read the 422 and say the true thing.
The bytes are discarded, never written, and the whole request is already bounded by
`bodylimit`'s per-route ceiling — so the worst case is bandwidth an authenticated caller was
sending anyway, against the old behaviour which wrote exactly those bytes to disk."""
size = 0
over = False
async for chunk in request.stream():
size += len(chunk)
if over:
continue # draining — see the docstring; counted, not stored
if size > limit:
over = True
continue
out.write(chunk)
if over:
raise refusal
return size
def issue_refused(exc: gitea.GiteaError) -> HTTPException:
"""The refusal an issue-tracker failure (E16) answers as — the send, the preview and the
admin's connection test alike. Literal reasons, one per case: the frontend's contract test reads
them to require a sentence for each in both languages."""
match exc.reason:
case "not_configured":
return refused(400, "issues_not_configured")
case "unreachable":
return refused(422, "issues_unreachable")
case "bad_token":
return refused(422, "issues_bad_token")
case "repo_not_found":
return refused(422, "issues_repo_not_found")
case "public_repo":
return refused(422, "issues_public_repo")
case "issues_disabled":
return refused(422, "issues_disabled")
case _:
return refused(422, "issues_rejected")
+3 -21
View File
@@ -26,10 +26,11 @@ from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy import Text, and_, cast, func, or_, select
from sqlalchemy.orm import Session
from app import logsink
from app.auth import admin_user
from app.db import get_db
from app.models import LogEvent, User
from app.utils import iso, like_escape
from app.utils import like_escape
router = APIRouter(prefix="/api/admin/logs", tags=["admin"])
@@ -66,25 +67,6 @@ def _decode_cursor(cursor: str) -> tuple[datetime, int]:
raise HTTPException(status_code=400, detail={"code": "bad_cursor"}) from exc
def _serialize(row: LogEvent) -> dict:
return {
"id": row.id,
"ts": iso(row.ts),
"level": row.level,
"source": row.source,
"logger": row.logger,
"event": row.event,
"msg": row.msg,
"attrs": row.attrs,
"exc": row.exc,
"request_id": row.request_id,
"user_id": row.user_id,
"job_id": row.job_id,
"debug_session_id": row.debug_session_id,
"version": row.version,
}
def _accounts(db: Session, user_ids: set[int]) -> dict[str, str]:
"""The accounts a page names, by id → address. An id with no account left (erased after the
line was written) is simply absent, and the client shows the bare id."""
@@ -196,7 +178,7 @@ def list_logs(
more = len(rows) > limit
rows = rows[:limit]
return {
"items": [_serialize(r) for r in rows],
"items": [logsink.serialize(r) for r in rows],
"accounts": _accounts(db, {r.user_id for r in rows if r.user_id is not None}),
"next_cursor": _encode_cursor(rows[-1]) if more else None,
}
+3 -3
View File
@@ -39,7 +39,7 @@ from app.auth import (
)
from app.config import settings
from app.db import SessionLocal, get_db
from app.models import AuthToken, DemoWhitelist, Invite, OAuthToken, User
from app.models import AuthToken, DemoWhitelist, Invite, OAuthToken, User, display_text
from app.ratelimit import RateLimiter, client_key
from app.security import encrypt, hash_password, hash_token, verify_password
from app.utils import canonical_email, valid_email
@@ -472,7 +472,7 @@ async def callback(
if email:
user.google_email = email
_sync_google_email(db, user, email)
user.display_name = userinfo.get("name")
user.display_name = display_text(userinfo.get("name"))
user.avatar_url = userinfo.get("picture")
# ADMIN_EMAILS (env) is the bootstrap admin list and always wins, so the configured admin can't
# be locked out. Everyone else's role is managed in the admin UI and stored in the DB, so a
@@ -594,7 +594,7 @@ def _complete_link(
if linked_email:
user.google_email = linked_email # see the sign-in callback: the other half of the identity
if not user.display_name:
user.display_name = userinfo.get("name")
user.display_name = display_text(userinfo.get("name"))
if not user.avatar_url:
user.avatar_url = userinfo.get("picture")
_store_token(db, user, token)
+15 -1
View File
@@ -18,7 +18,8 @@ from app.auth import admin_user
from app.db import get_db
from app.models import User
from app.refusal import refused
from app.routes._common import not_found
from app.issues import gitea
from app.routes._common import issue_refused, not_found
from app.refusal import setting_refused
router = APIRouter(prefix="/api/admin/config", tags=["admin"])
@@ -131,3 +132,16 @@ def send_test_email(
if not sent:
raise refused(422, "smtp_failed")
return {"ok": True, "sent": True, "to": user.email}
@router.post("/test-issues")
def test_issue_tracker(
_: User = Depends(admin_user),
db: Session = Depends(get_db),
) -> dict:
"""Check the configured issue tracker (E16): the repository answers with this token, takes
issues, and is PRIVATE — the one it must be, since an issue carries its reporter's log."""
try:
return {"ok": True, **gitea.check(db)}
except gitea.GiteaError as exc:
raise issue_refused(exc) from None
+418
View File
@@ -0,0 +1,418 @@
"""Debug sessions (E16): Start → reproduce → Stop, the screenshots, and the evidence under review.
The session itself — what tags a request, its cap, its level — is `app/debugsession.py`. These routes
are the user's handle on their OWN sessions only: another user's id 404s (`owned_or_404`), including
for an admin, who reads everyone's lines in the diagnostics console instead. The shared demo account
cannot record: its log would interleave every visitor under one account.
Screenshots arrive as a raw image body (`image/png`, `image/jpeg`, `image/webp`), not a multipart
form — one image per request, capped while it arrives, and kept only if its bytes really are one of
those types, whatever the Content-Type claimed.
"""
import json
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Request, Response
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from app import audit, debugsession, logs, logsink
from app.audit import AuditAction
from app.auth import require_human
from app.db import get_db, release_read_connection
from app.issues import bundle, gitea
from app.ratelimit import RateLimiter
from app.refusal import refused
from app.models import DebugScreenshot, DebugSession, User
from app.routes._common import capped_body, issue_refused, not_found, owned_or_404
from app.utils import iso
router = APIRouter(prefix="/api/debug-sessions", tags=["diagnostics"])
log = logs.get_logger(__name__)
# Sessions listed in Settings → Diagnosztika, newest first: enough to reach last week's unsent draft.
LIST_MAX = 20
class StartIn(BaseModel):
level: debugsession.SessionLevel
_view = debugsession.view
# New sessions a user may open per hour (joining the running one is not counted): a session is a
# place to keep screenshots, and this — with the per-user storage cap — bounds how many there are.
_new_sessions = RateLimiter(max_events=20, window_seconds=3600)
@router.post("")
def start_session(
body: StartIn,
user: User = Depends(require_human),
db: Session = Depends(get_db),
) -> dict:
"""Start recording, or join the recording already running (another tab started it)."""
joining = debugsession.open_session(db, user) is not None
if not joining and not _new_sessions.allow(str(user.id)):
raise refused(429, "debug_session_rate_limited", limit=_new_sessions.max_events)
try:
session = debugsession.start(db, user, body.level)
db.commit()
except IntegrityError:
# A concurrent Start won the one-open-per-user index: join that one.
db.rollback()
session = debugsession.open_session(db, user)
if session is None:
raise refused(409, "debug_session_conflict") from None
# The rest of THIS request is part of the session too.
level, until = debugsession.level_for(user, session)
logs.set_user(user.id, level, until, session.id)
log.info(
"debug session started",
extra=logs.event("debug.session.started", level=session.level, until=iso(session.until)),
)
return {"ok": True, "session": _view(db, session)}
@router.post("/{sid}/stop")
def stop_session(
sid: str, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
session = owned_or_404(db, DebugSession, sid, user, what="debug_session")
if session.ended_at is None:
# Logged BEFORE the session closes, while this request still carries its tag: the last line
# of the evidence is the Stop itself.
log.info("debug session stopped", extra=logs.event("debug.session.stopped"))
debugsession.stop(session)
db.commit()
logs.end_session_elevation(session.id)
return {"ok": True, "session": _view(db, session)}
@router.get("")
def list_sessions(user: User = Depends(require_human), db: Session = Depends(get_db)) -> dict:
sessions = db.scalars(
select(DebugSession)
.where(DebugSession.user_id == user.id)
.order_by(DebugSession.started_at.desc())
.limit(LIST_MAX)
).all()
return {"items": [_view(db, s) for s in sessions]}
@router.get("/{sid}")
def get_session(
sid: str, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
return _view(db, owned_or_404(db, DebugSession, sid, user, what="debug_session"))
@router.get("/{sid}/events")
def session_events(
sid: str, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
"""The rows an issue from this session would carry — browser and server, in time order."""
session = owned_or_404(db, DebugSession, sid, user, what="debug_session")
rows, total = debugsession.bundle(db, session)
return {"items": [logsink.serialize(r) for r in rows], "total": total}
# --- the draft and its export ---------------------------------------------------------------
# The composer's own fields (steps, description…) are its business, kept whole so it reopens as it
# was — bounded here so a draft cannot become a place to store anything else.
FORM_MAX_BYTES = 64 * 1024
# Zips a user may build per minute: each holds every screenshot in memory while it is built.
_exports = RateLimiter(max_events=6, window_seconds=60)
class DraftIn(BaseModel):
title: str = Field(max_length=200)
# The rendered markdown the export and the issue carry.
body: str = Field(max_length=50_000)
# Log rows taken out of the evidence in the review.
excluded: list[int] = Field(default_factory=list, max_length=debugsession.BUNDLE_MAX)
form: dict[str, Any] = Field(default_factory=dict)
@router.put("/{sid}/draft")
def save_draft(
sid: str,
body: DraftIn,
user: User = Depends(require_human),
db: Session = Depends(get_db),
) -> dict:
session = owned_or_404(db, DebugSession, sid, user, what="debug_session")
if len(json.dumps(body.form, ensure_ascii=False).encode()) > FORM_MAX_BYTES:
raise HTTPException(status_code=413, detail={"code": "too_large"})
# Under the lock, and refused while a send is under way: the issue is being built from the draft
# as it was saved, and this would change it under the send.
debugsession.lock(db, session)
if debugsession.sending(session):
raise refused(409, "issue_sending")
if session.issue_url is not None:
# Sent: the issue is on the tracker now, and a draft edited after it would say otherwise.
raise refused(409, "issue_already_sent")
debugsession.save_draft(session, body.model_dump())
db.commit()
return {"ok": True, "session": _view(db, session)}
@router.get("/{sid}/export")
def export_session(
sid: str, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> Response:
"""The issue as a zip — the text, the two logs and the screenshots — for an instance without an
issue tracker, or for sending it by hand. Built from the SAVED draft: what the composer shows is
what it saved."""
session = owned_or_404(db, DebugSession, sid, user, what="debug_session")
if not _exports.allow(str(user.id)):
raise refused(429, "export_rate_limited")
data = bundle.zipped(bundle.files(db, session))
name = f"siftlode-debug-{session.started_at:%Y%m%d-%H%M}-{session.id[:6]}.zip"
return Response(
content=data,
media_type="application/zip",
headers={
"Content-Disposition": f'attachment; filename="{name}"',
"Cache-Control": "no-store",
},
)
# --- the issue tracker (S4) -----------------------------------------------------------------
# Issues a user may open per hour — counted once the tracker is ASKED to open one (an answer that
# goes missing may still have created it, so an attempt cannot be given back); finishing a half-sent
# one, or an attempt refused before the tracker was asked, is not counted. The composer sends one at
# a time; this is room for a bad afternoon, not a pace anything legitimate keeps up.
_new_issues = RateLimiter(max_events=5, window_seconds=3600)
# Attempts on the tracker per hour, creating or finishing. Deliberately NOT the budget above: a
# half-sent issue must always be finishable, so the pace that guards creating one must never be what
# stands between a user and their own report. This one only bounds a loop — every attempt re-reads
# the whole bundle into memory and re-offers the pending file to the tracker, and a retry that is
# refused for good (the tracker's own limits) never leaves `pending`. No hand reaches it.
_sends = RateLimiter(max_events=60, window_seconds=3600)
# The composer's live preview, rendered BY the tracker: one authenticated call each, with our token.
# The composer's toggle fires it once per switch INTO preview — the editor is the other branch, so
# no keystroke can change the query key while the preview is up — and react-query's cache makes an
# unchanged draft free. (It does NOT debounce, whatever this comment used to say: a key that changes
# per keystroke is one fetch per keystroke, measured. Showing the preview BESIDE the fields would
# need a real debounce first.) This bounds what a loop can aim at the tracker.
_previews = RateLimiter(max_events=60, window_seconds=60)
def _issues_rate_limited() -> HTTPException:
return refused(429, "issues_rate_limited", limit=_new_issues.max_events)
@router.post("/{sid}/issue")
def send_issue(
sid: str, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
"""Send the session's issue to the tracker — from the SAVED draft, as the export is built — or
finish a send that stopped half-way (the issue exists; only what is missing is sent)."""
session = owned_or_404(db, DebugSession, sid, user, what="debug_session")
# The session is CLAIMED for the send rather than locked through it: the tracker is slow (a
# dozen calls, 20 s each), and a row lock held across them would queue this user's Stop, their
# autosave and their screenshots behind it. They see the claim instead and are refused at once.
debugsession.lock(db, session)
if debugsession.sending(session):
raise refused(409, "issue_sending")
new = session.issue_url is None
if not new and not debugsession.unfinished(session):
return {"ok": True, "session": _view(db, session)}
key = str(user.id)
if new:
if not ((session.draft or {}).get("title") or "").strip():
raise refused(422, "issue_title_required")
if _new_issues.exhausted(key):
raise _issues_rate_limited()
if not _sends.allow(key):
raise refused(429, "issue_rate_limited", limit=_sends.max_events)
token = debugsession.claim_send(session)
db.commit() # the claim is what everything else sees; the lock ends here
def count_new_issue() -> None:
if not _new_issues.allow(key):
raise _issues_rate_limited()
try:
gitea.deliver(
db,
session,
token=token,
checkpoint=db.commit,
before_create=count_new_issue,
on_created=lambda: _record_created(db, user, session),
)
except debugsession.SendClaimLost:
# This send outlived its lease and another one took the session over while it was still
# talking to the tracker. It stopped rather than writing over that one's progress — and a
# send IS under way, just not ours, which is what the caller needs to hear.
raise refused(409, "issue_sending") from None
except gitea.GiteaError as exc:
# Keep what did happen: an issue created before an upload failed stays linked, and the
# retry does only what is still missing.
raise issue_refused(exc) from None
except HTTPException:
raise # the pace, refused before the tracker was asked
finally:
_end_send(db, session, token)
return {"ok": True, "session": _view(db, session)}
def _end_send(db: Session, session: DebugSession, token: str) -> None:
"""Give the claim back, whatever else happened. A failure here must not replace the failure that
brought us: a dead transaction (the database went away mid-send) is rolled back, and the claim
then simply ages out."""
try:
debugsession.lock(db, session)
debugsession.release_send(session, token)
db.commit()
except Exception: # noqa: BLE001 — whatever went wrong here, the send's own failure is the news
db.rollback()
log.warning(
"a send could not give its claim back",
extra=logs.event("issues.claim.stuck", session_id=session.id),
)
def _record_created(db: Session, user: User, session: DebugSession) -> None:
"""The trail for the one act that takes a user's log off this instance. Staged inside the send,
so the checkpoint that first writes `issue_url` commits BOTH — recorded later it could be lost
on its own (the process dies mid-send), and no retry could write it afterwards: the next send
sees the link and knows the issue is not new."""
audit.record(
db,
user.id,
AuditAction.ISSUE_CREATE,
target_type="debug_session",
target_id=session.id,
summary=session.issue_url or "",
)
class PreviewIn(BaseModel):
text: str = Field(max_length=50_000)
@router.post("/{sid}/preview")
def preview_issue(
sid: str,
body: PreviewIn,
user: User = Depends(require_human),
db: Session = Depends(get_db),
) -> dict:
"""The body as the tracker will render it. Gitea sanitises its own output; the composer still
shows it in a sandboxed frame, never in the app's own document."""
owned_or_404(db, DebugSession, sid, user, what="debug_session")
if not _previews.allow(str(user.id)):
raise refused(429, "preview_rate_limited")
try:
return {"ok": True, "html": gitea.preview(db, body.text)}
except gitea.GiteaError as exc:
raise issue_refused(exc) from None
# --- screenshots ---------------------------------------------------------------------------
def _shot_view(shot: DebugScreenshot) -> dict:
return {"id": shot.id, "mime": shot.mime, "size": shot.size, "created_at": iso(shot.created_at)}
@router.get("/{sid}/screenshots")
def list_screenshots(
sid: str, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
session = owned_or_404(db, DebugSession, sid, user, what="debug_session")
return {"items": [_shot_view(s) for s in debugsession.screenshots(db, session)]}
def _upload_target(
sid: str, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> DebugSession:
return owned_or_404(db, DebugSession, sid, user, what="debug_session")
async def _image_body(request: Request, db: Session = Depends(get_db)) -> bytes:
# The owner check above has already run, and it was a SELECT — so this session is holding a
# pooled connection `idle in transaction` and would hold it for the whole 10 MiB upload, from a
# client that may be on a phone. Hand it back first; the handler's own work re-takes one. The
# `session` object the handler holds is expired by this and reloads on its next attribute read,
# which is the correct moment to touch the database again.
release_read_connection(db)
return await capped_body(request, debugsession.SCREENSHOT_MAX_BYTES)
@router.post("/{sid}/screenshots")
def add_screenshot(
# The owner check FIRST: dependencies resolve in order, so a stranger's upload is refused
# before a byte of it is read.
session: DebugSession = Depends(_upload_target),
data: bytes = Depends(_image_body),
db: Session = Depends(get_db),
) -> dict:
try:
shot = debugsession.add_screenshot(db, session, data)
except debugsession.ScreenshotRefused as exc:
raise _screenshot_refused(exc.code) from None
db.commit()
return {"ok": True, "screenshot": _shot_view(shot)}
def _screenshot_refused(code: str) -> HTTPException:
# Literal reasons, one per case: the frontend's contract test reads them to require a sentence
# for each in both languages.
if code == "already_sent":
return refused(409, "screenshot_already_sent")
if code == "too_many":
return refused(422, "screenshot_too_many", limit=debugsession.SCREENSHOTS_MAX)
if code == "sending":
return refused(409, "issue_sending")
if code == "storage_full":
mb = debugsession.SCREENSHOT_USER_MAX_BYTES // (1024 * 1024)
return refused(422, "screenshot_storage_full", limit=mb)
if code == "empty":
return refused(422, "screenshot_empty")
return refused(422, "screenshot_not_an_image")
def _session_screenshot(db: Session, sid: str, shot_id: int, user: User) -> DebugScreenshot:
return _screenshot_of(
db, owned_or_404(db, DebugSession, sid, user, what="debug_session"), shot_id
)
def _screenshot_of(db: Session, session: DebugSession, shot_id: int) -> DebugScreenshot:
shot = db.get(DebugScreenshot, shot_id)
if shot is None or shot.session_id != session.id:
raise not_found("screenshot")
return shot
@router.get("/{sid}/screenshots/{shot_id}")
def get_screenshot(
sid: str, shot_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> Response:
shot = _session_screenshot(db, sid, shot_id, user)
# `nosniff`: the type was checked against the bytes at upload, and the browser must not guess.
return Response(
content=shot.data,
media_type=shot.mime,
headers={"Cache-Control": "private, max-age=3600", "X-Content-Type-Options": "nosniff"},
)
@router.delete("/{sid}/screenshots/{shot_id}")
def delete_screenshot(
sid: str, shot_id: int, user: User = Depends(require_human), db: Session = Depends(get_db)
) -> dict:
session = owned_or_404(db, DebugSession, sid, user, what="debug_session")
shot = _screenshot_of(db, session, shot_id)
try:
debugsession.remove_screenshot(db, session, shot)
except debugsession.ScreenshotRefused as exc:
raise _screenshot_refused(exc.code) from None
db.commit()
return {"ok": True, "deleted": shot_id}
+32 -21
View File
@@ -13,7 +13,9 @@ Nothing from the body is trusted as-is:
* everything is redacted AGAIN with the server's own rules — the client redacts at capture, but a
modified client would not;
* an event below the consented level is dropped (a Részletes consent does not store what only
Teljes may carry), and so is the whole batch once the level has lapsed;
Teljes may carry), and so is the whole batch once the level has lapsed — where an open debug
session the request names counts as consent to ITS level (E16), and only its own events keep
its tag;
* the timestamp is clamped to the consent window, so a row cannot dodge the retention job by
claiming to come from the future.
"""
@@ -27,11 +29,12 @@ from pydantic import BaseModel, Field, ValidationError
from sqlalchemy import insert
from sqlalchemy.orm import Session
from app import diagnostics, logs, logsink
from app import debugsession, diagnostics, logs, logsink
from app.auth import current_user
from app.db import get_db
from app.models import LogEvent, User
from app.db import get_db, release_read_connection
from app.models import DebugSession, LogEvent, User
from app.ratelimit import RateLimiter
from app.routes._common import capped_body
router = APIRouter(prefix="/api/diagnostics", tags=["diagnostics"])
@@ -60,6 +63,9 @@ class ClientEvent(BaseModel):
attrs: dict[str, Any] | None = None
request_id: str | None = Field(default=None, pattern=r"^[A-Za-z0-9._-]{8,64}$")
tab_id: str | None = Field(default=None, pattern=r"^[A-Za-z0-9_-]{1,32}$")
# The debug session the event was captured in (E16); kept only when it is the one this request
# names in `X-Debug-Session` and the auth dependency found it the caller's own and open.
debug_session_id: str | None = Field(default=None, pattern=r"^[0-9a-f]{16}$")
version: str | None = Field(default=None, max_length=64)
@@ -67,19 +73,7 @@ class ClientEventsIn(BaseModel):
events: list[ClientEvent] = Field(max_length=EVENTS_MAX)
async def _capped_body(request: Request) -> bytes:
"""The request body, refused the moment it passes BODY_MAX_BYTES while still arriving. Counted,
not taken from Content-Length: a declared length can be absent (chunked) or wrong, and counting
stops reading at the same point a declared-length check would have."""
body = bytearray()
async for chunk in request.stream():
body += chunk
if len(body) > BODY_MAX_BYTES:
raise HTTPException(status_code=413, detail={"code": "too_large"})
return bytes(body)
async def _parsed_events(request: Request) -> ClientEventsIn:
async def _parsed_events(request: Request, db: Session = Depends(get_db)) -> ClientEventsIn:
"""The body as the model — parsed only after the size cap, which is why this is a dependency and
not a plain body parameter (FastAPI reads a declared body in full before anything else runs).
@@ -89,14 +83,24 @@ async def _parsed_events(request: Request) -> ClientEventsIn:
media_type = request.headers.get("content-type", "").split(";")[0].strip().lower()
if media_type != "application/json" and not media_type.endswith("+json"):
raise HTTPException(status_code=415, detail={"code": "json_required"})
raw = await _capped_body(request)
# `current_user` resolved above on a SELECT, so this session is holding a pooled connection.
# Give it back before awaiting the body — the sink is a background beacon and its sender may be
# on a link far slower than the 512 KiB suggests. See `db.release_read_connection`.
release_read_connection(db)
raw = await capped_body(request, BODY_MAX_BYTES)
try:
return ClientEventsIn.model_validate_json(raw)
except ValidationError as exc:
raise HTTPException(status_code=422, detail=json.loads(exc.json())) from exc
def _row(item: ClientEvent, user: User, now: datetime, window_start: datetime) -> dict[str, Any]:
def _row(
item: ClientEvent,
user: User,
now: datetime,
window_start: datetime,
session: DebugSession | None,
) -> dict[str, Any]:
attrs = logs.redact(item.attrs) if item.attrs else None
if attrs is not None:
size = len(json.dumps(attrs, ensure_ascii=False, default=str).encode())
@@ -117,6 +121,9 @@ def _row(item: ClientEvent, user: User, now: datetime, window_start: datetime) -
"attrs": attrs,
"request_id": item.request_id,
"user_id": user.id,
"debug_session_id": (
session.id if session and item.debug_session_id == session.id else None
),
"version": item.version,
}
)
@@ -126,11 +133,15 @@ def _row(item: ClientEvent, user: User, now: datetime, window_start: datetime) -
def ingest_events(
# The account first: dependencies resolve in order, so an anonymous caller is refused before a
# byte of its body is read.
request: Request,
user: User = Depends(current_user),
body: ClientEventsIn = Depends(_parsed_events),
db: Session = Depends(get_db),
) -> dict:
level = diagnostics.effective_level(user)
# A debug session the request names raises the level for its own events (E16): starting one is
# the consent to that level for its duration, whatever the user's own level is.
session = debugsession.from_header(db, user, request.headers.get(debugsession.HEADER))
level, _ = debugsession.level_for(user, session)
if level == diagnostics.BASE:
# Lapsed or withdrawn: the client stops streaming when it sees this.
raise HTTPException(status_code=409, detail={"code": "diagnostics_off"})
@@ -140,7 +151,7 @@ def ingest_events(
window_start = now - timedelta(hours=diagnostics.ELEVATION_HOURS)
floor = logs.SEVERITY[level]
rows = [
_row(item, user, now, window_start)
_row(item, user, now, window_start, session)
for item in body.events
if logs.SEVERITY[item.level] >= floor
]
+121 -47
View File
@@ -9,11 +9,13 @@ serves finished files (range-aware, with the user's custom display name), and sh
import os
import re
import tempfile
from collections.abc import AsyncIterator
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path
from urllib.parse import parse_qs, urlparse
from urllib.parse import parse_qs, unquote, urlparse
from fastapi import APIRouter, Depends, File, Form, HTTPException, Request, UploadFile
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import FileResponse
from pydantic import BaseModel
from sqlalchemy import func, select
@@ -22,7 +24,7 @@ from sqlalchemy.orm import Session
from app import audit, sysconfig
from app.auth import admin_user, require_human
from app.config import settings
from app.db import get_db
from app.db import get_db, release_read_connection
from app.downloads import credentials as dlcreds
from app.downloads import edit as editmod
from app.downloads import links as linksmod
@@ -37,8 +39,9 @@ from app.models import (
DownloadShare,
MediaAsset,
User,
display_text,
)
from app.routes._common import get_or_404, not_found, owned_or_404
from app.routes._common import capped_stream_to, get_or_404, not_found, owned_or_404
from app.security import hash_password
from app.userscope import is_messageable_user, messageable_clauses
from app.utils import iso
@@ -398,7 +401,7 @@ def enqueue_download(
status_code=422, detail={"code": "url_blocked", "reason": e.reason}
) from e
spec, profile_id = _resolve_spec(db, user, body.profile_id, body.spec)
display_name = (body.display_name or "").strip() or None
display_name = display_text(body.display_name)
try:
job = service.enqueue(db, user.id, source_kind, source_ref, spec, profile_id, display_name)
except quota.QuotaExceeded as e:
@@ -406,61 +409,119 @@ def enqueue_download(
return _serialize_job(db, job)
# Read the upload in 1 MiB chunks so a large file never sits fully in memory, and the size cap is
# enforced on the ACTUAL bytes read (a client-set Content-Length can't be trusted to gate it).
_UPLOAD_CHUNK = 1024 * 1024
# The upload's filename and display name travel as HEADERS, not as multipart parts. That is not a
# style choice: a DECLARED body parameter (`file: UploadFile = File(...)`) is read and spooled by
# FastAPI *before the first dependency runs*, so `require_human` could not gate it and an anonymous
# caller could make the process write the whole per-file allowance to temp, per request (P1; SEC-20
# bounded the number but could not stop the writing). Reading the body ourselves, from a dependency
# declared AFTER the auth one, is the only shape that closes it — the same one
# `POST /api/debug-sessions/{sid}/screenshots` uses, and `test_body_limit.py` now enforces it for
# every route with a raised ceiling rather than leaving it to a comment like this one.
#
# Headers are latin-1 on the wire, so both are percent-encoded by the client and unquoted here; a
# Hungarian filename would otherwise be unsendable.
UPLOAD_FILENAME_HEADER = "x-upload-filename"
UPLOAD_DISPLAY_NAME_HEADER = "x-upload-display-name"
@router.post("/upload")
def upload_download(
file: UploadFile = File(...),
display_name: str | None = Form(None),
@dataclass(frozen=True)
class SpooledUpload:
"""What the body dependency hands the handler: the bytes already on disk, plus the two names
that came alongside them."""
path: Path
size: int
user_id: int
filename: str
display_name: str | None
def _header_text(request: Request, name: str) -> str:
"""One percent-encoded header, decoded. A header that is not valid UTF-8 once unquoted is the
caller's doing, not a server fault — it degrades to empty rather than 500ing, and the route's
own fallback names the file."""
raw = request.headers.get(name, "")
try:
return unquote(raw, errors="strict").strip()
except (UnicodeDecodeError, ValueError):
# silent-ok: a mis-encoded header is the caller's own malformed request, and this one is
# cosmetic — it names the file. Refusing the upload over it would fail a 500 MiB transfer on
# a label; logging it would hand any client a free log line per request.
return ""
async def spool_upload(
request: Request,
user: User = Depends(require_human),
db: Session = Depends(get_db),
) -> dict:
"""Ingest a local file into the caller's library (E7 S5). Any file type is accepted; media
(video/audio) gets the full function set, a generic file is download-only. The file is streamed
to a temp path under the download volume (enforcing the per-file cap as it goes), then handed to
the non-yt-dlp ingest which probes + places it and returns an already-`done` job.
) -> AsyncIterator[SpooledUpload]:
"""Stream the request body to a staging temp file, enforcing the per-file cap as it arrives.
A sync (threadpool) handler on purpose: the ingest does blocking work — an ffprobe of up to 30s
plus sync DB writes — so it must NOT run on the event loop. `file.file` is the sync file object
Starlette spools the multipart body into."""
`user` is taken but unused: it is the whole point. Dependencies resolve in declaration order, so
naming `require_human` here makes the auth check a precondition of reading the body at all — a
stranger is refused before a byte moves. The `yield` shape owns the temp file's life: ingest
MOVES it into place on success (the unlink is then a no-op), and on any other exit — cap hit,
empty, quota, probe or move error, a client that vanished mid-upload — it is removed."""
cap = sysconfig.get_int(db, "download_upload_max_bytes")
user_id = user.id
# Hand the pooled connection back BEFORE the transfer starts — see `db.release_read_connection`
# for why, and for the precondition it enforces. Everything this dependency still needs is
# already an int or a header in hand: touching an ORM attribute below would reload `user` and
# check a connection straight back out.
release_read_connection(db)
staging = Path(settings.download_root) / ".staging" / "uploads"
staging.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=str(staging))
tmp_path = Path(tmp)
try:
size = 0
with os.fdopen(fd, "wb") as out:
while chunk := file.file.read(_UPLOAD_CHUNK):
size += len(chunk)
if cap and size > cap:
raise HTTPException(
status_code=422, detail={"code": "upload_too_large", "limit": cap}
)
out.write(chunk)
if size == 0:
raise HTTPException(status_code=400, detail={"code": "upload_empty"})
try:
job = uploadmod.ingest_upload(
db,
user.id,
filename=file.filename or "upload",
src_path=tmp_path,
size=size,
display_name=(display_name or "").strip() or None,
size = await capped_stream_to(
request,
out,
cap,
HTTPException(status_code=422, detail={"code": "upload_too_large", "limit": cap}),
)
except quota.QuotaExceeded as e:
raise HTTPException(status_code=422, detail=_quota_detail(e))
return _serialize_job(db, job)
yield SpooledUpload(
path=tmp_path,
size=size,
user_id=user_id,
filename=_header_text(request, UPLOAD_FILENAME_HEADER) or "upload",
display_name=display_text(_header_text(request, UPLOAD_DISPLAY_NAME_HEADER)),
)
finally:
# Ingest MOVES the temp file into place on success (unlink is then a no-op); on any early exit
# — cap hit, empty, quota, probe/move error — this removes the abandoned staging file.
tmp_path.unlink(missing_ok=True)
@router.post("/upload")
def upload_download(
upload: SpooledUpload = Depends(spool_upload),
db: Session = Depends(get_db),
) -> dict:
"""Ingest a local file into the caller's library (E7 S5). Any file type is accepted; media
(video/audio) gets the full function set, a generic file is download-only. The body IS the file
(see the header note above); `spool_upload` has already authenticated the caller and streamed it
to a temp path under the download volume, so what is left here is the non-yt-dlp ingest which
probes + places it and returns an already-`done` job.
A sync (threadpool) handler on purpose: the ingest does blocking work — an ffprobe of up to 30s
plus sync DB writes — so it must NOT run on the event loop. The streaming half is async because
reading the body is; keeping them in separate callables is what lets each run where it belongs."""
if upload.size == 0:
raise HTTPException(status_code=400, detail={"code": "upload_empty"})
try:
job = uploadmod.ingest_upload(
db,
upload.user_id,
filename=upload.filename,
src_path=upload.path,
size=upload.size,
display_name=upload.display_name,
)
except quota.QuotaExceeded as e:
raise HTTPException(status_code=422, detail=_quota_detail(e))
return _serialize_job(db, job)
class EnqueueEditIn(BaseModel):
source_job_id: int = 0
edit_spec: dict
@@ -480,7 +541,7 @@ def enqueue_edit(
if source_job.status != "done" or source_job.asset_id is None:
raise refused(409, "edit_unfinished")
spec = body.edit_spec
display_name = (body.display_name or "").strip() or None
display_name = display_text(body.display_name)
try:
job = service.enqueue_edit(db, user.id, source_job, spec, display_name)
except quota.QuotaExceeded as e:
@@ -516,7 +577,20 @@ def list_downloads(
@router.get("/usage")
def my_usage(user: User = Depends(require_human), db: Session = Depends(get_db)) -> dict:
return quota.usage(db, user.id)
"""The caller's download quota, plus the per-file upload cap.
The cap rides here rather than in `quota.usage` because it is not a quota value — it is a
`sysconfig` knob about one route — and this is the route layer, where composing the two is the
job. The upload dialog needs it to refuse an over-cap file BEFORE sending it: both server-side
caps answer too late to be pleasant (the edge ceiling refuses the request outright, see
`bodylimit.ROUTE_MAX_BODY_BYTES`), and making the user push 600 MB up the wire to be told the
limit is 500 is a bad answer even when it is a correct one. It is a convenience, not the
boundary: the caps stay where they are and still refuse a client that ignores this.
"""
return {
**quota.usage(db, user.id),
"upload_max_bytes": sysconfig.get_int(db, "download_upload_max_bytes"),
}
# Priority when a source has several jobs (e.g. a done one plus a fresh queued one).
@@ -611,9 +685,9 @@ def update_download_meta(
job = owned_or_404(db, DownloadJob, job_id, user, what="download")
sent = body.model_fields_set
if "display_name" in sent:
job.display_name = ((body.display_name or "").strip())[:255] or None
job.display_name = display_text(body.display_name)
if "display_uploader" in sent:
job.display_uploader = ((body.display_uploader or "").strip())[:255] or None
job.display_uploader = display_text(body.display_uploader)
if "display_uploader_url" in sent:
job.display_uploader_url = _clean_url(body.display_uploader_url)
if "extra_links" in sent:
+11 -1
View File
@@ -17,7 +17,8 @@ from app.auth import (
optional_current_user,
purge_user,
)
from app import diagnostics
from app import debugsession, diagnostics
from app.issues import gitea
from app import email as email_mod
from app import logs
from app import sysconfig
@@ -158,9 +159,18 @@ def get_me(
# The consented diagnostics level in force and when it lapses (E22 S5). The client applies
# it to its own log and streams that log while it is raised.
"diagnostics": diagnostics.state(db, user),
# The live debug session, if one is recording (E16) — a reload resumes it.
"debug_session": _live_session(db, user),
# Where a bug report goes (E16 S4): the tracker's repository, or None — download only.
"issues": gitea.target(db),
}
def _live_session(db: Session, user: User) -> dict | None:
session = debugsession.open_session(db, user)
return None if session is None else debugsession.view(db, session)
@router.delete("/account")
def delete_my_account(
request: Request,
+7 -3
View File
@@ -9,7 +9,7 @@ from typing import Callable
from apscheduler.schedulers.background import BackgroundScheduler
from sqlalchemy import select
from app import audit, logs, logsink, progress, quota
from app import audit, debugsession, logs, logsink, progress, quota
from app.audit import AuditAction
from app.config import settings
from app.db import SessionLocal
@@ -386,9 +386,13 @@ def _log_gc_job() -> None:
def work(db):
from app import sysconfig
reaped = logsink.gc(db, sysconfig.get_int(db, "log_retention_days"))
days = sysconfig.get_int(db, "log_retention_days")
reaped = logsink.gc(db, days)
# Debug sessions (E16) go with the rows they bundle; an open one past its cap is closed.
closed = debugsession.sweep(db)
sessions = debugsession.gc(db, days)
db.commit()
return {"reaped": reaped}
return {"reaped": reaped, "sessions_closed": closed, "sessions_reaped": sessions}
_job("log_gc", work)
+58 -10
View File
@@ -137,10 +137,29 @@ SPECS: tuple[ConfigSpec, ...] = (
min=1,
max=100_000,
),
# min=1, NOT 0: the per-file cap is enforced as `if cap and size > cap`, so a stored 0 would read
# as falsy and SILENTLY disable the limit (an unbounded streamed write to the volume). Floor it at
# 1 so the cap is always active; the overall footprint is separately bounded by the byte quota.
ConfigSpec("download_upload_max_bytes", "int", "downloads", "download_upload_max_bytes", min=1),
# min=1, NOT 0: a stored 0 used to read as "no per-file cap" (the cap was enforced as
# `if cap and size > cap`, so a falsy value disabled it). Since P1 the route passes this number
# straight to `capped_stream_to`, where 0 would refuse every upload instead — the opposite
# failure, and the reason `config.download_upload_max_bytes` carries its own `ge=1`. Either way
# a stored 0 is wrong, so it is floored at 1; the overall footprint is separately bounded by the
# byte quota.
# `max` is not cosmetic: `bodylimit.ROUTE_MAX_BODY_BYTES` refuses an upload body over exactly
# this number at the edge, before the route's own check ever sees it. An admin allowed to raise
# the knob past the edge would get 413s from a layer the admin page never mentions, so the two
# are pinned together here and `tests/test_body_limit.py` holds them that way. Below the edge is
# the useful direction and the one the knob exists for: a lowered knob is what makes the route's
# own 422 — the one carrying this figure — the answer the user gets. `_coerce` also
# REJECTS a stored value outside these bounds on read (falling back to the env default, with a
# warning) — neither bound has always been what it is now, so a legal-when-written override can
# be sitting in the database already.
ConfigSpec(
"download_upload_max_bytes",
"int",
"downloads",
"download_upload_max_bytes",
min=1,
max=settings.download_upload_max_bytes,
),
ConfigSpec(
"download_retention_days", "int", "downloads", "download_retention_days", min=0, max=3_650
),
@@ -192,6 +211,10 @@ SPECS: tuple[ConfigSpec, ...] = (
# --- TMDB enrichment (E5 S2; optional) — the API key is the only secret ---
ConfigSpec("tmdb_api_key", "str", "tmdb", "tmdb_api_key", secret=True),
ConfigSpec("tmdb_language", "str", "tmdb", "tmdb_language"),
# --- Bug reports (E16): the Gitea issue tracker a debug session's issue goes to ---
ConfigSpec("issues_gitea_url", "str", "issues", "issues_gitea_url"),
ConfigSpec("issues_gitea_repo", "str", "issues", "issues_gitea_repo"),
ConfigSpec("issues_gitea_token", "str", "issues", "issues_gitea_token", secret=True),
# (tmdb_enrich_interval_min is a scheduler job, tuned from the Scheduler dashboard; tmdb_enrich_batch
# is a config.py default — both stay out of SPECS, like plex_sync_interval_min.)
)
@@ -232,9 +255,29 @@ def _choice(s: ConfigSpec, raw: object) -> str:
return option
class OutOfRange(ValueError):
"""A stored int override that no longer satisfies its spec's bounds."""
def _coerce(s: ConfigSpec, raw: str):
if s.type == "int":
return int(raw)
# Bounds are checked on READ, not only in `set_value`. Bounds get ADDED to a spec
# (download_upload_max_bytes gained a `max` when SEC-20 started enforcing the same number at
# the edge; its `min` went 0 → 1 during E7) and env defaults get LOWERED, and either leaves
# a stored value that was legal when it was written. Serving it is how one layer ends up
# acting on a number another layer refuses, with the admin page showing a value outside the
# range it renders.
#
# It raises rather than CLAMPING, and that distinction cost a round: clamping is a silent
# rewrite, and silently rewriting is exactly what this module refuses to do elsewhere. The
# case that settles it is a stored 0 for download_upload_max_bytes — legal when `min` was 0,
# and meaning "no per-file cap" to the route that reads it. Clamped to `min=1` it becomes a
# ONE-BYTE cap and uploads stop working with nothing in the log. Raising drops it into the
# same fallback the uncoercible path uses: the env default, and one warning.
v = int(raw)
if (s.min is not None and v < s.min) or (s.max is not None and v > s.max):
raise OutOfRange(f"{v} is outside [{s.min}, {s.max}]")
return v
if s.type == "bool":
return raw == "true"
if s.type == "choice":
@@ -287,14 +330,19 @@ def get(db: Session, key: str):
return _coerce(s, raw)
except (TypeError, ValueError) as exc:
# The admin set this in the UI and it is quietly not in force: the app runs on the env
# default while the settings page goes on showing the value that was saved.
# default while the settings page goes on showing the value that was saved. An OutOfRange is
# the same situation arrived at from the other direction — the value was fine when it was
# written and the spec moved under it — so it gets the same treatment and its own code.
event = (
"sysconfig.override_out_of_range"
if isinstance(exc, OutOfRange)
else "sysconfig.override_uncoercible"
)
if (s.key, raw) not in _coerce_warned:
_coerce_warned.add((s.key, raw))
log.warning(
"a stored setting could not be coerced; the default is used instead",
extra=logs.event(
"sysconfig.override_uncoercible", key=s.key, reason=type(exc).__name__
),
"a stored setting is not usable; the default is used instead",
extra=logs.event(event, key=s.key, reason=type(exc).__name__),
)
return _default(s)
+2 -2
View File
@@ -43,7 +43,7 @@ from app.downloads.jobdb import (
_staging_dir,
)
from app.downloads.status import STOPPED_BY_ROUTE, JobStatus
from app.models import DownloadJob, MediaAsset
from app.models import DISPLAY_TEXT_MAX, DownloadJob, MediaAsset
log = logs.get_logger(__name__)
_stop = threading.Event()
@@ -562,7 +562,7 @@ def _finish_from_cache(job_id: int, asset_id: int) -> None:
fields = {"status": JobStatus.DONE, "progress": 100, "phase": None}
if not job.display_name and asset.title:
# display_name is VARCHAR(255); a title can be longer (e.g. a recipe in the title).
fields["display_name"] = asset.title[:255]
fields["display_name"] = asset.title[:DISPLAY_TEXT_MAX]
asset.last_access_at = datetime.now(timezone.utc)
# Atomic finalize: only marks the job done while it is still running, so a concurrent
# cancel/pause is never resurrected (see service.finalize_if_running).
+31
View File
@@ -0,0 +1,31 @@
// The backend type-check lane (E27 S3) — the gate's ninth lane, and the first thing that has ever
// type-checked the ~36k lines of `app/`. Read as JSONC: pyright accepts comments here.
//
// basedpyright, not mypy: measured on this tree with the dependencies actually installed, both land
// ~190-200 errors with the same distribution, so the choice was made on the things that differ —
// basedpyright is ~2x faster, needs no MYPYPATH to resolve our `app.*` imports, and has a FIRST-CLASS
// baseline (`--writebaseline`), which is the whole ratchet. mypy would have needed a third-party tool
// for that.
//
// It runs in the pytest test image, never on the host: `requirements.lock` cannot be installed on
// Windows at all (uvloop refuses to build there), so a host run would type-check against absent
// third-party packages and report nonsense. The version is pinned in requirements-dev.txt, which the
// image installs — so the lane's version IS the pin, with no host-vs-pin check to drift.
//
// `pythonVersion` is not free-standing: `/.python-version` is the single source of truth and
// `tests/test_python_engine_contract.py` fails this gate if this file, the two Dockerfiles, the CI
// container or ruff's target-version disagree with it.
//
// `include` covers `alembic` for the same reason the ruff lane was widened to it: migrations run on
// every container start, so one that raises there crash-loops prod, and "it is generated code" is
// not an argument when the generated code is what boots the system. `tests/` is deliberately OUT —
// a test's job is to fail loudly when it is wrong, which is the opposite of the silent runtime
// failure this lane exists to catch, and annotating fixtures to a checker's satisfaction is churn
// the ratchet exists to avoid.
{
"include": ["app", "alembic"],
"extraPaths": ["."],
"typeCheckingMode": "basic",
"pythonVersion": "3.12",
"pythonPlatform": "Linux"
}
+4
View File
@@ -4,6 +4,10 @@
# backend lint lane runs on the host (the gate fails when the host's ruff is not this one). httpx2:
# Starlette 1.x's TestClient is built on it (`import httpx2 as httpx`) and only falls back to the
# app's own httpx with a deprecation warning on every run.
# basedpyright: the backend type-check lane (E27 S3). Pinned like everything else here — a type
# checker whose version floats silently changes what "clean" means, and the ratchet baseline it
# writes is keyed to the diagnostics THIS version produces.
basedpyright==1.40.1
pytest==8.4.2
ruff==0.15.21
httpx2==2.13.0
-1
View File
@@ -47,7 +47,6 @@ pydantic-settings==2.15.0
pydantic==2.13.4
pydantic_core==2.46.4
python-dotenv==1.2.3
python-multipart==0.0.32
requests==2.34.2
rich==15.0.0
starlette==1.6.0
-3
View File
@@ -1,7 +1,4 @@
fastapi>=0.115,<1.0
# Multipart form parsing — required by Starlette/FastAPI to accept file uploads (UploadFile/Form).
# The download center's local-file Upload (E7) is the only multipart endpoint in the app.
python-multipart>=0.0.9,<1.0
uvicorn[standard]>=0.30,<1.0
sqlalchemy>=2.0,<2.1
psycopg[binary]>=3.2,<4.0
+4
View File
@@ -7,6 +7,10 @@
# the repo wrap at the same width, and it's the width `ruff format` enforces (gated via
# `ruff format --check` alongside `ruff check`).
line-length = 100
# The interpreter this code actually runs on, so ruff judges syntax and upgrade hints against the
# right one. Left unset it falls back to a default that has nothing to do with us. `/.python-version`
# is the single source of truth; tests/test_python_engine_contract.py holds this to it.
target-version = "py312"
[lint]
# Beyond the default set, the rules that hold a LOG LINE to what E22.5 makes of it:
+30 -2
View File
@@ -15,6 +15,7 @@ tests that COMMIT (the whole point of this lane — transaction-ownership behavi
import os
import shutil
from urllib.parse import quote
import pytest
from alembic import command
@@ -111,8 +112,11 @@ def _request_dep(db, unfinished: list[str]):
request that ends normally with a write it did not commit — flushed, or still only staged — is
recorded, and the test fails on it at teardown. Only the ROOT transaction's end clears it: a
savepoint released inside the handler commits nothing. The test's own staged rows are flushed before the request starts — they are the
test's business, not the handler's. A request that raises is exempt: its writes are meant to
roll back."""
test's business, not the handler's. A request that raises is rolled back, as production's
`close()` rolls it back: without that, what a failing handler flushed but never committed stayed
readable to the rest of the test — the one state production cannot be in (E16: an issue created
on the tracker before a failed upload was kept only by an explicit commit, and nothing saw it
missing)."""
def dep(conn: HTTPConnection):
db.flush()
@@ -133,6 +137,9 @@ def _request_dep(db, unfinished: list[str]):
staged = db.new or db.deleted or any(db.is_modified(o) for o in db.dirty)
if state["dirty"] or staged:
unfinished.append(f"{conn.scope.get('method', 'WS')} {conn.url.path}")
except Exception:
db.rollback()
raise
finally:
for name, fn in hooks:
event.remove(db, name, fn)
@@ -242,6 +249,27 @@ def as_user(db, override_dep, _configured):
return _as
@pytest.fixture
def upload_headers():
"""Build the headers `POST /api/downloads/upload` takes since P1.
Shared, because two test modules drive that route and a second copy of this is exactly the drift
that lets the wire shape change in one place only. The body IS the file now; the names ride as
percent-encoded headers (`quote` mirrors the client's `encodeURIComponent`, since headers are
latin-1 and a Hungarian filename would not survive raw)."""
def _headers(filename: str, display_name: str | None = None) -> dict[str, str]:
headers = {
"content-type": "application/octet-stream",
"x-upload-filename": quote(filename),
}
if display_name is not None:
headers["x-upload-display-name"] = quote(display_name)
return headers
return _headers
@pytest.fixture(autouse=True)
def _reset_rate_limiters():
"""Every RateLimiter is a module-level singleton keyed by client IP, and TestClient always
+705
View File
@@ -0,0 +1,705 @@
"""SEC-20 — the global request-body cap (`app/bodylimit.py`).
Two halves, deliberately kept apart:
* the MIDDLEWARE tests drive a toy ASGI app, so they pin the mechanism (declared length, counted
bytes, the exemption match) with no router, no auth and no database in the way;
* the APP tests drive the real application, so they pin that the middleware is actually wired in,
sits ahead of the auth dependencies, and refuses in the same shape `capped_body` does.
The pair that matters most is "an oversized body is refused" together with "a normal body still
gets through": the first alone would pass just as well if the cap refused *everything*.
"""
import ast
import json
import pathlib
import pytest
from fastapi.testclient import TestClient
from starlette.requests import Request
from app import bodylimit, debugsession, sysconfig
from app.config import settings
from app.models import SystemConfig
from app.bodylimit import DEFAULT_MAX_BODY_BYTES, BodyLimitMiddleware
from app.routes import diagnostics as diagnostics_routes
SMALL = b"x" * 16
OVER = b"x" * (DEFAULT_MAX_BODY_BYTES + 1)
async def _reading_app(scope, receive, send):
"""A toy app that reads the whole body the way a route does, then reports its length.
It uses a real Starlette `Request`, so the faked `http.disconnect` reaches it as the real
ClientDisconnect a real handler would see — the middleware's exception path is exercised, not
stubbed around.
"""
body = await Request(scope, receive).body()
payload = json.dumps({"read": len(body)}).encode()
await send(
{
"type": "http.response.start",
"status": 200,
"headers": [(b"content-type", b"application/json")],
}
)
await send({"type": "http.response.body", "body": payload})
def _client(**kwargs) -> TestClient:
return TestClient(BodyLimitMiddleware(_reading_app, **kwargs))
def _chunks(total: int, size: int = 64 * 1024):
"""A body with NO Content-Length: httpx sends an iterator chunked, which is exactly the shape a
declared-length check cannot see."""
sent = 0
while sent < total:
n = min(size, total - sent)
yield b"x" * n
sent += n
# --- the mechanism ------------------------------------------------------------------------------
def test_a_declared_length_over_the_cap_is_refused_without_reading_the_body():
r = _client().post("/anything", content=OVER)
assert r.status_code == 413
assert r.content == bodylimit.TOO_LARGE_BODY
def test_a_body_under_the_cap_is_passed_through_untouched():
# The mutation guard for the test above: if the cap refused everything, this would fail.
r = _client().post("/anything", content=SMALL)
assert r.status_code == 200
assert r.json() == {"read": len(SMALL)}
def test_a_chunked_body_is_refused_on_the_COUNTED_bytes():
# No Content-Length at all, so the only thing that can stop this is counting as it arrives.
r = _client().post("/anything", content=_chunks(DEFAULT_MAX_BODY_BYTES + 64 * 1024))
assert r.status_code == 413
assert r.content == bodylimit.TOO_LARGE_BODY
def test_a_chunked_body_under_the_cap_still_arrives_whole():
size = DEFAULT_MAX_BODY_BYTES - 1
r = _client().post("/anything", content=_chunks(size))
assert r.status_code == 200
assert r.json() == {"read": size}
def test_an_understated_content_length_does_not_buy_a_pass():
# The declared-length check is the cheap half; a client that lies must still be counted out.
r = _client().post("/anything", content=OVER, headers={"content-length": "10"})
assert r.status_code == 413
def test_exactly_at_the_cap_is_allowed():
size = DEFAULT_MAX_BODY_BYTES
r = _client(limit=size).post("/anything", content=b"x" * size)
assert r.status_code == 200
assert r.json() == {"read": size}
# --- the per-route ceilings ---------------------------------------------------------------------
BIG = {("POST", "/api/things/{tid}/blob"): DEFAULT_MAX_BODY_BYTES * 4}
def test_a_route_with_a_raised_ceiling_may_send_a_body_over_the_global_cap():
r = _client(routes=BIG).post("/api/things/abc/blob", content=OVER)
assert r.status_code == 200
assert r.json() == {"read": len(OVER)}
def test_a_raised_ceiling_is_still_a_ceiling():
"""The difference between this and an exemption, and the reason the list is a mapping: a route
that needs more bytes does not thereby get unlimited bytes. Left uncounted, the upload route
would let an ANONYMOUS caller stream without bound — FastAPI parses a declared body before the
first dependency, so `require_human` has not run yet when the bytes land."""
over = b"x" * (BIG[("POST", "/api/things/{tid}/blob")] + 1)
r = _client(routes=BIG).post("/api/things/abc/blob", content=over)
assert r.status_code == 413
assert r.content == bodylimit.TOO_LARGE_BODY
def test_a_raised_ceiling_counts_a_chunked_body_too():
cap = BIG[("POST", "/api/things/{tid}/blob")]
r = _client(routes=BIG).post("/api/things/abc/blob", content=_chunks(cap + 64 * 1024))
assert r.status_code == 413
def test_the_raised_ceiling_is_scoped_to_its_method_and_its_path():
client = _client(routes=BIG)
# Same path, different method.
assert client.put("/api/things/abc/blob", content=OVER).status_code == 413
# A neighbour that merely starts the same way.
assert client.post("/api/things/abc/blob/extra", content=OVER).status_code == 413
def test_a_path_parameter_cannot_swallow_a_slash():
# `{tid}` matching greedily across `/` would hand the raised ceiling to routes without one.
mw = BodyLimitMiddleware(_reading_app, routes={("POST", "/a/{tid}/b"): 99})
assert mw.limit_for("POST", "/a/one/b") == 99
assert mw.limit_for("POST", "/a/one/two/b") == DEFAULT_MAX_BODY_BYTES
def test_every_request_gets_a_finite_ceiling():
"""There is no pass-through path. A route that is not listed gets the default; one that is
listed gets its own; nothing is simply not counted."""
mw = BodyLimitMiddleware(_reading_app)
assert mw.limit_for("GET", "/api/anything") == DEFAULT_MAX_BODY_BYTES
for (method, template), cap in bodylimit.ROUTE_MAX_BODY_BYTES.items():
assert cap > DEFAULT_MAX_BODY_BYTES # the admission rule: raised, never lowered
assert mw.limit_for(method, template.replace("{sid}", "abc")) == cap
def _declared_routes(root) -> set[tuple[str, str]]:
"""Every (method, path template) the app serves.
It has to RECURSE: since FastAPI 0.141 / Starlette 1.6 an `include_router` leaves an opaque
`_IncludedRouter` in `app.routes` and keeps the real routes under its `original_router`, instead
of flattening them into the parent as older versions did. Reading `app.routes` alone finds 35
entries and none of ours; walking the tree finds all 252.
"""
found: set[tuple[str, str]] = set()
stack = [root]
while stack:
node = stack.pop()
for route in getattr(node, "routes", ()) or ():
inner = getattr(route, "original_router", None)
if inner is not None:
stack.append(inner)
continue
for method in getattr(route, "methods", None) or ():
found.add((method, route.path))
return found
@pytest.mark.parametrize(("method", "template"), sorted(bodylimit.ROUTE_MAX_BODY_BYTES))
def test_every_listed_template_still_resolves_to_a_real_route(method, template):
"""The drift guard. The list names route templates that live in another file; a renamed route
must fail the gate here, not lose its allowance silently in production."""
from app.main import app
assert (method, template) in _declared_routes(app)
def test_the_default_ceiling_clears_the_largest_body_a_default_route_takes():
"""The headroom the module's docblock claims, held as a RATIO rather than as a figure in prose.
The client log sink caps itself at 512 KiB — below the default, which is why it is not listed:
the global ceiling never fires on it and raising it could only widen what it accepts.
`<=` alone would not hold the claim. The docblock calls the 2x margin load-bearing precisely
because a ceiling that refuses a legitimate request is a worse bug than the one this module
fixes, and a margin of 1x is no margin at all.
"""
assert diagnostics_routes.BODY_MAX_BYTES * 2 <= DEFAULT_MAX_BODY_BYTES
assert not any(
path.startswith("/api/diagnostics") for _, path in bodylimit.ROUTE_MAX_BODY_BYTES
)
# Every body-reading call site in the routers, as (module, function, limit expression). A route that
# reads its own body declares the size it MEANS to accept; if that is above the global default and
# its route is not in ROUTE_MAX_BODY_BYTES, this middleware refuses a request the route would have
# taken — silently, in production, on a path nobody tested. The point of the list is the third entry.
#
# BOTH helpers, not just `capped_body`: P1 added `capped_stream_to` as its sibling and the registry
# did not follow, so the upload route — the one this whole invariant was written for — was the one
# site the drift guard could not see (deep-review round 1). The names come from `_common.py` itself
# below, so a THIRD sibling cannot be added without this list noticing either — and a route that
# skips the helpers altogether and awaits `request.body()` lands here as `<no ceiling declared>`
# rather than vanishing (round 3).
#
# The module is the path relative to `app/routes/`, so a site under `plex/` is spelled
# `plex/admin.py` and cannot collide with the top-level `admin.py`.
CAPPED_BODY_SITES = {
("diagnostics.py", "_parsed_events", "capped_body", "BODY_MAX_BYTES"),
("debug_sessions.py", "_image_body", "capped_body", "debugsession.SCREENSHOT_MAX_BYTES"),
("downloads.py", "spool_upload", "capped_stream_to", "cap"),
}
def _body_readers() -> set[str]:
"""The names of the body-reading helpers `_common.py` exports — asked of the module rather than
listed here, so adding a fourth one cannot quietly leave the registry below behind."""
common = pathlib.Path(__file__).resolve().parents[1] / "app" / "routes" / "_common.py"
tree = ast.parse(common.read_text("utf-8"), filename=str(common))
return {
node.name
for node in tree.body
if isinstance(node, (ast.AsyncFunctionDef, ast.FunctionDef))
and node.name.startswith("capped_")
}
def test_the_registry_covers_every_body_reading_helper():
"""The mutation guard for the tests below: if a helper were renamed or added, the site list
would still match itself while the new one went uncovered."""
assert _body_readers() == {"capped_body", "capped_stream_to"}, (
"routes/_common.py grew or renamed a body-reading helper — add its call sites to "
"CAPPED_BODY_SITES and this expectation."
)
# The ways a route can read its own body WITHOUT one of `_common.py`'s helpers. Every `Request`
# parameter under `app/routes/` is named `request`, so the base name is an exact filter and not a
# guess — the only other `.json()` in the package is `exc.json()` on a ValidationError.
#
# Without this set the three guards below know only the `capped_*` naming convention, which means
# they are blind to the shape `_common.py` itself is written in. A route spelling `await
# request.body()` inline declares no ceiling, hands no connection back, and would be invisible to
# all three — reading as "no new sites" rather than as a failure, which is the exact way a drift
# guard dies quietly (deep-review round 3).
RAW_BODY_READS = {"body", "stream", "form", "json"}
def _call_name(node: ast.Call) -> str | None:
if isinstance(node.func, ast.Attribute):
return node.func.attr
return getattr(node.func, "id", None)
def _body_read_calls(fn: ast.AST, readers: set[str]) -> list[tuple[ast.Call, str]]:
"""Every call inside `fn` that reads the request body, as (node, name) in source order."""
out: list[tuple[ast.Call, str]] = []
for node in ast.walk(fn):
if not isinstance(node, ast.Call):
continue
if (
isinstance(node.func, ast.Attribute)
and isinstance(node.func.value, ast.Name)
and node.func.value.id == "request"
and node.func.attr in RAW_BODY_READS
):
out.append((node, f"request.{node.func.attr}"))
elif _call_name(node) in readers:
out.append((node, _call_name(node) or ""))
return sorted(out, key=lambda pair: (pair[0].lineno, pair[0].col_offset))
def _body_reading_functions() -> dict[tuple[str, str], dict]:
"""Every function under `app/routes/` that reads a request body itself, keyed by
(module, function).
Parsed, not grepped. A regex over the call text has to guess how the arguments are spelled, and
the shapes it guesses wrong — a bare `4 * 1024 * 1024`, a parameter not named `request` — are
the natural ways the NEXT site gets written, so a miss reads as "no new sites" rather than as a
failure. `ast` sees every call whatever it looks like.
Keyed by the ENCLOSING FUNCTION, not just the module: a set keyed by module and limit
expression collapses two call sites that spell their limit the same way, which is exactly what
a second route copied from the first would do.
The module half of the key is the path RELATIVE to `app/routes/`, not the basename: the scan
recurses, and `admin.py`, `discover.py` and `playlists.py` each exist both at the top level and
under `plex/`. Keying by basename would let the second of a colliding pair overwrite the first
and take its `releases` flag out of the comparison with it — the same entry-collapse this
registry was widened to close.
`_common.py` is skipped: it is where the helpers live, so its own `request.stream()` loops are
the implementation of the rule rather than a violation of it.
"""
readers = _body_readers()
routes_dir = pathlib.Path(__file__).resolve().parents[1] / "app" / "routes"
found: dict[tuple[str, str], dict] = {}
for py in sorted(routes_dir.rglob("*.py")):
if py.name == "_common.py":
continue
module = py.relative_to(routes_dir).as_posix()
tree = ast.parse(py.read_text("utf-8"), filename=str(py))
for fn in ast.walk(tree):
if not isinstance(fn, (ast.AsyncFunctionDef, ast.FunctionDef)):
continue
calls = _body_read_calls(fn, readers)
if not calls:
continue
names = [name for _, name in calls]
limits = []
for node, name in zip([n for n, _ in calls], names):
if name.startswith("request."):
limits.append("<no ceiling declared>")
continue
limit = next((kw.value for kw in node.keywords if kw.arg == "limit"), None)
if limit is None:
pos = 2 if name == "capped_stream_to" else 1
limit = node.args[pos] if len(node.args) > pos else None
limits.append(ast.unparse(limit) if limit is not None else "<none>")
# POSITION, not presence. `any(...)` over the function accepted a release that ran
# AFTER the await it exists to precede — the natural spelling for someone reading the
# name as "release it when you're done" — while pinning a connection for the whole
# transfer. The first body read is what the release has to beat.
releases = [
node
for node in ast.walk(fn)
if isinstance(node, ast.Call) and _call_name(node) == "release_read_connection"
]
first_read = calls[0][0].lineno
found[(module, fn.name)] = {
"helpers": names,
"limits": limits,
"releases": any(node.lineno < first_read for node in releases),
"releases_late": bool(releases)
and not any(node.lineno < first_read for node in releases),
}
return found
def test_every_body_reading_site_is_known_and_fits_under_its_ceiling():
found = {
(module, fn, site["helpers"][i], site["limits"][i])
for (module, fn), site in _body_reading_functions().items()
for i in range(len(site["helpers"]))
}
assert found == CAPPED_BODY_SITES, (
f"a body-reading call site appeared, moved or changed shape: {found ^ CAPPED_BODY_SITES}. "
"Each one declares a body size; check it against bodylimit.DEFAULT_MAX_BODY_BYTES and, if "
"it is larger, add its route to bodylimit.ROUTE_MAX_BODY_BYTES — then update this list."
)
# And the known ones really do satisfy the rule.
assert diagnostics_routes.BODY_MAX_BYTES <= DEFAULT_MAX_BODY_BYTES
assert debugsession.SCREENSHOT_MAX_BYTES > DEFAULT_MAX_BODY_BYTES
assert ("POST", "/api/debug-sessions/{sid}/screenshots") in bodylimit.ROUTE_MAX_BODY_BYTES
# The upload site's limit is the sysconfig knob, which is bounded BY the edge ceiling rather
# than being a constant to compare against — that pairing is its own test below.
assert ("POST", "/api/downloads/upload") in bodylimit.ROUTE_MAX_BODY_BYTES
def test_release_read_connection_ends_the_transaction_it_was_called_for(db):
"""The helper itself: a SELECT has autobegun, and afterwards it has not."""
from sqlalchemy import select
from app.db import release_read_connection
db.execute(select(SystemConfig).limit(1))
assert db.in_transaction(), "nothing autobegan, so this test would pass vacuously"
release_read_connection(db)
assert not db.in_transaction()
def test_release_read_connection_refuses_to_discard_pending_work(db):
"""The precondition, ENFORCED rather than described.
The session belongs to the whole request, not to the dependency calling this — so if something
ahead of it ever stages a write, ending the transaction would throw that write away with
nothing in the log. `service.enqueue` already learned that a whole-session rollback takes the
caller's pending work with it, which is why it uses a SAVEPOINT. A comment cannot stop the next
dependency from being added in the wrong place; this can.
"""
from app.db import release_read_connection
db.add(SystemConfig(key="probe_pending_write", value="1"))
with pytest.raises(RuntimeError, match="pending work"):
release_read_connection(db)
db.rollback() # the staged row is this test's own
def test_release_read_connection_refuses_to_discard_a_FLUSHED_write(db):
"""The half the unit-of-work check cannot see.
`db.new`/`dirty`/`deleted` empty out the moment a helper flushes, and CLAUDE.md's canonical
split is "the handler commits; helpers flush" — so the single most likely future violator is
the one shape the obvious check is blind to. Measured before the fix: all three sets empty, the
guard silent, the row gone after the rollback.
"""
from app.db import release_read_connection
db.add(SystemConfig(key="probe_flushed_write", value="1"))
db.flush()
assert not (db.new or db.dirty or db.deleted), (
"the flush did not clear the unit-of-work, so this test would pass for the wrong reason"
)
with pytest.raises(RuntimeError, match="already WRITTEN"):
release_read_connection(db)
db.rollback()
def test_release_read_connection_refuses_to_discard_a_core_dml_write(db):
"""The other half: `db.execute(update(...))` never enters the unit-of-work at all, and
`downloads/service.py` writes that way on purpose (a conditional `UPDATE … WHERE status=…` is
the honest-states pattern)."""
from sqlalchemy import insert
from app.db import release_read_connection
db.execute(insert(SystemConfig).values(key="probe_core_dml", value="1"))
assert not (db.new or db.dirty or db.deleted), (
"Core DML entered the unit-of-work, so this test would pass for the wrong reason"
)
with pytest.raises(RuntimeError, match="already WRITTEN"):
release_read_connection(db)
db.rollback()
def test_every_body_reading_site_hands_its_pooled_connection_back_first():
"""Reading the body yourself is what makes a request SLOW inside its own database session.
The auth (or ownership) dependency that must resolve first is a SELECT, so SQLAlchemy has
autobegun and the session is holding one of the pool's connections; awaiting the body then pins
it for as long as the client keeps sending — one connection per in-flight upload, `idle in
transaction`, with nothing to time it out. `db.release_read_connection` ends that transaction,
and every one of these sites needs it.
Held generically because the three sites arrived one at a time and only the newest was fixed:
a fourth body-reading route gets the same requirement automatically.
BEFORE the read, not merely somewhere in the function. A release that runs after the await
satisfies the letter of the rule and none of its purpose — the connection was held for exactly
the span the rule exists to protect — so `releases_late` is reported as its own, louder case.
"""
sites = _body_reading_functions()
late = [f"{module}:{fn}" for (module, fn), site in sites.items() if site["releases_late"]]
missing = [
f"{module}:{fn}"
for (module, fn), site in sites.items()
if not site["releases"] and not site["releases_late"]
]
assert not late, (
"a route calls db.release_read_connection AFTER it reads the body, which holds the pooled "
f"connection for exactly the transfer it was supposed to free: {late}. Move the call above "
"the await."
)
assert not missing, (
"a route reads its own request body while still inside its read transaction, pinning a "
f"pooled connection for the whole transfer: {missing}. Call "
"db.release_read_connection(db) before the await — see routes/downloads.py:spool_upload."
)
def test_the_upload_ceiling_and_the_admin_knob_cannot_drift_apart():
"""The upload route's in-handler cap is a `sysconfig` value an admin can change; the edge is a
constant. If the knob could be set above the edge, an admin raising it would silently start
getting 413s from a layer the admin page never mentions.
Since P1 the two count the SAME bytes — the body is the file, with no multipart envelope
wrapped round it — so the old allowance is gone and the edge is the knob's own maximum,
exactly. A gap reappearing means someone re-wrapped the upload in an envelope without saying so.
Equality also means the EDGE is what answers on a default install, where no override has lowered
the knob: the declared-length check refuses before a byte is read. That is not a gap to close
with headroom (measured: headroom only moves the boundary by its own size, and the route's drain
pushes the count back over it) — it is a refusal the client has to be able to read, which is why
`errors.tooLarge` exists and why the dialog checks the file against `upload_max_bytes` first.
"""
spec = next(s for s in sysconfig.SPECS if s.key == "download_upload_max_bytes")
edge = bodylimit.ROUTE_MAX_BODY_BYTES[("POST", "/api/downloads/upload")]
assert spec.max == edge
def _route_objects(root):
"""`_declared_routes`' sibling, keeping the route OBJECT so its FastAPI dependant can be read."""
found: dict[tuple[str, str], object] = {}
stack = [root]
while stack:
node = stack.pop()
for route in getattr(node, "routes", ()) or ():
inner = getattr(route, "original_router", None)
if inner is not None:
stack.append(inner)
continue
for method in getattr(route, "methods", None) or ():
found[(method, route.path)] = route
return found
@pytest.mark.parametrize(("method", "template"), sorted(bodylimit.ROUTE_MAX_BODY_BYTES))
def test_no_route_with_a_raised_ceiling_declares_a_body_parameter(method, template):
"""P1's invariant, and the reason this is a test rather than a comment.
A raised ceiling is only honest when the route's own cap applies to the bytes AS THEY ARRIVE.
FastAPI reads and spools a DECLARED body (`= File(...)`, `= Form(...)`, a Pydantic model)
BEFORE the first dependency resolves — so on such a route this middleware's ceiling is the only
thing between an anonymous caller and the whole allowance, `require_human` never gets a say, and
the route's in-handler cap measures bytes that have already landed. That is exactly what P1
closed on the upload route, which sat in `ROUTE_MAX_BODY_BYTES` with a 500 MiB ceiling and a
`file: UploadFile = File(...)` body for one release.
Asked of FastAPI itself rather than parsed or grepped: the router already knows which parameters
it will read the body for, and a shape a regex or an AST walk guesses wrong is precisely how the
next route slips through.
`route.body_field`, NOT `route.dependant.body_params`, and the difference is the whole test.
`body_params` holds only the parameters of the route's OWN callable; `body_field` is built from
`_get_flat_body_params`, which recurses into sub-dependencies — and that is also what FastAPI
itself consults when it decides to read the body. Measured on the pinned FastAPI: a body declared
inside a sub-dependency leaves `body_params` EMPTY while the body is still read in full before
auth, so the first version of this test was green over a reproduction of the very defect it
exists to forbid (deep-review round 1). The upload route's own idiom — a body read from a
`yield` dependency — is exactly the shape that would have slipped through."""
from app.main import app
route = _route_objects(app).get((method, template))
assert route is not None, f"{method} {template} is not a route any more"
assert route.body_field is None, (
f"{method} {template} declares a body ({route.body_field.name}) on the route or in one of "
"its dependencies, so FastAPI reads it before the first dependency runs and the raised "
"ceiling is all that stands between an anonymous caller and the whole allowance. Read the "
"body yourself, from a dependency declared AFTER the auth one — see "
"routes/downloads.py:spool_upload."
)
def test_a_stored_override_outside_the_bounds_falls_back_on_READ(db):
"""`set_value` refuses an out-of-range int, but neither bound has always been what it is now —
the upload knob had no `max` before SEC-20 and its `min` was 0 before E7 — so a value that was
legal when it was written can be sitting in the database. Served as-is it is exactly the drift
the test above forbids: the route would accept bytes the edge refuses.
The fallback is the env default and a warning, NOT a clamp. A clamp is a silent rewrite, and on
the min side it would be a wrong one: a stored 0 meant "no per-file cap" to the route, and
clamped to 1 it becomes a one-byte cap that kills uploads with nothing in the log.
"""
spec = next(s for s in sysconfig.SPECS if s.key == "download_upload_max_bytes")
assert spec.max is not None and spec.min is not None
default = settings.download_upload_max_bytes
row = SystemConfig(key="download_upload_max_bytes", value=str(spec.max * 4))
db.add(row)
db.commit()
assert sysconfig.get_int(db, "download_upload_max_bytes") == default
row.value = "0" # the legacy "no per-file cap", now below min
db.commit()
assert sysconfig.get_int(db, "download_upload_max_bytes") == default
# An override that IS in range must still be honoured — otherwise a bounds predicate written
# backwards would make every override inert and the two assertions above could not tell.
# Deliberately not `spec.max`: that equals the env default for this key, so the assertion would
# be satisfied by the fallback it is meant to rule out.
row.value = str(spec.max - 1)
db.commit()
assert spec.max - 1 != default
assert sysconfig.get_int(db, "download_upload_max_bytes") == spec.max - 1
def test_the_cap_sits_inside_the_response_decorators_and_outside_the_setup_gate():
"""The registration order, held in code rather than in the comment beside it. `add_middleware`
inverts the reading order (last registered = OUTERMOST), which is exactly the kind of thing a
prose description gets backwards — it already did once.
The layers are named, not counted by class: all four `@app.middleware("http")` decorators —
including the setup gate, which must be INSIDE — register as the same `BaseHTTPMiddleware`, so a
class-shaped assertion passes for every arrangement anyone would plausibly get wrong.
Outside the cap: only response decorators, so the 413 is dressed like any other answer. Inside
it: everything that does work on the caller's behalf — the setup gate opens a DB session per
request while an instance is unconfigured, which is the anonymous pre-auth work SEC-20 exists to
remove.
"""
from app.main import app
def layer(m) -> str:
dispatch = m.kwargs.get("dispatch") if m.kwargs else None
return dispatch.__name__ if dispatch is not None else getattr(m.cls, "__name__", str(m.cls))
names = [layer(m) for m in app.user_middleware] # outer → inner
i = names.index("BodyLimitMiddleware")
assert set(names[:i]) <= {
"log_requests",
"security_headers",
"static_cache_headers",
"CORSMiddleware",
}, f"something that is not a response decorator sits outside the cap: {names[:i]}"
assert "log_requests" in names[:i], "the access log must stay outside, so a 413 is logged"
assert "setup_gate" in names[i + 1 :], "the setup gate must not run for an oversized body"
assert "SessionMiddleware" in names[i + 1 :]
def test_the_screenshot_ceiling_matches_the_cap_the_route_itself_enforces():
assert debugsession.SCREENSHOT_MAX_BYTES > DEFAULT_MAX_BODY_BYTES
assert (
bodylimit.ROUTE_MAX_BODY_BYTES[("POST", "/api/debug-sessions/{sid}/screenshots")]
== debugsession.SCREENSHOT_MAX_BYTES
)
# --- the real application -----------------------------------------------------------------------
def test_the_global_refusal_carries_the_baseline_security_headers():
"""The 413 is a response like any other, so it has to be dressed like one. The middleware sits
INSIDE the security-header and CORS layers for exactly this reason: a refusal a browser drops
for a missing header is indistinguishable from a network failure, and the user would be shown a
generic error instead of the translated `too_large`.
A bare TestClient on purpose, not the `client` fixture: this is the only test that exercises the
middleware ORDER, and that fixture is DB-gated — so on a host without Postgres the one assertion
holding the move would have skipped while the file still reported green. The refusal never
reaches a database; it is answered above the router.
"""
from app.main import app
r = TestClient(app).post(
"/auth/password-login", content=OVER, headers={"content-type": "text/plain"}
)
assert r.status_code == 413
assert r.headers.get("x-content-type-options") == "nosniff"
assert "referrer-policy" in r.headers
def test_the_login_route_is_refused_before_a_single_dependency_runs(client, monkeypatch):
"""`/auth/password-login` is the reason SEC-20 exists: it is anonymous, it declares a body, and
FastAPI reads that body in full before the first dependency. `_login_limiter.allow` is the very
first statement of the handler, so it is the sharpest available proof that nothing ran."""
from app.routes import auth as auth_routes
reached = False
def _record(*args, **kwargs):
nonlocal reached
reached = True
return True
monkeypatch.setattr(auth_routes._login_limiter, "allow", _record)
r = client.post("/auth/password-login", content=OVER, headers={"content-type": "text/plain"})
assert r.status_code == 413
assert not reached, "the handler ran — the cap is inside the router instead of ahead of it"
def test_a_normal_login_body_still_reaches_the_handler(client, monkeypatch):
"""The mutation check for the test above. If the cap refused every body, the 413 test would
pass while the app was broken; this one would not."""
from app.routes import auth as auth_routes
reached = False
def _record(*args, **kwargs):
nonlocal reached
reached = True
return True
monkeypatch.setattr(auth_routes._login_limiter, "allow", _record)
r = client.post("/auth/password-login", json={"email": "nobody@example.com", "password": "x"})
assert r.status_code == 401
assert reached
def test_the_global_refusal_is_byte_identical_to_the_per_route_one(db, as_user, make_user):
"""Two shapes for one fact is how the `cancelled`/`canceled` bug happened. The log sink caps
itself with `capped_body`; its 413 and the middleware's must be the same bytes.
Signed in on purpose: `current_user` resolves before the body dependency, so an anonymous call
would be answered 401 and never reach the per-route cap this compares against. The body is over
the SINK's 512 KiB but under the global 1 MiB, so it is `capped_body` that refuses it.
"""
user = make_user()
db.commit()
over_the_sink = b"x" * (diagnostics_routes.BODY_MAX_BYTES + 1)
assert len(over_the_sink) < DEFAULT_MAX_BODY_BYTES
r = as_user(user).post(
"/api/diagnostics/events",
content=over_the_sink,
headers={"content-type": "application/json"},
)
assert r.status_code == 413
# RAW BYTES, not parsed JSON: the two used to differ by the two spaces `json.dumps` puts in by
# default and `JSONResponse` does not, and a structural comparison could not see it — which is
# the only kind of drift the docblock's promise was ever at risk from.
assert r.content == bodylimit.TOO_LARGE_BODY
File diff suppressed because it is too large Load Diff
+2
View File
@@ -68,6 +68,7 @@ def test_raising_the_level_sets_the_servers_expiry_whatever_the_client_sends(db,
"level": "INFO",
"until": None,
"retention_days": 7,
"session_minutes": 60,
}
@@ -82,6 +83,7 @@ def test_a_lapsed_level_reads_as_the_base_in_me(db, client, make_user):
"level": "INFO",
"until": None,
"retention_days": 7,
"session_minutes": 60,
}
@@ -58,3 +58,27 @@ def test_the_commit_check_counts_only_the_root_transaction(db):
next(request)
assert db.info["unfinished"] == ["POST /probe"]
db.info["unfinished"].clear() # recorded as it should be — not this test's failure
def test_a_request_that_raises_leaves_nothing_it_did_not_commit(db):
"""Production closes a failing request's session, which rolls back whatever it flushed and did
not commit; the shared test session must do the same, or a later read in the test sees a write
production would have lost (E16: the tracker's issue link, kept only by an explicit commit in
the handler's failure path). Driven directly, as a request would drive the dependency."""
import pytest
from sqlalchemy import select
from starlette.requests import HTTPConnection
from app.models import User
conn = HTTPConnection(
{"type": "http", "method": "POST", "path": "/probe", "headers": [], "query_string": b""}
)
request = db.info["request"](conn)
session = next(request)
session.add(User(email="rolled-back-probe@example.com"))
session.flush()
with pytest.raises(RuntimeError):
request.throw(RuntimeError("the handler failed"))
assert db.scalar(select(User).where(User.email == "rolled-back-probe@example.com")) is None
assert db.info["unfinished"] == [] # a failing request is rolled back, not reported
+2 -2
View File
@@ -18,12 +18,12 @@ _PW = "open sesame"
@pytest.fixture
def unlock_url(as_user, make_user, tmp_path, monkeypatch):
def unlock_url(as_user, make_user, tmp_path, monkeypatch, upload_headers):
monkeypatch.setattr(settings, "download_root", str(tmp_path))
monkeypatch.setattr(probemod, "probe_media", lambda _p: ("h264", "aac", 320, 240, 5))
owner = as_user(make_user())
job = owner.post(
"/api/downloads/upload", files={"file": ("clip.mp4", b"fake-video", "video/mp4")}
"/api/downloads/upload", content=b"fake-video", headers=upload_headers("clip.mp4")
)
assert job.status_code == 200, job.text
link = owner.post(f"/api/downloads/{job.json()['id']}/links", json={"password": _PW})
@@ -0,0 +1,66 @@
"""The Python major is a contract, not a coincidence — the backend half (E27 S3).
`0.93.2` was released to fix exactly this bug on the JavaScript side: four machines built or checked
the app and none of them agreed on which Node they ran, the server gate went red for 73 of its last
90 runs, and nothing told anyone. The Python side had the same shape and nobody had looked: prod
(`Dockerfile`), the test image (`Dockerfile.test`) and CI (`check.yml`) each spelled **3.12** as an
independent literal, there was no `.python-version`, no `requires-python`, and ruff had no
`target-version` — while the dev machine had no 3.12 installed at all, only 3.13 and 3.14.
**The contract is a chain, and it is split across two test files because no single runner can see
every link.** This suite runs inside the test image, which mounts `backend/` and nothing else, so it
cannot read the repo-root Dockerfile or the workflow. It therefore holds the link it CAN see, and the
one that matters most here:
the interpreter actually running == backend/pyrightconfig.json == backend/ruff.toml
and `frontend/src/lib/pythonEngine.contract.test.ts` — which runs on the host, with the whole tree —
holds the rest of the chain:
/.python-version == pyrightconfig.json == ruff.toml == Dockerfile == Dockerfile.test == check.yml
Together they close the loop. To move the version: edit `/.python-version`, then follow the failures
in both files. Every place they name is a place that would otherwise have been forgotten.
"""
import re
import sys
from pathlib import Path
BACKEND = Path(__file__).resolve().parents[1]
def _version_in(path: Path, pattern: str) -> tuple[int, int]:
"""The (major, minor) the given file declares. A rename or a moved line lands here — re-point
the test, the contract did not go away with it."""
text = path.read_text(encoding="utf-8")
match = re.search(pattern, text, re.MULTILINE)
assert match is not None, f"{pattern!r} not found in {path.name}"
return int(match.group(1)), int(match.group(2))
# pyrightconfig.json is JSONC (it carries comments), so it is read by regex rather than json.loads.
PINNED = _version_in(BACKEND / "pyrightconfig.json", r'"pythonVersion"\s*:\s*"(\d+)\.(\d+)"')
def test_pins_a_plausible_python_version():
"""Guards against a typo agreeing with nothing at all."""
assert PINNED >= (3, 12), PINNED
def test_this_interpreter_is_the_pinned_one():
"""The assertion that pays. A green suite on another minor does not mean the shipped image is
green — it means the suite and the image have drifted and nobody noticed yet."""
running = sys.version_info[:2]
assert running == PINNED, (
f"these tests are running on Python {running[0]}.{running[1]}, but the app is pinned to "
f"{PINNED[0]}.{PINNED[1]} (backend/pyrightconfig.json, /.python-version). The suite runs in "
f"the test image, so this failing means backend/Dockerfile.test has drifted from the pin."
)
def test_ruff_judges_the_pinned_version():
"""Unset, ruff falls back to a default that has nothing to do with us, and its syntax and
upgrade advice are then aimed at some other interpreter."""
major, minor = _version_in(BACKEND / "ruff.toml", r'^target-version\s*=\s*"py(\d)(\d+)"')
assert (major, minor) == PINNED
+2
View File
@@ -28,6 +28,8 @@ COMMITTING_HELPERS = {
"commits nothing)",
"messages.py:ensure_welcome": "the websocket push after it must announce a row the client "
"can already fetch",
"debug_sessions.py:_end_send": "gives back the claim a send took, in a `finally`: it runs when "
"the handler is already unwinding and has no commit left to reach",
}
+513 -26
View File
@@ -12,15 +12,23 @@ parsing (image → file, cover-art skip, dims, duration) is covered directly by
below, which stubs one level down (`run_ffprobe_json`) and exercises the real `probe_media`.
"""
import ast
import asyncio
import io
import pathlib
from pathlib import Path
import pytest
from fastapi import HTTPException
from app.config import settings
from app.routes._common import capped_stream_to
from sqlalchemy import select
from app.downloads import mediakind, quota, storage
from app.downloads import probe as probemod
from app.downloads import upload as uploadmod
from app.models import DownloadQuota, MediaAsset
from app.models import DISPLAY_TEXT_MAX, DownloadJob, DownloadQuota, MediaAsset
# --- pure: the kind invariant ---------------------------------------------------------------
@@ -123,15 +131,143 @@ def test_ingest_rejects_over_footprint_cap_before_writing(db, make_user, _root,
assert db.query(MediaAsset).count() == 0 # and no orphan asset row
# --- route: the auth-first body (P1) --------------------------------------------------------
def _counting_body(chunks: int = 4, size: int = 1024):
"""A request body that records how much of itself the server actually pulled.
httpx's ASGI transport feeds the app lazily, one `http.request` message per yielded chunk, so a
chunk that was never yielded is a chunk the app never asked for. That is the whole assertion
below: it is not "the upload was refused", it is "the refusal happened before the bytes moved".
"""
pulled: list[int] = []
def body():
for _ in range(chunks):
pulled.append(size)
yield b"x" * size
return body(), pulled
_BOUNDARY = "p1boundary"
def _counting_multipart(chunks: int = 4, size: int = 1024):
"""The same counter, wrapped in a real multipart envelope.
The content-type is the load-bearing part. Starlette's `request.form()` returns an EMPTY
FormData without touching the stream when the content-type is not multipart/urlencoded — so a
raw-body probe cannot tell "the route stopped reading strangers" apart from "there was nothing
here to parse". Only a well-formed multipart body makes the old shape actually spool.
"""
pulled: list[int] = []
head = (
f"--{_BOUNDARY}\r\n"
'Content-Disposition: form-data; name="file"; filename="clip.mp4"\r\n'
"Content-Type: application/octet-stream\r\n\r\n"
).encode()
tail = f"\r\n--{_BOUNDARY}--\r\n".encode()
def body():
pulled.append(len(head))
yield head
for _ in range(chunks):
pulled.append(size)
yield b"x" * size
pulled.append(len(tail))
yield tail
return body(), pulled
def test_an_anonymous_upload_is_refused_before_a_byte_of_it_is_read(client):
"""P1 — the defect E27 bounded but did not close.
While the route declared `file: UploadFile = File(...)`, FastAPI parsed and spooled the body
BEFORE the first dependency ran, so `require_human` could not gate it: an unauthenticated
caller could make the server write the whole per-file allowance (500 MiB by default) to temp,
per request. SEC-20's ceiling capped the number; it could not stop the writing.
The route now reads its own body from a dependency declared AFTER the auth one, the shape
`POST /api/debug-sessions/{sid}/screenshots` already uses. The claim under test is the strong
one — a stranger's bytes are never read WHATEVER they claim to send, so the probe sends the
multipart the old route would have spooled, not the raw body the new one expects."""
body, pulled = _counting_multipart()
r = client.post(
"/api/downloads/upload",
content=body,
headers={"content-type": f"multipart/form-data; boundary={_BOUNDARY}"},
)
assert r.status_code == 401
assert pulled == [], f"the server read {sum(pulled)} bytes from an anonymous caller"
def test_an_authenticated_upload_still_has_its_body_read(as_user, make_user, _root, monkeypatch):
"""The mutation check for the test above: a route that read NOBODY's body would pass it just as
well as the fixed one. The same counting body, signed in, must be pulled in full."""
_probe(monkeypatch, (None, None, None, None, None))
body, pulled = _counting_body()
c = as_user(make_user())
r = c.post(
"/api/downloads/upload",
content=body,
headers={"content-type": "application/octet-stream", "x-upload-filename": "clip.bin"},
)
assert r.status_code == 200, r.text
assert sum(pulled) == 4096
def test_capped_stream_to_drains_the_rest_before_refusing():
"""Measured in a real browser, then pinned HERE rather than at the route.
The cap now fires while the client is still sending, and answering + closing mid-upload is what
a browser reports as a NETWORK failure: the dialog said "the server couldn't be reached"
instead of naming the limit, because XHR never got to read the 422. `capped_stream_to` reads
the remainder to nowhere before raising, so the client finishes sending and reads the real
answer.
It is asserted against the helper and not through TestClient on purpose — httpx's ASGI
transport pumps the request body whether or not the app reads it, so a route-level version of
this test passes just as happily with the drain deleted. It was written that way first and
caught by its own mutation check.
"""
pulled = []
async def stream():
for i in range(8):
pulled.append(i)
yield b"x" * 1024
class _Req:
def stream(self):
return stream()
out = io.BytesIO()
refusal = HTTPException(status_code=422, detail={"code": "upload_too_large", "limit": 2048})
# `asyncio.run` rather than an async test: the suite has no async plugin, and adding one to
# pin four lines of helper would mean a new dev dependency and a lock regeneration.
with pytest.raises(HTTPException) as exc:
asyncio.run(capped_stream_to(_Req(), out, 2048, refusal))
assert exc.value is refusal
assert len(pulled) == 8, "the body was abandoned mid-stream, not drained"
# Drained means DISCARDED: nothing past the limit is written.
assert len(out.getvalue()) <= 2048
# --- route ----------------------------------------------------------------------------------
def test_upload_route_generic_forces_download(as_user, make_user, _root, monkeypatch):
def test_upload_route_generic_forces_download(
as_user, make_user, _root, monkeypatch, upload_headers
):
_probe(monkeypatch, (None, None, None, None, None))
c = as_user(make_user())
r = c.post(
"/api/downloads/upload",
files={"file": ("report.html", b"<script>alert(1)</script>", "text/html")},
content=b"<script>alert(1)</script>",
headers=upload_headers("report.html"),
)
assert r.status_code == 200, r.text
job = r.json()
@@ -145,7 +281,7 @@ def test_upload_route_generic_forces_download(as_user, make_user, _root, monkeyp
def test_upload_route_media_html_extension_not_served_as_html(
as_user, make_user, _root, monkeypatch
as_user, make_user, _root, monkeypatch, upload_headers
):
"""The XSS guard: a MEDIA-classified upload (real codecs) named with an executable extension must
NOT be served with a guessed text/html type. The served Content-Type comes from the verified
@@ -156,7 +292,8 @@ def test_upload_route_media_html_extension_not_served_as_html(
c = as_user(make_user())
r = c.post(
"/api/downloads/upload",
files={"file": ("evil.html", b"<script>alert(1)</script>", "text/html")},
content=b"<script>alert(1)</script>",
headers=upload_headers("evil.html"),
)
assert r.status_code == 200, r.text
job = r.json()
@@ -177,7 +314,9 @@ def test_upload_route_media_html_extension_not_served_as_html(
assert "attachment" in pr.headers["content-disposition"].lower()
def test_public_watch_file_forces_download_for_generic_file(as_user, make_user, _root, monkeypatch):
def test_public_watch_file_forces_download_for_generic_file(
as_user, make_user, _root, monkeypatch, upload_headers
):
"""The login-free public serve of a generic upload (media_kind='file') must be a neutral
force-download attachment — the branch that stops uploaded HTML/SVG executing on the public
origin. This is a SEPARATE code path from the authenticated /file route, so it needs its own
@@ -186,7 +325,8 @@ def test_public_watch_file_forces_download_for_generic_file(as_user, make_user,
c = as_user(make_user())
job = c.post(
"/api/downloads/upload",
files={"file": ("notes.html", b"<script>alert(1)</script>", "text/html")},
content=b"<script>alert(1)</script>",
headers=upload_headers("notes.html"),
).json()
assert job["asset"]["media_kind"] == "file"
link = c.post(f"/api/downloads/{job['id']}/links", json={"allow_download": True}).json()
@@ -197,7 +337,7 @@ def test_public_watch_file_forces_download_for_generic_file(as_user, make_user,
def test_public_watch_file_serves_media_inline_with_real_type(
as_user, make_user, _root, monkeypatch
as_user, make_user, _root, monkeypatch, upload_headers
):
"""A recognised media container IS still served inline with its real audio/video type (so it
plays in the watch page) — the fix must not break legitimate playback. A stream-only link →
@@ -206,7 +346,8 @@ def test_public_watch_file_serves_media_inline_with_real_type(
c = as_user(make_user())
job = c.post(
"/api/downloads/upload",
files={"file": ("clip.mp4", b"fake-mp4-bytes", "video/mp4")},
content=b"fake-mp4-bytes",
headers=upload_headers("clip.mp4"),
).json()
link = c.post(f"/api/downloads/{job['id']}/links", json={"allow_download": False}).json()
pr = c.get(f"/api/public/watch/{link['token']}/file")
@@ -215,7 +356,9 @@ def test_public_watch_file_serves_media_inline_with_real_type(
assert "inline" in pr.headers["content-disposition"].lower()
def test_upload_route_enforces_per_file_cap(as_user, make_user, db, _root, monkeypatch):
def test_upload_route_enforces_per_file_cap(
as_user, make_user, db, _root, monkeypatch, upload_headers
):
from app import sysconfig
sysconfig.set_value(db, "download_upload_max_bytes", 5)
@@ -223,47 +366,391 @@ def test_upload_route_enforces_per_file_cap(as_user, make_user, db, _root, monke
c = as_user(make_user())
r = c.post(
"/api/downloads/upload",
files={"file": ("big.bin", b"way over the cap", "application/octet-stream")},
content=b"way over the cap",
headers=upload_headers("big.bin"),
)
assert r.status_code == 422
assert r.json()["detail"]["code"] == "upload_too_large"
def test_the_cap_admits_a_body_of_EXACTLY_the_cap_and_refuses_one_byte_more(
as_user, make_user, db, _root, monkeypatch, upload_headers
):
"""The boundary, which the test above cannot see: it sends 16 bytes against a cap of 5, so a
`>` silently loosened to `>=` refuses it just the same and the mutation survives the whole
suite. The cap is `size > limit`, so a file of exactly the cap must be ACCEPTED.
"""
from app import sysconfig
_probe(monkeypatch, (None, None, None, None, None))
payload = b"x" * 64
sysconfig.set_value(db, "download_upload_max_bytes", len(payload))
db.commit()
c = as_user(make_user())
at_the_cap = c.post(
"/api/downloads/upload", content=payload, headers=upload_headers("exact.bin")
)
assert at_the_cap.status_code == 200, at_the_cap.text
over = c.post(
"/api/downloads/upload", content=payload + b"x", headers=upload_headers("over.bin")
)
assert over.status_code == 422
assert over.json()["detail"] == {"code": "upload_too_large", "limit": len(payload)}
def test_a_non_ascii_filename_survives_the_header_round_trip(
as_user, make_user, db, _root, monkeypatch, upload_headers
):
"""The percent-decoding, which every other test in this file leaves vacuous.
Headers are latin-1 on the wire, so the client `encodeURIComponent`s the name and the route
`unquote`s it. Every filename the suite sends is pure ASCII — and `quote()` is the IDENTITY
function on ASCII — so deleting the `unquote` call changes nothing anywhere else. A Hungarian
name is the only input that can tell the two apart: without the decode the sanitizer would be
handed the literal `nyaral%C3%A1s.bin` and would keep the percent-escapes in the stored name.
"""
_probe(monkeypatch, (None, None, None, None, None))
c = as_user(make_user())
r = c.post(
"/api/downloads/upload", content=b"data", headers=upload_headers("nyaralás ősszel.bin")
)
assert r.status_code == 200, r.text
asset = db.execute(select(MediaAsset).order_by(MediaAsset.id.desc())).scalars().first()
assert asset is not None
stored = Path(asset.rel_path).name
assert "%" not in stored and "c3" not in stored.lower(), stored
assert stored.startswith("nyaralas_osszel"), stored
def test_a_long_display_name_is_truncated_on_EVERY_route_that_takes_one(
db, as_user, make_user, _root, monkeypatch, upload_headers
):
"""The sweep, held end to end rather than at one site.
`display_text` was introduced for the upload path and applied to three of the five writers of
this varchar(255) column; `POST /api/downloads` and `POST /api/downloads/edit` kept the old
`(x or "").strip() or None` and would still have answered a bare 500 (a psycopg DataError,
which main.py re-raises for everything but the NUL case) to a name a user simply pasted. Found
by four independent readers in deep-review round 2 — a sweep is exactly the kind of thing that
gets done four-fifths of the way, so this holds all of it at once.
"""
_probe(monkeypatch, ("h264", "aac", 640, 480, 60))
monkeypatch.setattr(uploadmod.editmod, "ensure_poster", lambda _a, _root: None)
long_name = "n" * 300
bound = "n" * DISPLAY_TEXT_MAX
c = as_user(make_user())
# 1. the upload route — the name arrives in a request header.
up = c.post(
"/api/downloads/upload", content=b"video", headers=upload_headers("a.mp4", long_name)
)
assert up.status_code == 200, up.text
assert up.json()["display_name"] == bound
# 2. the enqueue route — a plain JSON field, from the Add-download dialog. An inline `spec`
# rather than a profile id, so the test does not have to seed the builtin profile table.
enq = c.post(
"/api/downloads",
json={"source": "dQw4w9WgXcQ", "spec": {"kind": "video"}, "display_name": long_name},
)
assert enq.status_code == 200, enq.text
assert enq.json()["display_name"] == bound
# 3. the edit route — the same field, from the editor.
edit = c.post(
"/api/downloads/edit",
json={
"source_job_id": up.json()["id"],
"edit_spec": {"trim": {"start_s": 0, "end_s": 1}},
"display_name": long_name,
},
)
assert edit.status_code == 200, edit.text
assert edit.json()["display_name"] == bound
# 4 + 5. the metadata PATCH's two fields.
patched = c.patch(
f"/api/downloads/{up.json()['id']}",
json={"display_name": long_name, "display_uploader": long_name},
)
assert patched.status_code == 200, patched.text
assert patched.json()["display_name"] == bound
assert patched.json()["display_uploader"] == bound
def _display_write_sites() -> dict[tuple[str, str], set[str]]:
"""Every place under `backend/app/` that puts a value into a display field, as
(module relative to `app/`, enclosing function) -> the set of value expressions written there.
TWO SHAPES, because the first version of this guard walked only `ast.Assign` and was therefore
blind to the shape the upload route itself uses — `SpooledUpload(..., display_name=...)` is an
`ast.keyword`, not an assignment, so the one site with an unbounded REQUEST HEADER behind it was
the site the guard could not see (deep-review round 3).
ANNOTATED assignments are excluded on purpose: `display_name: Mapped[str | None] =
mapped_column(...)` in `models.py` and `display_name: str | None = None` in a Pydantic model are
declarations of the field, not writes into it.
The whole of `app/`, not just `app/routes/`: a job row is constructed in `downloads/service.py`
and `downloads/upload.py`, so a walk stopping at the HTTP boundary misses the writers that
actually reach the column.
"""
app_dir = pathlib.Path(__file__).resolve().parents[1] / "app"
found: dict[tuple[str, str], set[str]] = {}
for py in sorted(app_dir.rglob("*.py")):
module = py.relative_to(app_dir).as_posix()
tree = ast.parse(py.read_text("utf-8"), filename=str(py))
enclosing: dict[int, str] = {}
for fn in ast.walk(tree):
if isinstance(fn, (ast.AsyncFunctionDef, ast.FunctionDef)):
for node in ast.walk(fn):
enclosing.setdefault(id(node), fn.name)
for node in ast.walk(tree):
if isinstance(node, ast.Assign):
names = {
t.id if isinstance(t, ast.Name) else t.attr
for t in node.targets
if isinstance(t, (ast.Name, ast.Attribute))
}
value = node.value
elif isinstance(node, ast.keyword):
names = {node.arg} if node.arg else set()
value = node.value
else:
continue
if not names & {"display_name", "display_uploader"}:
continue
key = (module, enclosing.get(id(node), "<module>"))
found.setdefault(key, set()).add(ast.unparse(value))
return found
# Every write into a display field, with the expression each one writes. A varchar(255) column with
# a request header behind it turns a too-long name into a psycopg DataError — a 500 AFTER the whole
# file has transferred — so each of these has to be bounded, and each one here has been read.
#
# Bounded by `display_text` unless noted:
# auth.get_or_create_demo_user a literal.
# service.redownload CLEARS the field so the fresh metadata wins — `None` is the
# one value that needs no bound, and a clear is still a write.
# service.enqueue / enqueue_edit `display_name` is already bounded by its caller; the fallback
# slices the YouTube title to the same width.
# downloads.upload_download passes through `spool_upload`'s value, bounded there.
#
# Held as EXACT EQUALITY rather than as a list of offenders. The offender form could only fail
# upward: refactoring the writers into a shape the walker did not recognise left it asserting
# `not []` and enforcing nothing, which is how a drift guard dies quietly (round 3). This one fails
# when a site appears, when one disappears, and when one changes what it writes.
DISPLAY_WRITE_SITES = {
("auth.py", "get_or_create_demo_user"): {"'Demo'"},
("downloads/service.py", "redownload"): {"None"},
("downloads/service.py", "enqueue"): {
"display_name or (asset.title[:DISPLAY_TEXT_MAX] "
"if asset.status == 'ready' and asset.title else None)"
},
("downloads/service.py", "enqueue_edit"): {"display_name or asset.title[:DISPLAY_TEXT_MAX]"},
("downloads/upload.py", "ingest_upload"): {"display_text(display_name)"},
("routes/auth.py", "callback"): {"display_text(userinfo.get('name'))"},
("routes/auth.py", "_complete_link"): {"display_text(userinfo.get('name'))"},
("routes/downloads.py", "enqueue_download"): {"display_text(body.display_name)"},
("routes/downloads.py", "enqueue_edit"): {"display_text(body.display_name)"},
("routes/downloads.py", "update_download_meta"): {
"display_text(body.display_name)",
"display_text(body.display_uploader)",
},
("routes/downloads.py", "upload_download"): {"upload.display_name"},
("routes/downloads.py", "spool_upload"): {
"display_text(_header_text(request, UPLOAD_DISPLAY_NAME_HEADER))"
},
}
def test_no_code_path_assigns_a_display_field_without_the_shared_helper():
"""The drift guard for the test above: it holds five routes, this holds the rule.
Parsed, not grepped. Every value that reaches a display field takes a string a user typed (or
YouTube supplied) and puts it on its way to a varchar(255) column, so every one of them must be
bounded. A writer added tomorrow fails here instead of 500ing in production.
"""
found = _display_write_sites()
moved = set(found) ^ set(DISPLAY_WRITE_SITES)
changed = {
key: (found[key], DISPLAY_WRITE_SITES[key])
for key in set(found) & set(DISPLAY_WRITE_SITES)
if found[key] != DISPLAY_WRITE_SITES[key]
}
assert found == DISPLAY_WRITE_SITES, (
"a display-field write appeared, moved or changed shape. Every value reaching "
"display_name/display_uploader must be bounded to models.DISPLAY_TEXT_MAX — normally by "
"models.display_text — or a name longer than the column is a 500 rather than a truncation. "
f"Sites that appeared or vanished: {moved}. Sites that changed: {changed}."
)
def test_the_display_text_bound_matches_the_columns_it_guards():
"""A number living in two places is how the `cancelled`/`canceled` bug happened. `display_text`
truncates to `DISPLAY_TEXT_MAX`; the columns it feeds are `String(255)`. Held against the model
itself, so widening one without the other fails here rather than as a DataError in production."""
for name in ("display_name", "display_uploader"):
assert DownloadJob.__table__.c[name].type.length == DISPLAY_TEXT_MAX
def test_the_display_name_header_reaches_the_job_and_is_bounded(
as_user, make_user, _root, monkeypatch, upload_headers
):
"""The second header, which had no backend test at all — and the length nothing bounded.
Since P1 the display name arrives in a REQUEST HEADER rather than as a form field, so the only
limit on it was the header size. The column is varchar(255), so a longer one reached Postgres as
a DataError: a 500 AFTER the whole file had been transferred. It is truncated now, by the same
helper the metadata PATCH uses.
"""
_probe(monkeypatch, (None, None, None, None, None))
c = as_user(make_user())
ok = c.post(
"/api/downloads/upload",
content=b"data",
headers=upload_headers("clip.bin", "Nyári vakáció"),
)
assert ok.status_code == 200, ok.text
assert ok.json()["display_name"] == "Nyári vakáció"
long_name = "n" * 300
bounded = c.post(
"/api/downloads/upload", content=b"data2", headers=upload_headers("clip2.bin", long_name)
)
assert bounded.status_code == 200, bounded.text
assert bounded.json()["display_name"] == "n" * DISPLAY_TEXT_MAX
def test_the_upload_holds_no_db_transaction_while_the_body_streams(
as_user, make_user, db, _root, monkeypatch, upload_headers
):
"""P1 moved the body read behind the auth dependency, which put the whole transfer INSIDE the
request's database session: resolving the caller and reading the cap are SELECTs, SQLAlchemy 2.0
autobegins on the first one, and the session then holds one of the twenty pooled connections —
`idle in transaction` to Postgres, with no `idle_in_transaction_session_timeout` to end it — for
as long as the client keeps sending. `db.py` sizes the pool on that not happening and
`plex/playback.py` records the production incident behind the rule.
Asserted on the session rather than on `engine.pool.checkedout()`: the transaction is the cause,
the checkout is its consequence (measured), and the pool counter is shared with whatever else
the test process is holding.
"""
from app.routes import downloads as downloads_routes
_probe(monkeypatch, (None, None, None, None, None))
seen: dict[str, bool] = {}
real_stream = downloads_routes.capped_stream_to
real_release = downloads_routes.release_read_connection
def spy_release(session):
# The PRECONDITION, so this test cannot pass vacuously. Asserting only "no transaction
# while streaming" would stay green if the auth/cap reads stopped opening one at all —
# and under `as_user` the auth SELECT is already overridden away, so that is a live risk
# rather than a theoretical one.
seen["before"] = bool(session.in_transaction())
real_release(session)
seen["after_release"] = bool(session.in_transaction())
async def spy_stream(request, out, limit, refusal):
seen["while_streaming"] = bool(db.in_transaction())
return await real_stream(request, out, limit, refusal)
monkeypatch.setattr(downloads_routes, "release_read_connection", spy_release)
monkeypatch.setattr(downloads_routes, "capped_stream_to", spy_stream)
c = as_user(make_user())
r = c.post("/api/downloads/upload", content=b"data", headers=upload_headers("held.bin"))
assert r.status_code == 200, r.text
assert seen["before"] is True, (
"nothing had opened a transaction before the release, so this test proves nothing — the "
"reads it is meant to follow must have moved or been stubbed away"
)
assert seen["after_release"] is False
assert seen["while_streaming"] is False, (
"the request's session was inside a transaction when the transfer began, so a pooled "
"connection is pinned for the whole upload"
)
def test_the_usage_route_reports_the_per_file_upload_cap(as_user, make_user, db):
"""The dialog refuses an over-cap file off this number, and it is the only layer that produces
a refusal carrying the limit on a default install — so the field is load-bearing, not
decorative. Nothing held it: the route had no test at all, and the frontend's `DownloadUsage`
is an unvalidated cast, so dropping the key would have been green everywhere (round 2).
"""
from app import sysconfig
c = as_user(make_user())
r = c.get("/api/downloads/usage")
assert r.status_code == 200, r.text
assert r.json()["upload_max_bytes"] == sysconfig.get_int(db, "download_upload_max_bytes")
# And it is the KNOB, not a constant: a lowered override must show through.
sysconfig.set_value(db, "download_upload_max_bytes", 4096)
db.commit()
assert c.get("/api/downloads/usage").json()["upload_max_bytes"] == 4096
def test_a_refused_upload_leaves_nothing_in_the_staging_directory(
as_user, make_user, db, _root, monkeypatch, upload_headers
):
"""`spool_upload` writes the body to a staging temp file as it arrives, so a refusal happens
with a partial file already on disk — its `finally` is what removes it. Nothing asserted that:
deleting the unlink left the whole suite green while every refused upload leaked its partial
bytes onto the volume, one file per attempt, with no job to ever clean them up.
"""
from app import sysconfig
sysconfig.set_value(db, "download_upload_max_bytes", 8)
db.commit()
staging = Path(settings.download_root) / ".staging" / "uploads"
c = as_user(make_user())
r = c.post("/api/downloads/upload", content=b"x" * 4096, headers=upload_headers("leaky.bin"))
assert r.status_code == 422
assert staging.exists(), "the dependency did not get as far as creating the staging dir"
assert list(staging.iterdir()) == [], "a refused upload left its partial file behind"
def test_upload_cap_cannot_be_disabled_with_zero(db):
"""download_upload_max_bytes floors at 1: a stored 0 would read falsy in the route's
`if cap and size > cap` guard and SILENTLY disable the per-file cap, so set_value must reject it.
Pins the min=1 invariant so a revert to min=0 can't pass the gate."""
"""download_upload_max_bytes floors at 1, and set_value must enforce it.
The falsy-0 footgun this originally guarded is gone — the route's old `if cap and size > cap`
read a stored 0 as "no cap at all", while `capped_stream_to` compares unconditionally and a 0
would refuse everything instead. Refusing everything is the safe direction, but it is still
wrong, and the knob is also the number the admin page and the 422 `limit` advertise. Pins the
min=1 invariant so a revert to min=0 can't pass the gate."""
from app import sysconfig
with pytest.raises(ValueError):
sysconfig.set_value(db, "download_upload_max_bytes", 0)
def test_upload_route_rejects_empty_file(as_user, make_user, _root, monkeypatch):
def test_upload_route_rejects_empty_file(as_user, make_user, _root, monkeypatch, upload_headers):
c = as_user(make_user())
r = c.post(
"/api/downloads/upload", files={"file": ("empty.bin", b"", "application/octet-stream")}
)
r = c.post("/api/downloads/upload", content=b"", headers=upload_headers("empty.bin"))
assert r.status_code == 400
assert r.json()["detail"]["code"] == "upload_empty"
def test_redownload_rejects_an_upload(as_user, make_user, _root, monkeypatch):
def test_redownload_rejects_an_upload(as_user, make_user, _root, monkeypatch, upload_headers):
_probe(monkeypatch, (None, None, None, None, None))
c = as_user(make_user())
job = c.post(
"/api/downloads/upload", files={"file": ("a.bin", b"data", "application/octet-stream")}
).json()
job = c.post("/api/downloads/upload", content=b"data", headers=upload_headers("a.bin")).json()
r = c.post(f"/api/downloads/{job['id']}/redownload")
assert r.status_code == 409
def test_edit_rejects_a_generic_upload_source(as_user, make_user, _root, monkeypatch):
def test_edit_rejects_a_generic_upload_source(
as_user, make_user, _root, monkeypatch, upload_headers
):
_probe(monkeypatch, (None, None, None, None, None))
c = as_user(make_user())
job = c.post(
"/api/downloads/upload", files={"file": ("a.bin", b"data", "application/octet-stream")}
).json()
job = c.post("/api/downloads/upload", content=b"data", headers=upload_headers("a.bin")).json()
r = c.post(
"/api/downloads/edit",
json={"source_job_id": job["id"], "edit_spec": {"trim": {"start_s": 0, "end_s": 1}}},
+11
View File
@@ -3,6 +3,17 @@
How to run Siftlode locally and pass the gate. Conventions for the backend code itself live in
[`backend/app/README.md`](../backend/app/README.md).
## Node
One major, everywhere. `.nvmrc` pins it; the CI container, the `Dockerfile`'s frontend build stage,
`frontend/package.json`'s `engines` and `@types/node` must all match it — and so must the machine you
run the gate on. `frontend/src/lib/nodeEngine.contract.test.ts` fails the gate the moment any of them
drift apart, including when yours is the odd one out. `nvm use` in the repo root picks it up.
This is a contract, not a preference: tsc and vitest read the engine's own `Intl` and web APIs, and
those change between majors — the same currency format renders `$160M` on one and `$160.0M` on the
next. A gate that is green on a major you do not ship is green about nothing.
## Run it locally
The whole stack (Postgres + API + worker) comes up from the localdev compose file:
+22
View File
@@ -187,6 +187,28 @@ export default tseslint.config(
},
rules: {
...reactHooks.configs.recommended.rules,
// eslint-plugin-react-hooks 7's `recommended` is no longer two rules — it carries the whole
// React Compiler rule set (E27 S1). On this codebase that landed 283 findings, and they were
// triaged rather than bulk-suppressed, because the two answers are different:
//
// react-hooks/refs 217 — OFF, see below.
// the other 66 — real. They went into eslint-suppressions.json as a ratchet for one
// sprint and E27 S2 burned the file down to nothing: what survives is
// an inline directive at the site, each naming the invariant it
// protects. There is DELIBERATELY no suppressions file any more — a
// bulk `--suppress-all` would hide the next violation instead of
// answering it, and it cannot hold a warning-level rule anyway.
//
// `refs` is off because it fires on this app's popover architecture, not on its history, so
// suppressing it would not help the next popover anyone writes. Measured over the 217:
// 79 are `pop.refs.setReference` / `getReferenceProps` — @floating-ui/react's callback-ref
// setters, which are the documented way to use it — and 74 more are passing the popover handle
// down as a prop (`popover={pop}`, `ref={fade.ref}`), which is what `components/ui`'s popover
// engine is FOR. That is 153 of 217 flagged for having the word `refs` in an object that is not
// a React ref. The remaining ~64 are genuine render-phase `someRef.current = x` writes and are
// real (E27 S2 answered the ones the other rules could see, at their sites).
// Re-evaluate when the rule can: it shipped in 1.0.0 and this looks like over-approximation.
"react-hooks/refs": "off",
"local/jsx-comment-text": "error",
"local/focus-safe-disabled": "error",
"local/literal-a11y-text": "error",
+102 -72
View File
@@ -5,6 +5,7 @@
"packages": {
"": {
"name": "siftlode-frontend",
"license": "AGPL-3.0-only",
"dependencies": {
"@dnd-kit/core": "^6.3.1",
"@dnd-kit/sortable": "^10.0.0",
@@ -17,22 +18,23 @@
"hls.js": "^1.6.16",
"i18next": "^23.11.5",
"lucide-react": "^0.408.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-i18next": "^14.1.2"
"react": "19.3.0",
"react-dom": "19.3.0",
"react-i18next": "^15.7.4"
},
"devDependencies": {
"@eslint/js": "9.39.1",
"@playwright/test": "^1.61.1",
"@tailwindcss/postcss": "^4.3.3",
"@testing-library/react": "^16.3.2",
"@types/node": "^26.1.1",
"@types/react": "^18.3.3",
"@types/react-dom": "^18.3.0",
"@types/node": "^24.13.6",
"@types/react": "19.3.0",
"@types/react-dom": "19.3.0",
"@vitejs/plugin-react": "^4.3.1",
"babel-plugin-react-compiler": "1.0.0",
"eslint": "9.39.1",
"eslint-config-prettier": "10.1.8",
"eslint-plugin-react-hooks": "5.2.0",
"eslint-plugin-react-hooks": "7.1.1",
"eslint-plugin-react-refresh": "0.4.24",
"globals": "16.5.0",
"jsdom": "^25.0.1",
@@ -44,6 +46,9 @@
"typescript-eslint": "8.46.0",
"vite": "^5.3.4",
"vitest": "^3.2.7"
},
"engines": {
"node": "24.x"
}
},
"node_modules/@alloc/quick-lru": {
@@ -2909,41 +2914,33 @@
"license": "MIT"
},
"node_modules/@types/node": {
"version": "26.1.1",
"resolved": "https://registry.npmjs.org/@types/node/-/node-26.1.1.tgz",
"integrity": "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw==",
"version": "24.13.6",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.13.6.tgz",
"integrity": "sha512-SGrw/h3KPFshy3OE6ZL53LMBG5vGQQ8/gIpiqz/kRZhPJ7HgwCEs8LBuNtWLa8dvGZVpSF7+Bf+c11HUrCb/yg==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~8.3.0"
"undici-types": "~7.18.0"
}
},
"node_modules/@types/prop-types": {
"version": "15.7.15",
"resolved": "https://registry.npmjs.org/@types/prop-types/-/prop-types-15.7.15.tgz",
"integrity": "sha512-F6bEyamV9jKGAFBEmlQnesRPGOQqS2+Uwi0Em15xenOxHaf2hv6L8YCVn3rPdPJOiJfPiCnLIRyvwVaqMY3MIw==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/react": {
"version": "18.3.31",
"resolved": "https://registry.npmjs.org/@types/react/-/react-18.3.31.tgz",
"integrity": "sha512-vfEqpXTvwT91yhmwdfouStN2hSKwTvyRs8qpLfADyrq/kxDw0hZM7Wk9Ug1FELj8hIby+S/+kQCSRFF32nv2Qw==",
"version": "19.3.0",
"resolved": "https://registry.npmjs.org/@types/react/-/react-19.3.0.tgz",
"integrity": "sha512-N0rFCuH9YoxG9/m61l9MfpJKfmLOVU0em7ipIz6TRgSSkvReLB9vL85GB+yr8Bs5leqpvg96JSwF4ZS1s4viQg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/prop-types": "*",
"csstype": "^3.2.2"
}
},
"node_modules/@types/react-dom": {
"version": "18.3.7",
"resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-18.3.7.tgz",
"integrity": "sha512-MEe3UeoENYVFXzoXEWsvcpg6ZvlrFNlOQ7EOsvhI3CfAXwzPfO8Qwuxd40nepsYKqyyVQnTdEfv68q91yLcKrQ==",
"version": "19.3.0",
"resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-19.3.0.tgz",
"integrity": "sha512-ZI7bU42mZXXKHn/qNLEw2IrbiINU7X5+vfgdixBHkCNpYWXjKgfQ/P+uyGb5CjOLB9UcnTeg3rylQtV2hym44Q==",
"dev": true,
"license": "MIT",
"peerDependencies": {
"@types/react": "^18.0.0"
"@types/react": "^19.3.0"
}
},
"node_modules/@typescript-eslint/eslint-plugin": {
@@ -3465,6 +3462,16 @@
"dev": true,
"license": "MIT"
},
"node_modules/babel-plugin-react-compiler": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/babel-plugin-react-compiler/-/babel-plugin-react-compiler-1.0.0.tgz",
"integrity": "sha512-Ixm8tFfoKKIPYdCCKYTsqv+Fd4IJ0DQqMyEimo+pxUOMUR9cVPlwTrFt9Avu+3cb6Zp3mAzl+t1MrG2fxxKsxw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/types": "^7.26.0"
}
},
"node_modules/balanced-match": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
@@ -4080,16 +4087,23 @@
}
},
"node_modules/eslint-plugin-react-hooks": {
"version": "5.2.0",
"resolved": "https://registry.npmjs.org/eslint-plugin-react-hooks/-/eslint-plugin-react-hooks-5.2.0.tgz",
"integrity": "sha512-+f15FfK64YQwZdJNELETdn5ibXEUQmW1DZL6KXhNnc2heoy/sg9VJJeT7n8TlMWouzWqSWavFkIhHyIbIAEapg==",
"version": "7.1.1",
"resolved": "https://registry.npmjs.org/eslint-plugin-react-hooks/-/eslint-plugin-react-hooks-7.1.1.tgz",
"integrity": "sha512-f2I7Gw6JbvCexzIInuSbZpfdQ44D7iqdWX01FKLvrPgqxoE7oMj8clOfto8U6vYiz4yd5oKu39rRSVOe1zRu0g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/core": "^7.24.4",
"@babel/parser": "^7.24.4",
"hermes-parser": "^0.25.1",
"zod": "^3.25.0 || ^4.0.0",
"zod-validation-error": "^3.5.0 || ^4.0.0"
},
"engines": {
"node": ">=10"
"node": ">=18"
},
"peerDependencies": {
"eslint": "^3.0.0 || ^4.0.0 || ^5.0.0 || ^6.0.0 || ^7.0.0 || ^8.0.0-0 || ^9.0.0"
"eslint": "^3.0.0 || ^4.0.0 || ^5.0.0 || ^6.0.0 || ^7.0.0 || ^8.0.0-0 || ^9.0.0 || ^10.0.0"
}
},
"node_modules/eslint-plugin-react-refresh": {
@@ -4582,6 +4596,23 @@
"node": ">= 0.4"
}
},
"node_modules/hermes-estree": {
"version": "0.25.1",
"resolved": "https://registry.npmjs.org/hermes-estree/-/hermes-estree-0.25.1.tgz",
"integrity": "sha512-0wUoCcLp+5Ev5pDW2OriHC2MJCbwLwuRx+gAqMTOkGKJJiBCLjtrvy4PWUGn6MIVefecRpzoOZ/UV6iGdOr+Cw==",
"dev": true,
"license": "MIT"
},
"node_modules/hermes-parser": {
"version": "0.25.1",
"resolved": "https://registry.npmjs.org/hermes-parser/-/hermes-parser-0.25.1.tgz",
"integrity": "sha512-6pEjquH3rqaI6cYAXYPcz9MS4rY6R4ngRgrgfDshRptUZIc3lw0MCIJIGDj9++mfySOuPTHB4nrSW99BCvOPIA==",
"dev": true,
"license": "MIT",
"dependencies": {
"hermes-estree": "0.25.1"
}
},
"node_modules/hls.js": {
"version": "1.6.16",
"resolved": "https://registry.npmjs.org/hls.js/-/hls.js-1.6.16.tgz",
@@ -4772,6 +4803,7 @@
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz",
"integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==",
"dev": true,
"license": "MIT"
},
"node_modules/js-yaml": {
@@ -5275,18 +5307,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/loose-envify": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.4.0.tgz",
"integrity": "sha512-lyuxPGr/Wfhrlem2CL/UcnUc1zcqKAImBDzukY7Y5F/yQiNdko6+fRLevlw1HgMySw7f611UIY408EtxRSoK3Q==",
"license": "MIT",
"dependencies": {
"js-tokens": "^3.0.0 || ^4.0.0"
},
"bin": {
"loose-envify": "cli.js"
}
},
"node_modules/loupe": {
"version": "3.2.1",
"resolved": "https://registry.npmjs.org/loupe/-/loupe-3.2.1.tgz",
@@ -5820,42 +5840,39 @@
"license": "MIT"
},
"node_modules/react": {
"version": "18.3.1",
"resolved": "https://registry.npmjs.org/react/-/react-18.3.1.tgz",
"integrity": "sha512-wS+hAgJShR0KhEvPJArfuPVN1+Hz1t0Y6n5jLrGQbkb4urgPE/0Rve+1kMB1v/oWgHgm4WIcV+i7F2pTVj+2iQ==",
"version": "19.3.0",
"resolved": "https://registry.npmjs.org/react/-/react-19.3.0.tgz",
"integrity": "sha512-E8LUcbtBWt20bbl2YoHfx4ZDBdxVTfOKtCZn9cDSJ4l6/nuoApcpIBcj47t2wZoVX8g2ZHuMHbiShgCR1T5Sog==",
"license": "MIT",
"dependencies": {
"loose-envify": "^1.1.0"
},
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/react-dom": {
"version": "18.3.1",
"resolved": "https://registry.npmjs.org/react-dom/-/react-dom-18.3.1.tgz",
"integrity": "sha512-5m4nQKp+rZRb09LNH59GM4BxTh9251/ylbKIbpe7TpGxfJ+9kv6BLkLBXIjjspbgbnIBNqlI23tRnTWT0snUIw==",
"version": "19.3.0",
"resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.3.0.tgz",
"integrity": "sha512-JDk8dgif51OjFoDE70+OT9ICyYr+69HlmihNwp1+Nsfbna3t5sIiCa9ZJktDmQ4/1b/rn26hIAR2uYXDMr5r0Q==",
"license": "MIT",
"dependencies": {
"loose-envify": "^1.1.0",
"scheduler": "^0.23.2"
"scheduler": "^0.28.0"
},
"peerDependencies": {
"react": "^18.3.1"
"react": "^19.3.0"
}
},
"node_modules/react-i18next": {
"version": "14.1.3",
"resolved": "https://registry.npmjs.org/react-i18next/-/react-i18next-14.1.3.tgz",
"integrity": "sha512-wZnpfunU6UIAiJ+bxwOiTmBOAaB14ha97MjOEnLGac2RJ+h/maIYXZuTHlmyqQVX1UVHmU1YDTQ5vxLmwfXTjw==",
"version": "15.7.4",
"resolved": "https://registry.npmjs.org/react-i18next/-/react-i18next-15.7.4.tgz",
"integrity": "sha512-nyU8iKNrI5uDJch0z9+Y5XEr34b0wkyYj3Rp+tfbahxtlswxSCjcUL9H0nqXo9IR3/t5Y5PKIA3fx3MfUyR9Xw==",
"license": "MIT",
"dependencies": {
"@babel/runtime": "^7.23.9",
"@babel/runtime": "^7.27.6",
"html-parse-stringify": "^3.0.1"
},
"peerDependencies": {
"i18next": ">= 23.2.3",
"react": ">= 16.8.0"
"i18next": ">= 23.4.0",
"react": ">= 16.8.0",
"typescript": "^5"
},
"peerDependenciesMeta": {
"react-dom": {
@@ -5863,6 +5880,9 @@
},
"react-native": {
"optional": true
},
"typescript": {
"optional": true
}
}
},
@@ -6012,13 +6032,10 @@
}
},
"node_modules/scheduler": {
"version": "0.23.2",
"resolved": "https://registry.npmjs.org/scheduler/-/scheduler-0.23.2.tgz",
"integrity": "sha512-UOShsPwz7NrMUqhR6t0hWjFduvOzbtv7toDH1/hIrfRNIDBnnBWd0CwJTGvTpngVlmwGCdP9/Zl/tVrDqcuYzQ==",
"license": "MIT",
"dependencies": {
"loose-envify": "^1.1.0"
}
"version": "0.28.0",
"resolved": "https://registry.npmjs.org/scheduler/-/scheduler-0.28.0.tgz",
"integrity": "sha512-juorfCmIkIw8tT+p5BXSm6PJjQF/ycEYmKyzURCIt/RaZIhL+PulbQ9Yu2z1HdOJDdqDTlxA1+xKBmHXJsczAw==",
"license": "MIT"
},
"node_modules/semver": {
"version": "6.3.1",
@@ -6364,7 +6381,7 @@
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"dev": true,
"devOptional": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
@@ -6409,9 +6426,9 @@
}
},
"node_modules/undici-types": {
"version": "8.3.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.3.0.tgz",
"integrity": "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==",
"version": "7.18.2",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.18.2.tgz",
"integrity": "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==",
"dev": true,
"license": "MIT"
},
@@ -6832,6 +6849,19 @@
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
},
"node_modules/zod-validation-error": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/zod-validation-error/-/zod-validation-error-4.0.2.tgz",
"integrity": "sha512-Q6/nZLe6jxuU80qb/4uJ4t5v2VEZ44lzQjPDhYJNztRQ4wyWc6VF3D3Kb/fAuPetZQnhS3hnajCf9CsWesghLQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=18.0.0"
},
"peerDependencies": {
"zod": "^3.25.0 || ^4.0.0"
}
}
}
}
+11 -7
View File
@@ -3,6 +3,9 @@
"private": true,
"license": "AGPL-3.0-only",
"type": "module",
"engines": {
"node": "24.x"
},
"scripts": {
"dev": "vite",
"typecheck": "npm run typecheck:app && npm run typecheck:node && npm run typecheck:e2e",
@@ -31,22 +34,23 @@
"hls.js": "^1.6.16",
"i18next": "^23.11.5",
"lucide-react": "^0.408.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-i18next": "^14.1.2"
"react": "19.3.0",
"react-dom": "19.3.0",
"react-i18next": "^15.7.4"
},
"devDependencies": {
"@eslint/js": "9.39.1",
"@playwright/test": "^1.61.1",
"@tailwindcss/postcss": "^4.3.3",
"@testing-library/react": "^16.3.2",
"@types/node": "^26.1.1",
"@types/react": "^18.3.3",
"@types/react-dom": "^18.3.0",
"@types/node": "^24.13.6",
"@types/react": "19.3.0",
"@types/react-dom": "19.3.0",
"@vitejs/plugin-react": "^4.3.1",
"babel-plugin-react-compiler": "1.0.0",
"eslint": "9.39.1",
"eslint-config-prettier": "10.1.8",
"eslint-plugin-react-hooks": "5.2.0",
"eslint-plugin-react-hooks": "7.1.1",
"eslint-plugin-react-refresh": "0.4.24",
"globals": "16.5.0",
"jsdom": "^25.0.1",
+26
View File
@@ -0,0 +1,26 @@
import react from "@vitejs/plugin-react";
import reactCompiler from "babel-plugin-react-compiler";
// ONE React transform, shared by vite.config.ts and vitest.config.ts (E27 S1).
//
// It is a shared factory rather than two `react()` calls because the two configs must not drift:
// if the compiler ran only in the production build, every vitest run would be exercising code the
// user never receives, and a compiler-induced behaviour change would ship green. That is precisely
// the defect `0.93.2` was released to fix — a gate and a build image that had quietly stopped being
// the same thing — so the rule here is that the gate compiles the code the same way the build does.
//
// The React Compiler auto-memoises: it infers what `useCallback`/`useMemo` were being hand-written
// for and applies the equivalent itself. It is enabled repo-wide with no opt-out list because it
// bails out on its own for anything it cannot prove safe (its rules are also linted, via
// eslint-plugin-react-hooks). `target` follows the installed React major and must move with it.
//
// The plugin is IMPORTED rather than named by the usual `"babel-plugin-react-compiler"` string. Two
// reasons, both about the gate: a string is invisible to knip, which then reports the dependency as
// unused, and — worse — a typo in that string is a silent no-op that ships an uncompiled bundle,
// while a bad import is a resolve error on the tsc lane.
export const reactWithCompiler = () =>
react({
babel: {
plugins: [[reactCompiler, { target: "19" }]],
},
});
+80 -33
View File
@@ -1,4 +1,4 @@
import { lazy, Suspense, useEffect, useState } from "react";
import { lazy, Suspense, useEffect, useRef, useState } from "react";
import { qk } from "./lib/queryKeys";
import { useTranslation } from "react-i18next";
import { useQuery, useQueryClient } from "@tanstack/react-query";
@@ -34,6 +34,7 @@ import BackToTop from "./components/ui/BackToTop";
import PageShell from "./components/ui/PageShell";
import GlassTuner from "./components/GlassTuner";
import ChatDock from "./components/ChatDock";
import DebugRecorder from "./components/DebugRecorder";
import Toaster from "./components/ui/Toaster";
import KeyboardHelp from "./components/ui/KeyboardHelp";
import AppCommandPalette from "./components/AppCommandPalette";
@@ -130,6 +131,9 @@ export default function App() {
});
const needsSetup = setupQuery.data?.configured === false;
const meQuery = useQuery({ queryKey: qk.me(), queryFn: api.me, enabled: false });
// The signed-in account, or null/undefined while signed out. Hoisted here because half a dozen
// effects below depend on it, and a dep list is only honest if it names the value the body reads.
const me = meQuery.data;
// The `global` key scope: `h` opens the cheat sheet, Mod+K the jump-to palette, Shift+W jumps to
// Home ("mit nézzek?"). Bound only once signed in, because that is where the overlays are mounted
@@ -140,26 +144,40 @@ export default function App() {
// page underneath (owner decision 2026-09-09: the key always ends with the answer on screen).
// The `page` scope rides the same binding: the `g` chords jump to a module, but only to one this
// account can reach (the nav rail's own list, plus Settings, which is not in the rail's order).
useGlobalKeymap(
!!meQuery.data,
setPage,
meQuery.data ? [...moduleOrder(meQuery.data), "settings"] : [],
);
useGlobalKeymap(!!me, setPage, me ? [...moduleOrder(me), "settings"] : []);
// The account's consented diagnostics level (E22 S5), applied to this tab's log: raised, the log is
// kept in more detail and streamed to the server until the level lapses. A lapse (its own timer, or
// the server saying it is off) refetches `me`, so Settings shows the base level again.
const diagnostics = meQuery.data?.diagnostics;
// kept in more detail and streamed to the server until the level lapses. A debug session that is
// recording (E16) raises it for its own run and tags what it keeps. A lapse (its own timer, or the
// server saying it is off) refetches `me`, so Settings and the recorder show what is in force.
// Re-running this on every `me` refetch is FREE and therefore the deps are the honest ones:
// applyDiagnostics computes an effective-level key and returns early when it has not moved
// (lib/diagnostics.ts), so an unchanged level is a no-op. It does refresh the `lapsed` callback
// before that early return, which is the half we WANT re-run. The old dep list picked out four
// primitive fields to dodge the fresh object identity a refetch produces; that is the checker's
// job to verify, not a comment's to assert, and it silently missed any other field the module
// may come to read.
const diagnostics = me?.diagnostics;
const debugSession = me?.debug_session;
useEffect(() => {
applyDiagnostics(diagnostics, () => void queryClient.invalidateQueries({ queryKey: qk.me() }));
}, [diagnostics?.level, diagnostics?.until]); // eslint-disable-line react-hooks/exhaustive-deps
applyDiagnostics(
diagnostics,
debugSession,
() => void queryClient.invalidateQueries({ queryKey: qk.me() }),
);
}, [diagnostics, debugSession, queryClient]);
// Fetch the code of every module this account can open — and the channel page — once the first page
// is on screen, so a module's first visit renders at once instead of flashing the chunk fallback.
// Idle-scheduled so it never competes with the first page's own requests (Safari has no
// requestIdleCallback, hence the timer).
// Depending on the whole `me` rather than its id is both honest and MORE correct: the module set
// is derived from `me` through each module's own gate (lib/modules moduleOrder), so a permission
// change — an admin grant, a Plex server appearing — alters it without the id moving, and the old
// `[me.id]` dep never re-preloaded for those. Re-running costs nothing: `preload()` is idempotent
// (lazyWithPreload memoises the pending promise), so a refetch that changes nothing fetches
// nothing; only the idle callback is re-scheduled.
useEffect(() => {
const me = meQuery.data;
if (!me) return;
const run = () => {
preloadPages([...moduleOrder(me), "settings"]);
@@ -172,7 +190,7 @@ export default function App() {
}
const id = window.setTimeout(run, 1000);
return () => window.clearTimeout(id);
}, [meQuery.data?.id]); // eslint-disable-line react-hooks/exhaustive-deps
}, [me]);
// Measure the scrollbar's gutter width once and publish it as --sbw, so the floating header and the
// toolbar band can pad their right edge to line up with the scrolled cards (the app scroller reserves
@@ -190,9 +208,13 @@ export default function App() {
// Once signed in, drop any leftover ONE-TIME query params from the pre-login flow (?access, ?verify,
// ?reset) — but NOT the nav/filter params, which the logged-in app now reads from and keeps in the
// URL. Gated on auth so the logged-out Welcome page still reads its params first.
// Keyed on "is there a session at all", which is the whole condition the body tests. Re-running
// on an account switch was never the point — by then the params are already gone and a second
// clearParams is a no-op.
const signedIn = !!me;
useEffect(() => {
if (meQuery.data) clearParams("access", "verify", "reset");
}, [meQuery.data?.id]); // eslint-disable-line react-hooks/exhaustive-deps
if (signedIn) clearParams("access", "verify", "reset");
}, [signedIn]);
// Any request that 401s means the session ended server-side. If we were signed in (cached
// `me` is a user object), reload to the login page at once (also covers any click that hits
@@ -211,12 +233,20 @@ export default function App() {
// Returning from an in-app Google link/upgrade round-trip (auth.py redirects to ?link=…).
// Surface the outcome, refresh `me` (can_read/can_write/has_google may have flipped), land on
// Settings → Account so the new state is visible, then strip the param for a clean URL.
// "Exactly once per mount" is now ENFORCED by the ref rather than asserted by an empty dep list:
// `t` changes identity on a language switch, so the honest dep list would otherwise re-fire the
// toast when the user changes language. (The old `[]` got away with it only because clearParams
// had already removed the param — an invariant held by a side effect two lines further down.)
const linkAnswered = useRef(false);
const { refetch: refetchMe } = meQuery;
useEffect(() => {
if (linkAnswered.current) return;
const result = new URLSearchParams(window.location.search).get("link");
if (!result) return;
linkAnswered.current = true;
if (result === "ok") {
notify({ level: "success", message: t("settings.account.googleLink.linked") });
void meQuery.refetch();
void refetchMe();
setAccountRaw(LS.settingsTab, "account");
setPage("settings");
} else {
@@ -224,42 +254,59 @@ export default function App() {
notify({ level: "error", message: t(`settings.account.googleLink.${key}`) });
}
clearParams("link");
}, []); // eslint-disable-line react-hooks/exhaustive-deps
}, [t, setPage, refetchMe]);
// A Google OAuth flow started while already signed in (add-another-account, link, or upgrade) was
// cancelled at Google's consent screen. The backend only emits this for a live session, so a
// neutral toast is the right feedback — the pre-auth login banner would never render here.
// Same one-shot shape as the `?link=` handler above, same reason for the ref.
const oauthAnswered = useRef(false);
useEffect(() => {
if (oauthAnswered.current) return;
if (new URLSearchParams(window.location.search).get("oauth") !== "cancelled") return;
oauthAnswered.current = true;
notify({ level: "info", message: t("settings.account.googleCancelled") });
clearParams("oauth");
}, []); // eslint-disable-line react-hooks/exhaustive-deps
}, [t]);
// Deep-link from the "new access request" admin email (?admin=access-requests) → jump straight to
// the admin Users → Access requests view. Snapshot the param up front (read once at mount); act once
// `me` is known, and only for an admin (others just land on their default page).
const [adminDeepLink] = useState(() => new URLSearchParams(window.location.search).get("admin"));
// Ref-guarded like the two above, and here it closes a real hole rather than only satisfying the
// checker: the snapshot lives for the whole session, so with honest deps a later role change —
// or any `me` refetch, under the old id-only key — would yank the user to Users long after the
// deep link was answered.
const adminDeepLinkAnswered = useRef(false);
useEffect(() => {
if (adminDeepLink !== "access-requests" || !meQuery.data) return;
if (meQuery.data.role === "admin") {
if (adminDeepLinkAnswered.current) return;
if (adminDeepLink !== "access-requests" || !me) return;
adminDeepLinkAnswered.current = true;
if (me.role === "admin") {
focusAccessRequestsTab();
setPage("users");
}
clearParams("admin");
}, [adminDeepLink, meQuery.data?.id]); // eslint-disable-line react-hooks/exhaustive-deps
}, [adminDeepLink, me, setPage]);
// Nudge admins when access requests are waiting (in lieu of a server-push bell). Keyed on the
// pending COUNT so it re-resolves the moment an approval/denial changes it — not only on a
// fresh load (which is why the notice used to linger until F5 after an approval).
// Honest deps, and `t` among them is a fix rather than a concession: the nudge is a persisted
// toast whose text is translated at issue time, so a language switch must re-issue it. It did not
// before — the notice stayed in the old language until the count changed.
const role = me?.role;
const pendingInvites = me?.pending_invites ?? 0;
const pendingActivations = me?.pending_activations ?? 0;
useEffect(() => {
if (meQuery.data?.role !== "admin") return;
if (role !== "admin") return;
// Keep a single, current nudge: drop any prior one (persisted copies reload as dismissed
// history, so they'd otherwise pile up one-per-reload) before re-issuing the live notice.
removeByMetaKind("access-requests");
// Both kinds of "someone is waiting on you": a request needing a decision, and a sign-up whose
// address was approved before they registered. The second used to produce NO signal at all —
// nothing lands in the queue for it — which is how the 2026-09-03 registrant went unnoticed.
const pending = (meQuery.data.pending_invites ?? 0) + (meQuery.data.pending_activations ?? 0);
const pending = pendingInvites + pendingActivations;
if (pending > 0) {
notify({
level: "warning",
@@ -278,8 +325,7 @@ export default function App() {
},
});
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [meQuery.data?.role, meQuery.data?.pending_invites, meQuery.data?.pending_activations]);
}, [role, pendingInvites, pendingActivations, t, setPage]);
// Language is edited outside the Settings page (sidebar/login), so it stays instant.
function changeLanguage(code: LangCode) {
@@ -295,7 +341,7 @@ export default function App() {
if (needsSetup) return <SetupWizard />;
// /api/me answers 200 always (null body = logged out), and BootGate has already handled a failed
// probe — so no data here means "not signed in" → the public landing.
if (!meQuery.data) return <Welcome verify={verifyEntry} />;
if (!me) return <Welcome verify={verifyEntry} />;
// Shown while a page's chunk is still on its way: only if its visit beats the preload above (or the
// preload failed), and like every loading state it stays invisible for a load that is quick anyway.
@@ -316,16 +362,15 @@ export default function App() {
// page the account can't reach (a gated module via stale nav / a deep link) falls back to the feed,
// exactly as the old ternary's fallthrough did. A channel page overlays the content column, so no
// side rail shows behind it.
const me = meQuery.data!;
const pageGate = moduleDef(page).gate;
const activePage: Page = !pageGate || pageGate(me) ? page : "feed";
const activePage: Page = !pageGate || pageGate(me!) ? page : "feed";
const { Component: PageContent, maxWidth } = PAGE_CONTENT[activePage];
const railDef = channelView ? undefined : PAGE_RAIL[activePage];
return (
<div className="h-screen flex text-fg">
<NavSidebar
me={meQuery.data!}
me={me!}
onOpenAbout={() => setAboutOpen(true)}
onChangeLanguage={changeLanguage}
language={i18n.language as LangCode}
@@ -361,7 +406,7 @@ export default function App() {
key={channelView.id}
channelId={channelView.id}
initialName={channelView.name}
me={meQuery.data!}
me={me!}
view={view}
setView={setView}
onShowInFeed={() => {
@@ -386,11 +431,11 @@ export default function App() {
<PageShell
scrollKey={normalScrollKey}
maxWidth={maxWidth}
header={<Header me={meQuery.data!} onYtSearch={enterYtSearch} />}
header={<Header me={me!} onYtSearch={enterYtSearch} />}
banners={
<>
<VerifyBanner entry={verifyEntry} />
{meQuery.data!.is_demo && <DemoBanner />}
{me!.is_demo && <DemoBanner />}
<VersionBanner onOpen={() => openReleaseNotes(FRONTEND_VERSION)} />
</>
}
@@ -410,7 +455,9 @@ export default function App() {
"take me to X" the same way. Their keys are bound by useGlobalKeymap above. */}
<KeyboardHelp />
<AppCommandPalette />
{meQuery.data && !meQuery.data.is_demo && <ChatDock meId={meQuery.data.id} />}
{me && !me.is_demo && <ChatDock meId={me.id} />}
{/* The debug recorder (E16): its start dialog, and its pill docked under the toasts. */}
{me && !me.is_demo && <DebugRecorder />}
<Suspense fallback={null}>
{aboutOpen && (
<About
@@ -10,7 +10,9 @@ import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { CommandGroup } from "./ui/CommandPalette";
vi.mock("react-i18next", () => ({
vi.mock("react-i18next", async (importOriginal) => ({
// The rest of the module too: the recorder's actions reach `i18n/index`, which needs it.
...(await importOriginal<typeof import("react-i18next")>()),
useTranslation: () => ({ t: (k: string) => k, i18n: { language: "en" } }),
}));
@@ -140,4 +142,21 @@ describe("AppCommandPalette", () => {
then?.();
expect(requestSurprise).toHaveBeenCalledTimes(1);
});
it("offers the debug recorder by its state, and not to the demo account", () => {
const recorder = () => ids("actions").filter((id) => id.startsWith("action:recorder:"));
mount();
expect(recorder()).toEqual(["action:recorder:start"]);
cleanup();
const until = new Date(Date.now() + 60_000).toISOString();
me = user({ debug_session: { id: "0123456789abcdef", until, ended_at: null } });
mount();
expect(recorder()).toEqual(["action:recorder:step", "action:recorder:stop"]);
cleanup();
me = user({ is_demo: true });
mount();
expect(recorder()).toEqual([]);
});
});
+36 -1
View File
@@ -1,7 +1,7 @@
import { useMemo } from "react";
import { useTranslation } from "react-i18next";
import { useQuery } from "@tanstack/react-query";
import { Keyboard, ListVideo, Palette, Sparkles, Tv } from "lucide-react";
import { Bug, Flag, Keyboard, ListVideo, Palette, Sparkles, Square, Tv } from "lucide-react";
import CommandPalette, { type CommandGroup } from "./ui/CommandPalette";
import { api } from "../lib/api";
import { qk } from "../lib/queryKeys";
@@ -10,6 +10,8 @@ import { moduleDef, moduleOrder } from "../lib/modules";
import { SCHEMES, type Scheme } from "../lib/theme";
import { closePalette, toggleKeyboardHelp, usePaletteOpen } from "../lib/keymap";
import { requestSurprise } from "../lib/homeIntent";
import { openRecorderStart } from "../lib/debugRecorder";
import { useDebugRecording } from "./useDebugRecording";
import { useNavigationActions } from "./navigationContext";
import { usePlaylistsActions } from "./playlistsContext";
import { usePrefs } from "./prefsContext";
@@ -34,6 +36,8 @@ export default function AppCommandPalette() {
const { setSelectedId } = usePlaylistsActions();
const { theme, setTheme } = usePrefs();
const savedViews = useSavedViews();
const { session: recordingSession, stop: stopRecording, markStep } = useDebugRecording();
const stopMutate = stopRecording.mutate;
// Only fetch the lists while the palette is actually open — it is a rarely-opened surface and
// these are otherwise the feed's and the playlist rail's own queries, shared through the cache.
@@ -147,6 +151,34 @@ export default function AppCommandPalette() {
icon: <Keyboard className="w-4 h-4" />,
run: toggleKeyboardHelp,
},
// The debug recorder (E16) — not for the shared demo account, which cannot record. While
// it runs, Stop and a step marker take Start's place: the palette is reachable inside a
// player or a dialog where the pill's buttons are a long Tab away.
...(me?.is_demo
? []
: recordingSession
? [
{
id: "action:recorder:step",
label: t("recorder.palette.step"),
icon: <Flag className="w-4 h-4" />,
run: () => markStep(""),
},
{
id: "action:recorder:stop",
label: t("recorder.palette.stop"),
icon: <Square className="w-4 h-4" />,
run: () => stopMutate(recordingSession.id),
},
]
: [
{
id: "action:recorder:start",
label: t("recorder.palette.start"),
icon: <Bug className="w-4 h-4" />,
run: openRecorderStart,
},
]),
...SCHEMES.map((s) => ({
id: `action:theme:${s.id}`,
label: t("palette.theme", { name: s.name }),
@@ -166,6 +198,9 @@ export default function AppCommandPalette() {
savedViews,
theme,
setTheme,
recordingSession,
markStep,
stopMutate,
setPage,
openChannel,
setSelectedId,
@@ -191,6 +191,26 @@ describe("BootGate", () => {
expect(sessionStorage.getItem(ACTIVE_ACCOUNT_KEY)).toBe("5");
});
// The adoption effect is idempotent for the identity it already adopted, and that is load-bearing
// rather than tidy: a re-run reloads the page, so an effect that cannot tell "already adopted this
// account" from "adopted a different one" turns an ordinary boot into a reload loop. The spy is
// installed BEFORE mounting on purpose — the other reload tests stub it after the app is up, which
// is exactly late enough to miss the re-run that setting `readyFor` triggers.
it("a plain boot never reloads, however many times the adoption effect re-runs", async () => {
const reload = vi.fn();
vi.stubGlobal("location", { ...window.location, reload });
vi.spyOn(api, "me").mockResolvedValue(account());
const qc = mount();
await screen.findByTestId("app");
// Anything that re-renders the gate must leave it alone: a pref save, a background refetch.
act(() => qc.setQueryData(qk.me(), account({ preferences: { startup: "last" } })));
await act(() => new Promise((r) => setTimeout(r, 20)));
expect(reload).not.toHaveBeenCalled();
expect(mounts).toBe(1);
});
it("a change to the account's data, not its identity, leaves the app as it is", async () => {
vi.spyOn(api, "me").mockResolvedValue(account());
const qc = mount();
+19 -11
View File
@@ -1,4 +1,4 @@
import { useLayoutEffect, useState, type ReactNode } from "react";
import { useLayoutEffect, useRef, useState, type ReactNode } from "react";
import { useQuery } from "@tanstack/react-query";
import { useTranslation } from "react-i18next";
import { api, getActiveAccount, setActiveAccount } from "../lib/api";
@@ -83,23 +83,31 @@ export default function BootGate({ children }: { children: ReactNode }) {
// • Adopt the account's language, which follows the account across devices — the local copy only
// knows what this browser last showed.
const [readyFor, setReadyFor] = useState<number | null | undefined>(undefined);
// `me` is read for the identity it produced, so it rides a ref: a background refetch of the same
// account must not re-run the adoption over a language the user has just changed. That rule used
// to live in a dep list that excluded `me`, which made the dep list load-bearing for correctness —
// and the failure mode of getting it wrong was a page-reload loop, not a stale value.
const meRef = useRef(me);
meRef.current = me;
useLayoutEffect(() => {
if (identity === undefined) return;
if (readyFor !== undefined) {
// The tree was built for another identity — see the header. The pin stays as it is: the tree
// being torn down still runs its unmount saves before the page goes, and with the pin moved they
// would land under the new account's local keys. The reloaded page's first probe corrects it.
location.reload();
// Already adopted. Only a DIFFERENT identity reloads: the tree was built for another account —
// see the header. The pin stays as it is: the tree being torn down still runs its unmount saves
// before the page goes, and with the pin moved they would land under the new account's local
// keys. The reloaded page's first probe corrects it. Comparing rather than merely testing for
// "set" is what makes this effect idempotent, so a re-run for any other reason is a no-op
// instead of a reload.
if (readyFor !== identity) location.reload();
return;
}
if (me && getActiveAccount() !== me.id) setActiveAccount(me.id);
const lang = me?.preferences?.language;
const account = meRef.current;
if (account && getActiveAccount() !== account.id) setActiveAccount(account.id);
const lang = account?.preferences?.language;
if (typeof lang === "string" && isSupported(lang) && lang !== i18n.language) setLanguage(lang);
// eslint-disable-next-line react-hooks/set-state-in-effect -- the adoption latch. It cannot be derived: the whole point is that the tree mounts only AFTER the pin and the language have been applied, and both are writes to things outside React (sessionStorage, i18n). A layout effect so the frame that first shows the app already has them.
setReadyFor(identity);
// `me` is read for the identity it produced; a background refetch of the same account must not
// re-run the adoption over a language the user has just changed.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [identity]);
}, [identity, readyFor]);
// No answer at all (a network or 5xx failure on the very first setup or account probe — past the
// one retry `req` already makes): say so, with a retry. An unanswered setup probe is NOT "signed
+8 -1
View File
@@ -95,7 +95,13 @@ export default function ChannelPage({
intervalMs: channelDetailPollMs,
});
const ch = detail.data;
useEffect(() => setBannerAttempt(0), [ch?.banner_url]); // fresh channel/banner → try again
// A fresh channel/banner gets the retries back, in the render that changes it: as an effect the
// new banner was committed with the previous one's exhausted attempt count still in place.
const [bannerFor, setBannerFor] = useState(ch?.banner_url);
if (bannerFor !== ch?.banner_url) {
setBannerFor(ch?.banner_url);
setBannerAttempt(0);
}
// The poll above is also the ONLY signal that the running backfill has written videos, so it
// drives the video grid's refresh too (the "Get full history" mutation below can't: it fires on
// the PATCH response, before the background job has stored anything).
@@ -137,6 +143,7 @@ export default function ChannelPage({
if (ch.subscribed || ch.blocked) return;
if (ch.explored && ch.stored_videos > 0) return;
autoTried.current = channelId;
// eslint-disable-next-line react-hooks/set-state-in-effect -- a one-off INGEST of an unsubscribed channel's uploads on first visit — a network side effect, not state. The autoTried ref keyed on the channel id is what makes it once per channel.
void runExplore();
}, [ch, channelId, me.is_demo, runExplore]);
+28 -13
View File
@@ -1,4 +1,4 @@
import { useEffect, useRef, useState } from "react";
import { useRef, useState } from "react";
import { Check, Plus, Sparkles, X } from "lucide-react";
import { useTranslation } from "react-i18next";
import { type Tag } from "../lib/api";
@@ -117,18 +117,28 @@ function SuggestedChipsLane({
decide: (g) =>
fitChipsWithPill({ chipRights: g.chipRights, pillW: g.pillW, avail: g.avail, gap: g.gap }),
});
const hidden = suggestions.length - visible;
// Clamped, and not defensively: `useChipOverflowFit` keeps `visible` in state and re-decides in a
// layout effect, so the render in which the chip set SHRINKS still holds the previous measurement
// and a raw subtraction goes negative. An unclamped `hidden` slips past the `=== 0` reset below —
// the flag survives exactly the window it exists to close, and the popover springs back on the
// next re-fit.
const hidden = Math.max(0, suggestions.length - visible);
const [overflowOpen, setOverflowOpen] = useState(false);
// Open only while there is a "+N" trigger to anchor it — derived, not closed by an effect, so the
// frame in which the last collapsed chip fits again cannot also hold a menu pointing at a
// reference that has gone. Same shape as TagFilterChips and the second row below, INCLUDING the
// render-phase reset: deriving hides the card but never clears the flag (a hover card whose pill
// unmounts under the cursor gets no mouseleave), so without this it springs back the next time
// the lane overflows. Convergent — the write falsifies its own condition.
const [overflowOpenRaw, setOverflowOpen] = useState(false);
if (overflowOpenRaw && hidden === 0) setOverflowOpen(false);
const overflowOpen = overflowOpenRaw && hidden > 0;
const overflowPop = useAnchoredPopover({
open: overflowOpen,
onOpenChange: setOverflowOpen,
mode: "hover",
placement: "bottom",
});
useEffect(() => {
if (hidden === 0 && overflowOpen) setOverflowOpen(false);
}, [hidden, overflowOpen]);
return (
<div className={`${chipLaneClass} flex items-center gap-1 min-w-0 flex-1`}>
@@ -340,7 +350,12 @@ export function ChannelTagsControl({
fitChipsWithPill({ chipRights: g.chipRights, pillW: g.pillW, avail: g.avail, gap: g.gap }),
});
const hidden = attached.length - visible;
// Clamped, and not defensively: `useChipOverflowFit` keeps `visible` in state and re-decides in a
// layout effect, so the render in which the chip set SHRINKS still holds the previous measurement
// and a raw subtraction goes negative. An unclamped `hidden` slips past the `=== 0` reset below —
// the flag survives exactly the window it exists to close, and the popover springs back on the
// next re-fit.
const hidden = Math.max(0, attached.length - visible);
const [editorOpen, setEditorOpen] = useState(false);
const editPop = useAnchoredPopover({
@@ -353,18 +368,18 @@ export function ChannelTagsControl({
hostsInput: true,
});
const [overflowOpen, setOverflowOpen] = useState(false);
// Derived for the same reason as the suggestion row above: if tags change so nothing is collapsed
// anymore, the "+N" trigger unmounts, and the popover must already be shut in that same render —
// and the flag reset with it, or the card returns unasked on the next overflow.
const [overflowOpenRaw, setOverflowOpen] = useState(false);
if (overflowOpenRaw && hidden === 0) setOverflowOpen(false);
const overflowOpen = overflowOpenRaw && hidden > 0;
const overflowPop = useAnchoredPopover({
open: overflowOpen,
onOpenChange: setOverflowOpen,
mode: "hover",
placement: "bottom",
});
// If tags change so nothing is collapsed anymore, the "+N" trigger unmounts — force the reveal-all
// popover's state closed so it can't linger open against a gone reference.
useEffect(() => {
if (hidden === 0 && overflowOpen) setOverflowOpen(false);
}, [hidden, overflowOpen]);
const pillClass =
"shrink-0 text-3xs px-1.5 py-0.5 rounded-full bg-card text-muted border border-border";
+16 -3
View File
@@ -86,6 +86,7 @@ export default function Channels() {
setLayout,
setSort,
focusChannel: onFocusChannel,
consumeFocus,
} = useChannelsActions();
const { openChannel: onViewChannel, setPage } = useNavigationActions();
const feedFilters = useFeedFilters();
@@ -281,10 +282,22 @@ export default function Channels() {
},
];
// A focus-channel intent (header "without full history" / tag manager) seeds the search box.
// The TOKEN is the intent (it advances even when the same channel is re-focused); the name is the
// payload, so it rides a ref. Naming it would re-seed the box on any render that changed it,
// overwriting whatever the user had since typed.
//
// `consumeFocus` is what stops this page's OWN remounts from re-applying it: this effect runs on
// every mount, and this page unmounts on every navigation away — and on opening a channel page,
// which does not even change `page`. The provider then reports `focusName` as null, so a later
// visit leaves the box exactly as the user left it.
const focusChannelNameRef = useRef(focusChannelName);
focusChannelNameRef.current = focusChannelName;
useEffect(() => {
if (focusChannelName) setSearch(focusChannelName);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [focusChannelToken]);
const name = focusChannelNameRef.current;
if (!name) return;
setSearch(name);
consumeFocus();
}, [focusChannelToken, setSearch, consumeFocus]);
// The header's reset intent clears the search box (skip the initial mount); the tag chips ride in
// `filters.tags`, which goToFullHistory already clears by setting a fresh map.
const resetRef = useRef(filtersResetToken);
@@ -8,8 +8,8 @@
// useAccountPersistedState / useSortState / useFilterState all resolve their account-scoped keys; only
// navigationContext (the sole runtime dependency) is mocked, pinned to the channels page.
import { describe, it, expect, beforeEach, vi } from "vitest";
import { renderHook, act } from "@testing-library/react";
import type { ReactNode } from "react";
import { renderHook, act, render as renderRoot } from "@testing-library/react";
import { useEffect, useRef, type ReactNode } from "react";
vi.mock("./navigationContext", () => ({
useNavigation: () => ({ page: "channels" }),
@@ -127,3 +127,85 @@ describe("ChannelsProvider — S8 shareable URL", () => {
});
});
});
// --- the focus-a-channel intent is consumed, not masked -------------------------------------------
// `focusChannel` sends the user to the manager with its name filter seeded. The manager is a page
// chunk: it unmounts on every navigation away, and on opening a channel page (which does not even
// change `page`), and its seeding effect runs again on each mount. So the "already applied" marker
// has to live in the PROVIDER — inside the page it would reset with the page and re-apply an intent
// from earlier in the session over whatever the user has typed into the search box since.
describe("ChannelsProvider — the focus-a-channel intent", () => {
/** Stands in for Channels.tsx:287-297 — the same effect, the same consume call. */
function Manager() {
const { focusName, focusToken } = useChannels();
const { setSearch, consumeFocus } = useChannelsActions();
const nameRef = useRef(focusName);
nameRef.current = focusName;
useEffect(() => {
const name = nameRef.current;
if (!name) return;
setSearch(name);
consumeFocus();
}, [focusToken, setSearch, consumeFocus]);
return null;
}
function Harness({ mounted }: { mounted: boolean }) {
return <>{mounted && <Manager />}</>;
}
it("seeds the search box once, and a later visit does not re-seed it", () => {
let api!: ReturnType<typeof useChannels> & ReturnType<typeof useChannelsActions>;
function Probe() {
api = { ...useChannels(), ...useChannelsActions() };
return null;
}
const { rerender } = renderRoot(
<ChannelsProvider>
<Probe />
<Harness mounted />
</ChannelsProvider>,
);
act(() => api.focusChannel("Kurzgesagt"));
expect(api.search).toBe("Kurzgesagt");
// The user looks for something else instead.
act(() => api.setSearch("veritasium"));
// Leave the manager (the page chunk unmounts) and come back.
rerender(
<ChannelsProvider>
<Probe />
<Harness mounted={false} />
</ChannelsProvider>,
);
rerender(
<ChannelsProvider>
<Probe />
<Harness mounted />
</ChannelsProvider>,
);
expect(api.search).toBe("veritasium");
});
it("re-seeds when the channel is focused again — the token is the intent", () => {
let api!: ReturnType<typeof useChannels> & ReturnType<typeof useChannelsActions>;
function Probe() {
api = { ...useChannels(), ...useChannelsActions() };
return null;
}
renderRoot(
<ChannelsProvider>
<Probe />
<Harness mounted />
</ChannelsProvider>,
);
act(() => api.focusChannel("Kurzgesagt"));
act(() => api.setSearch("veritasium"));
act(() => api.focusChannel("Kurzgesagt")); // the same channel, a new intent
expect(api.search).toBe("Kurzgesagt");
});
});
+33 -9
View File
@@ -110,7 +110,7 @@ export function ChannelsProvider({ children }: { children: ReactNode }) {
// it against the live columns for display, so the provider stores it un-healed (no options passed).
const [sort, setSort] = useSortState(LS.channelSort);
const [filtersResetToken, setFiltersResetToken] = useState(0);
const [focusName, setFocusName] = useState<string | null>(null);
const [focusNameRaw, setFocusName] = useState<string | null>(null);
const [focusToken, setFocusToken] = useState(0);
// S8 — shareable manager view. Captured once at mount (or null when no manager URL keys are present).
@@ -120,14 +120,18 @@ export function ChannelsProvider({ children }: { children: ReactNode }) {
// Apply the shared-link seed once, BEFORE paint, through the slice setters — which persist it, so the
// shared view also survives a later PLAIN reload (no URL). setViewRaw (not setView) restores the tab
// without pushing a phantom history entry. A normal load has urlSeed === null and this is a no-op.
// `view` is frozen at mount for the same reason `urlSeed` is: the comparison is against the tab
// this load started on, and the three setters are stable by construction (useFilterState,
// useSortState and useAccountPersistedState all return a useCallback). So the dep list can name
// every value the body reads without the seed re-arming on a later tab switch.
const [mountView] = useState(view);
useLayoutEffect(() => {
if (!urlSeed || seededRef.current) return;
seededRef.current = true;
if (urlSeed.view !== view) setViewRaw(urlSeed.view);
if (urlSeed.view !== mountView) setViewRaw(urlSeed.view);
setFilters(urlSeed.filters);
setSort(urlSeed.sort);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
}, [urlSeed, mountView, setViewRaw, setFilters, setSort]);
// Latest-value ref so the STABLE setView can compare against the current tab without depending on
// it (a dep would recreate the action on every switch and re-render every actions consumer).
@@ -175,10 +179,20 @@ export function ChannelsProvider({ children }: { children: ReactNode }) {
setPage("channels");
}, [setFilters, setView, setPage]);
// Clear the manager focus when leaving the Channels page so it doesn't re-apply on a later visit.
useEffect(() => {
if (page !== "channels") setFocusName(null);
}, [page]);
// The manager focus only means anything ON the Channels page, so it is DERIVED from the page
// rather than cleared by an effect when you leave. focusChannel sets the name and navigates in the
// same handler, so the value is live by the time channels renders; everywhere else it reads null,
// one render earlier than the effect used to make it.
//
// But an intent has to be CONSUMED, not merely masked, and the consumed marker has to live HERE.
// Channels is a page chunk that unmounts on every navigation (and on opening a channel page, which
// does not even change `page`), so its seeding effect runs again on each remount. A marker inside
// it would reset with it and re-apply an intent from earlier in the session over whatever the user
// has since typed into the search box. Keyed on the token rather than the name, because
// re-focusing the same channel is a new intent.
const [focusApplied, setFocusApplied] = useState(0);
const consumeFocus = useCallback(() => setFocusApplied(focusToken), [focusToken]);
const focusName = page === "channels" && focusApplied !== focusToken ? focusNameRaw : null;
// Label the entry the manager was REACHED on (setPage pushes a bare { sfPage }), so stepping back
// onto it later restores the tab it belonged to rather than leaving whatever tab is current.
@@ -211,9 +225,19 @@ export function ChannelsProvider({ children }: { children: ReactNode }) {
setLayout,
setSort,
focusChannel,
consumeFocus,
goToFullHistory,
}),
[setSearch, setFilters, setView, setLayout, setSort, focusChannel, goToFullHistory],
[
setSearch,
setFilters,
setView,
setLayout,
setSort,
focusChannel,
consumeFocus,
goToFullHistory,
],
);
// S8: mirror the content state (tab + sort + column filters) into the address bar under the namespaced
+8 -4
View File
@@ -42,6 +42,10 @@ export default function ChatThread({
{ intervalMs: POLL_MS, enabled: !needsKey },
);
const items = q.data?.items ?? [];
// The two effects below take the FETCH as their trigger and read the payload through this ref, so
// their dep lists name everything they touch without making the identity of `q.data` a trigger.
const threadRef = useRef(q.data);
threadRef.current = q.data;
// Decrypt the thread when fresh server data arrives. Keyed on dataUpdatedAt (not `items`): the
// `?? []` fallback is a new array reference every render, which would otherwise re-run this
@@ -56,7 +60,7 @@ export default function ChatThread({
// hasn't re-set-up, so has no key) degrades to "can't decrypt" instead of rejecting the pass.
let pub: string | null = null;
let refreshed = false;
for (const m of q.data?.items ?? []) {
for (const m of threadRef.current?.items ?? []) {
if (!m.ciphertext || !m.iv) continue;
try {
if (!pub) pub = await partnerPub(partnerId);
@@ -84,7 +88,8 @@ export default function ChatThread({
return () => {
cancelled = true;
};
// eslint-disable-next-line react-hooks/exhaustive-deps
// `dataUpdatedAt` is the trigger and the payload is read through the ref above — see Messages,
// where keying this family on `q.data` itself produced a render loop.
}, [q.dataUpdatedAt, ready, isSystem, partnerId, t]);
// Opening the thread (or a new incoming message arriving) marks messages read server-side, so
@@ -101,13 +106,12 @@ export default function ChatThread({
lastPartner.current = partnerId;
lastIncoming.current = -1;
}
const incoming = (q.data?.items ?? []).filter((m) => m.recipient_id === meId).length;
const incoming = (threadRef.current?.items ?? []).filter((m) => m.recipient_id === meId).length;
if (incoming !== lastIncoming.current) {
lastIncoming.current = incoming;
qc.invalidateQueries({ queryKey: qk.conversations() });
qc.invalidateQueries({ queryKey: qk.messageUnread() });
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [q.dataUpdatedAt, partnerId, meId, qc]);
// Scroll to the newest message when the count changes (open / new message) — NOT on `plain`,
+31 -3
View File
@@ -1,4 +1,4 @@
import { useEffect, useMemo, useState } from "react";
import { useEffect, useMemo, useRef, useState } from "react";
import { qk } from "../lib/queryKeys";
import { useTranslation } from "react-i18next";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
@@ -59,10 +59,14 @@ export default function ConfigPanel() {
// load, registry change) or an explicit post-save bump — so a refetch that resolves mid-edit
// (e.g. an invalidation) can no longer clobber the admin's in-progress draft (CB2).
const dataVersion = useMemo(() => items.map((i) => i.key).join("|"), [items]);
// The baseline is read through a ref, not named as a dependency: an unchanged refetch produces a
// new object, and re-seeding on that is exactly the clobbering of an in-progress draft that CB2
// was about. `dataVersion` (the config SHAPE) and the post-save token are the real triggers.
const baselineRef = useRef(baseline);
baselineRef.current = baseline;
useEffect(() => {
setDraft(baseline);
setDraft(baselineRef.current);
setSecretReset({});
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [dataVersion, reseedToken]);
const isDirty = (it: ConfigItem): boolean => {
@@ -134,6 +138,14 @@ export default function ConfigPanel() {
notifyActionError(e, t("config.plexTestFailed"));
},
});
// Bug reports' "Check the connection" (E16): the repository answers, takes issues, is private.
const testIssues = useMutation({
mutationFn: () => api.testIssues(),
onSuccess: (r) =>
notify({ level: "success", message: t("config.issuesTestOk", { repo: r.repo }) }),
onError: (e) => notifyActionError(e, t("config.issuesTestFailed")),
});
const plexLibs = (draft["plex_libraries"] ?? "")
.split(",")
.map((s) => s.trim())
@@ -218,6 +230,22 @@ export default function ConfigPanel() {
</div>
)}
{activeGroup === "issues" && (
<div className="mt-3 pt-3 border-t border-border/60">
<Tooltip hint={t("config.issuesTestHint")}>
<FocusSafeButton
onClick={() => testIssues.mutate()}
disabled={testIssues.isPending || dirty}
className="glass-card glass-hover inline-flex items-center gap-2 px-3 py-2 rounded-xl text-sm disabled:opacity-50 transition"
>
<Plug className={`w-4 h-4 ${testIssues.isPending ? "animate-pulse" : ""}`} />
{t("config.issuesTest")}
</FocusSafeButton>
</Tooltip>
{dirty && <p className="text-2xs text-muted mt-2">{t("config.plexTestDirty")}</p>}
</div>
)}
{activeGroup === "plex" && (
<div className="mt-3 pt-3 border-t border-border/60">
<Tooltip hint={t("config.plexTestHint")}>
@@ -40,6 +40,7 @@ export default function DateFilterSelect({
// useLayoutEffect (not useEffect) so the reset commits BEFORE paint — otherwise the trigger flashes
// one stale "Custom range" frame between the external clear and this reset.
useLayoutEffect(() => {
// eslint-disable-next-line react-hooks/set-state-in-effect -- the reveal is NOT derivable from the range — the user opens it on an empty range and fills it in afterwards. This undoes it only when the range is cleared from OUTSIDE while the popover is shut, and it is a layout effect so the trigger never paints a stale "Custom range".
if (!open && customOpen && !filters.dateFrom && !filters.dateTo) setCustomOpen(false);
}, [open, customOpen, filters.dateFrom, filters.dateTo]);
@@ -0,0 +1,382 @@
// @vitest-environment jsdom
//
// E16 S2: the debug recorder. Its pill rides the toast hosts (lib/toastDock) and stays out of their
// live region; Start is consent-first; Stop sends what is waiting BEFORE the server closes the
// session; a step is one tagged event; Escape in the step prompt closes only the prompt, never the
// dialog the pill sits in. i18n is a key-passthrough with its params; the API is stubbed and `me`
// lives in a real query cache, as in the app.
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { useState } from "react";
import { act, cleanup, fireEvent, render, screen, waitFor, within } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { DebugSession, Me } from "../lib/api";
import { qk } from "../lib/queryKeys";
vi.mock("react-i18next", async (importOriginal) => {
const actual = await importOriginal<typeof import("react-i18next")>();
return {
...actual,
useTranslation: () => ({
t: (key: string, opts?: Record<string, unknown>) =>
opts && Object.keys(opts).length ? `${key}:${Object.values(opts).join(",")}` : key,
i18n: { language: "en" },
}),
};
});
const { order, session, startDebugSession, stopDebugSession, hold, past } = vi.hoisted(() => {
const past: { items: unknown[] } = { items: [] };
// A Stop held open until released, when a test needs it in flight.
const hold = { stop: null as Promise<void> | null };
const order: string[] = [];
const session = (over: Partial<DebugSession> = {}): DebugSession => ({
id: "0123456789abcdef",
level: "DEBUG",
started_at: new Date().toISOString(),
until: new Date(Date.now() + 3_600_000).toISOString(),
ended_at: null,
ended_reason: null,
issue_url: null,
pending_files: [],
images_pending: false,
sending: false,
screenshots: 0,
draft: null,
...over,
});
const startDebugSession = vi.fn((level: "DEBUG" | "TRACE") =>
Promise.resolve({ ok: true as const, session: session({ level }) }),
);
const stopDebugSession = vi.fn(async (id: string) => {
order.push("stop");
if (hold.stop) await hold.stop;
return {
ok: true as const,
session: session({ id, ended_at: new Date().toISOString(), ended_reason: "stopped" }),
};
});
return { order, session, startDebugSession, stopDebugSession, hold, past };
});
vi.mock("../lib/api", async (importOriginal) => {
const actual = await importOriginal<typeof import("../lib/api")>();
return {
...actual,
api: {
...actual.api,
startDebugSession,
stopDebugSession,
debugSessions: () => Promise.resolve(past),
},
};
});
vi.mock("../lib/diagnostics", async (importOriginal) => {
const actual = await importOriginal<typeof import("../lib/diagnostics")>();
return {
...actual,
flushNow: () => {
order.push("flush");
return Promise.resolve();
},
};
});
import DebugRecorder, { RecorderSettings } from "./DebugRecorder";
import Modal from "./ui/Modal";
import Toaster from "./ui/Toaster";
import {
closeComposer,
openRecorderStart,
setStepPrompt,
useComposerSession,
} from "../lib/debugRecorder";
import { anyBlockingLayer, anyBlockingLayerAbove, pushLayer, removeLayer } from "../lib/layerStack";
import { events } from "../lib/log";
beforeEach(() => {
order.length = 0;
hold.stop = null;
sessionStorage.clear();
});
afterEach(() => {
cleanup();
closeComposer();
setStepPrompt(null);
vi.clearAllMocks();
vi.useRealTimers();
});
let composing: string | null = null;
function ComposerProbe() {
// eslint-disable-next-line react-hooks/globals -- a TEST probe: a module-level handle is how the test reads a hook's value from outside the tree without a wrapper that would change what is being measured.
composing = useComposerSession();
return null;
}
function mount(me: Partial<Me>, extra: React.ReactNode = null) {
const qc = new QueryClient({ defaultOptions: { mutations: { retry: false } } });
qc.setQueryData(qk.me(), {
id: 1,
is_demo: false,
preferences: {},
diagnostics: { level: "INFO", until: null, retention_days: 7, session_minutes: 60 },
debug_session: null,
...me,
});
render(
<QueryClientProvider client={qc}>
<Toaster />
<DebugRecorder />
<ComposerProbe />
{extra}
</QueryClientProvider>,
);
return qc;
}
const pill = () => screen.queryByRole("group", { name: "recorder.pill.label" });
/** A dialog the test opens and closes while the pill is up: the pill moves into its toast host. */
function DialogToggle() {
const [open, setOpen] = useState(false);
return (
<>
<button onClick={() => setOpen((v) => !v)}>toggle dialog</button>
{open && (
<Modal title="dialog" onClose={() => setOpen(false)}>
<p>inside</p>
</Modal>
)}
</>
);
}
describe("the recorder pill", () => {
it("shows while a session records, under the toasts and outside their live region", () => {
mount({ debug_session: session() });
const p = pill()!;
expect(p).not.toBeNull();
expect(p.closest("[data-toast-region]")).not.toBeNull();
expect(p.closest("[aria-live]")).toBeNull();
});
it("is not there for a session that ended or ran past its cap", () => {
mount({ debug_session: session({ until: new Date(Date.now() - 1000).toISOString() }) });
expect(pill()).toBeNull();
cleanup();
mount({ debug_session: session({ ended_at: new Date().toISOString() }) });
expect(pill()).toBeNull();
});
it("Stop sends what is waiting first, then stops, then opens the composer", async () => {
const qc = mount({ debug_session: session() });
fireEvent.click(within(pill()!).getByRole("button", { name: "recorder.pill.stop" }));
await waitFor(() => expect(stopDebugSession).toHaveBeenCalledWith("0123456789abcdef"));
expect(order).toEqual(["flush", "stop"]);
await waitFor(() => expect(pill()).toBeNull());
expect(qc.getQueryData<Me>(qk.me())?.debug_session).toBeNull();
expect(composing).toBe("0123456789abcdef");
});
it("a step is one tagged event, numbered, and the focus comes back to Step", () => {
mount({ debug_session: session() });
const step = within(pill()!).getByRole("button", { name: "recorder.pill.step" });
fireEvent.click(step);
const input = screen.getByLabelText("recorder.pill.stepPrompt:1");
expect(document.activeElement).toBe(input);
fireEvent.change(input, { target: { value: "opened the player" } });
fireEvent.keyDown(input, { key: "Enter" });
const marked = events().filter((e) => e.event === "debug.step");
expect(marked.slice(-1)[0]?.attrs).toMatchObject({ n: 1, label: "opened the player" });
expect(screen.queryByLabelText(/recorder.pill.stepPrompt/)).toBeNull();
expect(document.activeElement).toBe(step);
expect(step.textContent).toContain("· 1");
});
it("Escape in the step prompt closes the prompt, not the dialog the pill sits in", () => {
const onClose = vi.fn();
mount(
{ debug_session: session() },
<Modal title="dialog" onClose={onClose}>
<p>inside</p>
</Modal>,
);
// The pill moved into the dialog's own toast host.
expect(pill()!.closest('[role="dialog"]')).not.toBeNull();
const marked = () => events().filter((e) => e.event === "debug.step").length;
const before = marked();
fireEvent.click(within(pill()!).getByRole("button", { name: "recorder.pill.step" }));
fireEvent.keyDown(screen.getByLabelText(/recorder.pill.stepPrompt/), { key: "Escape" });
expect(screen.queryByLabelText(/recorder.pill.stepPrompt/)).toBeNull();
expect(onClose).not.toHaveBeenCalled();
expect(marked()).toBe(before);
});
it("Tab from the step prompt stays inside the dialog the pill sits in", () => {
mount(
{ debug_session: session() },
<Modal title="dialog" onClose={() => {}}>
<p>inside</p>
</Modal>,
);
fireEvent.click(within(pill()!).getByRole("button", { name: "recorder.pill.step" }));
const mark = screen.getByRole("button", { name: "recorder.pill.mark" });
mark.focus();
// The dialog's trap takes the Tab (jsdom has no layout, so it can only be seen taking it — the
// browser would otherwise walk out of the dialog into the page behind its scrim).
const unhandled = fireEvent.keyDown(mark, { key: "Tab" });
expect(unhandled).toBe(false);
});
it("the step prompt quiets the page's keys and leaves the Tab to the dialog it sits in", () => {
mount({ debug_session: session() });
expect(anyBlockingLayer()).toBe(false);
// The pill docks inside whatever dialog is open; stand one in for it, under the prompt.
const dialog = pushLayer();
fireEvent.click(within(pill()!).getByRole("button", { name: "recorder.pill.step" }));
// The prompt's field has the keyboard, so `j` and the `g` chords stay off the page behind...
expect(anyBlockingLayer()).toBe(true);
// ...while the dialog around it keeps its focus trap. The prompt cannot be that trap: its two
// controls are two of the dialog's many, and taking it left the rest of them untrapped, with
// Tab walking out into the page behind the scrim.
expect(anyBlockingLayerAbove(dialog)).toBe(false);
// And the quiet is the prompt's own doing, not the stand-in's: it outlasts the dialog.
removeLayer(dialog);
expect(anyBlockingLayer()).toBe(true);
});
it("the step prompt keeps what was typed when the pill moves to a dialog and back", () => {
mount({ debug_session: session() }, <DialogToggle />);
fireEvent.click(within(pill()!).getByRole("button", { name: "recorder.pill.step" }));
fireEvent.change(screen.getByLabelText(/recorder.pill.stepPrompt/), {
target: { value: "opened the player" },
});
fireEvent.click(screen.getByText("toggle dialog"));
expect(pill()!.closest('[role="dialog"]')).not.toBeNull(); // moved: a new pill
expect((screen.getByLabelText(/recorder.pill.stepPrompt/) as HTMLInputElement).value).toBe(
"opened the player",
);
});
it("a Stop still running keeps its button busy after the pill moves", async () => {
let release = () => {};
hold.stop = new Promise((r) => (release = r));
mount({ debug_session: session() }, <DialogToggle />);
fireEvent.click(within(pill()!).getByRole("button", { name: "recorder.pill.stop" }));
await waitFor(() => expect(order).toContain("stop"));
fireEvent.click(screen.getByText("toggle dialog"));
const stop = within(pill()!).getByRole("button", { name: "recorder.pill.stop" });
expect(stop.getAttribute("aria-busy")).toBe("true");
release();
await waitFor(() => expect(pill()).toBeNull());
});
it("a Start or a Stop in another tab makes this one ask the server again", async () => {
const qc = mount({ debug_session: session() });
const invalidate = vi.spyOn(qc, "invalidateQueries");
const otherTab = new BroadcastChannel("siftlode.debugSession");
otherTab.postMessage("changed");
await waitFor(() => expect(invalidate).toHaveBeenCalledWith({ queryKey: qk.me() }));
otherTab.close();
});
it("offers a screenshot only where the browser can capture a tab", () => {
mount({ debug_session: session() });
// jsdom has no Screen Capture API — like iOS Safari.
expect(within(pill()!).queryByRole("button", { name: "recorder.pill.screenshot" })).toBeNull();
});
it("at its cap it says so and opens the composer", () => {
vi.useFakeTimers();
mount({ debug_session: session({ until: new Date(Date.now() + 5_000).toISOString() }) });
act(() => void vi.advanceTimersByTime(5_000));
expect(screen.getByText("recorder.pill.expired:60")).not.toBeNull();
expect(composing).toBe("0123456789abcdef");
});
});
describe("the start dialog", () => {
it("states the retention and the cap, and starts at the level chosen", async () => {
const qc = mount({});
act(() => openRecorderStart());
expect(screen.getByText(/recorder.start.kept:7/)).not.toBeNull();
expect(screen.getByText(/recorder.start.cap:60/)).not.toBeNull();
fireEvent.click(screen.getByRole("button", { name: "settings.diagnostics.levels.TRACE" }));
expect(screen.getByText("recorder.start.what.TRACE")).not.toBeNull();
fireEvent.click(screen.getByRole("button", { name: /recorder.start.confirm/ }));
await waitFor(() => expect(startDebugSession).toHaveBeenCalledWith("TRACE"));
await waitFor(() => expect(pill()).not.toBeNull());
expect(qc.getQueryData<Me>(qk.me())?.debug_session?.level).toBe("TRACE");
expect(screen.queryByText("recorder.start.title")).toBeNull();
});
});
describe("the past sessions a user goes back to check", () => {
function settings() {
const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
qc.setQueryData(qk.me(), {
id: 1,
is_demo: false,
preferences: {},
diagnostics: { level: "INFO", until: null, retention_days: 7, session_minutes: 60 },
debug_session: null,
});
render(
<QueryClientProvider client={qc}>
<RecorderSettings />
</QueryClientProvider>,
);
}
it("does not call a half-delivered report sent", async () => {
// An issue that still owes its logs, or an edit that shows its screenshots, is not done. This
// list is where a user checks back, and a green "Sent" here is how one gets forgotten.
const over = { ended_at: new Date().toISOString(), ended_reason: "stopped" as const };
past.items = [
session({
...over,
id: "aaaaaaaaaaaaaaa1",
issue_url: "https://git.example/o/r/issues/7",
pending_files: ["client.log"],
}),
session({
...over,
id: "aaaaaaaaaaaaaaa2",
issue_url: "https://git.example/o/r/issues/8",
images_pending: true,
}),
session({ ...over, id: "aaaaaaaaaaaaaaa3", issue_url: "https://git.example/o/r/issues/9" }),
];
settings();
expect(await screen.findAllByText("recorder.history.partial")).toHaveLength(2);
expect(screen.getAllByText("recorder.history.sent")).toHaveLength(1);
});
it("says a send is still running rather than naming what it has got to so far", async () => {
// This is the list a user opens TO CHECK on a send. Its two honest-looking words are both
// wrong for a running one: "draft" before it has an issue, "half sent" after.
const over = { ended_at: new Date().toISOString(), ended_reason: "stopped" as const };
past.items = [
session({
...over,
id: "bbbbbbbbbbbbbbb1",
sending: true,
draft: { title: "t", body: "b", excluded: [], form: {} },
}),
session({
...over,
id: "bbbbbbbbbbbbbbb2",
sending: true,
issue_url: "https://git.example/o/r/issues/7",
pending_files: ["client.log"],
}),
];
settings();
expect(await screen.findAllByText("recorder.history.sending")).toHaveLength(2);
expect(screen.queryByText("recorder.history.draft")).toBeNull();
expect(screen.queryByText("recorder.history.partial")).toBeNull();
});
});
+412
View File
@@ -0,0 +1,412 @@
import { useEffect, useRef, useState, type KeyboardEvent as ReactKeyboardEvent } from "react";
import { useTranslation } from "react-i18next";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { Camera, Flag, Square } from "lucide-react";
import { api, type DebugSession, type DebugSessionLevel } from "../lib/api";
import { canCaptureTab } from "../lib/captureTab";
import {
closeRecorderStart,
onSessionChangeElsewhere,
openComposer,
openRecorderStart,
setStepPrompt,
useComposerSession,
useRecorderStartOpen,
useStepCount,
useStepPrompt,
} from "../lib/debugRecorder";
import { formatDuration } from "../lib/format";
import { useLayer } from "../lib/layerStack";
import { notify } from "../lib/notifications";
import { qk } from "../lib/queryKeys";
import { dockUnderToasts } from "../lib/toastDock";
import { useMe } from "../lib/useMe";
import { Button } from "./ui/Button";
import Modal from "./ui/Modal";
import { LoadingState, StateMessage } from "./ui/QueryState";
import SegmentedToggle from "./ui/SegmentedToggle";
import { inputCls } from "./ui/form";
import IssueComposer from "./IssueComposer";
import { useDebugRecording } from "./useDebugRecording";
// The debug recorder (E16): Start → reproduce → Stop.
//
// While a session records, its pill sits under the toasts (lib/toastDock), so it rides every toast
// host — the page, each dialog, both players, the fullscreen player — and is reachable wherever the
// bug is being reproduced, above a player's control band rather than on it. Mounted once, in the
// app shell, for a signed-in account other than the demo.
/** The time since `since`, as the app's other clocks show it, re-rendered every second. */
function useElapsed(since: string): string {
const [, tick] = useState(0);
useEffect(() => {
const id = window.setInterval(() => tick((n) => n + 1), 1000);
return () => window.clearInterval(id);
}, []);
// eslint-disable-next-line react-hooks/purity -- the clock is READ at render and the interval above only re-renders — a `now` held in state is the moment the component mounted, which is the defect DiagnosticsSettings records from prod UAT.
return formatDuration(Math.max(0, Math.floor((Date.now() - Date.parse(since)) / 1000)));
}
export default function DebugRecorder() {
const { t } = useTranslation();
const qc = useQueryClient();
const me = useMe();
const { session } = useDebugRecording();
const startOpen = useRecorderStartOpen();
const composing = useComposerSession();
const id = session?.id;
useEffect(() => (id ? dockUnderToasts(RecorderPill) : undefined), [id]);
// Started or stopped in another tab: this one asks the server what is in force now.
useEffect(
() => onSessionChangeElsewhere(() => void qc.invalidateQueries({ queryKey: qk.me() })),
[qc],
);
// The cap: the server ends the session at `until` whether or not anyone stops it. This tab says
// so, and opens the composer for it — the recording is as useful as one stopped by hand.
const until = session?.until;
const minutes = me?.diagnostics?.session_minutes;
useEffect(() => {
if (!id || !until) return;
const wait = Math.min(Date.parse(until) - Date.now(), 2 ** 31 - 1);
const timer = window.setTimeout(() => {
notify({ level: "warning", message: t("recorder.pill.expired", { count: minutes ?? 0 }) });
openComposer(id);
void qc.invalidateQueries({ queryKey: qk.me() });
}, wait);
return () => window.clearTimeout(timer);
}, [id, until, minutes, qc, t]);
return (
<>
{startOpen && <StartDialog />}
{composing && <IssueComposer id={composing} />}
</>
);
}
function StartDialog() {
const { t } = useTranslation();
const me = useMe();
const { start } = useDebugRecording();
const [level, setLevel] = useState<DebugSessionLevel>("DEBUG");
const days = me.diagnostics.retention_days;
const minutes = me.diagnostics.session_minutes;
const levels: readonly DebugSessionLevel[] = ["DEBUG", "TRACE"];
return (
<Modal title={t("recorder.start.title")} onClose={closeRecorderStart}>
<div className="flex flex-col gap-4">
<p className="text-sm text-muted leading-relaxed">{t("recorder.start.intro")}</p>
<div className="flex flex-col items-start gap-1.5">
<span className="text-sm text-muted">{t("recorder.start.level")}</span>
<SegmentedToggle
size="md"
ariaLabel={t("recorder.start.level")}
options={levels.map((l) => ({
value: l,
label: t(`settings.diagnostics.levels.${l}`),
}))}
value={level}
onChange={setLevel}
/>
</div>
<div className="rounded-lg border border-border bg-surface/60 px-4 py-3 flex flex-col gap-2 text-sm leading-relaxed">
<div className="font-semibold">{t("recorder.start.whatTitle")}</div>
<p>{t(`recorder.start.what.${level}`)}</p>
<p className="text-muted">{t("settings.diagnostics.consent.never")}</p>
<p className="text-muted">
{t("recorder.start.kept", { count: days })}{" "}
{t("recorder.start.cap", { count: minutes })}
</p>
</div>
<div className="flex justify-end gap-2">
<Button variant="ghost" onClick={closeRecorderStart}>
{t("common.cancel")}
</Button>
<Button
variant="primary"
loading={start.isPending}
icon={<span className="w-2.5 h-2.5 rounded-full bg-danger" aria-hidden />}
onClick={() => start.mutate(level)}
>
{t("recorder.start.confirm")}
</Button>
</div>
</div>
</Modal>
);
}
/** The pill: how long it has been recording, and the three things to do while it does. */
function RecorderPill() {
const { t } = useTranslation();
const { session, stop, stopping, shooting, markStep, screenshot } = useDebugRecording();
const [hidden, setHidden] = useState(false);
const stepOpen = useStepPrompt() !== null;
const stepButton = useRef<HTMLButtonElement>(null);
const steps = useStepCount(session?.id ?? "");
if (!session) return null;
return (
<div
role="group"
aria-label={t("recorder.pill.label")}
className={`relative glass rounded-full h-12 pl-3 sm:pl-4 pr-1.5 flex items-center gap-1.5 sm:gap-2 shadow-lg ${
hidden ? "invisible" : ""
}`}
>
<span
aria-hidden
className="w-2.5 h-2.5 shrink-0 rounded-full bg-danger ring-4 ring-danger/20 motion-safe:animate-pulse"
/>
<Elapsed since={session.started_at} />
<span className="chip hidden sm:inline-flex px-2 py-0.5 text-2xs">
{t(`settings.diagnostics.levels.${session.level}`)}
</span>
<span aria-hidden className="hidden sm:block w-px h-6 bg-border" />
<Button
ref={stepButton}
variant="outline-accent"
size="sm"
className="rounded-full"
icon={<Flag className="w-4 h-4" />}
label={t("recorder.pill.step")}
aria-expanded={stepOpen}
onClick={() => setStepPrompt(stepOpen ? null : "")}
>
<span className="hidden sm:inline tabular-nums">
{t("recorder.pill.step")}
{steps ? ` · ${steps}` : ""}
</span>
</Button>
{canCaptureTab() && (
<Button
variant="outline"
size="sm"
className="rounded-full"
icon={<Camera className="w-4 h-4" />}
label={t("recorder.pill.screenshot")}
loading={shooting}
onClick={() => void screenshot(setHidden)}
>
<span className="hidden sm:inline tabular-nums">
{t("recorder.pill.screenshot")}
{session.screenshots ? ` · ${session.screenshots}` : ""}
</span>
</Button>
)}
<Button
variant="danger"
size="sm"
className="rounded-full"
icon={<Square className="w-3.5 h-3.5 fill-current" />}
label={t("recorder.pill.stop")}
loading={stopping}
onClick={() => stop.mutate(session.id)}
>
<span className="hidden sm:inline">{t("recorder.pill.stop")}</span>
</Button>
{stepOpen && (
<StepPrompt
id={session.id}
onDone={(label) => {
if (label !== null) markStep(label);
setStepPrompt(null);
stepButton.current?.focus();
}}
/>
)}
</div>
);
}
function Elapsed({ since }: { since: string }) {
const { t } = useTranslation();
const time = useElapsed(since);
return (
// On a phone only the clock: the pill has to fit the content column beside the rail, and the
// red dot already says what it is.
<span className="text-sm font-semibold tabular-nums whitespace-nowrap">
<span className="hidden sm:inline">{t("recorder.pill.recording", { time })}</span>
<span className="sm:hidden">{time}</span>
</span>
);
}
/** Settings → Diagnosztika: start a recording or stop the one running, and reopen a past one. */
export function RecorderSettings() {
return (
<div className="flex flex-col gap-4">
<RecorderControl />
<SessionHistory />
</div>
);
}
function RecorderControl() {
const { t } = useTranslation();
const { session, stop, stopping } = useDebugRecording();
if (!session)
return (
<div className="flex flex-wrap items-center justify-between gap-3">
<p className="text-sm text-muted max-w-prose">{t("recorder.settings.hint")}</p>
<Button
variant="outline"
icon={<span className="w-2.5 h-2.5 rounded-full bg-danger" aria-hidden />}
onClick={openRecorderStart}
>
{t("recorder.settings.start")}
</Button>
</div>
);
return (
<div className="flex flex-wrap items-center justify-between gap-2 rounded-xl border border-danger/40 bg-danger/10 px-3 py-2 text-sm">
<span className="inline-flex items-center gap-2">
<span className="h-2 w-2 shrink-0 rounded-full bg-danger" aria-hidden />
<SettingsElapsed since={session.started_at} />
</span>
<Button
variant="danger-soft"
size="sm"
loading={stopping}
onClick={() => stop.mutate(session.id)}
>
{t("recorder.settings.stop")}
</Button>
</div>
);
}
function SettingsElapsed({ since }: { since: string }) {
const { t } = useTranslation();
const time = useElapsed(since);
return <span className="tabular-nums">{t("recorder.settings.recording", { time })}</span>;
}
/** "What did you just do?" — optional; Enter marks the step, Escape leaves it unmarked. */
function StepPrompt({ id, onDone }: { id: string; onDone: (label: string | null) => void }) {
const { t } = useTranslation();
const n = useStepCount(id) + 1;
const label = useStepPrompt() ?? "";
// A layer of its own, so Escape closes only the prompt — not the dialog or the player the pill may
// be sitting in: the topmost layer takes Escape (lib/layerStack), and this one opened last. It
// OWNS the keyboard while it is up (an autofocused field does), so the page's own keys stay quiet.
// It does NOT take the Tab, though: it is a field inside whatever dialog the pill is docked in,
// not a dialog of its own, and the trap belongs to that one — it spans the prompt as it spans the
// rest of the toast host. Claiming it here switched the dialog's off and left every control
// OUTSIDE this little box untrapped (deep-review round 3).
useLayer(true, () => onDone(null), { trapsTab: false });
const onKeyDown = (e: ReactKeyboardEvent<HTMLInputElement>) => {
if (e.key === "Enter") {
e.preventDefault();
onDone(label);
}
};
return (
<div className="absolute bottom-full left-0 mb-2 w-80 max-w-[calc(100vw-2rem)] glass rounded-xl p-3 flex flex-col gap-2 shadow-lg">
<label htmlFor="debug-step-label" className="text-sm text-muted">
{t("recorder.pill.stepPrompt", { n })}
</label>
<input
id="debug-step-label"
autoFocus
value={label}
onChange={(e) => setStepPrompt(e.target.value)}
onKeyDown={onKeyDown}
maxLength={200}
className={`${inputCls} w-full`}
/>
<div className="flex items-center justify-between gap-2 text-xs text-muted">
<span>{t("recorder.pill.stepHint")}</span>
<Button variant="primary" size="sm" onClick={() => onDone(label)}>
{t("recorder.pill.mark")}
</Button>
</div>
</div>
);
}
/** The five words a past session can read as. A union, not `string`, because `chipTone` below
* matches on these same keys: untyped, a rename in one of them left the other matching nothing
* and every chip untoned, with the gate green (deep-review round 4). */
type HistoryState =
| "recorder.history.sending"
| "recorder.history.draft"
| "recorder.history.unwritten"
| "recorder.history.partial"
| "recorder.history.sent";
/** What a past session reads as. An issue that still owes the tracker its attachments, or an edit
* that shows its screenshots, is NOT "Sent": this list is where a user goes back to check, and a
* green chip there is how a half-delivered report gets forgotten. */
function historyState(s: DebugSession): HistoryState {
// A send under way outranks every other word here: this is the list a user comes back to in
// order to check on one, and both "draft" (no issue yet) and "half sent" (the steps it has
// already committed) would describe a send that STOPPED.
if (s.sending) return "recorder.history.sending";
if (!s.issue_url) return s.draft ? "recorder.history.draft" : "recorder.history.unwritten";
return s.pending_files.length > 0 || s.images_pending
? "recorder.history.partial"
: "recorder.history.sent";
}
/** The colour that reinforces that word. Exhaustive over `HistoryState`, so a key renamed in
* `historyState` is a compile error here rather than a chip that quietly loses its tone. */
const chipTone = (s: DebugSession): string => {
switch (historyState(s)) {
case "recorder.history.sent":
return "text-success";
case "recorder.history.sending":
return "text-accent";
case "recorder.history.partial":
return "text-warn";
case "recorder.history.draft":
case "recorder.history.unwritten":
return "";
}
};
/** The last sessions, newest first — reopening one opens its composer, with what was written. */
function SessionHistory() {
const { t, i18n } = useTranslation();
const q = useQuery({ queryKey: qk.debugSessions(), queryFn: api.debugSessions });
const items = q.data?.items.filter((s) => s.ended_at) ?? [];
if (q.isError && !q.data)
return (
<StateMessage tone="error" compact onRetry={() => void q.refetch()}>
{t("recorder.history.failed")}
</StateMessage>
);
if (!q.data) return <LoadingState />;
return (
<div className="flex flex-col gap-1.5">
<div className="text-sm text-muted">{t("recorder.history.title")}</div>
{items.length === 0 ? (
<p className="text-sm text-muted">{t("recorder.history.empty")}</p>
) : (
<ul className="flex flex-col divide-y divide-border/60 rounded-xl border border-border">
{items.map((s) => (
<li
key={s.id}
className="flex flex-wrap items-center gap-x-3 gap-y-1 px-3 py-2 text-sm"
>
<span className="tabular-nums">
{new Date(s.started_at).toLocaleString(i18n.language)}
</span>
<span className="min-w-0 flex-1 truncate text-muted">
{s.draft?.title || t(`settings.diagnostics.levels.${s.level}`)}
</span>
<span className={`chip px-2 py-0.5 text-2xs ${chipTone(s)}`}>
{t(historyState(s))}
</span>
<Button variant="outline" size="xs" onClick={() => openComposer(s.id)}>
{t("recorder.history.open")}
</Button>
</li>
))}
</ul>
)}
</div>
);
}
@@ -37,6 +37,7 @@ const setLevel = vi.fn(
level,
until: level === "INFO" ? null : new Date(Date.now() + 86_400_000).toISOString(),
retention_days: 7,
session_minutes: 60,
},
}),
);
@@ -61,7 +62,7 @@ function mount(me: Partial<Me>) {
id: 1,
is_demo: false,
preferences: {},
diagnostics: { level: "INFO", until: null, retention_days: 7 },
diagnostics: { level: "INFO", until: null, retention_days: 7, session_minutes: 60 },
...me,
});
render(
@@ -131,7 +132,7 @@ describe("DiagnosticsSettings", () => {
it("going back to the base level needs no consent", async () => {
const until = new Date(Date.now() + 3_600_000).toISOString();
mount({ diagnostics: { level: "DEBUG", until, retention_days: 7 } });
mount({ diagnostics: { level: "DEBUG", until, retention_days: 7, session_minutes: 60 } });
fireEvent.click(levelButton("settings.diagnostics.levels.INFO"));
expect(screen.queryByTestId("diagnostics-consent")).toBeNull();
await waitFor(() => expect(setLevel).toHaveBeenCalledWith("INFO"));
@@ -169,7 +170,7 @@ describe("DiagnosticsSettings", () => {
try {
// Raised, so the status line's countdown is mounted and its 30 s tick keeps re-rendering.
const until = new Date(Date.now() + 3_600_000).toISOString();
mount({ diagnostics: { level: "DEBUG", until, retention_days: 7 } });
mount({ diagnostics: { level: "DEBUG", until, retention_days: 7, session_minutes: 60 } });
fireEvent.click(levelButton("settings.diagnostics.levels.TRACE"));
const elsewhere = document.createElement("button");
document.body.append(elsewhere);
@@ -194,7 +195,7 @@ describe("DiagnosticsSettings", () => {
it("the ticking status line is not a live region", async () => {
const until = new Date(Date.now() + 3_600_000).toISOString();
mount({ diagnostics: { level: "DEBUG", until, retention_days: 7 } });
mount({ diagnostics: { level: "DEBUG", until, retention_days: 7, session_minutes: 60 } });
const status = screen.getByTestId("diagnostics-status");
expect(status.closest('[role="status"], [aria-live]')).toBeNull();
});
@@ -215,7 +216,7 @@ describe("DiagnosticsSettings", () => {
id: 1,
is_demo: false,
preferences: {},
diagnostics: { level: "INFO", until: null, retention_days: 7 },
diagnostics: { level: "INFO", until: null, retention_days: 7, session_minutes: 60 },
});
await stale;
expect(qc.getQueryData<Me>(qk.me())?.diagnostics.level).toBe("TRACE");
@@ -224,7 +225,7 @@ describe("DiagnosticsSettings", () => {
it("a lapsed level shows as the base level", () => {
const until = new Date(Date.now() - 60_000).toISOString();
mount({ diagnostics: { level: "TRACE", until, retention_days: 7 } });
mount({ diagnostics: { level: "TRACE", until, retention_days: 7, session_minutes: 60 } });
expect(levelButton("settings.diagnostics.levels.INFO").getAttribute("aria-pressed")).toBe(
"true",
);
@@ -10,6 +10,7 @@ import InlineNotice from "./ui/InlineNotice";
import SegmentedToggle from "./ui/SegmentedToggle";
import { Section, SettingRow } from "./ui/form";
import MyLogViewer from "./MyLogViewer";
import { RecorderSettings } from "./DebugRecorder";
const LEVELS: readonly DiagnosticsLevel[] = ["INFO", "DEBUG", "TRACE"];
@@ -25,6 +26,7 @@ function useRemaining(until: string | null): { h: number; m: number } | null {
return () => window.clearInterval(id);
}, [until]);
if (!until) return null;
// eslint-disable-next-line react-hooks/purity -- same as DebugRecorder's elapsed clock, and the comment above records the prod UAT finding that a state-held `now` produced: a level raised two minutes after opening Settings read "24 h 2 m" until the first tick.
const minutes = Math.max(0, Math.ceil((Date.parse(until) - Date.now()) / 60_000));
return { h: Math.floor(minutes / 60), m: minutes % 60 };
}
@@ -181,6 +183,12 @@ export default function DiagnosticsSettings() {
)}
</Section>
{!me.is_demo && (
<Section title={t("recorder.settings.title")}>
<RecorderSettings />
</Section>
)}
<Section title={t("settings.myLog.title")}>
<MyLogViewer />
</Section>
+5 -5
View File
@@ -1,4 +1,4 @@
import { lazy, Suspense, useEffect, useState } from "react";
import { lazy, Suspense, useState } from "react";
import { qk } from "../lib/queryKeys";
import { useTranslation } from "react-i18next";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
@@ -152,10 +152,10 @@ export default function DownloadCenter() {
];
// Guard against the active tab going away under us (e.g. admin on "system" gets demoted mid-session)
// — that would render a blank page with no active tab. Fall back to the default.
useEffect(() => {
if (!tabs.some((tb) => tb.id === tab)) setTab("queue");
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [tab, me.role]);
// Adjusted during render rather than from an effect: as an effect the blank page it exists to
// prevent was committed and painted first. `setTab` is this component's own state (persisted per
// account), so a render-phase write is allowed, and it converges — "queue" is always in the list.
if (!tabs.some((tb) => tb.id === tab)) setTab("queue");
const saveBtn = (job: DownloadJob) => (
<a
+18 -6
View File
@@ -220,17 +220,21 @@ export default function Feed({
if (qEmpty && parseSort(filters.sort).key === "relevance") {
setFilters({ ...filters, sort: buildSort("date", "desc") });
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [qEmpty]);
// Naming `filters` is safe precisely because the write below falsifies the condition above: the
// sort it sets is "date", so the one re-run it causes does nothing. The old `[qEmpty]` was
// guarding against a loop that the guard already prevents.
}, [qEmpty, filters, setFilters]);
// Drop optimistic status overrides when filters change OR fresh server data
// arrives — after a refetch the server is authoritative, so a stale override
// (e.g. a video reverted to "new" from the notification center) won't keep it
// filtered out of the current view.
useEffect(() => {
const [overridesFor, setOverridesFor] = useState(filters);
if (overridesFor !== filters) {
setOverridesFor(filters);
setOverrides({});
setSavedOverrides({});
}, [filters]);
}
// ...but NOT on infinite-scroll appends: fetchNextPage also bumps dataUpdatedAt while page 1's
// rows are unchanged, so clearing overrides there would make a just-hidden card flash back.
// Only a real refetch (page count doesn't grow) is authoritative.
@@ -284,7 +288,7 @@ export default function Feed({
message: v?.title
? i18n.t("feed.hiddenNamed", { title: v.title })
: i18n.t("feed.hidden"),
action: { label: i18n.t("feed.undo"), onClick: () => onState(id, "new") },
action: { label: i18n.t("feed.undo"), onClick: () => onStateRef.current(id, "new") },
meta: {
kind: "video-hidden",
videoId: id,
@@ -299,7 +303,10 @@ export default function Feed({
message: v?.title
? i18n.t("feed.markedWatchedNamed", { title: v.title })
: i18n.t("feed.markedWatched"),
action: { label: i18n.t("feed.unwatch"), onClick: () => onState(id, "new") },
action: {
label: i18n.t("feed.unwatch"),
onClick: () => onStateRef.current(id, "new"),
},
meta: {
kind: "video-watched",
videoId: id,
@@ -328,6 +335,11 @@ export default function Feed({
},
[qc],
);
// The toast's Undo action re-enters onState, which is a forward reference to the binding still
// being defined — the same shape PlexPlayer's session reclaim has, and answered the same way.
const onStateRef = useRef(onState);
// eslint-disable-next-line react-hooks/immutability
onStateRef.current = onState;
const onResetState = useCallback(
(id: string) => {
@@ -67,10 +67,16 @@ export function FeedFiltersProvider({ children }: { children: ReactNode }) {
// A share link's filters (captured at mount) win over the account's stored view and are persisted
// to it once, so the view the link opened also survives a later plain reload.
//
// The mount-time filters are FROZEN into their own state rather than left to an empty dep list:
// `filters` changes on every edit, so listing it honestly would re-persist — and setFilters
// setting the state it depends on would spin. Frozen, all three deps are stable and the effect
// means exactly what it says: write what the URL brought, once.
const [urlSeededFilters] = useState(filters);
useEffect(() => {
if (initialHadUrlFilters) setFilters(filters);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
// eslint-disable-next-line react-hooks/set-state-in-effect -- setFilters also WRITES to per-account localStorage, which is the point here — persisting what a share link brought. That is a side effect, not a value to derive, and the frozen seed above makes it happen exactly once.
if (initialHadUrlFilters) setFilters(urlSeededFilters);
}, [initialHadUrlFilters, urlSeededFilters, setFilters]);
const actions = useMemo<FeedFiltersActions>(() => ({ setFilters }), [setFilters]);
+5 -1
View File
@@ -141,7 +141,10 @@ export default function GlassTuner() {
// Opt-in via Settings → Appearance (theme.showTuner, off by default); the flag lives in PrefsProvider.
const enabled = usePrefs().theme.showTuner;
const { t } = useTranslation();
const initial = useMemo(loadSaved, []);
// Read once at mount. useState's lazy initializer rather than a useMemo with an empty dep list:
// a memo is a caching hint React may discard and recompute, while "the saved values as they were
// when this mounted" is state — and the effect below applies them exactly once.
const [initial] = useState(loadSaved);
const [vars, setVars] = useState<Record<string, number>>(() => {
const base: Record<string, number> = {};
ALL.forEach((s) => (base[s.k] = s.def));
@@ -167,6 +170,7 @@ export default function GlassTuner() {
const v = (initial.palette?.[p.k] || readVar(p.k)).trim();
sw[p.k] = /^#[0-9a-fA-F]{6}$/.test(v) ? v : "#000000";
});
// eslint-disable-next-line react-hooks/set-state-in-effect -- the swatches are READ FROM THE DOM (readVar on the computed style) after the saved overrides have been applied to it, so they cannot exist before a commit.
setSwatches(sw);
}, [initial]);
+2
View File
@@ -474,6 +474,7 @@ export default function Home() {
hero: DiscoverPayload["hero"];
} | null>(null);
useEffect(() => {
// eslint-disable-next-line react-hooks/set-state-in-effect -- deliberately OUTLIVES the query: a failed reroll is terminal (retry:false) and must leave the previous answer on screen with the budget tag the SERVER echoed for it. Derived from pickQ it would vanish with the query.
if (pickQ.data !== undefined) setLastPick({ budget: pickQ.data.budget, hero: pickQ.data.hero });
}, [pickQ.data]);
@@ -695,6 +696,7 @@ export default function Home() {
// `reroll`'s changing identity causes — and to a token minted for another history entry.
const surpriseRequest = useSurpriseRequest();
useEffect(() => {
// eslint-disable-next-line react-hooks/set-state-in-effect -- consuming a cross-page token (lib/homeIntent) that was minted before this module's chunk existed. takeSurprise() clears it and answers false to every later pass, including StrictMode's second mount.
if (surpriseRequest !== null && takeSurprise()) reroll("surprise");
}, [surpriseRequest, reroll]);
@@ -0,0 +1,558 @@
// @vitest-environment jsdom
//
// E16 S3: the issue composer. It drafts the steps from the session, saves as it goes (the rendered
// text included — the export is built from what it SAVED), reopens as it was left, takes a log line
// out of the evidence in the review, adds a pasted image, and a sent session is read-only. i18n is a
// key-passthrough with its params; the API is stubbed.
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { cleanup, fireEvent, render, screen, waitFor, within } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { DebugDraft, DebugSession, ServerLogRow } from "../lib/api";
import { qk } from "../lib/queryKeys";
vi.mock("react-i18next", async (importOriginal) => {
const actual = await importOriginal<typeof import("react-i18next")>();
return {
...actual,
useTranslation: () => ({
t: (key: string, opts?: Record<string, unknown>) =>
opts && Object.keys(opts).length ? `${key}:${Object.values(opts).join(",")}` : key,
i18n: { language: "en" },
}),
};
});
const api = vi.hoisted(() => ({
session: null as unknown as DebugSession,
rows: [] as ServerLogRow[],
saved: [] as DebugDraft[],
uploaded: [] as Blob[],
failSession: false,
failShots: false,
sent: 0,
previewed: [] as string[],
// What happened, in order: a save is "save" when asked and "saved" when it answered.
order: [] as string[],
holdSend: null as Promise<void> | null,
// The server's half-failure: the issue is committed, then an upload dies and the call 422s.
failSendAfterCreate: false,
}));
vi.mock("../lib/api", async (importOriginal) => {
const actual = await importOriginal<typeof import("../lib/api")>();
return {
...actual,
api: {
...actual.api,
debugSession: () =>
api.failSession ? Promise.reject(new Error("down")) : Promise.resolve(api.session),
debugSessionEvents: () => Promise.resolve({ items: api.rows, total: api.rows.length }),
debugScreenshots: () =>
api.failShots ? Promise.reject(new Error("down")) : Promise.resolve({ items: [] }),
saveDebugDraft: (_id: string, draft: DebugDraft) => {
api.saved.push(draft);
api.order.push("save");
// Answers a task later, so a caller that does not wait for it is seen not waiting.
return new Promise((resolve) =>
setTimeout(() => {
api.order.push("saved");
resolve({ ok: true, session: { ...api.session, draft } });
}, 0),
);
},
addDebugScreenshot: (_id: string, image: Blob) => {
api.uploaded.push(image);
return Promise.resolve({
ok: true,
screenshot: { id: 1, mime: image.type, size: 4, created_at: "" },
});
},
debugExport: () => {
api.order.push("export");
return Promise.resolve({ blob: new Blob(["zip"]), name: "siftlode-debug.zip" });
},
sendDebugIssue: async () => {
api.order.push("send");
if (api.holdSend) await api.holdSend;
api.sent += 1;
api.session = {
...api.session,
issue_url: "https://git.example/o/r/issues/7",
pending_files: api.failSendAfterCreate ? ["server.log"] : [],
// What the server does on the way through: a send that got there shows the images too,
// so the banner (and the Retry in it) goes. Leaving this true made the retry's own
// banner outlive its success, which is not a state the server can produce.
images_pending: false,
};
if (api.failSendAfterCreate) throw new Error("issues_unreachable");
return { ok: true as const, session: api.session };
},
previewDebugIssue: (_id: string, text: string) => {
api.previewed.push(text);
return Promise.resolve({ ok: true, html: "<h3>Steps</h3><script>alert(1)</script>" });
},
},
};
});
import IssueComposer from "./IssueComposer";
const SID = "0123456789abcdef";
let n = 1;
const row = (over: Partial<ServerLogRow>): ServerLogRow => ({
id: n++,
ts: "2026-09-19T14:05:31.000Z",
level: "INFO",
source: "web",
logger: "web",
event: "x.y",
msg: "m",
attrs: null,
exc: null,
request_id: null,
user_id: 1,
job_id: null,
debug_session_id: SID,
version: null,
...over,
});
const baseSession = (over: Partial<DebugSession> = {}): DebugSession => ({
id: SID,
level: "DEBUG",
started_at: "2026-09-19T14:02:00.000Z",
until: "2026-09-19T15:02:00.000Z",
ended_at: "2026-09-19T14:16:00.000Z",
ended_reason: "stopped",
issue_url: null,
pending_files: [],
images_pending: false,
sending: false,
screenshots: 0,
draft: null,
...over,
});
beforeEach(() => {
api.session = baseSession();
api.rows = [
row({ event: "debug.step", msg: "resume", attrs: { n: 1, label: "Resume" } }),
row({ event: "ui.click", msg: "private click" }),
row({ source: "api", event: "http.request", msg: "GET /api/x 200" }),
];
api.saved = [];
api.uploaded = [];
api.failSession = false;
api.failShots = false;
api.order = [];
api.holdSend = null;
api.failSendAfterCreate = false;
api.sent = 0;
api.previewed = [];
tracker = null;
});
afterEach(() => {
cleanup();
vi.useRealTimers();
});
let tracker: { repo: string } | null = null;
function mount() {
const qc = new QueryClient({ defaultOptions: { queries: { retry: false } } });
qc.setQueryData(qk.me(), { id: 1, issues: tracker });
render(
<QueryClientProvider client={qc}>
<IssueComposer id={SID} />
</QueryClientProvider>,
);
}
const lastSaved = () => api.saved[api.saved.length - 1]!;
const title = () => screen.getByLabelText("recorder.composer.fieldTitle") as HTMLInputElement;
/** A button once it can be pressed: `loading` holds it while the draft's first save settles. */
async function ready(name: string): Promise<HTMLButtonElement> {
const button = (await screen.findByRole("button", { name })) as HTMLButtonElement;
await waitFor(() => expect(button.disabled).toBe(false));
return button;
}
describe("the issue composer", () => {
it("drafts the steps from the session and saves the rendered text after a pause", async () => {
mount();
const first = await screen.findByLabelText("recorder.composer.stepLabel:1");
expect((first as HTMLInputElement).value).toBe("Resume");
fireEvent.change(screen.getByLabelText("recorder.composer.fieldTitle"), {
target: { value: "Spinner stays" },
});
await waitFor(() => expect(api.saved.length).toBeGreaterThan(0), { timeout: 2000 });
expect(lastSaved().title).toBe("Spinner stays");
expect(lastSaved().body).toContain("1. Resume");
expect(lastSaved().body).toContain("`client.log` — recorder.issue.clientLog:2");
});
it("reopens a saved draft as it was left", async () => {
api.session = baseSession({
draft: {
title: "Saved title",
body: "",
excluded: [],
form: {
title: "Saved title",
description: "kept",
steps: [{ id: "m1", kind: "manual", text: "typed" }],
excluded: [],
},
},
});
mount();
expect(
((await screen.findByLabelText("recorder.composer.fieldTitle")) as HTMLInputElement).value,
).toBe("Saved title");
expect((screen.getByLabelText("recorder.composer.stepLabel:1") as HTMLInputElement).value).toBe(
"typed",
);
});
it("takes an unticked line out of the evidence", async () => {
mount();
fireEvent.click(
(await screen.findAllByRole("button", { name: "recorder.composer.review" }))[0]!,
);
fireEvent.click(
screen.getByRole("checkbox", { name: "recorder.composer.keepRow:ui.click,private click" }),
);
fireEvent.click(screen.getByRole("button", { name: "recorder.composer.reviewDone" }));
expect(screen.getByText(/recorder.composer.linesKept:1,2/)).not.toBeNull();
await waitFor(() => expect(api.saved.length).toBeGreaterThan(0), { timeout: 2000 });
const clickId = api.rows[1]!.id;
expect(lastSaved().excluded).toEqual([clickId]);
expect(lastSaved().body).toContain("recorder.issue.clientLog:1");
});
it("adds a pasted image, and nothing else that was pasted", async () => {
mount();
const zone = (await screen.findByText("recorder.composer.attachments")).parentElement!;
const png = new File([new Uint8Array([137, 80, 78, 71])], "shot.png", { type: "image/png" });
const text = new File(["hi"], "note.txt", { type: "text/plain" });
fireEvent.paste(zone, { clipboardData: { files: [png, text] } });
await waitFor(() => expect(api.uploaded).toHaveLength(1));
expect(api.uploaded[0]!.type).toBe("image/png");
});
it("takes a dropped or picked image, and nothing else", async () => {
mount();
const zone = (await screen.findByText(/recorder.composer.drop/)).closest("div")!;
const png = new File([new Uint8Array([137, 80, 78, 71])], "a.png", { type: "image/png" });
const pdf = new File(["%PDF"], "a.pdf", { type: "application/pdf" });
fireEvent.drop(zone, { dataTransfer: { files: [pdf, png] } });
await waitFor(() => expect(api.uploaded).toHaveLength(1));
expect(api.uploaded[0]!.type).toBe("image/png");
});
it("takes an image pasted while typing, anywhere in the composer", async () => {
mount();
await screen.findByText("recorder.composer.attachments");
const png = new File([new Uint8Array([137, 80, 78, 71])], "shot.png", { type: "image/png" });
fireEvent.paste(title(), { clipboardData: { files: [png] } });
await waitFor(() => expect(api.uploaded).toHaveLength(1));
});
it("saves an edit still waiting when it is closed", async () => {
mount();
await ready("recorder.composer.export");
fireEvent.change(title(), { target: { value: "Typed and closed" } });
cleanup(); // closed well inside the pause
await waitFor(() => expect(api.saved).toHaveLength(1));
expect(lastSaved().title).toBe("Typed and closed");
});
it("does not call a newer edit saved", async () => {
mount();
await ready("recorder.composer.export");
fireEvent.change(title(), { target: { value: "one" } });
expect(
await screen.findByText("recorder.composer.saved", {}, { timeout: 2000 }),
).not.toBeNull();
fireEvent.change(title(), { target: { value: "two" } });
expect(screen.queryByText("recorder.composer.saved")).toBeNull();
});
it("says so when the screenshots cannot be read, and still counts them from the session", async () => {
api.failShots = true;
api.session = baseSession({ screenshots: 2 });
mount();
expect(await screen.findByText("recorder.composer.shotsFailed")).not.toBeNull();
// Everything still works: the images are the one thing that could not be listed.
expect(screen.queryByText("recorder.composer.attachments")).not.toBeNull();
expect(
(screen.getByRole("button", { name: "recorder.composer.export" }) as HTMLButtonElement)
.disabled,
).toBe(false);
// Including the control standing right beside the failed list: more can still be added.
expect(screen.queryByText("recorder.composer.pick")).not.toBeNull();
// The draft is still saved — and says two screenshots, the server's own count, not none.
fireEvent.change(title(), { target: { value: "x" } });
await waitFor(() => expect(api.saved).toHaveLength(1), { timeout: 2000 });
expect(lastSaved().body).toContain("recorder.issue.screenshots:2");
});
it("says a send is under way instead of calling it half-failed, and holds the draft", async () => {
// Every step of a send commits its progress, so a reload or a second tab reads exactly the
// shape of a send that STOPPED. Only the server can tell the two apart — and it does.
api.session = baseSession({
issue_url: "https://git.example/o/r/issues/7",
pending_files: ["server.log", "screenshot-1.png"],
sending: true,
});
mount();
expect(await screen.findByText("recorder.composer.sendingNow")).not.toBeNull();
expect(screen.queryByText(/recorder.composer.partial/)).toBeNull();
// No retry to press: it would only be refused while the send it invites is still running.
expect(screen.queryByRole("button", { name: "recorder.composer.retry" })).toBeNull();
fireEvent.change(title(), { target: { value: "typed while it went out" } });
await new Promise((r) => setTimeout(r, 1000));
expect(api.saved).toHaveLength(0);
});
it("holds the draft through a first send, before there is an issue to point at", async () => {
tracker = { repo: "o/r" };
// A send with no issue_url yet is what the read-only lock is for: an edit made mid-flight
// would be saved over the very body the send is at that moment uploading, and the issue that
// arrives would be neither draft. The half-sent case above is locked by its link instead.
api.session = baseSession({ sending: true });
mount();
expect(await screen.findByText("recorder.composer.sendingNow")).not.toBeNull();
fireEvent.change(title(), { target: { value: "typed mid-flight" } });
await new Promise((r) => setTimeout(r, 1000));
expect(api.saved).toHaveLength(0);
// Nor can a screenshot join what is already on its way up.
expect(screen.queryByText("recorder.composer.pick")).toBeNull();
// And the button that would start a SECOND send is out — the server would only refuse it.
expect(
(screen.getByRole("button", { name: "recorder.composer.send" }) as HTMLButtonElement)
.disabled,
).toBe(true);
});
it("announces the half-sent banner instead of only drawing it", async () => {
// A screen-reader user hears the failure toast — "couldn't send the report" — and nothing else.
// WHICH attachments are missing, and that a Retry now stands beside them, live only in this
// banner, which is inserted already carrying its first message. A live region created together
// with its content is unreliably announced (the rule ui/Toaster.tsx states and follows), so the
// announcement has to come from a ROLE, which announces on insertion.
api.session = baseSession({
issue_url: "https://git.example/o/r/issues/7",
pending_files: ["server.log"],
});
mount();
const said = await screen.findByText(/recorder\.composer\.partial/);
const announced = said.closest('[role="status"]');
expect(announced).not.toBeNull();
// ...and the Retry is inside what gets read out, not left behind silently beside it.
expect(announced!.querySelector("button")).not.toBeNull();
});
it("hands the focus on when a send half-fails, not only when it works", async () => {
tracker = { repo: "o/r" };
// The issue is created and an upload then dies, so the call FAILS with the link already
// committed. The refetch brings the link back, `sent` flips, and "Create issue" — the control
// holding the focus — unmounts under it. Without a hand-off the focus lands on <body> in the
// middle of an open dialog: a screen reader loses its place and the Retry that just appeared
// is several Tabs away, past the dialog's own ✕.
api.failSendAfterCreate = true;
mount();
const send = (await ready("recorder.composer.send")) as HTMLButtonElement;
send.focus();
expect(document.activeElement).toBe(send);
fireEvent.click(send);
await waitFor(() => expect(api.sent).toBe(1));
const retry = await screen.findByRole("button", { name: "recorder.composer.retry" });
await waitFor(() => expect(document.activeElement).toBe(retry));
expect(screen.queryByRole("button", { name: "recorder.composer.send" })).toBeNull();
});
it("still builds the zip while a send holds the draft, instead of saving into a refusal", async () => {
tracker = { repo: "o/r" };
// The zip is the only way out of a send that is stuck — a claim left by a dead request stands
// for ten minutes. Its pre-save is pointless here (the draft is frozen, and the server would
// 409 `issue_sending` and take the export down with it), and unnecessary: `GET /export` has no
// send guard and builds from the saved draft, which is exactly what the send is uploading.
api.session = baseSession({ sending: true });
const click = vi.spyOn(HTMLAnchorElement.prototype, "click");
click.mockImplementation(() => void api.order.push("click"));
const real = { make: URL.createObjectURL, revoke: URL.revokeObjectURL };
URL.createObjectURL = () => "blob:zip";
URL.revokeObjectURL = () => {};
try {
mount();
expect(await screen.findByText("recorder.composer.sendingNow")).not.toBeNull();
fireEvent.click(await ready("recorder.composer.export"));
await waitFor(() => expect(click).toHaveBeenCalled());
expect(api.order).toEqual(["export", "click"]);
expect(api.saved).toHaveLength(0);
} finally {
click.mockRestore();
URL.createObjectURL = real.make;
URL.revokeObjectURL = real.revoke;
}
});
it("asks to show the images, not to send what is already there", async () => {
tracker = { repo: "o/r" };
api.session = baseSession({
issue_url: "https://git.example/o/r/issues/7",
images_pending: true,
});
mount();
await screen.findByText("recorder.composer.partialImages");
// Nothing is missing from the issue here — only its text has still to show what it carries.
expect(screen.queryByRole("button", { name: "recorder.composer.retry" })).toBeNull();
fireEvent.click(screen.getByRole("button", { name: "recorder.composer.retryImages" }));
await waitFor(() => expect(api.sent).toBe(1));
// The button just pressed is gone with its banner: the focus goes to what takes its place,
// not to <body> in the middle of an open dialog.
await waitFor(() =>
expect(document.activeElement?.getAttribute("href")).toBe("https://git.example/o/r/issues/7"),
);
});
it("leaves an image pasted into a dialog opened over it alone", async () => {
mount();
await screen.findByText("recorder.composer.attachments");
const above = document.createElement("div");
above.setAttribute("aria-modal", "true");
const field = document.createElement("input");
above.appendChild(field);
document.body.appendChild(above);
const png = new File([new Uint8Array([137, 80, 78, 71])], "shot.png", { type: "image/png" });
fireEvent.paste(field, { clipboardData: { files: [png] } });
await new Promise((r) => setTimeout(r, 50));
expect(api.uploaded).toHaveLength(0);
above.remove();
});
it("moves the focus to what stays when a control goes", async () => {
mount();
fireEvent.click(
(await screen.findAllByRole("button", { name: "recorder.composer.review" }))[0]!,
);
const done = screen.getByRole("button", { name: "recorder.composer.reviewDone" });
expect(document.activeElement).toBe(done);
fireEvent.click(done);
expect(document.activeElement).toBe(
screen.getAllByRole("button", { name: "recorder.composer.review" })[0],
);
fireEvent.click(screen.getByRole("button", { name: "recorder.composer.removeStep:1" }));
expect(document.activeElement).toBe(
screen.getByRole("button", { name: "recorder.composer.addStep" }),
);
});
it("leaves a paste without an image to the field it was typed into", async () => {
mount();
const zone = (await screen.findByText("recorder.composer.attachments")).parentElement!;
const pasted = fireEvent.paste(zone, { clipboardData: { files: [] } });
expect(pasted).toBe(true); // not prevented
expect(api.uploaded).toHaveLength(0);
});
it("saves before it asks for the zip, and hands it to the browser", async () => {
const click = vi.spyOn(HTMLAnchorElement.prototype, "click");
click.mockImplementation(() => void api.order.push("click"));
const urls = { made: 0, revoked: 0 };
// jsdom implements neither, so there is nothing to spy on: they are DEFINED here — and put
// back afterwards, or every later test in this file would keep running against them.
const real = { make: URL.createObjectURL, revoke: URL.revokeObjectURL };
URL.createObjectURL = () => {
urls.made += 1;
return "blob:zip";
};
URL.revokeObjectURL = () => void (urls.revoked += 1);
mount();
fireEvent.click(await ready("recorder.composer.export"));
await waitFor(() => expect(click).toHaveBeenCalled());
// A request, not a bare navigation: a refusal (the export's own pace) is shown like any other.
expect(api.order).toEqual(["save", "saved", "export", "click"]);
expect([urls.made, urls.revoked]).toEqual([1, 1]);
click.mockRestore();
URL.createObjectURL = real.make;
URL.revokeObjectURL = real.revoke;
});
it("a sent session is read-only and links its issue", async () => {
api.session = baseSession({ issue_url: "https://git.example/o/r/issues/7" });
mount();
const link = await screen.findByRole("link", { name: /recorder.composer.openIssue/ });
expect(link.getAttribute("href")).toBe("https://git.example/o/r/issues/7");
const title = screen.getByLabelText("recorder.composer.fieldTitle") as HTMLInputElement;
expect(title.readOnly).toBe(true);
expect(screen.queryByRole("button", { name: /recorder.composer.addStep/ })).toBeNull();
});
it("says so when the session cannot be loaded, with a retry", async () => {
api.failSession = true;
mount();
const message = await screen.findByText("recorder.composer.loadFailed");
expect(within(message.closest('[role="alert"]')!).getByRole("button")).not.toBeNull();
});
it("offers the tracker only where one is set up, and sends the saved draft", async () => {
mount();
await screen.findByRole("button", { name: "recorder.composer.export" });
expect(screen.queryByRole("button", { name: "recorder.composer.send" })).toBeNull();
cleanup();
tracker = { repo: "o/r" };
mount();
expect(await screen.findByText("recorder.composer.target:o/r")).not.toBeNull();
fireEvent.click(await ready("recorder.composer.send"));
await waitFor(() => expect(api.sent).toBe(1));
// Saved, and answered, first: the issue is built from what was saved.
expect(api.order).toEqual(["save", "saved", "send"]);
expect(await screen.findByRole("link", { name: /recorder.composer.openIssue/ })).not.toBeNull();
});
it("holds the draft while it is being sent", async () => {
tracker = { repo: "o/r" };
let release = () => {};
api.holdSend = new Promise((r) => (release = r));
mount();
fireEvent.click(await ready("recorder.composer.send"));
await waitFor(() => expect(api.order).toContain("send"));
// What is typed now would not reach the issue: the fields do not take it.
expect(title().readOnly).toBe(true);
expect(screen.queryByRole("button", { name: /recorder.composer.addStep/ })).toBeNull();
release();
await waitFor(() => expect(api.sent).toBe(1));
});
it("finishes a send that stopped half-way, and says what is missing", async () => {
tracker = { repo: "o/r" };
api.session = baseSession({
issue_url: "https://git.example/o/r/issues/7",
pending_files: ["server.log"],
});
mount();
expect(await screen.findByText("recorder.composer.partial:server.log")).not.toBeNull();
fireEvent.click(screen.getByRole("button", { name: "recorder.composer.retry" }));
await waitFor(() => expect(api.sent).toBe(1));
expect(api.saved).toHaveLength(0); // sent: nothing to save, only the rest to send
});
it("previews through the tracker in a frame where no script runs", async () => {
tracker = { repo: "o/r" };
mount();
fireEvent.click(await screen.findByRole("button", { name: "recorder.composer.preview" }));
const frame = (await screen.findByTitle("recorder.composer.previewTitle")) as HTMLIFrameElement;
expect(frame.getAttribute("sandbox")).not.toContain("allow-scripts");
expect(frame.getAttribute("srcdoc")).toContain("<h3>Steps</h3>");
expect(api.previewed[0]).toContain("1. Resume");
});
it("previews the zip's own text where there is no tracker", async () => {
mount();
fireEvent.click(await screen.findByRole("button", { name: "recorder.composer.preview" }));
expect(screen.getByText(/1\. Resume/).tagName).toBe("PRE");
expect(api.previewed).toHaveLength(0);
});
});
+849
View File
@@ -0,0 +1,849 @@
import { useEffect, useMemo, useRef, useState, type DragEvent, type MutableRefObject } from "react";
import { useTranslation } from "react-i18next";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { ExternalLink, FileText, Plus, Server, X } from "lucide-react";
import { api, type DebugScreenshot, type DebugSession, type ServerLogRow } from "../lib/api";
import { closeComposer } from "../lib/debugRecorder";
import { formatBytes } from "../lib/format";
import { notify } from "../lib/notifications";
import { useMe } from "../lib/useMe";
import {
draftSteps,
environment,
renderIssue,
type DraftForm,
type DraftStep,
type IssueText,
} from "../lib/issueDraft";
import { moduleDef } from "../lib/modules";
import type { Page } from "../lib/urlState";
import { qk } from "../lib/queryKeys";
import { LEVEL_TAG, logClock } from "./logLineParts";
import { Button } from "./ui/Button";
import Modal from "./ui/Modal";
import { LoadingState, StateMessage } from "./ui/QueryState";
import SegmentedToggle from "./ui/SegmentedToggle";
import { inputCls } from "./ui/form";
// The issue composer (E16 S3, D4 A): one dialog — the draft on the left (title, description, the
// steps drafted from the session, the environment), what goes with it on the right (the two logs,
// each reviewable line by line, and the screenshots, removable one by one, with paste / drop / pick
// to add more — an image pasted anywhere in it is taken). It saves as it goes, and what is still
// waiting to be saved when it closes is saved then, so closing it loses nothing: Settings →
// Diagnosztika reopens any past session. The export and (S4) the issue are built from what it SAVED.
const SAVE_DELAY_MS = 800;
type Source = "web" | "server";
const sourceOf = (row: ServerLogRow): Source => (row.source === "web" ? "web" : "server");
export default function IssueComposer({ id }: { id: string }) {
const { t } = useTranslation();
const sessionQ = useQuery({ queryKey: qk.debugSession(id), queryFn: () => api.debugSession(id) });
const eventsQ = useQuery({
queryKey: qk.debugSessionEvents(id),
queryFn: () => api.debugSessionEvents(id),
});
const failed = sessionQ.isError || eventsQ.isError;
// Closed after a Stop, the focus has nowhere to go back to — the Stop button went with the pill —
// and would be left on <body>. The page the user is on is the next best place. After the dialog's
// own restore (a timeout: its cleanup runs after this one), and only when that had nothing.
useEffect(
() => () => {
window.setTimeout(() => {
if (document.activeElement === document.body)
document.querySelector<HTMLElement>('nav [aria-current="page"]')?.focus();
}, 0);
},
[],
);
return (
<Modal title={t("recorder.composer.title")} onClose={closeComposer} maxWidth="max-w-5xl">
{failed && !(sessionQ.data && eventsQ.data) ? (
<StateMessage
tone="error"
onRetry={() => {
void sessionQ.refetch();
void eventsQ.refetch();
}}
>
{t("recorder.composer.loadFailed")}
</StateMessage>
) : !sessionQ.data || !eventsQ.data ? (
<LoadingState />
) : (
<ComposerBody
session={sessionQ.data}
rows={eventsQ.data.items}
total={eventsQ.data.total}
/>
)}
</Modal>
);
}
/** The saved form if it has the shape this composer writes; otherwise null (an older or foreign
* draft starts over from the session rather than half-applying). */
function savedForm(session: DebugSession): DraftForm | null {
const form = session.draft?.form as Partial<DraftForm> | undefined;
if (!form || typeof form.title !== "string" || !Array.isArray(form.steps)) return null;
return {
title: form.title,
description: typeof form.description === "string" ? form.description : "",
steps: form.steps as DraftStep[],
excluded: Array.isArray(form.excluded) ? (form.excluded as number[]) : [],
};
}
function ComposerBody({
session,
rows,
total,
}: {
session: DebugSession;
rows: ServerLogRow[];
total: number;
}) {
const { t, i18n } = useTranslation();
const qc = useQueryClient();
const pageName = (page: string) => {
const def = moduleDef(page as Page);
return def ? t(def.labelKey) : page;
};
const [form, setForm] = useState<DraftForm>(
() =>
savedForm(session) ?? {
title: "",
description: "",
steps: draftSteps(rows, pageName, logClock),
excluded: [],
},
);
const [reviewing, setReviewing] = useState<Source | null>(null);
const issueLink = useRef<HTMLAnchorElement>(null);
const retryButton = useRef<HTMLButtonElement>(null);
// The log whose review just closed: its Review button takes the focus back.
const reviewed = useRef<Source | null>(null);
const sent = session.issue_url !== null;
const shotsQ = useQuery({
queryKey: qk.debugScreenshots(session.id),
queryFn: () => api.debugScreenshots(session.id),
});
const shots = shotsQ.data?.items ?? [];
// How many the ISSUE will carry. From the list while it is loaded, else from the session itself
// (the server's own count) — a failed read must not quietly become "no screenshots".
const shotCount = shotsQ.data ? shots.length : session.screenshots;
const excluded = useMemo(() => new Set(form.excluded), [form.excluded]);
const kept = (source: Source) =>
rows.filter((r) => sourceOf(r) === source && !excluded.has(r.id)).length;
const env = useMemo(() => environment(i18n.language), [i18n.language]);
const text: IssueText = {
steps: t("recorder.issue.steps"),
environment: t("recorder.issue.environment"),
envKeys: Object.fromEntries(env.map(([k]) => [k, t(`recorder.composer.env.${k}`)])),
screenshot: t("recorder.issue.screenshot"),
attachments: t("recorder.issue.attachments"),
clientLog: (count) => t("recorder.issue.clientLog", { count }),
serverLog: (count) => t("recorder.issue.serverLog", { count }),
screenshots: (count) => t("recorder.issue.screenshots", { count }),
truncated:
total > rows.length
? t("recorder.composer.truncated", { kept: rows.length, total })
: undefined,
};
const body = renderIssue(
form,
env,
{ client: kept("web"), server: kept("server"), screenshots: shotCount },
text,
);
// Saves as it goes — after the typing pauses, not on every key. `edits` counts the edits, so a
// save that finishes after a newer edit does not call that one saved.
const edits = useRef(0);
const [unsaved, setUnsaved] = useState(false);
const save = useMutation({
mutationFn: async () => {
const upTo = edits.current;
const r = await api.saveDebugDraft(session.id, {
title: form.title.trim(),
body,
excluded: form.excluded,
form: form as unknown as Record<string, unknown>,
});
return { ...r, upTo };
},
meta: { failTitle: "recorder.composer.saveFailed" },
onSuccess: (r) => {
qc.setQueryData(qk.debugSession(session.id), r.session);
void qc.invalidateQueries({ queryKey: qk.debugSessions(), exact: true });
if (r.upTo === edits.current) setUnsaved(false);
},
});
const edit = (next: (f: DraftForm) => DraftForm) => {
edits.current += 1;
setUnsaved(true);
setForm(next);
};
// The issue (S4), like the export, is built from the SAVED draft: save what is on screen first.
// A send that stopped half-way is finished by the same call, which does only what is missing.
// While it runs the draft is held as it was saved: nothing typed then would reach the issue.
const target = useMe()?.issues ?? null;
const pending = session.pending_files;
// A send is running — this tab's or another's. Its progress is committed step by step, so the
// half-sent shape below is also what a LIVE send looks like; only the server can tell them apart.
const sending = session.sending;
const send = useMutation({
mutationFn: async () => {
if (!sent) await save.mutateAsync();
return api.sendDebugIssue(session.id);
},
meta: { failTitle: "recorder.composer.sendFailed" },
onSuccess: (r) => {
qc.setQueryData(qk.debugSession(session.id), r.session);
notify({ level: "success", message: t("recorder.composer.sentOk") });
},
// A failure half-way still created the issue: show what the server now has.
onSettled: async () => {
void qc.invalidateQueries({ queryKey: qk.debugSessions(), exact: true });
await qc.invalidateQueries({ queryKey: qk.debugSession(session.id), exact: true });
// Whichever way it went, the button that was just pressed is gone: a success unmounts the
// Retry's banner, and a half-failure unmounts Create issue (the refetch above brings back the
// issue_url it created before it died). Either leaves the focus on <body> inside an open
// dialog, so hand it on — to whatever now stands where the pressed control was. AFTER the
// refetch, not in onSuccess/onError: before it lands, neither target is rendered yet.
window.setTimeout(() => (retryButton.current ?? issueLink.current)?.focus(), 0);
},
});
const locked = sent || send.isPending || sending;
const saveRef = useRef(save.mutate);
saveRef.current = save.mutate;
useEffect(() => {
if (locked || !unsaved) return;
const timer = window.setTimeout(() => saveRef.current(), SAVE_DELAY_MS);
return () => window.clearTimeout(timer);
}, [body, form.title, locked, unsaved]);
// Closed before the typing paused: save what is still waiting, now.
const flushRef = useRef(() => {});
flushRef.current = () => {
if (unsaved && !locked) save.mutate();
};
useEffect(() => () => flushRef.current(), []);
// Screenshots are added here, not in the attachments column: an image pasted ANYWHERE in the
// composer is taken — while typing the description is exactly when one is pasted.
const addShot = useMutation({
mutationFn: (image: Blob) => api.addDebugScreenshot(session.id, image),
meta: { failTitle: "recorder.pill.shotFailed" },
onSuccess: () => refreshShots(qc, session.id),
});
const addRef = useRef(addShot.mutate);
addRef.current = addShot.mutate;
const root = useRef<HTMLDivElement>(null);
useEffect(() => {
if (locked) return;
// Only an image paste is taken over: text pasted into the title or a step goes where it was typed.
// And only OUR dialog's: something opened over the composer (the command palette) keeps its own.
const onPaste = (e: ClipboardEvent) => {
const dialog = e.target instanceof Element ? e.target.closest('[aria-modal="true"]') : null;
if (dialog && !dialog.contains(root.current)) return;
const images = [...(e.clipboardData?.files ?? [])].filter(isImage);
if (!images.length) return;
e.preventDefault();
for (const image of images) addRef.current(image);
};
document.addEventListener("paste", onPaste);
return () => document.removeEventListener("paste", onPaste);
}, [locked]);
const [mode, setMode] = useState<"edit" | "preview">("edit");
// The export is built from the SAVED draft: save what is on screen first — except when the
// draft is already frozen (`locked`), where the save is both pointless and refused. While a send
// holds the claim the server 409s `issue_sending`, and that refusal would take the zip with it:
// `GET /export` has no send guard and would have served exactly what the send is uploading, so
// the one tracker-free way out of a stuck send used to fail at the last step (round 4).
const exportZip = useMutation({
mutationFn: async () => {
if (!locked) await save.mutateAsync();
return api.debugExport(session.id);
},
meta: { failTitle: "recorder.composer.exportFailed" },
onSuccess: ({ blob, name }) => {
const url = URL.createObjectURL(blob);
const a = document.createElement("a");
a.href = url;
a.download = name;
a.click();
URL.revokeObjectURL(url);
},
});
const started = new Date(session.started_at);
const minutes = Math.max(
1,
Math.round(
(Date.parse(session.ended_at ?? new Date().toISOString()) - started.getTime()) / 60_000,
),
);
return (
<div ref={root} className="flex flex-col gap-4">
<p className="text-sm text-muted">
{t("recorder.composer.meta", {
start: started.toLocaleString(i18n.language),
minutes,
level: t(`settings.diagnostics.levels.${session.level}`),
})}
</p>
{(sending || (sent && (pending.length > 0 || session.images_pending))) && (
<div
className={`flex flex-wrap items-center justify-between gap-2 rounded-xl border px-3 py-2 text-sm ${
sending ? "border-accent/40 bg-accent/10" : "border-warn/40 bg-warn/10"
}`}
// `role="status"` rather than `aria-live`: this box is mounted TOGETHER with its first
// message (a half-failed send inserts it already populated), and a live region created
// with its content is unreliably announced — the rule ui/Toaster.tsx states and follows.
// A role announces on insertion, and its implied atomic re-read is right for one line.
role="status"
>
<span>
{sending
? t("recorder.composer.sendingNow")
: pending.length > 0
? t("recorder.composer.partial", { files: pending.join(", ") })
: t("recorder.composer.partialImages")}
</span>
{/* Nothing to retry while it is still going — the button would only be refused. */}
{!sending && (
<Button
ref={retryButton}
variant="outline"
size="sm"
loading={send.isPending}
onClick={() => send.mutate()}
>
{t(pending.length > 0 ? "recorder.composer.retry" : "recorder.composer.retryImages")}
</Button>
)}
</div>
)}
{sent && (
<div className="flex flex-wrap items-center justify-between gap-2 rounded-xl border border-success/40 bg-success/10 px-3 py-2 text-sm">
<span>{t("recorder.composer.sent")}</span>
<a
ref={issueLink}
href={session.issue_url!}
target="_blank"
rel="noreferrer"
className="inline-flex items-center gap-1 font-semibold text-accent hover:underline"
>
{t("recorder.composer.openIssue")}
<ExternalLink className="w-3.5 h-3.5" aria-hidden />
</a>
</div>
)}
{!reviewing && (
<SegmentedToggle
size="md"
className="self-start"
ariaLabel={t("recorder.composer.previewTitle")}
options={[
{ value: "edit", label: t("recorder.composer.edit") },
{ value: "preview", label: t("recorder.composer.preview") },
]}
value={mode}
onChange={setMode}
/>
)}
{mode === "preview" && !reviewing ? (
<IssuePreview
sessionId={session.id}
text={`# ${form.title.trim() || "…"}\n\n${body}`}
viaTracker={!!target}
/>
) : reviewing ? (
<ReviewList
rows={rows.filter((r) => sourceOf(r) === reviewing)}
name={t(
reviewing === "web" ? "recorder.composer.clientLog" : "recorder.composer.serverLog",
)}
excluded={excluded}
readOnly={locked}
onToggle={(rowId) =>
edit((f) => ({
...f,
excluded: f.excluded.includes(rowId)
? f.excluded.filter((x) => x !== rowId)
: [...f.excluded, rowId],
}))
}
onDone={() => {
reviewed.current = reviewing;
setReviewing(null);
}}
/>
) : (
<div className="grid grid-cols-1 gap-5 md:grid-cols-[minmax(0,1.45fr)_minmax(0,1fr)]">
<DraftFields form={form} edit={edit} env={env} readOnly={locked} />
<Attachments
session={session}
shotsQ={shotsQ}
counts={{
web: [kept("web"), rows.filter((r) => sourceOf(r) === "web").length],
server: [kept("server"), rows.filter((r) => sourceOf(r) === "server").length],
}}
readOnly={locked}
adding={addShot.isPending}
onAdd={(files) => {
for (const file of files) if (isImage(file)) addShot.mutate(file);
}}
onReview={setReviewing}
reviewed={reviewed}
/>
</div>
)}
<div className="flex flex-wrap items-center justify-end gap-2 border-t border-border pt-3">
<span className="mr-auto flex flex-col text-xs text-muted">
{target && <span>{t("recorder.composer.target", { repo: target.repo })}</span>}
<span aria-live="polite">
{send.isPending
? t("recorder.composer.sending")
: save.isPending
? t("recorder.composer.saving")
: save.isSuccess && !unsaved
? t("recorder.composer.saved")
: ""}
</span>
</span>
<Button variant="ghost" onClick={closeComposer}>
{t("recorder.composer.close")}
</Button>
<Button
variant="outline"
loading={save.isPending || exportZip.isPending}
disabled={send.isPending}
onClick={() => exportZip.mutate()}
>
{t("recorder.composer.export")}
</Button>
{target && !sent && (
// `sending` is a send ALREADY under way — from a reload, or from the other tab this one
// is a copy of. The button would only earn an `issue_sending` refusal; the banner above
// says why it is out.
<Button
variant="primary"
loading={send.isPending}
disabled={sending}
onClick={() => send.mutate()}
>
{t("recorder.composer.send")}
</Button>
)}
</div>
</div>
);
}
function DraftFields({
form,
edit,
env,
readOnly,
}: {
form: DraftForm;
edit: (next: (f: DraftForm) => DraftForm) => void;
env: [string, string][];
readOnly: boolean;
}) {
const { t } = useTranslation();
// A removed step's ✕ goes with it: the focus moves to "add a step", which stays.
const addStep = useRef<HTMLButtonElement>(null);
const setStep = (i: number, patch: Partial<DraftStep>) =>
edit((f) => ({ ...f, steps: f.steps.map((s, j) => (j === i ? { ...s, ...patch } : s)) }));
return (
<div className="flex flex-col gap-4 min-w-0">
<div className="flex flex-col gap-1.5">
<label htmlFor="issue-title" className="text-sm text-muted">
{t("recorder.composer.fieldTitle")}
</label>
<input
id="issue-title"
value={form.title}
readOnly={readOnly}
maxLength={200}
placeholder={t("recorder.composer.titlePlaceholder")}
onChange={(e) => edit((f) => ({ ...f, title: e.target.value }))}
className={inputCls}
/>
</div>
<div className="flex flex-col gap-1.5">
<label htmlFor="issue-description" className="text-sm text-muted">
{t("recorder.composer.description")}
</label>
<textarea
id="issue-description"
value={form.description}
readOnly={readOnly}
rows={4}
placeholder={t("recorder.composer.descriptionPlaceholder")}
onChange={(e) => edit((f) => ({ ...f, description: e.target.value }))}
className={`${inputCls} resize-y`}
/>
</div>
<fieldset className="flex flex-col gap-1.5 min-w-0">
<legend className="mb-1.5 text-sm text-muted">{t("recorder.composer.steps")}</legend>
<ol className="flex flex-col gap-1.5">
{form.steps.map((step, i) => (
<li
key={step.id}
className="flex items-center gap-2 rounded-lg bg-surface/60 pl-2.5 pr-1 py-1"
>
<span className="w-6 shrink-0 text-sm font-semibold text-accent tabular-nums">
{i + 1}.
</span>
<input
aria-label={t("recorder.composer.stepLabel", { n: i + 1 })}
value={step.text}
readOnly={readOnly}
placeholder={t("recorder.composer.stepPlaceholder")}
onChange={(e) => setStep(i, { text: e.target.value })}
className="min-w-0 flex-1 bg-transparent text-sm outline-none focus-visible:ring-2 focus-visible:ring-accent/60 rounded px-1 py-1"
/>
<span className="chip shrink-0 px-2 py-0.5 text-2xs">
{t(`recorder.composer.stepKind.${step.kind}`)}
{step.shot ? ` · ${t("recorder.composer.withShot")}` : ""}
</span>
{!readOnly && (
<Button
variant="ghost"
size="icon-sm"
iconOnly
icon={<X className="w-4 h-4" />}
label={t("recorder.composer.removeStep", { n: i + 1 })}
onClick={() => {
edit((f) => ({ ...f, steps: f.steps.filter((_, j) => j !== i) }));
addStep.current?.focus();
}}
/>
)}
</li>
))}
</ol>
{!readOnly && (
<Button
ref={addStep}
variant="link"
className="self-start"
icon={<Plus className="w-4 h-4" />}
onClick={() =>
edit((f) => ({
...f,
steps: [...f.steps, { id: `m${Date.now()}`, kind: "manual", text: "" }],
}))
}
>
{t("recorder.composer.addStep")}
</Button>
)}
</fieldset>
<div className="flex flex-col gap-1.5">
<div className="text-sm text-muted">{t("recorder.composer.environment")}</div>
<dl className="grid grid-cols-[auto_minmax(0,1fr)] gap-x-4 gap-y-1 text-sm">
{env.map(([k, v]) => (
<div key={k} className="contents">
<dt className="text-muted">{t(`recorder.composer.env.${k}`)}</dt>
<dd className="break-words tabular-nums">{v}</dd>
</div>
))}
</dl>
</div>
</div>
);
}
// The types the server keeps (backend debugsession.IMAGE_TYPES) — offered by the picker, and the
// only ones a paste or a drop hands over.
const IMAGE_TYPES = ["image/png", "image/jpeg", "image/webp"];
const isImage = (file: File) => IMAGE_TYPES.includes(file.type);
/** What changes with a screenshot: its list, and the session's count — not the session's events,
* which share the session's key prefix and are thousands of rows. */
function refreshShots(qc: ReturnType<typeof useQueryClient>, id: string) {
void qc.invalidateQueries({ queryKey: qk.debugScreenshots(id) });
void qc.invalidateQueries({ queryKey: qk.debugSession(id), exact: true });
}
function Attachments({
session,
shotsQ,
counts,
readOnly,
adding,
onAdd,
onReview,
reviewed,
}: {
session: DebugSession;
shotsQ: { data?: { items: DebugScreenshot[] }; isError: boolean; refetch: () => unknown };
counts: Record<Source, [kept: number, total: number]>;
readOnly: boolean;
adding: boolean;
onAdd: (files: Iterable<File>) => void;
onReview: (source: Source) => void;
reviewed: MutableRefObject<Source | null>;
}) {
const { t } = useTranslation();
const qc = useQueryClient();
const shots = shotsQ.data?.items ?? [];
// A removed screenshot's ✕ goes with it: the focus moves to the file picker, which stays.
const pick = useRef<HTMLInputElement>(null);
const reviewButtons = useRef<Partial<Record<Source, HTMLButtonElement | null>>>({});
// eslint-disable-next-line react-hooks/immutability
useEffect(() => {
const source = reviewed.current;
if (!source) return;
// eslint-disable-next-line react-hooks/immutability -- a ONE-SHOT token, consumed: the parent sets it to say "give this source's review button the focus back when the list re-renders", and clearing it here is what stops the focus being stolen on every later render. The ref is the caller's, which is why it is a dependency rather than a local.
reviewed.current = null;
reviewButtons.current[source]?.focus();
}, [reviewed]);
const remove = useMutation({
mutationFn: (shotId: number) => api.removeDebugScreenshot(session.id, shotId),
meta: { failTitle: "recorder.composer.removeShotFailed" },
onSuccess: () => refreshShots(qc, session.id),
});
const onDrop = (e: DragEvent) => {
e.preventDefault();
onAdd(e.dataTransfer.files);
};
const logRow = (source: Source, icon: React.ReactNode) => {
const [kept, total] = counts[source];
return (
<div className="flex items-center gap-3 rounded-xl border border-border bg-card/60 px-3 py-2.5">
{icon}
<div className="flex min-w-0 flex-1 flex-col">
<span className="text-sm">
{t(source === "web" ? "recorder.composer.clientLog" : "recorder.composer.serverLog")}
</span>
<span className="text-xs text-muted tabular-nums">
{kept === total
? t("recorder.composer.lines", { count: total })
: t("recorder.composer.linesKept", { kept, total })}
{` · ${source === "web" ? "client.log" : "server.log"}`}
</span>
</div>
{total > 0 && (
<Button
ref={(el) => {
reviewButtons.current[source] = el;
}}
variant="outline"
size="sm"
onClick={() => onReview(source)}
>
{t("recorder.composer.review")}
</Button>
)}
</div>
);
};
return (
<div className="flex flex-col gap-3 min-w-0 rounded-xl bg-surface/40 p-3 md:p-4">
<div className="text-sm text-muted">{t("recorder.composer.attachments")}</div>
{logRow("web", <FileText className="w-4.5 h-4.5 shrink-0 text-muted" aria-hidden />)}
{logRow("server", <Server className="w-4.5 h-4.5 shrink-0 text-muted" aria-hidden />)}
{shotsQ.isError && !shotsQ.data && (
<StateMessage tone="error" compact onRetry={() => void shotsQ.refetch()}>
{t("recorder.composer.shotsFailed")}
</StateMessage>
)}
{shots.map((shot, i) => (
<div
key={shot.id}
className="flex items-center gap-3 rounded-xl border border-border bg-card/60 px-3 py-2"
>
<img
src={api.debugScreenshotUrl(session.id, shot.id)}
alt={t("recorder.composer.screenshot", { n: i + 1 })}
className="h-10 w-16 shrink-0 rounded-md object-cover bg-surface"
/>
<div className="flex min-w-0 flex-1 flex-col">
<span className="text-sm">{t("recorder.composer.screenshot", { n: i + 1 })}</span>
<span className="text-xs text-muted tabular-nums">{formatBytes(shot.size)}</span>
</div>
{!readOnly && (
<Button
variant="ghost"
size="icon-sm"
iconOnly
icon={<X className="w-4 h-4" />}
label={t("recorder.composer.removeShot", { n: i + 1 })}
loading={remove.isPending && remove.variables === shot.id}
onClick={() => {
remove.mutate(shot.id);
pick.current?.focus();
}}
/>
)}
</div>
))}
{!readOnly && (
<div
onDragOver={(e) => e.preventDefault()}
onDrop={onDrop}
aria-busy={adding}
className="rounded-xl border border-dashed border-border px-3 py-4 text-center text-sm text-muted"
>
{t("recorder.composer.drop")}{" "}
<label className="cursor-pointer rounded font-semibold text-accent hover:underline focus-within:underline focus-within:ring-2 focus-within:ring-accent/60">
{t("recorder.composer.pick")}
<input
ref={pick}
type="file"
accept={IMAGE_TYPES.join(",")}
multiple
className="sr-only"
onChange={(e) => {
onAdd(e.target.files ?? []);
e.target.value = "";
}}
/>
</label>
</div>
)}
</div>
);
}
/** One log, line by line: an unticked line is not sent. */
function ReviewList({
rows,
name,
excluded,
readOnly,
onToggle,
onDone,
}: {
rows: ServerLogRow[];
name: string;
excluded: Set<number>;
readOnly: boolean;
onToggle: (id: number) => void;
onDone: () => void;
}) {
const { t } = useTranslation();
// Opened from its Review button, which is gone now: the focus comes here.
const done = useRef<HTMLButtonElement>(null);
useEffect(() => done.current?.focus(), []);
return (
<div className="flex flex-col gap-3">
<div className="flex flex-wrap items-center justify-between gap-2">
<div>
<div className="font-semibold">{t("recorder.composer.reviewTitle", { name })}</div>
<div className="text-xs text-muted">{t("recorder.composer.reviewHint")}</div>
</div>
<Button ref={done} variant="primary" size="sm" onClick={onDone}>
{t("recorder.composer.reviewDone")}
</Button>
</div>
<ul className="max-h-[60vh] overflow-y-auto rounded-xl border border-border bg-bg/60 font-mono text-xs">
{rows.map((row) => {
const tag = LEVEL_TAG[row.level];
const off = excluded.has(row.id);
return (
<li key={row.id} className="border-b border-border/50 last:border-b-0">
<label
className={`grid cursor-pointer grid-cols-[auto_6.5em_3.5em_minmax(0,1fr)] items-baseline gap-x-2 px-3 py-1 hover:bg-card sm:grid-cols-[auto_6.5em_3.5em_minmax(0,10em)_minmax(0,1fr)] ${
off ? "opacity-50 line-through" : ""
}`}
>
<input
type="checkbox"
checked={!off}
disabled={readOnly}
aria-label={t("recorder.composer.keepRow", {
event: row.event ?? "",
msg: row.msg,
})}
onChange={() => onToggle(row.id)}
/>
<span className="tabular-nums text-muted">{logClock(row.ts)}</span>
<span className={`font-semibold ${tag.className}`}>{tag.text}</span>
<span className="truncate">{row.event}</span>
<span className="col-span-4 truncate sm:col-span-1">{row.msg}</span>
</label>
</li>
);
})}
</ul>
</div>
);
}
/** The report as it will read. Through the tracker, its OWN rendering (Gitea's markdown and
* sanitiser) — drawn in a sandboxed frame: no script runs in it, and nothing it holds reaches the
* app's document. Without a tracker, the markdown itself: it is the zip's issue.md. */
function IssuePreview({
sessionId,
text,
viaTracker,
}: {
sessionId: string;
text: string;
viaTracker: boolean;
}) {
const { t } = useTranslation();
const q = useQuery({
queryKey: qk.debugSessionPreview(sessionId, text),
queryFn: () => api.previewDebugIssue(sessionId, text),
enabled: viaTracker,
});
const frame = useRef<HTMLIFrameElement>(null);
const [height, setHeight] = useState(240);
if (!viaTracker)
return (
<div className="flex flex-col gap-1.5">
<p className="text-xs text-muted">{t("recorder.composer.previewLocal")}</p>
<pre className="max-h-[60vh] overflow-auto whitespace-pre-wrap rounded-xl border border-border bg-bg/60 p-3 font-mono text-xs">
{text}
</pre>
</div>
);
if (q.isError && !q.data)
return (
<StateMessage tone="error" compact onRetry={() => void q.refetch()}>
{t("recorder.composer.previewFailed")}
</StateMessage>
);
if (!q.data) return <LoadingState />;
const css = getComputedStyle(document.documentElement);
const color = (name: string, fallback: string) => css.getPropertyValue(name).trim() || fallback;
// The frame is a document of its own: the app's text size (the root font, which the text-size
// setting drives) is handed to it, and everything in it is sized off that.
const doc = `<!doctype html><html><head><meta charset="utf-8"><base target="_blank"><style>
html{font-size:${css.fontSize}}
body{margin:0;padding:.75rem .875rem;font:.875rem/1.55 system-ui,sans-serif;color:${color("--fg", "#e6e9f0")};background:${color("--card", "#161f38")}}
a{color:${color("--accent", "#6d8cff")}}table{border-collapse:collapse}td,th{border:1px solid ${color("--border", "#243049")};padding:.25rem .5rem}
code{font-family:ui-monospace,Consolas,monospace;font-size:.75rem}h1,h2,h3{line-height:1.25}
</style></head><body>${q.data.html}</body></html>`;
return (
<iframe
ref={frame}
title={t("recorder.composer.previewTitle")}
sandbox="allow-same-origin allow-popups allow-popups-to-escape-sandbox"
srcDoc={doc}
onLoad={() => {
const body = frame.current?.contentDocument?.body;
if (body) setHeight(Math.min(body.scrollHeight + 4, window.innerHeight * 0.6));
}}
style={{ height }}
className="w-full rounded-xl border border-border"
/>
);
}
+1 -16
View File
@@ -2,28 +2,13 @@ import type { ReactNode } from "react";
import type { Level } from "../lib/log";
import { navItemAttrs } from "../lib/listNav";
import { LEVEL_TAG, logClock } from "./logLineParts";
// One log line as both log viewers draw it — "Saját napló" (this tab's ring) and the admin
// diagnostics console (the server's stored log): time, level, event code, message, and the line opens
// in place to whatever detail its screen has. One component, so a level reads the same on both.
// Inside a NavGrid the keyboard walks the lines (`j`/`k`) and Enter opens one, as a click does.
const LEVEL_TAG: Record<Level, { text: string; className: string }> = {
TRACE: { text: "TRACE", className: "text-muted opacity-80" },
DEBUG: { text: "DEBUG", className: "text-muted" },
INFO: { text: "INFO", className: "text-info" },
WARNING: { text: "WARN", className: "text-warn" },
ERROR: { text: "ERROR", className: "text-danger" },
CRITICAL: { text: "FATAL", className: "text-danger" },
};
/** `HH:MM:SS.mmm` in local time — a log is read to the millisecond. */
function logClock(ts: string): string {
const d = new Date(ts);
const pad = (n: number, w = 2) => String(n).padStart(w, "0");
return `${pad(d.getHours())}:${pad(d.getMinutes())}:${pad(d.getSeconds())}.${pad(d.getMilliseconds(), 3)}`;
}
export default function LogLine({
ts,
level,
+10 -6
View File
@@ -1,4 +1,4 @@
import { useEffect, useState } from "react";
import { useEffect, useRef, useState } from "react";
import { qk } from "../lib/queryKeys";
import { useTranslation } from "react-i18next";
import { useQuery } from "@tanstack/react-query";
@@ -85,15 +85,20 @@ function ConversationList({
const items = q.data?.items ?? [];
const [previews, setPreviews] = useState<Record<number, string>>({});
// Decrypt conversation previews when fresh data arrives. Keyed on dataUpdatedAt (not `items`):
// the `?? []` fallback is a new array each render, which would otherwise spin this effect
// (and setPreviews) every render while the query is loading.
// Decrypt conversation previews when fresh data arrives. The TRIGGER is `dataUpdatedAt` and the
// payload is read through a ref, which is what makes the dep list below complete without making
// the identity of `q.data` a trigger. That distinction is load-bearing: the `?? []` fallback is a
// new array every render, and anything handing back a fresh object per render (a stub, a caller
// that rebuilds it) turns "re-run when the data changes" into a render loop through setPreviews.
// Measured the hard way — E27 S2 keyed this on `q.data`, and Messages.keyboard.dom.test hung.
const conversationsRef = useRef(q.data);
conversationsRef.current = q.data;
useEffect(() => {
if (!ready) return;
let cancelled = false;
(async () => {
const out: Record<number, string> = {};
for (const c of q.data?.items ?? []) {
for (const c of conversationsRef.current?.items ?? []) {
const m = c.last_message;
if (m.kind === "user" && m.ciphertext && m.iv) {
try {
@@ -109,7 +114,6 @@ function ConversationList({
return () => {
cancelled = true;
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [q.dataUpdatedAt, ready]);
function previewText(c: Conversation): string {
+1
View File
@@ -33,6 +33,7 @@ function useLogEvents(): LogEvent[] {
setList(events());
});
});
// eslint-disable-next-line react-hooks/set-state-in-effect -- this IS the subscribe-to-an-external-store shape the rule's own text allows: the ring fills outside React, and a sync at subscribe time is what closes the gap between the first render and the listener. useSyncExternalStore would drop the rAF coalescing, which is the point at a few hundred events a second.
setList(events()); // whatever arrived between the first render and the subscription
return () => {
unsubscribe();
+17 -8
View File
@@ -153,10 +153,16 @@ export function NavigationProvider({ children }: { children: ReactNode }) {
// Expose setPage to decoupled callers (download toast "View", etc.) via the lib/nav singleton.
useEffect(() => setNavigator(setPage), [setPage]);
// Leaving the feed ends any live YouTube search, so coming back shows the normal feed.
useEffect(() => {
// Leaving the feed ends any live YouTube search, so coming back shows the normal feed. Adjusted
// DURING the render that changes the page, against the page we were last on: as an effect it ran
// after the new page had already committed, so for one frame a module rendered with a live search
// still in the nav state. Keyed on the CHANGE rather than derived from `page`, because "ended" has
// to be permanent — a derived null would quietly come back when you returned to the feed.
const [pageSearchWasOn, setPageSearchWasOn] = useState(page);
if (pageSearchWasOn !== page) {
setPageSearchWasOn(page);
if (page !== "feed") setYtSearch(null);
}, [page]);
}
// Reflect the current module in the browser tab title. Feed = brand only; an open channel page
// shows the channel name. Signed out, the page on screen is the public landing whatever `page`
@@ -174,6 +180,10 @@ export function NavigationProvider({ children }: { children: ReactNode }) {
document.title = label ? `${label} · ${brand}` : brand;
}, [signedIn, page, channelView, i18n.language, t]);
// The page this tab LOADED on. Frozen, because the stamp below belongs to the initial history
// entry: every later page change writes its own entry through setPage.
const [mountPage] = useState(page);
// In-app Back/Forward: stamp the initial entry with the current page (mirrored to the address bar),
// then sync state from history.state on popstate — the browser restores each entry's URL for us, so
// the popstate handler only needs to read history.state.
@@ -185,12 +195,12 @@ export function NavigationProvider({ children }: { children: ReactNode }) {
// Mirror the resolved initial page to the address bar (feed implicit) so a page reached from the
// stored landing pref — not just from the URL — is itself bookmarkable. urlWithParams preserves any
// ?chan / ?v / ?pl / filter params the loaders already read.
const url = urlWithParams({ page: page === "feed" ? null : page });
const url = urlWithParams({ page: mountPage === "feed" ? null : mountPage });
if (st._yt && st._ytScrape) {
window.history.replaceState({ ...st, sfPage: page }, "", url);
window.history.replaceState({ ...st, sfPage: mountPage }, "", url);
} else {
const { _yt: _staleYt, _ytScrape: _staleScrape, ...rest } = st;
window.history.replaceState({ ...rest, sfPage: page }, "", url);
window.history.replaceState({ ...rest, sfPage: mountPage }, "", url);
}
function onPop(e: PopStateEvent) {
const p = (e.state?.sfPage as Page) ?? "feed";
@@ -207,8 +217,7 @@ export function NavigationProvider({ children }: { children: ReactNode }) {
}
window.addEventListener("popstate", onPop);
return () => window.removeEventListener("popstate", onPop);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
}, [mountPage]);
const state = useMemo<NavigationState>(
() => ({ page, channelView, ytSearch }),
+5 -3
View File
@@ -87,6 +87,7 @@ export default function OnboardingWizard({ me, onClose }: { me: Me; onClose: ()
meta: { errorShownInline: true },
});
const { mutate: startImport } = importSubs;
// First time we see read access with zero imported channels, pull the user's
// subscriptions automatically so they don't have to hunt for a button.
useEffect(() => {
@@ -97,10 +98,11 @@ export default function OnboardingWizard({ me, onClose }: { me: Me; onClose: ()
!importTriggered.current
) {
importTriggered.current = true;
importSubs.mutate();
startImport();
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [me.can_read, myStatus.data?.channels_total]);
// `mutate` is stable (react-query memoises it) while the mutation object around it is not, so
// the effect can name what it calls. The ref keeps it to once regardless.
}, [me.can_read, myStatus.data, startImport]);
// Closing without finishing the essential read step suppresses future auto-popups
// (it stays reachable from Settings). Once read is granted, closing is just "done".
+10 -7
View File
@@ -1,4 +1,4 @@
import { useEffect, useState } from "react";
import { useEffect, useRef, useState } from "react";
import { useTranslation } from "react-i18next";
import { BookmarkPlus, Check, GripVertical, Play, Save, Star, Trash2, X } from "lucide-react";
import {
@@ -107,13 +107,16 @@ export default function PersonalViewsModal() {
},
});
const resetDraft = filtersDraft.reset;
// Re-seed only when the SELECTION changes, not on every same-view refetch — a refetch returns the
// value we just wrote, and re-seeding on it would throw away the prune's undo history. The chosen
// view is therefore read through a ref: `selectedId` is the trigger, `selected` is only the data.
const selectedRef = useRef(selected);
selectedRef.current = selected;
useEffect(() => {
resetDraft(selected?.filters ?? {});
setNameDraft(selected?.name ?? "");
// Re-seed only when the selection changes (not on every same-view refetch), so pruning keeps its
// undo history. Save-current re-seeds explicitly in its own handler.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [selectedId]);
const view = selectedRef.current;
resetDraft(view?.filters ?? {});
setNameDraft(view?.name ?? "");
}, [selectedId, resetDraft]);
// The selected view's captured filters, as the same human labels its module's chip row shows, read
// from the undoable draft. The feed hook runs unconditionally (fed empty filters off-feed); Plex
@@ -351,3 +351,53 @@ describe("PlayerModal — the dialog contract", () => {
}
});
});
// --- the watch toggle belongs to the item it describes -------------------------------------------
// A late writer exists by construction: useYtProgress's cleanup for the OUTGOING item runs after the
// commit that already advanced `active`, and within the finish margin it auto-marks that item
// watched. While the mirror was a bare boolean that write landed last and painted the INCOMING item
// as watched. Keying the mirror by id makes the stray write not this item's, so it is ignored.
describe("PlayerModal — the watch toggle after a queue advance", () => {
const A = vid("A", 0);
const B = vid("B", 0);
beforeEach(() => {
vi.spyOn(api, "videoDetail").mockReturnValue(new Promise(() => {}));
vi.spyOn(api, "saveProgress").mockImplementation(async (id: string) => ({
video_id: id,
position_seconds: 0,
}));
});
it("does not inherit the outgoing item's auto-watch when stepping away near the end", async () => {
const onState = vi.fn();
clock.current = 2995; // inside min(FINISH_MARGIN, duration/2) = 10s of 3000
renderModal(
<PlayerModal video={A} queue={[A, B]} startIndex={0} onClose={() => {}} onState={onState} />,
);
await waitFor(() => expect(lastStart()?.videoId).toBe("A"));
next();
await waitFor(() => expect(lastStart()?.videoId).toBe("B"));
await settle();
// A really is marked watched — that part is correct and must keep working.
expect(onState.mock.calls).toContainEqual(["A", "watched"]);
// B is not, and its toggle says so. Before the fix both toggles read "watchedUnmark" here.
expect(screen.queryAllByTitle("player.watchedUnmark")).toHaveLength(0);
expect(screen.queryAllByTitle("player.markWatched").length).toBeGreaterThan(0);
});
it("still reflects a watch toggled on the item you are actually on", async () => {
const onState = vi.fn();
clock.current = 0;
renderModal(
<PlayerModal video={A} queue={[A, B]} startIndex={0} onClose={() => {}} onState={onState} />,
);
await waitFor(() => expect(lastStart()?.videoId).toBe("A"));
fireEvent.click(screen.getAllByTitle("player.markWatched")[0]!);
expect(onState.mock.calls).toContainEqual(["A", "watched"]);
expect(screen.queryAllByTitle("player.watchedUnmark").length).toBeGreaterThan(0);
});
});
+103 -69
View File
@@ -1,4 +1,4 @@
import { useEffect, useId, useRef, useState } from "react";
import { useCallback, useEffect, useEffectEvent, useId, useRef, useState } from "react";
import { useTranslation } from "react-i18next";
import { createPortal } from "react-dom";
import { useNavigationActions } from "./navigationContext";
@@ -170,7 +170,10 @@ export default function PlayerModal({
// The whole modal (backdrop + card) — the wheel-to-volume target, so scrolling anywhere over the
// modal adjusts volume (not just the small player area).
const dialogRef = useRef<HTMLDivElement | null>(null);
const focusModal = () => cardRef.current?.focus();
// Stable: it reads a ref and nothing else, and three hooks take it as a parameter (useYtVolume,
// useYtNativeControls, useYtBigView) where a fresh identity each render would spread into their
// dep lists too.
const focusModal = useCallback(() => cardRef.current?.focus(), []);
// The dialog contract every other `aria-modal` shell shares (ui/dialogFocus): the focus goes back
// to whatever opened the player, and the page lock is the one counted lock. It opens focused on the
// CARD, not its first button, so Space/F/arrows reach the shortcuts. Until E11 this was the one
@@ -231,22 +234,51 @@ export default function PlayerModal({
setLiveData(null);
};
// Local mirror of watch status so the toggle reacts instantly; changes are
// propagated to the feed (and server) via onState.
const [status, setStatus] = useState(active.status);
const watched = status === "watched";
// Local mirror of watch status so the toggle reacts instantly; changes are propagated to the feed
// (and server) via onState. The mirror carries the id it DESCRIBES, and a write for any other id
// is ignored rather than displayed — that is the enforcement, and it has to be in code because
// one writer arrives late by construction: useYtProgress's cleanup for the OUTGOING item runs
// after the commit that already advanced `active`, and near the end of a clip it auto-marks that
// item watched (playerModalHooks.ts `maybeAutoWatch`). Keyed, that stray write is simply not this
// item's, so the toggle falls back to `active.status` and no ordering has to be argued about.
// Unkeyed it landed last and painted the NEXT video as watched; a corrective click then sent
// onState(next, "new") for an item that was already new.
const [status, setStatus] = useState<{ id: string; value: VideoStatus }>({
id: active.id,
value: active.status,
});
const watched = (status.id === active.id ? status.value : active.status) === "watched";
const setWatched = (on: boolean) => {
const next = on ? "watched" : "new";
setStatus(next);
setStatus({ id: active.id, value: next });
onState(active.id, next);
};
// On queue advance (active changes), sync local state to the new item.
useEffect(() => {
setStatus(active.status);
// On queue advance (active changes), point `currentVideoId` at the new item — DURING that render,
// not one commit later. As an effect, the frame between still had it on the previous id, so the
// "you have navigated away" affordance flashed on every advance. React discards this pass instead
// of committing it, so there is no such frame; the guard falsifies itself immediately.
//
// The watch status is NOT synced here: it is keyed by id above, which is both stale-proof and
// immune to the late writer this block cannot see (see that comment).
const [syncedTo, setSyncedTo] = useState(active.id);
if (syncedTo !== active.id) {
setSyncedTo(active.id);
setCurrentVideoId(active.id);
}
// `currentIdRef` must NOT move into the block above, and the reason is EFFECT ORDERING, which the
// committed test "resumes an item played earlier in the SAME session" measures. useYtProgress's
// per-item effect cleanup persists the outgoing item's position, and its persist() is guarded on
// `currentIdRef.current === id` so a video navigated to inside the player is never checkpointed
// against the library item. React runs every cleanup BEFORE any setup, so in this effect the ref
// still names the item being left while that cleanup runs, and flips to the new one just before
// useYtProgress (declared below) sets up for it. Advanced to render time the flip happens first,
// the outgoing checkpoint is skipped, and stepping next→prev back to an item resumes from the
// frozen launch queue's stale position — which is the exact bug the session-position map exists
// to prevent (748 instead of 830, in the test's numbers).
useEffect(() => {
currentIdRef.current = active.id;
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [active.id]);
const seekTo = (seconds: number) => {
@@ -412,7 +444,11 @@ export default function PlayerModal({
// scope) so the help sheet lists them; only the behaviour behind each id is here.
// These fire only while focus is on our page, not inside the cross-origin player iframe — so
// we focus the modal card on open and again on player-ready (autoplay can grab focus).
useEffect(() => {
//
// The handler itself is a useEffectEvent (React 19.2+): it is bound ONCE per player, but it reads
// whatever is current at the moment a key arrives, which is what the disabled dep list used to
// claim in prose and now is by construction.
const onPlayerKey = useEffectEvent((e: KeyboardEvent) => {
// Volume by keyboard (YouTube/Plex-style ±5). The only volume control that survives fullscreen,
// where the wheel goes to YouTube's own iframe. But the modal card scrolls (a long description)
// and Up/Down is how you scroll it from the keyboard — so defer whenever the focused element can
@@ -432,59 +468,61 @@ export default function PlayerModal({
e.preventDefault();
nudgeSeek(secs);
};
const onKey = (e: KeyboardEvent) => {
// The shared dialog Tab trap — only while nothing that owns the keyboard is layered above the
// player: a Modal opened over it (the download dialog) traps for itself. The title's hover card
// does not, so it must not switch the trap off.
if (e.key === "Tab") {
if (layerIdRef.current != null && anyBlockingLayerAbove(layerIdRef.current)) return;
if (dialogRef.current) trapTab(e, dialogRef.current, cardRef.current);
return;
}
dispatch(e, "ytPlayer", {
close: (ev) => {
// Defer to any layer opened above the player FIRST (a Modal like the download dialog, or an
// anchored popover): it takes the Escape and we do nothing — so we don't also step out of
// maximise or close the player underneath it. The consumed-event flag is the reliable signal
// (UX-8): the layer's `document` listener runs before ours and its close can pop it off the
// stack synchronously, so `anyLayerAbove` alone would miss it and close the player too.
if (escapeHandled(ev)) return;
if (layerIdRef.current != null && anyLayerAbove(layerIdRef.current)) return;
// In fullscreen, let the browser's Esc exit fullscreen only — don't also close the modal.
if (fullscreenElement()) return;
// Same for our own maximise: Esc steps back out of it before it closes the player.
if (maximizedRef.current) {
ev.preventDefault();
setMaximized(false);
return;
}
onClose();
},
fullscreen: (ev) => {
// The shared dialog Tab trap — only while nothing that owns the keyboard is layered above the
// player: a Modal opened over it (the download dialog) traps for itself. The title's hover card
// does not, so it must not switch the trap off.
if (e.key === "Tab") {
if (layerIdRef.current != null && anyBlockingLayerAbove(layerIdRef.current)) return;
if (dialogRef.current) trapTab(e, dialogRef.current, cardRef.current);
return;
}
dispatch(e, "ytPlayer", {
close: (ev) => {
// Defer to any layer opened above the player FIRST (a Modal like the download dialog, or an
// anchored popover): it takes the Escape and we do nothing — so we don't also step out of
// maximise or close the player underneath it. The consumed-event flag is the reliable signal
// (UX-8): the layer's `document` listener runs before ours and its close can pop it off the
// stack synchronously, so `anyLayerAbove` alone would miss it and close the player too.
if (escapeHandled(ev)) return;
if (layerIdRef.current != null && anyLayerAbove(layerIdRef.current)) return;
// In fullscreen, let the browser's Esc exit fullscreen only — don't also close the modal.
if (fullscreenElement()) return;
// Same for our own maximise: Esc steps back out of it before it closes the player.
if (maximizedRef.current) {
ev.preventDefault();
toggleFullscreen();
},
playPause: (ev) => {
// Let Space activate a focused button/link; otherwise toggle playback (and stop the
// page from scrolling).
if (isActivationTarget(ev)) return;
ev.preventDefault();
togglePlay();
},
volumeUp: (ev) => volume(ev, true),
volumeDown: (ev) => volume(ev, false),
seekBack: (ev) => seek(ev, -5),
seekFwd: (ev) => seek(ev, 5),
prevVideo: (ev) => {
ev.preventDefault();
goPrev();
},
nextVideo: (ev) => {
ev.preventDefault();
goNext();
},
});
};
setMaximized(false);
return;
}
onClose();
},
fullscreen: (ev) => {
ev.preventDefault();
toggleFullscreen();
},
playPause: (ev) => {
// Let Space activate a focused button/link; otherwise toggle playback (and stop the
// page from scrolling).
if (isActivationTarget(ev)) return;
ev.preventDefault();
togglePlay();
},
volumeUp: (ev) => volume(ev, true),
volumeDown: (ev) => volume(ev, false),
seekBack: (ev) => seek(ev, -5),
seekFwd: (ev) => seek(ev, 5),
prevVideo: (ev) => {
ev.preventDefault();
goPrev();
},
nextVideo: (ev) => {
ev.preventDefault();
goNext();
},
});
});
useEffect(() => {
const onKey = (e: KeyboardEvent) => onPlayerKey(e);
window.addEventListener("keydown", onKey);
focusModal();
// (The scroll lock is the shared, counted one in useDialogChrome.)
@@ -492,11 +530,7 @@ export default function PlayerModal({
window.removeEventListener("keydown", onKey);
// (The debounced volume write is flushed by useYtVolume's own unmount cleanup.)
};
// Bound once per player on purpose: every helper the handler calls reads its live state through
// a ref (playerRef, navRef, maximizedRef, isFullscreenRef, the volume hook's own ref), so
// re-binding on each render would only churn the listener, never change what a key does.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [onClose]);
}, [focusModal]);
// Create the player, resume from the saved position, checkpoint progress, and auto-mark watched at
// the end — recreated per active item. The once-bound event closures read fresh queue/mode/volume via
+18 -7
View File
@@ -290,6 +290,9 @@ export default function Playlists() {
});
const detail = detailQuery.data;
// Destructured so the effect below can name what it calls: both are stable useCallbacks, while
// the object around them is rebuilt every render (lib/useUndoable).
const { reset: resetOrder, rebase: rebaseOrder } = order;
useEffect(() => {
if (!detail) return;
// Scoped to the playlist: two lists holding the SAME videos would otherwise look identical to
@@ -297,7 +300,7 @@ export default function Playlists() {
const key = `${selectedId}:${idsKey(detail.items)}`;
if (key !== lastSetRef.current) {
lastSetRef.current = key;
order.reset(detail.items);
resetOrder(detail.items);
} else {
// Same membership, fresher ROWS — a watch position the player just stored, a status. The
// guard above looks at the id set only, so without this the list kept showing, and handing the
@@ -305,17 +308,23 @@ export default function Playlists() {
// reopen it, and it resumed at the old spot and the next checkpoint wrote that over the 830.
// `rebase` takes the fresh rows into every snapshot and keeps the order and its undo history.
const fresh = new Map(detail.items.map((v) => [v.id, v]));
order.rebase((list) => list.map((v) => fresh.get(v.id) ?? v));
rebaseOrder((list) => list.map((v) => fresh.get(v.id) ?? v));
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [detail]);
// `reset` and `rebase` are stable useCallbacks (lib/useUndoable), so naming them costs nothing;
// `selectedId` moves only together with `detail` anyway, since detail IS the selected playlist's.
}, [detail, selectedId, resetOrder, rebaseOrder]);
// Reset the view when switching playlists (the stored order itself is per-playlist).
useEffect(() => {
// Reset the view when switching playlists (the stored order itself is per-playlist) — during the
// render that switches, so the incoming playlist is never drawn once through the previous one's
// sort, filters and grouping. All three are this component's own state, which is what makes a
// render-phase write legal here.
const [viewFor, setViewFor] = useState(selectedId);
if (viewFor !== selectedId) {
setViewFor(selectedId);
setViewSort(STORED_SORT);
setFilters({});
setGroupBy(false);
}, [selectedId]);
}
const builtin = detail?.kind === "watch_later";
// YouTube-mirrored playlists can be edited locally and synced back (the read-sync skips a
@@ -385,6 +394,7 @@ export default function Playlists() {
// Read by the drag announcements, which must report the CURRENT positions without making the
// announcement closures a render dependency of DndContext.
const shownRef = useRef(shown);
// eslint-disable-next-line react-hooks/immutability -- the house latest-value ref: the announcement closures must read the CURRENT list without the list becoming a render dependency of DndContext. A render-phase ref write is never part of the rendered output, which is what makes it safe; the checker is conservative about refs an effect also reads.
shownRef.current = shown;
const viewActive = !isStoredSort(viewSort) || groupBy || filterActive;
@@ -595,6 +605,7 @@ export default function Playlists() {
}
// Kept in a ref so the memoized column callbacks don't rebuild on every state change.
const removeRef = useRef(removeItem);
// eslint-disable-next-line react-hooks/immutability -- same latest-value ref as shownRef above — it keeps the memoised column callbacks from rebuilding on every state change.
removeRef.current = removeItem;
async function moveItemTo(video: Video, target: Playlist) {
+11 -9
View File
@@ -1,4 +1,4 @@
import { useEffect, useMemo, useRef, useState } from "react";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { useTranslation } from "react-i18next";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { qk } from "../lib/queryKeys";
@@ -91,7 +91,9 @@ export default function PlaylistsRail({
queryFn: () => api.playlists(),
refetchOnWindowFocus: true,
});
const playlists = listQuery.data ?? [];
// Memoised so the effect and the sort below depend on a value that only changes with the query —
// `?? []` would otherwise mint a new empty array on every render while the list is loading.
const playlists = useMemo(() => listQuery.data ?? [], [listQuery.data]);
// Keep the selection valid: default to the first playlist, and if the selected one disappears
// drop to the next available (or clear when none remain). Waits for the first load.
@@ -104,10 +106,12 @@ export default function PlaylistsRail({
if (selectedId == null || !playlists.some((p) => p.id === selectedId)) {
setSelectedId(playlists[0]!.id); // playlists is non-empty (the length guard above returns)
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [playlists, selectedId, listQuery.data]);
}, [playlists, selectedId, listQuery.data, setSelectedId]);
const plName = (p: { kind: string; name: string }) => playlistName(p, t);
// useCallback because the two memos below sort by it: as a fresh closure each render it could not
// be named in their dep lists, and `t` — which is what actually changes it, on a language switch —
// was therefore invisible to them. A rename in the other language now re-sorts.
const plName = useCallback((p: { kind: string; name: string }) => playlistName(p, t), [t]);
const sortedPlaylists = useMemo(() => {
const sign = plSort.dir === "asc" ? 1 : -1;
@@ -126,15 +130,13 @@ export default function PlaylistsRail({
return 0;
}
});
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [playlists, plSort]);
}, [playlists, plSort, plName]);
const q = search.trim().toLowerCase();
const visiblePlaylists = useMemo(
() =>
q ? sortedPlaylists.filter((p) => plName(p).toLowerCase().includes(q)) : sortedPlaylists,
// eslint-disable-next-line react-hooks/exhaustive-deps
[sortedPlaylists, q],
[sortedPlaylists, q, plName],
);
useEffect(() => {
+25 -11
View File
@@ -112,14 +112,14 @@ export default function PlexBrowse() {
const sub = useHistorySubview<Sub>({ kind: "discover" }, PLEX_SUB_URL);
// Publish the top-level view (discover vs grid) + opener to the header's Discover/Browse toggle via
// the bridge; clear it on unmount so the toggle disappears when you leave the plex page. Keyed on the
// view KIND (open is behaviourally stable — it takes the next view as an arg, closing over nothing).
// the bridge; clear it on unmount so the toggle disappears when you leave the plex page. `sub.open`
// is referentially stable (lib/history memoises it), so naming it costs nothing and the dep list is
// the honest one: the toggle is republished when the view KIND changes, and on nothing else.
const publishView = usePublishPlexView();
useLayoutEffect(() => {
publishView({ kind: sub.view.kind, open: sub.open });
return () => publishView(null);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [sub.view.kind]);
}, [sub.view.kind, sub.open, publishView]);
// Sidebar asked to open a playlist → push it as a subview (browser Back returns to the grid), then
// clear the signal so it doesn't re-open.
@@ -128,8 +128,7 @@ export default function PlexBrowse() {
sub.open({ kind: "playlist", id: openPlaylist });
setPlaylistOpen(null);
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [openPlaylist]);
}, [openPlaylist, sub, setPlaylistOpen]);
// The header search typeahead picked a title → open it the same way a card click would (a movie's
// info page like the card's ⓘ, a show's drill-down), then clear the hand-off so it doesn't re-open.
@@ -150,8 +149,7 @@ export default function PlexBrowse() {
);
setJumpTo(null);
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [jumpTo]);
}, [jumpTo, sub, setJumpTo]);
// Discover → grid on a SEARCH: a header search has no rail on Discover, so entering one FIRST while on
// the Discover landing switches to the flat grid (which consumes q). Only on the off→on edge, so a Back
@@ -180,6 +178,17 @@ export default function PlexBrowse() {
// (down to a collection strip) instead of snapping to the top. Reset on a fresh info open so a
// different title's info starts at the top.
const infoScrollRef = useRef(0);
// ONE writer for that memory, so "stash on the way out of an info page, reset on a fresh one" is a
// rule the code holds rather than one spread over four call sites — and so the compiler's
// complaint about it is answered once, here, instead of at each of them.
//
// The complaint is a false positive of a conservative analysis, not a hazard: the rule objects to a
// ref being written outside the effect that reads it, but that IS the arrangement a scroll memory
// needs — the navigation handlers record where the page was, and the restore layout effect above
// reads it on the way back. React's own guarantee (a ref write is never part of rendering) is what
// makes it safe, and nothing here reads the value during render.
// eslint-disable-next-line react-hooks/immutability
const rememberInfoScroll = (px: number) => void (infoScrollRef.current = px);
// One comfort-pin menu for the titles grid (see ui/ContextMenu).
const pinMenu = useContextMenu<PlexCard>();
const scroller = () => document.querySelector("main");
@@ -261,6 +270,11 @@ export default function PlexBrowse() {
// then a second after the clear; the query waits for both to settle into a single fetch.
useEffect(() => {
if (committedLang !== lang) {
// Not adjustable during render, which is where a state reset like this would otherwise
// belong: `setFilters` belongs to PlexProvider, and React only permits a render-phase setState
// on the component's OWN state. The cascade the rule warns about is bounded to one extra pass
// — the guard above is false immediately afterwards.
// eslint-disable-next-line react-hooks/set-state-in-effect
setCommittedLang(lang);
if (filters.genres.length) setFilters({ ...filters, genres: [] });
}
@@ -308,7 +322,7 @@ export default function PlexBrowse() {
}
function onInfo(card: PlexCard) {
scrollRef.current = scroller()?.scrollTop ?? 0;
infoScrollRef.current = 0; // fresh info page starts at the top
rememberInfoScroll(0); // fresh info page starts at the top
sub.open({ kind: "info", id: card.id });
}
// Open a cast/crew member's filmography page (from an info/show cast row, or an actor filter-chip).
@@ -316,7 +330,7 @@ export default function PlexBrowse() {
// it was, like applyFilter) — from a show/grid context infoScrollRef must stay untouched, else a
// later info page would wrongly restore this stale offset.
function onOpenPerson(name: string) {
if (sub.view.kind === "info") infoScrollRef.current = scroller()?.scrollTop ?? 0;
if (sub.view.kind === "info") rememberInfoScroll(scroller()?.scrollTop ?? 0);
sub.open({ kind: "person", name });
}
// Optimistically flip a card's status in every cached library page — both the title grid (`items`)
@@ -361,7 +375,7 @@ export default function PlexBrowse() {
}
setFilters(merged as unknown as PlexFilters);
if (patch.actors?.length || patch.directors?.length) setScope("both");
infoScrollRef.current = scroller()?.scrollTop ?? 0; // restore on Back to the info/show page
rememberInfoScroll(scroller()?.scrollTop ?? 0); // restore on Back to the info/show page
sub.open({ kind: "grid" });
}
+43 -18
View File
@@ -35,9 +35,46 @@ const money = (n: number) =>
notation: "compact",
style: "currency",
currency: "USD",
// The minimum is load-bearing, not a default written out: given only a maximum, a compact
// format is free to pad to it. The very same options render "$160.0M" on Node 22 and "$160M"
// on Node 24 (measured 2026-09-20 — the server gate was red on exactly this while the local one
// was green), and browsers split along the same line. Pin it rather than inherit the engine's.
minimumFractionDigits: 0,
maximumFractionDigits: 1,
}).format(n);
// One rating chip's number: a filter button when this is the score the backend's ratingMin column
// tracks, plain text otherwise. It lives at MODULE level, not inside RatingsRow, because a component
// declared during render is a NEW component type on every render — React unmounts and remounts the
// subtree rather than updating it, which throws away DOM state and, for this one, the keyboard focus
// on the button while the info page re-renders around it.
function Score({
score,
filterable,
onFilterRating,
}: {
score: number;
filterable: boolean;
onFilterRating?: (score: number) => void;
}) {
const { t } = useTranslation();
return filterable && onFilterRating ? (
<button
onClick={() => onFilterRating(Math.floor(score))}
className="inline-flex items-center gap-1 hover:opacity-80"
title={t("plex.info.filterRating", { n: Math.floor(score) })}
>
<Star className="h-3.5 w-3.5 fill-current" />
{score.toFixed(1)}
</button>
) : (
<span className="inline-flex items-center gap-1">
<Star className="h-3.5 w-3.5 fill-current" />
{score.toFixed(1)}
</span>
);
}
// Multi-source ratings cluster (E5 S5): IMDb (links out) · TMDb (with vote count) · Plex audience —
// whichever are present. The "primary" score (IMDb, else audience) mirrors the backend rating column
// (coalesce(imdb, audience)), so only it carries the ratingMin filter; the others are informational.
@@ -60,29 +97,17 @@ function RatingsRow({
const primary = imdbRating ?? audience ?? null; // the filterable score
const chipCls =
"inline-flex items-center gap-1 rounded-md bg-amber-400/15 px-1.5 py-0.5 font-semibold text-amber-500";
const Score = ({ score, filterable }: { score: number; filterable: boolean }) =>
filterable && onFilterRating ? (
<button
onClick={() => onFilterRating(Math.floor(score))}
className="inline-flex items-center gap-1 hover:opacity-80"
title={t("plex.info.filterRating", { n: Math.floor(score) })}
>
<Star className="h-3.5 w-3.5 fill-current" />
{score.toFixed(1)}
</button>
) : (
<span className="inline-flex items-center gap-1">
<Star className="h-3.5 w-3.5 fill-current" />
{score.toFixed(1)}
</span>
);
const has = (x?: number | null): x is number => x != null;
if (!has(imdbRating) && !has(audience) && !has(tmdbRating)) return null;
return (
<div className="flex flex-wrap items-center gap-2 text-sm">
{has(imdbRating) && (
<span className={chipCls}>
<Score score={imdbRating} filterable={imdbRating === primary} />
<Score
score={imdbRating}
filterable={imdbRating === primary}
onFilterRating={onFilterRating}
/>
<span className="text-2xs font-normal opacity-70">IMDb</span>
{imdbUrl && (
<a
@@ -106,7 +131,7 @@ function RatingsRow({
)}
{has(audience) && imdbRating !== audience && (
<span className={chipCls}>
<Score score={audience} filterable={!has(imdbRating)} />
<Score score={audience} filterable={!has(imdbRating)} onFilterRating={onFilterRating} />
<span className="text-2xs font-normal opacity-70">{t("plex.info.audience")}</span>
</span>
)}
@@ -0,0 +1,185 @@
// @vitest-environment jsdom
//
// The Plex player paints an opaque, full-viewport layer over the whole app — and until deep-review
// round 4 it was the only overlay in the tree with no Tab trap at all. Tab from its transport bar,
// from its menus, or from the debug recorder's pill (which docks in THIS player's toast host) walked
// straight out into the page behind it, where the focus ring is invisible and the next Enter fires
// something the user cannot see. `ui/Modal`, `PlayerModal`, `OnboardingWizard` and `Welcome` all trap;
// the round-3 comment that handed the step prompt's trap to "the dialog below" assumed this one did
// too. Its own file because `layerStack` is module state and these tests push into it.
import { describe, it, expect, vi, beforeAll, afterEach } from "vitest";
import { render, cleanup, waitFor, act } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
vi.mock("react-i18next", async (importOriginal) => ({
...(await importOriginal<typeof import("react-i18next")>()),
useTranslation: () => ({ t: (key: string) => key, i18n: { language: "en" } }),
}));
import PlexPlayer from "./PlexPlayer";
import { api, type PlexItemDetail } from "../lib/api";
import { pushLayer, removeLayer } from "../lib/layerStack";
beforeAll(() => {
class Stub {
observe() {}
unobserve() {}
disconnect() {}
takeRecords() {
return [];
}
}
globalThis.ResizeObserver ??= Stub as unknown as typeof ResizeObserver;
globalThis.IntersectionObserver ??= Stub as unknown as typeof IntersectionObserver;
for (const [name, value] of Object.entries({
play: () => Promise.resolve(),
pause: () => {},
load: () => {},
}))
Object.defineProperty(HTMLMediaElement.prototype, name, { configurable: true, value });
Object.defineProperty(HTMLMediaElement.prototype, "seekable", {
configurable: true,
get: () => ({ length: 0, end: () => 0 }),
});
Object.defineProperty(HTMLMediaElement.prototype, "textTracks", {
configurable: true,
get: () => Object.assign([], { addEventListener() {}, removeEventListener() {} }),
});
});
const behind: HTMLButtonElement[] = [];
afterEach(() => {
cleanup();
while (behind.length) behind.pop()!.remove();
vi.restoreAllMocks();
});
const detail = {
id: "1",
title: "Title 1",
type: "movie",
duration_seconds: 3000,
status: "new",
position_seconds: 0,
audio_streams: [],
subtitle_streams: [],
markers: [],
} as unknown as PlexItemDetail;
async function mount() {
vi.spyOn(api, "plexItem").mockResolvedValue(detail);
vi.spyOn(api, "plexSession").mockResolvedValue({
mode: "direct",
url: "/media/1",
start_s: 0,
} as Awaited<ReturnType<typeof api.plexSession>>);
vi.spyOn(api, "plexProgress").mockResolvedValue(undefined as never);
render(
<QueryClientProvider
client={new QueryClient({ defaultOptions: { queries: { retry: false } } })}
>
<PlexPlayer itemId="1" onClose={() => {}} />
</QueryClientProvider>,
);
await waitFor(() =>
expect(document.querySelector("video")?.getAttribute("src")).toBe("/media/1"),
);
}
/** A control on the page the player is painted over — the thing focus used to escape onto. */
function pageBehind() {
const b = document.createElement("button");
b.textContent = "on the page behind";
document.body.insertBefore(b, document.body.firstChild);
behind.push(b);
return b;
}
const player = () => document.querySelector<HTMLElement>(".fixed.z-overlay")!;
/** jsdom has no layout, so the trap's visibility filter (`offsetParent`) would see nothing tabbable
* anywhere. Give every element one for the duration of a check — the house pattern (PlayerModal,
* OnboardingWizard, ui/dialogFocus all do this). */
function withLayout<T>(fn: () => T): T {
const spy = vi.spyOn(HTMLElement.prototype, "offsetParent", "get").mockReturnValue(document.body);
try {
return fn();
} finally {
spy.mockRestore();
}
}
const press = (key: string, shiftKey = false) =>
act(() => {
window.dispatchEvent(
new KeyboardEvent("keydown", { key, shiftKey, bubbles: true, cancelable: true }),
);
});
describe("PlexPlayer · Tab stays inside the player", () => {
it("pulls the focus back in when it is on the page behind", async () => {
await mount();
const outside = pageBehind();
outside.focus();
expect(document.activeElement).toBe(outside);
withLayout(() => press("Tab"));
// Not left on a control under an opaque overlay: the trap takes it into the player.
expect(document.activeElement).not.toBe(outside);
expect(player().contains(document.activeElement)).toBe(true);
});
it("wraps from the last of its own controls back to the first", async () => {
await mount();
const items = [
...player().querySelectorAll<HTMLElement>(
'button:not([disabled]), a[href], [tabindex]:not([tabindex="-1"])',
),
];
expect(items.length).toBeGreaterThan(1);
const first = items[0]!;
const last = items[items.length - 1]!;
last.focus();
withLayout(() => press("Tab"));
expect(document.activeElement).toBe(first);
// ...and backwards, which is the direction that used to walk straight out behind the player.
first.focus();
withLayout(() => press("Tab", true));
expect(document.activeElement).toBe(last);
});
it("keeps trapping while one of its OWN menus is open", async () => {
await mount();
// The settings and track menus are `useDismiss` layers. That hook owns Escape and the
// outside-press and installs no Tab trap at all — but it used to register as a layer that
// DOES trap, so the player stood its own trap down for it and Tab walked out past the last
// control into the nav rail. Measured in a real browser: 30 Tabs, 9 escapes. The menus render
// inside the player, so the player's trap is the one that must stay on.
const gear = [...player().querySelectorAll<HTMLElement>("button")].find(
(b) => b.getAttribute("aria-label") === "plex.player.settings",
);
expect(gear).toBeDefined();
act(() => gear!.click());
const items = [...player().querySelectorAll<HTMLElement>("button:not([disabled])")];
// The menu really opened — the trap now has to span its controls too.
expect(items.length).toBeGreaterThan(12);
const last = items[items.length - 1]!;
last.focus();
withLayout(() => press("Tab"));
expect(player().contains(document.activeElement)).toBe(true);
expect(document.activeElement).toBe(items[0]);
});
it("leaves the Tab to a dialog opened above it, which traps for itself", async () => {
await mount();
const outside = pageBehind();
outside.focus();
const above = pushLayer(); // a Modal over the player — the download dialog, say
try {
withLayout(() => press("Tab"));
// The player does not reach past it: the dialog above owns the keyboard and its own trap.
expect(document.activeElement).toBe(outside);
} finally {
removeLayer(above);
}
});
});
+56 -11
View File
@@ -74,7 +74,14 @@ import { LS, useAccountPersistedObject } from "../lib/storage";
import { useDismiss } from "../lib/useDismiss";
import { useScrollFade } from "../lib/useScrollFade";
import { useBackToClose } from "../lib/history";
import { pushLayer, removeLayer, anyLayerAbove, escapeHandled } from "../lib/layerStack";
import { trapTab } from "./ui/dialogFocus";
import {
pushLayer,
removeLayer,
anyLayerAbove,
anyBlockingLayerAbove,
escapeHandled,
} from "../lib/layerStack";
// The rich info overlay (poster/cast/ratings) reuses the same component as the card's info page.
const PlexInfo = lazy(() => import("./PlexInfo"));
@@ -201,6 +208,13 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) {
// Refs for values read inside stable callbacks / the keyboard handler (avoid stale closures).
const barAutoHideRef = useRef(prefs.barAutoHide);
barAutoHideRef.current = prefs.barAutoHide;
// The two language prefs are read only at ITEM LOAD (the detail effect below), so they go through
// a ref for the same reason `tRef` above does: naming them in that effect's deps would re-run it
// when the user picks a track from the player's own menu, double-loading the session the picker
// has already reloaded. The ref says "latest value, but not a trigger" in code, which an excused
// dep list only said in prose. Same shape as barAutoHideRef/seekStepRef either side of this.
const langPrefsRef = useRef({ audioLang: prefs.audioLang, subLang: prefs.subLang });
langPrefsRef.current = { audioLang: prefs.audioLang, subLang: prefs.subLang };
const seekStepRef = useRef(prefs.seekStep);
seekStepRef.current = prefs.seekStep;
const fineSeekStepRef = useRef(prefs.fineSeekStep);
@@ -467,7 +481,11 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) {
restartsRef.current.at = now;
if (restartsRef.current.n < MAX_SESSION_RECLAIMS) {
restartsRef.current.n += 1;
loadSession(playhead(absRef, loadStartRef), !video.paused);
// Through the ref, not by name: this is loadSession calling ITSELF from inside its own
// useCallback, which the checker rightly reads as "accessed before it is declared".
// The ref (assigned right below the callback) is the file's existing answer to that —
// detachSessionRef and cancelAutoSkipRef do the same for the same reason.
void loadSessionRef.current(playhead(absRef, loadStartRef), !video.paused);
return;
}
}
@@ -490,6 +508,11 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) {
},
[id], // NOT `t` — see tRef above (a language switch must not rebuild this / reload the session).
);
// The self-restart path above reaches the CURRENT loadSession through this, so the recursion is
// not a forward reference to a binding that is still being defined.
const loadSessionRef = useRef(loadSession);
// eslint-disable-next-line react-hooks/immutability -- the latest-value ref the self-restart path above reads; see the comment there for why the recursion cannot name loadSession directly.
loadSessionRef.current = loadSession;
// Audio/subtitle track selection + subtitle rendering. Owns the selected-ordinal state; the shared
// media refs stay component-owned (loadSession above + the detail effect below write audioRef too).
@@ -545,16 +568,14 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) {
restartsRef.current = { n: 0, at: 0 };
// Restore the per-account audio/subtitle language choice, resolved to THIS item's tracks. Set the
// audio ref synchronously so the loadSession below picks the right track on the first request.
const aOrd = audioOrdForLang(detail.audio_streams ?? [], prefs.audioLang);
const { audioLang, subLang } = langPrefsRef.current;
const aOrd = audioOrdForLang(detail.audio_streams ?? [], audioLang);
audioRef.current = aOrd;
setAudioOrd(aOrd);
setSubOrd(subOrdForLang(detail.subtitle_streams ?? [], prefs.subLang));
setSubOrd(subOrdForLang(detail.subtitle_streams ?? [], subLang));
const resume = detail.status !== "watched" ? detail.position_seconds || 0 : 0;
loadSession(resume);
// Prefs are applied at item load only; changing one via the picker must NOT re-run this (it would
// double-load the session — the picker handlers already reload/toggle).
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [detail, id, loadSession]);
void loadSession(resume);
}, [detail, id, loadSession, audioRef, setAudioOrd, setSubOrd]);
// Leaving the item: a session still in flight for it must not land after it, and its stream goes.
useEffect(
() => () => {
@@ -586,7 +607,11 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) {
const v = videoRef.current;
return () => {
aliveRef.current = false;
// eslint-disable-next-line react-hooks/exhaustive-deps -- the live counter, as above
// The rule's advice — copy the ref into a variable and use the copy — is precisely wrong here.
// sessGenRef is a LIVE counter: bumping it is how the unmount tells every in-flight session
// that its response no longer belongs to anything, and a value captured when the effect ran
// would invalidate a generation that has since been superseded instead of the current one.
// eslint-disable-next-line react-hooks/exhaustive-deps
sessGenRef.current++;
if (hlsRef.current) {
hlsRef.current.destroy();
@@ -832,7 +857,14 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) {
setUpNextOpen(false);
if (nextId) go(nextId);
}, [nextId, go]);
useEffect(() => setUpNextOpen(false), [id]);
// The "up next" card belongs to the item that was playing, so a queue advance closes it. Adjusted
// during the render that changes `id` rather than in an effect: as an effect it closed one commit
// after the new episode had already drawn behind it.
const [upNextForId, setUpNextForId] = useState(id);
if (upNextForId !== id) {
setUpNextForId(id);
setUpNextOpen(false);
}
// "Back" cascade shared by the on-screen Back arrow AND the keyboard/remote Backspace·Escape: peel
// off whatever is layered on top (a tuning/tracks menu, then the info overlay) before leaving the
@@ -896,6 +928,19 @@ export default function PlexPlayer({ itemId, onClose, queue }: Props) {
// remote", and `dispatch` still lets Escape through from inside a field (the only key that
// opts in), which is why this is a guard around `wake` rather than an early return.
if (!isTypingTarget(e)) wake();
// The shared dialog Tab trap. This player paints an opaque full-viewport layer over the app,
// so Tab leaving it lands on a control nobody can see, where the ring is invisible and the
// next Enter fires something blind. Every other overlay in the tree traps (ui/Modal,
// PlayerModal, OnboardingWizard, Welcome); this one never did, and deep-review round 4 caught
// it when the recorder's step prompt — which docks in this player's toast host — handed its
// own trap to "the dialog below" and found none. Only while nothing that owns the keyboard is
// layered above: a Modal opened over the player traps for itself. `wake()` above has already
// revealed the control band, so the trap cannot leave focus on a faded-out button.
if (e.key === "Tab") {
if (layerIdRef.current != null && anyBlockingLayerAbove(layerIdRef.current)) return;
if (wrapRef.current) trapTab(e, wrapRef.current);
return;
}
dispatch(e, "plexPlayer", {
playPause: (ev) => {
// Let Space activate a focused button/link natively (e.g. "Skip intro"); only toggle
@@ -12,7 +12,10 @@ type SettingsTab = "sync" | "playback" | "subtitle" | "clock";
// panel owns no state of its own. `now` is the live wall clock (passed in, not a second ticker) used
// only for the clock-date-style preview buttons.
export function PlexPlayerSettings(props: {
menuRef: RefObject<HTMLDivElement>;
// `| null` is not optional under React 19's types: `useRef<T>(null)` is now `RefObject<T | null>`,
// which is the honest shape — the element is null until the div mounts. The old `RefObject<T>` lied
// about that and 18's types let it.
menuRef: RefObject<HTMLDivElement | null>;
settingsTabs: { key: SettingsTab; label: string }[];
activeTab: SettingsTab;
setSettingsTab: (tab: SettingsTab) => void;
+8 -2
View File
@@ -48,8 +48,14 @@ export default function PlexSearchBox() {
const suggestions = suggestEnabled ? (suggestQ.data?.items ?? []) : [];
const open = suggestEnabled && (suggestQ.isFetching || suggestions.length > 0);
// A fresh query resets the highlight.
useEffect(() => setHi(-1), [dq, activeField, scope]);
// A fresh query resets the highlight — during THAT render, so no frame ever draws the new
// suggestion list with the old row highlighted.
const suggestKey = `${scope}|${activeField}|${dq}`;
const [hiFor, setHiFor] = useState(suggestKey);
if (hiFor !== suggestKey) {
setHiFor(suggestKey);
setHi(-1);
}
// Keep the highlighted option scrolled into view during arrow-key navigation.
useEffect(() => {
if (hi < 0) return;
+20 -11
View File
@@ -129,11 +129,15 @@ export function SavedViewsProvider({ children }: { children: ReactNode }) {
const [modalOpen, setModalOpen] = useState(false);
// Switching modules (feed ⟷ plex) starts fresh: a feed view can't be the sticky/active one on plex,
// and a feed modal shouldn't linger open on plex.
useEffect(() => {
// and a feed modal shouldn't linger open on plex. Adjusted DURING RENDER against the previous
// source rather than from an effect, so the first plex render never commits carrying the feed's
// sticky id — which is exactly long enough for a stale view chip to paint.
const [prevSource, setPrevSource] = useState(source);
if (prevSource !== source) {
setPrevSource(source);
setStickyId(null);
setModalOpen(false);
}, [source]);
}
// The ONE default-application path for both modules. On the views query's FIRST resolve for a
// source, seed the starred default view's filters — but only into a genuinely fresh module (no
@@ -146,17 +150,21 @@ export function SavedViewsProvider({ children }: { children: ReactNode }) {
// per-account stored-filters key, which is always readable here: BootGate pins the tab's account
// before this provider mounts.
const seededRef = useRef<Set<ViewSource>>(new Set());
// `cfg` re-memoizes on every filter edit, so it is read through a ref: the seed must fire on the
// views query's first resolve for a source and never again, which `seededRef` enforces. Listing
// cfg would re-arm nothing (the guard still holds) but would re-run the effect on every keystroke
// in the filter bar. The ref keeps the dep list honest without that.
const cfgRef = useRef(cfg);
cfgRef.current = cfg;
useEffect(() => {
const data = viewsQuery.data;
if (!data || seededRef.current.has(source)) return;
seededRef.current.add(source);
const def = pickDefaultToSeed(data, cfg.canSeedDefault());
const c = cfgRef.current;
const def = pickDefaultToSeed(data, c.canSeedDefault());
if (!def) return;
cfg.setFilters(cfg.asFilters(def.filters));
c.setFilters(c.asFilters(def.filters));
setStickyId(def.id);
// cfg is read fresh on this resolve and the per-source guard makes the body run once, so cfg
// (which re-memoizes on every filter edit) is deliberately kept out of the deps.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [viewsQuery.data, source]);
const currentKey = cfg.keyOf(cfg.filters);
@@ -168,9 +176,10 @@ export function SavedViewsProvider({ children }: { children: ReactNode }) {
),
[views, currentKey, stickyId, cfg],
);
useEffect(() => {
if (activeView) setStickyId(activeView.id);
}, [activeView]);
// The sticky id follows whichever view is active — also adjusted during render. It converges by
// construction: preferSticky returns the sticky match when there is one, so the very write below
// makes its own condition false on the next pass.
if (activeView && activeView.id !== stickyId) setStickyId(activeView.id);
const stickyView = useMemo(
() => (stickyId == null ? null : (views.find((v) => v.id === stickyId) ?? null)),

Some files were not shown because too many files have changed in this diff Show More