7fce7a7ce5cdaf894d6099c9322b496362e55143
375
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 peter
|
f5018bdcbc |
fix(uploads): read the body after authenticating, not before it
`POST /api/downloads/upload` declared `file: UploadFile = File(...)`, and FastAPI parses and spools a declared body before the first dependency runs -- `require_human` included. An unauthenticated caller could therefore make the process write the whole per-file allowance (500 MiB by default) to temp, once per request. SEC-20's ceiling bounded that number but could not stop the writing, which is why E27 parked it rather than closing it. The route now takes no body parameter. `spool_upload` names `require_human` first and only then streams `request.stream()` to the staging temp file, so a stranger is refused before a byte moves -- the shape the screenshot route has used since E16. Filename and display name travel as percent-encoded headers (latin-1 on the wire, so a Hungarian filename needs the encoding) instead of multipart parts. `capped_stream_to` joins `capped_body` in routes/_common.py: the same counting rule, written to a file descriptor rather than accumulated, because holding 500 MiB would trade a disk problem for a memory one. Its refusal is the caller's, so the upload route keeps answering 422 `upload_too_large` with the admin knob's own number. With no envelope to allow for, the edge is now exactly that knob, and MULTIPART_ENVELOPE_ALLOWANCE is gone. The invariant stops being prose: test_body_limit.py asks FastAPI itself (`route.dependant.body_params`) and fails the gate if any route with a raised ceiling declares a body. A comment could not have prevented the defect it described; that test would have. Both new tests were seen red for the right reason -- before the fix the anonymous probe got its 401 only after the server had read 4236 bytes of a multipart body, and re-declaring the File parameter turns the invariant test red naming the route and the parameter. |
||
|
peter
|
90fc86d437 |
fix(diagnostics): deep-review round 4 — a claim only its owner may write, and a player that keeps the Tab
Round 4 was targeted at the surfaces rounds 2 and 3 changed. 29 findings raised, 26 survived deduplication, 12 survived three adversarial readings each. `set_delivery` renewed the claim but never checked it was still ours, while its sibling `release_send` always did. `expire_on_commit` is off, so the token it re-stamped came from the caller's own memory, not the row: a send that outlived its 600 s lease came back, overwrote the take-over send's committed links with a full-column write, and put its own dead token back. It now takes the token, reads the row's claim with a targeted scalar (not a refresh, which would discard the `issue_url` assigned one line earlier) and raises `SendClaimLost` rather than writing. An issue created by a send that had already lost its claim is logged with its number — that record is the only one that will exist. The tracker's settings are read above the pre-call checkpoint instead of inside `_client`. Reading them there was a SELECT one line after the commit that was meant to end the transaction, and held it — idle, snapshot pinned — across the repository check and the create. The comment and the module docstring said otherwise; measuring it in pg_stat_activity is what settled it. Two more comments were saying things the code had stopped doing: the `**state` spread does not carry the claim (`set_delivery` strips and re-stamps it), and react-query does not debounce the preview — the editor being unmounted is what paces it. The composer stopped stranding the focus on `<body>`: a send that creates the issue and then fails on an attachment unmounts the button that was just pressed, and only the success path handed the focus on. Export no longer saves into a refusal it does not need — the zip is the way out of a stuck send, and its pre-save was the one step that could not survive one. The delivery banner announces through a role instead of an `aria-live` created with its own message. `historyState` returns a union, so `chipTone` matching on the same keys is checked by tsc rather than by hope. PlexPlayer was the only full-screen overlay in the tree with no focus containment at all — Tab walked out of an opaque player into a page nobody can see. It now carries the shared trap. Closing that hole exposed the one underneath it: `useDismiss` registered as a layer that traps the Tab while installing no trap, so the player stood its own down for its own menus. Measured in a real browser: 30 Tabs with the settings panel open, 9 escapes into the nav rail, and 0 after. Escape ordering is unchanged — the panel first, then the player. Three tests that did not hold their line: the advisory lock's call site (deleting it left the suite green), the trail assertion whose `all()` was vacuous over an empty filter, and a `layerStack` assertion the stand-in dialog already forced. All eleven executable fixes mutation-checked red for the right reason; the two comment corrections have nothing to pin. |
||
|
peter
|
2fe58073e4 |
fix(diagnostics): deep-review round 3 — a keyboard owner that is not a Tab trap
Round 2 made the step prompt a blocking layer so that `j` and the `g` chords would not reach the page behind it. The dialog it sits in gave up its Tab trap because of that, while the prompt's own replacement trap covered only its own two controls — so Tab from anywhere else in the dialog (the bar's Screenshot or Stop, the dialog's own buttons) walked out behind the scrim. `layerStack` now separates the two questions it had folded into one flag: a layer can own the keyboard and still leave the Tab trap to the dialog below it. The mirror case, a dialog opening over the prompt, falls out of the same split. A send's claim is now a lease with an owner. Every progress report re-stamps it, so a send slower than the lease cannot lose it while it is still running, and only the send that took a claim can give it back. The issue_create audit row is staged inside the send and rides the same checkpoint as `issue_url`, so the trail cannot be missing for an issue that exists. The attachment-name prefix has one definition, shared by the producer and both readers. A running send is reported to the UI. A reload or a second tab used to render it exactly like a send that STOPPED, and offered a retry the server was certain to refuse; the composer now says the report is being sent, holds the draft, and puts both the retry and Create issue out. The past-sessions list says the same for the same reason — it is the list a user opens to check on a send, and both "draft" and "half sent" describe one that died. `/preview` and the retry path got a pace of their own, and the per-user byte cap took the user-scoped lock that makes it a cap instead of a guess. Verified in a real browser on localdev: Tab cycles the whole dialog with the prompt open, the prompt still swallows the page's keys, and the issue link is focusable where the composer puts it. All 24 regression tests mutation-checked. |
||
|
peter
|
3f983e6d76 |
fix(diagnostics): deep-review round 2 — a send that claims, not one that blocks
The send no longer holds the session's row lock while it talks to the tracker: it claims the session, gives the lock back, and checkpoints each step. A Stop is never held behind a send, while a draft, a screenshot or a second send are refused at once instead of queueing. The claim ages out, so a request that died mid-send does not strand the session. What the composer says is what happened: an attachment that went is no longer named as one that did not (the missing body edit is its own line), and the issue's screenshot count comes from the server when the list cannot be read, so a failed read no longer blocks saving the draft. The zip is fetched as a request, so its pace is a message and not a saved error page. Screenshots of a finished report can be removed again; only one still being sent is held. A recording's log elevation can no longer outlive it. The step prompt keeps the keyboard (the page's keys stay quiet) and traps Tab itself. An image pasted into a dialog opened over the composer stays that dialog's. |
||
|
peter
|
134ec9a68e |
fix(diagnostics): deep-review round 1 — a send that cannot race, lose or repeat itself
The send, the draft and the screenshots take the session's row lock, so an autosave or a paste during a send waits for it and then sees it, and two sends cannot open two issues. A save keeps the delivery's progress; a failed edit that shows the screenshots is finished by the retry; an answer that is not the tracker's API (a redirect, a proxy page) is refused, not a crash that loses the issue's record. Only an attempt that opened an issue counts toward the pace. Screenshots: deleting one after the send is refused, a user's storage is capped across sessions, new sessions and zip exports are paced, images are stored uncompressed in the zip. One liveness rule, one session view, one image type table. Logging: a session's elevation is tracked apart from the user's own, so the browser's own requests (no header) no longer lower the floor a session needs; Stop ends it at once. Composer: an image pasted anywhere is taken; what is still unsaved is saved on close, and "saved" is shown only when it is; the draft is held while it is sent; a failed screenshot read says so instead of reading as none; focus moves to what stays; the preview follows the text size. Pill: the step prompt and a running Stop survive the pill moving between hosts, the prompt keeps the dialog's Tab trap, the toast region lets clicks through, other tabs hear of a Start or Stop, the last seconds are sent before the cap, a capture with no frame is an error. |
||
|
peter
|
9cf177b198 |
feat(diagnostics): send a bug report to a private Gitea issue tracker (E16 S4)
- admin config group "Bug reports": Gitea address, repository, token (secret),
and "Check the connection" — the repository must answer, take issues and
be PRIVATE (a report carries its reporter's log); checked on every send too
- POST /api/debug-sessions/{id}/issue sends the SAVED draft: create the issue,
upload each file on its own request (the measured API), then show the
screenshots inline; progress is kept in the draft, so a send that stops
half-way keeps its issue and the retry sends only what is missing; 5 new
issues an hour per user; audited as issue_create
- the composer: Create issue (only where a tracker is set up, naming the
repository), a partial-send banner with retry, and a Preview — the tracker's
own rendering in a sandboxed frame, or the zip's markdown without one
- the recorder's refusals answer in the localized family (debug_session_*,
screenshot_*, issue(s)_*), not bare codes the UI could only call generic
- a tracker refusal logs the status only: its message can quote the title back
|
||
|
peter
|
8d0083d4e4 |
feat(diagnostics): the issue composer — draft, review, screenshots, zip export (E16 S3)
After Stop (or at the cap, or from Settings → Diagnosztika's past sessions) one dialog drafts the issue: title, description, the steps drafted from the session's markers and page changes (a marker without a label is named by its time), the environment (version, browser, system, window, language). Beside it, what goes with it: the browser and server logs — each reviewable line by line, an unticked line is not sent — and the screenshots, removable, with paste / drop / pick to add more. - it saves as it goes (debug_sessions.draft, migration 0090): the rendered markdown in the user's language, the excluded rows, and its own fields to reopen as it was; a sent session is read-only - the zip export (issue.md, client.log, server.log, screenshots) is built from the saved draft by app/issues/bundle.py — the one definition the Gitea delivery (S4) will share; .log, not .jsonl, which Gitea refuses - the log line's level tags and clock move to components/logLineParts so the review list draws a level the way both log viewers do |
||
|
peter
|
a7d7456279 |
feat(diagnostics): debug sessions — start, stop, the tag on every line, screenshots (E16 S1)
A session is one recording, capped at 60 minutes by the server. While it is open, a request naming the caller's own session in X-Debug-Session is logged at the session's level with every line tagged; a header naming anything else is ignored. The browser's batches count a session as consent to its level and keep the tag only on its own events. Screenshots are judged by their bytes, capped in size and count, and served back to their owner only. - debug_sessions / debug_screenshots tables, one open session per user (partial unique index), log_events.debug_session_id indexed - /api/debug-sessions: start (joins a running one), stop, list, get, events, screenshots; /api/me reports the live session - the hourly log job closes expired sessions and drops them with the logs - logsink.serialize and routes._common.capped_body shared by their readers |
||
|
peter
|
0ca0cac534 |
fix(api): a waiting reorder deals from the fresh order; the commit check sees staged writes
Round 2 of the deep review, on round 1's own fixes:
- `ORDER BY position … FOR UPDATE` sorts before it locks, so the reorder that waited got the
fresh rows in its STALE order and dealt the pins it left out (a tab with Plex off) back to their
old slots. It locks in id order — one global order, so two reorders cannot deadlock — and sorts
the fresh rows itself. The race test now sends that partial list and fails unless tab B really
blocked.
- The request check saw only writes the handler flushed: a `db.delete` left staged, or a flush
followed by a released savepoint, passed. Staged changes count at the end of the request, and
only the ROOT transaction's end clears it (a savepoint test drives the dependency directly).
- The `ok` test exempted every call as a serializer, hiding four domain REPORTS (YouTube sync,
revert, push; Plex sync) without `ok`. They are spread into `{"ok": True, …}`; the exemption is a
named list of real serializers.
|
||
|
peter
|
955c8c3445 |
fix(api): a token hiccup is not "reconnect"; reorders and approvals are one unit each
Deep review of the debt burn: - `refusal_reason` told the user to reconnect on ANY token-endpoint failure — a 5xx, a 429, a misconfigured client — while the grant stayed and the account still read as connected, with no connect button. Only `invalid_grant` (the one that drops the grant) means reconnect now. - `pins.reorder` writes only the rows that change, so two overlapping reorders each skipped rows that matched in their own stale snapshot and interleaved into two pins on one slot. It locks the user's pins first; the second waits and deals from the first's result. - Adding a whitelist entry committed the invite and the activation it implies separately, and its audit row was written before a new invite had an id (target NULL). One transaction, flushed for the id. - The mutation convention test skipped async routes and dicts returned by name: request-access, purge-discovery and the subscription sync answered without `ok`. They do now, and the test covers both. The helper-commit check looked only at `_`-names; it reads every non-route function, which listed two deliberate committers (upsert_pending_invite, ensure_welcome) — now allowlisted with their reasons. |
||
|
peter
|
99df44d7fb |
fix(api): mutations answer ok; route helpers flush and leave the commit to the handler
Two rows of the canonical-patterns table had drifted, found while closing the roadmap audit:
- 63 mutation responses carried no `ok` — `{"unsubscribed": true}`, `{"status": "ok"}`,
`{"added": 3, …}` and so on. Each now leads with `"ok": true` (additive: every field a client
reads is still there).
- Twelve route helpers committed on their own. Six were just the end of their handler's work and now
flush: the playlist item add/remove and Watch later get-or-create, the Plex series/season bulk
mark, and the invite decision + account activation — which an approval used to land in separate
transactions. The other six commit on purpose (a token durable before its link is mailed, a
single-use token burned as its own unit, the Google callback's tail, a background registration
with its own session, the staged search ingest around YouTube calls), each listed with its reason.
`tests/test_route_conventions.py` holds both rules in code; CLAUDE.md's two rows point at it.
|
||
|
peter
|
7b94e9a651 |
fix(home): shelf honours the Plex gate; hero unpins its origin; palette asks for a surprise
- A13: with Plex off, the Home shelf and the hero's comfort tier go YouTube-only like the rails beside them (owner's call: hide the tiles). The pins are kept; GET /pins still lists them. - Reorder deals the listed pins into the slots they already hold, so a partial list (hidden Plex pins, a stale tab) no longer puts two pins on one position; repeats are ignored. - A21: a comfort hero carries `pinned_from`, the shelf entry it was resolved from; its menu acts on that pin, so a pinned show's next episode offers "Unpin" instead of creating a second pin. - B12: the palette's "Meglepetés" asks for the surprise tier (the next press leading with it); the rotation carries on from there. - B13: the surprise token names the history entry it was requested for (minted by setPage's new `then`, after the push) and is dropped on any other entry, so a navigation that never mounted Home cannot replay it later. |
||
|
peter
|
7ae5c57b6b |
fix(i18n): every refusal the API answers is structured and said in the reader's language
~150 HTTPExceptions carried a hand-written English sentence as their detail,
shown verbatim under the Hungarian UI — in the blocking error dialog and on
every inline surface that printed HttpError.detail (sign-in, setup wizard,
change password, the passphrase gate, live search). Four also quoted an
upstream exception's text back to the user.
- app/refusal.py: `refused(status, reason, **params)` → {code: "refused",
reason, …params}; localizeDetail translates errors.refused.<reason> with the
params interpolated. "Unknown X" 404s use the existing not_found(what).
- sysconfig.set_value raises InvalidSetting (reason + bounds) instead of an
English ValueError; the three writers answer it through setting_refused.
- The Plex connection/write/image failures log what the upstream said and
answer a reason; the search scraper's failure is logged and answered as
youtube/unavailable (it crashed refusal_reason before).
- HttpError.detail now only ever holds a translated reason: a string detail
(only the framework's own can remain) is left out, so reading .detail is
safe everywhere — including destructured, the hole the raw-detail source
scan could not see. That scan and its five-file exception list are gone.
- Enforced: test_http_details.py fails on any HTTPException with a sentence
for a detail; the contract test requires every refused reason and every
not_found "what" in both languages, and no key for one that is gone.
- local/jsx-comment-text: a // inside JSX children renders as text; the rule
is ten lines in eslint.config.js rather than eslint-plugin-react's ~1800
lockfile lines.
|
||
|
peter
|
dc4e8b9fcd |
fix(plex): playback sessions per viewer, bounded in total; a fair share of subtitle passes
HLS sessions were keyed by the item, so a second account opening the same film killed the first one's session; the displaced player reclaimed at its playhead and evicted the other in turn. The registry is now keyed by (viewer, rating_key), segment requests resolve by both, and the session directory carries the viewer (the orphan sweep still recognises the older shapes). Only transcodes counted toward a cap, so remuxes — each a whole segmented copy of the film on the scratch volume — were bounded by the library size. Three bounds now apply after every start, each dropping the least recently used: two sessions per account, the transcode cap, and a new admin knob, plex_max_sessions (default 8), over every session. A superseded session request returns at once, since it waits on the killed ffmpeg, so an account cannot park more request workers than it may hold sessions. The subtitle extractor's full-pass slots were global and minutes long each; one account opening subtitles on film after film could hold them all. An account may now hold three of the six, and still gets windows past that. |
||
|
peter
|
6f9ba428d5 |
fix(playlists): the move's conflict fallback says why it is silent
The house guard requires a silent handler to carry a silent-ok note; the
savepoint fallback added in
|
||
|
peter
|
e6b76e156a |
refactor(discover): one shelf query per kind, one RAIL_LIMIT, a true rollup comment
The comfort shelf resolved each pinned channel with three queries (the row, its unwatched count, its newest video) and each playlist with two. channels_summary gets the count and the newest row per channel as window functions over one filtered set, playlists_summary reads every pinned list in one query, and the rows come in one IN() each — so the shelf's cost no longer grows with its length. A test counts the statements. RAIL_LIMIT had two definitions held in step by a comment; the Plex Discover route now imports the domain's. And rollup's "continue the latest-started episode" was false: it takes the last in-progress one in broadcast order, which is also what Home's next_episode picks. |
||
|
peter
|
e74986cc9a |
fix(playlists): a racing move is not a 500; ownership through owned_or_404 everywhere
move_item checked "already in the target?" and then re-parented without a lock, so a second tab's add or the scheduler's YouTube merge committing the same video in between hit uq_playlist_video and answered a raw 500. The re-parent now runs in a savepoint and a conflict falls back to the already-present outcome. The target is touched only when the row landed in it (an untouched list was promoted in the rail's "recently changed" sort), and _touch stamps the database clock like the mixin's own onupdate. The canonical-patterns table names owned_or_404 against ad-hoc ownership helpers; six were left (_own_playlist, _own_playlist_or_404, _own_view, _own_tag, notifications._owned, downloads._own_job), two of them hand-rolled with a bare English 404. All call sites now use owned_or_404, and the download/link/notification 404s are structured like the rest. The contract test reads every "what is gone" the routes name, and the YouTube refusal reasons, and requires both languages to name each. |
||
|
peter
|
bacd6aec4f |
fix(plex): one scrobble per finish, and no resume push lands after it
Two near-end checkpoints arrive together routinely (the 10 s tick and the debounced seeked flush); both read "not watched" before writing and both scrobbled, bumping Plex's viewCount twice. The transition is now decided by the write: an insert that did not conflict, or an update conditional on the old status. The finish also popped the push-ordering entry that _resume_push_superseded reads, so an older resume push still in flight could land at Plex after the scrobble and revive a resume point on a watched item. It now stamps the entry instead; the existing eviction bounds the map. |
||
|
peter
|
fad301e7a4 |
fix(youtube): a refused YouTube action answers a 422 that says what to do
The two manual syncs (subscriptions import, playlist pull) had no YouTubeError
handler, so a quota, token or transport failure was a raw 500 and the blocking
"Server error" dialog (E9 F1). The other YouTube routes answered either the
upstream's English or a playlist-scoped "youtube_failed".
One helper, routes/_common.youtube_refusal: 422 {code: "youtube", reason} with
the reason sorted by what the user can do — quota (wait for the reset),
reconnect (the token step refused), unavailable (retry). Used by every route
that surfaces a YouTube failure; localizeDetail translates it.
"Reconnect" had no path: a grant Google revoked still read as granted, so
Settings and the wizard offered no connect button while every call failed.
An invalid_grant from the token endpoint now clears the stored grant.
|
||
|
peter
|
373415a2eb |
feat(account): show the account its own last sign-in
Settings → Account names the latest sign-in on any device (only the admin's user list did, though the 0.88 notes said otherwise), and the role in the reader's language instead of the raw `user`. |
||
|
peter
|
8f6f89832f |
fix(diagnostics): honest live console after review
- Live stops polling on a failed read and marks the list as possibly stale (with Retry), instead of re-raising the error dialog every tick. - While live, a refetch reads the newest page only, whatever the cache holds (a late "Load older", a revisited filter set). - Copy is disabled and the list dimmed while a filter change loads. - Dates follow the UI language; the pivot chip uses the shared .chip. - Tests: query-string wire test, Copy, every facet's skip key. |
||
|
peter
|
b5996a0e56 |
feat(diagnostics): the admin log console
Admin → Diagnostics reads the server's stored log: one list, newest first, where a line opens in place to its full record (time, source and logger, account, request and job ids, version, message, attributes, stack). A request or job id narrows the list to that request or job with every other filter off, shown as a removable chip. Filters: level group, source, account, event, period and a free-text search; all of them live in the URL (`l*` keys, which setPage clears like the Plex and Channels keys), so a filtered view is a link and Back restores it. "Live" re-reads the newest page every few seconds while the tab is visible; paging into older rows turns it off. Copy puts the loaded lines on the clipboard as JSON lines. The facet counts now honour every filter but their own, as the feed's do: a request pivot narrows them, and a picked level does not zero its siblings. A value in use stays offered even when the counts drop it. "Saját napló" and the console draw a line with one shared component, `LogLine`, whose phone layout gives the message its own full-width line (sharing the code's line left it a few characters). The level groups moved to lib/log.ts for both. The console's own reads are logged at DEBUG on success, like the healthcheck: at INFO its live tail filled the log with itself. |
||
|
peter
|
5e208353e3 |
feat(diagnostics): admin query API over the stored log
GET /api/admin/logs pages `log_events` newest first on a (ts, id) keyset: a browser row's ts is clamped into its consent window, so ts alone is not unique. Filters AND together: level, source, account (or `none`), event code (or `none`), request id, job id, a free-text search over the message, code, request id, stack and attributes (typed text is a literal), and a time window. Each page names the accounts it mentions. GET /api/admin/logs/facets counts what the filter chips offer over the window alone, so a chip row does not shrink as filters are picked. Admin-only and read-only; reading is not audited (the console's live tail polls). There is deliberately no "ids after N" tail: several processes write this table and a batch takes its ids at insert, not at commit, so a tail keyed on ids could pass a row that commits late. |
||
|
peter
|
66dbd0af81 |
fix(search): typed text is a literal in every LIKE pattern
The feed's channel-name match and the live search's term fold built their LIKE pattern from the typed text as it was, so `_` and `%` acted as wildcards: `build_log` also matched `buildXlog`. The term fold also tested a substring, so a term already present inside a longer word (`cat` in `category`) was never folded into search_terms. It now compares whole terms, escaped. `like_escape` moves from the Plex query module to `app.utils`, beside `to_tsquery_str`, and every pattern built from typed text goes through it. |
||
|
peter
|
54d19cc386 |
fix(logs): deep-review round 1 — say each failure once, and scrub it everywhere
- yt-dlp's text is scrubbed wherever it is logged, not only in the adapter:
`formats.log_safe` takes out the URL, the `[extractor] <id>:` prefix and
file paths, and the job-failed, refused, edit-failed and backfill lines use
it. Done lines log the asset id, not the title-built path. The PII contract
now bans `rel`/`rel_path`/`title`.
- A traceback goes with a download/edit failure only when it is our own code
failing; for yt-dlp's DownloadError, our RuntimeErrors and OSErrors its foot
is the unscrubbed message.
- yt-dlp's `error` maps to WARNING: it fires on attempts the fallback client
set recovers from; the job's verdict is `download.job.failed`.
- The sink groups a bare `%s` pass-through by its text (numbers folded), and
a process's last batch closes the repeat window, so a short-lived download
child still reports what it dropped.
- Worker-side failures (stall, wall-clock, no asset, cached asset error) log
`download.job.failed` with a cause; they left no line at all before. The
crash, supervisor-error and non-zero-exit lines carry the code too.
- An edit failure is one coded line in `_fail_edit`, for every path.
- `logs.get_logger("__main__")` resolves to the `-m` module, so the worker
and the download child keep their names.
- TRACE lines carrying user content are gated on `logs.consented_trace()`,
not on the process level; a contract test pins it.
- The OAuth token refresh goes through `_upstream_failure`; transport errors
carry reason `transport`. Subscribe recognises a duplicate by reason code
only (the prose fallback was dead).
- Once-only guards for a stale sysconfig override and a refusing HLS session
directory; item_detail splits Plex failures from our own parse bugs.
- The playlist breaker line names the playlist and the action; comments that
were false (runner noqa, `_merge_vtt`, sink grouping) are corrected.
|
||
|
peter
|
04b58e5c6f |
feat(logs): give the log the events that no request line already carries
The gap this sprint was scoped against turned out to be smaller than the survey said, and in a way worth writing down: `log_requests` already records every HTTP call with its route template, status, duration, request id and user id, and the audit log already records every admin act on someone else's account. Putting a line in each of routes/downloads.py's twenty mutating endpoints would have added a second copy of the first and nothing else. So this instruments what neither of them can see. yt-dlp now explains itself to our log. `quiet: True` was doing more than quieting: with no `logger` its whole account of a download went to the child's stdout, so "Requested format is not available", a signature-extraction failure or a POT refusal reached us only as whatever text ended up in the raised exception — the real reason a download failed was routinely one line the app never kept. Measured against the pinned yt-dlp 2026.07.04 before wiring it: with `logger` set it calls `debug` for extraction and progress lines and `warning`/`error` for the rest, and neither `quiet` nor `no_warnings` gates those calls. `debug` lands at TRACE, being several lines a second, so the noisy half is the half the user's own consent gates. The full source URL is trimmed to its host on the way in — it is a row of someone's download history, and `logs.set_job` already puts the job id on every one of these lines. The option is set AFTER the adapter merge, like `cookiefile` and the size cap: a source adapter must not be able to take away the one account of what went wrong that a failed job leaves behind. An HLS session now says which pipeline it got (transcode/remux, hw, tonemap, variants, offset) — everything downstream is about that choice, and it was only inferable from an ffmpeg command line nobody keeps. A session culled by the concurrent-transcode cap says so: somebody's playback stops there, the route turns it into a retryable 404, and "my film keeps rebuffering" had no other explanation anywhere. The HW→SW fallback keeps its line and gains an event code. A live search refused by a quota cap says WHICH cap: the 429 is in the request line either way, but the per-user daily cap is answered by tomorrow and the shared budget by the admin. The download and edit job lifecycle lines gain event codes, which is what E22.6's console filters on — the job id was already a structured field. tests/test_ytdlp_logging.py; three of its four cases mutation-checked. A fourth (a `%` in a progress line) survived its mutation — stdlib only applies `%` formatting to a record that has args — so it is gone rather than kept as a case nothing can break; the reasoning moved into the comment on `_emit`. |
||
|
peter
|
6b0401b922 |
fix(logs): rule on every swallowed exception in the backend
114 exception handlers neither logged nor re-raised. The defect was never that
they are silent — plenty of them should be — it is that their silence said
nothing, so nobody could tell a decision from an oversight. Each one has now been
read. 44 got a line; the other 70 carry `# silent-ok: <why>`.
What the silent ones had been hiding, in the cases that got a line:
- A failed ffprobe sent every HDR file down the 8-bit pipeline, so playback
degraded to a software transcode with nothing saying why; a failed audio probe
collapsed a multi-audio file to one track and the player showed no language
picker; a failed first-PTS probe left subtitles seconds out of sync.
- A subtitle sidecar that existed but would not parse made a PARTIAL track read as
complete, permanently — the viewer loses every cue past the cached window.
- An unreadable crop/trim/cut-list in an edit spec was dropped, so the clip came
out uncropped, untrimmed, or shorter than the one the user built.
- A dropped `max_height` downloaded at best quality against an explicit cap.
- A failed videos.list enrich made every candidate undroppable-from-judgement and
therefore dropped: a working search rendered as "no results".
- An unusable password hash answered as "wrong password", locking the owner out
and telling them they mistyped; a rotated TOKEN_ENCRYPTION_KEY turned every
stored token into "not connected".
- A stored admin setting that would not coerce fell back to the env default while
the settings page went on showing the saved value.
- A one-off InnerTube config failure pinned the whole process to the hard-coded
fallback key, and searches return nothing when that goes stale.
- The HLS sweep reported 0 swept — like a clean run — when it could not list its
own root, leaving every crashed session's segments on disk forever.
Four of them are counted rather than logged per item (dropped cues, dropped cut
list segments, unreclaimed GC files) and reported once per call, because the
alternative is one line per cue over a whole subtitle track.
ruff gains S110 and BLE001. Every blind `except Exception` that remains carries
`# noqa: BLE001 — <why>`, which is the point: catching everything is sometimes
right, and always worth a sentence. plex/sync.py's three best-effort backfill
catches now keep the traceback — a blind catch over a batch that fetches AND
writes could be Plex, the database or our own bug, and `%s` of the exception told
them apart badly. main.py's boot-time log-level read likewise.
logs.event's `code` is positional-only now: `event("x", code=...)` was a TypeError,
and `code` is exactly the word a caller reaches for when the thing that failed has
one of its own.
tests/test_swallowed_exceptions.py is the wider rule ruff cannot see: a handler
that neither logs nor re-raises must say why. Both cases mutation-checked — a
marker removed and a log line removed each fail it, naming the handler.
|
||
|
peter
|
0cff964ff2 |
refactor(logs): name every logger after its module, and hold it there
25 literal names covered 34 modules. plex/client.py, plex/stream.py, plex/subtitles.py, plex/sync.py and two Plex route modules all called themselves `siftlode.plex`; downloads/pipeline.py called itself `siftlode.worker`; routes/channels.py called itself `siftlode.api`. The `logger` column is in every stored row so E22.6's console can filter on it, and so an operator can pin a level to one part of the system — neither works when six modules share a name. `logs.get_logger(__name__)` derives it: app/plex/sync.py -> siftlode.plex.sync. One name per module, and it cannot drift from the file it sits in. tests/test_logger_names.py pins the derivation, that everything stays under the one `siftlode` tree the handler and the tests attach to, and that no module hand-rolls a name again unless the line says why (`# logger-name-ok: <why>`). Two do: get_logger's own call, and `siftlode.http`, which is named for the traffic it carries rather than the module it lives in — it replaced uvicorn's access log and is what an operator pins when they want request lines and nothing else. app/scheduler.py called its logger `logger`; every other module calls it `log`. ruff gains LOG, G, T20, TRY400, TRY401 and S112. Four findings cleared with them: worker.py logged `str(exc)[:120]` where the traceback was the whole question, and three prints/f-strings that are deliberate now say so on the line. S110 and BLE001 stay off until S3 — 23 handlers are outstanding and each is a judgement about one of them; enabling the rules first would just mean 23 blanket noqas. |
||
|
peter
|
dc221338be |
fix(logs): keep what a person typed out of the backend log lines
The E22 redactor is a safety net over addresses and credential params, and every one of these call sites was leaning on it to do the whole job. What it does not cover went in whole, and what it does cover came out as a constant. Not covered, so a real leak: - routes/search.py logged the raw search term at ERROR. The privacy policy shipped in 0.88.0 says a search string is kept only at Teljes/TRACE, with the user's own consent. The ERROR line now carries the shape of the failure (source, candidate and collected counts) and the term follows on a TRACE line, which nothing emits unless that consent is in force. - downloads/backfill.py logged the full source URL at INFO — a row of the user's download history. Now the asset id and the host. - youtube/client.py logged the upstream error body, which is prose that quotes the request back. Every failing YouTube call now goes through one _upstream_failure() that logs the API's `reason` code, keeps the body for the same consented TRACE line, and raises a YouTubeError carrying `reason` as a field — callers put that error in log lines of their own (routes/channels.py) and in a 422 answered to the user, so the body must not travel in it either. Covered, so merely useless: routes/auth.py named the account by an address that reached both sinks as `[email]`, and app/email.py printed recipient lists that read as `['[email]']`. Both now carry the uid (or a count) and a reason code, which is also the first half of E22.5 S4's failed-sign-in coverage. routes/channels.py branched on the text of that error message to recognise an already-existing subscription. It now branches on `exc.reason`, with the substring check kept beside it for a body that carried no reason at all. tests/test_log_pii.py pins the call sites and the family: a source-level contract test walks every log call in app/ and fails on an argument naming user-typed content unless the line says why it is safe (`# log-pii-ok: <why>`). Eight sites carry that marker today, each with its reason. Gate 8/8 green; all five cases mutation-checked. |
||
|
peter
|
07e3ea3737 |
docs: the deep-backfill card counts the backlog, not the next pass
Two reviewers in a row read the admin card's selection against `run_deep_backfill`'s, saw it was missing the job's `recent_synced_at` filter, and called it a card that counts channels the job will never touch. It is the opposite: the card is the same backlog figure /api/sync/my-status shows the user, and a channel still missing its recent pull is one the same job tick fetches first. Narrowing it would hide genuinely queued work. Written down where the next reader will look. |
||
|
peter
|
b2eaa9e00b |
fix: a credential, an erased account and a stale level stay out of the log
Round 2 of the E22 review, on what round 1 changed. `redact_query` decodes a parameter before the text rules see it, which is what lets `%40` be recognised as an address — but `_SECRET_PARAM_RE` was written for the raw, encoded form, where a value holds no space, `&` or quote. On decoded text its value class stops at the first one, so `?state=a%20b` kept its tail and `?code=%20abc` was not matched at all. The parameter-NAME rule now decides, on the key, before any text rule runs. `forget_user` dropped the account's level elevation without recomputing the root level, so a process whose only elevated account had just deleted itself kept building TRACE records to throw away, until some later request happened to recompute it. Every other path that drops an elevation recomputes. The sink's writer took the erasure lock and THEN waited for a pooled connection, while every request path already holds a connection when it reaches that lock — so under pool pressure the writer held a process-global lock against the very requests that could hand a connection back. Same order on both sides now; the lock still spans the check, the insert and the commit. The storage-budget notice said the rest of the hour was not kept, and then smaller batches kept landing underneath it: the window slides and a refused batch is not charged to it. Says what it does. Tests: the route's half of the erasure rule had no test — only the sink's did — and a source-level contract test now covers the third insert nobody has written yet. `logs._erased` is process-global with an hour's memory while the DB fixture hands serial ids back out from 1, so it is swept before each test like the rate limiters are. |
||
|
peter
|
4bfba4aa86 |
refactor: one definition of a followed channel and of a choice spelling
The explore purge rebuilt followed_channel's two EXISTS clauses inline, so what is polled and what may be deleted could drift; it now uses the predicate, which moved to sync/explore.py. The env validator and sysconfig each folded a choice value on their own; both call config.canonical_choice. The deep-backfill comment no longer calls recent backfill unconditional. |
||
|
peter
|
fac5c4465a |
fix: stored logs keep bad rows, erased accounts and unconsented queries out
Deep-review round 1 of E22. - A NUL character (`GET /assets/%00` logs the decoded path) made Postgres refuse the row and with it the sink's whole batch, every user's rows included. `to_row` replaces NUL in every text column and in attrs. - An erased account's lines kept arriving after purge_user's DELETE: the erasing request's own access line, a queued row, a background task. `logs.forget_user` runs first; later rows are stored without the account, and the check and the insert share a lock with the erasure. - An admin setting the whole process to TRACE logged every user's query string. The query is now logged only under the user's own Teljes consent, and decoded before redaction, so `%40` no longer hides an address. - The browser log ingest has an hourly storage budget per account (5 MiB, one notice when spent) and reads only a JSON Content-Type, so a same-site text/plain form post cannot write into a user's log. RateLimiter takes a per-attempt cost for the byte budget. - The root level is applied under the elevation lock: two concurrent changes could leave it at the base while an elevation was active. - Tests for the running sink (level gate, drain on stop) and for the admin log level applied at runtime and at boot. |
||
|
peter
|
5883ea509d |
feat: Settings → Diagnostics, a consented log level and the tab's own log
A user can raise their diagnostics level to Részletes (DEBUG) or Teljes (TRACE) for 24 hours from a new Settings tab, after a consent box that says what is recorded, that the server keeps it and for how long (the instance's own retention), and that it lapses. Alap is the base and needs no consent. - users.diagnostics_level/_until (migration 0087), written only by PUT /api/me/diagnostics, which computes the expiry itself; the demo account cannot raise it. /api/me reports the level in force. - The server logs an elevated user's requests at that level and stores every line of them (logs.LevelGate on the stdout handler and the sink; the root is lowered only while an elevation is active). Teljes adds the redacted query string to the request line. - POST /api/diagnostics/events stores the browser's log as source=web rows: account from the session, body capped while streaming, events/attrs capped, re-redacted, events under the consented level dropped, timestamps clamped, 30 batches a minute, 409 once the level lapsed. - lib/diagnostics.ts applies the level to the tab and streams only what happens after consent, in batches, never its own request lines, with a keepalive flush on pagehide; stops on lapse (its own timer), on withdrawal and on 409. - Detail hooks at DEBUG: which control was clicked (test id, icon, app copy in English — never user text, input values or a link's query), visibility/bfcache/resize, filter changes (lib/filterLog.ts; typed search only at TRACE), player events (lib/mediaLog.ts; media errors are warnings at every level). Offline/online at the base level. - "Saját napló" viewer: live, newest first, level groups with counts, search, row details, copy the filtered events as JSON lines, clear. - Privacy policy and Terms exist in Hungarian and English (a contract test holds both languages to the same sections); the policy now describes the server logs and diagnostic logging, self-service deletion and password sign-in. Fixed on the way: - A sync auth dependency ran logs.set_user on a copied context, so the account never reached a handler's own log lines (measured); a per-request mutable scope carries it now. - The Settings tab rail left a phone no room (clipped labels, buttons over text): below `sm` it is a scrolling row above the content. - PUT /api/me/preferences took a bare dict body; it is a Pydantic model now. |
||
|
peter
|
0517de1f59 |
fix: poll and backfill only channels somebody follows
The scheduled RSS poll and recent backfill covered every channel row, but rows also come from search results, playlist sync and dropped subscriptions, and nothing deletes them. On prod 582 of 979 polled channels had no follower, three of them dead feeds answering 404 on every poll. Both jobs now take subscribed or explored channels only, and the scheduler card counts the same set. As a deliberate trade-off, the shared Library no longer receives new uploads from channels nobody follows. |
||
|
peter
|
07766c43e8 |
feat: closed-set admin knobs are pickers, not free text
log_level, search_source, download_layout and the transcode hw/preset were typed by hand, and every consumer carried its own guess at what a typo meant. The value set is now a Literal in config.py: pydantic rejects a bad env value at boot (any letter case accepted), sysconfig refuses a bad override and derives the admin picker's options from the same Literal, and the consumers read the value as-is. The Configuration page renders a choice key as a segmented bar, or a dropdown past six options, with translated option names. Its tabs now follow the registry's order instead of a hand-kept copy that had already dropped the explore group. A single-choice FilterSelect now opens on the option in use: in the browser it could land on the first row instead. |
||
|
peter
|
6c63df6704 |
feat: one structured log for every process, with a request id
A deployment printed four different log formats and none of them could be joined up: uvicorn's log_config.json for the API, a hand-rolled StreamHandler in main.py for anything not started by uvicorn, logging.basicConfig() in the worker and in the download child (no timestamp at all), and alembic's fileConfig. app/logs.py replaces all four with one JSON line per event (OpenTelemetry's log data model, no SDK): ts, level, severity, source, logger, event code, message, attributes, plus the correlation the context carries — request id, account, download job. Redaction is applied at the HANDLER, not at the call sites: every message, every attribute and every traceback is scrubbed on the way out, so a secret cannot leak by being forgotten at one of ~125 call sites. Passwords, tokens, cookies, secret config values and email addresses never reach a line. The access log is ours now (`http.request`): the route template instead of the concrete path, the status, the duration and the account — and never the query string, which uvicorn's access log printed on every single request (`?account=`, `?source=`, the `?token=` of a share link). A well-formed client request id is accepted and echoed in X-Request-Id, so a browser line and a server line join up. The account comes from `request.state`, filled by the auth dependency: the middleware is the outermost one, so it runs outside SessionMiddleware and in a different task context. `log_level` becomes an admin knob (sysconfig), applied without a restart and re-applied at startup — best-effort on purpose, since a logging knob must never be what stops the app from booting. Every compose service gets bounded json-file logs (3×50 MB): Docker keeps every line forever by default, and the API now writes one line per request. Verified on the localdev stack: JSON everywhere including alembic and the worker, the query strings gone, a supplied X-Request-Id echoed and logged, and three /healthz hits of which only the one made while the level was DEBUG produced a line. |
||
|
peter
|
11aef4ab7a |
feat: record when an account last signed in
users.last_login_at (migration 0085, no backfill) is stamped by _establish_session, which the Google callback now shares with password login instead of repeating its session setup inline. The demo entry does not stamp its shared account. The admin user list carries the field. |
||
|
peter
|
47d968eeea |
fix: review round 4 — an account switch unloads as the old account, a share link's lockout costs five networks
Deep-review round 4, targeted at round 3's changes (f569b19c..e7f38396). Accounts - Switching, adding or signing out an account moved the tab's pin before the page unloaded, so the players' resume beacons fired on the way out under the NEW account: one account's playback position saved into the other. The pin is now handed to the next page (pinOnNextPage) and taken over at import; clearActiveAccount is gone. - Switching to a suspended account was refused but left it in the switcher, refused on every click; a deactivated one was not refused at all, and made the default it signed every account in the browser out on the next request. Every account _session_user would refuse — gone, deactivated, suspended, revoked — is refused and dropped from the list. Share page (/watch) - An IPv6 client is limited by its /64 (client_ip becomes client_key, for every limiter): keyed on the full address, one subscriber could rotate through endless identities. The comments now state what the per-link ceiling costs: guessing from five networks can keep a link shut for its window, and guesses already running when it fills are still checked. - The link is looked up before the limiter records the attempt, so made-up tokens no longer grow it by an entry each. Forms - Failing the same way twice is announced again: the sign-in, passphrase and setup errors are keyed on the attempt (a clear and a set in one handler left the alert node untouched). - The install wizard's own validation showed the generic error — it threw plain objects its catch did not read. It sets the step's message. Tests: a re-pin leaving the previous account's notices behind; the pin hand-over; switching to an ended account; the /64 viewer; made-up tokens; the repeated alerts. Each mutation-checked RED for its own reason. |
||
|
peter
|
e7f38396a2 |
fix: review round 3 — a new tab keeps its account's notices, a revoked tab account lands on the default
Deep-review round 3 over main..improvement/watch-background-audio. Account boot - The notification history and settings, and the hints toggle, are read when their modules load, which in a new tab (bookmark, typed URL) is before BootGate pins the account: they read as nobody's, and the tab's first notice replaced the account's stored history (up to 100 entries) with a one-item list. They now read again when the tab is pinned (onAccountPinned, run by setActiveAccount); a notice raised before the pin stays, after the history. - A tab pinned to an account still in this browser's wallet but whose session ended (suspended, deleted, or revoked by "log out everywhere") got "signed out" from /api/me while the default account was still signed in, and signing in again as anyone else left it there. A refused per-tab account now leaves the wallet, and the probe answers as the default, so the tab reloads onto it. Share page (/watch) - The password unlock is limited per viewer (link + client IP), under a per-link ceiling that counts only wrong passwords. Keyed on the link alone, ten wrong guesses every five minutes locked out every viewer, the ones with the password included. client_ip moves to app.ratelimit so the public routes can key on it. - Its failure messages and the load-failure screen are announced (role=alert, the password input described by its error). The sign-in, passphrase, password-change and setup form errors get role=alert too. Comments that still described removed machinery are corrected: App's collapse flags, SettingsPanel's Save bar, ConfigPanel, authed_format_sig. Tests: the account's history and hints across the pin; a revoked and a suspended pinned account; the per-viewer limit and the ceiling; the limiter's exhausted(); the share page's announced failures; a share link's feed filters persisted. Each mutation-checked RED for its own reason. |
||
|
peter
|
f569b19c49 |
fix: review round 2 — a signed-out tab acts as nobody, honest share-page failures
Deep-review round 2 over main..improvement/watch-background-audio. Account resolution (server) - A request naming an account that is no longer signed in to this browser (X-Siftlode-Account or ?account=) now acts as nobody instead of the session's default account. A tab still pinned to an account signed out in another tab went on acting as the default: its player checkpoints and preference saves landed in that account, the messages socket and Plex playback ran as it, and its Sign out signed THAT account out. A hidden tab never polls /api/me, so a tab playing in the background kept doing it. The /api/me bootstrap probe alone keeps the fallback, which is how such a tab learns who it now is: the app's 401 handler reloads, and BootGate re-pins the tab. BootGate - On an identity change the tab reloads without moving its pin first. The tree being torn down still runs its unmount saves before the page goes, and with the pin already moved they wrote into the new account's local keys (a player volume still pending). The reloaded page corrects the pin. Share page (/watch) - Only a 404 says the link is gone. A 5xx, a dropped connection, and the unlock's own rate limit all rendered "no longer available ... expired or revoked" to a viewer with a valid link, with no way to try again. A load failure now offers Retry; too many password tries says so and keeps the form; an unlock that gets no answer says so instead of doing nothing. Strings in en/hu/de. Downloads - A cookie-authed asset is recognised under the current signature version and the one before it, so a job queued before a deploy that bumps the version still downloads with its cookie instead of failing on the login wall with an error that says no cookie was used. The app chunk is preloaded through lazyWithPreload, so a failed fetch no longer leaves an unhandled rejection beside the ErrorBoundary. The panel layout seed is one helper (seedLayout). Tests: the stale-tab refusal, its harmless Sign out and the probe fallback; the pre-bump authed asset; BootGate keeping the pin on reload and staying mounted through a same-account change; the reveal clock's later-commit hand-off; the prefs, panel layout and demo-scope seeds from the me cache; the share page's failure states. Each mutation-checked RED for its own reason. |
||
|
peter
|
2a9ddb78ac |
fix(downloads): E11.5 epic-end review round 2
Pause now re-reads the job under the same asset-then-job row lock as resume, cancel, delete and redownload. Its unlocked write waited for a concurrent cancel's lock and then set the released job back to paused, a holding status, so a later delete released the same hold again and could free a file another user's job still held (measured: the overlapping-request test ends 'paused' instead of 'canceled' with the old read). The lock test now calls the route handlers, so it also pins which helper each route uses. Comments follow the code: resolve_url's contract names both verdicts and that a failed page read is raised as itself; the inherited-failure diagnosis asks the cookie gate as it stands when the job is claimed, so its advice fits the cookie that job's Retry would use; the undecryptable-cookie warning no longer claims a download goes ahead, since the worker parent reaches it only to diagnose. |
||
|
peter
|
e887233210 |
fix(downloads): E11.5 epic-end review round 1
A refused Facebook cookie on a group post is no longer "no video in this post":
measured on dev, Facebook answers with its login page (no post id at all), so the
adapter now raises its own POST_NOT_SHOWN verdict and the catalog explains it as
the refused cookie. The adapter owns both post verdicts and the "cannot parse
data" wall wording, so the Chrome retry and the catalog read one constant. The
group-post "no formats" entry is cookie-scoped (with a cookie yt-dlp only gets the
post's video link) and uses the adapter's own group-post test instead of a
"/groups/" substring.
Job transitions that move an asset's ref count (resume, cancel, delete,
redownload) re-read the job under a row lock, asset first, so two overlapping
requests can no longer release the same hold. get_or_create_asset loses an insert
race inside a SAVEPOINT instead of rolling back the caller's pending writes (Retry
lost its cleared error). Every worker-side job failure goes through
finalize_if_running and writes error_detail, so a cancel that landed first is kept
and a failure never shows an earlier one's copy. The worker asks the download
child's own cookie gate (credentials.usable_cookie) when a job inherits a failed
asset.
UI: the Facebook copy no longer points at a Retry button the failure window does
not have, says "paste" like Settings, and the HU wording is fixed
("Újraletöltés" on the re-download confirm). A category this client has no copy
for falls back to the generic sentence instead of a raw key.
Tests: route serialization of error_detail, the probed duration through a real
_download, the page-read cap and a failed read, a Retry off a pending asset, the
confirm-button labels, the lock and savepoint races.
|
||
|
peter
|
98ec1ffcab |
fix(downloads): Retry and Re-download take a cookie added after the failure
The login-wall copy tells the user to add their cookies and press Retry, but Retry kept the job on the shared asset the worker never attaches a cookie to, so it failed on the same wall. Resume now moves a pending/errored job onto the asset enqueue would pick with the cookie (never off a file or a download in flight). Re-download salted its fresh asset with the job's previous asset id, which the cookie gate cannot recompute, so even a cookie download rebuilt without the cookie. The rebuild salt is now the new asset's own id (plus the user's cookie salt when one applies), and is_authed_asset recognises it. |
||
|
peter
|
18c11ca437 |
feat(downloads): a per-site failure catalog — Facebook's walls say why and what to do
A failed Facebook download read "Download failed": yt-dlp words the login wall (18+ reel, closed group) as "Cannot parse data", which no category matched. Measured on dev with the owner's cookie before writing a rule — the same text is the login wall without a cookie and a refused cookie with one. - errors.diagnose(msg, url, cookie_attached) tries a host-scoped catalog first (all-of markers, cookie state, link fragment) and falls back to the category. First entries: facebook.login_wall, facebook.cookie_rejected, facebook.group_post_video. The module doc is the recipe for the next entry. - download_jobs.error_detail (0084) + backfill of the stored Facebook parse failures that were unknown. - The pipeline's failure branch passes the link and whether the cookie went with it; resume and redownload clear code and detail (redownload never cleared error_code). - A job meeting an already-failed asset got the text but no code, so it read "Download failed" where the first job said why. It is diagnosed again, with the cookie state from credentials.is_authed_asset (now shared with the cookie gate); the redacted network text classifies back to network. - FE: failCopyKeys gives the row and the failure window one lookup — catalog copy, else the category. Copy in EN and HU; a contract test pins every backend entry to both locales. |
||
|
peter
|
f130a1c05a |
fix(keys): epic-end review round 1 — Back, menus and dialogs keep the focus where it belongs
Deep-review of E11 (19 raised, 14 confirmed; 2 refuted, 3 latent test-hardening notes dropped). - Back from a sub-view (a Plex show page, a Messages thread) restores the item it was opened from even when focus was still inside the view: the first attempt now waits one task, past the sub-view's own popstate listener (measured in Chrome: it left focus on <body>). - The restore listener is installed at load, so the first Back after a reload finds it; a push cancels a restore still waiting for the last landing's lists. - Navigating over an overlay asks it again, a render apart, while its entry stays: the Plex player peels one panel per Back step, so Shift+W over info + stats closed one panel and dropped the navigation. - j/k: a native <button> item keeps its own Tab stop; landing twice on the same item no longer leaks its tabindex; after the focused item is removed (a hidden card) the next j goes on from its slot when that slot is on screen. - An abandoned `g` no longer swallows a modifier chord (g, then Mod+K opened the browser search). - A hover card above a dialog no longer switches its Tab trap off (player, Modal, wizard, Lightbox): new layerStack `anyBlockingLayerAbove`. - "Manage views…" closes its menu before opening the dialog, through a shared `closeThenAct` that ActionMenu now uses too. - Copy: the EN welcome says "Feed"; the EN unreachable reason no longer claims a change; stale comments about PlayerModal's scroll lock and dialog contract. |
||
|
peter
|
8c51062e5b |
fix(api,ui): the demo refusal says why at every site, and the contract guard cannot fail open
Round 6 converted three of four `is_demo` 403s to a structured detail and wrote a docstring saying it
had converted all the sites there are. The account-delete route in `routes/me.py` still answered a
hand-written English sentence, which `notifyActionError` discards in favour of the caller's
translated fallback — so the demo user gets no hint that the demo account is the reason. It is
UI-gated today, so the string is unreachable; what was reachable was the false completeness claim,
which is precisely what stops the next reader from looking. The site is converted, and the list is
now DERIVED from the source: a new test walks `app/` for every `is_demo`-guarded 403, asserts each
raises a structured detail, and fails when the count moves — so this file cannot fall behind the code
the way it just did.
The same round's frontend contract guard was failing open. Its pattern required a bare identifier
before `.detail` and excluded `.detail` used as a last call argument, so `setErr(e.detail)`,
`(e as HttpError).detail`, `String(e.detail)` and four more idiomatic forms were invisible — and
because the file set is compared with `toEqual`, a file the pattern cannot see is simply never listed
and the gate stays green over a fresh copy of the contract. Widened to cover cast receivers, optional
chains and call-argument reads; the pinned set is unchanged, so the narrowing had removed no false
positive. A destructured `detail` stays out of reach (a pattern loose enough for it fires on
react-query's own `const { data: detail }`), and that hole is written into the test rather than
papered over with two innocent files.
Its anti-vacuity guard also did not guard anything: the comment says the glob must have matched
something, but the assertion under it read a sibling file by its own URL, which resolves whether or
not the glob enumerated anything. That only mattered while the pinned list was non-empty — the moment
the deferral sweep lands and it becomes [], the whole file would pass over a glob matching zero
files. The scan size is asserted now.
|
||
|
peter
|
71d7646bf1 |
fix(plex): a name this session cannot write fails at once, and the layout has one spelling
Round 6 turned `artifact_may_arrive` into a positive rule, but keyed it to the HLS vocabulary in
general rather than to what THIS session produces: the playlist regex accepted master/index/stream_*
for every session and the `seq == 0` branch accepted any variant token unconditionally. The two argv
branches are disjoint — a single-audio session is given `seg_%d.ts` + `index.m3u8` and no `%v` at
all, so `master.m3u8`, `stream_*.m3u8` and every `seg_<variant>_<n>.ts` are names it can never write,
and vice versa — so each of those was read as "an HLS name, so wait and see" and held a threadpool
worker, plus its pooled DB connection, for the full timeout. That is verbatim the round-5 outage the
route's own comment claims it prevents.
Measured through the real route against a live single-audio session, before: master.m3u8 404 after
30.0s, stream_q7z.m3u8 404 after 30.0s, seg_a0_0.ts / seg_zzzz_0.ts / seg_0_0.ts 404 after 20.1s
each. After, on the running localdev API: 404 in 44-316ms for all of them, while index.m3u8,
seg_0.ts and seg_1.ts still answer 200 — nothing is over-refused.
The session now carries the rendition names its ffmpeg was told to write, and `entry` is derived from
them instead of passed. And "one place" now means the argv BUILDS its patterns from the same
functions the parser reads back (`segment_name("%d", "%v")`, `variant_playlist_name("%v")`): the
layout was still spelled out independently in six places, with `_VIDEO_VARIANT` read by no consumer,
which is the drift that caused round 6's HIGH. A test asserts the argv's own patterns are what those
functions produce, and that every name they expand to is one the guard accepts.
The load-bearing MEASURED comment cited ffmpeg 8.1.1 — the Windows host's. The binary that writes
these files is the API container's 7.1.5; re-measured there, the conclusion holds.
|
||
|
peter
|
a0114deff1 |
fix(api,ui): a refused write says why, wherever the refusal happens
Round 5 taught `notifyActionError` to discard a plain-string `detail` in favour
of the caller's translated fallback, because `req` leaves 403/404/429 to the
caller and those routes hand-write English. Three places did not get the
message.
**The demo account lost its explanation.** `require_human` refused with a plain
string, so from that commit on the shared demo account got its caller's generic
"couldn't do that" and no hint that the demo account was the reason — and the
gate is not backed by a UI gate everywhere: the comfort shelf has no `is_demo`
check, so the pin button is live and simply fails. A structured `{code: "demo"}`
and an `errors.demo` string in both locales. The two auth routes that hand-roll
the same check were swept with it.
**`ChatThread` carried a second copy of the contract.** It filtered 404/429
itself and then did `e.detail || t(…)`, the pre-fix order, so the central fix
could not reach it and "That member hasn't set up messaging yet." stayed English
under a Hungarian UI. It goes through the shared helper now, and a contract test
pins the remaining raw-`detail` consumers as an explicit SET — never a count —
so a new copy fails the gate instead of shipping. The five on that list are
inline surfaces (a form field's error line, a setup banner) and are deferred
with the ~60 backend routes whose details they print.
**The Plex "Add to playlist" dialog was still silent.** Same shape as the
collection editor round 5 fixed and the YouTube twin that already had the fix:
no catch, so a playlist deleted on another device answered a structured 404 that
nothing reported — the tick never flipped, the gone row stayed, and the floating
promise became an unhandled rejection.
Also: the `ABORTED` sentinel's VALUE was asserted nowhere. The consumer test
hand-built the error it expected, so replacing `xhr.onabort`'s rejection with a
friendlier message left the lane green — and every upload the user cancels would
then raise a red "the upload failed" toast, which is what the sentinel prevents.
Both producer tests assert through `isAborted` now.
And two backend tests were pinning fewer sites than their own docstrings claimed:
the channel-404 loop said "all five" and issued three requests (explore and
subscribe unexercised, the excuse for subscribe naming a different call site
with a different reason slug), and the recipient/job test named three refusals
and requested two — `share_download`'s `not_found("recipient")` had no test in
the suite at all, nor did the locale keys added beside it. Every site named now
has a request behind it.
6 tests, mutation-checked against 7 mutations: the demo detail at both the gate
and an auth route, a re-hand-rolled ChatThread, the silent Plex dialog, the
abort sentinel, and each unpinned 404 — all red on the test that names them.
|
||
|
peter
|
7f17b17c13 |
fix(plex): an HLS session's file names are one vocabulary, and a name it cannot produce fails at once
Round 5 added a guard so a request for a segment beyond the produced region answers instead of polling out the full timeout — the route is a SYNC one, so every such wait parks a threadpool worker and, through `current_user`, a pooled DB connection. Measured against real ffmpeg, the guard could not fire where it mattered. `-hls_segment_filename seg_%v_%d.ts` substitutes the VARIANT first: a dual-audio session writes seg_0_0.ts (video) and seg_a0_0.ts / seg_a1_0.ts (the `name:aN` renditions). The regex read those as sequence-first, so every video segment parsed as segment 0 and every audio segment did not parse at all — the guard was inert on every multi-audio item, which is most of this library. `_seg()` carried the same inversion: the warm-up waited 8 s for `seg_1_0.ts`, a name ffmpeg never writes, on every multi-audio session start AND every seek-restart. The layout is now one constant plus one parser (`segment_name` / `_SEG_NAME_RE`), with a test pinning them to each other, instead of three comments that could drift — and they had drifted, all three ways. The guard is also a positive rule now. It was a blocklist, so anything that was not a single-audio segment name was waved through: `GET …/anything.m3u8` still parked a worker for 30 s, while the route's comment claimed a flood of hopeless names was what it prevented. A session produces its playlists and its segments and nothing else. `.vtt` leaves `_HLS_CT` with it — ffmpeg runs with `-sn` and subtitles are a separate route, so that extension could only ever name a file that will not arrive. Also here: - A 5000-digit segment index reached the client as a 500: `int()` on an unbounded digit run raises past 4300 digits and only OSError was caught. The name is bounded on the route and the index in the regex — the same bug class the subtitle routes were bounded for. - A session torn down between the wait and the read is a 404, not a 500. `_kill` rmtrees the directory and no lock spans those two steps; the player's recovery is keyed on the STATUS, so a 500 skipped the reclaim and ended playback with "couldn't play" instead of restarting at the playhead. The trigger is ordinary — `loadSession` awaits the new session before destroying the old hls instance. - Two comments claimed an open player checkpoints every 10 s "whether or not it is playing", which is the stated reason the 30 s finished-idle reap is safe. `flush` sends nothing once the finish latch is up, or before a position is known. The states it does cover — playing or paused mid-item — are the ones the rule needs; the comments now say that instead of more. - The route's tests leave the DB lane: the route reads nothing from the database since the auto-start was removed, and there those four regressions SKIP on a machine with no Postgres while pytest prints green. 14 tests, mutation-checked: the old parse, the old warm-up name, the blocklist, both bounds off the absurd name, the unguarded read, and the route dropping `proc` — each red on the test that names it. |