FastAPI reads a declared body IN FULL before the first dependency, so
`POST /auth/password-login` drained an arbitrary upload from an anonymous
caller and only then answered 401. 70 of the 72 body-declaring routes
behaved the same way: one unauthenticated client could spend the
process's memory and time at will.
A raw ASGI middleware, not `@app.middleware("http")` — BaseHTTPMiddleware
cannot wrap `receive`, so it could only trust `Content-Length`, which a
client may omit (chunked) or understate. This refuses an oversized
declaration outright and counts the bytes of everything else as they
arrive, the same rule `capped_body` already applies per-route. Registered
directly inside the access log: a refusal is still logged with its
request id, but the session, CORS, the setup gate and every auth
dependency never run for a body that is already too large.
Two routes opt out, under one stated rule: an exemption is ONLY for a
route whose own cap is larger than the global one. That is why
`/api/diagnostics/events` is absent — it caps itself at 512 KiB, below
the ceiling, so exempting it could only widen what it accepts. The list
names route templates and a test resolves each against the live route
table, so a rename fails the gate instead of silently dropping an
exemption in prod.
The 413 is byte-identical to `capped_body`'s and the test compares the
two against each other rather than against a literal — two shapes for one
fact is how the `cancelled`/`canceled` bug happened.
The composer's preview did not fail a call — it killed the Gitea process, and
with it the git server and the container registry. `POST /api/v1/markdown` with
`{"Mode": "gfm", "Context": "<repo>"}` makes Gitea resolve every hex-shaped word
as a commit link, and on the API's render path the commit checker it asks is nil:
panic: runtime error: invalid memory address or nil pointer dereference
renderhelper.(*commitChecker).IsCommitIDExisting(0x0)
<- markup.hashCurrentPatternProcessor (html_commit.go:217)
Every report carries hex — a tab id, a request id — so this was not an edge case
but the only case: the preview answered "the tracker isn't answering" every time,
because the tracker had just died.
Measured on throwaway instances, not argued. Same body, same call:
gfm + Context + "5645b1c0ae01" -> 1.26.4 CRASH, 1.27.1 CRASH, 1.27.3 200
gfm + Context, no hex -> 200
gfm, no Context -> 200, and the HTML keeps its tables, task
lists and chroma-highlighted code blocks
Upstream fixed it in 1.27.2 ("fix: avoid markup render panic", #38698), but a
self-hoster runs whatever version they run, so the payload is what gets fixed.
Sending was never affected: the created issue's own page renders the same body
fine (measured) — a real repo context has a real commit checker. The only loss is
that a hash or a `#123` is not yet a link in the preview; the issue page links it.
The test pins the payload dict, because the defect is in what we send.
Round 4 was targeted at the surfaces rounds 2 and 3 changed. 29 findings raised,
26 survived deduplication, 12 survived three adversarial readings each.
`set_delivery` renewed the claim but never checked it was still ours, while its
sibling `release_send` always did. `expire_on_commit` is off, so the token it
re-stamped came from the caller's own memory, not the row: a send that outlived
its 600 s lease came back, overwrote the take-over send's committed links with a
full-column write, and put its own dead token back. It now takes the token, reads
the row's claim with a targeted scalar (not a refresh, which would discard the
`issue_url` assigned one line earlier) and raises `SendClaimLost` rather than
writing. An issue created by a send that had already lost its claim is logged with
its number — that record is the only one that will exist.
The tracker's settings are read above the pre-call checkpoint instead of inside
`_client`. Reading them there was a SELECT one line after the commit that was
meant to end the transaction, and held it — idle, snapshot pinned — across the
repository check and the create. The comment and the module docstring said
otherwise; measuring it in pg_stat_activity is what settled it. Two more comments
were saying things the code had stopped doing: the `**state` spread does not carry
the claim (`set_delivery` strips and re-stamps it), and react-query does not
debounce the preview — the editor being unmounted is what paces it.
The composer stopped stranding the focus on `<body>`: a send that creates the
issue and then fails on an attachment unmounts the button that was just pressed,
and only the success path handed the focus on. Export no longer saves into a
refusal it does not need — the zip is the way out of a stuck send, and its
pre-save was the one step that could not survive one. The delivery banner
announces through a role instead of an `aria-live` created with its own message.
`historyState` returns a union, so `chipTone` matching on the same keys is checked
by tsc rather than by hope.
PlexPlayer was the only full-screen overlay in the tree with no focus containment
at all — Tab walked out of an opaque player into a page nobody can see. It now
carries the shared trap. Closing that hole exposed the one underneath it:
`useDismiss` registered as a layer that traps the Tab while installing no trap,
so the player stood its own down for its own menus. Measured in a real browser:
30 Tabs with the settings panel open, 9 escapes into the nav rail, and 0 after.
Escape ordering is unchanged — the panel first, then the player.
Three tests that did not hold their line: the advisory lock's call site (deleting
it left the suite green), the trail assertion whose `all()` was vacuous over an
empty filter, and a `layerStack` assertion the stand-in dialog already forced.
All eleven executable fixes mutation-checked red for the right reason; the two
comment corrections have nothing to pin.
Round 2 made the step prompt a blocking layer so that `j` and the `g` chords
would not reach the page behind it. The dialog it sits in gave up its Tab trap
because of that, while the prompt's own replacement trap covered only its own
two controls — so Tab from anywhere else in the dialog (the bar's Screenshot or
Stop, the dialog's own buttons) walked out behind the scrim. `layerStack` now
separates the two questions it had folded into one flag: a layer can own the
keyboard and still leave the Tab trap to the dialog below it. The mirror case,
a dialog opening over the prompt, falls out of the same split.
A send's claim is now a lease with an owner. Every progress report re-stamps
it, so a send slower than the lease cannot lose it while it is still running,
and only the send that took a claim can give it back. The issue_create audit
row is staged inside the send and rides the same checkpoint as `issue_url`, so
the trail cannot be missing for an issue that exists. The attachment-name
prefix has one definition, shared by the producer and both readers.
A running send is reported to the UI. A reload or a second tab used to render
it exactly like a send that STOPPED, and offered a retry the server was certain
to refuse; the composer now says the report is being sent, holds the draft, and
puts both the retry and Create issue out. The past-sessions list says the same
for the same reason — it is the list a user opens to check on a send, and both
"draft" and "half sent" describe one that died. `/preview` and the retry path
got a pace of their own, and the per-user byte cap took the user-scoped lock
that makes it a cap instead of a guess.
Verified in a real browser on localdev: Tab cycles the whole dialog with the
prompt open, the prompt still swallows the page's keys, and the issue link is
focusable where the composer puts it. All 24 regression tests mutation-checked.
Creating the issue rebuilt the delivery state from scratch and dropped the
claim with it, so for a new issue the claim covered only the creation — a
draft, a screenshot or a second send could change the report under the uploads
and the body edit that follow, which is exactly what the claim is for. The
state is now built on what was read, and a test walks the send's checkpoints
to see the claim still held at each one.
Found by mutation-checking the round-2 tests: removing the claim's release
left them green, because for a new issue there was nothing left to release.
The send no longer holds the session's row lock while it talks to the tracker:
it claims the session, gives the lock back, and checkpoints each step. A Stop
is never held behind a send, while a draft, a screenshot or a second send are
refused at once instead of queueing. The claim ages out, so a request that
died mid-send does not strand the session.
What the composer says is what happened: an attachment that went is no longer
named as one that did not (the missing body edit is its own line), and the
issue's screenshot count comes from the server when the list cannot be read,
so a failed read no longer blocks saving the draft. The zip is fetched as a
request, so its pace is a message and not a saved error page. Screenshots of a
finished report can be removed again; only one still being sent is held.
A recording's log elevation can no longer outlive it. The step prompt keeps
the keyboard (the page's keys stay quiet) and traps Tab itself. An image
pasted into a dialog opened over the composer stays that dialog's.
The send, the draft and the screenshots take the session's row lock, so an
autosave or a paste during a send waits for it and then sees it, and two sends
cannot open two issues. A save keeps the delivery's progress; a failed edit
that shows the screenshots is finished by the retry; an answer that is not the
tracker's API (a redirect, a proxy page) is refused, not a crash that loses the
issue's record. Only an attempt that opened an issue counts toward the pace.
Screenshots: deleting one after the send is refused, a user's storage is
capped across sessions, new sessions and zip exports are paced, images are
stored uncompressed in the zip. One liveness rule, one session view, one image
type table.
Logging: a session's elevation is tracked apart from the user's own, so the
browser's own requests (no header) no longer lower the floor a session needs;
Stop ends it at once.
Composer: an image pasted anywhere is taken; what is still unsaved is saved on
close, and "saved" is shown only when it is; the draft is held while it is
sent; a failed screenshot read says so instead of reading as none; focus moves
to what stays; the preview follows the text size. Pill: the step prompt and a
running Stop survive the pill moving between hosts, the prompt keeps the
dialog's Tab trap, the toast region lets clicks through, other tabs hear of a
Start or Stop, the last seconds are sent before the cap, a capture with no
frame is an error.
- admin config group "Bug reports": Gitea address, repository, token (secret),
and "Check the connection" — the repository must answer, take issues and
be PRIVATE (a report carries its reporter's log); checked on every send too
- POST /api/debug-sessions/{id}/issue sends the SAVED draft: create the issue,
upload each file on its own request (the measured API), then show the
screenshots inline; progress is kept in the draft, so a send that stops
half-way keeps its issue and the retry sends only what is missing; 5 new
issues an hour per user; audited as issue_create
- the composer: Create issue (only where a tracker is set up, naming the
repository), a partial-send banner with retry, and a Preview — the tracker's
own rendering in a sandboxed frame, or the zip's markdown without one
- the recorder's refusals answer in the localized family (debug_session_*,
screenshot_*, issue(s)_*), not bare codes the UI could only call generic
- a tracker refusal logs the status only: its message can quote the title back
After Stop (or at the cap, or from Settings → Diagnosztika's past sessions)
one dialog drafts the issue: title, description, the steps drafted from the
session's markers and page changes (a marker without a label is named by its
time), the environment (version, browser, system, window, language). Beside
it, what goes with it: the browser and server logs — each reviewable line by
line, an unticked line is not sent — and the screenshots, removable, with
paste / drop / pick to add more.
- it saves as it goes (debug_sessions.draft, migration 0090): the rendered
markdown in the user's language, the excluded rows, and its own fields to
reopen as it was; a sent session is read-only
- the zip export (issue.md, client.log, server.log, screenshots) is built
from the saved draft by app/issues/bundle.py — the one definition the Gitea
delivery (S4) will share; .log, not .jsonl, which Gitea refuses
- the log line's level tags and clock move to components/logLineParts so the
review list draws a level the way both log viewers do
- the start dialog is the consent: level (Részletes/Teljes), what is recorded,
retention and the cap (now served as diagnostics.session_minutes)
- the pill docks under the toasts: elapsed time, Step (an optional label,
its own Escape layer), Screenshot (tab capture where the browser has it),
Stop; compact on a phone
- while recording, lib/log tags every event and every request path (req,
beacon, uploadReq via one identityHeaders) names the session; applyDiagnostics
raises the level for the session's run and never lowers a raised one, and
lapses at whichever ends first
- Stop sends what is waiting before the server closes the session; the cap
says so and opens the composer
- Settings → Diagnosztika and the command palette start/step/stop it
- the evidence leaves the log upload's own request lines out
- the step shortcut is deliberately palette-only: the keymap's D1 rule bars a
new modifier chord, and a bare letter collides with the players'
A session is one recording, capped at 60 minutes by the server. While it is
open, a request naming the caller's own session in X-Debug-Session is logged
at the session's level with every line tagged; a header naming anything else
is ignored. The browser's batches count a session as consent to its level and
keep the tag only on its own events. Screenshots are judged by their bytes,
capped in size and count, and served back to their owner only.
- debug_sessions / debug_screenshots tables, one open session per user
(partial unique index), log_events.debug_session_id indexed
- /api/debug-sessions: start (joins a running one), stop, list, get, events,
screenshots; /api/me reports the live session
- the hourly log job closes expired sessions and drops them with the logs
- logsink.serialize and routes._common.capped_body shared by their readers
Round 2 of the deep review, on round 1's own fixes:
- `ORDER BY position … FOR UPDATE` sorts before it locks, so the reorder that waited got the
fresh rows in its STALE order and dealt the pins it left out (a tab with Plex off) back to their
old slots. It locks in id order — one global order, so two reorders cannot deadlock — and sorts
the fresh rows itself. The race test now sends that partial list and fails unless tab B really
blocked.
- The request check saw only writes the handler flushed: a `db.delete` left staged, or a flush
followed by a released savepoint, passed. Staged changes count at the end of the request, and
only the ROOT transaction's end clears it (a savepoint test drives the dependency directly).
- The `ok` test exempted every call as a serializer, hiding four domain REPORTS (YouTube sync,
revert, push; Plex sync) without `ok`. They are spread into `{"ok": True, …}`; the exemption is a
named list of real serializers.
Deep review of the debt burn:
- `refusal_reason` told the user to reconnect on ANY token-endpoint failure — a 5xx, a 429, a
misconfigured client — while the grant stayed and the account still read as connected, with no
connect button. Only `invalid_grant` (the one that drops the grant) means reconnect now.
- `pins.reorder` writes only the rows that change, so two overlapping reorders each skipped rows
that matched in their own stale snapshot and interleaved into two pins on one slot. It locks the
user's pins first; the second waits and deals from the first's result.
- Adding a whitelist entry committed the invite and the activation it implies separately, and its
audit row was written before a new invite had an id (target NULL). One transaction, flushed for
the id.
- The mutation convention test skipped async routes and dicts returned by name: request-access,
purge-discovery and the subscription sync answered without `ok`. They do now, and the test
covers both. The helper-commit check looked only at `_`-names; it reads every non-route
function, which listed two deliberate committers (upsert_pending_invite, ensure_welcome) — now
allowlisted with their reasons.
Two rows of the canonical-patterns table had drifted, found while closing the roadmap audit:
- 63 mutation responses carried no `ok` — `{"unsubscribed": true}`, `{"status": "ok"}`,
`{"added": 3, …}` and so on. Each now leads with `"ok": true` (additive: every field a client
reads is still there).
- Twelve route helpers committed on their own. Six were just the end of their handler's work and now
flush: the playlist item add/remove and Watch later get-or-create, the Plex series/season bulk
mark, and the invite decision + account activation — which an approval used to land in separate
transactions. The other six commit on purpose (a token durable before its link is mailed, a
single-use token burned as its own unit, the Google callback's tail, a background registration
with its own session, the staged search ingest around YouTube calls), each listed with its reason.
`tests/test_route_conventions.py` holds both rules in code; CLAUDE.md's two rows point at it.
- A13: with Plex off, the Home shelf and the hero's comfort tier go YouTube-only like the rails
beside them (owner's call: hide the tiles). The pins are kept; GET /pins still lists them.
- Reorder deals the listed pins into the slots they already hold, so a partial list (hidden Plex
pins, a stale tab) no longer puts two pins on one position; repeats are ignored.
- A21: a comfort hero carries `pinned_from`, the shelf entry it was resolved from; its menu acts on
that pin, so a pinned show's next episode offers "Unpin" instead of creating a second pin.
- B12: the palette's "Meglepetés" asks for the surprise tier (the next press leading with it); the
rotation carries on from there.
- B13: the surprise token names the history entry it was requested for (minted by setPage's new
`then`, after the push) and is dropped on any other entry, so a navigation that never mounted
Home cannot replay it later.
~150 HTTPExceptions carried a hand-written English sentence as their detail,
shown verbatim under the Hungarian UI — in the blocking error dialog and on
every inline surface that printed HttpError.detail (sign-in, setup wizard,
change password, the passphrase gate, live search). Four also quoted an
upstream exception's text back to the user.
- app/refusal.py: `refused(status, reason, **params)` → {code: "refused",
reason, …params}; localizeDetail translates errors.refused.<reason> with the
params interpolated. "Unknown X" 404s use the existing not_found(what).
- sysconfig.set_value raises InvalidSetting (reason + bounds) instead of an
English ValueError; the three writers answer it through setting_refused.
- The Plex connection/write/image failures log what the upstream said and
answer a reason; the search scraper's failure is logged and answered as
youtube/unavailable (it crashed refusal_reason before).
- HttpError.detail now only ever holds a translated reason: a string detail
(only the framework's own can remain) is left out, so reading .detail is
safe everywhere — including destructured, the hole the raw-detail source
scan could not see. That scan and its five-file exception list are gone.
- Enforced: test_http_details.py fails on any HTTPException with a sentence
for a detail; the contract test requires every refused reason and every
not_found "what" in both languages, and no key for one that is gone.
- local/jsx-comment-text: a // inside JSX children renders as text; the rule
is ten lines in eslint.config.js rather than eslint-plugin-react's ~1800
lockfile lines.
HLS sessions were keyed by the item, so a second account opening the same
film killed the first one's session; the displaced player reclaimed at its
playhead and evicted the other in turn. The registry is now keyed by
(viewer, rating_key), segment requests resolve by both, and the session
directory carries the viewer (the orphan sweep still recognises the older
shapes).
Only transcodes counted toward a cap, so remuxes — each a whole segmented
copy of the film on the scratch volume — were bounded by the library size.
Three bounds now apply after every start, each dropping the least recently
used: two sessions per account, the transcode cap, and a new admin knob,
plex_max_sessions (default 8), over every session. A superseded session
request returns at once, since it waits on the killed ffmpeg, so an account
cannot park more request workers than it may hold sessions.
The subtitle extractor's full-pass slots were global and minutes long each;
one account opening subtitles on film after film could hold them all. An
account may now hold three of the six, and still gets windows past that.
The house guard requires a silent handler to carry a silent-ok note; the
savepoint fallback added in e74986cc did not, and the gate's pytest lane
caught it.
A full pass that ran out of time kept what it reached, but its retry an hour
later started again at 0, so a file too big to read inside the extraction
timeout never completed. The pass now starts where every embedded track's
coverage from the head ends (less the window pre-roll, so a cue across the
join is read whole). The seek needs -avoid_negative_ts on this read as on a
window, measured on a real file: without it the cue running at the start
shifts the whole output.
A partial track whose full pass had failed never reached a settled state, so
the player polled it every 2 s for the rest of the film. The status now says
whether the track is still growing, and a stalled one is polled every 15 s —
still inside the 60 s lookahead that reads the next window before an edge.
The comfort shelf resolved each pinned channel with three queries (the
row, its unwatched count, its newest video) and each playlist with two.
channels_summary gets the count and the newest row per channel as window
functions over one filtered set, playlists_summary reads every pinned list
in one query, and the rows come in one IN() each — so the shelf's cost no
longer grows with its length. A test counts the statements.
RAIL_LIMIT had two definitions held in step by a comment; the Plex Discover
route now imports the domain's. And rollup's "continue the latest-started
episode" was false: it takes the last in-progress one in broadcast order,
which is also what Home's next_episode picks.
move_item checked "already in the target?" and then re-parented without a
lock, so a second tab's add or the scheduler's YouTube merge committing the
same video in between hit uq_playlist_video and answered a raw 500. The
re-parent now runs in a savepoint and a conflict falls back to the
already-present outcome. The target is touched only when the row landed in
it (an untouched list was promoted in the rail's "recently changed" sort),
and _touch stamps the database clock like the mixin's own onupdate.
The canonical-patterns table names owned_or_404 against ad-hoc ownership
helpers; six were left (_own_playlist, _own_playlist_or_404, _own_view,
_own_tag, notifications._owned, downloads._own_job), two of them
hand-rolled with a bare English 404. All call sites now use owned_or_404,
and the download/link/notification 404s are structured like the rest. The
contract test reads every "what is gone" the routes name, and the YouTube
refusal reasons, and requires both languages to name each.
Two near-end checkpoints arrive together routinely (the 10 s tick and the
debounced seeked flush); both read "not watched" before writing and both
scrobbled, bumping Plex's viewCount twice. The transition is now decided by
the write: an insert that did not conflict, or an update conditional on the
old status.
The finish also popped the push-ordering entry that _resume_push_superseded
reads, so an older resume push still in flight could land at Plex after the
scrobble and revive a resume point on a watched item. It now stamps the
entry instead; the existing eviction bounds the map.
The two manual syncs (subscriptions import, playlist pull) had no YouTubeError
handler, so a quota, token or transport failure was a raw 500 and the blocking
"Server error" dialog (E9 F1). The other YouTube routes answered either the
upstream's English or a playlist-scoped "youtube_failed".
One helper, routes/_common.youtube_refusal: 422 {code: "youtube", reason} with
the reason sorted by what the user can do — quota (wait for the reset),
reconnect (the token step refused), unavailable (retry). Used by every route
that surfaces a YouTube failure; localizeDetail translates it.
"Reconnect" had no path: a grant Google revoked still read as granted, so
Settings and the wizard offered no connect button while every call failed.
An invalid_grant from the token endpoint now clears the stored grant.
A failed connection or token step answered nothing about the channel, yet stamped it: one proxy blip
during an RSS blackout delayed every due channel by a whole window. YouTubeError now names the
endpoint that answered; a failure no endpoint gave, or a spent quota, ends the round and stamps
nothing.
A channel whose uploads playlist also failed was never stamped, so a terminated channel was charged
again on every 20-minute poll instead of once per rss_fallback_hours. An upstream quotaExceeded now
stops the round instead of charging every remaining channel the same refusal.
An INFO line outside a request never reaches log_events, so the fallback's
own summary was invisible in the Server log. Its counts ride on the
sync.rss.failed warning, the one line of a poll the console keeps.
Measured on localdev: 375 of 375 dark channels read, 403 new videos.
The server now logs polled reads at DEBUG, and the browser's copy of each
request line promises the server's level: at INFO the unread-count polls
filled My log within minutes and, at Detailed, the shipped log. The server
marks a line it logged at DEBUG with X-Log-Quiet, so the polled-route list
stays in one place.
Settings → Account names the latest sign-in on any device (only the
admin's user list did, though the 0.88 notes said otherwise), and the role
in the reader's language instead of the raw `user`.
The reads the frontend polls (unread counts, /api/me, the download index,
sync status, subtitle progress) are logged at DEBUG: at INFO they were ~95%
of the stored log on prod. A scheduler job failure and a playlist sync
failure carry event codes, and a Google account without a YouTube channel
has an empty playlist list instead of a warning every six hours.
YouTube's RSS goes dark per channel for hours (prod 2026-09-18/19: 393 of
396 feeds answered 404 for four hours, sticky per channel, so a retry does
not help), and nothing else read a followed channel's new uploads. A
channel whose feed has not answered for `rss_fallback_hours` (admin knob,
default 2, 0 = off) now gets one page of its uploads playlist through the
Data API (1 unit), at most once per window, yielding to the backfill
reserve. Spend is attributed to its own quota action.
- Live stops polling on a failed read and marks the list as possibly
stale (with Retry), instead of re-raising the error dialog every tick.
- While live, a refetch reads the newest page only, whatever the cache
holds (a late "Load older", a revisited filter set).
- Copy is disabled and the list dimmed while a filter change loads.
- Dates follow the UI language; the pivot chip uses the shared .chip.
- Tests: query-string wire test, Copy, every facet's skip key.
A request that hung for the full timeout says the way out is hanging, not
that the endpoint is flaky; three of them per channel stretched a poll over
a dead egress to about three times its interval.
YouTube's feed endpoint answers the same channel with 404, 500 and 200 in
turn: on prod 182, then 296 of 396 channels failed a poll, and a probe of
24 channels saw 15 fail a first request while 16 had succeeded by the
fifth. A channel that failed once waited for the next poll, 20 minutes
later, so new uploads showed up late.
A feed is now tried up to three times (after 1 s and 2 s) on a transport
error, a 404, a 429 or a 5xx; a refusal such as 403 is not retried. The
last failure is what gets counted. A feed that is really gone costs two
more requests per poll.
Admin → Diagnostics reads the server's stored log: one list, newest first,
where a line opens in place to its full record (time, source and logger,
account, request and job ids, version, message, attributes, stack). A
request or job id narrows the list to that request or job with every other
filter off, shown as a removable chip. Filters: level group, source,
account, event, period and a free-text search; all of them live in the URL
(`l*` keys, which setPage clears like the Plex and Channels keys), so a
filtered view is a link and Back restores it. "Live" re-reads the newest
page every few seconds while the tab is visible; paging into older rows
turns it off. Copy puts the loaded lines on the clipboard as JSON lines.
The facet counts now honour every filter but their own, as the feed's do:
a request pivot narrows them, and a picked level does not zero its
siblings. A value in use stays offered even when the counts drop it.
"Saját napló" and the console draw a line with one shared component,
`LogLine`, whose phone layout gives the message its own full-width line
(sharing the code's line left it a few characters). The level groups
moved to lib/log.ts for both.
The console's own reads are logged at DEBUG on success, like the
healthcheck: at INFO its live tail filled the log with itself.
GET /api/admin/logs pages `log_events` newest first on a (ts, id) keyset:
a browser row's ts is clamped into its consent window, so ts alone is not
unique. Filters AND together: level, source, account (or `none`), event
code (or `none`), request id, job id, a free-text search over the
message, code, request id, stack and attributes (typed text is a literal),
and a time window. Each page names the accounts it mentions.
GET /api/admin/logs/facets counts what the filter chips offer over the
window alone, so a chip row does not shrink as filters are picked.
Admin-only and read-only; reading is not audited (the console's live tail
polls). There is deliberately no "ids after N" tail: several processes
write this table and a batch takes its ids at insert, not at commit, so a
tail keyed on ids could pass a row that commits late.
The feed's channel-name match and the live search's term fold built their
LIKE pattern from the typed text as it was, so `_` and `%` acted as
wildcards: `build_log` also matched `buildXlog`. The term fold also tested
a substring, so a term already present inside a longer word (`cat` in
`category`) was never folded into search_terms. It now compares whole
terms, escaped.
`like_escape` moves from the Plex query module to `app.utils`, beside
`to_tsquery_str`, and every pattern built from typed text goes through it.
- The download and edit failure lines go back after their DB write: a job the
write could not fail is not a failed job, and the worker's child_exit line
already says it failed.
- The download child's runner is the last stop before Python's hook: an
escaping exception is one download.child.failed line with its chain scrubbed,
then exit 1, instead of a raw stderr traceback carrying the URL.
- A crash the database will not record carries its stack once, and the
fail-write line has an event code.
- The job's own generic-extractor name is taken out as a literal (formats.hiding),
so brackets and colons in a direct link's file name no longer leak; the
pattern stays as the fallback. `Downloading just the video <id>` is covered.
- Tests: scrubbed_stack's cause, suppress, unprintable and text-format paths,
the image steps' failure kinds, the watchdog error through the real
_fail_running, exact poster/storyboard messages, ffmpeg skip marker.
- A failure keeps its traceback unless it is yt-dlp's or our own StatedFailure (a
message that is the whole story). A plain RuntimeError, RecursionError or OSError
is also what a bug of ours raises, and its stack is the only lead.
- The stack is scrubbed link by link (logs.scrubbed_stack): a chained DownloadError
under a DB error no longer prints yt-dlp's text unscrubbed.
- Engine hide_parameters: a DB error's message no longer carries the title or path
it was writing.
- The download and edit failure lines go out even when their DB write fails.
- The worker's child_exit/crash/supervisor lines say download.job.failed only when
the job failed there; a crash the DB will not record no longer ends the thread.
Stopping a paused child is download.child.stopped, not a second abort.
- log_safe also covers generic ids with spaces, playlist titles and bare ids.
- The ffmpeg image steps log the failure kind, not the argv with its path.
- The consent contract accepts only the body of a plain consented_trace() check,
and any level that may be TRACE.
- Tests for each; every one mutation-checked.
- yt-dlp's text is scrubbed wherever it is logged, not only in the adapter:
`formats.log_safe` takes out the URL, the `[extractor] <id>:` prefix and
file paths, and the job-failed, refused, edit-failed and backfill lines use
it. Done lines log the asset id, not the title-built path. The PII contract
now bans `rel`/`rel_path`/`title`.
- A traceback goes with a download/edit failure only when it is our own code
failing; for yt-dlp's DownloadError, our RuntimeErrors and OSErrors its foot
is the unscrubbed message.
- yt-dlp's `error` maps to WARNING: it fires on attempts the fallback client
set recovers from; the job's verdict is `download.job.failed`.
- The sink groups a bare `%s` pass-through by its text (numbers folded), and
a process's last batch closes the repeat window, so a short-lived download
child still reports what it dropped.
- Worker-side failures (stall, wall-clock, no asset, cached asset error) log
`download.job.failed` with a cause; they left no line at all before. The
crash, supervisor-error and non-zero-exit lines carry the code too.
- An edit failure is one coded line in `_fail_edit`, for every path.
- `logs.get_logger("__main__")` resolves to the `-m` module, so the worker
and the download child keep their names.
- TRACE lines carrying user content are gated on `logs.consented_trace()`,
not on the process level; a contract test pins it.
- The OAuth token refresh goes through `_upstream_failure`; transport errors
carry reason `transport`. Subscribe recognises a duplicate by reason code
only (the prose fallback was dead).
- Once-only guards for a stale sysconfig override and a refusing HLS session
directory; item_detail splits Plex failures from our own parse bugs.
- The playlist breaker line names the playlist and the action; comments that
were false (runner noqa, `_merge_vtt`, sink grouping) are corrected.
The gap this sprint was scoped against turned out to be smaller than the survey
said, and in a way worth writing down: `log_requests` already records every HTTP
call with its route template, status, duration, request id and user id, and the
audit log already records every admin act on someone else's account. Putting a
line in each of routes/downloads.py's twenty mutating endpoints would have added
a second copy of the first and nothing else. So this instruments what neither of
them can see.
yt-dlp now explains itself to our log. `quiet: True` was doing more than quieting:
with no `logger` its whole account of a download went to the child's stdout, so
"Requested format is not available", a signature-extraction failure or a POT
refusal reached us only as whatever text ended up in the raised exception — the
real reason a download failed was routinely one line the app never kept. Measured
against the pinned yt-dlp 2026.07.04 before wiring it: with `logger` set it calls
`debug` for extraction and progress lines and `warning`/`error` for the rest, and
neither `quiet` nor `no_warnings` gates those calls. `debug` lands at TRACE, being
several lines a second, so the noisy half is the half the user's own consent
gates. The full source URL is trimmed to its host on the way in — it is a row of
someone's download history, and `logs.set_job` already puts the job id on every
one of these lines. The option is set AFTER the adapter merge, like `cookiefile`
and the size cap: a source adapter must not be able to take away the one account
of what went wrong that a failed job leaves behind.
An HLS session now says which pipeline it got (transcode/remux, hw, tonemap,
variants, offset) — everything downstream is about that choice, and it was only
inferable from an ffmpeg command line nobody keeps. A session culled by the
concurrent-transcode cap says so: somebody's playback stops there, the route turns
it into a retryable 404, and "my film keeps rebuffering" had no other explanation
anywhere. The HW→SW fallback keeps its line and gains an event code.
A live search refused by a quota cap says WHICH cap: the 429 is in the request
line either way, but the per-user daily cap is answered by tomorrow and the shared
budget by the admin.
The download and edit job lifecycle lines gain event codes, which is what E22.6's
console filters on — the job id was already a structured field.
tests/test_ytdlp_logging.py; three of its four cases mutation-checked. A fourth
(a `%` in a progress line) survived its mutation — stdlib only applies `%`
formatting to a record that has args — so it is gone rather than kept as a case
nothing can break; the reasoning moved into the comment on `_emit`.
114 exception handlers neither logged nor re-raised. The defect was never that
they are silent — plenty of them should be — it is that their silence said
nothing, so nobody could tell a decision from an oversight. Each one has now been
read. 44 got a line; the other 70 carry `# silent-ok: <why>`.
What the silent ones had been hiding, in the cases that got a line:
- A failed ffprobe sent every HDR file down the 8-bit pipeline, so playback
degraded to a software transcode with nothing saying why; a failed audio probe
collapsed a multi-audio file to one track and the player showed no language
picker; a failed first-PTS probe left subtitles seconds out of sync.
- A subtitle sidecar that existed but would not parse made a PARTIAL track read as
complete, permanently — the viewer loses every cue past the cached window.
- An unreadable crop/trim/cut-list in an edit spec was dropped, so the clip came
out uncropped, untrimmed, or shorter than the one the user built.
- A dropped `max_height` downloaded at best quality against an explicit cap.
- A failed videos.list enrich made every candidate undroppable-from-judgement and
therefore dropped: a working search rendered as "no results".
- An unusable password hash answered as "wrong password", locking the owner out
and telling them they mistyped; a rotated TOKEN_ENCRYPTION_KEY turned every
stored token into "not connected".
- A stored admin setting that would not coerce fell back to the env default while
the settings page went on showing the saved value.
- A one-off InnerTube config failure pinned the whole process to the hard-coded
fallback key, and searches return nothing when that goes stale.
- The HLS sweep reported 0 swept — like a clean run — when it could not list its
own root, leaving every crashed session's segments on disk forever.
Four of them are counted rather than logged per item (dropped cues, dropped cut
list segments, unreclaimed GC files) and reported once per call, because the
alternative is one line per cue over a whole subtitle track.
ruff gains S110 and BLE001. Every blind `except Exception` that remains carries
`# noqa: BLE001 — <why>`, which is the point: catching everything is sometimes
right, and always worth a sentence. plex/sync.py's three best-effort backfill
catches now keep the traceback — a blind catch over a batch that fetches AND
writes could be Plex, the database or our own bug, and `%s` of the exception told
them apart badly. main.py's boot-time log-level read likewise.
logs.event's `code` is positional-only now: `event("x", code=...)` was a TypeError,
and `code` is exactly the word a caller reaches for when the thing that failed has
one of its own.
tests/test_swallowed_exceptions.py is the wider rule ruff cannot see: a handler
that neither logs nor re-raises must say why. Both cases mutation-checked — a
marker removed and a log line removed each fail it, naming the handler.
25 literal names covered 34 modules. plex/client.py, plex/stream.py,
plex/subtitles.py, plex/sync.py and two Plex route modules all called themselves
`siftlode.plex`; downloads/pipeline.py called itself `siftlode.worker`;
routes/channels.py called itself `siftlode.api`. The `logger` column is in every
stored row so E22.6's console can filter on it, and so an operator can pin a level
to one part of the system — neither works when six modules share a name.
`logs.get_logger(__name__)` derives it: app/plex/sync.py -> siftlode.plex.sync.
One name per module, and it cannot drift from the file it sits in.
tests/test_logger_names.py pins the derivation, that everything stays under the
one `siftlode` tree the handler and the tests attach to, and that no module
hand-rolls a name again unless the line says why (`# logger-name-ok: <why>`).
Two do: get_logger's own call, and `siftlode.http`, which is named for the traffic
it carries rather than the module it lives in — it replaced uvicorn's access log
and is what an operator pins when they want request lines and nothing else.
app/scheduler.py called its logger `logger`; every other module calls it `log`.
ruff gains LOG, G, T20, TRY400, TRY401 and S112. Four findings cleared with them:
worker.py logged `str(exc)[:120]` where the traceback was the whole question, and
three prints/f-strings that are deliberate now say so on the line. S110 and BLE001
stay off until S3 — 23 handlers are outstanding and each is a judgement about one
of them; enabling the rules first would just mean 23 blanket noqas.
The E22 redactor is a safety net over addresses and credential params, and every
one of these call sites was leaning on it to do the whole job. What it does not
cover went in whole, and what it does cover came out as a constant.
Not covered, so a real leak:
- routes/search.py logged the raw search term at ERROR. The privacy policy shipped
in 0.88.0 says a search string is kept only at Teljes/TRACE, with the user's own
consent. The ERROR line now carries the shape of the failure (source, candidate
and collected counts) and the term follows on a TRACE line, which nothing emits
unless that consent is in force.
- downloads/backfill.py logged the full source URL at INFO — a row of the user's
download history. Now the asset id and the host.
- youtube/client.py logged the upstream error body, which is prose that quotes the
request back. Every failing YouTube call now goes through one _upstream_failure()
that logs the API's `reason` code, keeps the body for the same consented TRACE
line, and raises a YouTubeError carrying `reason` as a field — callers put that
error in log lines of their own (routes/channels.py) and in a 422 answered to the
user, so the body must not travel in it either.
Covered, so merely useless: routes/auth.py named the account by an address that
reached both sinks as `[email]`, and app/email.py printed recipient lists that read
as `['[email]']`. Both now carry the uid (or a count) and a reason code, which is
also the first half of E22.5 S4's failed-sign-in coverage.
routes/channels.py branched on the text of that error message to recognise an
already-existing subscription. It now branches on `exc.reason`, with the substring
check kept beside it for a body that carried no reason at all.
tests/test_log_pii.py pins the call sites and the family: a source-level contract
test walks every log call in app/ and fails on an argument naming user-typed
content unless the line says why it is safe (`# log-pii-ok: <why>`). Eight sites
carry that marker today, each with its reason. Gate 8/8 green; all five cases
mutation-checked.
Two reviewers in a row read the admin card's selection against `run_deep_backfill`'s, saw it was
missing the job's `recent_synced_at` filter, and called it a card that counts channels the job will
never touch. It is the opposite: the card is the same backlog figure /api/sync/my-status shows the
user, and a channel still missing its recent pull is one the same job tick fetches first. Narrowing
it would hide genuinely queued work. Written down where the next reader will look.
Round 2 of the E22 review, on what round 1 changed.
`redact_query` decodes a parameter before the text rules see it, which is what lets `%40` be
recognised as an address — but `_SECRET_PARAM_RE` was written for the raw, encoded form, where a
value holds no space, `&` or quote. On decoded text its value class stops at the first one, so
`?state=a%20b` kept its tail and `?code=%20abc` was not matched at all. The parameter-NAME rule now
decides, on the key, before any text rule runs.
`forget_user` dropped the account's level elevation without recomputing the root level, so a process
whose only elevated account had just deleted itself kept building TRACE records to throw away, until
some later request happened to recompute it. Every other path that drops an elevation recomputes.
The sink's writer took the erasure lock and THEN waited for a pooled connection, while every request
path already holds a connection when it reaches that lock — so under pool pressure the writer held a
process-global lock against the very requests that could hand a connection back. Same order on both
sides now; the lock still spans the check, the insert and the commit.
The storage-budget notice said the rest of the hour was not kept, and then smaller batches kept
landing underneath it: the window slides and a refused batch is not charged to it. Says what it does.
Tests: the route's half of the erasure rule had no test — only the sink's did — and a source-level
contract test now covers the third insert nobody has written yet. `logs._erased` is process-global
with an hour's memory while the DB fixture hands serial ids back out from 1, so it is swept before
each test like the rate limiters are.
The explore purge rebuilt followed_channel's two EXISTS clauses inline, so what is polled and what may be deleted could drift; it now uses the predicate, which moved to sync/explore.py. The env validator and sysconfig each folded a choice value on their own; both call config.canonical_choice. The deep-backfill comment no longer calls recent backfill unconditional.
The bind sat inside the try, so it was already reset when the crash line and the cleanup were logged — the one line most worth finding by job id had none.
Deep-review round 1 of E22.
- A NUL character (`GET /assets/%00` logs the decoded path) made Postgres
refuse the row and with it the sink's whole batch, every user's rows
included. `to_row` replaces NUL in every text column and in attrs.
- An erased account's lines kept arriving after purge_user's DELETE: the
erasing request's own access line, a queued row, a background task.
`logs.forget_user` runs first; later rows are stored without the account,
and the check and the insert share a lock with the erasure.
- An admin setting the whole process to TRACE logged every user's query
string. The query is now logged only under the user's own Teljes consent,
and decoded before redaction, so `%40` no longer hides an address.
- The browser log ingest has an hourly storage budget per account (5 MiB,
one notice when spent) and reads only a JSON Content-Type, so a
same-site text/plain form post cannot write into a user's log.
RateLimiter takes a per-attempt cost for the byte budget.
- The root level is applied under the elevation lock: two concurrent
changes could leave it at the base while an elevation was active.
- Tests for the running sink (level gate, drain on stop) and for the
admin log level applied at runtime and at boot.
A user can raise their diagnostics level to Részletes (DEBUG) or Teljes (TRACE) for 24 hours from a
new Settings tab, after a consent box that says what is recorded, that the server keeps it and for
how long (the instance's own retention), and that it lapses. Alap is the base and needs no consent.
- users.diagnostics_level/_until (migration 0087), written only by PUT /api/me/diagnostics, which
computes the expiry itself; the demo account cannot raise it. /api/me reports the level in force.
- The server logs an elevated user's requests at that level and stores every line of them
(logs.LevelGate on the stdout handler and the sink; the root is lowered only while an elevation
is active). Teljes adds the redacted query string to the request line.
- POST /api/diagnostics/events stores the browser's log as source=web rows: account from the
session, body capped while streaming, events/attrs capped, re-redacted, events under the
consented level dropped, timestamps clamped, 30 batches a minute, 409 once the level lapsed.
- lib/diagnostics.ts applies the level to the tab and streams only what happens after consent, in
batches, never its own request lines, with a keepalive flush on pagehide; stops on lapse (its
own timer), on withdrawal and on 409.
- Detail hooks at DEBUG: which control was clicked (test id, icon, app copy in English — never user
text, input values or a link's query), visibility/bfcache/resize, filter changes
(lib/filterLog.ts; typed search only at TRACE), player events (lib/mediaLog.ts; media errors are
warnings at every level). Offline/online at the base level.
- "Saját napló" viewer: live, newest first, level groups with counts, search, row details, copy
the filtered events as JSON lines, clear.
- Privacy policy and Terms exist in Hungarian and English (a contract test holds both languages to
the same sections); the policy now describes the server logs and diagnostic logging, self-service
deletion and password sign-in.
Fixed on the way:
- A sync auth dependency ran logs.set_user on a copied context, so the account never reached a
handler's own log lines (measured); a per-request mutable scope carries it now.
- The Settings tab rail left a phone no room (clipped labels, buttons over text): below `sm` it is
a scrolling row above the content.
- PUT /api/me/preferences took a bare dict body; it is a Pydantic model now.
peter