74f58ea00173dfa6acfcfa49650e437ba04411de
817
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 peter
|
8cfab498cf |
fix(sync): try a channel's feed again before counting it as failed
YouTube's feed endpoint answers the same channel with 404, 500 and 200 in turn: on prod 182, then 296 of 396 channels failed a poll, and a probe of 24 channels saw 15 fail a first request while 16 had succeeded by the fifth. A channel that failed once waited for the next poll, 20 minutes later, so new uploads showed up late. A feed is now tried up to three times (after 1 s and 2 s) on a transport error, a 404, a 429 or a 5xx; a refusal such as 403 is not retried. The last failure is what gets counted. A feed that is really gone costs two more requests per poll. |
||
|
peter
|
b5996a0e56 |
feat(diagnostics): the admin log console
Admin → Diagnostics reads the server's stored log: one list, newest first, where a line opens in place to its full record (time, source and logger, account, request and job ids, version, message, attributes, stack). A request or job id narrows the list to that request or job with every other filter off, shown as a removable chip. Filters: level group, source, account, event, period and a free-text search; all of them live in the URL (`l*` keys, which setPage clears like the Plex and Channels keys), so a filtered view is a link and Back restores it. "Live" re-reads the newest page every few seconds while the tab is visible; paging into older rows turns it off. Copy puts the loaded lines on the clipboard as JSON lines. The facet counts now honour every filter but their own, as the feed's do: a request pivot narrows them, and a picked level does not zero its siblings. A value in use stays offered even when the counts drop it. "Saját napló" and the console draw a line with one shared component, `LogLine`, whose phone layout gives the message its own full-width line (sharing the code's line left it a few characters). The level groups moved to lib/log.ts for both. The console's own reads are logged at DEBUG on success, like the healthcheck: at INFO its live tail filled the log with itself. |
||
|
peter
|
5e208353e3 |
feat(diagnostics): admin query API over the stored log
GET /api/admin/logs pages `log_events` newest first on a (ts, id) keyset: a browser row's ts is clamped into its consent window, so ts alone is not unique. Filters AND together: level, source, account (or `none`), event code (or `none`), request id, job id, a free-text search over the message, code, request id, stack and attributes (typed text is a literal), and a time window. Each page names the accounts it mentions. GET /api/admin/logs/facets counts what the filter chips offer over the window alone, so a chip row does not shrink as filters are picked. Admin-only and read-only; reading is not audited (the console's live tail polls). There is deliberately no "ids after N" tail: several processes write this table and a batch takes its ids at insert, not at commit, so a tail keyed on ids could pass a row that commits late. |
||
|
peter
|
66dbd0af81 |
fix(search): typed text is a literal in every LIKE pattern
The feed's channel-name match and the live search's term fold built their LIKE pattern from the typed text as it was, so `_` and `%` acted as wildcards: `build_log` also matched `buildXlog`. The term fold also tested a substring, so a term already present inside a longer word (`cat` in `category`) was never folded into search_terms. It now compares whole terms, escaped. `like_escape` moves from the Plex query module to `app.utils`, beside `to_tsquery_str`, and every pattern built from typed text goes through it. |
||
|
peter
|
89c77ae6b5 |
fix(logs): deep-review round 3 — a failure nobody could record is one scrubbed line
- The download and edit failure lines go back after their DB write: a job the write could not fail is not a failed job, and the worker's child_exit line already says it failed. - The download child's runner is the last stop before Python's hook: an escaping exception is one download.child.failed line with its chain scrubbed, then exit 1, instead of a raw stderr traceback carrying the URL. - A crash the database will not record carries its stack once, and the fail-write line has an event code. - The job's own generic-extractor name is taken out as a literal (formats.hiding), so brackets and colons in a direct link's file name no longer leak; the pattern stays as the fallback. `Downloading just the video <id>` is covered. - Tests: scrubbed_stack's cause, suppress, unprintable and text-format paths, the image steps' failure kinds, the watchdog error through the real _fail_running, exact poster/storyboard messages, ffmpeg skip marker. |
||
|
peter
|
0b672ed6e7 |
fix(logs): deep-review round 2 — a stack names code, not data, and survives a failed write
- A failure keeps its traceback unless it is yt-dlp's or our own StatedFailure (a message that is the whole story). A plain RuntimeError, RecursionError or OSError is also what a bug of ours raises, and its stack is the only lead. - The stack is scrubbed link by link (logs.scrubbed_stack): a chained DownloadError under a DB error no longer prints yt-dlp's text unscrubbed. - Engine hide_parameters: a DB error's message no longer carries the title or path it was writing. - The download and edit failure lines go out even when their DB write fails. - The worker's child_exit/crash/supervisor lines say download.job.failed only when the job failed there; a crash the DB will not record no longer ends the thread. Stopping a paused child is download.child.stopped, not a second abort. - log_safe also covers generic ids with spaces, playlist titles and bare ids. - The ffmpeg image steps log the failure kind, not the argv with its path. - The consent contract accepts only the body of a plain consented_trace() check, and any level that may be TRACE. - Tests for each; every one mutation-checked. |
||
|
peter
|
54d19cc386 |
fix(logs): deep-review round 1 — say each failure once, and scrub it everywhere
- yt-dlp's text is scrubbed wherever it is logged, not only in the adapter:
`formats.log_safe` takes out the URL, the `[extractor] <id>:` prefix and
file paths, and the job-failed, refused, edit-failed and backfill lines use
it. Done lines log the asset id, not the title-built path. The PII contract
now bans `rel`/`rel_path`/`title`.
- A traceback goes with a download/edit failure only when it is our own code
failing; for yt-dlp's DownloadError, our RuntimeErrors and OSErrors its foot
is the unscrubbed message.
- yt-dlp's `error` maps to WARNING: it fires on attempts the fallback client
set recovers from; the job's verdict is `download.job.failed`.
- The sink groups a bare `%s` pass-through by its text (numbers folded), and
a process's last batch closes the repeat window, so a short-lived download
child still reports what it dropped.
- Worker-side failures (stall, wall-clock, no asset, cached asset error) log
`download.job.failed` with a cause; they left no line at all before. The
crash, supervisor-error and non-zero-exit lines carry the code too.
- An edit failure is one coded line in `_fail_edit`, for every path.
- `logs.get_logger("__main__")` resolves to the `-m` module, so the worker
and the download child keep their names.
- TRACE lines carrying user content are gated on `logs.consented_trace()`,
not on the process level; a contract test pins it.
- The OAuth token refresh goes through `_upstream_failure`; transport errors
carry reason `transport`. Subscribe recognises a duplicate by reason code
only (the prose fallback was dead).
- Once-only guards for a stale sysconfig override and a refusing HLS session
directory; item_detail splits Plex failures from our own parse bugs.
- The playlist breaker line names the playlist and the action; comments that
were false (runner noqa, `_merge_vtt`, sink grouping) are corrected.
|
||
|
peter
|
04b58e5c6f |
feat(logs): give the log the events that no request line already carries
The gap this sprint was scoped against turned out to be smaller than the survey said, and in a way worth writing down: `log_requests` already records every HTTP call with its route template, status, duration, request id and user id, and the audit log already records every admin act on someone else's account. Putting a line in each of routes/downloads.py's twenty mutating endpoints would have added a second copy of the first and nothing else. So this instruments what neither of them can see. yt-dlp now explains itself to our log. `quiet: True` was doing more than quieting: with no `logger` its whole account of a download went to the child's stdout, so "Requested format is not available", a signature-extraction failure or a POT refusal reached us only as whatever text ended up in the raised exception — the real reason a download failed was routinely one line the app never kept. Measured against the pinned yt-dlp 2026.07.04 before wiring it: with `logger` set it calls `debug` for extraction and progress lines and `warning`/`error` for the rest, and neither `quiet` nor `no_warnings` gates those calls. `debug` lands at TRACE, being several lines a second, so the noisy half is the half the user's own consent gates. The full source URL is trimmed to its host on the way in — it is a row of someone's download history, and `logs.set_job` already puts the job id on every one of these lines. The option is set AFTER the adapter merge, like `cookiefile` and the size cap: a source adapter must not be able to take away the one account of what went wrong that a failed job leaves behind. An HLS session now says which pipeline it got (transcode/remux, hw, tonemap, variants, offset) — everything downstream is about that choice, and it was only inferable from an ffmpeg command line nobody keeps. A session culled by the concurrent-transcode cap says so: somebody's playback stops there, the route turns it into a retryable 404, and "my film keeps rebuffering" had no other explanation anywhere. The HW→SW fallback keeps its line and gains an event code. A live search refused by a quota cap says WHICH cap: the 429 is in the request line either way, but the per-user daily cap is answered by tomorrow and the shared budget by the admin. The download and edit job lifecycle lines gain event codes, which is what E22.6's console filters on — the job id was already a structured field. tests/test_ytdlp_logging.py; three of its four cases mutation-checked. A fourth (a `%` in a progress line) survived its mutation — stdlib only applies `%` formatting to a record that has args — so it is gone rather than kept as a case nothing can break; the reasoning moved into the comment on `_emit`. |
||
|
peter
|
6b0401b922 |
fix(logs): rule on every swallowed exception in the backend
114 exception handlers neither logged nor re-raised. The defect was never that
they are silent — plenty of them should be — it is that their silence said
nothing, so nobody could tell a decision from an oversight. Each one has now been
read. 44 got a line; the other 70 carry `# silent-ok: <why>`.
What the silent ones had been hiding, in the cases that got a line:
- A failed ffprobe sent every HDR file down the 8-bit pipeline, so playback
degraded to a software transcode with nothing saying why; a failed audio probe
collapsed a multi-audio file to one track and the player showed no language
picker; a failed first-PTS probe left subtitles seconds out of sync.
- A subtitle sidecar that existed but would not parse made a PARTIAL track read as
complete, permanently — the viewer loses every cue past the cached window.
- An unreadable crop/trim/cut-list in an edit spec was dropped, so the clip came
out uncropped, untrimmed, or shorter than the one the user built.
- A dropped `max_height` downloaded at best quality against an explicit cap.
- A failed videos.list enrich made every candidate undroppable-from-judgement and
therefore dropped: a working search rendered as "no results".
- An unusable password hash answered as "wrong password", locking the owner out
and telling them they mistyped; a rotated TOKEN_ENCRYPTION_KEY turned every
stored token into "not connected".
- A stored admin setting that would not coerce fell back to the env default while
the settings page went on showing the saved value.
- A one-off InnerTube config failure pinned the whole process to the hard-coded
fallback key, and searches return nothing when that goes stale.
- The HLS sweep reported 0 swept — like a clean run — when it could not list its
own root, leaving every crashed session's segments on disk forever.
Four of them are counted rather than logged per item (dropped cues, dropped cut
list segments, unreclaimed GC files) and reported once per call, because the
alternative is one line per cue over a whole subtitle track.
ruff gains S110 and BLE001. Every blind `except Exception` that remains carries
`# noqa: BLE001 — <why>`, which is the point: catching everything is sometimes
right, and always worth a sentence. plex/sync.py's three best-effort backfill
catches now keep the traceback — a blind catch over a batch that fetches AND
writes could be Plex, the database or our own bug, and `%s` of the exception told
them apart badly. main.py's boot-time log-level read likewise.
logs.event's `code` is positional-only now: `event("x", code=...)` was a TypeError,
and `code` is exactly the word a caller reaches for when the thing that failed has
one of its own.
tests/test_swallowed_exceptions.py is the wider rule ruff cannot see: a handler
that neither logs nor re-raises must say why. Both cases mutation-checked — a
marker removed and a log line removed each fail it, naming the handler.
|
||
|
peter
|
0cff964ff2 |
refactor(logs): name every logger after its module, and hold it there
25 literal names covered 34 modules. plex/client.py, plex/stream.py, plex/subtitles.py, plex/sync.py and two Plex route modules all called themselves `siftlode.plex`; downloads/pipeline.py called itself `siftlode.worker`; routes/channels.py called itself `siftlode.api`. The `logger` column is in every stored row so E22.6's console can filter on it, and so an operator can pin a level to one part of the system — neither works when six modules share a name. `logs.get_logger(__name__)` derives it: app/plex/sync.py -> siftlode.plex.sync. One name per module, and it cannot drift from the file it sits in. tests/test_logger_names.py pins the derivation, that everything stays under the one `siftlode` tree the handler and the tests attach to, and that no module hand-rolls a name again unless the line says why (`# logger-name-ok: <why>`). Two do: get_logger's own call, and `siftlode.http`, which is named for the traffic it carries rather than the module it lives in — it replaced uvicorn's access log and is what an operator pins when they want request lines and nothing else. app/scheduler.py called its logger `logger`; every other module calls it `log`. ruff gains LOG, G, T20, TRY400, TRY401 and S112. Four findings cleared with them: worker.py logged `str(exc)[:120]` where the traceback was the whole question, and three prints/f-strings that are deliberate now say so on the line. S110 and BLE001 stay off until S3 — 23 handlers are outstanding and each is a judgement about one of them; enabling the rules first would just mean 23 blanket noqas. |
||
|
peter
|
dc221338be |
fix(logs): keep what a person typed out of the backend log lines
The E22 redactor is a safety net over addresses and credential params, and every one of these call sites was leaning on it to do the whole job. What it does not cover went in whole, and what it does cover came out as a constant. Not covered, so a real leak: - routes/search.py logged the raw search term at ERROR. The privacy policy shipped in 0.88.0 says a search string is kept only at Teljes/TRACE, with the user's own consent. The ERROR line now carries the shape of the failure (source, candidate and collected counts) and the term follows on a TRACE line, which nothing emits unless that consent is in force. - downloads/backfill.py logged the full source URL at INFO — a row of the user's download history. Now the asset id and the host. - youtube/client.py logged the upstream error body, which is prose that quotes the request back. Every failing YouTube call now goes through one _upstream_failure() that logs the API's `reason` code, keeps the body for the same consented TRACE line, and raises a YouTubeError carrying `reason` as a field — callers put that error in log lines of their own (routes/channels.py) and in a 422 answered to the user, so the body must not travel in it either. Covered, so merely useless: routes/auth.py named the account by an address that reached both sinks as `[email]`, and app/email.py printed recipient lists that read as `['[email]']`. Both now carry the uid (or a count) and a reason code, which is also the first half of E22.5 S4's failed-sign-in coverage. routes/channels.py branched on the text of that error message to recognise an already-existing subscription. It now branches on `exc.reason`, with the substring check kept beside it for a body that carried no reason at all. tests/test_log_pii.py pins the call sites and the family: a source-level contract test walks every log call in app/ and fails on an argument naming user-typed content unless the line says why it is safe (`# log-pii-ok: <why>`). Eight sites carry that marker today, each with its reason. Gate 8/8 green; all five cases mutation-checked. |
||
|
peter
|
4ab128ccab | Merge branch 'main' into feature/e22-activity-trail | ||
|
peter
|
07e3ea3737 |
docs: the deep-backfill card counts the backlog, not the next pass
Two reviewers in a row read the admin card's selection against `run_deep_backfill`'s, saw it was missing the job's `recent_synced_at` filter, and called it a card that counts channels the job will never touch. It is the opposite: the card is the same backlog figure /api/sync/my-status shows the user, and a channel still missing its recent pull is one the same job tick fetches first. Narrowing it would hide genuinely queued work. Written down where the next reader will look. |
||
|
peter
|
b2eaa9e00b |
fix: a credential, an erased account and a stale level stay out of the log
Round 2 of the E22 review, on what round 1 changed. `redact_query` decodes a parameter before the text rules see it, which is what lets `%40` be recognised as an address — but `_SECRET_PARAM_RE` was written for the raw, encoded form, where a value holds no space, `&` or quote. On decoded text its value class stops at the first one, so `?state=a%20b` kept its tail and `?code=%20abc` was not matched at all. The parameter-NAME rule now decides, on the key, before any text rule runs. `forget_user` dropped the account's level elevation without recomputing the root level, so a process whose only elevated account had just deleted itself kept building TRACE records to throw away, until some later request happened to recompute it. Every other path that drops an elevation recomputes. The sink's writer took the erasure lock and THEN waited for a pooled connection, while every request path already holds a connection when it reaches that lock — so under pool pressure the writer held a process-global lock against the very requests that could hand a connection back. Same order on both sides now; the lock still spans the check, the insert and the commit. The storage-budget notice said the rest of the hour was not kept, and then smaller batches kept landing underneath it: the window slides and a refused batch is not charged to it. Says what it does. Tests: the route's half of the erasure rule had no test — only the sink's did — and a source-level contract test now covers the third insert nobody has written yet. `logs._erased` is process-global with an hour's memory while the DB fixture hands serial ids back out from 1, so it is swept before each test like the rate limiters are. |
||
|
peter
|
4bfba4aa86 |
refactor: one definition of a followed channel and of a choice spelling
The explore purge rebuilt followed_channel's two EXISTS clauses inline, so what is polled and what may be deleted could drift; it now uses the predicate, which moved to sync/explore.py. The env validator and sysconfig each folded a choice value on their own; both call config.canonical_choice. The deep-backfill comment no longer calls recent backfill unconditional. |
||
|
peter
|
86bc008a81 |
fix: a crashed job's crash line carries its job id
The bind sat inside the try, so it was already reset when the crash line and the cleanup were logged — the one line most worth finding by job id had none. |
||
|
peter
|
fac5c4465a |
fix: stored logs keep bad rows, erased accounts and unconsented queries out
Deep-review round 1 of E22. - A NUL character (`GET /assets/%00` logs the decoded path) made Postgres refuse the row and with it the sink's whole batch, every user's rows included. `to_row` replaces NUL in every text column and in attrs. - An erased account's lines kept arriving after purge_user's DELETE: the erasing request's own access line, a queued row, a background task. `logs.forget_user` runs first; later rows are stored without the account, and the check and the insert share a lock with the erasure. - An admin setting the whole process to TRACE logged every user's query string. The query is now logged only under the user's own Teljes consent, and decoded before redaction, so `%40` no longer hides an address. - The browser log ingest has an hourly storage budget per account (5 MiB, one notice when spent) and reads only a JSON Content-Type, so a same-site text/plain form post cannot write into a user's log. RateLimiter takes a per-attempt cost for the byte budget. - The root level is applied under the elevation lock: two concurrent changes could leave it at the base while an elevation was active. - Tests for the running sink (level gate, drain on stop) and for the admin log level applied at runtime and at boot. |
||
|
peter
|
9c9b1cf6d5 |
test: the gate's output carries no noise to hide a real warning in
- Home tests mocked the api without downloadIndex, so every card's download button logged a missing-queryFn error (64 stderr blocks per run). - QueryState rendered without an i18n instance; Modal.fullscreen awaited a synchronous act, which React reports as an act environment it does not support. - pytest: Starlette 1.x's TestClient is built on httpx2 and warned on every run without it (pinned in requirements-dev); alembic.ini declares its path separator. |
||
|
peter
|
5883ea509d |
feat: Settings → Diagnostics, a consented log level and the tab's own log
A user can raise their diagnostics level to Részletes (DEBUG) or Teljes (TRACE) for 24 hours from a new Settings tab, after a consent box that says what is recorded, that the server keeps it and for how long (the instance's own retention), and that it lapses. Alap is the base and needs no consent. - users.diagnostics_level/_until (migration 0087), written only by PUT /api/me/diagnostics, which computes the expiry itself; the demo account cannot raise it. /api/me reports the level in force. - The server logs an elevated user's requests at that level and stores every line of them (logs.LevelGate on the stdout handler and the sink; the root is lowered only while an elevation is active). Teljes adds the redacted query string to the request line. - POST /api/diagnostics/events stores the browser's log as source=web rows: account from the session, body capped while streaming, events/attrs capped, re-redacted, events under the consented level dropped, timestamps clamped, 30 batches a minute, 409 once the level lapsed. - lib/diagnostics.ts applies the level to the tab and streams only what happens after consent, in batches, never its own request lines, with a keepalive flush on pagehide; stops on lapse (its own timer), on withdrawal and on 409. - Detail hooks at DEBUG: which control was clicked (test id, icon, app copy in English — never user text, input values or a link's query), visibility/bfcache/resize, filter changes (lib/filterLog.ts; typed search only at TRACE), player events (lib/mediaLog.ts; media errors are warnings at every level). Offline/online at the base level. - "Saját napló" viewer: live, newest first, level groups with counts, search, row details, copy the filtered events as JSON lines, clear. - Privacy policy and Terms exist in Hungarian and English (a contract test holds both languages to the same sections); the policy now describes the server logs and diagnostic logging, self-service deletion and password sign-in. Fixed on the way: - A sync auth dependency ran logs.set_user on a copied context, so the account never reached a handler's own log lines (measured); a per-request mutable scope carries it now. - The Settings tab rail left a phone no room (clipped labels, buttons over text): below `sm` it is a scrolling row above the content. - PUT /api/me/preferences took a bare dict body; it is a Pydantic model now. |
||
|
peter
|
a210992bfc |
feat: client log with request ids, joined to the server's
The browser now keeps its half of the one event shape logs.py defines: lib/log.ts is a levelled, redacted, bounded (2000 events) in-memory ring with the server's level names and severity numbers. Nothing leaves the browser yet; sending is the consented diagnostics level (S5). - lib/http.ts `send` is the one fetch: every request carries an X-Request-Id and becomes one http.request event. The server echoes the matched route template in X-Route (only for a matched route, so a raw path with a value in it is never sent back); without one the client templates the path itself. req, beacon, the upload XHR, the public watch page and the legal pages all go through it. - Hooks: app start snapshot, uncaught errors and rejections, History push/replace/popstate (a scroll-only replace is not an event; parameter values only at DEBUG, typed search only at TRACE), notifications, the error dialog, the crash boundary, failed queries and mutations (key head only), and the messages socket (lifecycle, frame types at DEBUG). - Redaction is one contract: backend/tests/fixtures/log_redaction_cases.json runs against both redact() implementations. - Lint: fetch, XMLHttpRequest and WebSocket each belong to one logging module, console to none; a contract test proves the rules fire. Fixes a socket takeover: the close of a socket released by the last subscriber cleared the newer socket a quick re-subscribe had opened and scheduled a third, leaving two live sockets that delivered every message twice (measured in Chromium: 3 created / 2 live before, 2 / 1 after). |
||
|
peter
|
0517de1f59 |
fix: poll and backfill only channels somebody follows
The scheduled RSS poll and recent backfill covered every channel row, but rows also come from search results, playlist sync and dropped subscriptions, and nothing deletes them. On prod 582 of 979 polled channels had no follower, three of them dead feeds answering 404 on every poll. Both jobs now take subscribed or explored channels only, and the scheduler card counts the same set. As a deliberate trade-off, the shared Library no longer receives new uploads from channels nobody follows. |
||
|
peter
|
b59fd38a9c |
feat: store warnings and request lines in log_events
The log now has a stored, queryable copy for the diagnostics console: every WARNING and above plus the request line of every request, kept for log_retention_days (default 7, no "forever") and pruned hourly by the new log_gc job. Rows are built on the logging thread from the same redacted fields as the stdout line, queued without blocking and written in batches by one background thread on its own connection. A full queue or a failed write drops and reports instead of retrying. Flood control keeps 30 of any one message per minute plus a count of the rest; anonymous request lines are capped at 60 a minute per client, and the client address is never stored. Deleting an account deletes its stored events. Measured on prod first: a YouTube RSS outage wrote 11k warnings in four hours, one per channel per poll. The poll now writes one warning with per-reason counts and a few examples, and a failure on our side gets one traceback per poll instead of one per channel. |
||
|
peter
|
07766c43e8 |
feat: closed-set admin knobs are pickers, not free text
log_level, search_source, download_layout and the transcode hw/preset were typed by hand, and every consumer carried its own guess at what a typo meant. The value set is now a Literal in config.py: pydantic rejects a bad env value at boot (any letter case accepted), sysconfig refuses a bad override and derives the admin picker's options from the same Literal, and the consumers read the value as-is. The Configuration page renders a choice key as a segmented bar, or a dropdown past six options, with translated option names. Its tabs now follow the registry's order instead of a hand-kept copy that had already dropped the explore group. A single-choice FilterSelect now opens on the option in use: in the browser it could land on the first row instead. |
||
|
peter
|
2b56c5c2eb |
fix: name the error yt-dlp actually raises for a cookie-emptied client set
A review round read yt-dlp's source and found the comment behind the previous commit's safety-net marker did not match it. Measured in the worker with a real account cookie and `player_client=android_vr`: yt-dlp logs `Skipping client "android_vr" since it does not support cookies`, drops it, and the request left with no client fails with "Requested format is not available" — not "No player clients have been requested". That raise sits EARLIER in yt-dlp, before the cookie-capability removal, so the cookie path can never reach it; and an unknown or renamed client is skipped with a warning and replaced by yt-dlp's defaults, so the other scenario the comment named cannot reach it either. So the marker bought nothing: the symptom that does occur was already retriable here before this branch. It is dropped, the comment now states what was measured, and the test asserts the real symptom instead of a message the code cannot produce. The cookie-capability filter itself is vindicated by the same measurement — without it every signed-in download would burn its first attempt on a client yt-dlp discards. Also from the round: an enqueue test for a `source_kind="youtube"` job, the one join nothing else made (a YouTube job stores a bare 11-char ref and the URL the cookie gate keys on is rebuilt by the adapter — both ends were pinned apart, the join was not, and a break there would stop every YouTube cookie silently while signed-out downloads kept working). And copy corrections: the group's only unhedged claim is hedged, "described the video" loses the pipeline vocabulary, and four Hungarian phrasings that read as translated English are rewritten. |
||
|
peter
|
c06a456a9a |
fix: a YouTube stream refused after extraction is the gate, not a network fault
Measured on the isolated test stack the same night, on the very same video ~90 minutes after it was answering with the bot message: yt-dlp got the player response and picked its formats, and the CDN then refused the stream itself — "unable to download video data: HTTP Error 403: Forbidden". Signed out it did that on every attempt; with the owner's cookie attached the video downloaded in full. So the wall moves between the player response and the stream, and the text it wears there was costing the download twice. retriable: the format URLs belong to the client that minted them, so the next client set is a real second chance — but this text classifies as `network` and nothing in the YouTube marker list matched it, so the ladder ended the job on attempt 1 and never tried the fallback. catalog: `youtube.media_forbidden` for the cookie-less case, so the UI stops calling it a network error when nothing was wrong with the network, and names what actually gets past it. Scoped exactly as narrowly as the measurement: another host keeps the generic `network` category, and so does a download that already carried a cookie — telling a signed-in user to sign in is the failure the catalog exists to avoid. EN + HU copy. |
||
|
peter
|
864f673ef5 |
fix: a YouTube video behind the bot wall is downloadable with cookies
A YouTube link can fail with "Sign in to confirm you're not a bot" while every other YouTube download works — YouTube gates INDIVIDUAL videos behind a signed-in session. Measured on prod job 273: all twelve of yt-dlp's player clients fail on that one video while its neighbours download fine, the bgutil PO-token sidecar is live and answering, and the newest yt-dlp fails identically. So the download is not broken by our stack or a stale yt-dlp, and no retry, client switch or upgrade gets past it. A cookie is the only way in. The failure copy already sends the user to Settings -> Account -> Download authentication, but YouTube was not in the cookie host registry, so that dropdown could not help them: the app asked for something it did not offer. credentials: YouTube joins SUPPORTED_HOSTS (youtube.com, youtu.be, youtube-nocookie.com). The Settings dropdown, the SEC-12 host scope and the private-asset cache salt all derive from the row, so that is the whole wiring. adapters: a signed-in YouTube download runs only cookie-capable player clients. Measured against yt-dlp's own INNERTUBE_CLIENTS table: with account cookies attached yt-dlp REMOVES every client whose SUPPORTS_COOKIES is False, and a set it empties that way raises "No player clients have been requested" — which is not a per-client flake it retries past. Our primary set is android_vr, one of those, so without this the cookie would have failed the download on attempt 1 and never reached the POT-backed web fallback it exists for. A test pins the client list against the installed yt-dlp so a version bump turns the gate red instead of silently un-authing every YouTube download, and the empty-set message joins the retriable markers as the second line of defence. errors: catalog entries for the wall with and without a cookie, so a user who has already added cookies is told they were rejected (YouTube rotates exported ones) rather than to add some. The marker is quote-agnostic: YouTube writes the apostrophe as U+2019, so "you're not a bot" with a straight quote never matches the real text. EN + HU copy. |
||
|
peter
|
6c63df6704 |
feat: one structured log for every process, with a request id
A deployment printed four different log formats and none of them could be joined up: uvicorn's log_config.json for the API, a hand-rolled StreamHandler in main.py for anything not started by uvicorn, logging.basicConfig() in the worker and in the download child (no timestamp at all), and alembic's fileConfig. app/logs.py replaces all four with one JSON line per event (OpenTelemetry's log data model, no SDK): ts, level, severity, source, logger, event code, message, attributes, plus the correlation the context carries — request id, account, download job. Redaction is applied at the HANDLER, not at the call sites: every message, every attribute and every traceback is scrubbed on the way out, so a secret cannot leak by being forgotten at one of ~125 call sites. Passwords, tokens, cookies, secret config values and email addresses never reach a line. The access log is ours now (`http.request`): the route template instead of the concrete path, the status, the duration and the account — and never the query string, which uvicorn's access log printed on every single request (`?account=`, `?source=`, the `?token=` of a share link). A well-formed client request id is accepted and echoed in X-Request-Id, so a browser line and a server line join up. The account comes from `request.state`, filled by the auth dependency: the middleware is the outermost one, so it runs outside SessionMiddleware and in a different task context. `log_level` becomes an admin knob (sysconfig), applied without a restart and re-applied at startup — best-effort on purpose, since a logging knob must never be what stops the app from booting. Every compose service gets bounded json-file logs (3×50 MB): Docker keeps every line forever by default, and the API now writes one line per request. Verified on the localdev stack: JSON everywhere including alembic and the worker, the query strings gone, a supplied X-Request-Id echoed and logged, and three /healthz hits of which only the one made while the level was DEBUG produced a line. |
||
|
peter
|
11aef4ab7a |
feat: record when an account last signed in
users.last_login_at (migration 0085, no backfill) is stamped by _establish_session, which the Google callback now shares with password login instead of repeating its session setup inline. The demo entry does not stamp its shared account. The admin user list carries the field. |
||
|
peter
|
47d968eeea |
fix: review round 4 — an account switch unloads as the old account, a share link's lockout costs five networks
Deep-review round 4, targeted at round 3's changes (f569b19c..e7f38396). Accounts - Switching, adding or signing out an account moved the tab's pin before the page unloaded, so the players' resume beacons fired on the way out under the NEW account: one account's playback position saved into the other. The pin is now handed to the next page (pinOnNextPage) and taken over at import; clearActiveAccount is gone. - Switching to a suspended account was refused but left it in the switcher, refused on every click; a deactivated one was not refused at all, and made the default it signed every account in the browser out on the next request. Every account _session_user would refuse — gone, deactivated, suspended, revoked — is refused and dropped from the list. Share page (/watch) - An IPv6 client is limited by its /64 (client_ip becomes client_key, for every limiter): keyed on the full address, one subscriber could rotate through endless identities. The comments now state what the per-link ceiling costs: guessing from five networks can keep a link shut for its window, and guesses already running when it fills are still checked. - The link is looked up before the limiter records the attempt, so made-up tokens no longer grow it by an entry each. Forms - Failing the same way twice is announced again: the sign-in, passphrase and setup errors are keyed on the attempt (a clear and a set in one handler left the alert node untouched). - The install wizard's own validation showed the generic error — it threw plain objects its catch did not read. It sets the step's message. Tests: a re-pin leaving the previous account's notices behind; the pin hand-over; switching to an ended account; the /64 viewer; made-up tokens; the repeated alerts. Each mutation-checked RED for its own reason. |
||
|
peter
|
e7f38396a2 |
fix: review round 3 — a new tab keeps its account's notices, a revoked tab account lands on the default
Deep-review round 3 over main..improvement/watch-background-audio. Account boot - The notification history and settings, and the hints toggle, are read when their modules load, which in a new tab (bookmark, typed URL) is before BootGate pins the account: they read as nobody's, and the tab's first notice replaced the account's stored history (up to 100 entries) with a one-item list. They now read again when the tab is pinned (onAccountPinned, run by setActiveAccount); a notice raised before the pin stays, after the history. - A tab pinned to an account still in this browser's wallet but whose session ended (suspended, deleted, or revoked by "log out everywhere") got "signed out" from /api/me while the default account was still signed in, and signing in again as anyone else left it there. A refused per-tab account now leaves the wallet, and the probe answers as the default, so the tab reloads onto it. Share page (/watch) - The password unlock is limited per viewer (link + client IP), under a per-link ceiling that counts only wrong passwords. Keyed on the link alone, ten wrong guesses every five minutes locked out every viewer, the ones with the password included. client_ip moves to app.ratelimit so the public routes can key on it. - Its failure messages and the load-failure screen are announced (role=alert, the password input described by its error). The sign-in, passphrase, password-change and setup form errors get role=alert too. Comments that still described removed machinery are corrected: App's collapse flags, SettingsPanel's Save bar, ConfigPanel, authed_format_sig. Tests: the account's history and hints across the pin; a revoked and a suspended pinned account; the per-viewer limit and the ceiling; the limiter's exhausted(); the share page's announced failures; a share link's feed filters persisted. Each mutation-checked RED for its own reason. |
||
|
peter
|
f569b19c49 |
fix: review round 2 — a signed-out tab acts as nobody, honest share-page failures
Deep-review round 2 over main..improvement/watch-background-audio. Account resolution (server) - A request naming an account that is no longer signed in to this browser (X-Siftlode-Account or ?account=) now acts as nobody instead of the session's default account. A tab still pinned to an account signed out in another tab went on acting as the default: its player checkpoints and preference saves landed in that account, the messages socket and Plex playback ran as it, and its Sign out signed THAT account out. A hidden tab never polls /api/me, so a tab playing in the background kept doing it. The /api/me bootstrap probe alone keeps the fallback, which is how such a tab learns who it now is: the app's 401 handler reloads, and BootGate re-pins the tab. BootGate - On an identity change the tab reloads without moving its pin first. The tree being torn down still runs its unmount saves before the page goes, and with the pin already moved they wrote into the new account's local keys (a player volume still pending). The reloaded page corrects the pin. Share page (/watch) - Only a 404 says the link is gone. A 5xx, a dropped connection, and the unlock's own rate limit all rendered "no longer available ... expired or revoked" to a viewer with a valid link, with no way to try again. A load failure now offers Retry; too many password tries says so and keeps the form; an unlock that gets no answer says so instead of doing nothing. Strings in en/hu/de. Downloads - A cookie-authed asset is recognised under the current signature version and the one before it, so a job queued before a deploy that bumps the version still downloads with its cookie instead of failing on the login wall with an error that says no cookie was used. The app chunk is preloaded through lazyWithPreload, so a failed fetch no longer leaves an unhandled rejection beside the ErrorBoundary. The panel layout seed is one helper (seedLayout). Tests: the stale-tab refusal, its harmless Sign out and the probe fallback; the pre-bump authed asset; BootGate keeping the pin on reload and staying mounted through a same-account change; the reveal clock's later-commit hand-off; the prefs, panel layout and demo-scope seeds from the me cache; the share page's failure states. Each mutation-checked RED for its own reason. |
||
|
peter
|
e795d191c5 |
fix(boot): review round 1 — reconcile the tab pin, one loading clock, honest probe failures
Deep-review round 1 over main..improvement/watch-background-audio. BootGate - The tab's account pin is corrected to the account the server answered as. The server honours the pin only for an account still signed in to this browser and otherwise answers as the default, so a tab pinned to an account signed out in another tab ran as the default on the server while reading and writing the gone account's local state. - A mounted tab whose identity changes (its session ended, or the case above) reloads instead of rebuilding the tree in place, like every account switch; that also drops the previous account's query cache. The account key on the tree goes: the readyFor gate already unmounted it on every change. - An unanswered setup probe shows the error with Retry, like an unanswered account probe, instead of the signed-out landing to a signed-in user. - Comments: /api/me never 401s; the ended-session path is identity → nobody. Loading states - LoadingState's reveal delay is one clock across loading states that hand over to each other (boot screen → page-chunk fallback → the page's own fetching state), instead of a CSS delay restarted by each, which blanked a visible spinner for another 300 ms at every hand-off. BootScreen is now the shared loading visual, full screen, counting from navigation start. Plex shared link - Persists only the dimensions the link carries again. Writing the default filters for an omitted `pf` created the stored-filters key, which is what tells SavedViewsProvider the account picked filters, so the account's default Plex saved view was never applied again. Watch page - The Repeat toggle uses the shared pressed-pill paint (a solid fill when on) instead of Button variants that differ only in colour, and its flag goes through storage.ts readJSON/writeJSON. - The actions wrap under the title when the row is too narrow, instead of squeezing the title into a column a word wide on a phone; the title breaks long words. Comments that still justified code by the removed adopt effects (usePref, PrefsProvider, FeedViewProvider) are rewritten against the seed-once model. Tests: BootGate error/retry for both probes, setup mode, a failed background refetch keeping the app, the pin correction and the reload on an identity change; the shared reveal clock; the Plex link persist; the audio signature actually moved off version 4; a video download (av and v) still remuxed to an mp4. Each mutation-checked RED for its own reason. |
||
|
peter
|
6169e32ca5 |
fix(downloads): a signed-in Facebook reel keeps its title, and an mp3 stays an mp3
Since a Facebook cookie was added on prod, every reel came down titled "Video": measured on dev, a signed-in read gives yt-dlp the placeholder "Video" (an 18+ reel read as Chrome gives "Facebook") while the post's text is still in the description. A new SourceAdapter.title hook takes the whole first line of the description in that case, and both title writes (the early fill and the finished download) go through it, so the row, the file name and the share preview carry the post's text again. The line is not cut with an ellipsis: normalize_title's NFKC turns "…" into "..." and collapses it to ".", which read as a finished sentence (seen in dev UAT). The "Audio (MP3)" profile's extracted .mp3 was remuxed into an .mp4 by the browser-safe step (the audio spec is clamped to container mp4 too) and served as video/mp4, which iPad's WebKit will not play (prod job 268). Audio mode now skips that step, so an .mp3 is served as audio/mpeg and an .m4a as audio/mp4. Audio downloads version their cache identity on their own (v5), so a cached remuxed audio re-derives without re-downloading every cached video. |
||
|
peter
|
2a9ddb78ac |
fix(downloads): E11.5 epic-end review round 2
Pause now re-reads the job under the same asset-then-job row lock as resume, cancel, delete and redownload. Its unlocked write waited for a concurrent cancel's lock and then set the released job back to paused, a holding status, so a later delete released the same hold again and could free a file another user's job still held (measured: the overlapping-request test ends 'paused' instead of 'canceled' with the old read). The lock test now calls the route handlers, so it also pins which helper each route uses. Comments follow the code: resolve_url's contract names both verdicts and that a failed page read is raised as itself; the inherited-failure diagnosis asks the cookie gate as it stands when the job is claimed, so its advice fits the cookie that job's Retry would use; the undecryptable-cookie warning no longer claims a download goes ahead, since the worker parent reaches it only to diagnose. |
||
|
peter
|
e887233210 |
fix(downloads): E11.5 epic-end review round 1
A refused Facebook cookie on a group post is no longer "no video in this post":
measured on dev, Facebook answers with its login page (no post id at all), so the
adapter now raises its own POST_NOT_SHOWN verdict and the catalog explains it as
the refused cookie. The adapter owns both post verdicts and the "cannot parse
data" wall wording, so the Chrome retry and the catalog read one constant. The
group-post "no formats" entry is cookie-scoped (with a cookie yt-dlp only gets the
post's video link) and uses the adapter's own group-post test instead of a
"/groups/" substring.
Job transitions that move an asset's ref count (resume, cancel, delete,
redownload) re-read the job under a row lock, asset first, so two overlapping
requests can no longer release the same hold. get_or_create_asset loses an insert
race inside a SAVEPOINT instead of rolling back the caller's pending writes (Retry
lost its cleared error). Every worker-side job failure goes through
finalize_if_running and writes error_detail, so a cancel that landed first is kept
and a failure never shows an earlier one's copy. The worker asks the download
child's own cookie gate (credentials.usable_cookie) when a job inherits a failed
asset.
UI: the Facebook copy no longer points at a Retry button the failure window does
not have, says "paste" like Settings, and the HU wording is fixed
("Újraletöltés" on the re-download confirm). A category this client has no copy
for falls back to the generic sentence instead of a raw key.
Tests: route serialization of error_detail, the probed duration through a real
_download, the page-read cap and a failed read, a Retry off a pending asset, the
confirm-button labels, the lock and savepoint races.
|
||
|
peter
|
90ee34a1b4 |
feat(downloads): a Facebook group post downloads its own video, or says it has none
Given a group post's link, yt-dlp either found no formats or took another video from the page's feed data, so a "successful" retry of such a post was a wrong download. With the user's cookie the Facebook adapter now reads the post page, takes the single video node in the script blocks that carry the post id, and downloads that video's watch link (the Chrome retry does the rest). No single video in the post fails with its own message and the group-post copy, reworded to say the post has no video; the post link is never handed to yt-dlp as a fallback. Without a cookie nothing changes. |
||
|
peter
|
118fc70e08 |
fix(downloads): take the duration from the file when yt-dlp gives none
A Facebook or Instagram reel extracts with no duration, so the asset kept a NULL duration_s although the file plays for minutes, and the Library row had no trim action. The finalize probe now returns the duration with the codecs (the file wins, info fills a gap), and runs before the sidecars so the .nfo runtime gets it too. |
||
|
peter
|
636372511b |
chore: remove the unreferenced backend/Dockerfile
Nothing builds from it: both compose files build the root Dockerfile, and the test image has its own Dockerfile.test. It installed the unpinned requirements.txt, contradicting the lock header that every image installs requirements.lock. |
||
|
peter
|
1354f5b503 |
feat(downloads): a Facebook cookie download stuck at "Cannot parse data" is retried as Chrome
Measured on dev: with the user's cookie some reels still answer "Cannot parse data" and only extract when the request impersonates Chrome. Impersonation is a second attempt behind that one error, only when a cookie is attached — links that already download with the cookie keep their plain path, and without a cookie the same error is the login wall. curl_cffi is pinned inside yt-dlp's supported range (0.10-0.15; outside it every target silently reports "unavailable"); the lock only gains the new packages. A test fails the gate if the locked pair can no longer impersonate. |
||
|
peter
|
98ec1ffcab |
fix(downloads): Retry and Re-download take a cookie added after the failure
The login-wall copy tells the user to add their cookies and press Retry, but Retry kept the job on the shared asset the worker never attaches a cookie to, so it failed on the same wall. Resume now moves a pending/errored job onto the asset enqueue would pick with the cookie (never off a file or a download in flight). Re-download salted its fresh asset with the job's previous asset id, which the cookie gate cannot recompute, so even a cookie download rebuilt without the cookie. The rebuild salt is now the new asset's own id (plus the user's cookie salt when one applies), and is_authed_asset recognises it. |
||
|
peter
|
18c11ca437 |
feat(downloads): a per-site failure catalog — Facebook's walls say why and what to do
A failed Facebook download read "Download failed": yt-dlp words the login wall (18+ reel, closed group) as "Cannot parse data", which no category matched. Measured on dev with the owner's cookie before writing a rule — the same text is the login wall without a cookie and a refused cookie with one. - errors.diagnose(msg, url, cookie_attached) tries a host-scoped catalog first (all-of markers, cookie state, link fragment) and falls back to the category. First entries: facebook.login_wall, facebook.cookie_rejected, facebook.group_post_video. The module doc is the recipe for the next entry. - download_jobs.error_detail (0084) + backfill of the stored Facebook parse failures that were unknown. - The pipeline's failure branch passes the link and whether the cookie went with it; resume and redownload clear code and detail (redownload never cleared error_code). - A job meeting an already-failed asset got the text but no code, so it read "Download failed" where the first job said why. It is diagnosed again, with the cookie state from credentials.is_authed_asset (now shared with the cookie gate); the redacted network text classifies back to network. - FE: failCopyKeys gives the row and the failure window one lookup — catalog copy, else the category. Copy in EN and HU; a contract test pins every backend entry to both locales. |
||
|
peter
|
f130a1c05a |
fix(keys): epic-end review round 1 — Back, menus and dialogs keep the focus where it belongs
Deep-review of E11 (19 raised, 14 confirmed; 2 refuted, 3 latent test-hardening notes dropped). - Back from a sub-view (a Plex show page, a Messages thread) restores the item it was opened from even when focus was still inside the view: the first attempt now waits one task, past the sub-view's own popstate listener (measured in Chrome: it left focus on <body>). - The restore listener is installed at load, so the first Back after a reload finds it; a push cancels a restore still waiting for the last landing's lists. - Navigating over an overlay asks it again, a render apart, while its entry stays: the Plex player peels one panel per Back step, so Shift+W over info + stats closed one panel and dropped the navigation. - j/k: a native <button> item keeps its own Tab stop; landing twice on the same item no longer leaks its tabindex; after the focused item is removed (a hidden card) the next j goes on from its slot when that slot is on screen. - An abandoned `g` no longer swallows a modifier chord (g, then Mod+K opened the browser search). - A hover card above a dialog no longer switches its Tab trap off (player, Modal, wizard, Lightbox): new layerStack `anyBlockingLayerAbove`. - "Manage views…" closes its menu before opening the dialog, through a shared `closeThenAct` that ActionMenu now uses too. - Copy: the EN welcome says "Feed"; the EN unreachable reason no longer claims a change; stale comments about PlayerModal's scroll lock and dialog contract. |
||
|
peter
|
bbd31c7143 |
feat(keys): Back returns focus to the item a view was opened from; "Feed" everywhere; every backend id named
Owner answers to the E11 S5 raises (2026-09-13): 1 → A, 2 → "Feed" in every language, 3 → translate the
proxy label and fix every config key missing in BOTH languages ("maintenance", "tmdb" — old bugs).
Focus across a view change (1-A). Opening a Plex title from Home's rail, a channel from the Channels
table or a title from a Plex grid switches the page or the sub-view, which unmounts the list; Back used
to hand focus to <body>, so the next `j` started over at the top. The YouTube player (an overlay)
already restored onto the card.
- `lib/listNav`: `navPosition()` (list's place in reading order, the page's list count, item index)
and `restoreNavPosition()`: focuses the item once the page has as many lists on screen as when it was
left, polls on a timer (a hidden tab runs no animation frames), gives up after 2 s or at once when
something already holds focus, focuses without scrolling (the scroll restore owns the position).
- `lib/history`: `rememberNavPosition()` stamps `sfNav` onto the entry being left and returns the state
without it (a new entry must not inherit it); one popstate listener restores and spends the stamp,
so a later landing on the same entry (an overlay closing over it) cannot pull focus to an old item.
- Called before every view push: `setPage`, `openChannel`, `enterYtSearch`, `useHistorySubview.open`,
the Channels tab push. `navPosition.wiring.contract.test` fails on any `history.pushState(` without
it (the overlay stack's own push is the one exemption).
"Feed" (2). The module is "Feed" in both languages: every "hírfolyam" in the Hungarian UI (≈45 strings,
suffixed: Feedből, Feeded, Feedben…) and the prose "feed" in English that means the module ("your
Feed", "Hide from Feed"); "RSS feed" keeps its meaning. The welcome email says it too.
Backend ids without a name (3). The Configuration page showed `maintenance`, `tmdb`,
`maintenance_revalidate_batch`, `download_upload_max_bytes`, `audit_retention_days`, `plex_overlay_map`,
`tmdb_api_key` and `tmdb_language` as raw identifiers — the lookup falls back to the id, and the i18n
parity test stayed green because both languages missed the same keys. Added labels + hints (EN/HU),
the two groups (and their tab order), the missing `scheduler.jobDesc.demo_reset`, removed the stale
`plex_sync_interval_min` field, and "YouTube API kimenő proxy". `i18n/backendIds.contract.test` reads
`sysconfig.SPECS`, `scheduler.JOB_INTERVALS` and the audit/quota action enums from the backend sources
and fails on an id without its name in either language, or a name left for an id the backend dropped.
|
||
|
peter
|
8c51062e5b |
fix(api,ui): the demo refusal says why at every site, and the contract guard cannot fail open
Round 6 converted three of four `is_demo` 403s to a structured detail and wrote a docstring saying it
had converted all the sites there are. The account-delete route in `routes/me.py` still answered a
hand-written English sentence, which `notifyActionError` discards in favour of the caller's
translated fallback — so the demo user gets no hint that the demo account is the reason. It is
UI-gated today, so the string is unreachable; what was reachable was the false completeness claim,
which is precisely what stops the next reader from looking. The site is converted, and the list is
now DERIVED from the source: a new test walks `app/` for every `is_demo`-guarded 403, asserts each
raises a structured detail, and fails when the count moves — so this file cannot fall behind the code
the way it just did.
The same round's frontend contract guard was failing open. Its pattern required a bare identifier
before `.detail` and excluded `.detail` used as a last call argument, so `setErr(e.detail)`,
`(e as HttpError).detail`, `String(e.detail)` and four more idiomatic forms were invisible — and
because the file set is compared with `toEqual`, a file the pattern cannot see is simply never listed
and the gate stays green over a fresh copy of the contract. Widened to cover cast receivers, optional
chains and call-argument reads; the pinned set is unchanged, so the narrowing had removed no false
positive. A destructured `detail` stays out of reach (a pattern loose enough for it fires on
react-query's own `const { data: detail }`), and that hole is written into the test rather than
papered over with two innocent files.
Its anti-vacuity guard also did not guard anything: the comment says the glob must have matched
something, but the assertion under it read a sibling file by its own URL, which resolves whether or
not the glob enumerated anything. That only mattered while the pinned list was non-empty — the moment
the deferral sweep lands and it becomes [], the whole file would pass over a glob matching zero
files. The scan size is asserted now.
|
||
|
peter
|
71d7646bf1 |
fix(plex): a name this session cannot write fails at once, and the layout has one spelling
Round 6 turned `artifact_may_arrive` into a positive rule, but keyed it to the HLS vocabulary in
general rather than to what THIS session produces: the playlist regex accepted master/index/stream_*
for every session and the `seq == 0` branch accepted any variant token unconditionally. The two argv
branches are disjoint — a single-audio session is given `seg_%d.ts` + `index.m3u8` and no `%v` at
all, so `master.m3u8`, `stream_*.m3u8` and every `seg_<variant>_<n>.ts` are names it can never write,
and vice versa — so each of those was read as "an HLS name, so wait and see" and held a threadpool
worker, plus its pooled DB connection, for the full timeout. That is verbatim the round-5 outage the
route's own comment claims it prevents.
Measured through the real route against a live single-audio session, before: master.m3u8 404 after
30.0s, stream_q7z.m3u8 404 after 30.0s, seg_a0_0.ts / seg_zzzz_0.ts / seg_0_0.ts 404 after 20.1s
each. After, on the running localdev API: 404 in 44-316ms for all of them, while index.m3u8,
seg_0.ts and seg_1.ts still answer 200 — nothing is over-refused.
The session now carries the rendition names its ffmpeg was told to write, and `entry` is derived from
them instead of passed. And "one place" now means the argv BUILDS its patterns from the same
functions the parser reads back (`segment_name("%d", "%v")`, `variant_playlist_name("%v")`): the
layout was still spelled out independently in six places, with `_VIDEO_VARIANT` read by no consumer,
which is the drift that caused round 6's HIGH. A test asserts the argv's own patterns are what those
functions produce, and that every name they expand to is one the guard accepts.
The load-bearing MEASURED comment cited ffmpeg 8.1.1 — the Windows host's. The binary that writes
these files is the API container's 7.1.5; re-measured there, the conclusion holds.
|
||
|
peter
|
a0114deff1 |
fix(api,ui): a refused write says why, wherever the refusal happens
Round 5 taught `notifyActionError` to discard a plain-string `detail` in favour
of the caller's translated fallback, because `req` leaves 403/404/429 to the
caller and those routes hand-write English. Three places did not get the
message.
**The demo account lost its explanation.** `require_human` refused with a plain
string, so from that commit on the shared demo account got its caller's generic
"couldn't do that" and no hint that the demo account was the reason — and the
gate is not backed by a UI gate everywhere: the comfort shelf has no `is_demo`
check, so the pin button is live and simply fails. A structured `{code: "demo"}`
and an `errors.demo` string in both locales. The two auth routes that hand-roll
the same check were swept with it.
**`ChatThread` carried a second copy of the contract.** It filtered 404/429
itself and then did `e.detail || t(…)`, the pre-fix order, so the central fix
could not reach it and "That member hasn't set up messaging yet." stayed English
under a Hungarian UI. It goes through the shared helper now, and a contract test
pins the remaining raw-`detail` consumers as an explicit SET — never a count —
so a new copy fails the gate instead of shipping. The five on that list are
inline surfaces (a form field's error line, a setup banner) and are deferred
with the ~60 backend routes whose details they print.
**The Plex "Add to playlist" dialog was still silent.** Same shape as the
collection editor round 5 fixed and the YouTube twin that already had the fix:
no catch, so a playlist deleted on another device answered a structured 404 that
nothing reported — the tick never flipped, the gone row stayed, and the floating
promise became an unhandled rejection.
Also: the `ABORTED` sentinel's VALUE was asserted nowhere. The consumer test
hand-built the error it expected, so replacing `xhr.onabort`'s rejection with a
friendlier message left the lane green — and every upload the user cancels would
then raise a red "the upload failed" toast, which is what the sentinel prevents.
Both producer tests assert through `isAborted` now.
And two backend tests were pinning fewer sites than their own docstrings claimed:
the channel-404 loop said "all five" and issued three requests (explore and
subscribe unexercised, the excuse for subscribe naming a different call site
with a different reason slug), and the recipient/job test named three refusals
and requested two — `share_download`'s `not_found("recipient")` had no test in
the suite at all, nor did the locale keys added beside it. Every site named now
has a request behind it.
6 tests, mutation-checked against 7 mutations: the demo detail at both the gate
and an auth route, a re-hand-rolled ChatThread, the silent Plex dialog, the
abort sentinel, and each unpinned 404 — all red on the test that names them.
|
||
|
peter
|
7f17b17c13 |
fix(plex): an HLS session's file names are one vocabulary, and a name it cannot produce fails at once
Round 5 added a guard so a request for a segment beyond the produced region answers instead of polling out the full timeout — the route is a SYNC one, so every such wait parks a threadpool worker and, through `current_user`, a pooled DB connection. Measured against real ffmpeg, the guard could not fire where it mattered. `-hls_segment_filename seg_%v_%d.ts` substitutes the VARIANT first: a dual-audio session writes seg_0_0.ts (video) and seg_a0_0.ts / seg_a1_0.ts (the `name:aN` renditions). The regex read those as sequence-first, so every video segment parsed as segment 0 and every audio segment did not parse at all — the guard was inert on every multi-audio item, which is most of this library. `_seg()` carried the same inversion: the warm-up waited 8 s for `seg_1_0.ts`, a name ffmpeg never writes, on every multi-audio session start AND every seek-restart. The layout is now one constant plus one parser (`segment_name` / `_SEG_NAME_RE`), with a test pinning them to each other, instead of three comments that could drift — and they had drifted, all three ways. The guard is also a positive rule now. It was a blocklist, so anything that was not a single-audio segment name was waved through: `GET …/anything.m3u8` still parked a worker for 30 s, while the route's comment claimed a flood of hopeless names was what it prevented. A session produces its playlists and its segments and nothing else. `.vtt` leaves `_HLS_CT` with it — ffmpeg runs with `-sn` and subtitles are a separate route, so that extension could only ever name a file that will not arrive. Also here: - A 5000-digit segment index reached the client as a 500: `int()` on an unbounded digit run raises past 4300 digits and only OSError was caught. The name is bounded on the route and the index in the regex — the same bug class the subtitle routes were bounded for. - A session torn down between the wait and the read is a 404, not a 500. `_kill` rmtrees the directory and no lock spans those two steps; the player's recovery is keyed on the STATUS, so a 500 skipped the reclaim and ended playback with "couldn't play" instead of restarting at the playhead. The trigger is ordinary — `loadSession` awaits the new session before destroying the old hls instance. - Two comments claimed an open player checkpoints every 10 s "whether or not it is playing", which is the stated reason the 30 s finished-idle reap is safe. `flush` sends nothing once the finish latch is up, or before a position is known. The states it does cover — playing or paused mid-item — are the ones the rule needs; the comments now say that instead of more. - The route's tests leave the DB lane: the route reads nothing from the database since the auto-start was removed, and there those four regressions SKIP on a machine with no Postgres while pytest prints green. 14 tests, mutation-checked: the old parse, the old warm-up name, the blocklist, both bounds off the absurd name, the unguarded read, and the route dropping `proc` — each red on the test that names it. |
||
|
peter
|
1f81684b6e |
fix(ui,plex): a dialog keeps the focus its child asked for, and a hopeless segment fails at once
Both found by UAT in a real browser, after the tests for them were already green — which is the whole reason this repo settles focus and timing claims by running them. - **The autoFocus fix worked, and then the dialog took the focus back.** React 18's StrictMode runs every effect mount→cleanup→mount in development: the cleanup handed focus back to the trigger, and the second mount, finding the focus outside the dialog, moved it to the first focusable — off the `autoFocus` field a beat after the browser had put it there. Chrome measured it: INPUT (commitMount) → BODY (cleanup) → ✕ (re-mount). The effect is idempotent now — it remembers what the dialog was focused on and restores THAT on a re-mount — and the close-time restore is conditional on the dialog still holding the focus, which it tracks as it changes rather than computing at cleanup, where the panel is already out of the document. A dialog no longer yanks focus back from wherever the user has since moved it. - **Passing ffmpeg's handle to the segment wait only shortened the FINISHED case.** Measured against the running localdev API: a request for `seg_99999.ts` on a live session still took 20 195 ms, because that ffmpeg has not exited and the wait has no other reason to stop. ffmpeg writes segments in order, so a segment whose predecessor does not exist is not the next one and cannot arrive — and that is knowable at once. Same request now: 45 ms. Verified on localdev against the real library (Westworld S01E06, remux, multi-audio): Mod+K opens the command palette with the focus in its search field and typing reaches it; playback runs on the new session-scoped HLS URL; a request naming a stale session id is a 404; and a viewer paused for longer than the finished-session rule keeps their 2.4 GB session across two reaper passes — the progress checkpoint touches it — and resumes cleanly. vitest 1009, pytest 1127. |
||
|
peter
|
33fddacee2 |
test(plex): the over-long-name guard is asserted the same way on every OS
Deep-review round 5. The assertion behind `subtitles._exists` was only red where pathlib re-raises ENAMETOOLONG: on Linux it does, on Windows the same stat fails with ERROR_INVALID_NAME, which pathlib's `exists()` already swallows — so the assertion passed with the fix reverted. It forces the errno now instead of relying on the filesystem. (The gate's pytest lane is the Linux container, so the branch was never actually unguarded; the test was just dishonest about why it passed.) |
||
|
peter
|
4a7b65d187 |
fix(api): a refused write is named, in the reader's language
Deep-review round 5, over round 4's own fixes.
- `notifyActionError` preferred the server's `detail` over the caller's translated fallback, and
`reportFailure` deliberately leaves 403/404/429 to the caller — so for exactly those statuses the
dozen call sites round 4 converted started printing whatever English a route hand-wrote into an
otherwise Hungarian UI ("No user with that email.", "Not subscribed to this channel", "Unknown
job"). A STRUCTURED detail is still preferred: `req` has already run it through `localizeDetail`,
so it is both localized and more specific. A plain string is not.
- The other half of the same defect, on the backend: round 4's `get_or_404(what="channel")` sweep
reached exactly ONE of the five identical sites in channels.py and shipped the HU/EN string with
it, so the Block button said it in Hungarian and the four controls beside it in English. All five
are converted, and the subscription / share-recipient / scheduler-job 404s with them (two new
reasons in both locales).
- `PlexCollectionEditor`'s add/remove, take-over and delete were try/finally with no catch — in the
very file round 4 edited to give `createNew` one. 403 and 404 are the caller's to report, so the
tick simply never appeared and a CONFIRMED "delete collection" did nothing, silently.
- `UploadDialog` is the only `uploadReq` caller and kept an empty catch whose comment still claimed
the global error modal had it. It had not since `reportFailure` became shared: a proxy 502 raised
only the self-clearing connection notice and a 413/403/404 nothing at all, so the dialog snapped
back to its pre-upload state with no error named. The user's own Cancel stays silent — and since
a cancel and a network drop both carry status 0, that difference is now a named `isAborted`
rather than a string literal at each end.
Regression tests for each (5 vitest, 2 pytest, plus two existing assertions corrected), all
mutation-checked: 6 mutations, each red on the assertion it is named after. vitest 1007,
pytest 1127.
|