Commit Graph
27 Commits
Author SHA1 Message Date
peter 9cf177b198 feat(diagnostics): send a bug report to a private Gitea issue tracker (E16 S4)
- admin config group "Bug reports": Gitea address, repository, token (secret),
  and "Check the connection" — the repository must answer, take issues and
  be PRIVATE (a report carries its reporter's log); checked on every send too
- POST /api/debug-sessions/{id}/issue sends the SAVED draft: create the issue,
  upload each file on its own request (the measured API), then show the
  screenshots inline; progress is kept in the draft, so a send that stops
  half-way keeps its issue and the retry sends only what is missing; 5 new
  issues an hour per user; audited as issue_create
- the composer: Create issue (only where a tracker is set up, naming the
  repository), a partial-send banner with retry, and a Preview — the tracker's
  own rendering in a sandboxed frame, or the zip's markdown without one
- the recorder's refusals answer in the localized family (debug_session_*,
  screenshot_*, issue(s)_*), not bare codes the UI could only call generic
- a tracker refusal logs the status only: its message can quote the title back
2026-09-19 22:48:20 +02:00
peter cedcbb1434 test(i18n): every locale key must be read by the app; 75 dead keys removed
A contract test scans the source (tests excluded) for each key as a whole literal, or under a
dynamic prefix like `plex.sort.${k}`, with plural forms counted through their base key. It found 75
keys that had outlived the code showing them (the old sidebar sort vocabulary, the old login page, a
pager the DataTable replaced, …); they are gone from EN and HU. A key read in a way the scan cannot
see goes in an explicit ALLOWED map with its reason.
2026-09-19 17:15:18 +02:00
peter dc4e8b9fcd fix(plex): playback sessions per viewer, bounded in total; a fair share of subtitle passes
HLS sessions were keyed by the item, so a second account opening the same
film killed the first one's session; the displaced player reclaimed at its
playhead and evicted the other in turn. The registry is now keyed by
(viewer, rating_key), segment requests resolve by both, and the session
directory carries the viewer (the orphan sweep still recognises the older
shapes).

Only transcodes counted toward a cap, so remuxes — each a whole segmented
copy of the film on the scratch volume — were bounded by the library size.
Three bounds now apply after every start, each dropping the least recently
used: two sessions per account, the transcode cap, and a new admin knob,
plex_max_sessions (default 8), over every session. A superseded session
request returns at once, since it waits on the killed ffmpeg, so an account
cannot park more request workers than it may hold sessions.

The subtitle extractor's full-pass slots were global and minutes long each;
one account opening subtitles on film after film could hold them all. An
account may now hold three of the six, and still gets windows past that.
2026-09-19 07:16:17 +02:00
peter 29094f5f51 fix(sync): read a dark RSS feed through the API instead
YouTube's RSS goes dark per channel for hours (prod 2026-09-18/19: 393 of
396 feeds answered 404 for four hours, sticky per channel, so a retry does
not help), and nothing else read a followed channel's new uploads. A
channel whose feed has not answered for `rss_fallback_hours` (admin knob,
default 2, 0 = off) now gets one page of its uploads playlist through the
Data API (1 unit), at most once per window, yielding to the backfill
reserve. Spend is attributed to its own quota action.
2026-09-19 04:51:31 +02:00
peter b59fd38a9c feat: store warnings and request lines in log_events
The log now has a stored, queryable copy for the diagnostics console:
every WARNING and above plus the request line of every request, kept
for log_retention_days (default 7, no "forever") and pruned hourly by
the new log_gc job.

Rows are built on the logging thread from the same redacted fields as
the stdout line, queued without blocking and written in batches by one
background thread on its own connection. A full queue or a failed write
drops and reports instead of retrying. Flood control keeps 30 of any
one message per minute plus a count of the rest; anonymous request
lines are capped at 60 a minute per client, and the client address is
never stored. Deleting an account deletes its stored events.

Measured on prod first: a YouTube RSS outage wrote 11k warnings in four
hours, one per channel per poll. The poll now writes one warning with
per-reason counts and a few examples, and a failure on our side gets
one traceback per poll instead of one per channel.
2026-09-17 00:20:14 +02:00
peter 07766c43e8 feat: closed-set admin knobs are pickers, not free text
log_level, search_source, download_layout and the transcode hw/preset
were typed by hand, and every consumer carried its own guess at what a
typo meant. The value set is now a Literal in config.py: pydantic
rejects a bad env value at boot (any letter case accepted), sysconfig
refuses a bad override and derives the admin picker's options from the
same Literal, and the consumers read the value as-is.

The Configuration page renders a choice key as a segmented bar, or a
dropdown past six options, with translated option names. Its tabs now
follow the registry's order instead of a hand-kept copy that had
already dropped the explore group.

A single-choice FilterSelect now opens on the option in use: in the
browser it could land on the first row instead.
2026-09-17 00:03:56 +02:00
peter 6c63df6704 feat: one structured log for every process, with a request id
A deployment printed four different log formats and none of them could be
joined up: uvicorn's log_config.json for the API, a hand-rolled StreamHandler in
main.py for anything not started by uvicorn, logging.basicConfig() in the worker
and in the download child (no timestamp at all), and alembic's fileConfig.
app/logs.py replaces all four with one JSON line per event (OpenTelemetry's log
data model, no SDK): ts, level, severity, source, logger, event code, message,
attributes, plus the correlation the context carries — request id, account,
download job.

Redaction is applied at the HANDLER, not at the call sites: every message, every
attribute and every traceback is scrubbed on the way out, so a secret cannot
leak by being forgotten at one of ~125 call sites. Passwords, tokens, cookies,
secret config values and email addresses never reach a line.

The access log is ours now (`http.request`): the route template instead of the
concrete path, the status, the duration and the account — and never the query
string, which uvicorn's access log printed on every single request (`?account=`,
`?source=`, the `?token=` of a share link). A well-formed client request id is
accepted and echoed in X-Request-Id, so a browser line and a server line join up.
The account comes from `request.state`, filled by the auth dependency: the
middleware is the outermost one, so it runs outside SessionMiddleware and in a
different task context.

`log_level` becomes an admin knob (sysconfig), applied without a restart and
re-applied at startup — best-effort on purpose, since a logging knob must never
be what stops the app from booting. Every compose service gets bounded json-file
logs (3×50 MB): Docker keeps every line forever by default, and the API now
writes one line per request.

Verified on the localdev stack: JSON everywhere including alembic and the
worker, the query strings gone, a supplied X-Request-Id echoed and logged, and
three /healthz hits of which only the one made while the level was DEBUG
produced a line.
2026-09-16 01:37:02 +02:00
peter bbd31c7143 feat(keys): Back returns focus to the item a view was opened from; "Feed" everywhere; every backend id named
Owner answers to the E11 S5 raises (2026-09-13): 1 → A, 2 → "Feed" in every language, 3 → translate the
proxy label and fix every config key missing in BOTH languages ("maintenance", "tmdb" — old bugs).

Focus across a view change (1-A). Opening a Plex title from Home's rail, a channel from the Channels
table or a title from a Plex grid switches the page or the sub-view, which unmounts the list; Back used
to hand focus to <body>, so the next `j` started over at the top. The YouTube player (an overlay)
already restored onto the card.
- `lib/listNav`: `navPosition()` (list's place in reading order, the page's list count, item index)
  and `restoreNavPosition()`: focuses the item once the page has as many lists on screen as when it was
  left, polls on a timer (a hidden tab runs no animation frames), gives up after 2 s or at once when
  something already holds focus, focuses without scrolling (the scroll restore owns the position).
- `lib/history`: `rememberNavPosition()` stamps `sfNav` onto the entry being left and returns the state
  without it (a new entry must not inherit it); one popstate listener restores and spends the stamp,
  so a later landing on the same entry (an overlay closing over it) cannot pull focus to an old item.
- Called before every view push: `setPage`, `openChannel`, `enterYtSearch`, `useHistorySubview.open`,
  the Channels tab push. `navPosition.wiring.contract.test` fails on any `history.pushState(` without
  it (the overlay stack's own push is the one exemption).

"Feed" (2). The module is "Feed" in both languages: every "hírfolyam" in the Hungarian UI (≈45 strings,
suffixed: Feedből, Feeded, Feedben…) and the prose "feed" in English that means the module ("your
Feed", "Hide from Feed"); "RSS feed" keeps its meaning. The welcome email says it too.

Backend ids without a name (3). The Configuration page showed `maintenance`, `tmdb`,
`maintenance_revalidate_batch`, `download_upload_max_bytes`, `audit_retention_days`, `plex_overlay_map`,
`tmdb_api_key` and `tmdb_language` as raw identifiers — the lookup falls back to the id, and the i18n
parity test stayed green because both languages missed the same keys. Added labels + hints (EN/HU),
the two groups (and their tab order), the missing `scheduler.jobDesc.demo_reset`, removed the stale
`plex_sync_interval_min` field, and "YouTube API kimenő proxy". `i18n/backendIds.contract.test` reads
`sysconfig.SPECS`, `scheduler.JOB_INTERVALS` and the audit/quota action enums from the backend sources
and fails on an id without its name in either language, or a name left for an id the backend dropped.
2026-09-13 07:36:41 +02:00
peter 5a42d76e8a feat(plex): an off-by-default loudness normalisation toggle for playback
Bug hunt 2026-09, item 14 — "how does the multichannel downmix work, and is it
normalised?" It isn't: `plex/stream.py` has always encoded `-c:a aac -ac 2 -b:a
192k`, i.e. fixed stereo with ffmpeg's default downmix matrix and no loudness
handling at all, which is why a film mastered for a cinema arrives with whispered
dialogue and deafening action.

New admin knob `plex_audio_loudnorm` (Configuration → Plex), DEFAULT OFF: it makes
a film markedly louder and narrows its dynamic range, which is a taste call rather
than a fix. It sits at the end of the Plex group, after the software-transcode
knobs — it is an audio concern, not one of the libx264 ones.

  -af aformat=channel_layouts=stereo,loudnorm=I=-16:TP=-1.5:LRA=11,aresample=48000

Three things about that chain are load-bearing and each has a test:

  * the downmix is FORCED FIRST rather than left to `-ac 2` — measuring 5.1 and
    folding it down afterwards would normalise something other than what is heard;
  * the trailing resample is not decoration: `loudnorm` runs its own resampler at
    192 kHz internally, which the AAC encoder will not take;
  * one `-af`, no stream index, so EVERY audio rendition of a multi-audio file is
    normalised — not just the first.

The knob is also part of the session directory name (`…_multi-ln_+0.00`), because
it changes the audio the segments contain, and a session left over from before the
admin flipped it must not be mistaken for one that honours it. The suffix is
appended with a DASH: `_SESSION_DIR_RE` treats the tag as one underscore-free
field, and that regex is the only thing keeping the orphan sweep off what may be a
shared scratch path. A test pins both halves — the name changes, and the sweep
still recognises it.

Measured end-to-end on the app's own HLS segments (a real 5.1 AC3 file, same item,
same 160 s offset, same ten segments): OFF -35.2 LUFS, ON -17.2 LUFS. On a 60 s
bench: I -27.6 → -16.6 LUFS with LRA 13.3 → 9.8 LU. A/V sync is untouched — under
this pipeline's own `-copyts` the audio start PTS is identical with and without the
filter, so the 3 s lookahead desyncs nothing.

Direct-played files are served untouched and stay so; a 5.1 mix is always `remux`,
so the reported case is fully covered.
2026-09-10 20:02:06 +02:00
peter 44f626f732 feat(admin): delete a whitelist entry, and stop hiding a broken mail setup
Three operator gaps, all found the same way — a person could not sign up and the
app showed nobody anything.

- DELETE /api/admin/invites/{id} with a delete control on both the pending
  requests and the previously-decided list. Deny keeps the row so the person
  cannot immediately re-ask; delete forgets the address. Refused with 409 while
  a User row exists for it, because is_allowed reads that row and removing it
  would revoke a live user's access with no other sign — the list ships
  has_account so the UI withholds the button instead of offering one that fails.
- Email is fail-soft by design, which had come to mean invisible. _send() now
  records an email_undelivered notification for every admin when a message is
  skipped or rejected, naming the message that was lost, throttled to one an
  hour. /api/me carries email_configured, driving a standing warning on
  Configuration → Email / SMTP and above the access-request list.
- The sign-in card's Create account / Forgot password / Didn't get the email
  views are history entries (useHistorySubview), so Back returns to the previous
  view instead of only moving the address bar.

Also: the per-test rate-limiter reset sweeps every loaded app.* module rather
than a hand-kept list of three — this change added a limiter outside it.
2026-09-01 23:09:25 +02:00
peter ce96032edb feat(plex): transcode resource discipline + admin knobs (E6 S1c)
The software-fallback re-encode now runs deprioritized and admin-configurable.
The SW transcode ffmpeg child is niced (preexec_fn → os.nice, applied per-child
because the Popen lives in the long-lived API process, which must NOT itself be
niced — unlike the download worker's whole-process _deprioritize) so a CPU-bound
re-encode can't starve request handling or playback; the VAAPI path needs none of
it.

New sysconfig knobs (plex group, rendered on the admin Configuration page, i18n
HU/EN): plex_transcode_hw (auto/on/off, layered onto _want_hw), plex_transcode_
preset, plex_transcode_crf, plex_transcode_max_height (software-only downscale-to
cap; the GPU path encodes at source resolution). plex_max_transcodes stays the
concurrency cap (comment corrected — the fallback is shipped, not "later").

pytest extends the command-builder + _want_hw coverage (preset/crf/max_height
plumbing, the on/off/auto policy). Browser-UAT'd: max_height=240 drops the live
resolution to 330x240; the SW ffmpeg child runs at nice 10 (ppid=api process).
2026-08-30 01:22:30 +02:00
peter 911f5100d8 feat(audit): admin audit log (Notification P3)
Append-only who/what/when trail for admin actions + scheduler run summaries,
generalizing QuotaEvent. Logged at ACTION / run-summary granularity — never
per-row (a scheduler run touching thousands of videos is ONE row).

Backend:
- migration 0054_audit_log + AuditLog model (actor_id FK SET NULL, action,
  target_type/id, summary, before/after JSON, created_at).
- app/audit.py: AuditAction constants (single source of truth, i18n'd on the
  client) + record() (adds to the caller's txn) + gc(retention_days).
- producers wired: role change / suspend / delete / invite approve-deny-add /
  demo add-remove-reset / discovery purge (admin.py); config set/reset — secret
  VALUES never logged, key only (config.py); scheduler interval + maintenance
  tune (scheduler routes); sync pause/resume (sync.py); and the scheduler _job()
  choke point writes ONE run-summary row per run — manual runs + errors always,
  automatic runs only when they changed something (no no-op-poll spam).
- retention: audit_gc scheduler job prunes rows older than audit_retention_days
  (sysconfig, default 180; 0 = keep forever).
- admin API routes/audit.py (/api/admin/audit): recent-window list (actor emails
  resolved, System for background) + clear (leaves one tamper-evidence row).

Frontend:
- new admin-only "Audit log" nav page (ScrollText) using the shared DataTable
  (first admin page to reuse it) — sort/filter/paginate/persist, action select
  filter, actor text filter, before→after detail, Clear-all with confirm.
- AuditLog.tsx + auditColumns.tsx + api.adminAudit/clearAudit + AuditRow type.
- trilingual audit + auditActions namespaces (HU/EN/DE) + nav + config-group +
  scheduler job labels; Audit config group (retention days).
2026-07-12 07:32:41 +02:00
peter 58a4ffd191 feat(plex): surface playback errors + media-mount status in the UI
PlexPlayer: when the stream session can't start, show why instead of an eternal
spinner — 404 (physical file unreachable: missing media mount / wrong path map),
501 (format needs transcoding), or a generic failure; also catch fatal hls.js
errors. ConfigPanel: render the Test-connection media_check (mount OK, or a
warning naming the unreadable local path). New plex/config i18n in en/hu/de.
2026-07-05 06:58:34 +02:00
peter 53b42c3775 chore(plex): use a generic example IP in the Plex server-URL hint (public repo) 2026-07-05 06:27:13 +02:00
peter e46099e0b7 i18n(config): friendly labels for the download_* config fields (en/hu/de)
The Downloads config group was added to the registry without field translations,
so it showed raw keys. Adds label+hint for all 9 download_* fields, matching the
Plex group's treatment.
2026-07-05 01:42:56 +02:00
peter 2aef2d94e2 feat(plex): P0 admin Config UI — connection test + library picker
- ConfigPanel 'plex' group auto-renders; adds a Test-connection button + a
  library-picker (checked sections write plex_libraries, applied on Save)
- api.testPlex + PlexTestResult/PlexSection types
- i18n config namespace: plex group + 7 fields + test strings (en/hu/de);
  also fills the previously-missing 'downloads' group label
2026-07-05 01:35:08 +02:00
peter 18a6cdff6c feat(search): ephemeral results UX, count selector, channel blocklist (frontend)
- Live-search view: a results-count selector (20/40/60/100) replaces manual load-more (the free
  scrape source pages until that many are gathered); an 'these results are temporary' banner with
  a 'Clear now' button that discards them 'as if never added' (api.clearSearch) and returns to
  the feed.
- Channel blocklist: a Block/Unblock toggle + 'Blocked' badge on the channel page (blocked
  channels don't auto-explore and their videos are hidden), and a 'Blocked channels' section in
  the Channel manager with one-click unblock. ChannelDetail.blocked from the backend.
- Admin: a 'Purge discovery' button on the Scheduler page (immediate un-kept search/explore
  cleanup). EN/HU/DE throughout.
2026-07-01 01:00:32 +02:00
peter c8123dd9ae feat(channels): dedicated channel page + ephemeral explore UI
Frontend for the channel-explore feature:
- ChannelPage: banner/avatar/stats header, Subscribe/unsubscribe, an "exploring" badge
  while browsing an un-subscribed channel, Videos/About tabs. Reuses Feed scoped to the
  channel (scope=all + source=all so the per-user ephemeral videos show). Auto-ingests
  recent uploads on first visit (background, with a loading note) + "Load more from
  YouTube" to page deeper; skipped for demo / already-subscribed channels.
- App: openChannel/closeChannel as a Back-aware sub-view (history.state._chan, mirrors the
  YT-search _yt pattern); ChannelPage takes over the content column, nav rail stays.
- ChannelLink/cards/player: the channel name now opens our channel page (onChannelFilter →
  onOpenChannel); the in-card "only this channel" filter button is dropped (the page
  subsumes it). PlayerModal channel-name wiring follows in the next commit.
- api: channelDetail + exploreChannel; ChannelDetail/ExploreResult types.
- i18n EN/HU/DE: channel namespace, explore_cleanup scheduler labels, explore config group,
  channels_explore quota label.
2026-06-30 03:08:52 +02:00
peter dca339bd3d feat(search): drop quota wording when the live search source is scrape
The search response now reports its source; in scrape mode (zero quota) the
results banner and Load-more button drop the 'uses quota' wording. Adds the
search_source toggle's labels/hints and updates the per-user-limit hint to note
the cost only applies to the api source. EN/HU/DE.
2026-06-29 22:30:03 +02:00
peter fb57abdf6a feat(search): live YouTube search UI
Surface live YouTube search in the existing feed, triggered explicitly so the
expensive API call is never per-keystroke.

- Header: the search box still filters the local catalog as you type; Enter or a
  YouTube button escalates the term to a live search (hidden for the demo account).
- Feed: a dedicated infinite query renders results in the same VirtualFeed cards +
  in-app player, under a banner with a back button and a quota note. No auto-paginate
  (each page spends 100 units) — an explicit 'Load more (uses quota)' button instead;
  quota/limit errors (incl. 429) shown inline. The empty local feed offers a
  'Search YouTube for <q>' CTA.
- Library: a 'Search-discovered' toggle reveals search-ingested videos (hidden by
  default); sent as exclude_search_discovered.
- Admin: search_daily_limit_per_user config field; new videos_search quota label.
- All new strings translated in HU/EN/DE.
2026-06-29 02:01:44 +02:00
peter e4b24b1b7d feat(youtube): optional egress proxy for YouTube API calls
New youtube_api_proxy setting (env fallback + admin Config UI, env YOUTUBE_API_PROXY):
when set, the YouTubeClient routes all its httpx traffic through that HTTP(S) proxy.
Lets a dynamic-IP host send API calls through a fixed-IP host (e.g. the server over
private tunnel) so an IP-restricted API key keeps working. Empty = direct.
2026-06-26 00:32:17 +02:00
peter 5597e9fd3f feat(auth): manage Google OAuth credentials from the database (epic 6b)
- Build the Google OAuth client lazily from sysconfig (DB override, env fallback) and re-register
  when the credentials change, so the install wizard / admin can set them without a restart.
- New 'google' config group (google_client_id/secret, encrypted) on the Configuration page; the
  token-refresh reads the creds the same way.
- Public GET /auth/config exposes google_enabled; the login page hides 'Continue with Google'
  when Google OAuth isn't configured (email+password still works).
2026-06-20 20:04:23 +02:00
peter 737da6bd96 feat(auth): admin Users page + allow_registration toggle (5a frontend)
New admin-only Users page (sidebar): Users & roles (list + promote/demote with
confirm; self/demo/last-admin guarded server-side), plus the Access requests
(Invite whitelist) and Demo whitelist+reset migrated out of Settings → Account
(same data/tables — UI relocated only). Settings → Account now holds personal
content only. ConfigPanel learns a boolean field type (toggle) for the new
allow_registration setting. api methods, new 'users' page/route/nav/header, and
EN/HU/DE strings (new users namespace + access group).
2026-06-19 14:16:48 +02:00
peter 2dc374cc7b feat(config): move youtube_api_key to DB (encrypted, admin-editable)
Register youtube_api_key in the sysconfig registry as a secret and resolve it from
the DB-override-or-env layer in YouTubeClient (self.db is in scope). Adds a YouTube
API group to the Configuration page (EN/HU/DE).

Google OAuth client id/secret stay env-only for now: the authlib client is
registered at module load (auth.py), and the token-refresh reads them too, so a
runtime DB override needs a lazy/re-registerable OAuth client — that belongs to the
install-wizard epic (first-boot OAuth config without restart). Admin-role-via-UI
moves to the auth-overhaul epic (user management lives there).
2026-06-19 13:19:06 +02:00
peter 060dbcffb0 feat(config): render quota/backfill/shorts/batch groups on the Configuration page
Add the four new groups to the page's group order and EN/HU/DE labels + field
hints for the 8 newly DB-overridable settings. The page is registry-driven, so
this is purely presentational (group order + i18n).
2026-06-19 13:07:55 +02:00
peter 8198c7712e refactor(config): one Save/Discard bar instead of per-field Save
Per-field Save buttons were an unconventional, endpoint-driven choice. Replace
them with a single drafted Save/Discard bar (consistent with the Settings page):
edits are buffered and applied together on Save (each via the existing per-key
PATCH/DELETE), Discard reverts. Reset-to-default is a per-field affordance applied
on Save too — clearing a non-secret falls back to default; a secret (which can't
be cleared by emptying) gets a toggle that resets it on save. Backend unchanged.
2026-06-19 12:40:23 +02:00
peter a71257f33f feat(config): admin Configuration page (registry-driven)
New admin-only Configuration page that renders the system_config registry grouped
(Email/SMTP first), with per-field save/reset, write-only encrypted secret fields
(disabled with a hint when TOKEN_ENCRYPTION_KEY is unset), and a Send-test-email
button. New 'config' page + sidebar nav item + header title; api methods and
EN/HU/DE strings.
2026-06-19 12:23:00 +02:00