feat: store warnings and request lines in log_events

The log now has a stored, queryable copy for the diagnostics console:
every WARNING and above plus the request line of every request, kept
for log_retention_days (default 7, no "forever") and pruned hourly by
the new log_gc job.

Rows are built on the logging thread from the same redacted fields as
the stdout line, queued without blocking and written in batches by one
background thread on its own connection. A full queue or a failed write
drops and reports instead of retrying. Flood control keeps 30 of any
one message per minute plus a count of the rest; anonymous request
lines are capped at 60 a minute per client, and the client address is
never stored. Deleting an account deletes its stored events.

Measured on prod first: a YouTube RSS outage wrote 11k warnings in four
hours, one per channel per poll. The poll now writes one warning with
per-reason counts and a few examples, and a failure on our side gets
one traceback per poll instead of one per channel.
This commit is contained in:
2026-09-17 00:20:14 +02:00
parent 88a166e4aa
commit b59fd38a9c
23 changed files with 832 additions and 67 deletions
+3 -1
View File
@@ -74,6 +74,7 @@
"plex_watch_sync": "Pulls recent Plex watch changes (finished episodes, resume positions) into Siftlode and re-pushes any local watch changes that didn't reach Plex. The cheap two-way keep-in-sync between full reconciles. If it stops, watched/resume made in the Plex app takes longer to show here.",
"plex_watch_reconcile": "A full watch-state reconcile that rescans every library — catches what the incremental pass can't, notably an episode un-marked as watched on the Plex side. Heavier, so it runs about once a day. If it stops, un-watches done in Plex aren't mirrored back.",
"audit_gc": "Prunes admin audit-log entries older than the configured retention window (Configuration → Audit log). If it stops, the audit log just keeps growing — harmless but unbounded.",
"log_gc": "Deletes stored log events older than the retention window (Configuration → Diagnostics). Runs hourly so nothing outlives the promised limit by more than an hour.",
"tmdb_enrich": "Enriches the mirrored Plex catalogue from TMDB (keywords, crew, budget/revenue, TMDb score, recommendations, artwork), matched by each title's canonical id and served from the app's own database. Optional — a no-op until a TMDB API key is set. If it stops, new Plex titles just miss their TMDB extras until it runs again."
},
"jobs": {
@@ -92,7 +93,8 @@
"plex_watch_sync": "Plex watch sync (incremental)",
"plex_watch_reconcile": "Plex watch reconcile (full)",
"tmdb_enrich": "TMDB enrichment (Plex)",
"audit_gc": "Audit-log cleanup"
"audit_gc": "Audit-log cleanup",
"log_gc": "Log cleanup"
},
"queue": {
"recentPending": "Channels to sync",