E2EE direct messages, the chat dock, the Messages module, the message
WebSocket and its realtime registry, and the messages / message_keys
tables (migration 0097; the downgrade restores the empty schema). Prod
held two welcome notes, no message between people and one key — never
used (owner decision, E34). WebSocket now has no owner module: the lint
bans it everywhere. Privacy policy updated: it no longer promises E2EE
for a feature that is gone.
The 7 baselined Optional accesses in plex/stream.py were one fact the type did not carry: the
initial spawn (replace_only=None) always supersedes, so only the fallback respawn can get None.
An assert at the call site says so; the pyright baseline drops 195 -> 188.
test_abuse_limits: the public-URL and YouTube enqueue tests stopped at the profile lookup
(400 no_profile) and passed on that unrelated refusal. They now send an inline spec and assert
the job is queued. The same shape — a request whose response is discarded — sat at 53 more
places across the suite (all returned what they should, measured with a TestClient probe);
each now asserts its status, and test_response_checked keeps it that way (seen red).
The server now logs polled reads at DEBUG, and the browser's copy of each
request line promises the server's level: at INFO the unread-count polls
filled My log within minutes and, at Detailed, the shipped log. The server
marks a line it logged at DEBUG with X-Log-Quiet, so the polled-route list
stays in one place.
The reads the frontend polls (unread counts, /api/me, the download index,
sync status, subtitle progress) are logged at DEBUG: at INFO they were ~95%
of the stored log on prod. A scheduler job failure and a playlist sync
failure carry event codes, and a Google account without a YouTube channel
has an empty playlist list instead of a warning every six hours.
Admin → Diagnostics reads the server's stored log: one list, newest first,
where a line opens in place to its full record (time, source and logger,
account, request and job ids, version, message, attributes, stack). A
request or job id narrows the list to that request or job with every other
filter off, shown as a removable chip. Filters: level group, source,
account, event, period and a free-text search; all of them live in the URL
(`l*` keys, which setPage clears like the Plex and Channels keys), so a
filtered view is a link and Back restores it. "Live" re-reads the newest
page every few seconds while the tab is visible; paging into older rows
turns it off. Copy puts the loaded lines on the clipboard as JSON lines.
The facet counts now honour every filter but their own, as the feed's do:
a request pivot narrows them, and a picked level does not zero its
siblings. A value in use stays offered even when the counts drop it.
"Saját napló" and the console draw a line with one shared component,
`LogLine`, whose phone layout gives the message its own full-width line
(sharing the code's line left it a few characters). The level groups
moved to lib/log.ts for both.
The console's own reads are logged at DEBUG on success, like the
healthcheck: at INFO its live tail filled the log with itself.
- The download and edit failure lines go back after their DB write: a job the
write could not fail is not a failed job, and the worker's child_exit line
already says it failed.
- The download child's runner is the last stop before Python's hook: an
escaping exception is one download.child.failed line with its chain scrubbed,
then exit 1, instead of a raw stderr traceback carrying the URL.
- A crash the database will not record carries its stack once, and the
fail-write line has an event code.
- The job's own generic-extractor name is taken out as a literal (formats.hiding),
so brackets and colons in a direct link's file name no longer leak; the
pattern stays as the fallback. `Downloading just the video <id>` is covered.
- Tests: scrubbed_stack's cause, suppress, unprintable and text-format paths,
the image steps' failure kinds, the watchdog error through the real
_fail_running, exact poster/storyboard messages, ffmpeg skip marker.
Deep-review round 1 of E22.
- A NUL character (`GET /assets/%00` logs the decoded path) made Postgres
refuse the row and with it the sink's whole batch, every user's rows
included. `to_row` replaces NUL in every text column and in attrs.
- An erased account's lines kept arriving after purge_user's DELETE: the
erasing request's own access line, a queued row, a background task.
`logs.forget_user` runs first; later rows are stored without the account,
and the check and the insert share a lock with the erasure.
- An admin setting the whole process to TRACE logged every user's query
string. The query is now logged only under the user's own Teljes consent,
and decoded before redaction, so `%40` no longer hides an address.
- The browser log ingest has an hourly storage budget per account (5 MiB,
one notice when spent) and reads only a JSON Content-Type, so a
same-site text/plain form post cannot write into a user's log.
RateLimiter takes a per-attempt cost for the byte budget.
- The root level is applied under the elevation lock: two concurrent
changes could leave it at the base while an elevation was active.
- Tests for the running sink (level gate, drain on stop) and for the
admin log level applied at runtime and at boot.
The browser now keeps its half of the one event shape logs.py defines:
lib/log.ts is a levelled, redacted, bounded (2000 events) in-memory ring
with the server's level names and severity numbers. Nothing leaves the
browser yet; sending is the consented diagnostics level (S5).
- lib/http.ts `send` is the one fetch: every request carries an
X-Request-Id and becomes one http.request event. The server echoes the
matched route template in X-Route (only for a matched route, so a raw
path with a value in it is never sent back); without one the client
templates the path itself. req, beacon, the upload XHR, the public
watch page and the legal pages all go through it.
- Hooks: app start snapshot, uncaught errors and rejections, History
push/replace/popstate (a scroll-only replace is not an event; parameter
values only at DEBUG, typed search only at TRACE), notifications, the
error dialog, the crash boundary, failed queries and mutations (key
head only), and the messages socket (lifecycle, frame types at DEBUG).
- Redaction is one contract: backend/tests/fixtures/log_redaction_cases.json
runs against both redact() implementations.
- Lint: fetch, XMLHttpRequest and WebSocket each belong to one logging
module, console to none; a contract test proves the rules fire.
Fixes a socket takeover: the close of a socket released by the last
subscriber cleared the newer socket a quick re-subscribe had opened and
scheduled a third, leaving two live sockets that delivered every message
twice (measured in Chromium: 3 created / 2 live before, 2 / 1 after).
The log now has a stored, queryable copy for the diagnostics console:
every WARNING and above plus the request line of every request, kept
for log_retention_days (default 7, no "forever") and pruned hourly by
the new log_gc job.
Rows are built on the logging thread from the same redacted fields as
the stdout line, queued without blocking and written in batches by one
background thread on its own connection. A full queue or a failed write
drops and reports instead of retrying. Flood control keeps 30 of any
one message per minute plus a count of the rest; anonymous request
lines are capped at 60 a minute per client, and the client address is
never stored. Deleting an account deletes its stored events.
Measured on prod first: a YouTube RSS outage wrote 11k warnings in four
hours, one per channel per poll. The poll now writes one warning with
per-reason counts and a few examples, and a failure on our side gets
one traceback per poll instead of one per channel.
log_level, search_source, download_layout and the transcode hw/preset
were typed by hand, and every consumer carried its own guess at what a
typo meant. The value set is now a Literal in config.py: pydantic
rejects a bad env value at boot (any letter case accepted), sysconfig
refuses a bad override and derives the admin picker's options from the
same Literal, and the consumers read the value as-is.
The Configuration page renders a choice key as a segmented bar, or a
dropdown past six options, with translated option names. Its tabs now
follow the registry's order instead of a hand-kept copy that had
already dropped the explore group.
A single-choice FilterSelect now opens on the option in use: in the
browser it could land on the first row instead.
A deployment printed four different log formats and none of them could be
joined up: uvicorn's log_config.json for the API, a hand-rolled StreamHandler in
main.py for anything not started by uvicorn, logging.basicConfig() in the worker
and in the download child (no timestamp at all), and alembic's fileConfig.
app/logs.py replaces all four with one JSON line per event (OpenTelemetry's log
data model, no SDK): ts, level, severity, source, logger, event code, message,
attributes, plus the correlation the context carries — request id, account,
download job.
Redaction is applied at the HANDLER, not at the call sites: every message, every
attribute and every traceback is scrubbed on the way out, so a secret cannot
leak by being forgotten at one of ~125 call sites. Passwords, tokens, cookies,
secret config values and email addresses never reach a line.
The access log is ours now (`http.request`): the route template instead of the
concrete path, the status, the duration and the account — and never the query
string, which uvicorn's access log printed on every single request (`?account=`,
`?source=`, the `?token=` of a share link). A well-formed client request id is
accepted and echoed in X-Request-Id, so a browser line and a server line join up.
The account comes from `request.state`, filled by the auth dependency: the
middleware is the outermost one, so it runs outside SessionMiddleware and in a
different task context.
`log_level` becomes an admin knob (sysconfig), applied without a restart and
re-applied at startup — best-effort on purpose, since a logging knob must never
be what stops the app from booting. Every compose service gets bounded json-file
logs (3×50 MB): Docker keeps every line forever by default, and the API now
writes one line per request.
Verified on the localdev stack: JSON everywhere including alembic and the
worker, the query strings gone, a supplied X-Request-Id echoed and logged, and
three /healthz hits of which only the one made while the level was DEBUG
produced a line.
peter