feat: one structured log for every process, with a request id

A deployment printed four different log formats and none of them could be
joined up: uvicorn's log_config.json for the API, a hand-rolled StreamHandler in
main.py for anything not started by uvicorn, logging.basicConfig() in the worker
and in the download child (no timestamp at all), and alembic's fileConfig.
app/logs.py replaces all four with one JSON line per event (OpenTelemetry's log
data model, no SDK): ts, level, severity, source, logger, event code, message,
attributes, plus the correlation the context carries — request id, account,
download job.

Redaction is applied at the HANDLER, not at the call sites: every message, every
attribute and every traceback is scrubbed on the way out, so a secret cannot
leak by being forgotten at one of ~125 call sites. Passwords, tokens, cookies,
secret config values and email addresses never reach a line.

The access log is ours now (`http.request`): the route template instead of the
concrete path, the status, the duration and the account — and never the query
string, which uvicorn's access log printed on every single request (`?account=`,
`?source=`, the `?token=` of a share link). A well-formed client request id is
accepted and echoed in X-Request-Id, so a browser line and a server line join up.
The account comes from `request.state`, filled by the auth dependency: the
middleware is the outermost one, so it runs outside SessionMiddleware and in a
different task context.

`log_level` becomes an admin knob (sysconfig), applied without a restart and
re-applied at startup — best-effort on purpose, since a logging knob must never
be what stops the app from booting. Every compose service gets bounded json-file
logs (3×50 MB): Docker keeps every line forever by default, and the API now
writes one line per request.

Verified on the localdev stack: JSON everywhere including alembic and the
worker, the query strings gone, a supplied X-Request-Id echoed and logged, and
three /healthz hits of which only the one made while the level was DEBUG
produced a line.
This commit is contained in:
2026-09-16 01:37:02 +02:00
parent a5bcd9fca2
commit 6c63df6704
18 changed files with 828 additions and 61 deletions
+6 -2
View File
@@ -28,6 +28,7 @@ from datetime import datetime, timezone
from sqlalchemy import text
from app import logs
from app import sysconfig
from app.config import settings
from app.db import SessionLocal
@@ -492,14 +493,17 @@ def _worker_loop(idx: int) -> None:
_stop.wait(settings.download_poll_seconds)
continue
try:
_process_job(job_id)
# Tag every line this job produces (including the crash below) with the job id: a worker
# runs N jobs at once on N threads, so without it the interleaved output is unreadable.
with logs.bind(job_id=job_id):
_process_job(job_id)
except Exception: # noqa: BLE001
log.exception("processing job %s crashed", job_id)
_fail_running(job_id, error="Worker error", error_code=None)
def main() -> None:
logging.basicConfig(level=logging.INFO)
logs.configure_logging("worker")
signal.signal(signal.SIGTERM, _handle_signal)
signal.signal(signal.SIGINT, _handle_signal)