feat: one structured log for every process, with a request id
A deployment printed four different log formats and none of them could be joined up: uvicorn's log_config.json for the API, a hand-rolled StreamHandler in main.py for anything not started by uvicorn, logging.basicConfig() in the worker and in the download child (no timestamp at all), and alembic's fileConfig. app/logs.py replaces all four with one JSON line per event (OpenTelemetry's log data model, no SDK): ts, level, severity, source, logger, event code, message, attributes, plus the correlation the context carries — request id, account, download job. Redaction is applied at the HANDLER, not at the call sites: every message, every attribute and every traceback is scrubbed on the way out, so a secret cannot leak by being forgotten at one of ~125 call sites. Passwords, tokens, cookies, secret config values and email addresses never reach a line. The access log is ours now (`http.request`): the route template instead of the concrete path, the status, the duration and the account — and never the query string, which uvicorn's access log printed on every single request (`?account=`, `?source=`, the `?token=` of a share link). A well-formed client request id is accepted and echoed in X-Request-Id, so a browser line and a server line join up. The account comes from `request.state`, filled by the auth dependency: the middleware is the outermost one, so it runs outside SessionMiddleware and in a different task context. `log_level` becomes an admin knob (sysconfig), applied without a restart and re-applied at startup — best-effort on purpose, since a logging knob must never be what stops the app from booting. Every compose service gets bounded json-file logs (3×50 MB): Docker keeps every line forever by default, and the API now writes one line per request. Verified on the localdev stack: JSON everywhere including alembic and the worker, the query strings gone, a supplied X-Request-Id echoed and logged, and three /healthz hits of which only the one made while the level was DEBUG produced a line.
This commit is contained in:
@@ -10,6 +10,7 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app import audit
|
||||
from app import email as email_mod
|
||||
from app import logs
|
||||
from app import mailtext
|
||||
from app import sysconfig
|
||||
from app.audit import AuditAction
|
||||
@@ -29,6 +30,14 @@ def _redact(v):
|
||||
return _URL_USERINFO.sub(r"\1***@", v) if isinstance(v, str) else v
|
||||
|
||||
|
||||
def _apply_runtime(db: Session, key: str) -> None:
|
||||
"""Some knobs change how THIS process behaves, not just what a later read returns — they have to
|
||||
take effect now, without a restart. One dispatch for both call sites (set and reset), so the next
|
||||
such key can't be wired into one of them and forgotten in the other."""
|
||||
if key == "log_level":
|
||||
logs.set_level(sysconfig.get_str(db, "log_level"))
|
||||
|
||||
|
||||
@router.get("")
|
||||
def get_config(_: User = Depends(admin_user), db: Session = Depends(get_db)) -> dict:
|
||||
return sysconfig.describe(db)
|
||||
@@ -80,6 +89,7 @@ def set_config(
|
||||
after=None if s.secret else {"value": _redact(new)},
|
||||
)
|
||||
db.commit()
|
||||
_apply_runtime(db, key)
|
||||
return sysconfig.describe(db)
|
||||
|
||||
|
||||
@@ -104,6 +114,7 @@ def reset_config(
|
||||
before=None if s.secret else {"value": _redact(old)},
|
||||
)
|
||||
db.commit()
|
||||
_apply_runtime(db, key)
|
||||
return sysconfig.describe(db)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user