Deep-review round 1 of E22.
- A NUL character (`GET /assets/%00` logs the decoded path) made Postgres
refuse the row and with it the sink's whole batch, every user's rows
included. `to_row` replaces NUL in every text column and in attrs.
- An erased account's lines kept arriving after purge_user's DELETE: the
erasing request's own access line, a queued row, a background task.
`logs.forget_user` runs first; later rows are stored without the account,
and the check and the insert share a lock with the erasure.
- An admin setting the whole process to TRACE logged every user's query
string. The query is now logged only under the user's own Teljes consent,
and decoded before redaction, so `%40` no longer hides an address.
- The browser log ingest has an hourly storage budget per account (5 MiB,
one notice when spent) and reads only a JSON Content-Type, so a
same-site text/plain form post cannot write into a user's log.
RateLimiter takes a per-attempt cost for the byte budget.
- The root level is applied under the elevation lock: two concurrent
changes could leave it at the base while an elevation was active.
- Tests for the running sink (level gate, drain on stop) and for the
admin log level applied at runtime and at boot.
A user can raise their diagnostics level to Részletes (DEBUG) or Teljes (TRACE) for 24 hours from a
new Settings tab, after a consent box that says what is recorded, that the server keeps it and for
how long (the instance's own retention), and that it lapses. Alap is the base and needs no consent.
- users.diagnostics_level/_until (migration 0087), written only by PUT /api/me/diagnostics, which
computes the expiry itself; the demo account cannot raise it. /api/me reports the level in force.
- The server logs an elevated user's requests at that level and stores every line of them
(logs.LevelGate on the stdout handler and the sink; the root is lowered only while an elevation
is active). Teljes adds the redacted query string to the request line.
- POST /api/diagnostics/events stores the browser's log as source=web rows: account from the
session, body capped while streaming, events/attrs capped, re-redacted, events under the
consented level dropped, timestamps clamped, 30 batches a minute, 409 once the level lapsed.
- lib/diagnostics.ts applies the level to the tab and streams only what happens after consent, in
batches, never its own request lines, with a keepalive flush on pagehide; stops on lapse (its
own timer), on withdrawal and on 409.
- Detail hooks at DEBUG: which control was clicked (test id, icon, app copy in English — never user
text, input values or a link's query), visibility/bfcache/resize, filter changes
(lib/filterLog.ts; typed search only at TRACE), player events (lib/mediaLog.ts; media errors are
warnings at every level). Offline/online at the base level.
- "Saját napló" viewer: live, newest first, level groups with counts, search, row details, copy
the filtered events as JSON lines, clear.
- Privacy policy and Terms exist in Hungarian and English (a contract test holds both languages to
the same sections); the policy now describes the server logs and diagnostic logging, self-service
deletion and password sign-in.
Fixed on the way:
- A sync auth dependency ran logs.set_user on a copied context, so the account never reached a
handler's own log lines (measured); a per-request mutable scope carries it now.
- The Settings tab rail left a phone no room (clipped labels, buttons over text): below `sm` it is
a scrolling row above the content.
- PUT /api/me/preferences took a bare dict body; it is a Pydantic model now.
The log now has a stored, queryable copy for the diagnostics console:
every WARNING and above plus the request line of every request, kept
for log_retention_days (default 7, no "forever") and pruned hourly by
the new log_gc job.
Rows are built on the logging thread from the same redacted fields as
the stdout line, queued without blocking and written in batches by one
background thread on its own connection. A full queue or a failed write
drops and reports instead of retrying. Flood control keeps 30 of any
one message per minute plus a count of the rest; anonymous request
lines are capped at 60 a minute per client, and the client address is
never stored. Deleting an account deletes its stored events.
Measured on prod first: a YouTube RSS outage wrote 11k warnings in four
hours, one per channel per poll. The poll now writes one warning with
per-reason counts and a few examples, and a failure on our side gets
one traceback per poll instead of one per channel.
A deployment printed four different log formats and none of them could be
joined up: uvicorn's log_config.json for the API, a hand-rolled StreamHandler in
main.py for anything not started by uvicorn, logging.basicConfig() in the worker
and in the download child (no timestamp at all), and alembic's fileConfig.
app/logs.py replaces all four with one JSON line per event (OpenTelemetry's log
data model, no SDK): ts, level, severity, source, logger, event code, message,
attributes, plus the correlation the context carries — request id, account,
download job.
Redaction is applied at the HANDLER, not at the call sites: every message, every
attribute and every traceback is scrubbed on the way out, so a secret cannot
leak by being forgotten at one of ~125 call sites. Passwords, tokens, cookies,
secret config values and email addresses never reach a line.
The access log is ours now (`http.request`): the route template instead of the
concrete path, the status, the duration and the account — and never the query
string, which uvicorn's access log printed on every single request (`?account=`,
`?source=`, the `?token=` of a share link). A well-formed client request id is
accepted and echoed in X-Request-Id, so a browser line and a server line join up.
The account comes from `request.state`, filled by the auth dependency: the
middleware is the outermost one, so it runs outside SessionMiddleware and in a
different task context.
`log_level` becomes an admin knob (sysconfig), applied without a restart and
re-applied at startup — best-effort on purpose, since a logging knob must never
be what stops the app from booting. Every compose service gets bounded json-file
logs (3×50 MB): Docker keeps every line forever by default, and the API now
writes one line per request.
Verified on the localdev stack: JSON everywhere including alembic and the
worker, the query strings gone, a supplied X-Request-Id echoed and logged, and
three /healthz hits of which only the one made while the level was DEBUG
produced a line.
Deep-review round 3 over main..improvement/watch-background-audio.
Account boot
- The notification history and settings, and the hints toggle, are read
when their modules load, which in a new tab (bookmark, typed URL) is
before BootGate pins the account: they read as nobody's, and the tab's
first notice replaced the account's stored history (up to 100 entries)
with a one-item list. They now read again when the tab is pinned
(onAccountPinned, run by setActiveAccount); a notice raised before the
pin stays, after the history.
- A tab pinned to an account still in this browser's wallet but whose
session ended (suspended, deleted, or revoked by "log out everywhere")
got "signed out" from /api/me while the default account was still
signed in, and signing in again as anyone else left it there. A refused
per-tab account now leaves the wallet, and the probe answers as the
default, so the tab reloads onto it.
Share page (/watch)
- The password unlock is limited per viewer (link + client IP), under a
per-link ceiling that counts only wrong passwords. Keyed on the link
alone, ten wrong guesses every five minutes locked out every viewer,
the ones with the password included. client_ip moves to app.ratelimit
so the public routes can key on it.
- Its failure messages and the load-failure screen are announced
(role=alert, the password input described by its error). The sign-in,
passphrase, password-change and setup form errors get role=alert too.
Comments that still described removed machinery are corrected: App's
collapse flags, SettingsPanel's Save bar, ConfigPanel, authed_format_sig.
Tests: the account's history and hints across the pin; a revoked and a
suspended pinned account; the per-viewer limit and the ceiling; the
limiter's exhausted(); the share page's announced failures; a share
link's feed filters persisted. Each mutation-checked RED for its own
reason.
Deep-review round 2 over main..improvement/watch-background-audio.
Account resolution (server)
- A request naming an account that is no longer signed in to this browser
(X-Siftlode-Account or ?account=) now acts as nobody instead of the
session's default account. A tab still pinned to an account signed out in
another tab went on acting as the default: its player checkpoints and
preference saves landed in that account, the messages socket and Plex
playback ran as it, and its Sign out signed THAT account out. A hidden tab
never polls /api/me, so a tab playing in the background kept doing it.
The /api/me bootstrap probe alone keeps the fallback, which is how such a
tab learns who it now is: the app's 401 handler reloads, and BootGate
re-pins the tab.
BootGate
- On an identity change the tab reloads without moving its pin first. The
tree being torn down still runs its unmount saves before the page goes,
and with the pin already moved they wrote into the new account's local
keys (a player volume still pending). The reloaded page corrects the pin.
Share page (/watch)
- Only a 404 says the link is gone. A 5xx, a dropped connection, and the
unlock's own rate limit all rendered "no longer available ... expired or
revoked" to a viewer with a valid link, with no way to try again. A load
failure now offers Retry; too many password tries says so and keeps the
form; an unlock that gets no answer says so instead of doing nothing.
Strings in en/hu/de.
Downloads
- A cookie-authed asset is recognised under the current signature version
and the one before it, so a job queued before a deploy that bumps the
version still downloads with its cookie instead of failing on the login
wall with an error that says no cookie was used.
The app chunk is preloaded through lazyWithPreload, so a failed fetch no
longer leaves an unhandled rejection beside the ErrorBoundary. The panel
layout seed is one helper (seedLayout).
Tests: the stale-tab refusal, its harmless Sign out and the probe fallback;
the pre-bump authed asset; BootGate keeping the pin on reload and staying
mounted through a same-account change; the reveal clock's later-commit
hand-off; the prefs, panel layout and demo-scope seeds from the me cache;
the share page's failure states. Each mutation-checked RED for its own
reason.
Round 5 taught `notifyActionError` to discard a plain-string `detail` in favour
of the caller's translated fallback, because `req` leaves 403/404/429 to the
caller and those routes hand-write English. Three places did not get the
message.
**The demo account lost its explanation.** `require_human` refused with a plain
string, so from that commit on the shared demo account got its caller's generic
"couldn't do that" and no hint that the demo account was the reason — and the
gate is not backed by a UI gate everywhere: the comfort shelf has no `is_demo`
check, so the pin button is live and simply fails. A structured `{code: "demo"}`
and an `errors.demo` string in both locales. The two auth routes that hand-roll
the same check were swept with it.
**`ChatThread` carried a second copy of the contract.** It filtered 404/429
itself and then did `e.detail || t(…)`, the pre-fix order, so the central fix
could not reach it and "That member hasn't set up messaging yet." stayed English
under a Hungarian UI. It goes through the shared helper now, and a contract test
pins the remaining raw-`detail` consumers as an explicit SET — never a count —
so a new copy fails the gate instead of shipping. The five on that list are
inline surfaces (a form field's error line, a setup banner) and are deferred
with the ~60 backend routes whose details they print.
**The Plex "Add to playlist" dialog was still silent.** Same shape as the
collection editor round 5 fixed and the YouTube twin that already had the fix:
no catch, so a playlist deleted on another device answered a structured 404 that
nothing reported — the tick never flipped, the gone row stayed, and the floating
promise became an unhandled rejection.
Also: the `ABORTED` sentinel's VALUE was asserted nowhere. The consumer test
hand-built the error it expected, so replacing `xhr.onabort`'s rejection with a
friendlier message left the lane green — and every upload the user cancels would
then raise a red "the upload failed" toast, which is what the sentinel prevents.
Both producer tests assert through `isAborted` now.
And two backend tests were pinning fewer sites than their own docstrings claimed:
the channel-404 loop said "all five" and issued three requests (explore and
subscribe unexercised, the excuse for subscribe naming a different call site
with a different reason slug), and the recipient/job test named three refusals
and requested two — `share_download`'s `not_found("recipient")` had no test in
the suite at all, nor did the locale keys added beside it. Every site named now
has a request behind it.
6 tests, mutation-checked against 7 mutations: the demo detail at both the gate
and an auth route, a re-hand-rolled ChatThread, the silent Plex dialog, the
abort sentinel, and each unpinned 404 — all red on the test that names them.
Eleven findings survived adversarial verification; these are the fixes.
**Address pre-emption.** S1 dropped the approval-time password revoke, arguing `password_login`
refuses an unverified account before it reads `is_active`. Two verifiers showed that holds only
while the address stays unverified — and S6's approval mail exists to get the real owner to end that
state, at which point the squatter's password works. While an address is unverified the NEWEST
registration now owns it: the password is replaced and every outstanding verify token deleted, so
the owner takes the row back simply by registering, and a link already in flight cannot confirm it
for anyone else. The residual (an owner who confirms and neither registers nor resets) is logged as
SEC-16 beside SEC-15, where the re-attestation path it needs already lives. The comments that
asserted the old, false invariant now say what actually guards this.
**The entry URL.** S2 moved `clearEntryUrl()` out of the signed-out auth card into a hook App mounts
unconditionally, and it blanket-stripped the whole query string, the fragment and `history.state` on
the FIRST commit. That broke first-run setup outright — the wizard mounts only after its query
resolves and then finds no `?token=` — and silently ate `?link=`/`?oauth=` (no toast, no refetch
after connecting Google), `?v=`/`?chan=` deep links, and scroll restoration. It now clears only the
fragment and the two one-time flags it owns, and preserves `history.state`.
**The env allow-list.** S3 canonicalised the Invite branch of `is_allowed` and left the
ALLOWED_EMAILS/ADMIN_EMAILS fallback matching literally — the 2026-09-03 lockout with a different
source of truth, plus a shadowing trick where an unauthenticated request under a sibling spelling
permanently masked an env grant. Canonical on both sides now, and the same for the delete dialog's
warning and the ADMIN_EMAILS role bootstrap.
**Honest states.** `send_invitation` stamped "I already wrote to them" before a fail-soft background
send; it sends inline and records only what left. The admin-side resend answered `{ok: true}` for a
dropped mail; it now says so. Both also activate/serialize consistently with the other decide paths.
**Correctness sweep.** The 7-day approval link said "expires in 168 hours" — durations render in
days once there is more than one. `_register_account`, `upsert_pending_invite` and `add_invite`
picked among canonical siblings with no ORDER BY and no exact-spelling tie-break, so which account
got the re-send, and whether a denied row was reopened over a pending one, was up to Postgres.
`purge_user` deleted every canonically-equal invite, revoking a surviving sibling account's access.
`ConfirmProvider` wrapped its ReactNode message in a `<p>`. Approving no longer leaves the
stuck-sign-ups list stale. The refusal panels announce and take focus.
**Tests.** Three mailers rewritten this epic never rendered in any test; every `send_*` now renders
in both languages. The orphan-key guard could not fail (it scanned the catalog that defines the
keys) and had no reverse direction. Newly pinned: canonical activation, the surviving sibling's
grant, a failed invitation and resend, an expired reset link, a throttled login's silence, the
per-language admin fan-out, and that the entry URL keeps what it does not own.
Twelve mails carried hardcoded English while the app itself has enforced HU/EN parity since E5.
A mail is the one user-facing surface nobody can re-render by switching the UI language: it arrives
once, in whatever language we picked, and stays that way. That made the epic's own headline mail —
"one mail, one click" — English for a Hungarian reader.
The text moved to a catalog (`mailtext.py`) with a `text(key, lang, **vars)` renderer, and the parity
rule is now a test rather than a habit: every key must carry both languages AND the same
`{placeholders}`. A translation that drops `{link}` sends a mail with no link in it; one that invents
a placeholder raises inside a background task nobody is watching. Both are silent in production.
Every `send_*` takes the language as its second argument, resolved by the REQUEST handler and never
inside the mail — these all run as background tasks, after the request's session is gone, which is
the same reason the approval mail computes its link in the handler.
Where the language comes from, in order: the account's own preference; failing that, the language the
person asked us in; failing that, English. The middle one is new — `invites.lang`, migration `0082` —
because the mails an access request produces go to an address that usually has no account yet, and
the signal does exist: the page their form was submitted from, or the locale Google reported for a
sign-in we refused. Captured rather than derived, for the third time in this epic (`google_email` in
S3, `preferences.language` in S6 taught the same lesson): a signal that exists only in one moment
cannot be backfilled later. So the admin is asked in exactly one place — the invitation to an address
they typed in themselves, where nobody has ever told us anything.
That send is now a dialog rather than a bare click, which is also where the "you already wrote to
them" warning belongs — it used to live only in a tooltip the admin may never hover, guarding the one
action whose failure mode is mailing a real person twice.
S6's registration `lang` field turned out to be dead on arrival: the backend accepted it, the
frontend never sent it, so every account has been storing nothing. The request builders now stamp the
UI language themselves — a signal that has to be passed in by every caller is one every caller can
forget, which is exactly how it shipped unset.
Admins are notified one mail per language group instead of one blast to everyone. They are people
with their own UI language; a single message would have to pick one of them and be wrong for the rest.
Also removes a duplicated `invited_at` column declaration on `Invite` (two identical definitions
landed in S6).
The funnel's common ordering made a person open TWO messages: one saying their access was approved,
another asking them to confirm their address. The approval mail now branches on what they actually
still have to do — carry the confirmation link, point at sign-in, or say a confirmation mail follows
once they register — so the usual case is one mail and one click. A mail that guesses wrong is worse
than a plain one: telling someone to sign in while their address is unconfirmed contradicts the
screen that will refuse them.
That embedded link lives 7 days rather than the usual 24 hours. Nobody sits waiting for an approval —
it can easily be read on Monday after a weekend, and an expired link spends the one-mail promise and
puts them back on two. The risk is small: the token only ever confirms an address; it signs nobody in
and sets no password.
The most likely root cause of the 2026-09-03 report that a confirmation mail "never arrived" is here
too, and it was never a delivery failure: re-registering an address that already had an unconfirmed
account returned in SILENCE — same cheerful "check your inbox", no mail sent at all. It now re-sends
the link. Nothing new leaks: the response and its timing are unchanged, and the public verify/resend
already does exactly this for the same input.
Whitelisting stays silent, and that is a decision, not an omission. The operator opens a door as a
possibility — usually already agreed with the person — and a row can sit approved for weeks with
nobody registering; a mail fired automatically there would be nudging someone who never asked.
Telling them is its own act on its own button, `invited_at` records it so a second click is a
decision rather than an accident, and the button only appears where nobody is behind the address yet:
"the door is open, come in" is meaningless to someone already inside.
Two things UAT caught that no test would have. Every fresh registration appeared in BOTH admin lists
at once, because the stuck-sign-ups filter keyed on "unconfirmed" — the rule is now "the address may
sign in, and this account still can't", which leaves an undecided request with the queue that owns
it. And the invitation button was offered on every approved row, half of which are existing users.
Also: the confirmation mail finally says what its link's rules are (once, 24 hours) — the reset mail
always has, while this one left the reader to discover both by meeting a dead link. The card's
"didn't get the email?" moves up under the form, and the three waiting messages name the spam folder,
which is where a first message to a fresh recipient sometimes lands.
Registration now records the language the sign-up page was in. Nothing reads it yet — the bilingual
templates are S7 — but it is the only moment that choice exists anywhere the server can see, and a
preference nobody recorded cannot be backfilled later.
UAT on localdev: re-registering re-sent the link (one token became two, one account); approval issued
a 7-day token and moved the account into the stuck list with only a re-send; whitelisting mailed
nobody; the invitation recorded itself and the row then warned that sending again would write twice.
Test data removed.
Approve an address BEFORE the person registers and their sign-up produces no queue row at all —
`_decide` only takes a still-pending invite — so the Access-requests screen sat empty, the tab badge
read zero, and the only working move was re-typing the address into the whitelist box. Nobody would
guess that, and on 2026-09-03 nobody did.
This is roadmap UX-25, where two earlier attempts were reverted after review. Both moved AUTHORITY:
auto-activating on registration turns a standing whitelist entry into a bearer grant for whoever
registers the address first, and reopening the approved invite to `pending` lets an anonymous sign-up
REVOKE a standing approval. What was actually missing was visibility, so this changes nothing about
what an unauthenticated caller can cause — the admin could already do exactly this by re-entering the
address; they just had no way to know they had to.
The section lists every account that cannot sign in yet and says WHICH gate is shut, because the two
dead ends look the same from the admin's chair: waiting for activation, or waiting on a confirmation
mail. Naming the unconfirmed ones is not decoration — on a no-SMTP instance that is the difference
between the owner and whoever registered the address first (SEC-15), and an admin about to let
someone in should see it. A person the queue above already covers is deliberately NOT listed here;
two buttons for one person is worse than one.
Activate is gated on `is_allowed` — the same function the sign-in paths consult, in core, so the
button that appears and the badge that counts cannot disagree. Without that gate it would
manufacture an inconsistent account: password sign-in only checks `is_active` and would let them in,
while their Google sign-in would be turned away and silently reopened as a fresh access request.
Letting in someone with no approved invite stays the whitelist box's job, which says so out loud.
The badge counts activations SEPARATELY from pending invites rather than widening an existing
field's meaning, and only counts what genuinely waits on the admin: an unconfirmed account waits on
its owner, so it is listed with a re-send but does not nag. The admin-side resend says plainly what
happened, including refusing when the instance has no working SMTP — mail is fail-soft everywhere
else, and that silence is exactly what left the 0.78.5 sign-ups with nothing to fall back on.
`send_verification_link` is now one function with three callers (registration, the public resend,
this one), so the link's shape and TTL cannot drift by who triggered it.
UAT on localdev, both languages: the UX-25 shape reproduced end to end (approve the address, then
register), the badge counting it, Activate letting them in and dropping the badge, the row losing
its Activate but keeping the re-send, a fresh token issued by the re-send, and both actions in the
audit log. Test data removed.
Two halves of one rule: the address someone typed is theirs to keep, and it must be
the one that gets them back in.
A plus-tagged or dotted Gmail is often a deliberate choice -- registering as
me+siftlode@gmail.com is how you find out later which site leaked your address. The
Google callback overwrote it with the account's primary, which can never contain a
'+', so the tag vanished on the first Google sign-in and, because sign-in matched the
stored address, took their chosen address with it. The rewrite line is older than
this epic, but S3 made it reachable here by matching adoption across spellings. It
now runs only when the mailbox GENUINELY changed, and lives in a named helper rather
than inline, so the rule has one home.
The three lookups that resolve a typed address -- password login, password reset,
verify resend -- go through find_user_by_email: exact spelling first, then the unique
canonical candidate, and None when several accounts share a mailbox rather than
guessing which one a password should open. Reset and resend previously answered a
different spelling with silence (they reply uniformly by design), which was a dead end
with no way out. Their mail now goes to the address on the account, never the one
typed into the form.
Gmail delivers x.y@, xy@ and xy+tag@ to one inbox; the invite gate only lowercased.
On 2026-09-03 that turned an approved user's Google sign-in into a silent SECOND
access request -- OAuth returns the account's stored primary, which need not be the
spelling they typed -- and left prod with two user rows for one mailbox (ids 13, 14).
The +tag case fails every time: a Google account cannot contain +, so OAuth can only
ever answer with the base address.
Mail still goes to the address AS TYPED; only identity matching uses the canonical
form. utils.canonical_email is the single definition, and a model validator derives
users.email_canonical / invites.email_canonical on every write, so no call site can
forget it. Scope is narrow on purpose: dots are significant on most providers and the
Gmail rule does not hold for Workspace custom domains.
Swept every sibling of the same principle, not just the reported one: is_allowed,
upsert_pending_invite, the Google-callback adoption, _register_account (so no new
duplicate pair can be created), _activate_user_by_email, admin _accounts_for (an
invite backing a live user under another spelling used to look deletable) and
add_invite, plus purge_user, which now erases every canonically-equal invite instead
of leaving one behind that still grants access.
Migration 0079 backfills both columns and indexes them NON-uniquely: prod already
carries canonical duplicates (approved invite pairs since 2026-06-16), and merging
accounts is a judgement call about people's data, not something a migration does on
its own. A test pins the backfill SQL against the Python helper.
A rotated or missing TOKEN_ENCRYPTION_KEY makes every stored secret undecryptable.
sysconfig._raw_override called security.decrypt() with no guard, and it sits on
hot paths (/api/me via google_enabled, login, every YouTube scheduler job,
account purge) — so a key mismatch raised InvalidToken and 500'd the whole app,
the SPA wouldn't even boot, with no documented recovery.
Catch the decrypt failure in _raw_override -> warn once per key and fall back to
the env/config default. The YouTube client's refresh-token decrypt now raises a
clear YouTubeError ("reconnect") that every caller already handles, and
purge_user uses decrypt_optional so a best-effort revoke can't 500 the delete.
Document that the key must never change once secrets are stored (recovery: clear
the affected system_config rows and re-enter).
Cleanup only, no behaviour change (ruff clean + pytest 193):
- keep the whole OAuth scope vocabulary together in app/auth.py: the bundles
(BASE_SCOPES/READ_SCOPES/WRITE_SCOPES/_YOUTUBE_SCOPES) move back next to the
atoms (READ_SCOPE/WRITE_SCOPE) as core; routes/auth.py imports the bundles it
needs instead of rebuilding them from the atoms.
- compact the const groups the split generator spread out (the six rate
limiters, the TTLs, the demo/scope constants) back into blocks.
R12 S2. app/auth.py (1154 lines) mixed the core auth surface with 18 HTTP
endpoints on one /auth router. Split by dependency graph:
- app/auth.py (223 lines) keeps ONLY the core surface other modules import
(current_user, resolved_user_id, admin_user, require_*, has_*_scope,
purge_user, count_admins, get_or_create_demo_user, google_enabled,
is_allowed, operator_contact, optional_current_user) plus everything it
transitively depends on.
- app/routes/auth.py (883 lines, new) gets the router + all 18 endpoints +
their endpoint-only helpers, importing the core surface from app.auth.
- main.py includes auth_routes.router (was app.auth.router).
- test_oauth_redirect imports _oauth_redirect_uri from its new home.
The STAY/MOVE partition is driven by an AST scan of the COMPLETE external
surface (every `from app.auth import` incl. multiline + `from app import
auth; auth.X`), so no still-referenced symbol was moved. Behaviour-equivalent:
every symbol relocated verbatim (get_source_segment equality), route set
proven identical, no undefined names (symtable sweep).
Gate: ruff clean (backend + scripts), pytest 193 passed.
Establish the R6 API-contract infra before the S1b payload:dict migration:
- iso(dt) in utils.py replaces the ~16 hand-rolled `x.isoformat() if x else None`
serializer idioms across the routes.
- get_or_404 / owned_or_404 in routes/_common.py — one 404 + existence-leak policy;
plex's _own_playlist_or_404 and saved_views' _own_view now delegate to it.
- require_write_scope dependency in auth.py collapses the inline has_write_scope→403
checks (channels subscribe/unsubscribe, playlists push/push-plan) that had drifted
into three different messages into one; the conditional delete_playlist guard stays
inline (its message was already canonical).
No response-shape change (iso == isoformat for a present value). Gate: ruff clean,
pytest 163 green (+ test_utils for iso).
- C-3.7: fold a per-link password_version into the signed watch grant (migration
0059 adds the column) and bump it on every password change, so rotating a share
link's password invalidates outstanding grants at once instead of letting them
live out the 6h TTL. make_grant/check_grant take the version; update_link bumps.
- C-3.8: the share recipient picker and share-by-email now reuse the messageable
filter (not-demo AND active AND not-suspended) instead of only excluding demo,
so suspended/deactivated addresses aren't leaked or reachable as targets.
- C-3.9: GET /keys/{user_id} adopts get_thread's MB2 guard — a non-messageable
target with no shared history returns the same "User not found" as a missing id,
so it can't be probed to enumerate users or fetch suspended users' keys.
- C-3.12: _register_account catches IntegrityError on the insert (the select-then-
insert TOCTOU) and treats it as the already-registered no-op.
- C-3.13: trigger_job claims the running flag atomically under _activity_lock
(compare-and-set) instead of a bare check, closing the double-start window.
- Move _messageable/is_messageable_user into a shared app/userscope.py so downloads
doesn't import the messages route module.
Re-add the 127.0.0.1 callbacks to the dev host allowlist (now registered in the
Google console alongside the localhost ones). Deriving the redirect_uri from the
request host keeps the whole round-trip — login, Google callback, and the
session/state cookie — on one origin; a cross-host fallback (127.0.0.1 → the
fixed localhost callback) would silently lose the OAuth state, since a session
cookie set on 127.0.0.1 is never sent to localhost. Clarify that in the comment
and cover all four hosts in the unit test.
- OAuth-cancel feedback now keys on the live session, not link_uid: ANY
signed-in cancel (add-another-account via /auth/login, link, or upgrade)
redirects to /?oauth=cancelled and gets a neutral "Google sign-in was
cancelled" toast — the add-account path set no marker and was silent, and
the shared ?link= message wrongly called an upgrade "account linking"
- drop 127.0.0.1 from the dev OAuth host allowlist: only the localhost
callbacks are registered in the Google console, so a 127.0.0.1 origin now
falls back to the fixed callback instead of a redirect_uri_mismatch
- Welcome: extract the shared goResend handler (banner CTA + sign-in link) and
merge the two adjacent signin-only blocks
- link/upgrade OAuth cancel now redirects to /?link=cancelled (a channel App.tsx
surfaces as a toast) instead of the pre-auth /?login=oauth_cancelled banner a
signed-in user never sees; a plain sign-in cancel is unchanged
- the login banner (suspended / oauth_cancelled) is now state, cleared on the
first deliberate action like the verify banner — no more stale lingering
- add a "Didn't get the verification email?" resend entry on the sign-in screen
so a never-arrived verification mail isn't a dead end (was only reachable from
the expired-link banner)
- unit-test _oauth_redirect_uri (dev-host derive, unknown-host fallback, prod
ignores the Host header — the injection guard)
- collapse the 5-deep title/button ternaries into Record<Mode,string> maps
A Google login started on the Vite dev server (:5173) always came back to the
fixed :8080 callback, because login/link/upgrade passed the static
settings.oauth_redirect_url. Derive the redirect_uri from the request origin
instead, but only in local dev and only for a known dev-host allowlist —
production still uses the fixed configured URL and never trusts the Host header
(host-injection guard). Set the Vite proxy to changeOrigin:false so it forwards
the real Host:localhost:5173 (it was rewriting it to the 127.0.0.1:8080 target).
Both callbacks must be registered in the Google console.
- password reset also sets email_verified (the mail proves the mailbox),
rescuing an account that never clicked the verify link; is_active (admin
approval) is deliberately left untouched
- add POST /auth/verify/resend (rate-limited, off-response-path, anti-enum)
+ a "resend" mode/CTA on the verifyInvalid banner, so an expired/lost
verification link is no longer a dead end (re-register was a silent no-op)
- OAuth cancel/deny now redirects to /?login=oauth_cancelled with a friendly
banner instead of a raw 400 JSON page; pop the oauth_link markers BEFORE the
token exchange so an aborted link flow can't poison the next clean sign-in
C-3.5: /auth/request-access had no rate limiter (unlike register/login/reset/demo)
and answered "approved" for an allow-listed email — a public enumeration oracle for
the allow-list, and an unthrottled invite/admin-mail firehose. It now rate-limits per
client IP and returns a uniform "pending" for every outcome (allowed, pending, new,
throttled); an already-allowed user can still just sign in with Google.
C-3.11: _consume_token was a read-check-write, so two concurrent requests could both
consume the same verify/reset token. It's now a single conditional UPDATE
(...WHERE used_at IS NULL AND expires_at >= now RETURNING user_id) — only the request
that flips used_at gets a user back.
Only English and Hungarian remain. Drops the de locale files, the LANGUAGES
entry, the Google-locale mapping in auth, and the German welcome-message
template. A legacy "de" preference falls back to English gracefully. Content-
language detection and subtitle languages are untouched (media, not UI).
purge_user's revoke used `decrypt(refresh) or tok.access_token`, but access_token is
stored ENCRYPTED — so a token row without a refresh token would send ciphertext to
Google's revoke endpoint and silently fail. Decrypt it. (Pre-existing; surfaced by the
review of the OAuth-scope fix.)
Correcting the earlier scope-union-only fix. The real damage from a logout→login isn't
just the stored `scope` field — verified via a live token refresh that the stored refresh
token itself had been REPLACED with a base-only one: a plain sign-in requests only
BASE_SCOPES and Google handed back a fresh base-only access+refresh token, which
_store_token adopted verbatim, destroying the read/write grant (the old refresh token
stays valid on Google's side, so overwriting it is what loses access).
_store_token now detects when an exchange would NARROW our YouTube scopes and, in that
case, keeps the WHOLE existing grant (refresh token, access token, scopes) untouched.
Broader-or-equal exchanges (first grant, read→write upgrade) adopt + union as before.
Unit-verified across base-relogin / upgrade / new-user / base-user cases.
Two user-reported signin bugs:
1) A plain re-login (or logout→login) wiped the user's YouTube grant: login requests
only BASE_SCOPES and Google's returned `scope` can list just those, but _store_token
wrote it verbatim — dropping a previously-granted youtube.readonly/youtube scope, so
can_read flipped to false and the feed demanded a reconnect. The underlying grant
(refresh token) survives such a login, so UNION the scopes instead of narrowing; they
only shrink on full disconnect (purge deletes the token row).
2) Admin 'new access request' email: (a) it named the wrong menu ('Settings → Account'
instead of Users → Access requests); (b) the requester address was invisible so the
'reply reaches them' note looked wrong (Reply-To is in fact set to the requester) —
now spelled out + a deep-link straight to the approve view (?admin=access-requests,
handled in App); (c) it emailed only the static env ADMIN_EMAILS — now notifies the
ACTUAL admins (active role=admin users) unioned with env, so a UI-promoted admin (who
CAN approve — the approve UI is role-gated) is notified too.
Loose ends to finish the auth security round:
- register + password-reset-request had an enumeration TIMING oracle: an already-
registered email skipped the create path (hash + row writes + email scheduling) and
responded measurably faster than a new one. Move the whole lookup+create (register)
and lookup+token+email (reset) into a background task with its own DB session, so the
endpoint returns in the same time for any valid email regardless of whether it exists.
Verified: existing vs new now ~equal (was 34ms vs 82ms on register); accounts/tokens
still created off-path.
- messages_ws re-implemented resolved_user_id's per-tab wallet-gated account resolution.
Generalize resolved_user_id to take any HTTPConnection (Request OR WebSocket) and call
it from the WS — one shared, wallet-gated resolution. Behavior-identical (unit-checked).
Post-review hardening (both fail-safe, not bugs):
- _client_ip normalizes IPs via ipaddress and unwraps IPv4-mapped IPv6, so a plain
TRUSTED_PROXY_IPS=10.10.0.1 also matches a peer surfaced as ::ffff:10.10.0.1 (a
Docker/IPv6 self-host footgun that would otherwise silently collapse everyone into
one rate-limit bucket).
- entrypoint.sh + _client_ip docstring: explicit warning never to add uvicorn
--proxy-headers, which would rewrite request.client from the forgeable XFF and defeat
the trust check.
_client_ip trusted the first X-Forwarded-For hop unconditionally, so anyone able to
reach the app port could forge XFF and dodge the login/register/reset/demo rate limits.
Now trust XFF ONLY when the request's socket peer is a configured reverse proxy
(settings.trusted_proxy_ips, e.g. the VPS Caddy's WireGuard peer IP), and take the
RIGHTMOST entry — the client our proxy actually saw and appended, immune to a client
pre-seeding a fake XFF. A request from any other peer (hitting the port directly) is
keyed on its real socket IP, so XFF can't be forged to bypass the limits.
New TRUSTED_PROXY_IPS env (empty default = no proxy, use the direct peer). Documented in
.env.example, docs/self-hosting.md, README. Unit-verified against spoof-through-proxy and
direct-bypass cases.
Adversarial re-review of the session-epoch work surfaced:
- WS auth (messages_ws) skipped the epoch check, so a revoked-but-unexpired cookie
could still open the live push channel after a reset/logout-others. Now mirrors
current_user: loads the user once, rejects a stale-epoch cookie before connecting.
- set_password + logout_others re-stamped the cookie BEFORE db.commit(); a failed
commit would strand the current session at a newer epoch than the DB and wrongly
401 it. Commit first, then re-stamp.
- Welcome verify effect could double-POST the single-use token (StrictMode/remount)
and flip the banner to a false 'invalid'. Fire-once useRef guard.
Left as-is (low value, documented): the Plex image proxy authenticates without a DB
load / epoch check (poster/art fetches only); adding one would cost a DB hit per image.
Signed client-side session cookies had no server-side kill switch: logout + password
reset couldn't evict a stolen/copied cookie (valid until expiry). Add User.session_epoch
(migration 0053), record it in the cookie at every login, and reject in current_user any
cookie whose recorded epoch is behind the account's current one.
- Bump the epoch on: password reset (kills ALL sessions — a reset is a compromise response),
password change + a new 'Log out other sessions' action (both re-stamp the CURRENT cookie
so the caller stays signed in, evicting only the others).
- Per-account epoch map in the session so one account's revocation doesn't evict the other
signed-in accounts in the same browser wallet.
- Missing epoch (pre-SA4 cookie) is treated as 0, so the first bump revokes grandfathered
sessions too.
- New POST /auth/logout-others + a Settings → Account 'Active sessions' button (trilingual).
Secret email tokens now ride the URL fragment (#reset=/#verify=), never the query
(?reset=/?token=): a fragment isn't sent to the server, so the token can't leak into
proxy/access logs or a Referer header.
- Reset: link → /#reset=; the SPA reads the token from location.hash and POSTs it
(unchanged /password-reset/confirm).
- Verify: link → /#verify=; new POST /auth/verify (token in body). The legacy GET
/auth/verify?token= is kept so pre-deploy emails in flight still work until they
expire. The SPA reads the fragment token, POSTs it, shows ok/invalid.
- Welcome: read secret tokens from the fragment, status flags from the query; strip
both after capture so nothing lingers in history.
From the auth /security-review (contained fixes; architectural SA3 proxy-trust +
SA4 session-revocation deferred to the user):
- SB1: encrypt the OAuth access_token at rest (was plaintext while refresh_token
was encrypted) — a ~1h Google bearer credential. New security.decrypt_optional()
falls back to a refresh for legacy plaintext tokens; column is unbounded String.
E2E-verified: token refreshed → stored as Fernet ciphertext → 319-subscription
YouTube sync succeeded.
- SA5: password_login is no longer a timing/enumeration oracle — it always runs
argon2 (against a decoy hash for unknown/passwordless emails), so response time
can't reveal whether an account exists.
- SB2: Google login only ADOPTS+activates a pre-existing password account when
Google actually attests email_verified (defense-in-depth against takeover); and
the email sync won't overwrite with a value another account owns (avoids a 500).
- demo_login clears the wallet first (demo can't switch to / act as a real account
via the multi-account header); switch_account rejects a suspended target (which
would otherwise clear the whole session on the next request).
Re-review clean; ruff clean; localdev boots; YouTube auth path E2E-verified.
The logged-out landing probed /api/me, which 401'd, and the browser logs every
non-2xx fetch as a console error regardless of how the app handles it. The
bootstrap probe now returns 200 with {authenticated:false} via a new
optional_current_user dependency; api.me() maps that to null. The render gate
treats no-data as 'signed out' -> the landing, and a thrown error as a real
network/5xx failure. Other protected endpoints still 401, so mid-session expiry
is still caught by the global handler. Lighthouse (dev): Best Practices 96->100.
A plain <a> file download can't send the X-Siftlode-Account header, so current_user resolved
it to the session-default account — 404 'Unknown download' when the tab acts as a non-default
wallet account that owns the file. resolved_user_id now also honours a ?account= query param
(the same wallet-gated selection the WebSocket already uses), and downloadFileUrl appends the
active account id. Verified: default account -> 404, ?account=<owner> -> 206 with the right
Content-Disposition.
Two tabs in one browser can now run two different signed-in accounts at once.
- The signed session cookie stays the browser's account WALLET (account_ids). Which account a
given tab acts as is a per-tab choice held in sessionStorage and sent per-request via the
X-Siftlode-Account header; current_user honours it only for an account already in the wallet,
without mutating the cookie's default account. Switching accounts sets the header + reloads
THIS tab only, instead of the old cookie-wide switch that changed every tab.
- WebSocket can't send headers, so the per-tab account rides in the ?account= query param
(validated against the wallet).
- Logout is per-tab aware: it signs the requesting tab's active account out of the wallet
(promoting a new default only if the removed one was the default), and the tab drops its
override. A stale per-tab header account 401s just that tab instead of clearing the session.
- Serve index.html with Cache-Control: no-cache so a deploy's new hashed bundle is picked up
immediately instead of the browser running a heuristically-cached stale index.html.
- Remove owner-specific / legacy deploy files (home/prod/server compose, deploy/).
The home compose stays as a local untracked file for the maintainer's own deploy.
- Genericise infra-specific code comments (egress-proxy examples) to neutral wording.
- Replace the hardcoded contact email on the legal pages with the instance operator's
configured admin email, served via the public /auth/config and shown with a neutral
fallback — so each self-hosted instance shows its own contact.
- Rewrite README for the current app + a copy-paste self-hosting quick start (prebuilt
image + first-run wizard) with a build-from-source alternative; tidy .env.example.
- app/utils.py: shared valid_email() + now_utc() (one email regex, was duplicated
in auth.py and admin.py with a looser "@"-in check elsewhere).
- security.hash_token(): one SHA-256 token hasher (was duplicated in auth.py + state.py).
- auth.admin_user dependency + count_admins() helper, replacing the inline role
checks and the last-admin count query repeated across admin.py/me.py/tags.py.
Replace the leftover 'subfeed' name across logger names + log_config,
frontend localStorage keys, Postgres user/db/volume defaults in the
compose files, .env.example, config.py, backup/restore scripts and the
README. Pure rename; no behavioural change. localStorage keys move from
subfeed.* to siftlode.* (one-time UI-state reset is acceptable).
- Build the Google OAuth client lazily from sysconfig (DB override, env fallback) and re-register
when the credentials change, so the install wizard / admin can set them without a restart.
- New 'google' config group (google_client_id/secret, encrypted) on the Configuration page; the
token-refresh reads the creds the same way.
- Public GET /auth/config exposes google_enabled; the login page hides 'Continue with Google'
when Google OAuth isn't configured (email+password still works).
- Link a Google account to a password account, and adopt the Google identity onto a matching
email account instead of 500ing on a duplicate; set or change a password from Settings.
- Expose has_google/has_password on /api/me for the Sign-in methods UI.
- Mark Google logins email-verified (backfill existing rows, migration 0024); stop a routine
login from clobbering an admin-assigned role (env ADMIN_EMAILS stays the bootstrap admin).
- Suspension login-gates (password + Google callback + current_user) with a rate-limited
'suspended' notice; shared purge_user (cascade delete + access-request cleanup + Google-grant
revoke) behind self- and admin-deletion; single app_base source for user-facing email links.
Add email+password auth alongside Google: argon2id hashing; users gains
password_hash/email_verified/is_active and google_sub becomes nullable (migration
0022); a single-use, hashed auth_tokens table for email verification + password
reset. Registration creates a pending (inactive, unverified) account + an access
request; sign-in needs verified email AND admin approval. Anti-enumeration: uniform
register/login/reset responses (a correct-password owner still gets a specific
pending reason). allow_registration flag (DB-tunable). Admin users-list + role
endpoint (guards: not self, not demo, not the last admin); approving access (or a
manual whitelist add) now activates the matching pending account. current_user
rejects deactivated accounts. Verified end-to-end via curl + DB.
The shared demo account no longer hits YouTube affordances: the onboarding
wizard never opens (or renders) for it, the empty-feed prompt nudges into
the shared library instead of the connect wizard, and the browser-facing
/auth/upgrade redirects the demo home instead of returning a raw 403 JSON.
Whitelisted emails enter the shared demo user via /auth/demo (lazily
created, no OAuth token/scope), rate-limited per IP and answering
uniformly so it can't be hammered as an enumeration oracle. Add a
require_human dependency that blocks the demo account from quota-spending
sync endpoints and the OAuth upgrade flow, and surface is_demo on /api/me.
Sessions created before multi-session existed had no account_ids, so adding a second
account left the first one (e.g. the long-lived session) absent from the switcher.
current_user now ensures the active user_id is always present in account_ids, so the
switch list is never missing the account you're using.
Track every account that completes OAuth in a browser session (session 'account_ids',
~14-day cookie), so the user can switch between them without another Google round-trip.
New GET /api/me/accounts (switchable accounts, active flagged) and POST /api/me/switch
(guarded: target must be in the session list — proof it signed in here — and re-checked
against is_allowed in case the invite was revoked). Logout now signs out the active
account and falls back to the most recent remaining one, else clears the session. UI: the
account popover lists the other signed-in accounts (click to switch, reloads) plus an
'Add another account' action (-> /auth/login, which uses prompt=select_account).
Trilingual. Third step of Epic N.
Set up react-i18next with locale files auto-loaded per area (Vite glob), a compact
LanguageSwitcher, and language as a server-persisted preference (preferences.language)
mirrored to localStorage. On first login the default UI language is guessed from the
Google-reported locale (hu/en/de, else English). vite-env.d.ts types the build-time env.
peter