peter
876cece881
Gmail delivers x.y@, xy@ and xy+tag@ to one inbox; the invite gate only lowercased. On 2026-09-03 that turned an approved user's Google sign-in into a silent SECOND access request -- OAuth returns the account's stored primary, which need not be the spelling they typed -- and left prod with two user rows for one mailbox (ids 13, 14). The +tag case fails every time: a Google account cannot contain +, so OAuth can only ever answer with the base address. Mail still goes to the address AS TYPED; only identity matching uses the canonical form. utils.canonical_email is the single definition, and a model validator derives users.email_canonical / invites.email_canonical on every write, so no call site can forget it. Scope is narrow on purpose: dots are significant on most providers and the Gmail rule does not hold for Workspace custom domains. Swept every sibling of the same principle, not just the reported one: is_allowed, upsert_pending_invite, the Google-callback adoption, _register_account (so no new duplicate pair can be created), _activate_user_by_email, admin _accounts_for (an invite backing a live user under another spelling used to look deletable) and add_invite, plus purge_user, which now erases every canonically-equal invite instead of leaving one behind that still grants access. Migration 0079 backfills both columns and indexes them NON-uniquely: prod already carries canonical duplicates (approved invite pairs since 2026-06-16), and merging accounts is a judgement call about people's data, not something a migration does on its own. A test pins the backfill SQL against the Python helper.
89 lines
4.6 KiB
Python
89 lines
4.6 KiB
Python
"""Small cross-cutting helpers shared across modules (kept dependency-light on purpose)."""
|
|
|
|
import re
|
|
from datetime import date, datetime, timezone
|
|
|
|
|
|
def iso(dt: date | None) -> str | None:
|
|
"""Serialize a date/datetime for a JSON response: ISO-8601, or None when the column is
|
|
NULL. Single source of truth for the `x.isoformat() if x else None` idiom the routes had
|
|
hand-rolled ~17 times — so a future tz/format tweak lands in one place. Accepts `date`
|
|
too (datetime is a subclass) — e.g. MediaAsset.upload_date is a plain `date`."""
|
|
return dt.isoformat() if dt is not None else None
|
|
|
|
|
|
# Single source of truth for "looks like an email" — used by every endpoint that accepts an
|
|
# address (auth register/reset/demo, admin invites + demo whitelist, the setup wizard). Keep
|
|
# the strictness in one place so routes don't drift between this and a looser "@"-in check.
|
|
_EMAIL_RE = re.compile(r"^[^@\s]+@[^@\s]+\.[^@\s]+$")
|
|
|
|
|
|
def valid_email(value: str | None) -> bool:
|
|
return bool(value and _EMAIL_RE.match(value))
|
|
|
|
|
|
# Gmail (and its googlemail.com alias) delivers to one mailbox regardless of dots in the local part
|
|
# and of any `+tag`; nobody else is guaranteed to. So the canonical form below strips those for
|
|
# Gmail only, and merely lowercases everything else.
|
|
_GMAIL_DOMAINS = {"gmail.com", "googlemail.com"}
|
|
|
|
|
|
def canonical_email(value: str | None) -> str:
|
|
"""The form used to decide whether two addresses are the SAME MAILBOX. Never used for sending —
|
|
mail always goes to the address as the user typed it.
|
|
|
|
Why this exists (observed on prod, 2026-09-03): someone registered as `npeter83.temu@gmail.com`
|
|
and was approved, but Google's OAuth `email` claim came back as the account's stored primary,
|
|
`npeter83temu@gmail.com`. The invite gate only lowercased, found no approved row for that literal
|
|
string, and turned an approved user's sign-in into a silent SECOND access request — plus a second
|
|
User row. The `+tag` case is worse and fails every time: a Google account cannot contain `+`, so
|
|
OAuth can only ever return the base address, meaning anyone who registers with
|
|
`user+siftlode@gmail.com` (a common throwaway habit) is guaranteed to be denied after approval.
|
|
|
|
Scope is deliberately narrow. The dot rule does NOT hold for Google Workspace custom domains
|
|
unless the admin turned it on, and dots are significant on most providers, so anything outside
|
|
gmail.com/googlemail.com is only lowercased. Approving one Gmail spelling always did mean
|
|
approving that mailbox, so collapsing them is the honest reading of an invite, not a widening
|
|
of it.
|
|
"""
|
|
email = (value or "").strip().lower()
|
|
local, sep, domain = email.partition("@")
|
|
if not sep or domain not in _GMAIL_DOMAINS:
|
|
return email
|
|
local = local.partition("+")[0].replace(".", "")
|
|
# Normalise the alias domain too, so googlemail and gmail spellings meet.
|
|
return f"{local}@gmail.com" if local else email
|
|
|
|
|
|
def now_utc() -> datetime:
|
|
"""Current time as a timezone-aware UTC datetime."""
|
|
return datetime.now(timezone.utc)
|
|
|
|
|
|
def to_tsquery_str(q: str) -> str | None:
|
|
"""Turn free-text into a to_tsquery() string: word runs ANDed together, the LAST word
|
|
prefix-matched (`:*`) for as-you-type search — e.g. "ti amo magyar" → "ti & amo & magyar:*".
|
|
Only `\\w` runs containing an alphanumeric are kept, so no tsquery operators can leak in
|
|
(the string is built from sanitised tokens, never the raw query). Shared by the YouTube feed
|
|
search and the Plex library search, hence this neutral home rather than either domain module."""
|
|
tokens = [t for t in re.findall(r"\w+", q, flags=re.UNICODE) if any(c.isalnum() for c in t)]
|
|
if not tokens:
|
|
return None
|
|
tokens[-1] = tokens[-1] + ":*"
|
|
return " & ".join(tokens)
|
|
|
|
|
|
def normalize_description(text: str | None) -> str | None:
|
|
"""Strip whitespace-only lines from a CHANNEL description before storing. YouTube channel
|
|
descriptions pad their contact/link blocks with runs of blank lines; keeping them wastes storage,
|
|
bloats the About popover, and adds nothing to the search document. Drops purely-empty lines and
|
|
right-strips each kept line; `str.splitlines()` also folds CR / CRLF and Unicode line separators
|
|
to "\\n" (a benign line-ending normalization). None stays None; all-blank collapses to "".
|
|
|
|
Deliberately CHANNEL-only — video descriptions are left with their original formatting: they are
|
|
user-facing prose where paragraph breaks carry meaning, unlike a channel's metadata block."""
|
|
if text is None:
|
|
return None
|
|
kept = [line.rstrip() for line in text.splitlines() if line.strip()]
|
|
return "\n".join(kept)
|