peter
4aaaaa7860
The code cited its plan instead of saying why: `E28 review J4`, `(R6 S3b)`, `S1 justified…`,
`deep-review round 5`. That plan lives in the private ops repo, so for every reader of this one the
citation was noise, and a sentence whose subject was the plan step ("round 2 showed…") hid its
lesson behind an id. Each one is gone and the "why" kept: a bare citation is deleted, a sentence
carrying a lesson is reworded around it ("a later attempt showed…").
Measured (`scripts/audit_metrics.py`, product code): 652 lines in 270 files carried an epic code by
the definition the next commit holds (507 / 241 by the old, two-digit-only one) -> 0. Across all
code, tests, scripts and configs: ~1100 epic-code lines plus ~320 sprint/round ids.
Behaviour unchanged, checked mechanically: every changed .py file has the same AST once docstrings
are blanked; css/sh/toml/yml/json files are identical once comments are stripped; the only
non-comment changes in TS/JS are test titles, one description string in a contract table, and five
ESLint messages, whose text is all the gates snapshot diff holds. Not touched: Markdown (docs and
the changelog keep their history), Alembic migrations, TV episode notation (`S2 · E5`), DESIGN.md
ids (`C1`, `B4`, `S3h`), tracker ids (`SEC-16`, `DX-17`).
One stale claim fixed on the way: test_hls_reaper's docstring said the player checkpoints "every
10 s whether or not it is playing", which plex/stream.py calls wrong; it now says what the code does.
84 lines
3.8 KiB
Python
84 lines
3.8 KiB
Python
"""Alphabetical order in the reader's language.
|
|
|
|
Why not the database default: the Postgres image is Alpine, and musl's `en_US.utf8` collation is BYTE
|
|
order — every accented initial sorted after "Z" ("Zorró < Életrajzi < Ötödik"), in every A→Z list the
|
|
server orders. Postgres ships the ICU collations on the same image (dev, test, prod alike), so a text
|
|
sort names one: Hungarian puts "Ö" after "O" and "Cs" after "Cu", English folds the accents in.
|
|
|
|
`collated` wraps an ORDER BY expression — and because a keyset cursor compares with the SAME expression
|
|
(`feedquery.sort_keys`), the cursor pages in the same order it sorted in. `sorted_words` orders a Python
|
|
list the same way through the database, so a list built in Python matches one built in SQL exactly.
|
|
|
|
The feed's title sorts ride indexes built in these collations (migration 0096), and an index keeps the
|
|
order of the ICU that built it. `postgres:16-alpine` is a floating tag: an image update can bring a newer
|
|
ICU, after which those indexes disagree with the comparisons a query makes (a keyset page skips or
|
|
repeats a title). `heal_drift` rebuilds them at startup once Postgres reports the version moved."""
|
|
|
|
from typing import Any
|
|
|
|
from sqlalchemy import Engine, SQLColumnExpression, column, func, select, text, values
|
|
from sqlalchemy.orm import Session
|
|
from sqlalchemy.sql.elements import ColumnElement
|
|
from sqlalchemy.types import String
|
|
|
|
from app import logs
|
|
from app.mailtext import normalize
|
|
|
|
log = logs.get_logger(__name__)
|
|
|
|
ICU: dict[str, str] = {"en": "en-x-icu", "hu": "hu-x-icu"}
|
|
|
|
|
|
def collation(lang: str | None) -> str:
|
|
"""The ICU collation for a UI language; anything unknown sorts as English."""
|
|
return ICU[normalize(lang)]
|
|
|
|
|
|
def collated(expr: SQLColumnExpression[Any], lang: str | None) -> ColumnElement[Any]:
|
|
"""`expr` compared in the reader's alphabet. Case-folded, so "apple" and "Apple" sit together."""
|
|
return func.lower(expr).collate(collation(lang))
|
|
|
|
|
|
def sorted_words(db: Session, words: list[str], lang: str | None) -> list[str]:
|
|
"""`words` in the reader's alphabetical order, by the same ICU collation as `collated`."""
|
|
if not words:
|
|
return []
|
|
w = values(column("w", String), name="words").data([(x,) for x in words])
|
|
return list(db.scalars(select(w.c.w).order_by(collated(w.c.w, lang), w.c.w)))
|
|
|
|
|
|
def heal_drift(engine: Engine) -> list[str]:
|
|
"""Rebuild every index on an ICU collation whose library version moved, then record the new version.
|
|
Returns the rebuilt indexes. CONCURRENTLY: the feed keeps reading while it runs."""
|
|
with engine.connect().execution_options(isolation_level="AUTOCOMMIT") as conn:
|
|
drifted = conn.execute(
|
|
text(
|
|
"SELECT oid, collname FROM pg_collation WHERE collname = ANY(:names) "
|
|
"AND collversion IS DISTINCT FROM pg_collation_actual_version(oid)"
|
|
),
|
|
{"names": list(ICU.values())},
|
|
).all()
|
|
if not drifted:
|
|
return []
|
|
indexes = list(
|
|
conn.execute(
|
|
text(
|
|
"SELECT DISTINCT d.objid::regclass::text FROM pg_depend d "
|
|
"JOIN pg_class c ON c.oid = d.objid AND c.relkind = 'i' "
|
|
"WHERE d.classid = 'pg_class'::regclass "
|
|
"AND d.refclassid = 'pg_collation'::regclass AND d.refobjid = ANY(:oids)"
|
|
),
|
|
{"oids": [row.oid for row in drifted]},
|
|
).scalars()
|
|
)
|
|
log.warning(
|
|
"ICU collation version changed (%s): rebuilding %d indexes",
|
|
", ".join(row.collname for row in drifted),
|
|
len(indexes),
|
|
)
|
|
for name in indexes:
|
|
conn.execute(text(f"REINDEX INDEX CONCURRENTLY {name}"))
|
|
for row in drifted:
|
|
conn.execute(text(f'ALTER COLLATION "{row.collname}" REFRESH VERSION'))
|
|
return indexes
|