perf(db): each feed sort rides an index, and an ICU upgrade rebuilds the title ones

The views, duration and title sorts had no index: every page read the whole videos heap and kept a
top-N (prod 350 ms a page, 550-780 for title). Migration 0096 adds one per direction and, for title,
per reader collation, in the exact ORDER BY feedquery builds. Localdev prod copy: plain sorts
180-310 ms -> ~1 ms, page 20 3-5 ms, narrow filters unchanged. view_count/duration are only written
by the enrich, which already rewrites indexed columns, so no HOT update is lost.

The title indexes are the first on an ICU collation, and postgres:16-alpine floats: collation.heal_drift
rebuilds them (CONCURRENTLY) at startup when Postgres reports the ICU version moved.
test_feed_sort_indexes holds every videos sort to an index and checks the rebuild.
This commit is contained in:
2026-10-02 21:17:41 +02:00
parent 540822d2b0
commit d928d50686
5 changed files with 226 additions and 4 deletions
+18 -2
View File
@@ -24,11 +24,12 @@ from sqlalchemy.exc import DataError
from starlette.middleware.sessions import SessionMiddleware
from app import bodylimit
from app import collation
from app import logsink
from app import state
from app import sysconfig
from app.config import settings
from app.db import SessionLocal
from app.db import SessionLocal, engine
from app.plex import stream as plex_stream
from app.plex import subtitles as plex_subtitles
from app.routes import (
@@ -97,6 +98,9 @@ async def lifespan(app: FastAPI):
# owns, so racing it against early playback is safe.
sweep = asyncio.create_task(_sweep_hls_orphans())
reaper = asyncio.create_task(_reap_hls_sessions())
# Fire and forget as well: a rebuild after an ICU upgrade takes seconds, and the title sorts serve
# meanwhile (CONCURRENTLY) in the old index's order, as they did before the restart.
heal = asyncio.create_task(_heal_collation_drift())
start_scheduler()
try:
yield
@@ -106,7 +110,7 @@ async def lifespan(app: FastAPI):
# pending ("Task was destroyed but it is pending!" — noise that reads like a bug during
# triage). Cancelling does NOT interrupt the rmtree thread itself; this only makes our own
# bookkeeping honest.
for task in (sweep, reaper):
for task in (sweep, reaper, heal):
task.cancel()
with suppress(asyncio.CancelledError):
await task
@@ -139,6 +143,18 @@ async def _reap_hls_sessions() -> None:
log.exception("HLS session reap failed")
async def _heal_collation_drift() -> None:
"""Background startup check of the ICU collation the title-sort indexes are built in (see
`collation.heal_drift`). A failure is logged, never fatal: the sorts still answer, only possibly in
the old order."""
try:
await asyncio.to_thread(collation.heal_drift, engine)
except asyncio.CancelledError:
raise
except Exception:
log.exception("ICU collation drift check failed")
async def _sweep_hls_orphans() -> None:
"""Background half of the startup HLS sweep (see lifespan). Owns its own errors: a failure here
must never take the app down, and a cancelled shutdown is not one."""