perf(feed): published_at NOT NULL + range-anchored keyset for deep pages (E2 S2, PERF-15)

A nullable published_at forced the feed keyset into a non-sargable
`... OR published_at IS NULL` disjunction: with the 0067 index the planner still
applied it as a FILTER and scanned from the top of the index, so a deep `newest`
page discarded every earlier row (O(depth)).

- Migration 0068: backfill any NULLs (0 in prod) -> SET NOT NULL + DEFAULT now().
- Model: Video.published_at is now non-Optional, nullable=False, server_default now().
- feedquery: the `pub` key is nullable=False, and keyset_predicate is rebuilt
  recursively so a non-nullable leading key gets a non-strict range bound
  (`published_at <= v`) as an AND conjunct — an index range Cond instead of a
  filter. Nullable keys keep the NULL-aware OR form (a `<= v` bound would drop
  their NULL group). Equivalence proven by test_feed_keyset (all 13 sorts,
  cursor-walk == unpaged).
- Ingest paths (stub_from_playlist_item, apply_video_details, playlists sync)
  fall back to now() so they never write an explicit NULL.

Measured on the prod-sized localdev reseed at a ~50k-deep cursor (0067 index +
prod planner): OLD `Rows Removed by Filter: 50001`, 25,687 buffers / 9.16 ms ->
NEW `Index Cond: published_at <= v`, 1 row filtered, 47 buffers / 0.037 ms. SET
NOT NULL on 257k rows: 0.32s. Gate green (334 pytest).
This commit is contained in:
2026-08-18 04:30:07 +02:00
parent 3af3799b25
commit 8c08ea3133
6 changed files with 78 additions and 15 deletions
+7 -2
View File
@@ -261,8 +261,13 @@ class Video(Base, TimestampMixin):
title: Mapped[str | None] = mapped_column(Text) # normalized for display (see app.titles)
original_title: Mapped[str | None] = mapped_column(Text) # raw YouTube title, preserved
description: Mapped[str | None] = mapped_column(Text)
published_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), index=True
# NOT NULL (migration 0068): in reality every video carries a publish date, and a nullable key
# forced the feed's keyset predicate into a non-sargable `... OR published_at IS NULL` disjunction
# that blocked the ix_videos_published_at_desc range (deep pages scanned from the top and filtered —
# PERF-15). server_default now() is the safety net for the rare date-less ingest (the enrich fills
# the real date); the ix_videos_published_at index stays for the `oldest` sort.
published_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), index=True, nullable=False, server_default=func.now()
)
thumbnail_url: Mapped[str | None] = mapped_column(String(1024))
duration_seconds: Mapped[int | None] = mapped_column(Integer)