Channel, Subscription, Video, SearchFind, ExploredChannel and BlockedChannel (and LIVE_OR_UPCOMING)
leave app/models.py for app/catalog/models.py; app.tables registers them. The User↔Subscription
relationship names its other side as a type only. No schema change.
A nullable published_at forced the feed keyset into a non-sargable
`... OR published_at IS NULL` disjunction: with the 0067 index the planner still
applied it as a FILTER and scanned from the top of the index, so a deep `newest`
page discarded every earlier row (O(depth)).
- Migration 0068: backfill any NULLs (0 in prod) -> SET NOT NULL + DEFAULT now().
- Model: Video.published_at is now non-Optional, nullable=False, server_default now().
- feedquery: the `pub` key is nullable=False, and keyset_predicate is rebuilt
recursively so a non-nullable leading key gets a non-strict range bound
(`published_at <= v`) as an AND conjunct — an index range Cond instead of a
filter. Nullable keys keep the NULL-aware OR form (a `<= v` bound would drop
their NULL group). Equivalence proven by test_feed_keyset (all 13 sorts,
cursor-walk == unpaged).
- Ingest paths (stub_from_playlist_item, apply_video_details, playlists sync)
fall back to now() so they never write an explicit NULL.
Measured on the prod-sized localdev reseed at a ~50k-deep cursor (0067 index +
prod planner): OLD `Rows Removed by Filter: 50001`, 25,687 buffers / 9.16 ms ->
NEW `Index Cond: published_at <= v`, 1 row filtered, 47 buffers / 0.037 ms. SET
NOT NULL on 257k rows: 0.32s. Gate green (334 pytest).
Address the /code-review findings on the S2 test-net:
- as_user no longer sets a global `lambda: user` override that two users in one
test would collide on; the shared current_user override now resolves the caller
from an x-test-user header, and each returned client tags its own requests, so
as_user(a)/as_user(b) are independent (#1).
- override_dep records the prior value per dependency key and restores it on
teardown instead of a blanket dependency_overrides.clear(), so fixtures compose
without clobbering each other (#5).
- a _configured fixture bakes the post-setup state into client/as_user, and the
per-test inline monkeypatch / autouse duplicates are removed (#4).
- the feed keyset walk asserts all seeded rows surfaced before comparing, so a
seeding regression can't make the sorts pass vacuously (#3).
The feed's cursor paging (sort_keys/order_by/keyset_predicate/encode/decode) is
the NULLS-LAST + tiebreaker algebra where a page-boundary bug hides as a
duplicated or skipped row on the busiest endpoint, and had zero tests. Seed 7
videos on 2 channels with duplicate sort values and NULLs across
published_at/view_count/duration/title/subscriber_count, then assert — for every
sort — that walking the feed in pages of 2 via the cursor reproduces exactly the
order a single unpaged query returns, with no duplicates.
peter