perf(feed): a shuffle walks a stored random key instead of sorting the whole scope

Every video carries a random number drawn once (videos.shuffle_key, migration 0118, indexed with
id). A seed picks a point on that ring: the feed's Shuffle sort pages along it and wraps, and the
Home surprise reads a 500-video window of it and shuffles that window by the seed, so no video is
favoured for sitting after a wide gap. The md5(id || seed) order read and sorted the whole feed
scope on every call: ~350 ms on prod per surprise, 381-767 ms per Shuffle page on localdev, now
14 ms and 4 ms.
This commit is contained in:
2026-11-07 04:00:00 +01:00
parent 9310ff85bf
commit 25aed6635f
7 changed files with 328 additions and 185 deletions
+4
View File
@@ -10,6 +10,7 @@ from sqlalchemy import (
Boolean,
Computed,
DateTime,
Float,
ForeignKey,
Integer,
JSON,
@@ -217,6 +218,9 @@ class Video(Base, TimestampMixin):
last_checked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
unavailable_since: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
unavailable_reason: Mapped[str | None] = mapped_column(String(24))
# A random number in [0, 1), drawn once when the row is written: the ring a seeded shuffle walks
# (`feedquery.shuffle_walk`, index ix_videos_shuffle_key on (shuffle_key, id), migration 0118).
shuffle_key: Mapped[float] = mapped_column(Float, nullable=False, server_default=func.random())
@hybrid_property
def length_seconds(self) -> int | None: