perf(db): the channel manager and the feed sidebar read an index, not the video heap

GET /api/channels summed every subscribed channel's videos by reading each video's heap page (681 MB on prod): 51-61 ms warm, but 2.25 s on a first open and 6.4 s under EXPLAIN once the table had left memory. /feed/summary and /feed/years did the same (~140 ms warm, ~800 ms cold). Migration 0095 replaces ix_videos_channel_pub_desc with the same key INCLUDING the columns those queries read, so they are Index Only Scans over a 25 MB index. A test holds the queries and the index's column list together. The admin Configuration page did a db.get per setting (two for an unset one): it now reads every override in one query (~40 ms of ~55 on prod).
2026-10-02 03:16:37 +02:00
parent 20010178b0
commit e995c664c6
7 changed files with 220 additions and 26 deletions
@@ -0,0 +1,47 @@
+"""videos: the per-channel index covers what the channel manager and the feed sidebar read (E31 S4d)
+
+The channel manager (`GET /api/channels`) sums each subscribed channel's videos: count, last upload, total
+duration. The feed sidebar (`/feed/summary`, `/feed/years`) counts the reader's videos per channel and per
+year under the default filters: available, not via explore, not a short, not live. Every one of them found
+its rows through `ix_videos_channel_pub_desc` (channel_id, published_at DESC NULLS LAST, id) and then read
+each video's heap page for the rest: ~270k rows at ~2.5 KB each, 681 MB on prod. That stays in cache only
+while nothing else needs the memory. Measured on prod (2026-10-02): `/api/channels` 51–61 ms warm, but
+2.25 s and 1.96 s on a first open in the browser and 6.4 s under EXPLAIN with 30k blocks read from disk;
+`/feed/summary` and `/feed/years` ~140 ms warm, ~800 ms cold.
+
+The same key with those columns INCLUDEd makes them Index Only Scans. On a prod-copy localdev, after a
+VACUUM, the manager's sums read 0 heap pages and the summary reads the 25 MB index instead of the heap.
+The visibility map is what lets them skip the heap; prod's autovacuum kept it complete (relallvisible ==
+relpages). The key columns are unchanged, so every reader of the old index (the per-channel feed, the FK
+cascade, the sync upsert, channel detail) rides this one. The old index is dropped.
+tests/test_channel_list_index_only.py holds the queries and this column list together.
+
+CONCURRENTLY (autocommit_block) builds it without blocking a live prod. IF (NOT) EXISTS keeps the drift
+test's downgrade → re-upgrade round trip idempotent.
+"""
+
+from alembic import op
+
+revision: str = "0095_videos_channel_cover_index"
+down_revision: str | None = "0094_plex_genres_hu_everywhere"
+branch_labels: str | None = None
+depends_on: str | None = None
+
+
+def upgrade() -> None:
+    with op.get_context().autocommit_block():
+        op.execute(
+            "CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_channel_pub_cover "
+            "ON videos (channel_id, published_at DESC NULLS LAST, id) "
+            "INCLUDE (duration_seconds, unavailable_since, via_explore, is_short, live_status)"
+        )
+        op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_channel_pub_desc")
+
+
+def downgrade() -> None:
+    with op.get_context().autocommit_block():
+        op.execute(
+            "CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_channel_pub_desc "
+            "ON videos (channel_id, published_at DESC NULLS LAST, id ASC)"
+        )
+        op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_channel_pub_cover")