peter
e995c664c6
GET /api/channels summed every subscribed channel's videos by reading each video's heap page (681 MB on prod): 51-61 ms warm, but 2.25 s on a first open and 6.4 s under EXPLAIN once the table had left memory. /feed/summary and /feed/years did the same (~140 ms warm, ~800 ms cold). Migration 0095 replaces ix_videos_channel_pub_desc with the same key INCLUDING the columns those queries read, so they are Index Only Scans over a 25 MB index. A test holds the queries and the index's column list together. The admin Configuration page did a db.get per setting (two for an unset one): it now reads every override in one query (~40 ms of ~55 on prod).
48 lines
2.6 KiB
Python
48 lines
2.6 KiB
Python
"""videos: the per-channel index covers what the channel manager and the feed sidebar read (E31 S4d)
|
||
|
||
The channel manager (`GET /api/channels`) sums each subscribed channel's videos: count, last upload, total
|
||
duration. The feed sidebar (`/feed/summary`, `/feed/years`) counts the reader's videos per channel and per
|
||
year under the default filters: available, not via explore, not a short, not live. Every one of them found
|
||
its rows through `ix_videos_channel_pub_desc` (channel_id, published_at DESC NULLS LAST, id) and then read
|
||
each video's heap page for the rest: ~270k rows at ~2.5 KB each, 681 MB on prod. That stays in cache only
|
||
while nothing else needs the memory. Measured on prod (2026-10-02): `/api/channels` 51–61 ms warm, but
|
||
2.25 s and 1.96 s on a first open in the browser and 6.4 s under EXPLAIN with 30k blocks read from disk;
|
||
`/feed/summary` and `/feed/years` ~140 ms warm, ~800 ms cold.
|
||
|
||
The same key with those columns INCLUDEd makes them Index Only Scans. On a prod-copy localdev, after a
|
||
VACUUM, the manager's sums read 0 heap pages and the summary reads the 25 MB index instead of the heap.
|
||
The visibility map is what lets them skip the heap; prod's autovacuum kept it complete (relallvisible ==
|
||
relpages). The key columns are unchanged, so every reader of the old index (the per-channel feed, the FK
|
||
cascade, the sync upsert, channel detail) rides this one. The old index is dropped.
|
||
tests/test_channel_list_index_only.py holds the queries and this column list together.
|
||
|
||
CONCURRENTLY (autocommit_block) builds it without blocking a live prod. IF (NOT) EXISTS keeps the drift
|
||
test's downgrade → re-upgrade round trip idempotent.
|
||
"""
|
||
|
||
from alembic import op
|
||
|
||
revision: str = "0095_videos_channel_cover_index"
|
||
down_revision: str | None = "0094_plex_genres_hu_everywhere"
|
||
branch_labels: str | None = None
|
||
depends_on: str | None = None
|
||
|
||
|
||
def upgrade() -> None:
|
||
with op.get_context().autocommit_block():
|
||
op.execute(
|
||
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_channel_pub_cover "
|
||
"ON videos (channel_id, published_at DESC NULLS LAST, id) "
|
||
"INCLUDE (duration_seconds, unavailable_since, via_explore, is_short, live_status)"
|
||
)
|
||
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_channel_pub_desc")
|
||
|
||
|
||
def downgrade() -> None:
|
||
with op.get_context().autocommit_block():
|
||
op.execute(
|
||
"CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_videos_channel_pub_desc "
|
||
"ON videos (channel_id, published_at DESC NULLS LAST, id ASC)"
|
||
)
|
||
op.execute("DROP INDEX CONCURRENTLY IF EXISTS ix_videos_channel_pub_cover")
|