perf(downloads): de-prioritize worker transcodes + idle-back-off polls to stop LAN slowdown
The download/edit child (yt-dlp + ffmpeg) shares one LAN box with the API, Postgres, and Plex playback. A CPU re-encode pegged every core for ~30s and starved web requests + playback until it finished. Full-scope S4 mitigations: - Child self-de-prioritizes at startup (inherited by every ffmpeg/yt-dlp descendant): CPU nice increment + best-effort idle ionice class + an OMP_NUM_THREADS budget, all config-gated and independently disable-able. - Cap the browser-safe re-encode's encoder threads (`-threads`) so a transcode reserves a core for foreground work instead of claiming all of them. - Throttle the retention GC's whole-tree orphan sweep (yield IO after each batch of files) so its rglob can't monopolize disk IO in the API process. - Idle-back-off the Download Center (2s→15s when nothing queued/running) and UsageBar (2s→15s when running_jobs is 0) polls; both snap back to 2s on the next fetch after an enqueue (invalidateDownloads refetches immediately). Tests: encode_thread_cap (pin/reserve-core/edge), -threads flag presence, _deprioritize (nice/ionice/OMP branches), GC sweep throttle batching.
This commit is contained in:
@@ -186,6 +186,21 @@ class Settings(BaseSettings):
|
||||
download_supervisor_poll_seconds: int = 3
|
||||
# Grace between SIGTERM and SIGKILL when stopping a child (matches run_ffmpeg's escalation).
|
||||
download_kill_grace_seconds: int = 5
|
||||
# --- worker-child resource de-prioritization ---
|
||||
# The download/edit child (yt-dlp + ffmpeg) shares ONE LAN box with the API, Postgres, and Plex
|
||||
# playback. Without a priority handicap a CPU re-encode pegs every core for ~30s and starves web
|
||||
# requests + playback until it finishes. The child lowers its OWN priority at startup — inherited
|
||||
# by every ffmpeg/yt-dlp descendant — so any foreground work always wins. Each knob is best-effort
|
||||
# and independently disable-able with 0.
|
||||
download_child_nice: int = 10 # CPU nice increment applied to the child; 0 = don't renice
|
||||
download_child_ionice_class: int = 3 # best-effort IO class (3 = idle); 0 = leave IO untouched
|
||||
# Cap the browser-safe re-encode's encoder threads so a transcode can't claim every core. 0 =
|
||||
# reserve one core for foreground work (max(1, cpu_count - 1)); >0 pins exactly that many.
|
||||
download_ffmpeg_threads: int = 0
|
||||
# Throttle the retention GC's whole-tree orphan file sweep so its rglob can't monopolize disk IO
|
||||
# in the API process: sleep `pause_ms` after every `batch` files touched. pause_ms=0 disables.
|
||||
download_gc_sweep_pause_ms: int = 20
|
||||
download_gc_sweep_batch: int = 400
|
||||
# Total cache size cap across ALL users (bytes). When >0 and exceeded, the GC evicts the
|
||||
# least-recently-accessed ready assets until under it. 0 = unlimited (disk is the limit).
|
||||
download_total_max_bytes: int = 0
|
||||
|
||||
Reference in New Issue
Block a user