fix(downloads): heartbeat a slow re-encode so the stall watchdog can't false-kill it (E3 S2, B-sub-4)

The supervisor kills a job that writes nothing for download_stall_timeout_seconds
(150s) and labels it a "network" stall. But the ffmpeg progress writer only wrote
on an integer-percent CHANGE, so a re-encode that sits on one percent for minutes
(AV1/VP9 -> H.264 or a crop on the CPU-only prod host) produced no row write and
was killed mid-encode as "stalled" — and the retry stalled the same way.

Fix: the writer now heartbeats (`_touch_job`, bumping updated_at without touching
the progress bar) once per interval even when the whole percent is unchanged, so
a live-but-quiet re-encode keeps a fresh updated_at. Reorders the rate-limit gate
first so a burst still costs one DB op — either a progress write or a heartbeat.

Pure-logic test with an injected clock: three stuck-at-10% intervals write no new
progress but heartbeat each time. Full lane green, ruff clean.
This commit is contained in:
2026-08-18 21:34:36 +02:00
parent 7d89bc8b8e
commit e2362e1ed1
2 changed files with 25 additions and 1 deletions
+8 -1
View File
@@ -625,8 +625,15 @@ def _ffmpeg_progress_writer(job_id: int, phase: str, *, now=time.monotonic):
state = {"pct": -1}
def write(pct: float) -> None:
if not allow():
return
whole = int(pct)
if whole == state["pct"] or not allow():
if whole == state["pct"]:
# Same integer percent, but ffmpeg IS still working — a slow re-encode (AV1/VP9 -> H.264
# or a crop on the CPU-only host) can sit on one percent for minutes. Heartbeat the row's
# updated_at so the stall watchdog doesn't kill a live job as "stalled" (B-sub-4), without
# rewriting the unchanged progress bar.
_touch_job(job_id)
return
state["pct"] = whole
_set_job(job_id, progress=whole, phase=phase)