fix(logs): deep-review round 1 — say each failure once, and scrub it everywhere

- yt-dlp's text is scrubbed wherever it is logged, not only in the adapter:
  `formats.log_safe` takes out the URL, the `[extractor] <id>:` prefix and
  file paths, and the job-failed, refused, edit-failed and backfill lines use
  it. Done lines log the asset id, not the title-built path. The PII contract
  now bans `rel`/`rel_path`/`title`.
- A traceback goes with a download/edit failure only when it is our own code
  failing; for yt-dlp's DownloadError, our RuntimeErrors and OSErrors its foot
  is the unscrubbed message.
- yt-dlp's `error` maps to WARNING: it fires on attempts the fallback client
  set recovers from; the job's verdict is `download.job.failed`.
- The sink groups a bare `%s` pass-through by its text (numbers folded), and
  a process's last batch closes the repeat window, so a short-lived download
  child still reports what it dropped.
- Worker-side failures (stall, wall-clock, no asset, cached asset error) log
  `download.job.failed` with a cause; they left no line at all before. The
  crash, supervisor-error and non-zero-exit lines carry the code too.
- An edit failure is one coded line in `_fail_edit`, for every path.
- `logs.get_logger("__main__")` resolves to the `-m` module, so the worker
  and the download child keep their names.
- TRACE lines carrying user content are gated on `logs.consented_trace()`,
  not on the process level; a contract test pins it.
- The OAuth token refresh goes through `_upstream_failure`; transport errors
  carry reason `transport`. Subscribe recognises a duplicate by reason code
  only (the prose fallback was dead).
- Once-only guards for a stale sysconfig override and a refusing HLS session
  directory; item_detail splits Plex failures from our own parse bugs.
- The playlist breaker line names the playlist and the action; comments that
  were false (runner noqa, `_merge_vtt`, sink grouping) are corrected.
This commit is contained in:
2026-09-18 03:16:15 +02:00
parent 04b58e5c6f
commit 54d19cc386
22 changed files with 834 additions and 141 deletions
+34
View File
@@ -90,6 +90,40 @@ def test_a_flood_of_one_message_is_stored_up_to_the_limit_and_then_summarised(st
assert notice.attrs["template"] == "RSS poll skipped for channel %s: HTTP 404"
def test_a_pass_through_template_groups_by_its_text_not_by_the_bare_placeholder(stored):
"""`"%s"` is how the yt-dlp adapter hands over yt-dlp's own line, and as a key it says nothing:
every distinct warning would share one bucket and the thirty-first — whatever it said — would be
dropped as a "repeat" of the first. The text groups instead, numbers folded, so a retry storm is
still one repeat and the warning that explains the failure still gets through."""
sink, log, rows = stored
limit = logsink.REPEATS_PER_MINUTE
for i in range(limit + 10):
log.warning("%s", f"[download] Got error: timed out. Retrying fragment {i} (1/10)...")
log.warning("%s", "[youtube] …: Requested format is not available")
stored_rows = rows()
assert len(stored_rows) == limit + 1
assert stored_rows[-1].msg == "[youtube] …: Requested format is not available"
sink._repeats.started -= logsink.REPEAT_WINDOW_SECONDS
notice = rows()[-1]
assert notice.attrs["suppressed"] == 10
assert notice.attrs["template"].startswith(
"[download] Got error: timed out. Retrying fragment #"
)
def test_a_process_that_ends_inside_the_minute_still_says_what_it_dropped(stored):
"""Every download child lives well under the limiter's minute. Its last batch closes the window
early, or the repeats it dropped would go unreported."""
sink, log, rows = stored
for i in range(logsink.REPEATS_PER_MINUTE + 3):
log.warning("RSS poll skipped for channel %s: HTTP 404", f"UC{i}")
rows()
sink.write_pending(final=True)
notice = rows()[-1]
assert notice.event == "log.repeats_suppressed"
assert notice.attrs["suppressed"] == 3
def test_request_lines_are_never_treated_as_a_flood(stored):
_sink, log, rows = stored
for _ in range(logsink.REPEATS_PER_MINUTE + 5):