fix(logs): deep-review round 1 — say each failure once, and scrub it everywhere

- yt-dlp's text is scrubbed wherever it is logged, not only in the adapter:
  `formats.log_safe` takes out the URL, the `[extractor] <id>:` prefix and
  file paths, and the job-failed, refused, edit-failed and backfill lines use
  it. Done lines log the asset id, not the title-built path. The PII contract
  now bans `rel`/`rel_path`/`title`.
- A traceback goes with a download/edit failure only when it is our own code
  failing; for yt-dlp's DownloadError, our RuntimeErrors and OSErrors its foot
  is the unscrubbed message.
- yt-dlp's `error` maps to WARNING: it fires on attempts the fallback client
  set recovers from; the job's verdict is `download.job.failed`.
- The sink groups a bare `%s` pass-through by its text (numbers folded), and
  a process's last batch closes the repeat window, so a short-lived download
  child still reports what it dropped.
- Worker-side failures (stall, wall-clock, no asset, cached asset error) log
  `download.job.failed` with a cause; they left no line at all before. The
  crash, supervisor-error and non-zero-exit lines carry the code too.
- An edit failure is one coded line in `_fail_edit`, for every path.
- `logs.get_logger("__main__")` resolves to the `-m` module, so the worker
  and the download child keep their names.
- TRACE lines carrying user content are gated on `logs.consented_trace()`,
  not on the process level; a contract test pins it.
- The OAuth token refresh goes through `_upstream_failure`; transport errors
  carry reason `transport`. Subscribe recognises a duplicate by reason code
  only (the prose fallback was dead).
- Once-only guards for a stale sysconfig override and a refusing HLS session
  directory; item_detail splits Plex failures from our own parse bugs.
- The playlist breaker line names the playlist and the action; comments that
  were false (runner noqa, `_merge_vtt`, sink grouping) are corrected.
This commit is contained in:
2026-09-18 03:16:15 +02:00
parent 04b58e5c6f
commit 54d19cc386
22 changed files with 834 additions and 141 deletions
+43 -1
View File
@@ -59,7 +59,10 @@ def sink():
("debug", "TRACE"),
("info", "DEBUG"),
("warning", "WARNING"),
("error", "ERROR"),
# yt-dlp calls `error` from `trouble()` just before it raises, on every failed ATTEMPT —
# the pipeline's fallback client set then often succeeds. The job's verdict is the
# pipeline's `download.job.failed`, not this.
("error", "WARNING"),
],
)
def test_each_ytdlp_level_lands_where_it_belongs(sink, method, level):
@@ -83,6 +86,45 @@ def test_the_source_url_does_not_travel_with_the_explanation(sink):
assert "Requested format is not available" in line["msg"]
@pytest.mark.parametrize(
("line", "kept", "gone"),
[
# The shape nearly every yt-dlp warning and error has: no URL, the id as a prefix.
(
"ERROR: [youtube] dQw4w9WgXcQ: Video unavailable",
"ERROR: [youtube] …: Video unavailable",
"dQw4w9WgXcQ",
),
# The generic extractor's "id" is the URL's own basename.
(
"WARNING: [generic] my-private-holiday-clip: Falling back on generic information",
"WARNING: [generic] …: Falling back on generic information",
"my-private-holiday-clip",
),
# `Destination` is yt-dlp's word, not an id; the path after it is built from the title.
(
"[download] Destination: /downloads/Some_Channel/Season_2024/A_Private_Title_a7.mp4",
"[download] Destination: …",
"A_Private_Title",
),
],
)
def test_what_names_the_download_comes_out_of_real_ytdlp_shapes(sink, line, kept, gone):
formats._YDL_LOG.warning(line)
(obj,) = sink.objects()
assert obj["msg"] == kept
assert gone not in json.dumps(obj)
def test_a_download_error_is_scrubbed_the_same_way_wherever_it_is_logged():
"""yt-dlp raises the text it has just logged, and the pipeline logs the exception again one
line later (`download.job.failed`). The same scrub, or that line puts the URL back."""
text = "ERROR: Unsupported URL: https://example.org/private/path?id=123"
assert formats.log_safe(text) == "ERROR: Unsupported URL: example.org/…"
def test_a_line_with_no_url_is_left_alone(sink):
formats._YDL_LOG.warning("nsig extraction failed: Some formats may be missing")