fix(logs): deep-review round 2 — a stack names code, not data, and survives a failed write

- A failure keeps its traceback unless it is yt-dlp's or our own StatedFailure (a
  message that is the whole story). A plain RuntimeError, RecursionError or OSError
  is also what a bug of ours raises, and its stack is the only lead.
- The stack is scrubbed link by link (logs.scrubbed_stack): a chained DownloadError
  under a DB error no longer prints yt-dlp's text unscrubbed.
- Engine hide_parameters: a DB error's message no longer carries the title or path
  it was writing.
- The download and edit failure lines go out even when their DB write fails.
- The worker's child_exit/crash/supervisor lines say download.job.failed only when
  the job failed there; a crash the DB will not record no longer ends the thread.
  Stopping a paused child is download.child.stopped, not a second abort.
- log_safe also covers generic ids with spaces, playlist titles and bare ids.
- The ffmpeg image steps log the failure kind, not the argv with its path.
- The consent contract accepts only the body of a plain consented_trace() check,
  and any level that may be TRACE.
- Tests for each; every one mutation-checked.
This commit is contained in:
2026-09-18 22:50:37 +02:00
parent 54d19cc386
commit 0b672ed6e7
14 changed files with 729 additions and 138 deletions
+24 -7
View File
@@ -26,16 +26,29 @@ from app import logs
log = logs.get_logger(__name__)
# yt-dlp names what it is working on in three shapes, and each one is a row of someone's download
# history: the source URL, the `[extractor] <id>:` prefix on nearly every warning and error (for
# the generic extractor the "id" is the URL's own basename), and the file path it writes to, which
# `storage.rel_path` builds from the channel and the title. The job id is already on every line
# (`logs.set_job` in the download child), so none of the three adds anything an operator needs.
# What stays: the host (it says which extractor is failing) and the extractor tag.
# yt-dlp names what it is working on in these shapes, and each one is a row of someone's download
# history: the source URL, the `[extractor] <id>:` prefix on nearly every warning and error, the
# file path it writes to (`storage.rel_path` builds it from the channel and the title), a
# playlist's title, and an id quoted mid-sentence. The job id is already on every line
# (`logs.set_job` in the download child), so none of them adds anything an operator needs. What
# stays: the host (it says which extractor is failing) and the extractor tag. Every pattern below
# is a shape measured in yt-dlp's own source (2026.07.04), pinned in test_ytdlp_logging.py.
_URL_RE = re.compile(r"https?://([^/\s]+)\S*")
# `Destination:` is yt-dlp's own word in `[download] Destination: <path>`, not an id — the path
# after it is what the third pattern takes out.
# after it is what `_PATH_RE` takes out.
_ID_PREFIX_RE = re.compile(r"(\[[\w:.-]+\] )(?!Destination:)[^\s:\[\]]+:")
# The generic extractor's "id" is the URL's basename, percent-DECODED: `My%20Holiday.mp4` is
# `My Holiday` — spaces and all, which the pattern above stops at. Only this extractor: every
# other one names a site id, and letting the prefix run to `: ` everywhere would take the words
# out of lines like `[VideoConvertor] Converting video from webm to mp4; Destination: …`.
_GENERIC_ID_RE = re.compile(r"(\[generic\] )(?!Destination:)[^\[\]\n]+?:(?= |$)", re.M)
# `[youtube:tab] Playlist <title>: Downloading 5 items` — the title is not an id, and has spaces.
_PLAYLIST_TITLE_RE = re.compile(r"(\] Playlist ).+?(?=: Downloading )")
# `[download] Finished downloading playlist: <title>`, `[download] Downloading playlist: <title>`,
# `… Skipping already downloaded playlist: <title>` — the title runs to the end of the line.
_TITLE_TAIL_RE = re.compile(r"(playlist: |\[download\] Downloading \w+: ).*", re.I)
# An id quoted mid-sentence, not as a prefix.
_BARE_ID_RE = re.compile(r"((?:subtitles not available for|Downloading just video) )[^\s,;]+")
# Two segments or more, and not glued to a word: `www.youtube.com/…` is what the URL pattern
# leaves behind, and it must survive this one.
_PATH_RE = re.compile(r"(?<![\w.…])/(?:[^/\s\"']+/)+[^/\s\"']*")
@@ -48,6 +61,10 @@ def log_safe(text: object) -> str:
it would put back one line later exactly what the adapter took out."""
text = _URL_RE.sub(lambda m: f"{m.group(1)}/…", str(text))
text = _ID_PREFIX_RE.sub(r"\1…:", text)
text = _GENERIC_ID_RE.sub(r"\1…:", text)
text = _PLAYLIST_TITLE_RE.sub(r"\1…", text)
text = _TITLE_TAIL_RE.sub(r"\1…", text)
text = _BARE_ID_RE.sub(r"\1…", text)
return _PATH_RE.sub("…", text)