fix(logs): deep-review round 1 — say each failure once, and scrub it everywhere
- yt-dlp's text is scrubbed wherever it is logged, not only in the adapter:
`formats.log_safe` takes out the URL, the `[extractor] <id>:` prefix and
file paths, and the job-failed, refused, edit-failed and backfill lines use
it. Done lines log the asset id, not the title-built path. The PII contract
now bans `rel`/`rel_path`/`title`.
- A traceback goes with a download/edit failure only when it is our own code
failing; for yt-dlp's DownloadError, our RuntimeErrors and OSErrors its foot
is the unscrubbed message.
- yt-dlp's `error` maps to WARNING: it fires on attempts the fallback client
set recovers from; the job's verdict is `download.job.failed`.
- The sink groups a bare `%s` pass-through by its text (numbers folded), and
a process's last batch closes the repeat window, so a short-lived download
child still reports what it dropped.
- Worker-side failures (stall, wall-clock, no asset, cached asset error) log
`download.job.failed` with a cause; they left no line at all before. The
crash, supervisor-error and non-zero-exit lines carry the code too.
- An edit failure is one coded line in `_fail_edit`, for every path.
- `logs.get_logger("__main__")` resolves to the `-m` module, so the worker
and the download child keep their names.
- TRACE lines carrying user content are gated on `logs.consented_trace()`,
not on the process level; a contract test pins it.
- The OAuth token refresh goes through `_upstream_failure`; transport errors
carry reason `transport`. Subscribe recognises a duplicate by reason code
only (the prose fallback was dead).
- Once-only guards for a stale sysconfig override and a refusing HLS session
directory; item_detail splits Plex failures from our own parse bugs.
- The playlist breaker line names the playlist and the action; comments that
were false (runner noqa, `_merge_vtt`, sink grouping) are corrected.
This commit is contained in:
@@ -37,7 +37,11 @@ def _error_reason(resp: httpx.Response) -> str:
|
||||
body; the `message` beside it is prose, and upstream prose can quote the request back — a
|
||||
search term, a playlist title."""
|
||||
try:
|
||||
errors = resp.json().get("error", {}).get("errors") or []
|
||||
error = resp.json().get("error")
|
||||
# The OAuth token endpoint's shape: `{"error": "invalid_grant", "error_description": …}`.
|
||||
if isinstance(error, str):
|
||||
return error or "<none>"
|
||||
errors = (error or {}).get("errors") or []
|
||||
reasons = ",".join(e.get("reason", "") for e in errors if e.get("reason"))
|
||||
# silent-ok: the sentinel it returns goes straight into the caller's warning line, which
|
||||
# is the report. (Any non-JSON body — an HTML error page from a proxy.)
|
||||
@@ -47,12 +51,13 @@ def _error_reason(resp: httpx.Response) -> str:
|
||||
|
||||
|
||||
def _upstream_failure(method: str, path: str, resp: httpx.Response) -> YouTubeError:
|
||||
"""Report one upstream failure and build the error to raise. Every failing YouTube call goes
|
||||
through here, so what may appear in a log line cannot drift apart between the read, the write
|
||||
and the delete path — and the error a caller raises carries the reason code rather than the
|
||||
body, because callers put it in a log line (`routes/channels.py`) and in a 422 answered to the
|
||||
user. The body itself is TRACE-only: it is the request quoted back, which the privacy policy
|
||||
keeps out of the log unless the user turned Teljes on for themselves."""
|
||||
"""Report one upstream failure and build the error to raise. Every YouTube response that is
|
||||
not a success goes through here — the Data API's reads, writes and deletes and the OAuth
|
||||
token refresh — so what may appear in a log line cannot drift apart between them, and the
|
||||
error a caller raises carries the reason code rather than the body, because callers put it in
|
||||
a log line (`routes/channels.py`) and in a 422 answered to the user. (A request that got no
|
||||
response at all is `_send`'s, with the reason `transport`.) The body is logged only under
|
||||
the requesting user's own Teljes consent: it is the request quoted back."""
|
||||
reason = _error_reason(resp)
|
||||
log.warning(
|
||||
"YouTube %s %s -> %s (%s)",
|
||||
@@ -68,12 +73,13 @@ def _upstream_failure(method: str, path: str, resp: httpx.Response) -> YouTubeEr
|
||||
reason=reason,
|
||||
),
|
||||
)
|
||||
log.log(
|
||||
logs.TRACE,
|
||||
"the upstream body behind it",
|
||||
# log-pii-ok: TRACE is the consented level, and this is what it is for
|
||||
extra=logs.event("youtube.api.body", body=resp.text[:500]),
|
||||
)
|
||||
if logs.consented_trace():
|
||||
log.log(
|
||||
logs.TRACE,
|
||||
"the upstream body behind it",
|
||||
# log-pii-ok: only under this user's own Teljes consent, which is what it is for
|
||||
extra=logs.event("youtube.api.body", body=resp.text[:500]),
|
||||
)
|
||||
return YouTubeError(f"{method} {path} -> {resp.status_code}: {reason}", reason=reason)
|
||||
|
||||
|
||||
@@ -119,7 +125,7 @@ class YouTubeClient:
|
||||
return self._http.request(method, url, **kwargs)
|
||||
except httpx.HTTPError as exc:
|
||||
raise YouTubeError(
|
||||
f"YouTube API unreachable ({exc.__class__.__name__}: {exc})"
|
||||
f"YouTube API unreachable ({exc.__class__.__name__}: {exc})", reason="transport"
|
||||
) from exc
|
||||
|
||||
# --- auth ---
|
||||
@@ -150,7 +156,7 @@ class YouTubeClient:
|
||||
},
|
||||
)
|
||||
if resp.status_code != 200:
|
||||
raise YouTubeError(f"Token refresh failed: {resp.status_code} {resp.text[:200]}")
|
||||
raise _upstream_failure("POST", "oauth2/token", resp)
|
||||
data = resp.json()
|
||||
access = data["access_token"]
|
||||
tok.access_token = encrypt(access)
|
||||
|
||||
Reference in New Issue
Block a user