docs(sync): scope normalize_description to channels + lock line-ending behavior

Review follow-up. The docstring said "channel/video description" but the helper is
wired only to the channel store sites; clarify it is deliberately channel-only
(video descriptions are user-facing prose where paragraph breaks carry meaning,
unlike a channel's metadata block) so it doesn't read as an asymmetric fix. Also
document that str.splitlines() folds CR/CRLF and Unicode line separators to "\n",
and lock it with a test.
This commit is contained in:
2026-08-19 01:32:26 +02:00
parent d065902404
commit 1e2b3c5d29
2 changed files with 12 additions and 4 deletions
+7 -4
View File
@@ -39,11 +39,14 @@ def to_tsquery_str(q: str) -> str | None:
def normalize_description(text: str | None) -> str | None:
"""Strip whitespace-only lines from a channel/video description before storing. YouTube
descriptions pad contact/link blocks with runs of blank lines; keeping them wastes storage,
"""Strip whitespace-only lines from a CHANNEL description before storing. YouTube channel
descriptions pad their contact/link blocks with runs of blank lines; keeping them wastes storage,
bloats the About popover, and adds nothing to the search document. Drops purely-empty lines and
right-strips each kept line. None stays None; all-blank collapses to "". Applied at the
sync/store site so the DB (and the derived search_vector) holds the clean text."""
right-strips each kept line; `str.splitlines()` also folds CR / CRLF and Unicode line separators
to "\\n" (a benign line-ending normalization). None stays None; all-blank collapses to "".
Deliberately CHANNEL-only — video descriptions are left with their original formatting: they are
user-facing prose where paragraph breaks carry meaning, unlike a channel's metadata block."""
if text is None:
return None
kept = [line.rstrip() for line in text.splitlines() if line.strip()]
+5
View File
@@ -50,3 +50,8 @@ def test_normalize_description_all_blank_collapses_to_empty():
def test_normalize_description_single_line_unchanged():
assert normalize_description("just one line") == "just one line"
def test_normalize_description_folds_crlf_line_endings():
# splitlines() folds CR/CRLF to \n — a documented, benign line-ending normalization.
assert normalize_description("a\r\nb\r\n\r\nc") == "a\nb\nc"