fix(downloads): Facebook files get their upload date, non-Latin names their letters

Since 2026-10-28 the signed-in Facebook read gives no upload date, so
every Facebook file since was named 0000-00-00 in Season_0 (31 on prod,
none before). The signed-out read that already titles these downloads
carries the date, so it now lends that too when the signed-in read has
none (backfill.public_title_info is now public_page_info).

On-disk names dropped every letter outside the Latin script, so a Hebrew,
Cyrillic, Greek or CJK uploader went to one shared "untitled" folder
(prod asset 503). anyascii now writes them in Latin letters
(Сергей Иванов -> Sergei_Ivanov); Latin and accented names are unchanged.
This commit is contained in:
2026-11-14 01:15:17 +01:00
parent 25198a936f
commit 452ff042ff
8 changed files with 92 additions and 30 deletions
+7 -3
View File
@@ -20,6 +20,8 @@ from datetime import date
from pathlib import Path
from xml.sax.saxutils import escape
from anyascii import anyascii
from app import logs
from app.downloads import mediakind
@@ -121,12 +123,14 @@ def files_root_problem(
def _ascii_fold(text: str) -> str:
"""Transliterate to pure ASCII: apply the explicit map, NFKD-split accented letters and drop
the combining marks (á→a, ő→o), then strip any leftover non-ASCII (non-Latin scripts that have
no ASCII base)."""
the combining marks (á→a, ő→o), then write what is left of other scripts in Latin letters
(`anyascii`: Сергей→Sergey). Before it, a non-Latin name was dropped whole, and every Hebrew,
Cyrillic, Greek or CJK uploader shared one "untitled" folder (prod asset 503). Anything still
not ASCII after that is stripped."""
text = text.translate(_FOLD_MAP)
text = unicodedata.normalize("NFKD", text)
text = "".join(ch for ch in text if unicodedata.category(ch) != "Mn")
return text.encode("ascii", "ignore").decode("ascii")
return anyascii(text).encode("ascii", "ignore").decode("ascii")
@dataclass