fix(downloads): Facebook files get their upload date, non-Latin names their letters
Since 2026-10-28 the signed-in Facebook read gives no upload date, so every Facebook file since was named 0000-00-00 in Season_0 (31 on prod, none before). The signed-out read that already titles these downloads carries the date, so it now lends that too when the signed-in read has none (backfill.public_title_info is now public_page_info). On-disk names dropped every letter outside the Latin script, so a Hebrew, Cyrillic, Greek or CJK uploader went to one shared "untitled" folder (prod asset 503). anyascii now writes them in Latin letters (Сергей Иванов -> Sergei_Ivanov); Latin and accented names are unchanged.
This commit is contained in:
@@ -20,6 +20,8 @@ from datetime import date
|
||||
from pathlib import Path
|
||||
from xml.sax.saxutils import escape
|
||||
|
||||
from anyascii import anyascii
|
||||
|
||||
from app import logs
|
||||
from app.downloads import mediakind
|
||||
|
||||
@@ -121,12 +123,14 @@ def files_root_problem(
|
||||
|
||||
def _ascii_fold(text: str) -> str:
|
||||
"""Transliterate to pure ASCII: apply the explicit map, NFKD-split accented letters and drop
|
||||
the combining marks (á→a, ő→o), then strip any leftover non-ASCII (non-Latin scripts that have
|
||||
no ASCII base)."""
|
||||
the combining marks (á→a, ő→o), then write what is left of other scripts in Latin letters
|
||||
(`anyascii`: Сергей→Sergey). Before it, a non-Latin name was dropped whole, and every Hebrew,
|
||||
Cyrillic, Greek or CJK uploader shared one "untitled" folder (prod asset 503). Anything still
|
||||
not ASCII after that is stripped."""
|
||||
text = text.translate(_FOLD_MAP)
|
||||
text = unicodedata.normalize("NFKD", text)
|
||||
text = "".join(ch for ch in text if unicodedata.category(ch) != "Mn")
|
||||
return text.encode("ascii", "ignore").decode("ascii")
|
||||
return anyascii(text).encode("ascii", "ignore").decode("ascii")
|
||||
|
||||
|
||||
@dataclass
|
||||
|
||||
Reference in New Issue
Block a user