fix(downloads): Facebook files get their upload date, non-Latin names their letters
Since 2026-10-28 the signed-in Facebook read gives no upload date, so every Facebook file since was named 0000-00-00 in Season_0 (31 on prod, none before). The signed-out read that already titles these downloads carries the date, so it now lends that too when the signed-in read has none (backfill.public_title_info is now public_page_info). On-disk names dropped every letter outside the Latin script, so a Hebrew, Cyrillic, Greek or CJK uploader went to one shared "untitled" folder (prod asset 503). anyascii now writes them in Latin letters (Сергей Иванов -> Sergei_Ivanov); Latin and accented names are unchanged.
This commit is contained in:
@@ -8,7 +8,7 @@ Facebook sources (a direct media URL like a reddit HLS manifest carries no chann
|
||||
URL may have expired — those just get a locally-generated poster instead). A Facebook download
|
||||
titled with the source's placeholder ("Facebook", "Video") is retitled from a signed-out read of its
|
||||
page, and so is each download name that only repeated the placeholder. That page read
|
||||
(`public_title_info`) also titles every new download, from the worker.
|
||||
(`public_page_info`) also titles and dates every new download, from the worker.
|
||||
|
||||
Run once per environment:
|
||||
docker exec siftlode-worker-1 python -c "from app.downloads import backfill; print(backfill.run())"
|
||||
@@ -71,16 +71,23 @@ def _extract_meta_scoped(url: str, adapter: adapters.SourceAdapter, asset_id: in
|
||||
return None
|
||||
|
||||
|
||||
def public_title_info(info: dict, url: str, adapter: adapters.SourceAdapter, asset_id: int) -> dict:
|
||||
"""`info` to title a download from: itself, unless its title is a source's placeholder with
|
||||
nothing behind it (a signed-in Facebook read: "Facebook", no description). Then one signed-out,
|
||||
metadata-only read of the page lends its text — the description, else its title — as the
|
||||
description the adapter takes a placeholder's title from. Nothing better there: `info` as it was."""
|
||||
def public_page_info(info: dict, url: str, adapter: adapters.SourceAdapter, asset_id: int) -> dict:
|
||||
"""`info` to title and date a download from: itself, unless its title is a source's placeholder
|
||||
with nothing behind it (a signed-in Facebook read: "Facebook", no description). Then one
|
||||
signed-out, metadata-only read of the page lends its text — the description, else its title — as
|
||||
the description the adapter takes a placeholder's title from, and its upload date when `info` has
|
||||
none (measured on prod: the signed-in read lost both on 2026-10-28, so every Facebook file since
|
||||
was named 0000-00-00 in Season_0). Nothing better there: `info` as it was."""
|
||||
if not adapter.is_placeholder_title(adapter.title(info)):
|
||||
return info
|
||||
public = extract_meta(url, adapter, asset_id)
|
||||
text = public and (public.get("description") or public.get("title"))
|
||||
return {**info, "description": text} if text else info
|
||||
public = extract_meta(url, adapter, asset_id) or {}
|
||||
lent: dict = {}
|
||||
text = public.get("description") or public.get("title")
|
||||
if text:
|
||||
lent["description"] = text
|
||||
if not info.get("upload_date") and public.get("upload_date"):
|
||||
lent["upload_date"] = public["upload_date"]
|
||||
return {**info, **lent} if lent else info
|
||||
|
||||
|
||||
def _retitle(db: Session, a: MediaAsset, adapter: adapters.SourceAdapter, url: str) -> bool:
|
||||
@@ -88,7 +95,7 @@ def _retitle(db: Session, a: MediaAsset, adapter: adapters.SourceAdapter, url: s
|
||||
names that only repeated it. A name its owner gave (anything else) stays."""
|
||||
if not adapter.is_placeholder_title(a.title):
|
||||
return False
|
||||
info = public_title_info({"title": a.title, "description": None}, url, adapter, a.id)
|
||||
info = public_page_info({"title": a.title, "description": None}, url, adapter, a.id)
|
||||
named = normalize_title(adapter.title(info))
|
||||
if not named or adapter.is_placeholder_title(named):
|
||||
return False
|
||||
|
||||
Reference in New Issue
Block a user