feat(downloads): every ready file is phone-safe — uploads and clips too
A share link is opened on whatever the friend holds. Measured on an iPhone and an iPad: AV1, Matroska, HEVC tagged hev1 and MP3 inside an .mp4 fail there; Chrome and Firefox add HEVC and 10-bit / 4:4:4 H.264. The old check judged the codec NAME only and ran on downloads only, so a 10-bit H.264 passed, an upload went out exactly as it came (an iPhone's HEVC .mov, a webm, an mkv) and a fast trim kept its source's codecs. One rule now, `downloads/playable.py`: video is H.264 8-bit 4:2:0 + AAC in .mp4, audio-only is .mp3 or AAC .m4a; `plan` says what must change, `build_cmd` re-encodes only the failing stream (or remuxes). The worker applies it to a download, an editor clip and — new — an upload: media that fails the rule is queued (asset pending, job queued) and the worker converts it, keeping only the converted file (owner's choice). Until then the original sits at rel_path, so Retry starts from it and a delete removes it; an upload that lost its last holder mid-conversion is reaped by the GC. Every libx264 run (edit, concat, conversion) now pins `-pix_fmt yuv420p` — without it a 10-bit source came out High 10. The download cache signature moves to v5 so a cached 10-bit asset re-derives. The upload toast says when the file is still converting (HU/EN). Tests: the plan for every measured class; real ffmpeg turning a 10-bit Matroska + Opus and an mp3-in-mp4 into files the rule accepts (red without the pixel-format pin); the queued upload's delete, cancel-then-GC, failed conversion and real worker conversion (the cleanup tests red without their rules); the toast (red without the branch). Live on localdev: 9 fixture uploads converted, every served file probed H.264 High yuv420p + AAC, or mp3 / m4a.
This commit is contained in:
@@ -2,14 +2,21 @@
|
||||
|
||||
A user-uploaded file joins the SAME library as a downloaded one, so it inherits the whole function
|
||||
set already built (metadata chips, share links, and — for media — in-app play/edit). Unlike a
|
||||
download it bypasses yt-dlp AND the worker entirely: the file is already on the host, so ingest
|
||||
probes it, places it on disk, and creates an already-`ready` MediaAsset + `done` DownloadJob in one
|
||||
step — the same "the file exists, mark it done now" shortcut `service.enqueue` takes on a cache hit.
|
||||
download it bypasses yt-dlp: the file is already on the host, so ingest probes it and places it on
|
||||
disk. A file that already is the phone-safe shape (`playable`) — or is not media at all — becomes an
|
||||
already-`ready` MediaAsset + `done` DownloadJob in one step, the same "the file exists, mark it done
|
||||
now" shortcut `service.enqueue` takes on a cache hit.
|
||||
|
||||
Media that is NOT phone-safe (an iPhone's HEVC .mov, a VP9 .webm, an .mkv) is placed as it came and
|
||||
queued: the asset stays `pending` and the job `queued`, and the download worker converts it
|
||||
(`pipeline._convert_upload`) — so a share link never hands a friend a file their phone cannot play.
|
||||
Only the converted file is kept (the owner's choice: one file per upload; a 10-bit HDR original
|
||||
becomes 8-bit SDR). Until then the original sits at the asset's `rel_path`, which is what a failed
|
||||
conversion's Retry starts from again, and what a delete removes (`service.release_asset`).
|
||||
|
||||
Kinds (the invariant lives in `mediakind`): a probe with a real video stream -> video, audio-only
|
||||
-> audio, neither -> a generic 'file' (pdf/zip/docx/…) that is download-only and never rendered
|
||||
inline (see the force-download serving in the file/watch routes). This module is the ingest seam a
|
||||
future sub-function (e.g. convert) plugs a new adapter into, mirroring how it sidesteps the worker.
|
||||
inline (see the force-download serving in the file/watch routes).
|
||||
|
||||
An upload does NOT get an `expires_at`: it's user-provided and may be irreplaceable (there's no
|
||||
source to re-download), so it doesn't age out on the retention clock the way a re-fetchable download
|
||||
@@ -24,7 +31,8 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import settings
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import mediakind, probe, quota, service, storage
|
||||
from app.downloads import mediakind, playable, probe, quota, service, storage
|
||||
from app.downloads.status import JobStatus
|
||||
from app.models import DownloadJob, MediaAsset, display_text
|
||||
|
||||
|
||||
@@ -44,15 +52,19 @@ def ingest_upload(
|
||||
size: int,
|
||||
display_name: str | None = None,
|
||||
) -> DownloadJob:
|
||||
"""Ingest an already-uploaded file at `src_path` into `user_id`'s library and return the `done`
|
||||
job. `src_path` is MOVED into place on success (the caller stages it to a temp file first).
|
||||
"""Ingest an already-uploaded file at `src_path` into `user_id`'s library and return its job —
|
||||
`done`, or `queued` for the worker's phone-safe conversion (see the module docstring). `src_path`
|
||||
is MOVED into place on success (the caller stages it to a temp file first).
|
||||
|
||||
Raises quota.QuotaExceeded (holdings/footprint cap) — checked BEFORE any disk write, so a
|
||||
rejected upload leaves nothing behind and the caller just cleans up its temp file."""
|
||||
quota.check_upload(db, user_id, size) # the file's size is known → charge it up front
|
||||
|
||||
vcodec, acodec, width, height, duration = probe.probe_media(src_path)
|
||||
data = probe.probe_json(src_path)
|
||||
vcodec, acodec, width, height, duration = probe.facts(data)
|
||||
kind = mediakind.derive(vcodec, acodec)
|
||||
safe = playable.plan(data, Path(filename).suffix) if kind != mediakind.FILE else None
|
||||
convert = safe is not None and safe.needs_change
|
||||
|
||||
# An upload never dedups (each file is its own asset): a random ref makes the cache key unique,
|
||||
# so `format_sig` is a constant marker rather than a format hash.
|
||||
@@ -73,7 +85,7 @@ def ingest_upload(
|
||||
final = storage.abs_path(settings.download_root, rel)
|
||||
actual_size = final.stat().st_size if final.exists() else size
|
||||
|
||||
asset.status = "ready"
|
||||
asset.status = "pending" if convert else "ready"
|
||||
asset.rel_path = rel
|
||||
asset.size_bytes = actual_size
|
||||
asset.container = storage.upload_ext(filename)
|
||||
@@ -100,8 +112,8 @@ def ingest_upload(
|
||||
source_ref=ref,
|
||||
profile_snapshot={},
|
||||
display_name=display_text(display_name),
|
||||
status="done",
|
||||
progress=100,
|
||||
status=JobStatus.QUEUED if convert else JobStatus.DONE,
|
||||
progress=0 if convert else 100,
|
||||
queue_pos=service.next_queue_pos(db),
|
||||
)
|
||||
db.add(job)
|
||||
|
||||
Reference in New Issue
Block a user