fix(files): review round 1, the server — the tray's window, the GC's locks, a file's origin, the upgrade path

- The queue keeps a done job by when it became done (ready_at), not by its last write: renaming,
  keeping or moving an old file, or the GC's expiry warning, no longer brings it back as just finished.
- GC expiry and eviction re-read each file under the asset→job locks the user routes take, and commit
  one file at a time: a delete, a re-download or a keep made while the GC runs is honoured, never a
  second release that frees a file another user still holds.
- A file records the origin it was placed under (download_jobs.source_key, migration 0105): an edit
  whose parent expired still goes back to its folder, not to Uploads.
- A folder a file is being put in is held (FOR SHARE) until it lands; a delete of it waits and takes
  that file home too, instead of failing on the foreign key.
- The hash route reads at most two files at once (429 past that) and holds no connection meanwhile.
- 0104's backfill dates old files by their request date only: no notice prod wrote names its job.
- The old DOWNLOAD_ROOT still finds the files; a files root that is not a mounted volume stops the
  worker and is logged by the API; the volume-copy recipe looks the project-prefixed names up.
- Stale comments about the /downloads mount and the Downloads page.
This commit is contained in:
2026-10-13 00:14:09 +02:00
parent 96509fe356
commit 815083bcc4
29 changed files with 593 additions and 92 deletions
+7 -1
View File
@@ -37,7 +37,7 @@ from app import sysconfig
from app.config import settings
from app.db import SessionLocal
from app.downloads import edit as editmod
from app.downloads import credentials, formats, quota, service
from app.downloads import credentials, formats, quota, service, storage
from app.downloads import errors as dlerrors
from app.downloads.jobdb import (
_edit_staging_dir,
@@ -710,6 +710,12 @@ def main() -> None:
# `log_events`, so on the deploy that first adds the table the worker may start before the API
# has migrated — those few seconds of batches fail and say so on stderr; nothing else breaks.
logsink.start()
problem = storage.files_root_problem(settings.files_root)
if problem:
# Refuse, loudly: downloading into a root that is not the volume loses every file with the
# container. The queue waits until the mount is fixed (see docs/self-hosting.md).
log.critical("The files library cannot be written: %s. The worker will not start.", problem)
raise SystemExit(1)
_recover_orphans()
global _target
_target = max(1, settings.download_worker_concurrency) # used only if the DB can't answer