fix(files): review round 1, the server — the tray's window, the GC's locks, a file's origin, the upgrade path
- The queue keeps a done job by when it became done (ready_at), not by its last write: renaming, keeping or moving an old file, or the GC's expiry warning, no longer brings it back as just finished. - GC expiry and eviction re-read each file under the asset→job locks the user routes take, and commit one file at a time: a delete, a re-download or a keep made while the GC runs is honoured, never a second release that frees a file another user still holds. - A file records the origin it was placed under (download_jobs.source_key, migration 0105): an edit whose parent expired still goes back to its folder, not to Uploads. - A folder a file is being put in is held (FOR SHARE) until it lands; a delete of it waits and takes that file home too, instead of failing on the foreign key. - The hash route reads at most two files at once (429 past that) and holds no connection meanwhile. - 0104's backfill dates old files by their request date only: no notice prod wrote names its job. - The old DOWNLOAD_ROOT still finds the files; a files root that is not a mounted volume stops the worker and is logged by the API; the volume-copy recipe looks the project-prefixed names up. - Stale comments about the /downloads mount and the Downloads page.
This commit is contained in:
@@ -37,7 +37,7 @@ from app import sysconfig
|
||||
from app.config import settings
|
||||
from app.db import SessionLocal
|
||||
from app.downloads import edit as editmod
|
||||
from app.downloads import credentials, formats, quota, service
|
||||
from app.downloads import credentials, formats, quota, service, storage
|
||||
from app.downloads import errors as dlerrors
|
||||
from app.downloads.jobdb import (
|
||||
_edit_staging_dir,
|
||||
@@ -710,6 +710,12 @@ def main() -> None:
|
||||
# `log_events`, so on the deploy that first adds the table the worker may start before the API
|
||||
# has migrated — those few seconds of batches fail and say so on stderr; nothing else breaks.
|
||||
logsink.start()
|
||||
problem = storage.files_root_problem(settings.files_root)
|
||||
if problem:
|
||||
# Refuse, loudly: downloading into a root that is not the volume loses every file with the
|
||||
# container. The queue waits until the mount is fixed (see docs/self-hosting.md).
|
||||
log.critical("The files library cannot be written: %s. The worker will not start.", problem)
|
||||
raise SystemExit(1)
|
||||
_recover_orphans()
|
||||
global _target
|
||||
_target = max(1, settings.download_worker_concurrency) # used only if the DB can't answer
|
||||
|
||||
Reference in New Issue
Block a user