fix(files): review round 1, the server — the tray's window, the GC's locks, a file's origin, the upgrade path

- The queue keeps a done job by when it became done (ready_at), not by its last write: renaming,
  keeping or moving an old file, or the GC's expiry warning, no longer brings it back as just finished.
- GC expiry and eviction re-read each file under the asset→job locks the user routes take, and commit
  one file at a time: a delete, a re-download or a keep made while the GC runs is honoured, never a
  second release that frees a file another user still holds.
- A file records the origin it was placed under (download_jobs.source_key, migration 0105): an edit
  whose parent expired still goes back to its folder, not to Uploads.
- A folder a file is being put in is held (FOR SHARE) until it lands; a delete of it waits and takes
  that file home too, instead of failing on the foreign key.
- The hash route reads at most two files at once (429 past that) and holds no connection meanwhile.
- 0104's backfill dates old files by their request date only: no notice prod wrote names its job.
- The old DOWNLOAD_ROOT still finds the files; a files root that is not a mounted volume stops the
  worker and is logged by the API; the volume-copy recipe looks the project-prefixed names up.
- Stale comments about the /downloads mount and the Downloads page.
This commit is contained in:
2026-10-13 00:14:09 +02:00
parent 96509fe356
commit 815083bcc4
29 changed files with 593 additions and 92 deletions
+5 -15
View File
@@ -1,5 +1,5 @@
"""Migration 0104's data step, DB-lane: a done file from before is dated by when it finished where
its "ready to watch" notice says so, else by when it was asked for; a job not done has no date."""
"""Migration 0104's data step, DB-lane: a done file from before is dated by when it was asked for
(where it already sat); a job not done has no date."""
from alembic import command
from alembic.config import Config
@@ -8,7 +8,7 @@ from sqlalchemy import text
from app.db import engine
def test_a_done_file_takes_its_notice_time_else_its_request_time(db, make_user):
def test_a_done_file_takes_its_request_time(db, make_user):
uid = make_user().id
cfg = Config("alembic.ini")
command.downgrade(cfg, "0103_files_sha256")
@@ -25,17 +25,8 @@ def test_a_done_file_takes_its_notice_time_else_its_request_time(db, make_user):
{"u": uid, "s": status, "c": created},
).scalar_one()
retried = job("done", "2026-10-01T10:00:00+00:00")
quiet = job("done", "2026-10-02T10:00:00+00:00")
done = job("done", "2026-10-02T10:00:00+00:00")
failed = job("error", "2026-10-03T10:00:00+00:00")
for at in ("2026-10-05T09:00:00+00:00", "2026-10-06T09:00:00+00:00"):
conn.execute(
text(
"INSERT INTO notifications (user_id, type, title, data, created_at) VALUES"
" (:u, 'download_done', 't', CAST(:d AS json), CAST(:c AS timestamptz))"
),
{"u": uid, "d": f'{{"job_id": {retried}}}', "c": at},
)
finally:
command.upgrade(cfg, "head")
@@ -47,6 +38,5 @@ def test_a_done_file_takes_its_notice_time_else_its_request_time(db, make_user):
)
).all()
)
assert rows[retried] == "10-06 09" # its last finish
assert rows[quiet] == "10-02 10" # no notice left: where it already sat
assert rows[done] == "10-02 10"
assert rows[failed] is None