peter
815083bcc4
- The queue keeps a done job by when it became done (ready_at), not by its last write: renaming, keeping or moving an old file, or the GC's expiry warning, no longer brings it back as just finished. - GC expiry and eviction re-read each file under the asset→job locks the user routes take, and commit one file at a time: a delete, a re-download or a keep made while the GC runs is honoured, never a second release that frees a file another user still holds. - A file records the origin it was placed under (download_jobs.source_key, migration 0105): an edit whose parent expired still goes back to its folder, not to Uploads. - A folder a file is being put in is held (FOR SHARE) until it lands; a delete of it waits and takes that file home too, instead of failing on the foreign key. - The hash route reads at most two files at once (429 past that) and holds no connection meanwhile. - 0104's backfill dates old files by their request date only: no notice prod wrote names its job. - The old DOWNLOAD_ROOT still finds the files; a files root that is not a mounted volume stops the worker and is logged by the API; the volume-copy recipe looks the project-prefixed names up. - Stale comments about the /downloads mount and the Downloads page.
43 lines
1.4 KiB
Python
43 lines
1.4 KiB
Python
"""Migration 0104's data step, DB-lane: a done file from before is dated by when it was asked for
|
|
(where it already sat); a job not done has no date."""
|
|
|
|
from alembic import command
|
|
from alembic.config import Config
|
|
from sqlalchemy import text
|
|
|
|
from app.db import engine
|
|
|
|
|
|
def test_a_done_file_takes_its_request_time(db, make_user):
|
|
uid = make_user().id
|
|
cfg = Config("alembic.ini")
|
|
command.downgrade(cfg, "0103_files_sha256")
|
|
try:
|
|
with engine.begin() as conn:
|
|
|
|
def job(status, created):
|
|
return conn.execute(
|
|
text(
|
|
"INSERT INTO download_jobs (user_id, source_kind, source_ref,"
|
|
" profile_snapshot, status, created_at) VALUES (:u, 'url', 'x', '{}', :s,"
|
|
" CAST(:c AS timestamptz)) RETURNING id"
|
|
),
|
|
{"u": uid, "s": status, "c": created},
|
|
).scalar_one()
|
|
|
|
done = job("done", "2026-10-02T10:00:00+00:00")
|
|
failed = job("error", "2026-10-03T10:00:00+00:00")
|
|
finally:
|
|
command.upgrade(cfg, "head")
|
|
|
|
with engine.connect() as conn:
|
|
rows = dict(
|
|
conn.execute(
|
|
text(
|
|
"SELECT id, to_char(ready_at AT TIME ZONE 'UTC', 'MM-DD HH24') FROM download_jobs"
|
|
)
|
|
).all()
|
|
)
|
|
assert rows[done] == "10-02 10"
|
|
assert rows[failed] is None
|