peter
815083bcc4
- The queue keeps a done job by when it became done (ready_at), not by its last write: renaming, keeping or moving an old file, or the GC's expiry warning, no longer brings it back as just finished. - GC expiry and eviction re-read each file under the asset→job locks the user routes take, and commit one file at a time: a delete, a re-download or a keep made while the GC runs is honoured, never a second release that frees a file another user still holds. - A file records the origin it was placed under (download_jobs.source_key, migration 0105): an edit whose parent expired still goes back to its folder, not to Uploads. - A folder a file is being put in is held (FOR SHARE) until it lands; a delete of it waits and takes that file home too, instead of failing on the foreign key. - The hash route reads at most two files at once (429 past that) and holds no connection meanwhile. - 0104's backfill dates old files by their request date only: no notice prod wrote names its job. - The old DOWNLOAD_ROOT still finds the files; a files root that is not a mounted volume stops the worker and is logged by the API; the volume-copy recipe looks the project-prefixed names up. - Stale comments about the /downloads mount and the Downloads page.
46 lines
1.7 KiB
Python
46 lines
1.7 KiB
Python
"""Migration 0105's data step, DB-lane: a file in a source folder records that folder's key as its
|
|
origin; a file in an own folder is left to be worked out, as before."""
|
|
|
|
from alembic import command
|
|
from alembic.config import Config
|
|
from sqlalchemy import text
|
|
|
|
from app.db import engine
|
|
|
|
|
|
def test_a_file_in_a_source_folder_records_its_origin(db, make_user):
|
|
uid = make_user().id
|
|
cfg = Config("alembic.ini")
|
|
command.downgrade(cfg, "0104_job_ready_at")
|
|
try:
|
|
with engine.begin() as conn:
|
|
|
|
def folder(kind, key, name):
|
|
return conn.execute(
|
|
text(
|
|
"INSERT INTO file_folders (user_id, kind, source_key, name) VALUES"
|
|
" (:u, :k, :s, :n) RETURNING id"
|
|
),
|
|
{"u": uid, "k": kind, "s": key, "n": name},
|
|
).scalar_one()
|
|
|
|
def job(folder_id):
|
|
return conn.execute(
|
|
text(
|
|
"INSERT INTO download_jobs (user_id, source_kind, source_ref,"
|
|
" profile_snapshot, status, folder_id) VALUES (:u, 'url', 'x', '{}',"
|
|
" 'queued', :f) RETURNING id"
|
|
),
|
|
{"u": uid, "f": folder_id},
|
|
).scalar_one()
|
|
|
|
in_source = job(folder("source", "site:vimeo.com", None))
|
|
in_own = job(folder("own", None, "Mine"))
|
|
finally:
|
|
command.upgrade(cfg, "head")
|
|
|
|
with engine.connect() as conn:
|
|
rows = dict(conn.execute(text("SELECT id, source_key FROM download_jobs")).all())
|
|
assert rows[in_source] == "site:vimeo.com"
|
|
assert rows[in_own] is None
|