fix(files): review round 2, the server — locks the GC and folders read under, the mount check, the old mount name
- GC: the expiry warning marks each file conditionally on the expiry it saw, one commit each (a file deleted meanwhile no longer fails the run; one re-timed keeps its fresh expiry unwarned); eviction counts the disk again when a candidate was freed meanwhile, so it never evicts for space already free. - Folders: a retention change locks the folder and its files, and a finishing file reads the retention under a share lock, so neither misses the other; a folder delete locks the files it takes home (a file moved out meanwhile stays put, one deleted meanwhile no longer fails the delete). - Upload: the folder is checked before the probe and held only once the job names it. - Dedup enqueue locks the cached asset, so a release meanwhile yields a fresh one instead of a 500. - Mount check reads the kernel's mount table instead of comparing devices (the vfs storage driver gave a mounted volume the device of /), and only inside a container; the API also logs a fresh, empty volume in the library's place. - Compose files fall back to DOWNLOAD_HOST_PATH; the upgrade notes cover the from-source pull. - Folder routes call owned_or_404 directly.
This commit is contained in:
@@ -65,6 +65,23 @@ def reference_url(job, asset) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def library_left_behind(db: Session, files_root: str) -> bool:
|
||||
"""True when the library names files and none of the newest of them is under `files_root` — a
|
||||
fresh, empty volume mounted in the library's place (a compose file changed under an install, its
|
||||
old volume left behind). The mount check cannot see that: the new volume is a real mount."""
|
||||
rels = (
|
||||
db.execute(
|
||||
select(MediaAsset.rel_path)
|
||||
.where(MediaAsset.status == "ready", MediaAsset.rel_path.is_not(None))
|
||||
.order_by(MediaAsset.id.desc())
|
||||
.limit(20)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
return bool(rels) and not any(storage.abs_path(files_root, r).exists() for r in rels if r)
|
||||
|
||||
|
||||
def get_or_create_asset(
|
||||
db: Session, source_kind: str, source_ref: str, format_sig: str
|
||||
) -> MediaAsset:
|
||||
@@ -77,7 +94,12 @@ def get_or_create_asset(
|
||||
MediaAsset.source_ref == source_ref,
|
||||
MediaAsset.format_sig == format_sig,
|
||||
)
|
||||
asset = db.execute(stmt).scalar_one_or_none()
|
||||
# Locked to the caller's commit: a release or the GC freeing this asset meanwhile (they lock it
|
||||
# too) either waits for the new hold, or has deleted the row by the time this read returns — and
|
||||
# the caller then gets a fresh one, never a job pointing at a row that is gone.
|
||||
asset = db.execute(
|
||||
stmt.with_for_update().execution_options(populate_existing=True)
|
||||
).scalar_one_or_none()
|
||||
if asset is not None:
|
||||
# A previously-FAILED shared asset must be re-attempted for the new job about to hold it —
|
||||
# reset it to pending (+clear the error) so the worker actually re-downloads instead of
|
||||
|
||||
Reference in New Issue
Block a user