peter
5561d9a379
A share link is opened on whatever the friend holds. Measured on an iPhone and an iPad: AV1, Matroska, HEVC tagged hev1 and MP3 inside an .mp4 fail there; Chrome and Firefox add HEVC and 10-bit / 4:4:4 H.264. The old check judged the codec NAME only and ran on downloads only, so a 10-bit H.264 passed, an upload went out exactly as it came (an iPhone's HEVC .mov, a webm, an mkv) and a fast trim kept its source's codecs. One rule now, `downloads/playable.py`: video is H.264 8-bit 4:2:0 + AAC in .mp4, audio-only is .mp3 or AAC .m4a; `plan` says what must change, `build_cmd` re-encodes only the failing stream (or remuxes). The worker applies it to a download, an editor clip and — new — an upload: media that fails the rule is queued (asset pending, job queued) and the worker converts it, keeping only the converted file (owner's choice). Until then the original sits at rel_path, so Retry starts from it and a delete removes it; an upload that lost its last holder mid-conversion is reaped by the GC. Every libx264 run (edit, concat, conversion) now pins `-pix_fmt yuv420p` — without it a 10-bit source came out High 10. The download cache signature moves to v5 so a cached 10-bit asset re-derives. The upload toast says when the file is still converting (HU/EN). Tests: the plan for every measured class; real ffmpeg turning a 10-bit Matroska + Opus and an mp3-in-mp4 into files the rule accepts (red without the pixel-format pin); the queued upload's delete, cancel-then-GC, failed conversion and real worker conversion (the cleanup tests red without their rules); the toast (red without the branch). Live on localdev: 9 fixture uploads converted, every served file probed H.264 High yuv420p + AAC, or mp3 / m4a.
Backend conventions
The rules below are already followed across backend/app — they lived only in a few module
docstrings (userscope.py, routes/_common.py) until this file collected them (E3 S3,
B-core-20). When you add code, match these so the next reader doesn't have to guess "which pattern do
I copy".
Module layering
Three layers, by who runs the code — not by feature:
-
routes/…— request-only. A route module holds HTTP concerns: auth dependencies, request/ response models, status-code mapping, serialization. It owns logic that runs only on a request. Business logic that a request happens to trigger does not live here if anything else runs it too. -
plex/,downloads/,sync/,issues/— domain logic shared by more than one caller. Anything the scheduler or the worker runs (not just a request) lives in one of these domain packages, so the route, the worker, and the scheduler share one definition. Examples from this codebase:downloads/service.pyowns enqueue + the job state transitions (routes call it);plex/sync.pyis run by both the admin route and the scheduler;downloads/pipeline.pyis the worker's child pipeline;issues/bundle.pyis the one definition of a bug report's files, for the zip export and the tracker. -
Flat
app/*.pycore modules — cross-cutting primitives.auth,sysconfig,quota,audit,notifications,userscope,ratelimit,progress,logs(the one logging configuration),logsink(itslog_eventstable),diagnostics(a user's consented log level),debugsession(a recording that tags its user's lines, E16) andcollation(A→Z in the reader's alphabet, E30), … are single-file primitives many layers import. They depend on nothing above them (a core module must never import aroutes/…module).
Extraction trigger: the SECOND consumer. Keep new logic inline in its first caller. The moment a second caller needs it (a worker as well as a route; a scheduler as well as a route), extract it down a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't pre-abstract for a single caller; don't leave shared logic duplicated once there are two.
Transaction ownership
This follows CLAUDE.md's canonical rule verbatim — "the handler/job-wrapper commits; helpers flush
(only quota owns its session)" — so read it that way, not as an exception to it. The commit belongs
to the operation owner; anything called inside another transaction only stages its change
(db.add / flush) and lets the owner commit, so the whole operation is one transaction.
audit.record() is the reference: it adds the audit row and the caller commits it together with the
mutation it records, so an audit entry can never exist without its change (or vice-versa).
- The operation owner is usually the route handler. A
service.*operation the handler delegates a whole transition to (e.g.service.cancel) is that owner and commits its one transition — it isn't a mid-operation helper. A helper that owns its OWN short-lived session (the worker'sjobdbwrites, the same carve-out CLAUDE.md grantsquota) commits that session. Everything else flushes. SessionLocalisexpire_on_commit=False: after a rawUPDATE+ commit, an ORM object you already loaded is stale. Read the fresh value from the row (RETURNING, or a re-SELECT), not the ORM attribute — seeservice.decr_ref_count.
Shared counters and status: do it atomically at the DB
- Never read-modify-write a shared counter in Python (
x.count = x.count + 1). Concurrent sessions lose an update. Use a DB-atomic statement —col = col + 1, orgreatest(col - 1, 0)withRETURNINGso the caller branches on the post-value under the row lock (service.incr_ref_count/decr_ref_count). - Download job status is a
JobStatusStrEnum (downloads/status.py), not bare strings. TheHOLDINGset defines which states hold aMediaAsset(so release accounting stays correct);STOPPED_BY_ROUTEis the route-initiated stop set. A worker child's terminal write goes throughservice.finalize_if_running(an atomicUPDATE … WHERE status='running') so a finishing child can't resurrect a job a route just stopped.
Config: one home per kind
app.config.settings— boot/env config (DB URL, download root, worker concurrency): read once at startup, not user-tunable at runtime.sysconfig.SPECS— the runtime admin knobs (DB-override-or-default). A new admin-tunable value goes inSPECS; don't invent a fourth store. (state.py'smaintenance_batchpredates this and is slated to fold intosysconfig— do not add more like it.)
The gate
Everything below must be green before a change is done (run via siftlode check):
ruff + basedpyright (backend) · the lock check + tsc ×3 + eslint + prettier + knip +
vite build (frontend) · vitest · pytest. The pytest lane runs in backend/docker-compose.test.yml
— an ephemeral Postgres + the backend tree bind-mounted — so the DB-backed integration tests (the db
fixture) actually run; a missing database is a hard failure, not a silent skip, unless a hand run asks
for pure-logic only with SIFTLODE_TEST_REQUIRE_DB=0. See docs/development.md, and docs/gates.md
for what each gate deliberately leaves out.