# Backend conventions The rules below are already followed across `backend/app` — they lived only in a few module docstrings (`userscope.py`, `routes/_common.py`) until this file collected them. When you add code, match these so the next reader doesn't have to guess "which pattern do I copy". ## Module layering Three layers, by *who* runs the code — not by feature: 1. **`routes/…` — request-only.** A route module holds HTTP concerns: auth dependencies, request/ response models, status-code mapping, serialization. It owns logic that runs *only* on a request. Business logic that a request happens to trigger does **not** live here if anything else runs it too. 2. **The units and the other domain packages — logic shared by more than one caller.** A unit's database work (below), and anything the **scheduler or the worker** runs (not just a request), lives in a domain package, so the route, the worker, and the scheduler share one definition. Examples from this codebase: `downloads/service.py` owns enqueue + the job state transitions (routes call it); `plex/sync.py` is run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline; `issues/bundle.py` is the one definition of a bug report's files, for the zip export and the tracker. 3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth` (with `apitokens`, a machine's folder-scoped way into the Files library), `sysconfig`, `quota`, `audit`, `jobruns` (every background run's durable row, its words in `runstatus`), `notifications`, `userscope`, `ratelimit`, `progress`, `logs` (the one logging configuration), `logsink` (its `log_events` table), `diagnostics` (a user's consented log level), `debugsession` (a recording that tags its user's lines, E16) and `collation` (A→Z in the reader's alphabet, E30), … are single-file primitives many layers import. They depend on nothing above them (a core module must never import a `routes/…` module). **Extraction trigger: the SECOND consumer.** Keep new logic inline in its first caller. The moment a *second* caller needs it (a worker as well as a route; a scheduler as well as a route), extract it down a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't pre-abstract for a single caller; don't leave shared logic duplicated once there are two. ## Units The domain is four units, each owning its tables (`models.py` in its package): | Unit | Package | Its routes | | --- | --- | --- | | Capture | `downloads/`, the worker | `downloads.py`, `files.py`, `public.py` | | Reader | `reader/`, `discover/`, `feedquery.py`, the playlist sync and tag suggestions in `sync/` | `feed.py`, `playlists.py`, `saved_views.py`, `discover.py`, `tags.py` | | Catalog | `catalog/`, the ingests in `sync/` | `channels.py`, `search.py`, `sync.py` | | Plex | `plex/` | `plex/` | - **A unit's route does no database work.** It validates, reads through the unit's `queries.py`, writes through its `service.py`, serializes, and commits. It may name a unit's model as a type (under `TYPE_CHECKING`), never use one as a value, and it passes a kind (`"movie"`), not a model, to a query. - **A unit's code keeps to its edges.** It reads another unit's tables only along a declared edge (Capture and Reader read Catalog's and Plex's: the feed joins `Video` with `VideoState` in one query) and writes them only through the owner's service. Core code reaches a unit's tables through that unit's `queries`. - **Whose a row is, `userscope` says** — in every module, core included: `owned_by(Model.user_id, user)` for a filter, `owned_row(db, Model, id, user)` for a row by id (someone else's row is the same answer as a missing one). A unit's `queries.own_x` returns None and its route answers `not_found`; a core route calls `owned_or_404`. A rule `owned_row` cannot say — a job its owner shared, a built-in profile nobody owns, a folder read under `FOR SHARE` — is written out in the unit's own code, beside the reason. All of it is held by `tests/test_unit_fences.py`, with no exemptions: the unit map, each unit's packages and edges (`UNIT_PACKAGES`, `READS`) and what may commit (`UNIT_COMMITS`, `CORE_COMMITS`) are listed there. It reads the code's AST: an inline owner filter in any form (an alias, `.in_()`, `filter_by`), a model built or a statement write (`insert`/`update`/`delete` under any name) across units, and a commit off the lists all fail it. An ORM attribute write on another unit's loaded row is beyond it — review's to catch. ## Transaction ownership This follows CLAUDE.md's canonical rule — **the handler commits; a unit's `service` and every helper flush** — so read it that way, not as an exception to it. The commit belongs to the **operation owner**; anything called *inside* another transaction only stages its change (`db.add` / `flush`) and lets the owner commit, so the whole operation is one transaction. `audit.record()` is the reference: it adds the audit row and the caller commits it together with the mutation it records, so an audit entry can never exist without its change (or vice-versa). - The **operation owner** is the route handler, or a job. A unit's `service` flushes even when it makes a whole transition (`service.cancel`): the handler commits it, together with whatever else the request changed. What owns its OWN session commits it: `quota` and `jobruns`' run rows, the scheduler's jobs, a unit's jobs (the worker's `jobdb` writes, the syncs and ingests that commit per page or batch so a later failure loses nothing already stored), and a few core steps that must land whatever their caller does next (a token's last-used stamp, a refreshed YouTube grant) — the fence's `UNIT_COMMITS` and `CORE_COMMITS` name each with its reason, `test_route_conventions` a route's own committing helpers. Everything else flushes. - `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM attribute — see `service.decr_ref_count`. ## Shared counters and status: do it atomically at the DB - **Never read-modify-write a shared counter in Python** (`x.count = x.count + 1`). Concurrent sessions lose an update. Use a DB-atomic statement — `col = col + 1`, or `greatest(col - 1, 0)` with `RETURNING` so the caller branches on the post-value under the row lock (`service.incr_ref_count` / `decr_ref_count`). - **Download job status is a `JobStatus` StrEnum** (`downloads/status.py`), not bare strings. The `HOLDING` set defines which states hold a `MediaAsset` (so release accounting stays correct); `STOPPED_BY_ROUTE` is the route-initiated stop set. A worker child's terminal write goes through `service.finalize_if_running` (an atomic `UPDATE … WHERE status='running'`) so a finishing child can't resurrect a job a route just stopped. ## Config: one home per kind - **`app.config.settings`** — boot/env config (DB URL, download root, worker concurrency): read once at startup, not user-tunable at runtime. - **`sysconfig.SPECS`** — the runtime admin knobs (DB-override-or-default). A new admin-tunable value goes in `SPECS`; don't invent a fourth store. Each spec states when a change takes effect (`applies`, required): `live` keys are read where they are used, never at import (test-held). The scheduler's job intervals are `SPECS` keys too, edited on the Scheduler page, not the Configuration page. ## The gate Everything below must be green before a change is done (run via `siftlode check`): `ruff` + `basedpyright` (backend) · the lock check + `tsc` ×3 + `eslint` + `prettier` + `knip` + `vite build` (frontend) · `vitest` · `pytest`. The pytest lane runs in `backend/docker-compose.test.yml` — an ephemeral Postgres + the backend tree bind-mounted — so the DB-backed integration tests (the `db` fixture) actually run; a missing database is a hard failure, not a silent skip, unless a hand run asks for pure-logic only with `SIFTLODE_TEST_REQUIRE_DB=0`. See `docs/development.md`, and `docs/gates.md` for what each gate deliberately leaves out.