Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5ce292cd28 | ||
|
|
ad6ccbac05 |
@@ -66,6 +66,12 @@ how it gets built. Concretely:
|
||||
`downloads/`, `reader/`, `discover/`, `catalog/`, `youtube/`, `issues/` (shared by routes + scheduler + worker) → flat core (`auth`, `apitokens`, `quota`, `jobruns`,
|
||||
`sysconfig`, `state`, `audit`, `notifications`, `userscope`, `ratelimit`, `bodylimit`, `progress`, `collation`,
|
||||
`security`, `logs`, `logsink`, `diagnostics`, `debugsession`) → `models/db/config`. **Dependency direction is one-way: core never imports routes.**
|
||||
The domain is four **units**, each owning its tables: Capture (`downloads/`), Reader (`reader/`, `discover/`, the playlist
|
||||
sync), Catalog (`catalog/`, the `sync/` ingests) and Plex (`plex/`). A unit's route does no database work: it reads through
|
||||
the unit's `queries`, writes through its `service` (which flushes) and commits. A unit's code reads another unit's tables
|
||||
only along a declared edge and writes them only through that unit's service; core reaches a unit's tables through its
|
||||
`queries`. **Fence-held, with no exemptions** (`backend/tests/test_unit_fences.py`); conventions in
|
||||
[`backend/app/README.md`](backend/app/README.md).
|
||||
Frontend: `main.tsx` provider tree → `App.tsx` shell (a shared `/watch` link boots from its own
|
||||
entry, `watch.html` → `watchMain.tsx`, held app-free by a contract test) → `moduleRegistry` route-chunks → module
|
||||
components → the **platform layer `components/ui/`** (domain-free primitives: the popover engine and
|
||||
@@ -87,12 +93,12 @@ CQRS, a repository/DI framework, a router library, a generic CRUD-router factory
|
||||
| ORM | SQLAlchemy 2.0 `select()` | `db.query(...)` |
|
||||
| Request body | a Pydantic model | `payload: dict` |
|
||||
| Request SIZE | nothing — `bodylimit` caps every route (SEC-20); need more? add a ceiling to `ROUTE_MAX_BODY_BYTES` | `capped_body` on a route that does not read raw bytes itself |
|
||||
| Ownership check | `owned_or_404` / `get_or_404` | ad-hoc `_own_playlist`/`_owned`/`_own_job` |
|
||||
| Ownership check | `userscope.owned_by` (a filter) / `owned_row` (a row by id): a unit's `queries.own_x` returns None and its route answers `not_found`; a core route `owned_or_404` | an inline `Model.user_id == …` (fence-held, in every module) or a `db.get` and a compare |
|
||||
| Admin gate import | `app.auth` | the `app.routes.admin` re-export |
|
||||
| Upstream failure status | `422` with JSON detail | `502` (reserve 502 for raw byte-proxies) |
|
||||
| User-facing error | structured `detail` (`{code, reason}`) | a bare English string under a bilingual UI |
|
||||
| Mutation response | `{"ok": true, …}` (`{"deleted": id}`), test-held | a bare `{"unsubscribed"}`/`{"cleared": n}`, no `ok` |
|
||||
| Transaction commit | the handler commits; helpers `flush` (test-held list) | helpers that commit (only `quota` and `jobruns` own their session) |
|
||||
| Transaction commit | the handler commits; a unit's `service` and every helper `flush` (fence-held) | a helper that commits (only `quota`, `jobruns` and the jobs the fence's `UNIT_COMMITS` names own their session) |
|
||||
| Admin knob storage | `sysconfig` (values) | env / `app_state` / a table of its own (the job intervals had one) |
|
||||
| Job status word | the `downloads/status.py` StrEnum | loose `"canceled"`/`"cancelled"` literals |
|
||||
| Current time | inline `datetime.now(timezone.utc)` | new `now_utc`/`_now` helpers |
|
||||
|
||||
+46
-16
@@ -1,9 +1,8 @@
|
||||
# Backend conventions
|
||||
|
||||
The rules below are already followed across `backend/app` — they lived only in a few module
|
||||
docstrings (`userscope.py`, `routes/_common.py`) until this file collected them (E3 S3,
|
||||
B-core-20). When you add code, match these so the next reader doesn't have to guess "which pattern do
|
||||
I copy".
|
||||
docstrings (`userscope.py`, `routes/_common.py`) until this file collected them. When you add code,
|
||||
match these so the next reader doesn't have to guess "which pattern do I copy".
|
||||
|
||||
## Module layering
|
||||
|
||||
@@ -13,12 +12,13 @@ Three layers, by *who* runs the code — not by feature:
|
||||
response models, status-code mapping, serialization. It owns logic that runs *only* on a request.
|
||||
Business logic that a request happens to trigger does **not** live here if anything else runs it too.
|
||||
|
||||
2. **`plex/`, `downloads/`, `sync/`, `issues/` — domain logic shared by more than one caller.** Anything the
|
||||
**scheduler or the worker** runs (not just a request) lives in one of these domain packages, so the
|
||||
route, the worker, and the scheduler share one definition. Examples from this codebase:
|
||||
`downloads/service.py` owns enqueue + the job state transitions (routes call it); `plex/sync.py` is
|
||||
run by both the admin route and the scheduler; `downloads/pipeline.py` is the worker's child pipeline;
|
||||
`issues/bundle.py` is the one definition of a bug report's files, for the zip export and the tracker.
|
||||
2. **The units and the other domain packages — logic shared by more than one caller.** A unit's
|
||||
database work (below), and anything the **scheduler or the worker** runs (not just a request), lives
|
||||
in a domain package, so the route, the worker, and the scheduler share one definition. Examples from
|
||||
this codebase: `downloads/service.py` owns enqueue + the job state transitions (routes call it);
|
||||
`plex/sync.py` is run by both the admin route and the scheduler; `downloads/pipeline.py` is the
|
||||
worker's child pipeline; `issues/bundle.py` is the one definition of a bug report's files, for the
|
||||
zip export and the tracker.
|
||||
|
||||
3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth` (with `apitokens`, a machine's
|
||||
folder-scoped way into the Files library), `sysconfig`, `quota`, `audit`, `jobruns` (every
|
||||
@@ -34,19 +34,49 @@ Three layers, by *who* runs the code — not by feature:
|
||||
a layer — into the relevant domain package, or a flat core module if it's cross-cutting. Don't
|
||||
pre-abstract for a single caller; don't leave shared logic duplicated once there are two.
|
||||
|
||||
## Units
|
||||
|
||||
The domain is four units, each owning its tables (`models.py` in its package):
|
||||
|
||||
| Unit | Package | Its routes |
|
||||
| --- | --- | --- |
|
||||
| Capture | `downloads/`, the worker | `downloads.py`, `files.py`, `public.py` |
|
||||
| Reader | `reader/`, `discover/`, `feedquery.py`, the playlist sync and tag suggestions in `sync/` | `feed.py`, `playlists.py`, `saved_views.py`, `discover.py`, `tags.py` |
|
||||
| Catalog | `catalog/`, the ingests in `sync/` | `channels.py`, `search.py`, `sync.py` |
|
||||
| Plex | `plex/` | `plex/` |
|
||||
|
||||
- **A unit's route does no database work.** It validates, reads through the unit's `queries.py`,
|
||||
writes through its `service.py`, serializes, and commits. It may name a unit's model as a type
|
||||
(under `TYPE_CHECKING`), never use one as a value, and it passes a kind (`"movie"`), not a model, to
|
||||
a query.
|
||||
- **A unit's code keeps to its edges.** It reads another unit's tables only along a declared edge
|
||||
(Capture and Reader read Catalog's and Plex's: the feed joins `Video` with `VideoState` in one
|
||||
query) and writes them only through the owner's service. Core code reaches a unit's tables through
|
||||
that unit's `queries`.
|
||||
- **Whose a row is, `userscope` says** — in every module, core included: `owned_by(Model.user_id,
|
||||
user)` for a filter, `owned_row(db, Model, id, user)` for a row by id (someone else's row is the
|
||||
same answer as a missing one). A unit's `queries.own_x` returns None and its route answers
|
||||
`not_found`; a core route calls `owned_or_404`.
|
||||
|
||||
All of it is held by `tests/test_unit_fences.py`, with no exemptions: the unit map, each unit's
|
||||
packages and edges (`UNIT_PACKAGES`, `READS`) and the jobs that commit (`UNIT_COMMITS`) are listed
|
||||
there.
|
||||
|
||||
## Transaction ownership
|
||||
|
||||
This follows CLAUDE.md's canonical rule verbatim — **"the handler/job-wrapper commits; helpers flush
|
||||
(only `quota` and `jobruns` own their session)"** — so read it that way, not as an exception to it. The commit belongs
|
||||
to the **operation owner**; anything called *inside* another transaction only stages its change
|
||||
This follows CLAUDE.md's canonical rule — **the handler commits; a unit's `service` and every helper
|
||||
flush** — so read it that way, not as an exception to it. The commit belongs to the **operation
|
||||
owner**; anything called *inside* another transaction only stages its change
|
||||
(`db.add` / `flush`) and lets the owner commit, so the whole operation is one transaction.
|
||||
`audit.record()` is the reference: it adds the audit row and the caller commits it together with the
|
||||
mutation it records, so an audit entry can never exist without its change (or vice-versa).
|
||||
|
||||
- The **operation owner** is usually the route handler. A `service.*` operation the handler delegates a
|
||||
whole transition to (e.g. `service.cancel`) *is* that owner and commits its one transition — it isn't
|
||||
a mid-operation helper. A helper that owns its OWN short-lived session (the worker's `jobdb` writes,
|
||||
`jobruns`' run rows — the same carve-out CLAUDE.md grants `quota`) commits that session. Everything else flushes.
|
||||
- The **operation owner** is the route handler, or a job. A unit's `service` flushes even when it
|
||||
makes a whole transition (`service.cancel`): the handler commits it, together with whatever else the
|
||||
request changed. What owns its OWN session commits it: `quota` and `jobruns`' run rows, and a unit's
|
||||
jobs (the worker's `jobdb` writes, the syncs and ingests that commit per page or batch so a later
|
||||
failure loses nothing already stored) — the fence's `UNIT_COMMITS` names each. Everything else
|
||||
flushes.
|
||||
- `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already
|
||||
loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM
|
||||
attribute — see `service.decr_ref_count`.
|
||||
|
||||
@@ -26,6 +26,7 @@ from fastapi import Depends, Request
|
||||
from sqlalchemy import delete, or_, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import audit, debugsession, logs
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
@@ -223,7 +224,9 @@ def revoke_all(db: Session, user_id: int) -> int:
|
||||
change, "log out everywhere", a revoked untrusted password — ends its tokens too: one minted
|
||||
from a stolen session must not outlive the owner's answer to that theft. Returns how many."""
|
||||
gone = (
|
||||
db.execute(delete(ApiToken).where(ApiToken.user_id == user_id).returning(ApiToken.id))
|
||||
db.execute(
|
||||
delete(ApiToken).where(owned_by(ApiToken.user_id, user_id)).returning(ApiToken.id)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
+2
-1
@@ -4,6 +4,7 @@ from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
from starlette.requests import HTTPConnection
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import debugsession
|
||||
from app import email as email_mod
|
||||
from app import logs
|
||||
@@ -230,7 +231,7 @@ def purge_user(db: Session, user: User, background: BackgroundTasks) -> None:
|
||||
# request's access line, a queued row, a later background task would otherwise put rows back
|
||||
# under the id after this DELETE (logs.forget_user).
|
||||
logs.forget_user(user_id)
|
||||
db.execute(delete(LogEvent).where(LogEvent.user_id == user_id))
|
||||
db.execute(delete(LogEvent).where(owned_by(LogEvent.user_id, user_id)))
|
||||
# Every canonically-equal invite goes, not just the literal spelling: is_allowed matches on the
|
||||
# canonical form, so leaving a sibling row behind would keep granting access to a mailbox whose
|
||||
# account was just erased — and prod really does carry approved pairs for one mailbox.
|
||||
|
||||
@@ -26,6 +26,7 @@ from uuid import uuid4
|
||||
from sqlalchemy import and_, delete, func, not_, select, update
|
||||
from sqlalchemy.orm import Session, defer
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import diagnostics
|
||||
from app.models import DebugScreenshot, DebugSession, LogEvent, User
|
||||
|
||||
@@ -76,7 +77,9 @@ def open_session(db: Session, user: User) -> DebugSession | None:
|
||||
|
||||
def _unclosed(db: Session, user: User) -> DebugSession | None:
|
||||
return db.scalar(
|
||||
select(DebugSession).where(DebugSession.user_id == user.id, DebugSession.ended_at.is_(None))
|
||||
select(DebugSession).where(
|
||||
owned_by(DebugSession.user_id, user), DebugSession.ended_at.is_(None)
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@@ -354,7 +357,7 @@ def stored_screenshot_bytes(db: Session, user_id: int) -> int:
|
||||
db.scalar(
|
||||
select(func.coalesce(func.sum(DebugScreenshot.size), 0))
|
||||
.join(DebugSession, DebugSession.id == DebugScreenshot.session_id)
|
||||
.where(DebugSession.user_id == user_id)
|
||||
.where(owned_by(DebugSession.user_id, user_id))
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
@@ -9,6 +9,7 @@ table can't grow without bound for a heavy user.
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app.models import Notification
|
||||
|
||||
# Keep at most this many READ notifications per user; older read ones are pruned. Unread
|
||||
@@ -52,7 +53,7 @@ def trim_read(db: Session, user_id: int, cap: int = READ_SOFT_CAP) -> int:
|
||||
ids = (
|
||||
db.execute(
|
||||
select(Notification.id)
|
||||
.where(Notification.user_id == user_id, Notification.read.is_(True))
|
||||
.where(owned_by(Notification.user_id, user_id), Notification.read.is_(True))
|
||||
.order_by(Notification.created_at.desc())
|
||||
.offset(cap)
|
||||
)
|
||||
|
||||
@@ -16,6 +16,7 @@ from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import logs, sysconfig
|
||||
from app.db import SessionLocal
|
||||
from app.models import ApiQuotaUsage, QuotaEvent
|
||||
@@ -173,7 +174,7 @@ def actions_today(db: Session, user_id: int, action: str) -> int:
|
||||
select(func.count())
|
||||
.select_from(QuotaEvent)
|
||||
.where(
|
||||
QuotaEvent.user_id == user_id,
|
||||
owned_by(QuotaEvent.user_id, user_id),
|
||||
QuotaEvent.action == action,
|
||||
QuotaEvent.created_at >= pacific_day_start_utc(),
|
||||
)
|
||||
|
||||
@@ -16,6 +16,7 @@ from sqlalchemy import select, update
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import apitokens
|
||||
from app import email as email_mod
|
||||
from app import logs
|
||||
@@ -1124,7 +1125,7 @@ def password_reset_confirm(body: TokenPasswordIn, db: Session = Depends(get_db))
|
||||
# Burn any other outstanding reset tokens for this user.
|
||||
for row in db.execute(
|
||||
select(AuthToken).where(
|
||||
AuthToken.user_id == user.id,
|
||||
owned_by(AuthToken.user_id, user),
|
||||
AuthToken.kind.in_(_PASSWORD_TOKEN_KINDS),
|
||||
AuthToken.used_at.is_(None),
|
||||
)
|
||||
|
||||
@@ -18,6 +18,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import audit, debugsession, logs, logsink
|
||||
from app.audit import AuditAction
|
||||
from app.auth import current_user
|
||||
@@ -95,7 +96,7 @@ def stop_session(
|
||||
def list_sessions(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
sessions = db.scalars(
|
||||
select(DebugSession)
|
||||
.where(DebugSession.user_id == user.id)
|
||||
.where(owned_by(DebugSession.user_id, user))
|
||||
.order_by(DebugSession.started_at.desc())
|
||||
.limit(LIST_MAX)
|
||||
).all()
|
||||
|
||||
@@ -5,6 +5,7 @@ from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import func, select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app.auth import current_user
|
||||
from app.db import get_db
|
||||
from app.models import Notification, User
|
||||
@@ -36,7 +37,7 @@ def list_notifications(
|
||||
user: User = Depends(current_user),
|
||||
db: Session = Depends(get_db),
|
||||
) -> dict:
|
||||
where = [Notification.user_id == user.id]
|
||||
where = [owned_by(Notification.user_id, user)]
|
||||
if not include_dismissed:
|
||||
where.append(Notification.dismissed.is_(False))
|
||||
total = db.scalar(select(func.count()).select_from(Notification).where(*where)) or 0
|
||||
@@ -61,7 +62,7 @@ def unread_count(user: User = Depends(current_user), db: Session = Depends(get_d
|
||||
select(func.count())
|
||||
.select_from(Notification)
|
||||
.where(
|
||||
Notification.user_id == user.id,
|
||||
owned_by(Notification.user_id, user),
|
||||
Notification.read.is_(False),
|
||||
Notification.dismissed.is_(False),
|
||||
)
|
||||
@@ -86,7 +87,7 @@ def mark_read(
|
||||
def mark_all_read(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
db.execute(
|
||||
update(Notification)
|
||||
.where(Notification.user_id == user.id, Notification.read.is_(False))
|
||||
.where(owned_by(Notification.user_id, user), Notification.read.is_(False))
|
||||
.values(read=True)
|
||||
)
|
||||
db.commit()
|
||||
@@ -113,7 +114,7 @@ def clear_all(user: User = Depends(current_user), db: Session = Depends(get_db))
|
||||
"""Dismiss every (still-visible) notification for this user."""
|
||||
db.execute(
|
||||
update(Notification)
|
||||
.where(Notification.user_id == user.id, Notification.dismissed.is_(False))
|
||||
.where(owned_by(Notification.user_id, user), Notification.dismissed.is_(False))
|
||||
.values(dismissed=True, read=True)
|
||||
)
|
||||
db.commit()
|
||||
|
||||
@@ -10,6 +10,7 @@ from fastapi import APIRouter, Depends, Query
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import quota
|
||||
from app.auth import admin_user, current_user
|
||||
from app.db import get_db
|
||||
@@ -27,7 +28,7 @@ def _sum(db: Session, *conds) -> int:
|
||||
@router.get("/my-usage")
|
||||
def my_usage(user: User = Depends(current_user), db: Session = Depends(get_db)) -> dict:
|
||||
now = datetime.now(timezone.utc)
|
||||
mine = QuotaEvent.user_id == user.id
|
||||
mine = owned_by(QuotaEvent.user_id, user)
|
||||
by_action = {
|
||||
action: int(units)
|
||||
for action, units in db.execute(
|
||||
|
||||
@@ -16,6 +16,7 @@ from pydantic import BaseModel, Field
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.userscope import owned_by
|
||||
from app import apitokens, audit
|
||||
from app.apitokens import GRANT_COLUMNS, Right
|
||||
from app.audit import AuditAction
|
||||
@@ -86,7 +87,7 @@ def _serialize(db: Session, token: ApiToken) -> dict:
|
||||
@router.get("")
|
||||
def list_tokens(user: User = Depends(current_user), db: Session = Depends(get_db)) -> list[dict]:
|
||||
tokens = (
|
||||
db.execute(select(ApiToken).where(ApiToken.user_id == user.id).order_by(ApiToken.id))
|
||||
db.execute(select(ApiToken).where(owned_by(ApiToken.user_id, user)).order_by(ApiToken.id))
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
@@ -134,7 +135,7 @@ def create_token(
|
||||
if (epoch or 0) != (user.session_epoch or 0):
|
||||
raise refused(401, "not_authenticated")
|
||||
count = db.execute(
|
||||
select(func.count(ApiToken.id)).where(ApiToken.user_id == user.id)
|
||||
select(func.count(ApiToken.id)).where(owned_by(ApiToken.user_id, user))
|
||||
).scalar_one()
|
||||
if count >= MAX_TOKENS:
|
||||
raise refused(409, "api_token_limit", limit=MAX_TOKENS)
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
"""Whose a row is, and who an account is to the others — one place for both.
|
||||
|
||||
**Ownership.** A user-owned table says whose a row is in one column (`user_id`, or `to_user_id` for
|
||||
something handed to them). A unit's reads and writes name that column through `owned_by` and fetch a
|
||||
something handed to them). Every read and write names that column through `owned_by` and fetches a
|
||||
row by id through `owned_row`, never with an inline `Model.user_id == …`: the rule "a row owned by
|
||||
someone else is answered like a missing one" then lives here, and a contract test
|
||||
(`test_unit_fences.py`) holds a switched-on unit to it.
|
||||
(`test_unit_fences.py`) holds every module to it, core included.
|
||||
|
||||
**Recipients.** "An active, non-suspended account" — the download share-by-email and the recipient
|
||||
picker filter on it. Kept here so the two forms (SQL clauses for a query, a Python check for a loaded
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
"""The core's owner filters, each held by a second account whose rows must not count, show or change.
|
||||
|
||||
Every one of these reads or writes a user-owned table through `userscope.owned_by`; the fence holds
|
||||
the form (no inline `Model.user_id == …`), these hold that the filter is there at all. Each test gives
|
||||
a second account the same kind of rows, so a filter that went missing answers with theirs. DB lane.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from app import debugsession, notifications, quota
|
||||
from app.models import DebugScreenshot, DebugSession, Notification, QuotaEvent
|
||||
from app.routes import tokens as tokens_routes
|
||||
from app.routes.auth import RESET_TTL, _issue_token
|
||||
|
||||
|
||||
def _notify(db, user, n=1, *, read=False):
|
||||
for i in range(n):
|
||||
db.add(Notification(user_id=user.id, type="t", title=f"n{i}", read=read))
|
||||
db.commit()
|
||||
|
||||
|
||||
def _visible(db, user) -> int:
|
||||
return db.scalar(
|
||||
select(func.count())
|
||||
.select_from(Notification)
|
||||
.where(Notification.user_id == user.id, Notification.dismissed.is_(False))
|
||||
)
|
||||
|
||||
|
||||
def test_the_notification_list_shows_only_mine(db, make_user, as_user):
|
||||
me, other = make_user(), make_user()
|
||||
_notify(db, me)
|
||||
_notify(db, other, 2)
|
||||
body = as_user(me).get("/api/me/notifications").json()
|
||||
assert (body["total"], len(body["items"])) == (1, 1)
|
||||
|
||||
|
||||
def test_the_unread_count_counts_only_mine(db, make_user, as_user):
|
||||
me, other = make_user(), make_user()
|
||||
_notify(db, me)
|
||||
_notify(db, other, 2)
|
||||
assert as_user(me).get("/api/me/notifications/unread_count").json() == {"count": 1}
|
||||
|
||||
|
||||
def test_reading_all_reads_only_mine(db, make_user, as_user):
|
||||
me, other = make_user(), make_user()
|
||||
_notify(db, me)
|
||||
_notify(db, other, 2)
|
||||
assert as_user(me).post("/api/me/notifications/read_all").status_code == 200
|
||||
assert as_user(other).get("/api/me/notifications/unread_count").json() == {"count": 2}
|
||||
|
||||
|
||||
def test_clearing_all_dismisses_only_mine(db, make_user, as_user):
|
||||
me, other = make_user(), make_user()
|
||||
_notify(db, me)
|
||||
_notify(db, other, 2)
|
||||
assert as_user(me).post("/api/me/notifications/clear").status_code == 200
|
||||
db.expire_all()
|
||||
assert (_visible(db, me), _visible(db, other)) == (0, 2)
|
||||
|
||||
|
||||
def test_trimming_read_notifications_trims_only_mine(db, make_user):
|
||||
me, other = make_user(), make_user()
|
||||
_notify(db, me, 3, read=True)
|
||||
_notify(db, other, 3, read=True)
|
||||
assert notifications.trim_read(db, me.id, cap=1) == 2
|
||||
db.commit()
|
||||
assert _visible(db, other) == 3
|
||||
|
||||
|
||||
def test_todays_actions_count_only_mine(db, make_user):
|
||||
me, other = make_user(), make_user()
|
||||
quota.log_action(me.id, quota.QuotaAction.VIDEOS_SEARCH)
|
||||
quota.log_action(other.id, quota.QuotaAction.VIDEOS_SEARCH)
|
||||
quota.log_action(other.id, quota.QuotaAction.VIDEOS_SEARCH)
|
||||
assert quota.actions_today(db, me.id, quota.QuotaAction.VIDEOS_SEARCH) == 1
|
||||
|
||||
|
||||
def test_my_usage_sums_only_mine(db, make_user, as_user):
|
||||
me, other = make_user(), make_user()
|
||||
db.add_all(
|
||||
[
|
||||
QuotaEvent(user_id=me.id, action="videos.list", units=5),
|
||||
QuotaEvent(user_id=other.id, action="videos.list", units=7),
|
||||
]
|
||||
)
|
||||
db.commit()
|
||||
body = as_user(me).get("/api/quota/my-usage").json()
|
||||
assert (body["today"], body["all_time"], body["by_action"]) == (5, 5, {"videos.list": 5})
|
||||
|
||||
|
||||
def test_the_token_ceiling_counts_only_mine(db, make_user, as_user, monkeypatch):
|
||||
monkeypatch.setattr(tokens_routes, "MAX_TOKENS", 1)
|
||||
me, other = make_user(), make_user()
|
||||
first = as_user(other).post("/api/tokens", json={"name": "theirs", "preset": "hub"})
|
||||
assert first.status_code == 200, first.text
|
||||
r = as_user(me).post("/api/tokens", json={"name": "mine", "preset": "hub"})
|
||||
assert r.status_code == 200, r.text
|
||||
|
||||
|
||||
def test_a_reset_burns_only_my_other_links(client, db, make_user):
|
||||
me, other = make_user(), make_user()
|
||||
mine = _issue_token(db, me, "reset", RESET_TTL)
|
||||
theirs = _issue_token(db, other, "reset", RESET_TTL)
|
||||
db.commit()
|
||||
r = client.post(
|
||||
"/auth/password-reset/confirm", json={"token": mine, "password": "a brand new passphrase"}
|
||||
)
|
||||
assert r.json() == {"ok": True}
|
||||
assert client.post("/auth/password-reset/check", json={"token": theirs}).json() == {"ok": True}
|
||||
|
||||
|
||||
def test_screenshot_storage_counts_only_mine(db, make_user):
|
||||
me, other = make_user(), make_user()
|
||||
now = datetime.now(timezone.utc)
|
||||
for i, (user, size) in enumerate([(me, 10), (other, 1000)]):
|
||||
db.add(
|
||||
DebugSession(
|
||||
id=f"s{i}",
|
||||
user_id=user.id,
|
||||
level="DEBUG",
|
||||
started_at=now,
|
||||
until=now + timedelta(hours=1),
|
||||
)
|
||||
)
|
||||
db.flush()
|
||||
db.add(
|
||||
DebugScreenshot(
|
||||
session_id=f"s{i}", created_at=now, mime="image/png", size=size, data=b"x" * size
|
||||
)
|
||||
)
|
||||
db.commit()
|
||||
assert debugsession.stored_screenshot_bytes(db, me.id) == 10
|
||||
@@ -5,17 +5,14 @@ What it may not do is build SQL, call the session's data methods, change a model
|
||||
import a unit's tables: when a route did those, ownership, the commit and the conditional `WHERE`
|
||||
lived in a different place in each handler, and a fix in one never reached its siblings.
|
||||
|
||||
The rule is switched on unit by unit. `ALLOWED` pins what each HTTP file still does today, and the
|
||||
pin is exact: a count that rises fails (new database work in a route), and one that falls fails
|
||||
until its entry is lowered (the fence only tightens, as `baseline.json` does for basedpyright). An
|
||||
empty `ALLOWED` is the end state — a file listed there is a debt, not an exemption.
|
||||
The rule has no exemptions: a single `select()` in a unit's route file fails it.
|
||||
|
||||
A core route (identity, admin, ops) is not a unit's HTTP layer and may use the session; it may
|
||||
still not reach into a unit's tables. A route may NAME a unit's model as a type (under
|
||||
`TYPE_CHECKING`), never use one as a value.
|
||||
|
||||
Below the HTTP layer, a fenced unit's own code keeps to its edges (`FENCED`, `READS`): see the
|
||||
second half of this file.
|
||||
Below the HTTP layer, a unit's own code keeps to its edges (`READS`): see the second half of this
|
||||
file. Whose a row is, every module says through `userscope` — a core one too.
|
||||
"""
|
||||
|
||||
import ast
|
||||
@@ -107,10 +104,6 @@ UNIT_ROUTES = {
|
||||
"plex/": "plex",
|
||||
}
|
||||
|
||||
# What each HTTP file still does, by file: the debt a unit's sprint pays off. Exact counts. A core
|
||||
# route's count is only the unit tables it imports; it is paid by the sprint of the unit it reaches.
|
||||
ALLOWED: dict[str, int] = {}
|
||||
|
||||
# The session methods that read or write data. `commit` and `rollback` stay with the handler.
|
||||
SESSION_DATA = {
|
||||
"add",
|
||||
@@ -257,19 +250,13 @@ def test_every_listed_route_exists():
|
||||
rels = [rel for rel, _ in _http_files()]
|
||||
for key in UNIT_ROUTES:
|
||||
assert any(rel == key or (key.endswith("/") and rel.startswith(key)) for rel in rels), key
|
||||
assert set(ALLOWED) <= set(rels), set(ALLOWED) - set(rels)
|
||||
|
||||
|
||||
def test_no_http_file_does_database_work_beyond_its_pin():
|
||||
drift = {}
|
||||
for rel, src in _http_files():
|
||||
found = violations(src, _unit_of(rel))
|
||||
pinned = ALLOWED.get(rel, 0)
|
||||
if len(found) != pinned:
|
||||
drift[rel] = (pinned, len(found), found if len(found) > pinned else "lower its pin")
|
||||
assert drift == {}, (
|
||||
"an HTTP file's database work moved off its pin (pinned, now, what): move new work into"
|
||||
" the unit's queries/service; when a count fell, lower ALLOWED to it"
|
||||
def test_no_http_file_does_database_work():
|
||||
hits = {rel: found for rel, src in _http_files() if (found := violations(src, _unit_of(rel)))}
|
||||
assert hits == {}, (
|
||||
"a route does database work: reads go through the unit's queries, writes through its"
|
||||
" service; a core route reaches a unit's tables through that unit's queries too"
|
||||
)
|
||||
|
||||
|
||||
@@ -307,11 +294,11 @@ def test_the_detector_sees_each_kind_of_database_work():
|
||||
|
||||
|
||||
# --- inside a unit: its package --------------------------------------------------------------
|
||||
# A unit whose sprint has landed is fenced beyond its HTTP layer too. Its own code reads another
|
||||
# unit's tables only along a declared edge (the feed joins Video with VideoState in one query; a
|
||||
# boundary that split it would cost the speed the app is for), and writes another unit's tables
|
||||
# never — that goes through the owner's service. Whose a row is, it says through `userscope`, and
|
||||
# it commits only where a job owns its session.
|
||||
# Every unit is fenced beyond its HTTP layer too. Its own code reads another unit's tables only
|
||||
# along a declared edge (the feed joins Video with VideoState in one query; a boundary that split
|
||||
# it would cost the speed the app is for), and writes another unit's tables never — that goes
|
||||
# through the owner's service. It commits only where a job owns its session. Whose a row is, it
|
||||
# says through `userscope`, and so does core code: the owner filter is checked in every module.
|
||||
|
||||
# Each unit's code outside the HTTP layer: its package, and the entry point that runs its jobs.
|
||||
UNIT_PACKAGES = {
|
||||
@@ -341,9 +328,7 @@ UNIT_PACKAGES = {
|
||||
# pages' reads and writes.
|
||||
"plex": ("plex/",),
|
||||
}
|
||||
# The units fenced so far; the rest join with their sprint.
|
||||
FENCED = {"capture", "reader", "catalog", "plex"}
|
||||
# Whose tables a unit's own code may READ. Core code reads a fenced unit's tables through its queries.
|
||||
# Whose tables a unit's own code may READ. Core code reads a unit's tables through its queries.
|
||||
READS = {
|
||||
"capture": {"catalog", "plex"},
|
||||
"reader": {"catalog", "plex"},
|
||||
@@ -352,7 +337,7 @@ READS = {
|
||||
}
|
||||
# The columns that say whose a row is.
|
||||
OWNER_COLUMNS = {"user_id", "to_user_id", "from_user_id", "created_by"}
|
||||
# Where a fenced unit's code may commit: the jobs that own their session (the worker and what it
|
||||
# Where a unit's code may commit: the jobs that own their session (the worker and what it
|
||||
# runs, the GC, the backfill — none of them inside a request), and the one write that must commit
|
||||
# where it stands: an upload's rows and the file it placed on disk land together or not at all, so
|
||||
# a failed commit can remove the file (`upload.ingest_upload`). Everything else flushes; the handler
|
||||
@@ -465,14 +450,14 @@ def _enclosing_functions(tree: ast.AST) -> dict[ast.AST, str]:
|
||||
|
||||
|
||||
def package_violations(rel: str, source: str) -> list[str]:
|
||||
"""What a module outside the HTTP layer does against a fenced unit's boundary, as `line: what`."""
|
||||
"""What a module outside the HTTP layer does against a unit's boundary, as `line: what`."""
|
||||
tree = ast.parse(source)
|
||||
mine = _package_unit(rel)
|
||||
found = []
|
||||
models = _imported_models(tree)
|
||||
for name in models.values():
|
||||
theirs = UNIT_MODEL_NAMES[name]
|
||||
if theirs == mine or (mine not in FENCED and theirs not in FENCED):
|
||||
if theirs == mine:
|
||||
continue
|
||||
if theirs not in READS.get(mine or "", set()):
|
||||
found.append(f"0: {mine or 'core'} imports {theirs}'s {name}")
|
||||
@@ -487,10 +472,10 @@ def package_violations(rel: str, source: str) -> list[str]:
|
||||
if written is None:
|
||||
continue
|
||||
theirs = UNIT_MODEL_NAMES[written]
|
||||
if theirs != mine and (mine in FENCED or theirs in FENCED):
|
||||
if theirs != mine:
|
||||
found.append(f"{node.lineno}: {mine or 'core'} writes {theirs}'s {written}")
|
||||
if mine in FENCED:
|
||||
found += _owner_filters(tree)
|
||||
found += _owner_filters(tree)
|
||||
if mine is not None:
|
||||
functions = _enclosing_functions(tree)
|
||||
allowed = UNIT_COMMITS.get(mine, set())
|
||||
for node in ast.walk(tree):
|
||||
@@ -522,7 +507,7 @@ def _owner_filters(tree: ast.AST) -> list[str]:
|
||||
return found
|
||||
|
||||
|
||||
def test_a_fenced_units_code_keeps_to_its_edges():
|
||||
def test_a_units_code_keeps_to_its_edges():
|
||||
hits = {rel: found for rel, src in _app_files() if (found := package_violations(rel, src))}
|
||||
assert hits == {}, (
|
||||
"read another unit's tables only along READS, write them only through their service, say"
|
||||
@@ -530,12 +515,8 @@ def test_a_fenced_units_code_keeps_to_its_edges():
|
||||
)
|
||||
|
||||
|
||||
def test_a_fenced_units_routes_filter_owners_through_userscope():
|
||||
hits = {
|
||||
rel: found
|
||||
for rel, src in _http_files()
|
||||
if _unit_of(rel) in FENCED and (found := _owner_filters(ast.parse(src)))
|
||||
}
|
||||
def test_every_route_filters_owners_through_userscope():
|
||||
hits = {rel: found for rel, src in _http_files() if (found := _owner_filters(ast.parse(src)))}
|
||||
assert hits == {}
|
||||
|
||||
|
||||
@@ -562,6 +543,7 @@ PACKAGE_CASES = {
|
||||
("sync/videos.py", "from app.models import Playlist"): 1, # no catalog→reader edge
|
||||
("downloads/x.py", "q = DownloadJob.user_id == 1"): 1,
|
||||
("downloads/x.py", "q = owned_by(DownloadJob.user_id, 1)"): 0,
|
||||
("quota.py", "q = QuotaEvent.user_id == 1"): 1, # core too
|
||||
("downloads/x.py", "def f(db):\n db.commit()"): 1,
|
||||
("downloads/gc.py", "def f(db):\n db.commit()"): 0,
|
||||
("downloads/upload.py", "def ingest_upload(db):\n db.commit()"): 0,
|
||||
|
||||
@@ -113,7 +113,6 @@ project checks needs a row.
|
||||
| CVE gate | HIGH/CRITICAL with a fix available only | `--ignore-unfixed` + `--severity HIGH,CRITICAL`: a finding with no fix cannot be acted on by an upgrade; it shows in the scan output. A fixable one fails the publish, and the only way past it is `.trivyignore.yaml`. | 0 open (E32 S1) |
|
||||
| npm audit | `--omit=dev --audit-level=high` | What ships to the browser, at high or above. The dev toolchain's advisories were burned down by hand in E32 S1 (18 → 0). | 0 |
|
||||
| knip | `entry` — one file added | Not a narrowing: knip's Vite plugin reads entries from index.html only, so the shared link's page (`watch.html` → `src/watchMain.tsx`) is named here, or the lane reports that whole entry unused. It adds files to the lane, never takes any out. | 1 entry |
|
||||
| unit fence | the database work the HTTP layer still does, pinned | `backend/tests/test_unit_fences.py`: a unit's route file may not build SQL, call the session's data methods, set a model's attributes or import a unit's tables; a core route may not import a unit's tables. Switched on unit by unit: `ALLOWED` pins each file's count exactly, so a rise fails and a fall fails until the pin is lowered — the list may only shrink. End state: empty, and the pin mechanism goes. A unit in `FENCED` is held below its HTTP layer too: its code reads another unit's tables only along `READS`, writes them never, filters owners through `userscope`, and commits only where `UNIT_COMMITS` names a job's own session. | 826 in 23 files (E42 S0) → 667 in 19 (S1: Capture 0, fenced) → 525 in 14 (S2: Reader 0, fenced) → 354 in 9 (S3: Catalog 0, fenced; core routes 0) → 178 in 4 (S4a: Plex browse, detail, people, collections 0) → 0 (S4b: Plex 0, fenced — every unit fenced) |
|
||||
| React Compiler | the functions it skips, pinned | `components/compilerCoverage.contract.test.ts` — the list may only shrink. End state: only VirtualGrid (TanStack Virtual, an IncompatibleLibrary) remains. | 93 (E32 S1) → 29 (E32 S3) → 1, VirtualGrid (E32 S4) |
|
||||
|
||||
## Accepted findings
|
||||
|
||||
Reference in New Issue
Block a user