feat(scheduler): every background run is a durable row

What a job last did lived in the API process's memory and was gone at
every restart; the audit log kept a run only when it was manual, failed
or changed something. Now each run is a job_runs row written at its
start and its end in its own session: who started it, its outcome as
data, what it spent of the YouTube quota (quota_events.job_run_id) and
the lines it logged (log_events.run_id, filterable in the log console).

- a run a stopped process left open is marked interrupted at startup
- "run now" is a one-shot job on the scheduler, which now always starts
  (interval jobs only with SCHEDULER_ENABLED); the manual actor is reset
  after the run, since pool threads are reused
- the dashboard's last run comes from job_runs, so it survives a deploy
- runs leave the audit log: migration 0109 moves the old job_run rows
  over (and back on downgrade); the audit page's run folding goes
- job_run_retention_days (default 30) prunes old runs in audit_gc
This commit is contained in:
2026-10-17 23:08:55 +02:00
parent 4b1f6d7ec2
commit 45c0ee7d58
31 changed files with 870 additions and 489 deletions
+4 -3
View File
@@ -21,7 +21,8 @@ Three layers, by *who* runs the code — not by feature:
`issues/bundle.py` is the one definition of a bug report's files, for the zip export and the tracker.
3. **Flat `app/*.py` core modules — cross-cutting primitives.** `auth` (with `apitokens`, a machine's
folder-scoped way into the Files library), `sysconfig`, `quota`, `audit`,
folder-scoped way into the Files library), `sysconfig`, `quota`, `audit`, `jobruns` (every
background run's durable row, its words in `runstatus`),
`notifications`, `userscope`, `ratelimit`, `progress`, `logs` (the one logging
configuration), `logsink` (its `log_events` table), `diagnostics` (a user's consented log
level), `debugsession` (a recording that tags its user's lines, E16) and `collation` (A→Z in the
@@ -36,7 +37,7 @@ pre-abstract for a single caller; don't leave shared logic duplicated once there
## Transaction ownership
This follows CLAUDE.md's canonical rule verbatim — **"the handler/job-wrapper commits; helpers flush
(only `quota` owns its session)"** — so read it that way, not as an exception to it. The commit belongs
(only `quota` and `jobruns` own their session)"** — so read it that way, not as an exception to it. The commit belongs
to the **operation owner**; anything called *inside* another transaction only stages its change
(`db.add` / `flush`) and lets the owner commit, so the whole operation is one transaction.
`audit.record()` is the reference: it adds the audit row and the caller commits it together with the
@@ -45,7 +46,7 @@ mutation it records, so an audit entry can never exist without its change (or vi
- The **operation owner** is usually the route handler. A `service.*` operation the handler delegates a
whole transition to (e.g. `service.cancel`) *is* that owner and commits its one transition — it isn't
a mid-operation helper. A helper that owns its OWN short-lived session (the worker's `jobdb` writes,
the same carve-out CLAUDE.md grants `quota`) commits that session. Everything else flushes.
`jobruns`' run rows — the same carve-out CLAUDE.md grants `quota`) commits that session. Everything else flushes.
- `SessionLocal` is `expire_on_commit=False`: after a raw `UPDATE` + commit, an ORM object you already
loaded is **stale**. Read the fresh value from the row (RETURNING, or a re-`SELECT`), not the ORM
attribute — see `service.decr_ref_count`.