DX-13: both shell scripts ran bare 'docker compose', which has no config
file to find in the layout docs/self-hosting.md tells people to use (only
docker-compose.selfhost.yml is downloaded) — they now fall back to it, while
a source checkout keeps resolving docker-compose.yml as before. The two
PowerShell scripts moved the dump through the PS pipeline, which corrupts
binary on 5.1 and cannot express -Encoding Byte at all on 7 ('Byte' is not a
supported encoding name), so they simply failed there; they now write inside
the container and 'docker compose cp' it across. backup.sh also wrote its
output straight into backups/, leaving a truncated file that looks like a
real backup when pg_dump fails — it writes a .part and promotes on success.
Both restores report a non-zero pg_restore rather than printing 'Restored'
over it. Docs now state that a backup is the dump PLUS .env: without
TOKEN_ENCRYPTION_KEY every encrypted secret in the restored database is
unreadable, which the old one-line 'back up the volume' never said.
DX-14: 'latest' moves with every release and migrations run on every boot,
so self-hosters had no way to choose when they move and no way back. Docs
gain IMAGE_TAG pinning, a rollback recipe (image-only vs schema-bearing) and
the migration-failed-at-boot case; CHANGELOG.md is generated from
releaseNotes.ts so the file cannot drift from what the app shows, and
'siftlode release' refuses on a stale one rather than regenerating it into
the clean tree it just demanded.
Verified: dump + restore round-trip on both platforms against localdev into
a scratch database (8 users, 262822 videos, schema 0082_invite_lang), and
IMAGE_TAG=0.80.0 resolves on both services in 'compose config'.
- CLAUDE.md: new "Collaboration" section (plan-first, options-with-marked-
optimal-and-why, mockups for design decisions, challenge unsound requests,
visual UAT before done, terse otherwise).
- CLAUDE.md: clarify release is user-owned in timing, agent-run in execution.
- CLAUDE.md + docs/development.md: document the ruff format --check gate step
(already enforced in .gitea/workflows/check.yml, was missing from the docs).
- CLAUDE.md: refresh the stale footer to point at the now-existing
docs/development.md and backend/app/README.md.
The layering rule ("request-only logic in routes/; scheduler+worker-shared logic
in the plex|downloads|sync packages; cross-cutting primitives as flat core
modules; extract at the SECOND consumer") was real but unwritten — it lived only
in three module docstrings and a one-line README mention. Collect it, plus the
transaction-ownership rule (a mid-operation helper stages, the caller commits),
the atomic-counter / JobStatus-StrEnum conventions, and the one-home-per-config
rule, into backend/app/README.md. Add docs/development.md (run locally + the gate
commands + branching), so a new session/contributor has one place to look instead
of copying whichever pattern they happen to find first.
A rotated or missing TOKEN_ENCRYPTION_KEY makes every stored secret undecryptable.
sysconfig._raw_override called security.decrypt() with no guard, and it sits on
hot paths (/api/me via google_enabled, login, every YouTube scheduler job,
account purge) — so a key mismatch raised InvalidToken and 500'd the whole app,
the SPA wouldn't even boot, with no documented recovery.
Catch the decrypt failure in _raw_override -> warn once per key and fall back to
the env/config default. The YouTube client's refresh-token decrypt now raises a
clear YouTubeError ("reconnect") that every caller already handles, and
purge_user uses decrypt_optional so a best-effort revoke can't 500 the delete.
Document that the key must never change once secrets are stored (recovery: clear
the affected system_config rows and re-enter).
_client_ip trusted the first X-Forwarded-For hop unconditionally, so anyone able to
reach the app port could forge XFF and dodge the login/register/reset/demo rate limits.
Now trust XFF ONLY when the request's socket peer is a configured reverse proxy
(settings.trusted_proxy_ips, e.g. the VPS Caddy's WireGuard peer IP), and take the
RIGHTMOST entry — the client our proxy actually saw and appended, immune to a client
pre-seeding a fake XFF. A request from any other peer (hitting the port directly) is
keyed on its real socket IP, so XFF can't be forged to bypass the limits.
New TRUSTED_PROXY_IPS env (empty default = no proxy, use the direct peer). Documented in
.env.example, docs/self-hosting.md, README. Unit-verified against spoof-through-proxy and
direct-bypass cases.
Prep the Download Center epic (Phase 1 + editor + share) for prod/self-host:
- Dockerfile: create /downloads owned by appuser so a named-volume mount is writable (prod Linux).
- docker-compose.{home,selfhost,yml}: add the 'worker' (yt-dlp/ffmpeg job loop) + 'bgutil-pot'
(PO-token) services + a downloads mount (DOWNLOAD_ROOT, WORKER_ENABLED). Media defaults to a
named volume; DOWNLOAD_HOST_PATH points it at a host dir (e.g. a Plex-readable folder).
- README / docs/self-hosting.md / .env.example / install.{sh,ps1}: document the Download Center,
the two extra containers, and DOWNLOAD_HOST_PATH.
- VERSION 0.22.0 + releaseNotes entry.
- docker-compose.selfhost.yml pulls the published image (forge.b1fr0st.eu/peter/siftlode) with a
bundled Postgres — no source build on the operator's host.
- install.sh / install.ps1 generate the four required secrets (.env: POSTGRES_PASSWORD, SECRET_KEY,
a valid Fernet TOKEN_ENCRYPTION_KEY, OAUTH_REDIRECT_URL), start the stack, and print the first-run
setup-wizard URL. Everything else is configured in the web wizard.
- docs/self-hosting.md: the copy-paste playbook (get the files, run the installer, finish in the
wizard, optional Google/SMTP, HTTPS/reverse-proxy, updates + backups).
- Verified end-to-end: a fresh install from the published image boots into setup mode and serves
the wizard.
peter
peter