fix(ops): make backup/restore work, and give upgrades a pin and a rollback

DX-13: both shell scripts ran bare 'docker compose', which has no config
file to find in the layout docs/self-hosting.md tells people to use (only
docker-compose.selfhost.yml is downloaded) — they now fall back to it, while
a source checkout keeps resolving docker-compose.yml as before. The two
PowerShell scripts moved the dump through the PS pipeline, which corrupts
binary on 5.1 and cannot express -Encoding Byte at all on 7 ('Byte' is not a
supported encoding name), so they simply failed there; they now write inside
the container and 'docker compose cp' it across. backup.sh also wrote its
output straight into backups/, leaving a truncated file that looks like a
real backup when pg_dump fails — it writes a .part and promotes on success.
Both restores report a non-zero pg_restore rather than printing 'Restored'
over it. Docs now state that a backup is the dump PLUS .env: without
TOKEN_ENCRYPTION_KEY every encrypted secret in the restored database is
unreadable, which the old one-line 'back up the volume' never said.

DX-14: 'latest' moves with every release and migrations run on every boot,
so self-hosters had no way to choose when they move and no way back. Docs
gain IMAGE_TAG pinning, a rollback recipe (image-only vs schema-bearing) and
the migration-failed-at-boot case; CHANGELOG.md is generated from
releaseNotes.ts so the file cannot drift from what the app shows, and
'siftlode release' refuses on a stale one rather than regenerating it into
the clean tree it just demanded.

Verified: dump + restore round-trip on both platforms against localdev into
a scratch database (8 users, 262822 videos, schema 0082_invite_lang), and
IMAGE_TAG=0.80.0 resolves on both services in 'compose config'.
This commit is contained in:
2026-09-05 00:50:07 +02:00
parent e23128b4a1
commit ca4e559cc7
8 changed files with 2061 additions and 17 deletions
+89 -7
View File
@@ -118,10 +118,6 @@ The directory must be writable by the container user (uid `1000`): `sudo chown -
## Day-to-day
```bash
# Update to the latest release
docker compose -f docker-compose.selfhost.yml pull
docker compose -f docker-compose.selfhost.yml up -d
# Logs / status (api = web, worker = downloads/edits)
docker compose -f docker-compose.selfhost.yml logs -f api
docker compose -f docker-compose.selfhost.yml logs -f worker
@@ -131,6 +127,92 @@ docker compose -f docker-compose.selfhost.yml ps
docker compose -f docker-compose.selfhost.yml down
```
Your data (accounts, subscriptions, playlists, the video catalog) lives in the `siftlode_pgdata`
Docker volume — back that up to keep your instance's state. Database migrations run automatically
when the app starts, so updating is just pull + up.
## Backups
**A backup is three things, not one.** Copying only the database gets you an instance that starts and
then cannot read its own secrets.
1. **The database** — accounts, subscriptions, playlists, the video catalog, and every setting you
made in the admin pages. It lives in the `siftlode_pgdata` Docker volume; dump it with the command
below.
2. **`.env`** — in particular `TOKEN_ENCRYPTION_KEY`. Your Google client secret, SMTP password, Plex
token and every stored refresh token are encrypted with it *inside* the database. Restore a dump
against a different key and those values are lost — you would have to clear the affected rows from
`system_config` and re-enter every secret by hand. Keep `.env` beside the dump, and just as private.
3. **Downloaded media** *(only if you use the Download Center)* — the `siftlode_downloads` volume, or
whatever `DOWNLOAD_HOST_PATH` points at. Not needed to bring the instance back; the jobs and their
metadata are in the database, the files themselves are not.
Dump and restore the database with these two commands. They also exist as scripts
(`scripts/backup.sh` + `scripts/restore.sh`, or `backup.ps1` / `restore.ps1` on Windows) if you have
the source checkout:
```bash
# Back up (writes backups/siftlode-<timestamp>.dump)
mkdir -p backups && docker compose -f docker-compose.selfhost.yml exec -T db \
pg_dump -U siftlode -Fc siftlode > "backups/siftlode-$(date +%Y%m%d-%H%M%S).dump"
# Restore into the running instance
docker compose -f docker-compose.selfhost.yml exec -T db \
pg_restore -U siftlode -d siftlode --clean --if-exists < backups/siftlode-<timestamp>.dump
```
On Windows, do **not** pipe the dump through PowerShell — it corrupts binary data. Write it inside the
container and copy it out instead:
```powershell
docker compose -f docker-compose.selfhost.yml exec -T db pg_dump -U siftlode -Fc siftlode -f /tmp/s.dump
docker compose -f docker-compose.selfhost.yml cp db:/tmp/s.dump backups\siftlode.dump
```
## Upgrading
**Take a backup first.** Every start runs any new database migrations automatically, and a migration
is not reversible by pulling the old image back — see rollback below.
```bash
docker compose -f docker-compose.selfhost.yml pull
docker compose -f docker-compose.selfhost.yml up -d
```
That tracks `latest`, which moves whenever a release is cut. **To decide for yourself when you move,
pin a version** by putting it in `.env`:
```bash
IMAGE_TAG=0.80.0
```
Then `pull` + `up -d` stays on that version until you change the line. Released versions and what
changed in them are listed in the app itself, under **Release Notes** in the account menu, and in
[`CHANGELOG.md`](../CHANGELOG.md).
### Rollback
If a release misbehaves, set `IMAGE_TAG` back to the previous version and `up -d`. That is enough
**as long as the release did not change the database schema**. If it did, the older image will refuse
to start against the newer schema, and you need to step the schema back first:
```bash
# 1. stop the app (leave the database up)
docker compose -f docker-compose.selfhost.yml stop api worker
# 2. step the schema back one revision
docker compose -f docker-compose.selfhost.yml run --rm api alembic downgrade -1
# 3. pin the previous version in .env, then start again
docker compose -f docker-compose.selfhost.yml up -d
```
If you are not sure whether the schema moved, restoring your pre-upgrade dump is the safer path — it
puts the database back exactly where the old image expects it.
### A migration failed on start
The container will keep restarting (`restart: unless-stopped`), and `worker` will sit waiting for an
`api` that never becomes healthy. Read the reason first:
```bash
docker compose -f docker-compose.selfhost.yml logs api | tail -50
```
Then either fix the cause and let it retry, or roll back: restore your pre-upgrade dump, pin the
previous `IMAGE_TAG`, and `up -d`. Do not delete the `siftlode_pgdata` volume to "start clean" unless
you have a dump you have actually tested restoring — that volume *is* your instance.