fix(ops): make backup/restore work, and give upgrades a pin and a rollback
DX-13: both shell scripts ran bare 'docker compose', which has no config
file to find in the layout docs/self-hosting.md tells people to use (only
docker-compose.selfhost.yml is downloaded) — they now fall back to it, while
a source checkout keeps resolving docker-compose.yml as before. The two
PowerShell scripts moved the dump through the PS pipeline, which corrupts
binary on 5.1 and cannot express -Encoding Byte at all on 7 ('Byte' is not a
supported encoding name), so they simply failed there; they now write inside
the container and 'docker compose cp' it across. backup.sh also wrote its
output straight into backups/, leaving a truncated file that looks like a
real backup when pg_dump fails — it writes a .part and promotes on success.
Both restores report a non-zero pg_restore rather than printing 'Restored'
over it. Docs now state that a backup is the dump PLUS .env: without
TOKEN_ENCRYPTION_KEY every encrypted secret in the restored database is
unreadable, which the old one-line 'back up the volume' never said.
DX-14: 'latest' moves with every release and migrations run on every boot,
so self-hosters had no way to choose when they move and no way back. Docs
gain IMAGE_TAG pinning, a rollback recipe (image-only vs schema-bearing) and
the migration-failed-at-boot case; CHANGELOG.md is generated from
releaseNotes.ts so the file cannot drift from what the app shows, and
'siftlode release' refuses on a stale one rather than regenerating it into
the clean tree it just demanded.
Verified: dump + restore round-trip on both platforms against localdev into
a scratch database (8 users, 262822 videos, schema 0082_invite_lang), and
IMAGE_TAG=0.80.0 resolves on both services in 'compose config'.
This commit is contained in:
+89
-7
@@ -118,10 +118,6 @@ The directory must be writable by the container user (uid `1000`): `sudo chown -
|
||||
## Day-to-day
|
||||
|
||||
```bash
|
||||
# Update to the latest release
|
||||
docker compose -f docker-compose.selfhost.yml pull
|
||||
docker compose -f docker-compose.selfhost.yml up -d
|
||||
|
||||
# Logs / status (api = web, worker = downloads/edits)
|
||||
docker compose -f docker-compose.selfhost.yml logs -f api
|
||||
docker compose -f docker-compose.selfhost.yml logs -f worker
|
||||
@@ -131,6 +127,92 @@ docker compose -f docker-compose.selfhost.yml ps
|
||||
docker compose -f docker-compose.selfhost.yml down
|
||||
```
|
||||
|
||||
Your data (accounts, subscriptions, playlists, the video catalog) lives in the `siftlode_pgdata`
|
||||
Docker volume — back that up to keep your instance's state. Database migrations run automatically
|
||||
when the app starts, so updating is just pull + up.
|
||||
## Backups
|
||||
|
||||
**A backup is three things, not one.** Copying only the database gets you an instance that starts and
|
||||
then cannot read its own secrets.
|
||||
|
||||
1. **The database** — accounts, subscriptions, playlists, the video catalog, and every setting you
|
||||
made in the admin pages. It lives in the `siftlode_pgdata` Docker volume; dump it with the command
|
||||
below.
|
||||
2. **`.env`** — in particular `TOKEN_ENCRYPTION_KEY`. Your Google client secret, SMTP password, Plex
|
||||
token and every stored refresh token are encrypted with it *inside* the database. Restore a dump
|
||||
against a different key and those values are lost — you would have to clear the affected rows from
|
||||
`system_config` and re-enter every secret by hand. Keep `.env` beside the dump, and just as private.
|
||||
3. **Downloaded media** *(only if you use the Download Center)* — the `siftlode_downloads` volume, or
|
||||
whatever `DOWNLOAD_HOST_PATH` points at. Not needed to bring the instance back; the jobs and their
|
||||
metadata are in the database, the files themselves are not.
|
||||
|
||||
Dump and restore the database with these two commands. They also exist as scripts
|
||||
(`scripts/backup.sh` + `scripts/restore.sh`, or `backup.ps1` / `restore.ps1` on Windows) if you have
|
||||
the source checkout:
|
||||
|
||||
```bash
|
||||
# Back up (writes backups/siftlode-<timestamp>.dump)
|
||||
mkdir -p backups && docker compose -f docker-compose.selfhost.yml exec -T db \
|
||||
pg_dump -U siftlode -Fc siftlode > "backups/siftlode-$(date +%Y%m%d-%H%M%S).dump"
|
||||
|
||||
# Restore into the running instance
|
||||
docker compose -f docker-compose.selfhost.yml exec -T db \
|
||||
pg_restore -U siftlode -d siftlode --clean --if-exists < backups/siftlode-<timestamp>.dump
|
||||
```
|
||||
|
||||
On Windows, do **not** pipe the dump through PowerShell — it corrupts binary data. Write it inside the
|
||||
container and copy it out instead:
|
||||
|
||||
```powershell
|
||||
docker compose -f docker-compose.selfhost.yml exec -T db pg_dump -U siftlode -Fc siftlode -f /tmp/s.dump
|
||||
docker compose -f docker-compose.selfhost.yml cp db:/tmp/s.dump backups\siftlode.dump
|
||||
```
|
||||
|
||||
## Upgrading
|
||||
|
||||
**Take a backup first.** Every start runs any new database migrations automatically, and a migration
|
||||
is not reversible by pulling the old image back — see rollback below.
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.selfhost.yml pull
|
||||
docker compose -f docker-compose.selfhost.yml up -d
|
||||
```
|
||||
|
||||
That tracks `latest`, which moves whenever a release is cut. **To decide for yourself when you move,
|
||||
pin a version** by putting it in `.env`:
|
||||
|
||||
```bash
|
||||
IMAGE_TAG=0.80.0
|
||||
```
|
||||
|
||||
Then `pull` + `up -d` stays on that version until you change the line. Released versions and what
|
||||
changed in them are listed in the app itself, under **Release Notes** in the account menu, and in
|
||||
[`CHANGELOG.md`](../CHANGELOG.md).
|
||||
|
||||
### Rollback
|
||||
|
||||
If a release misbehaves, set `IMAGE_TAG` back to the previous version and `up -d`. That is enough
|
||||
**as long as the release did not change the database schema**. If it did, the older image will refuse
|
||||
to start against the newer schema, and you need to step the schema back first:
|
||||
|
||||
```bash
|
||||
# 1. stop the app (leave the database up)
|
||||
docker compose -f docker-compose.selfhost.yml stop api worker
|
||||
# 2. step the schema back one revision
|
||||
docker compose -f docker-compose.selfhost.yml run --rm api alembic downgrade -1
|
||||
# 3. pin the previous version in .env, then start again
|
||||
docker compose -f docker-compose.selfhost.yml up -d
|
||||
```
|
||||
|
||||
If you are not sure whether the schema moved, restoring your pre-upgrade dump is the safer path — it
|
||||
puts the database back exactly where the old image expects it.
|
||||
|
||||
### A migration failed on start
|
||||
|
||||
The container will keep restarting (`restart: unless-stopped`), and `worker` will sit waiting for an
|
||||
`api` that never becomes healthy. Read the reason first:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.selfhost.yml logs api | tail -50
|
||||
```
|
||||
|
||||
Then either fix the cause and let it retry, or roll back: restore your pre-upgrade dump, pin the
|
||||
previous `IMAGE_TAG`, and `up -d`. Do not delete the `siftlode_pgdata` volume to "start clean" unless
|
||||
you have a dump you have actually tested restoring — that volume *is* your instance.
|
||||
|
||||
Reference in New Issue
Block a user