peter
ca4e559cc7
DX-13: both shell scripts ran bare 'docker compose', which has no config
file to find in the layout docs/self-hosting.md tells people to use (only
docker-compose.selfhost.yml is downloaded) — they now fall back to it, while
a source checkout keeps resolving docker-compose.yml as before. The two
PowerShell scripts moved the dump through the PS pipeline, which corrupts
binary on 5.1 and cannot express -Encoding Byte at all on 7 ('Byte' is not a
supported encoding name), so they simply failed there; they now write inside
the container and 'docker compose cp' it across. backup.sh also wrote its
output straight into backups/, leaving a truncated file that looks like a
real backup when pg_dump fails — it writes a .part and promotes on success.
Both restores report a non-zero pg_restore rather than printing 'Restored'
over it. Docs now state that a backup is the dump PLUS .env: without
TOKEN_ENCRYPTION_KEY every encrypted secret in the restored database is
unreadable, which the old one-line 'back up the volume' never said.
DX-14: 'latest' moves with every release and migrations run on every boot,
so self-hosters had no way to choose when they move and no way back. Docs
gain IMAGE_TAG pinning, a rollback recipe (image-only vs schema-bearing) and
the migration-failed-at-boot case; CHANGELOG.md is generated from
releaseNotes.ts so the file cannot drift from what the app shows, and
'siftlode release' refuses on a stale one rather than regenerating it into
the clean tree it just demanded.
Verified: dump + restore round-trip on both platforms against localdev into
a scratch database (8 users, 262822 videos, schema 0082_invite_lang), and
IMAGE_TAG=0.80.0 resolves on both services in 'compose config'.
124 lines
6.1 KiB
Markdown
124 lines
6.1 KiB
Markdown
# Siftlode
|
|
|
|
**Your YouTube subscriptions, the way a feed should work.** Siftlode pulls every upload from the
|
|
channels you follow into one clean, filterable feed — no algorithm deciding what you see, and no
|
|
Shorts or livestream noise unless you want it. Self-hosted, multi-user, and private: your data
|
|
stays on your own server.
|
|
|
|
|
|
|
|
Everything expensive (channels, videos, metadata) is fetched from YouTube once and stored locally,
|
|
so filtering, searching and sorting are instant and don't burn API quota. Click a video to watch it
|
|
in an in-app player that resumes where you left off — or open it on youtube.com so your own ad
|
|
blocker and SponsorBlock keep working.
|
|
|
|
## Features
|
|
|
|
- **A readable subscription feed** — sort and filter by channel, tag, language, topic, length,
|
|
upload date or watch state; hide channels without unsubscribing; save filter setups as named views.
|
|
- **Search all of YouTube** from the feed — live results you can play, save or add to playlists like
|
|
any other video; they stay temporary and out of your catalog unless you keep one.
|
|
- **Channel pages & a channel manager** — per-channel stats and uploads, priorities, and your own
|
|
tags to slice the feed by.
|
|
- **Playlists with two-way YouTube sync** — build them locally, keep them in sync in both directions.
|
|
- **In-app player** with resume, plus keyboard/scroll controls.
|
|
- **Download Center** — save videos to the server with yt-dlp in a Plex-friendly layout (format
|
|
presets, per-user storage quota), trim / crop / split & join them in a built-in editor, then save
|
|
to your device, share with another user, or hand out a public watch link.
|
|
- **Plex library** *(optional)* — point Siftlode at your own Plex server to browse and play its
|
|
movies and shows alongside your feed, with watch state kept in sync.
|
|
- **Multi-user** with per-user private state, a shared catalog, and a fair daily API-quota guard.
|
|
- **Self-hosted & private**, with a first-run web setup wizard and the interface in **English
|
|
and Hungarian**.
|
|
|
|
## Quick start (self-hosting)
|
|
|
|
You don't need to build anything — Siftlode runs from a prebuilt public image, and all
|
|
configuration (your admin account, Google sign-in, email) happens in a **first-run web wizard**.
|
|
You need [Docker](https://docs.docker.com/get-docker/) with the Compose plugin.
|
|
|
|
**1. Get the files and run the installer:**
|
|
|
|
```sh
|
|
git clone https://git.b1fr0st.eu/peter/siftlode.git
|
|
cd siftlode
|
|
./install.sh # Windows (PowerShell): ./install.ps1
|
|
```
|
|
|
|
The installer generates a private `.env` (secrets), pulls the image, starts the app + database, and
|
|
prints a one-time setup URL like `http://localhost:8080/setup?token=…`.
|
|
|
|
**2. Finish in your browser.** Open that URL and follow the wizard:
|
|
|
|
1. **Admin account** — the email + password you'll sign in with.
|
|
2. **Google sign-in** *(optional)* — paste a Google OAuth client to enable "Sign in with Google" and
|
|
pulling your YouTube subscriptions. Skip it to use email + password only.
|
|
3. **Email / SMTP** *(optional)* — for verification/notification emails. Skip it and you (the admin)
|
|
simply approve new accounts yourself.
|
|
|
|
Then sign in with your admin account. That's it. See **[docs/self-hosting.md](docs/self-hosting.md)**
|
|
for the full walkthrough.
|
|
|
|
> Just trying it out? Press Enter at the installer's URL prompt to run on `http://localhost:8080`.
|
|
|
|
## Build from source (alternative)
|
|
|
|
Prefer to build the image yourself instead of pulling it:
|
|
|
|
```sh
|
|
git clone https://git.b1fr0st.eu/peter/siftlode.git
|
|
cd siftlode
|
|
cp .env.example .env
|
|
# generate the two secrets and paste them into .env:
|
|
python -c "import secrets;print('SECRET_KEY='+secrets.token_urlsafe(48))"
|
|
python -c "import base64,os;print('TOKEN_ENCRYPTION_KEY='+base64.urlsafe_b64encode(os.urandom(32)).decode())"
|
|
docker compose up --build -d # builds from the included Dockerfile
|
|
```
|
|
|
|
Open `http://localhost:8080` and finish in the setup wizard as above. (Set a `POSTGRES_PASSWORD` in
|
|
`.env` too.)
|
|
|
|
## HTTPS / public access
|
|
|
|
Port `8080` over plain HTTP is fine for a LAN or a quick trial. For public access put a reverse
|
|
proxy (Caddy, Nginx, Traefik…) in front to terminate TLS, and set the **public URL** (the installer
|
|
prompt, or `OAUTH_REDIRECT_URL` in `.env`) to your `https://…` address — this also marks the session
|
|
cookie secure. Add that same `…/auth/callback` URL to your Google OAuth client's authorized redirect
|
|
URIs. Behind a proxy, also set `TRUSTED_PROXY_IPS` so the rate limiters see the real client IP and
|
|
can't be bypassed via a forged `X-Forwarded-For` — see [docs/self-hosting.md](docs/self-hosting.md#https--public-access).
|
|
|
|
## Updating & backups
|
|
|
|
```sh
|
|
docker compose -f docker-compose.selfhost.yml pull # or: docker compose pull
|
|
docker compose -f docker-compose.selfhost.yml up -d
|
|
```
|
|
|
|
That follows `latest`, which moves with every release. Set `IMAGE_TAG=0.80.0` in `.env` to pin a
|
|
version and upgrade on your own schedule; released versions are listed in [`CHANGELOG.md`](CHANGELOG.md)
|
|
and in the app's Release Notes.
|
|
|
|
**Back up before upgrading.** Migrations run automatically on startup and are not undone by pulling
|
|
the old image back. A backup is the database dump **plus your `.env`** — `TOKEN_ENCRYPTION_KEY` is
|
|
what decrypts the Google, SMTP and Plex secrets stored in that database, so a dump restored without it
|
|
is only half an instance. Use `scripts/backup.sh` (or `backup.ps1` on Windows) and `scripts/restore.sh`;
|
|
the equivalent one-liners, the pinning recipe and the rollback steps are in
|
|
[docs/self-hosting.md](docs/self-hosting.md#backups).
|
|
|
|
## How it works
|
|
|
|
- **Shared catalog, private state.** Channels and videos are stored once and shared; each user's
|
|
subscriptions, tags, playlists and watch/save/hide state are private.
|
|
- **Cheap by design.** Public reads are cached locally; a shared daily quota budget and a background
|
|
scheduler keep unattended syncing within YouTube's free API limits. An optional API key lets
|
|
backfill run without depending on a user's OAuth token.
|
|
|
|
## Tech
|
|
|
|
FastAPI + PostgreSQL (SQLAlchemy, Alembic) backend; React + Vite + Tailwind + TanStack Query
|
|
frontend; packaged as a single Docker image with Docker Compose.
|
|
|
|
## Note
|
|
|
|
This project is developed with AI assistance.
|