peter
fe38dfbcca
The deep review of the branch raised 47 findings, 18 of which survived adversarial verification. Grouped by what they actually were: **The size ceiling inverted at its boundary.** `remaining_bytes` clamped to 0 for a user already at their storage cap, and `build_ydl_opts` reads 0 as "no limit" — so the one user over quota was the only one downloading with no ceiling at all. `check_enqueue` could not catch it: it ran while the job was queued and the user was still under the cap. The arithmetic moves into `quota.remaining_bytes` (where every other max_bytes decision lives, and where it can be tested), and the worker now fails the job outright rather than proceeding unbounded — as a non-retryable `quota_exceeded`, since retrying cannot help until space is freed. **Silent failures in the new account panel.** All three actions went through `quiet: true` endpoints, which suppress the global error dialog on the promise that the caller reports inline — with no `onError`, that promise was broken and a refusal rendered as nothing. On the broken-SMTP instance this panel exists for, an admin would read silence as "the mail went out". The co-invalidation set and the email-off predicate move to `lib/adminUsers`, shared with the pending-access queue, so the two copies of these mutations cannot drift apart again — which is how half of this was lost in the first place. **A rollback recipe that destroyed data without working.** `alembic downgrade -1` steps back one migration, but a release routinely ships several (0.80.0 shipped four): it drops the newest one's columns and leaves the schema at a revision the older image also cannot resolve, i.e. the crash loop it was meant to escape, minus the data. The dump restore is now the recommended path, and the alembic route asks the target version for its own head revision instead of counting backwards. Also: redirects named in the urlguard docstring beside the DNS-rebinding residual they share a fix with (SEC-17); a malformed port refused as `bad_port` instead of escaping as a 500; the pull cooldown extended to the sibling route spending the same QuotaAction; restore stops the app before dropping its tables; the PowerShell cleanup can no longer decide the script's verdict on 5.1, nor report the wrong exit code; the documented one-liners hardened to match the scripts; the registry's retention window stated where pinning is advertised; the two banners made mutually exclusive so neither can shadow the other; `hover:bg-hover` (a utility this project never generated) replaced by the `selectable` recipe; the clipboard helper reused; and the sync cooldown's copy swept to the two callers that discarded it. Verified: gate green (vitest 678, pytest 949), and the panel, the row hover and the reset banner UAT'd on localdev.
132 lines
6.6 KiB
Markdown
132 lines
6.6 KiB
Markdown
# Siftlode
|
|
|
|
**Your YouTube subscriptions, the way a feed should work.** Siftlode pulls every upload from the
|
|
channels you follow into one clean, filterable feed — no algorithm deciding what you see, and no
|
|
Shorts or livestream noise unless you want it. Self-hosted, multi-user, and private: your data
|
|
stays on your own server.
|
|
|
|
|
|
|
|
Everything expensive (channels, videos, metadata) is fetched from YouTube once and stored locally,
|
|
so filtering, searching and sorting are instant and don't burn API quota. Click a video to watch it
|
|
in an in-app player that resumes where you left off — or open it on youtube.com so your own ad
|
|
blocker and SponsorBlock keep working.
|
|
|
|
## Features
|
|
|
|
- **A readable subscription feed** — sort and filter by channel, tag, language, topic, length,
|
|
upload date or watch state; hide channels without unsubscribing; save filter setups as named views.
|
|
- **Search all of YouTube** from the feed — live results you can play, save or add to playlists like
|
|
any other video; they stay temporary and out of your catalog unless you keep one.
|
|
- **Channel pages & a channel manager** — per-channel stats and uploads, priorities, and your own
|
|
tags to slice the feed by.
|
|
- **Playlists with two-way YouTube sync** — build them locally, keep them in sync in both directions.
|
|
- **In-app player** with resume, plus keyboard/scroll controls.
|
|
- **Download Center** — save videos to the server with yt-dlp in a Plex-friendly layout (format
|
|
presets, per-user storage quota), trim / crop / split & join them in a built-in editor, then save
|
|
to your device, share with another user, or hand out a public watch link.
|
|
- **Plex library** *(optional)* — point Siftlode at your own Plex server to browse and play its
|
|
movies and shows alongside your feed, with watch state kept in sync.
|
|
- **Multi-user** with per-user private state, a shared catalog, and a fair daily API-quota guard.
|
|
- **Self-hosted & private**, with a first-run web setup wizard and the interface in **English
|
|
and Hungarian**.
|
|
|
|
## Quick start (self-hosting)
|
|
|
|
You don't need to build anything — Siftlode runs from a prebuilt public image, and all
|
|
configuration (your admin account, Google sign-in, email) happens in a **first-run web wizard**.
|
|
You need [Docker](https://docs.docker.com/get-docker/) with the Compose plugin.
|
|
|
|
**1. Get the files and run the installer:**
|
|
|
|
```sh
|
|
git clone https://git.b1fr0st.eu/peter/siftlode.git
|
|
cd siftlode
|
|
./install.sh # Windows (PowerShell): ./install.ps1
|
|
```
|
|
|
|
The installer generates a private `.env` (secrets), pulls the image, starts the app + database, and
|
|
prints a one-time setup URL like `http://localhost:8080/setup?token=…`.
|
|
|
|
**2. Finish in your browser.** Open that URL and follow the wizard:
|
|
|
|
1. **Admin account** — the email + password you'll sign in with.
|
|
2. **Google sign-in** *(optional)* — paste a Google OAuth client to enable "Sign in with Google" and
|
|
pulling your YouTube subscriptions. Skip it to use email + password only.
|
|
3. **Email / SMTP** *(optional)* — for verification/notification emails. Skip it and you (the admin)
|
|
simply approve new accounts yourself.
|
|
|
|
Then sign in with your admin account. That's it. See **[docs/self-hosting.md](docs/self-hosting.md)**
|
|
for the full walkthrough.
|
|
|
|
> Just trying it out? Press Enter at the installer's URL prompt to run on `http://localhost:8080`.
|
|
|
|
## Build from source (alternative)
|
|
|
|
Prefer to build the image yourself instead of pulling it:
|
|
|
|
```sh
|
|
git clone https://git.b1fr0st.eu/peter/siftlode.git
|
|
cd siftlode
|
|
cp .env.example .env
|
|
# generate the two secrets and paste them into .env:
|
|
python -c "import secrets;print('SECRET_KEY='+secrets.token_urlsafe(48))"
|
|
python -c "import base64,os;print('TOKEN_ENCRYPTION_KEY='+base64.urlsafe_b64encode(os.urandom(32)).decode())"
|
|
docker compose up --build -d # builds from the included Dockerfile
|
|
```
|
|
|
|
Open `http://localhost:8080` and finish in the setup wizard as above. (Set a `POSTGRES_PASSWORD` in
|
|
`.env` too.)
|
|
|
|
## HTTPS / public access
|
|
|
|
Port `8080` over plain HTTP is fine for a LAN or a quick trial. For public access put a reverse
|
|
proxy (Caddy, Nginx, Traefik…) in front to terminate TLS, and set the **public URL** (the installer
|
|
prompt, or `OAUTH_REDIRECT_URL` in `.env`) to your `https://…` address — this also marks the session
|
|
cookie secure. Add that same `…/auth/callback` URL to your Google OAuth client's authorized redirect
|
|
URIs. Behind a proxy, also set `TRUSTED_PROXY_IPS` so the rate limiters see the real client IP and
|
|
can't be bypassed via a forged `X-Forwarded-For` — see [docs/self-hosting.md](docs/self-hosting.md#https--public-access).
|
|
|
|
## Updating & backups
|
|
|
|
```sh
|
|
docker compose -f docker-compose.selfhost.yml pull # or: docker compose pull
|
|
docker compose -f docker-compose.selfhost.yml up -d
|
|
```
|
|
|
|
That follows `latest`, which moves with every release. Set `IMAGE_TAG=0.80.0` in `.env` to pin a
|
|
version and upgrade on your own schedule; released versions are listed in [`CHANGELOG.md`](CHANGELOG.md)
|
|
and in the app's Release Notes. The registry keeps only the newest few of those, so a pin decides
|
|
*when* you take the next hop rather than letting you stay put forever — see
|
|
[docs/self-hosting.md](docs/self-hosting.md#upgrading).
|
|
|
|
**Back up before upgrading.** Migrations run automatically on startup and are not undone by pulling
|
|
the old image back. A backup is the database dump **plus your `.env`** — `TOKEN_ENCRYPTION_KEY` is
|
|
what decrypts the Google, SMTP and Plex secrets stored in that database, so a dump restored without it
|
|
is only half an instance. Use `scripts/backup.sh` (or `backup.ps1` on Windows) and `scripts/restore.sh`;
|
|
the equivalent one-liners, the pinning recipe and the rollback steps are in
|
|
[docs/self-hosting.md](docs/self-hosting.md#backups).
|
|
|
|
## How it works
|
|
|
|
- **Shared catalog, private state.** Channels and videos are stored once and shared; each user's
|
|
subscriptions, tags, playlists and watch/save/hide state are private.
|
|
- **Cheap by design.** Public reads are cached locally; a shared daily quota budget and a background
|
|
scheduler keep unattended syncing within YouTube's free API limits. An optional API key lets
|
|
backfill run without depending on a user's OAuth token.
|
|
|
|
## Tech
|
|
|
|
FastAPI + PostgreSQL (SQLAlchemy, Alembic) backend; React + Vite + Tailwind + TanStack Query
|
|
frontend; packaged as a single Docker image with Docker Compose.
|
|
|
|
## License
|
|
|
|
[GNU AGPL-3.0](LICENSE). You can run, study, modify and share Siftlode; if you modify it and let other
|
|
people use it over a network, they are entitled to your modified source. Copyright © 2026 Siftlode
|
|
contributors.
|
|
|
|
## Note
|
|
|
|
This project is developed with AI assistance.
|