The deep review of the branch raised 47 findings, 18 of which survived adversarial
verification. Grouped by what they actually were:
**The size ceiling inverted at its boundary.** `remaining_bytes` clamped to 0 for a user
already at their storage cap, and `build_ydl_opts` reads 0 as "no limit" — so the one
user over quota was the only one downloading with no ceiling at all. `check_enqueue`
could not catch it: it ran while the job was queued and the user was still under the cap.
The arithmetic moves into `quota.remaining_bytes` (where every other max_bytes decision
lives, and where it can be tested), and the worker now fails the job outright rather than
proceeding unbounded — as a non-retryable `quota_exceeded`, since retrying cannot help
until space is freed.
**Silent failures in the new account panel.** All three actions went through `quiet: true`
endpoints, which suppress the global error dialog on the promise that the caller reports
inline — with no `onError`, that promise was broken and a refusal rendered as nothing. On
the broken-SMTP instance this panel exists for, an admin would read silence as "the mail
went out". The co-invalidation set and the email-off predicate move to `lib/adminUsers`,
shared with the pending-access queue, so the two copies of these mutations cannot drift
apart again — which is how half of this was lost in the first place.
**A rollback recipe that destroyed data without working.** `alembic downgrade -1` steps
back one migration, but a release routinely ships several (0.80.0 shipped four): it drops
the newest one's columns and leaves the schema at a revision the older image also cannot
resolve, i.e. the crash loop it was meant to escape, minus the data. The dump restore is
now the recommended path, and the alembic route asks the target version for its own head
revision instead of counting backwards.
Also: redirects named in the urlguard docstring beside the DNS-rebinding residual they
share a fix with (SEC-17); a malformed port refused as `bad_port` instead of escaping as
a 500; the pull cooldown extended to the sibling route spending the same QuotaAction;
restore stops the app before dropping its tables; the PowerShell cleanup can no longer
decide the script's verdict on 5.1, nor report the wrong exit code; the documented
one-liners hardened to match the scripts; the registry's retention window stated where
pinning is advertised; the two banners made mutually exclusive so neither can shadow the
other; `hover:bg-hover` (a utility this project never generated) replaced by the
`selectable` recipe; the clipboard helper reused; and the sync cooldown's copy swept to
the two callers that discarded it.
Verified: gate green (vitest 678, pytest 949), and the panel, the row hover and the reset
banner UAT'd on localdev.
DX-10: the repo invited people to clone, build and self-host while carrying
no LICENSE at all — which legally means all rights reserved, so none of that
was permitted. Adds the verbatim GNU AGPL-3.0 text, the SPDX id in
package.json and a README section. AGPL rather than MIT deliberately: this is
a server application, so the case worth covering is someone running a
modified copy as a network service.
DX-12: .env.example documented PLEX_MEDIA_HOST_PATH -> /plex-media and the
README advertised the Plex library, but only the localdev and home compose
files carried the mount — on both PUBLIC files the module could read Plex's
catalogue and then fail to open a single file, because it streams the media
itself rather than proxying Plex. Both api and worker now mount it read-only
with an empty named volume as the fallback, and self-hosting.md gains the
section that explains the server URL/token, the path map and the mount
together (it had never mentioned Plex).
DX-15: the worker had no init: true on either public file, though localdev
and home both carry tini with the reason written out — without PID 1 reaping,
a watchdog hard-kill of a download's process group orphans its ffmpeg/yt-dlp
grandchildren.
Verified: 'compose config' resolves on both files, the Plex mount comes out
read_only: true on api and worker, and init lands on worker (not just the
bgutil-pot sidecar). Gate green: vitest 671, pytest 896.
DX-13: both shell scripts ran bare 'docker compose', which has no config
file to find in the layout docs/self-hosting.md tells people to use (only
docker-compose.selfhost.yml is downloaded) — they now fall back to it, while
a source checkout keeps resolving docker-compose.yml as before. The two
PowerShell scripts moved the dump through the PS pipeline, which corrupts
binary on 5.1 and cannot express -Encoding Byte at all on 7 ('Byte' is not a
supported encoding name), so they simply failed there; they now write inside
the container and 'docker compose cp' it across. backup.sh also wrote its
output straight into backups/, leaving a truncated file that looks like a
real backup when pg_dump fails — it writes a .part and promotes on success.
Both restores report a non-zero pg_restore rather than printing 'Restored'
over it. Docs now state that a backup is the dump PLUS .env: without
TOKEN_ENCRYPTION_KEY every encrypted secret in the restored database is
unreadable, which the old one-line 'back up the volume' never said.
DX-14: 'latest' moves with every release and migrations run on every boot,
so self-hosters had no way to choose when they move and no way back. Docs
gain IMAGE_TAG pinning, a rollback recipe (image-only vs schema-bearing) and
the migration-failed-at-boot case; CHANGELOG.md is generated from
releaseNotes.ts so the file cannot drift from what the app shows, and
'siftlode release' refuses on a stale one rather than regenerating it into
the clean tree it just demanded.
Verified: dump + restore round-trip on both platforms against localdev into
a scratch database (8 users, 262822 videos, schema 0082_invite_lang), and
IMAGE_TAG=0.80.0 resolves on both services in 'compose config'.
A rotated or missing TOKEN_ENCRYPTION_KEY makes every stored secret undecryptable.
sysconfig._raw_override called security.decrypt() with no guard, and it sits on
hot paths (/api/me via google_enabled, login, every YouTube scheduler job,
account purge) — so a key mismatch raised InvalidToken and 500'd the whole app,
the SPA wouldn't even boot, with no documented recovery.
Catch the decrypt failure in _raw_override -> warn once per key and fall back to
the env/config default. The YouTube client's refresh-token decrypt now raises a
clear YouTubeError ("reconnect") that every caller already handles, and
purge_user uses decrypt_optional so a best-effort revoke can't 500 the delete.
Document that the key must never change once secrets are stored (recovery: clear
the affected system_config rows and re-enter).
_client_ip trusted the first X-Forwarded-For hop unconditionally, so anyone able to
reach the app port could forge XFF and dodge the login/register/reset/demo rate limits.
Now trust XFF ONLY when the request's socket peer is a configured reverse proxy
(settings.trusted_proxy_ips, e.g. the VPS Caddy's WireGuard peer IP), and take the
RIGHTMOST entry — the client our proxy actually saw and appended, immune to a client
pre-seeding a fake XFF. A request from any other peer (hitting the port directly) is
keyed on its real socket IP, so XFF can't be forged to bypass the limits.
New TRUSTED_PROXY_IPS env (empty default = no proxy, use the direct peer). Documented in
.env.example, docs/self-hosting.md, README. Unit-verified against spoof-through-proxy and
direct-bypass cases.
Prep the Download Center epic (Phase 1 + editor + share) for prod/self-host:
- Dockerfile: create /downloads owned by appuser so a named-volume mount is writable (prod Linux).
- docker-compose.{home,selfhost,yml}: add the 'worker' (yt-dlp/ffmpeg job loop) + 'bgutil-pot'
(PO-token) services + a downloads mount (DOWNLOAD_ROOT, WORKER_ENABLED). Media defaults to a
named volume; DOWNLOAD_HOST_PATH points it at a host dir (e.g. a Plex-readable folder).
- README / docs/self-hosting.md / .env.example / install.{sh,ps1}: document the Download Center,
the two extra containers, and DOWNLOAD_HOST_PATH.
- VERSION 0.22.0 + releaseNotes entry.
- docker-compose.selfhost.yml pulls the published image (forge.b1fr0st.eu/peter/siftlode) with a
bundled Postgres — no source build on the operator's host.
- install.sh / install.ps1 generate the four required secrets (.env: POSTGRES_PASSWORD, SECRET_KEY,
a valid Fernet TOKEN_ENCRYPTION_KEY, OAUTH_REDIRECT_URL), start the stack, and print the first-run
setup-wizard URL. Everything else is configured in the web wizard.
- docs/self-hosting.md: the copy-paste playbook (get the files, run the installer, finish in the
wizard, optional Google/SMTP, HTTPS/reverse-proxy, updates + backups).
- Verified end-to-end: a fresh install from the published image boots into setup mode and serves
the wizard.
peter
peter