fa0b0edd634bc588e5d75f7bd6824db3280b20ce
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 peter
|
14f1c97c0a |
fix(e12): a bare up -d picks the wrong compose file, and recreates the database
Round four, on round three's diff. 47 -> 34 -> 15 -> 7 raw findings. **The printed restart command could strand the restored data.** Round three replaced `start` with `up -d` because `start` ignores a newly pinned tag — but printed it bare. In the git-clone layout BOTH compose files exist, so the scripts' COMPOSE_FILE fallback never fires and a bare `docker compose up -d` reads `docker-compose.yml`: different volume names, locally built images, no IMAGE_TAG at all. `stop` and `exec` had been fine there (they resolve running containers by project and service), but `up -d` RECREATES — so the operator pastes the hint straight after a successful restore and gets `db` on a new empty volume and a first-run setup wizard, with their data in an orphaned volume. Verified: the two files resolve `siftlode_pgdata` vs `pgdata` and different images. The scripts no longer print a command that recreates from a file they did not resolve. They name the compose file when they resolved one, and otherwise say to start it the way the operator normally does — with the caveats on their own lines rather than in a parenthetical at the end of a very long one. **The reset flag is now decided where the live hash is already parsed.** Round three cleared it in the token effect, which only runs when a NEW verify token arrives — so the back button, or re-pasting an already-consumed verify link, still left the banner promising a recovery whose token was gone. Setting it from the hash on every fragment change is both smaller and closes those. Also: the doc said `start` "still carries the old tag" four lines after "pin the older IMAGE_TAG", where "old" then reads as the tag being rolled back TO — it now reuses the scripts' unambiguous wording; the Windows recipe removes a partially-copied dump before throwing, so a mid-stream `cp` failure cannot leave a truncated file under the final name; and `$WarningPreference` is pinned beside `$ErrorActionPreference` so an inherited SilentlyContinue cannot drop the half-restored warning, with the comment corrected to say what the warning stream actually is. Gate green: vitest 685, pytest 950. |
||
|
peter
|
fc56f514b3 |
fix(e12): the restart hint pointed at the one command that undoes a rollback
Round three, on round two's diff. The trend across the loop is 47 -> 34 -> 15 raw findings; the ops half kept producing the sharpest ones because the gate cannot run it. **`docker compose start` ignores a newly pinned IMAGE_TAG.** Round two removed the automatic restart because it re-ran `alembic upgrade head` — and then told the operator to run that exact command by hand, in both scripts and the docs. `start` relaunches the container that already exists, which still carries the tag being rolled back FROM, so pinning 0.79.0 and following the printed instruction brings 0.80.0 back up and re-applies its migrations. Verified against a real daemon: after changing the pinned tag, `start` kept the old image and only `up -d` picked up the new one. Both scripts and the doc now say `up -d`, with the pin-first caveat where the operator reads it. **The reset flag outlived its token.** Reverting round one's "each link clears the other" went one step too far: a `#verify=` link pasted over a standing reset notice replaces the whole fragment, so the reset token is gone — but `resetPending` stayed true, and being checked first it hid the confirmation while promising a recovery that signing out could no longer complete. The flag is now cleared when the fragment it describes is gone, which is narrower than what was reverted and does not blank the signed-out card. **A test that could not fail.** The shared-asset regression test seeded the asset as `pending`, and `_reset_asset_pending` only acts on `downloading` — so the release half was a no-op and deleting it kept the suite green. Seeded as the worker actually leaves it now. Also: the malformed-URL guard answers with urlguard's structured detail rather than a bare English string, so both halves of one typo class get the same localized copy, and the third site in that module handing user input to `urlparse` was swept with it; the asset is released after the job is terminal, not before; the Stats 403 toast gets the wizard handle its own comment claimed it had; the pasted Windows recipe sets `$ErrorActionPreference` inside the block (a scriptblock inherits the caller's, so a `SilentlyContinue` profile swallowed every throw) and cleans up the container-side dump on the failure path too; the restore warning moved to the warning stream so a `> log` redirect cannot file it away; and two comments narrowed to what the code does. Gate green: vitest 685, pytest 950. |
||
|
peter
|
358d722ce9 |
fix(e12): close what round one broke, and revert two fixes that overreached
Round two of the review loop, on round one's own diff. Two of its findings were regressions the fixes themselves introduced, which is exactly what the loop is for: **A per-user refusal was writing onto a shared row.** MediaAssets are deduped across users, and the generic failure handler puts the message on the asset — right for an upstream failure, wrong for "your storage is full". Two users queuing the same video share one asset, so the first user hitting their cap failed the second user's job with the first one's private reason, telling someone with an empty library to free up space. The refusal now raises `_OwnerRefused`, caught separately: the job fails, the asset goes back to `pending` for whoever else holds it. Pinned by a test that drives `_download` directly — the refusal happens before extraction, so nothing needs stubbing. **The restore scripts restarted the app, undoing the restore.** `docker compose start` re-runs the container command, and that runs `alembic upgrade head` — so on the rollback path this documents as recommended (restore the dump, THEN pin the older tag) the newer image immediately re-applied the migrations the restore had just undone, and on a FAILED restore it put the app back on a half-restored schema. Both are the opposite of why the stop was added. Nothing is restarted automatically now; the scripts say what to run and when, and the docs' snippets say the same. **Reverted rather than patched:** the OnboardingWizard toast (the wizard already renders that failure inline, next to a Retry button — three recoveries for one failure), and `useVerifyEntry`'s "each new link clears the other". The latter blanked the signed-out card's only banner, where nothing renders the reset flag to replace it. The two states are independent again; what changes is that `dismiss` peels ONE layer, which is what the original finding was actually about. Also: `Stats` uses the YouTube helper rather than the bare one, so the route's English 403 detail no longer beats the translated fallback under a Hungarian UI; `urlparse` guarded alongside `.port` (an unclosed bracket raised just as late), the reason renamed `malformed` since "bad port" misdescribes it; three docstrings narrowed to what the code actually does (the worker's byte check is per-job, not an aggregate ledger, and not on the edit path; the redirect residual leaks a reachability oracle, which the note had denied; the invalidation helper does not cover suspend); the pasted Windows recipe wrapped so `throw` can stop it; the bash one-liner's failure branch no longer exits 0; and the tests that were missing for the round-one fixes — the account panel's error paths, the invalidation set, and the shared-asset case above. Gate green: vitest 685, pytest 950. |
||
|
peter
|
fe38dfbcca |
fix(e12): close the eighteen findings the E12 review round left standing
The deep review of the branch raised 47 findings, 18 of which survived adversarial verification. Grouped by what they actually were: **The size ceiling inverted at its boundary.** `remaining_bytes` clamped to 0 for a user already at their storage cap, and `build_ydl_opts` reads 0 as "no limit" — so the one user over quota was the only one downloading with no ceiling at all. `check_enqueue` could not catch it: it ran while the job was queued and the user was still under the cap. The arithmetic moves into `quota.remaining_bytes` (where every other max_bytes decision lives, and where it can be tested), and the worker now fails the job outright rather than proceeding unbounded — as a non-retryable `quota_exceeded`, since retrying cannot help until space is freed. **Silent failures in the new account panel.** All three actions went through `quiet: true` endpoints, which suppress the global error dialog on the promise that the caller reports inline — with no `onError`, that promise was broken and a refusal rendered as nothing. On the broken-SMTP instance this panel exists for, an admin would read silence as "the mail went out". The co-invalidation set and the email-off predicate move to `lib/adminUsers`, shared with the pending-access queue, so the two copies of these mutations cannot drift apart again — which is how half of this was lost in the first place. **A rollback recipe that destroyed data without working.** `alembic downgrade -1` steps back one migration, but a release routinely ships several (0.80.0 shipped four): it drops the newest one's columns and leaves the schema at a revision the older image also cannot resolve, i.e. the crash loop it was meant to escape, minus the data. The dump restore is now the recommended path, and the alembic route asks the target version for its own head revision instead of counting backwards. Also: redirects named in the urlguard docstring beside the DNS-rebinding residual they share a fix with (SEC-17); a malformed port refused as `bad_port` instead of escaping as a 500; the pull cooldown extended to the sibling route spending the same QuotaAction; restore stops the app before dropping its tables; the PowerShell cleanup can no longer decide the script's verdict on 5.1, nor report the wrong exit code; the documented one-liners hardened to match the scripts; the registry's retention window stated where pinning is advertised; the two banners made mutually exclusive so neither can shadow the other; `hover:bg-hover` (a utility this project never generated) replaced by the `selectable` recipe; the clipboard helper reused; and the sync cooldown's copy swept to the two callers that discarded it. Verified: gate green (vitest 678, pytest 949), and the panel, the row hover and the reset banner UAT'd on localdev. |
||
|
peter
|
688efb9ed6 |
feat(selfhost): license the repo, and make Plex actually usable on the public compose files
DX-10: the repo invited people to clone, build and self-host while carrying no LICENSE at all — which legally means all rights reserved, so none of that was permitted. Adds the verbatim GNU AGPL-3.0 text, the SPDX id in package.json and a README section. AGPL rather than MIT deliberately: this is a server application, so the case worth covering is someone running a modified copy as a network service. DX-12: .env.example documented PLEX_MEDIA_HOST_PATH -> /plex-media and the README advertised the Plex library, but only the localdev and home compose files carried the mount — on both PUBLIC files the module could read Plex's catalogue and then fail to open a single file, because it streams the media itself rather than proxying Plex. Both api and worker now mount it read-only with an empty named volume as the fallback, and self-hosting.md gains the section that explains the server URL/token, the path map and the mount together (it had never mentioned Plex). DX-15: the worker had no init: true on either public file, though localdev and home both carry tini with the reason written out — without PID 1 reaping, a watchdog hard-kill of a download's process group orphans its ffmpeg/yt-dlp grandchildren. Verified: 'compose config' resolves on both files, the Plex mount comes out read_only: true on api and worker, and init lands on worker (not just the bgutil-pot sidecar). Gate green: vitest 671, pytest 896. |
||
|
peter
|
ca4e559cc7 |
fix(ops): make backup/restore work, and give upgrades a pin and a rollback
DX-13: both shell scripts ran bare 'docker compose', which has no config
file to find in the layout docs/self-hosting.md tells people to use (only
docker-compose.selfhost.yml is downloaded) — they now fall back to it, while
a source checkout keeps resolving docker-compose.yml as before. The two
PowerShell scripts moved the dump through the PS pipeline, which corrupts
binary on 5.1 and cannot express -Encoding Byte at all on 7 ('Byte' is not a
supported encoding name), so they simply failed there; they now write inside
the container and 'docker compose cp' it across. backup.sh also wrote its
output straight into backups/, leaving a truncated file that looks like a
real backup when pg_dump fails — it writes a .part and promotes on success.
Both restores report a non-zero pg_restore rather than printing 'Restored'
over it. Docs now state that a backup is the dump PLUS .env: without
TOKEN_ENCRYPTION_KEY every encrypted secret in the restored database is
unreadable, which the old one-line 'back up the volume' never said.
DX-14: 'latest' moves with every release and migrations run on every boot,
so self-hosters had no way to choose when they move and no way back. Docs
gain IMAGE_TAG pinning, a rollback recipe (image-only vs schema-bearing) and
the migration-failed-at-boot case; CHANGELOG.md is generated from
releaseNotes.ts so the file cannot drift from what the app shows, and
'siftlode release' refuses on a stale one rather than regenerating it into
the clean tree it just demanded.
Verified: dump + restore round-trip on both platforms against localdev into
a scratch database (8 users, 262822 videos, schema 0082_invite_lang), and
IMAGE_TAG=0.80.0 resolves on both services in 'compose config'.
|
||
|
peter
|
7c431d73fb |
fix(config): degrade gracefully on a TOKEN_ENCRYPTION_KEY mismatch instead of a 500 (B-core-2)
A rotated or missing TOKEN_ENCRYPTION_KEY makes every stored secret undecryptable.
sysconfig._raw_override called security.decrypt() with no guard, and it sits on
hot paths (/api/me via google_enabled, login, every YouTube scheduler job,
account purge) — so a key mismatch raised InvalidToken and 500'd the whole app,
the SPA wouldn't even boot, with no documented recovery.
Catch the decrypt failure in _raw_override -> warn once per key and fall back to
the env/config default. The YouTube client's refresh-token decrypt now raises a
clear YouTubeError ("reconnect") that every caller already handles, and
purge_user uses decrypt_optional so a best-effort revoke can't 500 the delete.
Document that the key must never change once secrets are stored (recovery: clear
the affected system_config rows and re-enter).
|
||
 peter
|
df8a2a6d52 |
chore: point repo/registry URLs at git.b1fr0st.eu (Forgejo->Gitea migration)
- Welcome.tsx REPO_URL (footer link) — takes effect on next image rebuild - README clone URLs, docs/self-hosting.md image ref - docker-compose.selfhost.yml image refs |
||
|
peter
|
db2257c5e7 |
feat(auth): SA3 — trusted-proxy X-Forwarded-For for rate limiting
_client_ip trusted the first X-Forwarded-For hop unconditionally, so anyone able to reach the app port could forge XFF and dodge the login/register/reset/demo rate limits. Now trust XFF ONLY when the request's socket peer is a configured reverse proxy (settings.trusted_proxy_ips, e.g. the VPS Caddy's WireGuard peer IP), and take the RIGHTMOST entry — the client our proxy actually saw and appended, immune to a client pre-seeding a fake XFF. A request from any other peer (hitting the port directly) is keyed on its real socket IP, so XFF can't be forged to bypass the limits. New TRUSTED_PROXY_IPS env (empty default = no proxy, use the direct peer). Documented in .env.example, docs/self-hosting.md, README. Unit-verified against spoof-through-proxy and direct-bypass cases. |
||
|
peter
|
58dd06738f |
chore(release): v0.22.0 — Download Center worker/sidecar in all composes + docs
Prep the Download Center epic (Phase 1 + editor + share) for prod/self-host:
- Dockerfile: create /downloads owned by appuser so a named-volume mount is writable (prod Linux).
- docker-compose.{home,selfhost,yml}: add the 'worker' (yt-dlp/ffmpeg job loop) + 'bgutil-pot'
(PO-token) services + a downloads mount (DOWNLOAD_ROOT, WORKER_ENABLED). Media defaults to a
named volume; DOWNLOAD_HOST_PATH points it at a host dir (e.g. a Plex-readable folder).
- README / docs/self-hosting.md / .env.example / install.{sh,ps1}: document the Download Center,
the two extra containers, and DOWNLOAD_HOST_PATH.
- VERSION 0.22.0 + releaseNotes entry.
|
||
|
peter
|
4951c32f6d |
feat(selfhost): one-command self-host package (epic 6d)
- docker-compose.selfhost.yml pulls the published image (forge.b1fr0st.eu/peter/siftlode) with a bundled Postgres — no source build on the operator's host. - install.sh / install.ps1 generate the four required secrets (.env: POSTGRES_PASSWORD, SECRET_KEY, a valid Fernet TOKEN_ENCRYPTION_KEY, OAUTH_REDIRECT_URL), start the stack, and print the first-run setup-wizard URL. Everything else is configured in the web wizard. - docs/self-hosting.md: the copy-paste playbook (get the files, run the installer, finish in the wizard, optional Google/SMTP, HTTPS/reverse-proxy, updates + backups). - Verified end-to-end: a fresh install from the published image boots into setup mode and serves the wizard. |