peter
b30c60d1a5
The shell cleared a hand-kept key list on a page switch (it missed the Files and feed keys) and cleared nothing when a page opened from a link carrying another page's keys. ModuleDef.urlKeys now names every key a page's view writes, the shell derives both from the registry, and moduleUrl.contract.test holds the declarations true: one owner per key, no shell key, exactly what each codec writes, and no undeclared key literal in the app.
158 lines
8.6 KiB
Markdown
158 lines
8.6 KiB
Markdown
# Development
|
|
|
|
How to run Siftlode locally and pass the gate. Conventions for the backend code itself live in
|
|
[`backend/app/README.md`](../backend/app/README.md).
|
|
|
|
## Node
|
|
|
|
One major, everywhere. `.nvmrc` pins it; the `Dockerfile`'s frontend build stage,
|
|
`frontend/package.json`'s `engines` and `@types/node` must all match it — and so must the machine you
|
|
run the gate on. `frontend/src/lib/nodeEngine.contract.test.ts` fails the gate the moment any of them
|
|
drift apart, including when yours is the odd one out. `nvm use` in the repo root picks it up.
|
|
|
|
This is a contract, not a preference: tsc and vitest read the engine's own `Intl` and web APIs, and
|
|
those change between majors — the same currency format renders `$160M` on one and `$160.0M` on the
|
|
next. A gate that is green on a major you do not ship is green about nothing.
|
|
|
|
## Run it locally
|
|
|
|
The whole stack (Postgres + API + worker) comes up from the base compose file plus the localdev override:
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml -f docker-compose.localdev.yml up --build
|
|
```
|
|
|
|
Browse at **http://localhost:8080** (Google login works — the OAuth redirect is
|
|
`http://localhost:8080/auth/callback`). The API runs `alembic upgrade head` on startup, so the schema
|
|
is always current.
|
|
|
|
For a fast frontend edit loop, run the Vite dev server with HMR against that API:
|
|
|
|
```bash
|
|
cd frontend && npm install && npm run dev # http://localhost:5173, proxying the API on :8080
|
|
```
|
|
|
|
The dev server also serves the **design lab** at `http://localhost:5173/design-lab`: the open design
|
|
decisions and the chosen patterns, rendered with the app's real CSS. It never reaches a production build
|
|
(see [`DESIGN.md`](DESIGN.md)).
|
|
|
|
## Adding a page
|
|
|
|
A frontend page is one folder, `frontend/src/modules/<id>/` (the layering is in
|
|
[`CLAUDE.md`](../CLAUDE.md) § Architecture shape):
|
|
|
|
1. The page itself, default-exported, and anything only it uses. Header content (a search box, a toggle) the
|
|
page renders into the floating header itself through `PageHeader`; the shell knows no page.
|
|
2. `module.ts`: its `ModuleDef` (`lib/moduleDef.ts`): id, label keys, icon, an optional `gate`, the page as a
|
|
`lazyWithPreload(() => import(…))`, and optionally a rail and a `Store`. Of its own folder it imports statically
|
|
only its store (a `…Provider` / `…Store` file); the page and the rail load lazily. `urlKeys` names every
|
|
address-bar key its view writes (the URL is the view: a bookmark reopens it), its list kept in `lib/` beside
|
|
its codec; the shell clears them on a page switch and opens a page without another page's keys.
|
|
3. `index.ts`, the module's public face: the definition, and only what other folders must read (its store's
|
|
hooks). Another folder reaches the module through this file or not at all.
|
|
4. The id in `lib/urlState`'s `PAGES`, and the definition in `features/navigation/moduleRegistry.ts`, in
|
|
nav-rail order. The rail, the palette, the `g` chords, the tab title and the root's stores all read from there.
|
|
5. Its strings in both locales (`src/i18n/locales`).
|
|
6. A page with a side rail also names its panel below the module, in `lib/` (rails share one layout
|
|
and collapse machinery, which lib may not learn from a module): the `PanelId` and its default group
|
|
order, storage and pref keys in `lib/panelLayout.ts`, its collapse flag in `lib/useRail.ts`
|
|
(`COLLAPSE`), and those keys in `lib/storage.ts`. The rail calls `useRail("<id>")` and hands what it
|
|
returns to `ui/FilterRail`; tsc names each table still missing the new id.
|
|
|
|
What holds it: `moduleRegistry.contract.test.ts` (every id defined once, from its own folder, lazily),
|
|
`moduleStores.contract.test.tsx` (no store reads another module), `moduleUrl.contract.test.ts` (each URL key
|
|
belongs to one module and matches what its codec writes; no undeclared key literal), the `local/layers` lint rule (imports only go
|
|
down, a module only through its face) and `chunkGuard` in the build (no lazy page on the boot path), plus the
|
|
i18n parity test.
|
|
|
|
## The gate
|
|
|
|
A change is done when this is green, from the repo root:
|
|
|
|
```bash
|
|
bash scripts/check.sh # every lane
|
|
bash scripts/check.sh lint # the static lanes only (no tests, build or audits) — the quick loop
|
|
bash scripts/check.sh --brief # either mode: one line per lane, output only for a red one
|
|
bash scripts/check.sh --list # the lane names, one per line (runs nothing)
|
|
bash scripts/check.sh --only "vitest" # one lane, by its exact name — never "the gate"
|
|
```
|
|
|
|
It runs every lane even after one fails and names each red one at the end.
|
|
|
|
The codebase's health numbers — size, complexity, test ratio, gate weight, comment noise — come from
|
|
`python scripts/audit_metrics.py` (seconds; it reports, it never fails). `--speed` adds the speed
|
|
baseline (`frontend/e2e/speed.measure.ts`: the feed's first card and a shared `/watch` link's
|
|
time-to-play against the :8080 stack, with `E2E_PASSWORD` set). An epic that changes structure
|
|
compares its before and after. What it runs, and why
|
|
each lane runs where it does, is in the script's comments; `frontend/src/lib/gates.contract.test.ts`
|
|
holds its lines, so a lane dropped or narrowed there fails a test. What each gate deliberately does
|
|
NOT check — and why — is the [gate register](gates.md). There is no CI: the gate runs here, and a
|
|
release runs it (plus e2e and the CVE scan) before anything is merged.
|
|
|
|
The lanes, for running one by hand:
|
|
|
|
**Backend**
|
|
|
|
```bash
|
|
# lint + format check (ruff is pinned in backend/requirements-dev.txt)
|
|
python -m ruff check --config backend/ruff.toml backend scripts
|
|
python -m ruff format --check --config backend/ruff.toml backend scripts
|
|
|
|
# type check — in the test image (requirements.lock does not install on Windows); new errors only,
|
|
# the pre-existing ones are in backend/.basedpyright/baseline.json
|
|
docker compose -f backend/docker-compose.test.yml run --rm --no-deps --build test \
|
|
python -m basedpyright --project pyrightconfig.json
|
|
|
|
# tests — pure-logic + the DB-backed integration lane, against an ephemeral Postgres. The backend
|
|
# tree is bind-mounted, so it always tests your working files.
|
|
docker compose -f backend/docker-compose.test.yml up --build --abort-on-container-exit --exit-code-from test
|
|
docker compose -f backend/docker-compose.test.yml down -v
|
|
|
|
# the lock against the advisory databases, at any severity
|
|
docker compose -f backend/docker-compose.test.yml run --rm --no-deps test \
|
|
pip-audit -r requirements.lock --no-deps --disable-pip
|
|
```
|
|
|
|
**Shell scripts**
|
|
|
|
```bash
|
|
# every tracked *.sh at shellcheck's default bar (info included); the version is pinned in scripts/check.sh
|
|
git ls-files -z -- '*.sh' | xargs -0 shellcheck
|
|
```
|
|
|
|
A missing database or ffmpeg is a hard error, not a skip — so the gate can't print green while the
|
|
DB-backed tests were quietly not run (`SIFTLODE_TEST_REQUIRE_DB=0` is the explicit "pure-logic only"
|
|
run for a machine without Postgres). Tests that need a database take the `db` fixture
|
|
(`backend/tests/conftest.py`); the rest are pure-logic and run without one.
|
|
|
|
**Frontend** (from `frontend/`)
|
|
|
|
```bash
|
|
node ../scripts/check-node-modules.mjs # node_modules == package-lock.json (else: npm ci)
|
|
npm run typecheck # tsc x3 (app / node / e2e)
|
|
npm run lint # eslint (typed rules included; warnings fail)
|
|
npm run format:check # prettier
|
|
npm run knip # unused files/exports/deps
|
|
npm run test # vitest (incl. the compiler and gate-register contracts)
|
|
npx vite build # the production bundle (devOnlyGuard fails it on dev-only code, chunkGuard on a lazy page loaded at boot)
|
|
npm audit --omit=dev --audit-level=high # what ships to the browser, at high or above
|
|
```
|
|
|
|
Coverage is on demand, never enforced and not in the gate (E35): `npm run test -- --coverage` in
|
|
`frontend/`, and for the backend
|
|
`docker compose -f backend/docker-compose.test.yml run --rm test python -m pytest --cov=app` (then
|
|
`… down -v`).
|
|
The DB lane runs on every core (`pytest -n auto`, pytest-xdist): `tests/db_per_worker.py` gives each worker its own
|
|
database (`siftlode_test_gw<N>`, created and migrated on first use), so the same tests run side by side. A serial
|
|
run (no `-n`) still works and uses `siftlode_test` itself.
|
|
The Playwright suite (`npm run e2e`) runs against the `:8080` stack built from your tree; the release
|
|
requires it. Before an image is pushed, it is scanned with Trivy (HIGH/CRITICAL with a fix fails;
|
|
exceptions in `.trivyignore.yaml`, each with a reason and an expiry).
|
|
|
|
## Branching
|
|
|
|
Trunk-based: branch off `main`, keep the branch gate-green, and land the whole change back to `main`
|
|
in one reviewed step. Migrations are Alembic revisions under `backend/alembic/versions`; the API
|
|
applies them on startup, and the DB-lane rebuilds the schema via `alembic upgrade head`, so a broken
|
|
migration chain fails the tests.
|