test(gates): the snapshot holds a config per scope, not a path per file
gates.snapshot.json listed ~800 file paths under their config, so every new or deleted frontend file regenerated it (31% of commits, E34 S2) and the regen diff buried the gate changes it exists to show. Now a scope is named by path (scopeOf: src, src tests, ui, lib tests, …), the snapshot holds one effective config per scope (2345 → 1456 lines), and a new test fails when a file's config is not its scope's. Not narrowed: a new file in a known place changes nothing (it gets its scope's config, checked), a glob that moves one file into another config fails (seen: an override on Feed.tsx → "src: src/components/Feed.tsx"), a scope whose rules change is a snapshot diff, and a file no config lints is still a register miss.
This commit is contained in:
@@ -20,7 +20,7 @@ green **before** any merge, release, or handing work to review.
|
|||||||
|
|
||||||
- Full gate: `bash scripts/check.sh` (`lint` for the static lanes only); the private ops dispatcher's `siftlode check` runs the same script. Single lanes: [`docs/development.md`](docs/development.md) § The gate.
|
- Full gate: `bash scripts/check.sh` (`lint` for the static lanes only); the private ops dispatcher's `siftlode check` runs the same script. Single lanes: [`docs/development.md`](docs/development.md) § The gate.
|
||||||
- **There is no CI** (owner, 2026-10-04): this gate, run here, is the only one — green before every hand-over, and `release` refuses to merge without it.
|
- **There is no CI** (owner, 2026-10-04): this gate, run here, is the only one — green before every hand-over, and `release` refuses to merge without it.
|
||||||
- **A gate with any real hit is never switched off or narrowed** — not a lint rule, a compiler flag, a lane or a release step. Fix the sites, or suppress each one inline naming its invariant (`-- why`, lint-enforced). What stays narrowed is in [`docs/gates.md`](docs/gates.md), and a contract test fails when a narrowing is missing from it. The effective ESLint config of every file and the gate/lane lines are pinned in `frontend/src/lib/gates.snapshot.json`: a new code file or any gate change fails until you regenerate it (`GATE_SNAPSHOT=update`, see gates.md) — then read that diff as a gate change. A comment saying "X is enforced by Y" is checked by running Y (E32: one such comment hid 95 functions the React Compiler skipped).
|
- **A gate with any real hit is never switched off or narrowed** — not a lint rule, a compiler flag, a lane or a release step. Fix the sites, or suppress each one inline naming its invariant (`-- why`, lint-enforced). What stays narrowed is in [`docs/gates.md`](docs/gates.md), and a contract test fails when a narrowing is missing from it. The effective ESLint config of every scope and the gate/lane lines are pinned in `frontend/src/lib/gates.snapshot.json`: any gate change fails until you regenerate it (`GATE_SNAPSHOT=update`, see gates.md) — then read that diff as a gate change. A comment saying "X is enforced by Y" is checked by running Y (E32: one such comment hid 95 functions the React Compiler skipped).
|
||||||
- **The backend type lane runs in the test image, not on the host** — `requirements.lock` cannot be installed on Windows (uvloop), and a type checker without the real packages reports nonsense. It fails on NEW errors only; the pre-existing ones sit in `backend/.basedpyright/baseline.json` and burn down epic by epic. Fixed some? Regenerate with `--writebaseline`.
|
- **The backend type lane runs in the test image, not on the host** — `requirements.lock` cannot be installed on Windows (uvloop), and a type checker without the real packages reports nonsense. It fails on NEW errors only; the pre-existing ones sit in `backend/.basedpyright/baseline.json` and burn down epic by epic. Fixed some? Regenerate with `--writebaseline`.
|
||||||
- New DB-backed test? It must actually run, not silently skip — assume `SIFTLODE_TEST_REQUIRE_DB=1`.
|
- New DB-backed test? It must actually run, not silently skip — assume `SIFTLODE_TEST_REQUIRE_DB=1`.
|
||||||
|
|
||||||
|
|||||||
+8
-6
@@ -13,16 +13,18 @@ name its invariant. Whatever is left narrowed is listed here, and
|
|||||||
**effective** configuration and fails when a narrowing is not on this page. A row here is a decision,
|
**effective** configuration and fails when a narrowing is not on this page. A row here is a decision,
|
||||||
not an oversight; a narrowing missing from here is a test failure.
|
not an oversight; a narrowing missing from here is a test failure.
|
||||||
|
|
||||||
**The snapshot.** ESLint's effective config for every code file `eslint .` reaches, and the lines
|
**The snapshot.** ESLint's effective config for every scope `eslint .` reaches, and the lines that
|
||||||
that run the lanes (`scripts/check.sh`, `backend/docker-compose.test.yml`,
|
run the lanes (`scripts/check.sh`, `backend/docker-compose.test.yml`,
|
||||||
`backend/Dockerfile.test`, the npm scripts, `vitest.config.ts`, `.prettierignore`), are committed in
|
`backend/Dockerfile.test`, the npm scripts, `vitest.config.ts`, `.prettierignore`), are committed in
|
||||||
[`frontend/src/lib/gates.snapshot.json`](../frontend/src/lib/gates.snapshot.json). The test fails on
|
[`frontend/src/lib/gates.snapshot.json`](../frontend/src/lib/gates.snapshot.json). The test fails on
|
||||||
ANY difference — a rule, an option, a glob that moves one file to another config, a new ignore, a gate
|
ANY difference — a rule, an option, a new ignore, a gate flag. A scope is named by path (`scopeOf` in
|
||||||
flag. So a change to a gate is always a diff of that file in review, never a shape a detector failed
|
the test: `src`, `src tests`, `ui`, …) and every file in it must get the same config, so a glob that
|
||||||
|
moves one file into another config fails too, while a new file in a known place changes nothing
|
||||||
|
(until E35 the snapshot listed every file, and 31% of commits regenerated it). So a change to a gate is always a diff of that file in review, never a shape a detector failed
|
||||||
to model (E32 review: three rounds of a cleverer detector each found a bypass of the last one's
|
to model (E32 review: three rounds of a cleverer detector each found a bypass of the last one's
|
||||||
fixes). Regenerate it with `GATE_SNAPSHOT=update npx vitest run src/lib/gates.contract.test.ts`
|
fixes). Regenerate it with `GATE_SNAPSHOT=update npx vitest run src/lib/gates.contract.test.ts`
|
||||||
(from `frontend/`), and read its diff as a gate change: a new file only adds its path; anything else
|
(from `frontend/`), and read its diff as a gate change. A new kind of file (a new config, a new place)
|
||||||
needs a reason. A new code file is part of that diff by design.
|
needs a line in `scopeOf`.
|
||||||
|
|
||||||
**How the register is matched:** each narrowing must appear in a table row below as its
|
**How the register is matched:** each narrowing must appear in a table row below as its
|
||||||
`code-quoted` id, in a row whose **Scope** cell names the scope exactly as the test reports it (a cell
|
`code-quoted` id, in a row whose **Scope** cell names the scope exactly as the test reports it (a cell
|
||||||
|
|||||||
@@ -101,20 +101,23 @@ function restrictedEntries(entry: Linter.RuleEntry | undefined): Set<string> {
|
|||||||
|
|
||||||
// --- the snapshot ---------------------------------------------------------------------------------
|
// --- the snapshot ---------------------------------------------------------------------------------
|
||||||
// The configs that can narrow a gate in more shapes than a check can enumerate are held to a
|
// The configs that can narrow a gate in more shapes than a check can enumerate are held to a
|
||||||
// COMMITTED SNAPSHOT (E32 review): ESLint's effective config for every file `eslint .` reaches, and
|
// COMMITTED SNAPSHOT (E32 review): ESLint's effective config per scope, and the lines that run the
|
||||||
// the lines that run the lanes. Any change fails until the snapshot is regenerated, so a narrowing
|
// lanes. Any change fails until the snapshot is regenerated, so a narrowing arrives as a diff of
|
||||||
// arrives as a diff of `gates.snapshot.json` in review instead of slipping past a detector. (Three
|
// `gates.snapshot.json` in review instead of slipping past a detector. (Three review rounds kept
|
||||||
// review rounds kept finding shapes the detector this replaced did not model — a `basePath` global
|
// finding shapes the detector this replaced did not model — a `basePath` global ignore, a widened glob
|
||||||
// ignore, a widened glob filed under a narrow row, a substring check on a gate line. A comparison
|
// filed under a narrow row, a substring check on a gate line. A comparison against everything has no
|
||||||
// against everything has no shape to miss.)
|
// shape to miss.)
|
||||||
|
//
|
||||||
|
// A scope is named by PATH (`scopeOf`), and every file of a scope must get the same effective config.
|
||||||
|
// So a new file in a known place changes nothing (until E35 the snapshot listed every file, and 31% of
|
||||||
|
// commits regenerated it), while a glob that moves a file into another config splits its scope and
|
||||||
|
// fails below, and a scope whose rules change is a snapshot diff.
|
||||||
//
|
//
|
||||||
// Regenerate: GATE_SNAPSHOT=update npx vitest run src/lib/gates.contract.test.ts
|
// Regenerate: GATE_SNAPSHOT=update npx vitest run src/lib/gates.contract.test.ts
|
||||||
// then read the diff like any gate change. An ESLint config that changed keeps its scope name when
|
// then read the diff like any gate change. A new kind of file (a new config or a new place) needs a
|
||||||
// most of its files had it; one no name fits is written as `new config N` — rename it to the
|
// line in `scopeOf` and, when it narrows anything, its register rows.
|
||||||
// register scope it is (the register test below then asks for its rows).
|
|
||||||
|
|
||||||
const SNAPSHOT = "frontend/src/lib/gates.snapshot.json";
|
const SNAPSHOT = "frontend/src/lib/gates.snapshot.json";
|
||||||
const NOT_LINTED = "(not linted)";
|
|
||||||
type EslintConfig = {
|
type EslintConfig = {
|
||||||
rules: Record<string, unknown[]>;
|
rules: Record<string, unknown[]>;
|
||||||
linterOptions: Record<string, unknown>;
|
linterOptions: Record<string, unknown>;
|
||||||
@@ -123,10 +126,24 @@ type EslintConfig = {
|
|||||||
settings: unknown;
|
settings: unknown;
|
||||||
};
|
};
|
||||||
type Snapshot = {
|
type Snapshot = {
|
||||||
eslint: { configs: Record<string, EslintConfig>; files: Record<string, string[]> };
|
eslint: Record<string, EslintConfig>;
|
||||||
lanes: Record<string, unknown>;
|
lanes: Record<string, unknown>;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/** The register scope a code file belongs to, by its path (relative to frontend/). */
|
||||||
|
function scopeOf(file: string): string {
|
||||||
|
if (!file.includes("/")) return /[.][mc]?js$/.test(file) ? "js configs" : "ts configs";
|
||||||
|
if (file.startsWith("e2e/")) return "e2e";
|
||||||
|
if (file.startsWith("src/design-lab/")) return "design-lab";
|
||||||
|
// the network owners (NETWORK_OWNERS in eslint.config.js)
|
||||||
|
if (file === "src/lib/http.ts") return "lib/http.ts";
|
||||||
|
if (file === "src/lib/api/core.ts") return "lib/api/core.ts";
|
||||||
|
const test = /[.]test[.]tsx?$/.test(file);
|
||||||
|
if (file.startsWith("src/components/ui/")) return test ? "ui tests" : "ui";
|
||||||
|
if (file.startsWith("src/lib/")) return test ? "lib tests" : "src/lib";
|
||||||
|
return test ? "src tests" : "src";
|
||||||
|
}
|
||||||
|
|
||||||
const sortKeys = (v: unknown): unknown =>
|
const sortKeys = (v: unknown): unknown =>
|
||||||
Array.isArray(v)
|
Array.isArray(v)
|
||||||
? v.map(sortKeys)
|
? v.map(sortKeys)
|
||||||
@@ -173,7 +190,7 @@ function normalize(cfg: Linter.Config): EslintConfig {
|
|||||||
|
|
||||||
// Every code file `eslint .` walks into. The skipped directories are the global ignores' generated
|
// Every code file `eslint .` walks into. The skipped directories are the global ignores' generated
|
||||||
// output (the register's `everywhere` row) — untracked, and present on one machine but not another.
|
// output (the register's `everywhere` row) — untracked, and present on one machine but not another.
|
||||||
// Every other file is asked for its config, so a file a new ignore drops lands in `(not linted)`.
|
// Every other file is asked for its config, so a file a new ignore drops is reported as not linted.
|
||||||
const WALK_SKIP = new Set([
|
const WALK_SKIP = new Set([
|
||||||
"node_modules",
|
"node_modules",
|
||||||
"dist",
|
"dist",
|
||||||
@@ -192,7 +209,7 @@ function codeFiles(dir = ""): string[] {
|
|||||||
|
|
||||||
// On disk every config but src is written as its difference from src — the rules it sets otherwise
|
// On disk every config but src is written as its difference from src — the rules it sets otherwise
|
||||||
// and the src rules it does not apply — so a scope reads like its register rows, and one rule set
|
// and the src rules it does not apply — so a scope reads like its register rows, and one rule set
|
||||||
// is not repeated fourteen times. In memory it is always whole.
|
// is not repeated a dozen times. In memory it is always whole.
|
||||||
type Stored = Omit<EslintConfig, "rules"> & {
|
type Stored = Omit<EslintConfig, "rules"> & {
|
||||||
rules: Record<string, unknown[]>;
|
rules: Record<string, unknown[]>;
|
||||||
notApplied?: string[];
|
notApplied?: string[];
|
||||||
@@ -225,15 +242,22 @@ function fromStored(stored: Record<string, Stored>): Record<string, EslintConfig
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
const committed = (): Snapshot => {
|
const committed = (): Snapshot => {
|
||||||
const s = JSON.parse(read(SNAPSHOT)) as Snapshot & {
|
const s = JSON.parse(read(SNAPSHOT)) as Omit<Snapshot, "eslint"> & {
|
||||||
eslint: { configs: Record<string, Stored> };
|
eslint: Record<string, Stored>;
|
||||||
};
|
};
|
||||||
return canonical({ ...s, eslint: { ...s.eslint, configs: fromStored(s.eslint.configs) } });
|
return canonical({ ...s, eslint: fromStored(s.eslint) });
|
||||||
};
|
};
|
||||||
|
|
||||||
async function eslintActual(names: Snapshot["eslint"]): Promise<Snapshot["eslint"]> {
|
type EslintActual = {
|
||||||
|
configs: Record<string, EslintConfig>;
|
||||||
|
/** the files no config lints */
|
||||||
|
notLinted: string[];
|
||||||
|
/** `scope: file` for each file whose config is not its scope's (the scope's majority) */
|
||||||
|
strays: string[];
|
||||||
|
};
|
||||||
|
async function eslintActual(): Promise<EslintActual> {
|
||||||
const eslint = new ESLint({ cwd: FRONTEND });
|
const eslint = new ESLint({ cwd: FRONTEND });
|
||||||
const groups = new Map<string, { config: EslintConfig; files: string[] }>();
|
const byScope = new Map<string, Map<string, { config: EslintConfig; files: string[] }>>();
|
||||||
const notLinted: string[] = [];
|
const notLinted: string[] = [];
|
||||||
for (const file of codeFiles().sort()) {
|
for (const file of codeFiles().sort()) {
|
||||||
const raw = (await eslint.calculateConfigForFile(file)) as Linter.Config | undefined;
|
const raw = (await eslint.calculateConfigForFile(file)) as Linter.Config | undefined;
|
||||||
@@ -243,35 +267,22 @@ async function eslintActual(names: Snapshot["eslint"]): Promise<Snapshot["eslint
|
|||||||
}
|
}
|
||||||
const config = normalize(raw);
|
const config = normalize(raw);
|
||||||
const key = JSON.stringify(config);
|
const key = JSON.stringify(config);
|
||||||
|
const scope = scopeOf(file);
|
||||||
|
const groups =
|
||||||
|
byScope.get(scope) ?? new Map<string, { config: EslintConfig; files: string[] }>();
|
||||||
const group = groups.get(key) ?? { config, files: [] };
|
const group = groups.get(key) ?? { config, files: [] };
|
||||||
group.files.push(file);
|
group.files.push(file);
|
||||||
groups.set(key, group);
|
groups.set(key, group);
|
||||||
|
byScope.set(scope, groups);
|
||||||
}
|
}
|
||||||
// a config is named by the snapshot entry it equals; a changed one by the old name most of its
|
|
||||||
// files had (if still free), so the diff shows the rule that moved, not a renamed scope
|
|
||||||
const byContent = new Map(Object.entries(names.configs).map(([n, c]) => [JSON.stringify(c), n]));
|
|
||||||
const oldName = new Map(
|
|
||||||
Object.entries(names.files).flatMap(([n, files]) => files.map((f) => [f, n] as const)),
|
|
||||||
);
|
|
||||||
const configs: Record<string, EslintConfig> = {};
|
const configs: Record<string, EslintConfig> = {};
|
||||||
const files: Record<string, string[]> = { [NOT_LINTED]: notLinted };
|
const strays: string[] = [];
|
||||||
const unnamed: { config: EslintConfig; files: string[] }[] = [];
|
for (const [scope, groups] of byScope) {
|
||||||
for (const [key, group] of groups) {
|
const [first, ...rest] = [...groups.values()].sort((a, b) => b.files.length - a.files.length);
|
||||||
const name = byContent.get(key);
|
configs[scope] = first!.config;
|
||||||
if (name) [configs[name], files[name]] = [group.config, group.files];
|
for (const g of rest) strays.push(...g.files.map((f) => `${scope}: ${f}`));
|
||||||
else unnamed.push(group);
|
|
||||||
}
|
}
|
||||||
unnamed.forEach((group, i) => {
|
return { configs: canonical(configs), notLinted, strays };
|
||||||
const votes = new Map<string, number>();
|
|
||||||
for (const f of group.files) {
|
|
||||||
const n = oldName.get(f);
|
|
||||||
if (n && n !== NOT_LINTED) votes.set(n, (votes.get(n) ?? 0) + 1);
|
|
||||||
}
|
|
||||||
const best = [...votes].sort((a, b) => b[1] - a[1])[0]?.[0];
|
|
||||||
const name = best && !(best in configs) ? best : `new config ${i + 1}`;
|
|
||||||
[configs[name], files[name]] = [group.config, group.files];
|
|
||||||
});
|
|
||||||
return canonical({ configs, files });
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// the lines that run each lane, comments and blank lines aside
|
// the lines that run each lane, comments and blank lines aside
|
||||||
@@ -295,10 +306,10 @@ const lanes = () => ({
|
|||||||
"frontend/.prettierignore": meaningful("frontend/.prettierignore"),
|
"frontend/.prettierignore": meaningful("frontend/.prettierignore"),
|
||||||
});
|
});
|
||||||
|
|
||||||
let actual: Promise<Snapshot> | undefined;
|
let actual: Promise<EslintActual> | undefined;
|
||||||
const current = () =>
|
const linted = () => (actual ??= eslintActual());
|
||||||
(actual ??= (async () =>
|
const current = async (): Promise<Snapshot> =>
|
||||||
canonical({ eslint: await eslintActual(committed().eslint), lanes: lanes() }))());
|
canonical({ eslint: (await linted()).configs, lanes: lanes() });
|
||||||
|
|
||||||
// --- the register, read off the snapshot ------------------------------------------------------------
|
// --- the register, read off the snapshot ------------------------------------------------------------
|
||||||
|
|
||||||
@@ -310,11 +321,8 @@ const ignorable = (rule: string, all: Record<string, unknown>) =>
|
|||||||
const entry = (v: unknown[] | undefined) => v as Linter.RuleEntry | undefined;
|
const entry = (v: unknown[] | undefined) => v as Linter.RuleEntry | undefined;
|
||||||
|
|
||||||
/** Every narrowing in the effective ESLint configs, as (scope, id). */
|
/** Every narrowing in the effective ESLint configs, as (scope, id). */
|
||||||
function eslintNarrowings({ configs, files }: Snapshot["eslint"]): [string, string][] {
|
function eslintNarrowings({ configs, notLinted }: EslintActual): [string, string][] {
|
||||||
const found: [string, string][] = files[NOT_LINTED]!.map((f) => [
|
const found: [string, string][] = notLinted.map((f) => ["everywhere", `${f} — not linted`]);
|
||||||
"everywhere",
|
|
||||||
`${f} — not linted`,
|
|
||||||
]);
|
|
||||||
const src = configs.src?.rules;
|
const src = configs.src?.rules;
|
||||||
if (!src) return [...found, ["src", "(no config named src)"]];
|
if (!src) return [...found, ["src", "(no config named src)"]];
|
||||||
for (const [scope, { rules, linterOptions }] of Object.entries(configs)) {
|
for (const [scope, { rules, linterOptions }] of Object.entries(configs)) {
|
||||||
@@ -347,22 +355,29 @@ function eslintNarrowings({ configs, files }: Snapshot["eslint"]): [string, stri
|
|||||||
}
|
}
|
||||||
|
|
||||||
describe("the gate snapshot (gates.snapshot.json)", () => {
|
describe("the gate snapshot (gates.snapshot.json)", () => {
|
||||||
it("holds the effective ESLint config of every file and the lines that run each lane", async () => {
|
it("holds the effective ESLint config of every scope and the lines that run each lane", async () => {
|
||||||
const now = await current();
|
const now = await current();
|
||||||
if (process.env.GATE_SNAPSHOT === "update") {
|
if (process.env.GATE_SNAPSHOT === "update") {
|
||||||
const path = REPO + SNAPSHOT;
|
const path = REPO + SNAPSHOT;
|
||||||
const options = { ...(await resolveConfig(path)), filepath: path };
|
const options = { ...(await resolveConfig(path)), filepath: path };
|
||||||
const stored = { ...now, eslint: { ...now.eslint, configs: toStored(now.eslint.configs) } };
|
const stored = { ...now, eslint: toStored(now.eslint) };
|
||||||
writeFileSync(path, await format(JSON.stringify(sortKeys(stored)), options));
|
writeFileSync(path, await format(JSON.stringify(sortKeys(stored)), options));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
expect(now).toEqual(committed());
|
expect(now).toEqual(committed());
|
||||||
}, 120_000);
|
}, 120_000);
|
||||||
|
|
||||||
|
// A glob that moves one file into another config (a product file into the test config) leaves its
|
||||||
|
// scope's config as it was, so the snapshot alone would not see it: the file now differs from the
|
||||||
|
// rest of its scope, and that is what fails here.
|
||||||
|
it("gives every file of a scope the same config", async () => {
|
||||||
|
expect((await linted()).strays).toEqual([]);
|
||||||
|
}, 120_000);
|
||||||
});
|
});
|
||||||
|
|
||||||
describe("the gate register (docs/gates.md)", () => {
|
describe("the gate register (docs/gates.md)", () => {
|
||||||
it("lists every ESLint rule narrowed, by scope", async () => {
|
it("lists every ESLint rule narrowed, by scope", async () => {
|
||||||
expect(missingFrom(eslintNarrowings((await current()).eslint))).toEqual([]);
|
expect(missingFrom(eslintNarrowings(await linted()))).toEqual([]);
|
||||||
}, 120_000);
|
}, 120_000);
|
||||||
|
|
||||||
it("keeps --max-warnings 0, which is what makes a warn-level rule fail", () => {
|
it("keeps --max-warnings 0, which is what makes a warn-level rule fail", () => {
|
||||||
|
|||||||
+1264
-2060
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user