RUL-032 — A test runs once per code state; escalation never re-runs a proven test

Ruling

A test module that has passed is not run again until the code under test or the test itself changes. Mohamed, 2026-09-22, verbatim: "with the rule enforced that test only run once unless the code in the test or code is changed, and watch for escalation to full suite when the tests have already been run, this should never happen." This sharpens the 2026-09-21 ruling ("no test that passes must run twice … if the hook escalates to a full run then the logic is wrong") into a mechanism with a definition of "changed".

"Code state" of a test module is the content of:

the test module itself;
every src/ module in its transitive import closure (the serialized AST dependency graph, the same closure scripts/affected_tests.py walks);
every conftest.py from the repo root down to the module's directory;
pyproject.toml.

Nothing else is in the key. In particular an env/*.env change is not a code change: it no longer escalates the suite, and it never re-runs a module whose key is unchanged. A module whose key changed runs; a module whose key is unchanged is reported as reused, with the counts of its last green run, and is never scheduled.

Why this needed a ruling rather than a cleanup

Measured 2026-09-22 on orchestrata-ultimate: the last verified base was 66f18509d (2026-08-26). The affected-test selector resolved the change set since then to 1008 tests and escalated to the full suite (1608 modules, 400–490 s) on three grounds: env/test.env, env/dev.env and a module-level edit of the root conftest.py. The same day, 197 gateway, kernel and CXP modules (3977 tests) had already passed on the identical code, recorded in reports/run-tests-latest.json. The ship gate would have run them again. Every prior fix bounded one escalation cause at a time; the ruling names the invariant instead: a proven module is not re-run.

Enforcement

gate scripts/run_tests.py RUL-032

Since 2026-11-03 (ADR-0171) the ledger itself is codetools' deploy green-ledger verb; scripts/run_tests.py calls the installed codetools for every partition and record, and scripts/test_green_ledger.py is deleted. The mechanism text below names the old file; the codetools module codetools.testing.green_ledger computes byte-identical keys (2,008 of 2,008 orchestrata modules, stage E4).

Mechanism:

The ledger — scripts/test_green_ledger.py computes each module's content key and keeps reports/test-green-ledger.json ({module: {key, passed, run_id, head_commit, ts}}, git-ignored like the duration ledgers). --seed-from reports/run-tests-latest.json admits a recorded green run only for modules whose key inputs are byte-identical to that run's tree.
The runner — scripts/run_tests.py drops every module whose current key matches the ledger before batching, prints the reused count, folds the reused modules' recorded totals into the report under reused, and records a fresh key for every module that passed (batch or solo retry). --force bypasses the ledger. An escalation therefore never re-runs a proven module: the escalated set is pruned by the same key check.
The selector — scripts/affected_tests.py no longer escalates on env/*.env; a changed env key resolves to the tests and src/ modules that name that key, plus the smoke floor.

Verification

Recorded in the ADR-0058 / ship session of 2026-09-22; the first ship after this ruling must show reused covering the 197 already-green modules and no escalated to the FULL suite line in its gate output.

RUL-024 — "Pre-existing" is not a disposition
RUL-014 — A ruling that exists only in prose is incomplete work
Architecture migration 2026-09 — the 2026-09-21 ruling this sharpens.

Addendum 1, 2026-10-24 (append-only) — greens persist past their worktree

Ruling, Mohamed, 2026-10-24, verbatim: "but the ledger should persist and only changes and blast radius should run", then "before we run anything add logic on merge to copy successful tests ledger entries from the worktree to the master ledger and do that first".

Context (measured 2026-10-24): the ledger is reports/test-green-ledger.json inside each checkout. 26 checkouts held separate ledgers (1 to 1,997 entries); worktrees.py gc deleted 8 merged worktrees that morning together with their greens; the prod gate then ran cold in a fresh worktree (38,762 tests in 1,944 modules, 1,668 modules cut off by the 300 s cap, FAIL test test) although most of those modules had passed on identical bytes elsewhere.

Ruling as mechanism:

The master ledger is reports/test-green-ledger.json of the checkout that holds master (the git worktree list --porcelain rule deploy/ship-merge-master.sh already uses).
On merge, ship's "merge to master" step copies the worktree's green entries into the master ledger — only entries whose key equals the key recomputed in the merged tree (proven on master's exact bytes); a newer master entry is never overwritten. worktrees.py gc performs the same copy before it removes a merged worktree, and keeps the worktree if the copy fails.
Every run reads it. scripts/run_tests.py reuses a module from the master ledger when its own ledger has no entry at the current key, so a fresh worktree and the prod gate run only the modules whose key changed: the changes and their blast radius.
Writes to a ledger are serialized by a file lock (concurrent ships share the master ledger).

Relation to ADR-0088 (Accepted 2026-10-11; one shared SQLite ledger per repository under the git common dir): its implementation sits unmerged on feat/shared-ledger (22 commits, 509 behind master on 2026-10-24). This addendum is the mechanism on master's ledger now; ADR-0088's import rule ("only where the recorded key still equals the key recomputed now") is the same rule step 2 applies.