Orchestrata Ultimate › Gates › test-purity

Gate: test-purity

At a glance

What it refuses

A test that uses a mock, stub, fake, monkeypatch or skip instead of real infrastructure — and, in a separate sweep, an import of a banned library (fakeredis, pyswip, unittest.mock, xdist, …) anywhere in src/, scripts/, or tests/.

Why it exists

The module docstring is operational: "Audit test suite for forbidden patterns: mocks, stubs, fakes, monkeypatch, skips. Scans tests/ directory and reports every file that uses forbidden patterns." The doctrine behind it is RUL-002 / the repo-wide "no mocking — real implementations only" law (CLAUDE.md Key Rules); the gates.py registry comment states the scoping decision:

"test-purity's positional paths scope is tests/ ONLY — mock/stub/monkeypatch/skip are RUL-002 test-purity categories (real infra, no doubles) that do not generalize to production code … banned_import is different in kind: an import of a banned library … is an architecture violation wherever it appears, so --banned-import-only sweeps src/ and scripts/ for that ONE category, closing the gap where such an import outside tests/ was invisible to this gate entirely."

The banned-name set is read through the ONE canonical parser (core.banned_libraries), never a second copy — the script's own comment records why: "A hardcoded second parser is the exact defect that module's own docstring documents twice already … this checker is not adding a third."

How it runs

• Command (battery argv): python scripts/audit_test_purity.py tests/ --banned-import-only src/ scripts/
• When it fires: three places —
  1. the scripts/gates.py battery (registered as test-purity);
  2. the suite preflight (scripts/run_tests.py::_run_gates, blocking — it joined the preflight because it "previously only ran inside the manual health_check, so its findings could not block a suite run");
  3. scripts/health_check.py.
• The preflight run is narrower than the battery run. _run_one_gate() calls [sys.executable, gate_script, *extra_args], and production passes no extra arguments, so the preflight scans the default paths=["tests/"] with --banned-import-only empty: the src//scripts/ banned-import sweep happens only in the battery (read through the AST toolkit on 2026-10-23).
• Inputs: fingerprint roots ("tests", "src", "scripts", ".claude/banned-libraries.txt") — the registry itself is a root because "a package added there must invalidate a cached PASS even when no file under tests/src/scripts changed."
• Caching: the battery verdict is cached in .cache/gates.json against _fingerprint(roots): a sha256 over the relative path and bytes of every file under the directory roots whose suffix is .py, .toml, .pl, .json, .md or .sh (_interesting() in scripts/gates.py), plus the banned-libraries registry as a file root. There is no per-gate suffix mask: this page previously quoted one, and git log --all -S GATE_MASKS -- scripts/gates.py finds no commit that ever contained a GATE_MASKS table (checked 2026-10-23).

Typical failures and how to fix them

• Exit code: 1 if any violation is found (main() returns 1 if total_violations > 0 else 0); 0 clean.
• What a PASS looks like (real recorded battery output, .cache/gates.json):

  62.4s | ============================================================
  SUMMARY: 0 files with 0 total violations
  ============================================================
  

• What a failure looks like (representative): each offending file listed with its violation category (mock / stub / monkeypatch / skip / banned_import) and the matching pattern.
• Un-blocking: rewrite the test against real infrastructure (the Redis/graph/ gateway fleet exists for exactly this), remove the skip, or remove the banned import. There is no waiver flag; the pattern lists are the contract.
• A banned import in src/ or scripts/ that the suite run did not catch. Expected: only the battery sweeps those trees. Run python scripts/gates.py --only test-purity before calling the tree clean.
• A pytest.skip used as an infrastructure guard. RUL-002 bans skips; RUL-010 forbids skipping infrastructure verification. Make the infrastructure available instead.

Wiring

• Invokers: scripts/gates.py, scripts/run_tests.py (preflight), scripts/health_check.py.
• Prevention layer upstream: the session hooks no-doubles-py.sh (blocks writing a double) and forbidden-doubles-stop.sh (blocks a turn that wrote or planned one) — see the session-hooks family page. This gate is the detection layer for whatever slips past them.
• Tests: tests/scripts/test_audit_test_purity.py.

Related

• Gates master index
• Development workflow
• RUL-002 — No mocks, stubs, fakes, skips or monkeypatch
• RUL-010 — Never skip infrastructure verification
• source-provenance, prolog-boundary and rulings-lint — the other preflight gates
• Suite preflight family
• parallel-safety — the other test-discipline gate (flakiness rather than purity)