RUL-036 — Every test is categorised functional or integration, by what it depends on

Ruling

Mohamed, 2026-10-14, after asking why the suite tests against containers, verbatim: "also tests must be categorized, fnctional vs integration".

Asked what decides the category, he chose by dependency:

• Integration: the test uses any test-fleet service (Redis, the graph, the Prolog container, the kernel, the gateway) or the network.
• Functional: the test runs fully in-process. In-process janus Prolog counts as functional.
• The category is derived from the fixtures a test uses, never hand-set, and a gate fails any test that touches infrastructure without carrying the integration category.

Evidence

Measured 2026-10-14 on the tests AST snapshot (plans/ast-tests.json, 2026-10-09, worktree orchestrata-ultimate-architecture, behind master):

• 1,848 test modules, 36,442 test functions.
• pyproject.toml registers unit, integration, e2e, requires_redis, requires_graph and requires_prolog under --strict-markers. Test functions decorated with any of them: 0.
• Decorated test functions overall: llm_integration 25, slow 21, known_slow 2, incus_live 1. 56 modules set a module-level pytestmark (markers not resolved).
• Lower bound: at least 1,752 of 1,848 modules (95%) carry no category marker.
• The only categorisation in force is the default lane -m "not slow and not llm_integration and not incus_live".
• tests/ is organised by package; tests/integration/ and tests/e2e/ hold one module each. Under the no-mocks rule (RUL-002) most tests reach real Redis, graph or Prolog, so nothing visible says which ones need the 10.53.2.x test fleet.

Enforcement

• none (judgment)

Mechanism (RUL-014)

Judgment-only at recording time. Candidate mechanism, a follow-up packet (RUL-022 contract first):

1. A collection hook in tests/conftest.py computes each test's fixture closure and applies integration when it contains an infrastructure fixture (Redis, graph, kernel or gateway client, test-fleet endpoint), otherwise functional. Register functional in pyproject.toml and settle the overlap with the existing unit / integration / requires_* markers in the packet's design.
2. A gate that fails a test which opens a connection to a test-fleet endpoint without carrying integration, so a missing fixture mapping cannot misfile a test as functional.
3. scripts/run_tests.py reports pass/fail counts per category.

When the gate lands, the index row moves from none (judgment) to the gate and this section names the commit.

Verification

• Open violation today: the lower bound above (at least 1,752 of 1,848 modules uncategorised).
• Done when every collected test carries exactly one of functional / integration: a collection with -m "not functional and not integration" selects 0 tests, and the gate reports 0 infrastructure-touching tests marked functional.