Orchestrata Ultimate › Gates › coverage-claude

Gate: coverage-claude

At a glance

What it refuses

The claude/ scope's line coverage, measured by a real pytest+coverage run, dropping below the baseline recorded at the call site (--min 19.5 today). It does not demand any universal target — only that coverage never gets worse.

Why it exists

Two rulings meet here. From the gates.py registry comment:

"coverage-claude is the RATCHET half of the same 'claude/ is production, not scaffolding' ruling plugin-drift enforces: claude/ (the deployed hooks + tools — task_context_hook.py rewrites every dispatched agent's prompt, route_pytest_to_threads.py decides whether a test command runs at all) is in [tool.coverage.run] source but was NEVER gated on a floor (user ruling 2026-08-19)."

And from scripts/coverage_ratchet.py's docstring, the ratchet philosophy:

"Why a ratchet and not fail_under = 90 directly (user ruling 2026-08-19): the AGENTS.md 'Definition of done' 90% bar stays the AMBITION; this gate is the ENFORCEMENT MECHANISM that makes ANY floor real without demanding 90% on day one from code that has never been measured… A hard fail_under = 90 would have been red on arrival, and a gate that is red on arrival gets waived, then ignored, then defanged — this project watched exactly that happen to earlier, unenforced coverage prose. A ratchet starts GREEN and can only go up: the baseline is a CLI flag inlined at the call site… tightened by hand as real tests land, never loosened to make a red run green."

The baseline history is recorded in the registry comment: 12.5 at birth (measured 12.96%), tightened to 19.5 on 2026-08-19 after two new test modules "landed real coverage on claude/'s two highest-blast-radius previously-untested modules, moving the measured claude/ scope from 12.96% to 19.80%… 19.5 keeps ~0.3 points of float-jitter headroom." The standing instruction: "Tighten --min by hand as real tests land; never loosen it to paper over a drop."

How it runs

• Command (battery argv):

  python scripts/coverage_ratchet.py --scope claude --paths claude \
      --tests tests/claude tests/scripts/test_task_context_hook.py --min 19.5
  

• When it fires: the scripts/gates.py battery (registered as coverage-claude).
• The measurement is real, every time: "a real python -m pytest --cov=<paths> --cov-report=json:... subprocess against the given --tests targets… the number comes from an actual test run every time, never a stored number re-asserted." The live-observer plugin is always injected — "a gate subprocess is still a test run" (RUL-009).
• Epsilon: "A small epsilon (default 0.05 points, --epsilon) absorbs float rounding, not real regressions."
• Inputs: fingerprint roots ("claude", "tests/claude", "tests/scripts/test_task_context_hook.py").
• Caching: the verdict is cached in .cache/gates.json against _fingerprint(roots): a sha256 over the relative path and bytes of every file under those roots whose suffix is .py, .toml, .pl, .json, .md or .sh (_interesting() in scripts/gates.py, read through the AST toolkit on 2026-10-23). There is no per-gate suffix mask: this page previously quoted a GATE_MASKS comment, and git log --all -S GATE_MASKS -- scripts/gates.py finds no commit that ever contained one. Files under claude/ with other suffixes (5 .j2 templates, 2 .envrc files, counted 2026-10-23) do not invalidate the cached verdict (derived from the code, not measured).

Typical failures and how to fix them

• Exit codes: "Exit 0 at/above baseline (or when --min is omitted — advisory-only…). Exit 1 when coverage DROPS below --min. Exit 2 when the coverage run itself failed or its JSON report could not be produced/parsed."
• State over time: on 2026-08-29 the cached battery verdict was ok: false; the primary checkout's .cache/gates.json, last written 2026-09-22 10:01, records ok: true. The fix for a red verdict is landing tests, never lowering the flag.
• Exit 2 from the observer. The coverage run is a pytest subprocess with the live observer injected; since the RUL-038 amendment an unstamped pytest session that spans more than one test file stops at collection (rc 4, 0 tests). Whether coverage_ratchet.py stamps ORCHESTRATA_SANCTIONED_RUNNER was not measured; see test-phase-cap.
• Un-blocking: write real tests for the regression that dropped the number (or fix the failing test run). Loosening --min to green a red run is the exact defanging the docstring forbids.

Wiring

• Invokers: scripts/gates.py (battery).
• Pattern siblings: pyright_ratchet (the shape this gate copies; it left scripts/ for codetools in 71b4cd46f, 2026-10-20, ADR-0118 decision 2, and scripts/pyright_ratchet.py no longer exists in the tree), the AST toolkit's untyped --max N / check-traces --max N ratchets.
• Tests: tests/scripts/test_coverage_ratchet.py.

Related

• Gates master index
• Development workflow
• RUL-034 — Definition of Done, five items
• RUL-009 — Never run tests without live observation
• ADR-0069
• ADR-0118 — tier-1 scripts transfer to codetools
• plugin-drift — the currency half of the same "claude/ is production" ruling
• Session-hooks family — the code this gate keeps tested