Measurement retention, refuted prediction, and proposed rule induction

Written 2026-08-19, snapshotted against commit c24da1898e30a141a44098c97e93fc0f05c7d446, while a spec author, two builders, and a reviewer worked the same subsystem in parallel. Companion to The Prolog layer does not learn — this page goes deeper on the instrument and adds the 2026-08-19 rule-induction proposal that page does not yet cover. Full version with code excerpts: docs/architecture/measurement-retention.md in the repo.

Three states, checked against the live tree

docs/architecture/learning.md described a GovernanceRuleLearner, a CLIPS+Prolog dual rule-synthesis engine, and hot-reloaded generated rules. grep -rln "GovernanceRuleLearner" . --include="*.py" returns nothing — zero Python implementation, ever. plans/prolog-learning-subsystem-research-2026-08-12.md (14 agents, 528 tool calls) then measured the idea's prediction half and refuted it with numbers; two specs — SPEC-113 (measurement retention) and SPEC-114 (governance rule learning), both landed 2026-08-19 as Draft — are now the authoritative source for everything below.

Why the prediction half is dead

Learning beats the trivial estimator by 0.5–0.9s against a 28.2s noise band. test_executor._estimate_durations is literally history.get(m, median(history)) — the "learning" ever proposed for prediction is a last-value lookup that already ships, and every real learner measured worse.

The instrument — SPEC-113's five FRs, checked live

Real evidence, not retyped — direct head/tail on the live sink:

$ tail -1 reports/test-durations.ndjson
{"test": "tests/scripts/test_ast_branches.py::test_testgen_subcommand_wired_and_generates_passing_file", "duration": 18.1125, "ts": 1787163268.4934072, "outcome": "passed"}

$ head -3 reports/test-phase-durations.ndjson
{"kind": "run", "ts": 1787163195.3323667, "lane": "gw25"}
{"test": "tests/scripts/test_gate_cli.py::TestFactsSubcommand::test_stats_prints_the_table_and_exits_zero", "phase": "setup", "duration": 7.1453, "ts": 1787163193.9370055, "outcome": "passed"}

The phase-durations file is real, working output from an actual run — but FR-002/FR-003's code is uncommitted and untested as of this snapshot, so this page does not count it as BUILT even though SPEC-113's own text (also dated 2026-08-19) predates it and still says UNBUILT. Both facts are true at once: the spec is honest about what existed when it was written, and the tree has moved since.

FR-006, the standing prohibition, re-verified 2026-08-19 with fresher counts than the original research: 12 of 26 rulebases under src/orchestrata_ultimate/prolog/ declare no :- module (only explain.pl does, among the gate's own consult set); 176 \+ sites repo-wide (up from ADR-0027's 121 counted 2026-07-24). Both feed the same conclusion this page's companion piece already reached: statistical facts must never enter the shared, non-monotonic, hash-verified rulebase — SPEC-114's rule lifecycle (below) is the only sanctioned path from an observed defect to the rulebase, and what crosses is always a human-promoted deterministic rule, never a statistical fact.

Proposed (SPEC-114, all UNBUILT except where marked): LEARN, QUARANTINE, PROMOTE, EXPLAIN

Requested by the user 2026-08-19, quoted verbatim in SPEC-114's Directive:

"self learning from code review errors, anti patterns design and code antipatterns, lack of spec and traces for code, the more issues we fall into the prolog is watching and learning and adding rules but they need to be reviewed by us both objectively and then the rules that prolog adds automatically are in status pending until we promote them" — and — "why isn't this feature implemented should be in prolog so prolog can be like your sidekick."

An Opus reviewer is adjudicating right now whether the underlying idea survives at all. Nothing here is built; two existing mechanisms — SEE BELOW — are what it would extend.

A. LEARN (FR-001) — a miner over eight existing corpora (review codes, test-standard T-rules, trace_intel.py's orphans/untraced-reqs/ unspecced, wiring/spec-authority/requirement-corpus gates, forbidden- pattern and flakiness audits), shaped like the existing mine_prompt_history.py precedent (read-only, cluster, diff-vs-canon, report — "never writes a ruling"). Zero hits for any rule-inducer name in src/ or scripts/.

B. QUARANTINE + PROMOTE (FR-002/FR-003) — the safety property that makes (A) acceptable: a pending rule never changes a gate verdict. This is not a new channel — the user's own 2026-08-19 resolution puts pending as a severity VALUE alongside error/warning in the severity machinery that already governs every rule today:

% gate_severity.pl:64-80
severity(RuleId, Context, Severity) :-
    severity_exception(RuleId, Context, Severity).
severity(RuleId, Context, Severity) :-
    \+ severity_exception(RuleId, Context, _),
    default_severity(RuleId, Severity).
blocking(error).

SPEC-114 goes further than "no default_severity/2 clause": quarantine is structural, not only a severity value (REQ-114-FR2-003) — candidate clauses live in a dedicated quarantine rulebase, never in gate_rules.pl, and must pass prolog_meta_lint.py before consult at all, because the shared namespace and non-monotonic NAF (FR-006 above) bite a defective clause regardless of its severity. Promotion is a committed .pl diff reviewed by both reviewers — no automatic promotion path, no hot-reloading (the exact learning.md Phase-4 design this explicitly prohibits).

C. EXPLAIN / SIDEKICK (FR-004) — two different questions, and keeping them apart matters: "why did this gate rule fire at file:line" is BUILT and in production use (explain.pl's explain_call/explain_whynot, wired via GateEngine.explain, reachable as python scripts/gate.py explain no_relative_import src/pkg/a.py 7 — 9 test functions in tests/prolog/test_explain.py). "Why isn't feature X implemented anywhere" is a different, UNBUILT question over requirement/@traces facts that trace_intel.py's untraced-reqs/orphans already compute deterministically but only as a CLI report, not as a Prolog goal explain_whynot can reason over. A closely related predicate — covered/1/uncovered/1 over policy_mechanism/3 facts (REQ-GATE-011) — is landing in the working tree right now, uncommitted, alongside this page's writing: real, working code, but not yet a test, not yet a commit, so it is IN PROGRESS here too, not BUILT.

Sources

• specs/113-measurement-retention/spec.md (SPEC-113, Draft) — authoritative for the instrument.
• specs/114-governance-rule-learning/spec.md (SPEC-114, Draft) — authoritative for LEARN/QUARANTINE/PROMOTE/EXPLAIN.
• plans/prolog-learning-subsystem-research-2026-08-12.md — the measurement record both specs cite.
• docs/architecture/learning.md — the superseded vision doc, now carrying a status header naming both specs.
• docs/architecture/measurement-retention.md — this page's source repo document, with full code excerpts and five validated diagrams.
• The Prolog layer does not learn — the companion audit this page agrees with throughout.