Orchestrata Ultimate / Specs / 114-governance-rule-learning

114-governance-rule-learning: Governance Rule Learning

Three capabilities on machinery that already exists:

At a glance

Item Value
Spec directory specs/114-governance-rule-learning/
Status line in the spec Draft
Requirement IDs defined 17
With a @traces citation in src/ 1 (5.9%)
With a @traces citation in tests/ 5 (29.4%)
Tracked files 2
RTM page 114-governance-rule-learning

Reading this spec

This spec defines how governance rules can be learned from the project's own history without letting a statistical system enforce anything. It describes three capabilities on machinery that already exists. LEARN is a mining pass over the repository's open-vocabulary defect corpora (rulings, adjudications, postmortems and ruling-shaped prompt history) that induces candidate rules as reviewable Prolog clauses with provenance; closed-vocabulary tool output is measured as not inducible and stays a fact source. QUARANTINE and PROMOTE means every induced rule starts as pending, so it reports but never fails a build, and becomes enforcing only through explicit human promotion reviewed by both reviewers. EXPLAIN turns the question "why isn't this feature implemented or this policy mechanized" into a Prolog why-not query using the existing meta-interpreter.

The spec calls itself the disciplined survivor of two refuted designs. An earlier architecture note proposed a CLIPS and Prolog dual-engine rule synthesizer with hot reloading, but the spec records that CLIPS is not a project dependency and is imported nowhere in the repository. Section 4 explains why this design does not reopen those proposals.

The four functional requirement groups map to the 17 requirement IDs (REQ-114-NNN). FR1 (LEARN): the miner draws candidates only from open-vocabulary human and incident corpora, and rejections are recorded with reasons so a rejected candidate is never re-proposed. FR2 (QUARANTINE): pending is a severity value beside error and warning in one severity channel, isolation is by predicate head and module rather than by file, and gate hardening prerequisites include an expected clause-count invariant that returns INVALID on a mismatch. FR3 (PROMOTE): promotion is an explicit human act recorded in the clause header with date, reviewers and provenance, with no automatic path; promoted rules are ordinary rules using ground extractor facts and complete answer sets with no confidence thresholds; machine-authored rule files fall under the content-hash freshness discipline. FR4 (EXPLAIN): answers carry derivations with bound witnesses, and a positive, decomposable requirement_implemented/2 rule is a prerequisite.

The only module that cites the spec is the gate engine. The requirements traceability matrix lists each ID. The spec's own status line reads Draft and it tracks 2 files. Measured coverage is 1 of 17 requirements cited by at least one @traces in src/ (5.9%), spread over 1 module; the module citing the most of them is orchestrata_ultimate/gates/engine (1 requirement). In tests/, 5 requirements are cited by a test-side @traces. 16 requirements have no citing symbol, the largest groups being REQ-114-FR3 (5), REQ-114-FR2 (4) and REQ-114-FR4 (4); the behaviour may still exist in code that was never annotated.

Requirements

Checkbox is the state recorded in the requirements checklist; Traced by is the count of src/ symbols whose @traces names the ID.

ID Done Requirement Traced by
REQ-114-FR1-001 no Miner sources candidates from open-vocabulary human/incident corpora only (rulings canon, ADR-0014 adjudications, postmortems, ruling-shaped prompt history); NEVER from closed-vocabulary tool output (code-review's 39 fixed codes, T1–T20, the gate scripts — measured 0-residual on two disjoint subsets 2026-08-19), which stays gate FACT input per REQ-GATE-010 none
REQ-114-FR1-002 no Miner is read-only; output is candidate .pl clauses + per-candidate provenance (source artifact id, incident references, occurrence count, exemplar file:line list where applicable); provenance-less candidates invalid none
REQ-114-FR1-003 no Rejections recorded with reasons; rejected candidates never re-proposed (tombstone dedupe); review outcomes feed the next mining pass none
REQ-114-FR2-001 no Induced rules enter status pending; a pending rule never changes a gate verdict or exit code (gate run exits identically with the rule present or absent, finding reported as pending) none
REQ-114-FR2-002 no One severity channel: pending is a severity value beside error/warning; blocking/1 stays blocking(error); error / warning-forever / pending-awaiting-review remain three distinguishable states none
REQ-114-FR2-003 no Quarantine rulebase uses a DISTINCT head (candidate_violation/4, never violation/4) AND declares its own :- module with an explicit export list (file separation alone was demonstrated 2026-08-19 to REPLACE the enforcing violation/4 clauses 5→1 on consult); must pass prolog_meta_lint.py before being consulted at all none
REQ-114-FR2-004 no Gate hardening PREREQUISITES (live defenses independent of learning): (a) GateEngine._evaluate asserts an expected violation/4 clause-count invariant and returns INVALID on mismatch (replaces the rules_loaded == 0-only guard under which a 5→1 clobber passes as "1 rule clause(s)"); (b) prolog_meta_lint.py gains a cross-rulebase predicate-redefinition check 5
REQ-114-FR2-005 no Pending-rule store is a version-controlled file or governed-ledger KG artifact, NEVER asserted Prolog state (ADR-0005); factory/promotion/store.py is the reusable promotion-ledger substrate none
REQ-114-FR3-001 no Promotion is an explicit human act reviewed by BOTH reviewers, recorded in the clause header (date, reviewers, provenance); no automatic promotion path (ADR-0014 discipline) none
REQ-114-FR3-002 no Promotion mechanism is a committed .pl diff into gate_rules.pl (rewritten onto the enforcing violation/4 head) + a default_severity/2 clause; hot-reloading / runtime rule deployment prohibited none
REQ-114-FR3-003 no Promoted rules are ordinary rules: ground extractor facts only (ADR-0025), complete answer sets / no confidence thresholds (ADR-0026), explainable via explain.pl none
REQ-114-FR3-004 no Demotion/retirement path exists with the same review provenance, feeding the tombstone record none
REQ-114-FR3-005 no Machine-authored rule files come under the content-hash freshness discipline (extend .py-only extractor hashing to consulted .pl files, or verify a recorded hash at consult time); until then a KNOWN GAP — quarantine files not consulted alongside enforcing rules outside CI-observed runs none
REQ-114-FR4-001 yes "Why isn't requirement/feature X implemented" answerable as a why-not query over requirement/trace/coverage facts emitted from trace_intel's deterministic computations (never recomputed inside Prolog) — trace_intel why-not <REQ-ID> → tool_why_not_req/2; conjuncts read the same traces/2, spec_requirement/3 and entry_reachable/1 the Layer-3 finders walk none
REQ-114-FR4-002 yes Answers carry derivations with bound witnesses (ADR-0031); model shape = gate uncovered deriving unmechanized long-line policy from tool_rule_status(ruff,'E501',ignored) — verified on the live corpus: REQ-000-004 ⇒ failed_subgoal: req_traced_by_code('REQ-000-004',A) under head: user:requirement_implemented('000','REQ-000-004'), and the rendering is STABLE across repeated identical queries (head_to_atom/2 numbervars a copy_term; bare ~q leaked SWI's internal variable counter, which varied run to run) none
REQ-114-FR4-003 yes Sidekick queries are read-only and freshness-honest: stale fact base ⇒ INVALID-with-names, never a confident answer — sidekick_precondition_report/1 in traceability_db.pl gates tool_why_not_req/2; three-valued (fresh answers, stale_fact_base and unknown_freshness refuse), and trace_intel why-not exits 2 = invalid, matching gate run's own 0 pass / 1 fail / 2 invalid. Built on kb_module_changed/2 (codebase_db.pl) rather than freshness.pl::gate_precondition/1 — see spec.md §FR-004 for why the two live in different engines. Verified live 2026-08-22: editing two source files made the real corpus refuse to answer, naming scripts.trace_intel and tests.prolog.test_requirement_whynot with reason: size, was 12859, now 18160. none
REQ-114-FR4-004 yes PREREQUISITE — a positive, decomposable rule body exists: requirement_implemented/2 in traceability_db.pl (see the AMENDED note in spec.md §FR-004 for why it is sited there and not in spec_completeness.pl, whose implemented/2 remains fact-only and clause-less — pinned by a regression test); acceptance MET — a requirement lacking @traces gets that conjunct named with a witness, proved for all three conjuncts in tests/prolog/test_requirement_whynot.py none

Code that traces this spec

Module Symbols Requirements
orchestrata_ultimate/gates/engine 5 1

Checklists

checklists/requirements.md

The requirement list above comes from this file; 4 of 17 checkbox items are ticked.

Full spec text

Verbatim from specs/114-governance-rule-learning/spec.md; headings are pushed down two levels.

SPEC-114: Governance Rule Learning

Metadata

1. Overview

What

Three capabilities on machinery that already exists:

  1. LEARN — a mining pass over the repo's OPEN-vocabulary defect corpora (rulings, adjudications, postmortems, ruling-shaped prompt history) induces CANDIDATE governance rules as reviewable .pl clauses with provenance. Closed-vocabulary tool output is measurably not inducible (§2) and stays a fact source.
  2. QUARANTINE + PROMOTE — every induced rule enters status pending: it reports, it never fails a build. It becomes enforcing only on explicit human promotion, reviewed objectively by both reviewers. Promotion rides the existing severity machinery (gate_severity.pl), per the 2026-08-19 resolution of plans/prolog-gates-implementation-plan.md open question 1: advisory-vs-blocking is a severity VALUE in ONE channel, never a second channel. ADR-0014 ("Adjudications are promoted into the Project KB", Accepted 2026-07-24) is this promotion model already in ADR form; this spec extends it to rules.
  3. EXPLAIN — "why isn't this feature implemented / this policy mechanized" is a Prolog why-not query. The meta-interpreter exists (explain.pl, ADR-0031, explain_whynot/2,3); the work is emitting requirement/trace/coverage facts and giving implemented/2 a positive, decomposable definition (§FR-004).
Why

This is the disciplined survivor of two refuted designs. docs/architecture/learning.md proposed a CLIPS+Prolog dual-engine rule synthesizer with hot-reloading — CLIPS has zero entries in pyproject.toml and uv.lock, no clipspy in the venv, and zero imports repo-wide (/usr/bin/clips is a system package invoked by exactly one standalone benchmark script nothing depends on), and the doc's GovernanceRuleLearner has exactly one occurrence in the entire repository: the doc itself (verified 2026-08-19). The statistical-learning premise was separately measured dead by plans/prolog-learning-subsystem-research-2026-08-12.md. What survives both refutations is exactly this spec's shape: deterministic findings in → reviewable rule proposals out → human promotion gate → ordinary rules. Nothing statistical crosses into the rulebase (SPEC-113 FR-006), and nothing enforces without a human decision.

How

The miner follows the established read-only shape of scripts/mine_prompt_history.py (mine → cluster → diff against the ratified canon → report what is new; never writes a ruling). Candidates land in a quarantine rulebase that is isolated by PREDICATE HEAD and MODULE, not merely by file (§FR-002 — file separation alone was demonstrated to destroy enforcing rules, not protect them); promotion is a reviewed .pl diff into the enforcing rulebase — rules-as-reviewable-data, the property the research record names as one of the three reasons the gate layer earns its Prolog. Rejections are recorded with reasons so the miner never re-proposes them — the same discipline as the research record's "explicitly dropped, with numbers attached so they are not re-proposed".

2. The corpus — what is and is not inducible (measured 2026-08-19)

Closed-vocabulary tool output is NOT an induction source. scripts/code-review.py defines 39 fixed literal codes, and every CodeIssue.code is one of those, a ruff id, or the MYPY_ERROR catch-all — there is no open or uncategorized bucket. Measured on two DISJOINT subsets by two independent agents (2026-08-19 adversarial review): core+gates+governance → 740 findings across 11 codes (top-3 = 75.9%, zero ERROR); core+agents → 1,078 findings across 10 codes (top-3 = 72.0%, zero ERROR). Residual findings outside already-encoded checks: 0 of 740 and 0 of 1,078. Inducing rules from that stream can only rediscover the rules that produced it. The same holds for scripts/test-review.py's fixed T1–T20 and the deterministic gate scripts (check_wiring.py, check_spec_authority.py, check_requirement_corpus.py, audit_test_purity.py, gate_parallel_safety.py) — each IS already a mechanized rule. All of these remain what REQ-GATE-010 says they are — FACT sources (catalogue, configured status, run outcome, findings) — never learning input.

The genuinely open corpus is the human/incident channel — defects caught by a person or an incident that are NOT yet mechanized. That is what the directive's "code review errors and anti-patterns" means in the evidence:

Source What it contributes Evidence
Rulings canon (~/workspace/knowledge-wiki/rulings/RUL-*.md) ratified corrections awaiting mechanisms — RUL-014 ("every ruling must resolve to a mechanism") makes the miner's job precise: propose the missing mechanism rulings_lint.py resolution logic
Adjudications (ADR-0014) disputes resolved by the human, promoted into the project KB — the promotion pipeline this spec generalizes to rules docs/adr/0014-adjudications-promoted-into-project-kb.md
Postmortems / incident records (plans/*postmortem*.md) measured failure classes with named mechanisms e.g. plans/suite-runtime-failure-postmortem-2026-08-12.md
Ruling-shaped prompt history the PRECEDENT miner: read-only, cluster, diff-vs-canon, report scripts/mine_prompt_history.py docstring: "never writes a ruling"

Traceability gaps are large but already deterministic — they feed FR-004, not FR-001. Measured 2026-08-19: orphans 4,093 / untraced_reqs 2,433 / unspecced 321, against 5,531 spec requirements and 7,743 traces. That is thousands of positive labels where the refuted duration-prediction problem had zero — the economics genuinely invert relative to the research record's "no positive labels" finding. But scripts/trace_intel.py already answers those questions deterministically, so the honest form is fact emission plus why-not (REQ-114-FR4-001), not a learner rediscovering what a query already knows.

3. Functional Requirements

FR-001: LEARN — candidate induction from open-vocabulary corpora
FR-002: QUARANTINE — pending is non-enforcing, and isolation is by head and module, not by file

This is the load-bearing safety property of the whole design, and the naive version of it was demonstrated broken during adversarial review (2026-08-19, real swipl): consulting a second module-less .pl file that defines violation/4 REPLACES the enforcing clauses instead of adding to them — SWI emits only Warning: Redefined static procedure violation/4 and the measured clause count went 5 → 1, leaving ONE pending rule as the entire rulebase. Declaring :- dynamic violation/4. in a first-loaded file does not prevent it (tested as a variant). Production shape confirms the exposure: violation/4 is static with 5 clauses in gate_rules.pl, which declares no :- module; gates/engine.py::_loaded (lines 352–388) consults everything into one process-global janus engine; code_facts.pl's :- dynamic declarations cover fact predicates only, not violation/4. And the gate would have reported a clean PASS while it happened: _evaluate (engine.py lines 248–303) guards only rules_loaded == 0, so 1 surviving clause passes as "1 rule clause(s)". Neither existing safeguard catches it: prolog_meta_lint.py has exactly four checks (check_no_cut :683, check_no_dynamic_goal :762, check_no_rule_side_effects :791, check_stratification :961) — none about redefinition — and tests/prolog/test_module_exports.py covers only files that declare :- module.

FR-003: PROMOTE — dual human review, rules-as-reviewable-data
FR-004: EXPLAIN — why-not as the standing sidekick

The machinery exists: explain.pl implements proof trees (explain_call/2) and why-not (explain_whynot/2,3 — every candidate clause's first failing precondition, with bound witnesses via bind_witness/2), exercised by tests/prolog/test_explain.py and wired as gate explain / gate why-not since 2026-08-08 (plan changelog). Coverage inference is specced: REQ-GATE-011's uncovered(Policy) :- policy_requirement(Policy), \+ covered(Policy) (plan line 852), with policy_mechanism/3 declared and emitted (code_facts.pl lines 237–243; FactExtractor._tooling_facts, fact_extractor.py line 869).

4. Why this does not reopen the refuted designs

The five Prolog-risk mechanisms (SPEC-113 FR-006, re-verified 2026-08-19) were stated against statistical facts entering the enforcing rulebase. Under this spec's lifecycle:

The quarantine design is not merely compatible with the research record's warnings — it is the mitigation those warnings imply, provided isolation is by predicate head and module (measured: file isolation alone silently destroys the enforcing rulebase), because it moves all learning into the proposal step where nothing the gate trusts can be corrupted.

5. Non-Goals

6. Traceability

Implementation code SHALL carry @traces to REQ-114-*. Rule-lifecycle artifacts (candidate files, promotion diffs, tombstones, the FR2-005 ledger) are governed by FR-002/FR-003 and reviewable in git history. The superseded vision doc docs/architecture/learning.md carries a status header naming this spec, SPEC-113, and the research record.