Knowledge Wiki Orchestrata Ultimate

Orchestrata Ultimate › Plans › prolog-ast-reasoning-ladder

Prolog + AST deterministic-reasoning ladder

Source: plans/prolog-ast-reasoning-ladder.md at 8a2cb42ad · last changed 2026-08-13 · first committed 2026-08-02 · 169 lines.

Prose pending.

Plan text

Prolog + AST deterministic-reasoning ladder

Origin: user directive 2026-08-02 — "what more can prolog do, tasks that take a lot of time and tokens but are deterministic and we can build right now?" Then, on tests/coverage: consolidate the suite so branches run once/as-necessary, not redundantly.

Principle

Work that is expensive but deterministic — whole-program reachability, impact, dead code, cycles, requirement↔code↔test tracing, coverage consolidation — should be computed once by a logic engine over code facts and queried forever, instead of approximated per-question by an LLM reading files (slow, token-heavy, never exhaustive). The AST supplies ground-truth facts; Prolog (SWI 10.0.2 + janus, tabling, CLP(FD), s(CASP)) supplies exact, complete closures.

Substrate (already on disk — extend, don't rebuild)

Metric doctrine (governs T1b/T1c — do not violate)

  1. Coverage-hit ≠ verification. A branch executed N× but asserted 0× is false confidence, not coverage. The real metric is fault detection: mutate the branch — does a test fail?
  2. Redundant execution ≠ redundant verification. A unit test proves logic; a high-layer test proves the code is wired in and reached with real inputs. Built-but-unwired is this project's dominant defect class — the overlap is the safety net. Tier the suite, never delete for coverage overlap.
  3. AST is the denominator. branches/call-graph define the complete set of paths that should be covered, so gap reports are honest ("what's missing"), not just overlap reports.

The ladder

T0 — wire the structural KB for the orchestrator (building)

scripts/code_intel.py CLI + freshness over CodebaseTools: impact/cycles/unused/ callers/callees/dep-path/module-api/hotspots, auto-reindex on staleness, qname prefix normalization. Facts already exist. Immediate token win on every structural question.

T1 — traceability break-finders (no test evidence) (building, scope-corrected)

src/orchestrata_ultimate/tools/traceability_db.pl (consults codebase_db) + scripts/trace_intel.py. Facts: traces/2, spec_requirement/3, entrypoint/2. Rules (tabled forward reaches/2 over calls_resolved):

T1b — false_confidence via three witnesses (next; needs coverage regen)

test_reaches(Test, SrcQName) evidenced by ANY of:

T1c — suite consolidation / minimization (user-approved; rides T1b matrix)

Coverage matrix = test × branch × assertion. Outputs:

  1. Minimal fast tier — greedy/CLP(FD) set-cover over branches (reuse schedule_tests.py). Runs on each change alongside every directly-asserting test.
  2. Pyramid inversions — branches covered ONLY by high-layer tests → add a unit test.
  3. Redundancy vs false confidence — executed-but-unasserted (T1b) split from asserted-elsewhere. Drop-safety proven by mutation (removing a test must not lower mutation-kill count). Full suite stays nightly; consolidation only re-tiers the fast loop.

T2 — contract / security proofs (D1–D7)

Reachability constraints: violates_audit(Path) = gateway-handler→return path missing audit.write; ungoverned_decision(Q) = decision reachable without the Prolog gate. Proves the audit-loss / circuit-breaker defects or yields counterexample paths. Needs entrypoint + must-pass-through facts (builds on T1 entrypoints).

T3 — clone / consolidation (module-count reduction)

body_clone(Q1,Q2) on body_hash (exact dup functions); structural_twin(C1,C2) (identical method-signature sets). Feeds the duplicate audit/postman/temporal/graph consolidation + DP2.

Real findings already surfaced (live, this session)

Capability integration (post-T1-verify) — make the tools the DEFAULT path

User directive 2026-08-02: after T1 lands+verifies, update skills / tool-calls / workflow / instructions to exploit the progress. Goal: deterministic queries replace file-reading approximations, for the orchestrator AND every subagent, with drift caught at commit. Ordered by leverage:

  1. .claude/agents/{auditor,builder,reviewer,refactorer,wirer}.md — deterministic-first clause: query code_intel (impact/callers/cycles/unused/dep-path) + trace_intel (orphans/untraced/narrative) + source <qname> BEFORE reading files for structure/ impact/traceability. Highest leverage — propagates to every swarm fan-out.
  2. .claude/skills/ast-toolkit/SKILL.md — add code_intel + trace_intel + a decision table (serialize_codebase_ast→symbols/refs/source/refactor; code_intel→graph closures; trace_intel→req↔code↔test) + the KB freshness/reindex model.
  3. CLAUDE.md Key Rules + gen_level0.py→LEVEL0.md — deterministic-first rule + list the new CLIs in the L1 pointers (regenerate LEVEL0 via the generator).
  4. .claude/skills/test-loop/SKILL.md — fold in false_confidence (T1b) + unused + the T1c minimal-covering-set.
  5. .pre-commit-config.yaml — advisory gates (warn-first, like doc-lint): trace_intel orphan_feature/untraced_req drift + a code_intel cycles ratchet (baseline 8).
  6. orchestrata project memory — "deterministic-first tools exist; default to them". Decisions deferred to integration time (need T1's real counts): warn-only vs blocking for #5; whether trace_intel folds into code_intel or stays a separate CLI.

Ownership / sequencing

Opus designs + evaluates + owns the spec-consolidation ANALYSIS (interpreting the finder outputs into a reorganized, use-case-narrated spec set — the core issue: features with no usage scenario). Builders implement each tier to a settled brief. Order: T0 ∥ T1 → verify/commit → T1b (with coverage regen) → T1c → T2/T3.

T1-P2 — runtime call graph closes the dynamic-dispatch gap + derived RTM (DONE 2026-08-02)

T1 flagged that static reaches/2 over calls_resolved misses registry→handler / DI / CXP dispatch, inflating the dead-code surface. That gap is now closed with a RUNTIME call graph, and the payoff extended into per-spec attribution.

Open follow-on: expand entrypoint/2 to the real entry surface (volk handlers, registered tools, CXP/gRPC handlers) so the static finders' orphan/specless counts match the attribution engine's 74% reachability — the diagnosis (agents.volks 194 orphans, tools 100+, protocols 97 correctly-abstract) is in this session's analysis.