scripts.trace_intel
Reference › scripts › trace_intel
scripts/trace_intel.py is the T1 traceability and use-case-narrative engine: a semantic Prolog layer over the structural code-intelligence knowledge base. It answers which code implements which requirement, which code is unreachable from any entrypoint, and which requirements nothing traces to. A human or agent runs it; derive_rtm imports it and tests/scripts/test_derive_rtm.py covers it. It uses orchestrata_ultimate.tools.ast_pipeline and orchestrata_ultimate.tools.codebase_tools. Its sibling for the time dimension is history_intel, and the graph-side counterpart is sync_traces_to_graph.
The extract subcommand regenerates traceability_facts.pl from plans/ast-ultimate.json and specs/. _traces_facts builds traces(QName, ReqId) facts from every @traces decorator (_iter_symbols, _extract_req_ids), and entrypoint facts are emitted for classes, CLI commands and other kinds (_entrypoint_kind, _emit_entrypoint, with class inheritance followed transitively). Query subcommands run against that base: report (all five finders with counts; --merged compares static with merged static-plus-runtime reachability), narrative <qname> (modules transitively reachable from one entrypoint), orphans (traced code unreachable from any entrypoint), untraced-reqs (requirements with zero @traces coverage), other limited listings (--limit, default 50), and why-not <req_id>, which names the first missing conjunct. Queries call ensure_kb first (honouring --no-refresh). The exit code has three values, matching the gate layer: 2 when the outcome is invalid (the fact base could not answer for the bytes on disk), 1 for an error and 0 otherwise.
At a glance
| Source | scripts/trace_intel.py |
| Lines of code | 1209 |
| Classes | 1 |
| Functions | 38 |
| Imports (internal) | orchestrata_ultimate.tools.ast_pipeline, orchestrata_ultimate.tools.codebase_tools |
| Imported by | scripts.derive_rtm |
| Tests that import it | tests/scripts/test_derive_rtm.py |
Class diagram
Dependency diagram
API
Constants
| Name | Type | Value |
|---|---|---|
_MERGED_ROWS |
tuple[tuple[str, str, str], ...] |
(('orphans_static', 'orphans_merged', 'orphans'), ('unspecced_static', 'unspecc… |
_QUERY_DISPATCH |
dict[str, Callable[[argparse.Namespace], dict[str, Any]]] |
{'report': _cmd_report, 'narrative': _cmd_narrative, 'orphans': _cmd_orphans, '… |
Classes
_WarmPrologWorker
Line 622
One persistent swipl process holding the STANDARD repo KB.
| Method | Signature | Summary |
|---|---|---|
__init__ |
__init__() -> None |
|
_kb_key staticmethod |
_kb_key(kb_files: list[Path]) -> tuple[tuple[str, int, int], ...] |
|
_start |
_start(kb_files: list[Path], key: tuple) -> None |
|
_stop_locked |
_stop_locked() -> None |
|
shutdown |
shutdown() -> None |
|
query |
query(goal: str, kb_files: list[Path]) -> dict[str, Any] | None |
Run goal on the warm worker; None => use the one-shot path. |
_query_locked |
_query_locked(goal: str) -> dict[str, Any] | None |
Functions
| Function | Signature | Summary |
|---|---|---|
_strip_prefix |
_strip_prefix(qname: str) -> str |
Strip a leading orchestrata_ultimate. if present. |
_pl_atom |
_pl_atom(value: str) -> str |
Quote a Python string as a Prolog atom (single-quote doubling). |
_extract_req_ids |
_extract_req_ids(decorator_text: str) -> list[str] |
Pull every REQ-... id out of one traces(..., reqs=[...]) decorator source string (as captured verbatim by the AST serializer). |
_iter_symbols |
_iter_symbols(modules: list[dict[str, Any]]) -> Iterator[tuple[str, list[Any]]] |
Yield (qualified_name, decorators) for every class, method, and top-level function symbol across all modules. |
_traces_facts |
_traces_facts(modules: list[dict[str, Any]]) -> list[str] |
Build traces(QName, ReqId). facts from every @traces(...) decorator's reqs=[...] list. |
_entrypoint_kind |
_entrypoint_kind(file_path: str) -> str | None |
Classify a module's file path as an entrypoint-hosting file, or None. cli_main/server_main only (see module docstring). |
_build_class_index |
_build_class_index(modules: list[dict[str, Any]]) -> dict[str, list[dict[str, Any]]] |
Simple-name -> [class dict, ...] index across all modules. |
_inherits_transitively |
_inherits_transitively(target_name: str, class_index: dict[str, list[dict[str, Any]]], _visited: set[str] | None = None) -> bool |
True if cls inherits from target_name, directly or through any number of intermediate bases (e.g. a mixin chain), resolved via the bare-name class_index. |
_is_cli_command_decorator |
_is_cli_command_decorator(text: str) -> bool |
True for a click @x.command(...)/@x.group(...) decorator source string (covers @click.command(), @cli.command(), @agents.group(), and any decorator text merely containing… |
_emit_entrypoint |
_emit_entrypoint(facts: list[str], seen: set[tuple[str, str]], qname_raw: str, kind: str) -> None |
Append entrypoint(QName, Kind). to facts unless qname_raw is empty or (QName, Kind) was already emitted. |
_entrypoint_facts |
_entrypoint_facts(modules: list[dict[str, Any]]) -> list[str] |
Build every entrypoint(QName, Kind). fact -- the production entry surface the traceability finders (orphan_feature/unspecced_code/ specless_usecase) treat as reachable roots. |
_load_parse_spec_requirements |
_load_parse_spec_requirements() -> Any |
Load scripts/parse_spec_requirements.py by path (REUSED, not reimplemented — same pattern as scripts/ast_report.py's _load_tool: scripts/ isn't an importable package member for direct import). |
_spec_requirement_facts |
_spec_requirement_facts(specs_root: Path) -> list[str] |
Build spec_requirement(SpecId, ReqId, Desc). facts by running parse_spec_requirements.parse_requirements over every requirements.md checklist under specs_root (both the direct… |
extract_facts |
extract_facts(ast_json_path: Path = AST_ULTIMATE_JSON, specs_root: Path = SPECS_ROOT, output_path: Path = TRACEABILITY_FACTS) -> tuple[int, int, int] |
Regenerate output_path from ast_json_path + specs_root. |
cmd_extract |
cmd_extract(_args: argparse.Namespace) -> int |
|
_kb_files |
_kb_files() -> list[Path] |
|
_qlf_for |
_qlf_for(pl_path: Path) -> Path |
Return a fresh precompiled .qlf twin for large .pl fact files, compiling it when missing/stale; fall back to pl_path. |
_get_worker |
_get_worker() -> _WarmPrologWorker |
|
_run_prolog |
_run_prolog(goal: str, kb_files: list[Path]) -> dict[str, Any] |
Run goal against kb_files: the standard repo KB goes to the warm persistent worker (falling back to one-shot on any anomaly); every non-standard KB list (fixtures, empty) always uses the… |
_run_prolog_oneshot |
_run_prolog_oneshot(goal: str, kb_files: list[Path]) -> dict[str, Any] |
Consult every file in kb_files (in order) then run goal in one standalone swipl subprocess, returning its R binding (a json{} dict written via json_write_dict) as a Python… |
_needs_refresh |
_needs_refresh(artifact: Path, **sources: Path) -> bool |
True when artifact is missing or older than ANY of sources. |
_traceability_sources |
_traceability_sources() -> list[Path] |
Every input extract_facts reads, for the staleness gate. |
_active_snapshot |
_active_snapshot() -> Path |
The snapshot the structural KB is reindexed from. |
_newest_source_mtime |
_newest_source_mtime() -> float |
Newest mtime across the trees the snapshot is serialized from. |
_refresh_snapshot |
_refresh_snapshot() -> None |
Bring the LIVE snapshot up to date with the working tree. |
ensure_kb |
ensure_kb(no_refresh: bool) -> None |
Auto-run extract + reindex the structural KB when either artifact is missing or older than any of its inputs. |
_print_result |
_print_result(result: dict[str, Any], json_mode: bool) -> None |
|
_print_merged_result |
_print_merged_result(result: dict[str, Any]) -> None |
Text-mode renderer for report --merged: a static-vs-merged comparison (each row's static count, merged count, and delta) instead of _print_result's flat key/value dump -- the default… |
_cmd_report |
_cmd_report(args: argparse.Namespace) -> dict[str, Any] |
|
_cmd_narrative |
_cmd_narrative(args: argparse.Namespace) -> dict[str, Any] |
|
_cmd_orphans |
_cmd_orphans(args: argparse.Namespace) -> dict[str, Any] |
|
_cmd_untraced_reqs |
_cmd_untraced_reqs(args: argparse.Namespace) -> dict[str, Any] |
|
_cmd_unspecced |
_cmd_unspecced(args: argparse.Namespace) -> dict[str, Any] |
|
_cmd_specless_usecases |
_cmd_specless_usecases(args: argparse.Namespace) -> dict[str, Any] |
|
_cmd_why_not |
_cmd_why_not(args: argparse.Namespace) -> dict[str, Any] |
SPEC-114 FR-004 — why isn't this requirement implemented? |
_add_common_args |
_add_common_args(parser: argparse.ArgumentParser) -> None |
|
_build_parser |
_build_parser() -> argparse.ArgumentParser |
|
main |
main(argv: list[str] | None = None) -> int |
Parse argv and dispatch to the matching subcommand handler. |
Source
Permalink: https://gitlab.com/MSamy/orchestrata-ultimate-architecture/-/blob/a4cbc2bdf/scripts/trace_intel.py
#!/usr/bin/env python3
"""scripts/trace_intel.py — T1 traceability / use-case-narrative engine.
A semantic Prolog layer ON TOP OF the existing structural code-intel KB
(``src/orchestrata_ultimate/tools/codebase_db.pl`` + ``CodebaseTools``).
Answers questions the structural facts alone cannot: which entrypoint
reaches which module, which traced code is unreachable from any known
entrypoint, which spec requirements have zero implementing code, which
reachable code carries no ``@traces`` at all, and which specs never chain
to reachable code.
Facts (regenerated by the ``extract`` subcommand into
``.cache/trace_intel/traceability_facts.pl``):
- ``traces(QName, ReqId)`` — one row per ``REQ-...`` id inside every
``@traces(spec=..., story=..., reqs=[...])`` decorator found in
``plans/ast-ultimate.json`` (QName is the symbol's qualified name,
already prefix-stripped by the serializer — see ``_strip_prefix``
for the defensive fallback).
- ``spec_requirement(SpecId, ReqId, Desc)`` — via
``scripts/parse_spec_requirements.py`` (imported by path, REUSED not
reimplemented) over every ``specs/*/requirements.md`` and
``specs/*/checklists/requirements.md``.
- ``entrypoint(QName, Kind)`` — the real production entry surface, not
just the MVP CLI seed: ``cli_main`` (a ``main``/``cli_main`` module-
level function in a ``cli.py``/``cli_main.py``/``__main__.py``
module), ``server_main`` (a ``main``/``serve`` module-level function
in a ``*server.py``/``runtime_main.py`` module), ``volk_handler``
(``.process`` on a concrete ``BaseAgent`` subclass -- the volk
dispatch loop), ``tool_entry`` (``.execute`` on a concrete
``BaseTool`` subclass, or on ``ToolExecutor`` -- the tool-calling
loop), ``gateway_handler`` (``.handle_*`` on a
``gateway.handlers.*`` class -- the gateway RPC table),
``cxp_handler`` (the CXP gRPC servicer method set, or ``.__call__``
on a ``*PacketHandler`` class), and ``cli_command`` (a click
``@x.command()``/``@x.group()``-decorated function/method, or an
argparse ``_cmd_*`` dispatch function in a ``*.cli`` module). See the
detector-constants block above ``_entrypoint_facts`` for the full
rationale and why test-exercised roots are deliberately excluded.
Rules: ``src/orchestrata_ultimate/tools/traceability_db.pl`` — consulted
alongside ``codebase_db.pl`` (for ``symbol/7``/``calls_resolved/3``).
Standard-KB queries run on a persistent warm swipl worker (consulted
once per process, mtime-keyed, SLG tables persisting across queries --
see ``_WarmPrologWorker``); non-standard KB lists and any worker
anomaly use the original one-swipl-subprocess-per-query path
(``_run_prolog_oneshot``). Large generated fact files load from
mtime-gated ``.qlf`` precompilations (``_qlf_for``).
The structural KB itself (``symbol/7``, ``calls_resolved/3``, ...) is
built via ``CodebaseTools(facts_file=...).reindex(plans/ast-ultimate.json)``
into a facts file OWNED by this script
(``.cache/trace_intel/codebase_facts.pl``) — never the default
``CodebaseTools`` facts file, so this tool can never race another
process's structural KB (see the module docstring of codebase_tools.py).
Usage:
python scripts/trace_intel.py extract
python scripts/trace_intel.py report [--json]
python scripts/trace_intel.py narrative <entrypoint_qname> [--json]
python scripts/trace_intel.py orphans [--limit N] [--json]
python scripts/trace_intel.py untraced-reqs [--limit N] [--json]
python scripts/trace_intel.py unspecced [--limit N] [--json]
python scripts/trace_intel.py specless-usecases [--limit N] [--json]
Every query subcommand auto-runs ``extract`` (and reindexes the
structural KB) when ``traceability_facts.pl``/``codebase_facts.pl`` is
missing or older than ``plans/ast-ultimate.json`` — pass ``--no-refresh``
to skip that check and query whatever is already on disk.
"""
from __future__ import annotations
import argparse
import atexit
import contextlib
import importlib.util
import json
import os
import re
import subprocess
import sys
import tempfile
import threading
import time
import types
from collections.abc import Callable, Iterator
from pathlib import Path
from typing import Any
from orchestrata_ultimate.tools.codebase_tools import CodebaseTools
REPO_ROOT = Path(__file__).resolve().parent.parent
AST_ULTIMATE_JSON = REPO_ROOT / "plans" / "ast-ultimate.json"
SPECS_ROOT = REPO_ROOT / "specs"
CODEBASE_DB = REPO_ROOT / "src" / "orchestrata_ultimate" / "tools" / "codebase_db.pl"
TRACEABILITY_DB = (
REPO_ROOT / "src" / "orchestrata_ultimate" / "tools" / "traceability_db.pl"
)
# SPEC-114 FR-004: the why-not meta-interpreter (ADR-0031). Consulted into
# every session rather than only the `why-not` one, because a session is one
# swipl subprocess per query -- there is no session to add it to later, and
# loading a 262-line rulebase that defines no facts costs nothing measurable
# against the 14K-fact traceability file it sits beside.
EXPLAIN_PL = REPO_ROOT / "src" / "orchestrata_ultimate" / "prolog" / "explain.pl"
_PARSE_SPEC_REQ_PATH = Path(__file__).resolve().with_name("parse_spec_requirements.py")
# Never the default CodebaseTools() facts file (concurrency note in the
# brief): a fully private scratch dir, gitignored (.cache is in .gitignore).
SCRATCH_DIR = REPO_ROOT / ".cache" / "trace_intel"
STRUCTURAL_FACTS = SCRATCH_DIR / "codebase_facts.pl"
TRACEABILITY_FACTS = SCRATCH_DIR / "traceability_facts.pl"
_ORCH_PREFIX = "orchestrata_ultimate."
_CLI_BASENAMES = {"cli.py", "cli_main.py", "__main__.py"}
_CLI_FUNC_NAMES = {"main", "cli_main"}
_SERVER_BASENAMES = {"runtime_main.py"}
_SERVER_FUNC_NAMES = {"main", "serve"}
_REQS_LIST_RE = re.compile(r"reqs\s*=\s*\[([^\]]*)\]")
_REQ_ITEM_RE = re.compile(r"""['"]([^'"]+)['"]""")
_SPEC_DIR_RE = re.compile(r"(\d{3})-")
# ---------------------------------------------------------------------------
# extract — traceability_facts.pl generation
# ---------------------------------------------------------------------------
def _strip_prefix(qname: str) -> str:
"""Strip a leading ``orchestrata_ultimate.`` if present.
Defensive fallback only — every ``qualified_name`` observed in
``plans/ast-ultimate.json`` is already prefix-stripped by the
serializer (module ``path`` is relative to ``root_path``), matching
the KB's own convention (codebase_db.pl's ``symbol/7`` stores exactly
this form). Kept so extraction stays correct even if a future
serializer run changes that convention.
"""
return qname[len(_ORCH_PREFIX) :] if qname.startswith(_ORCH_PREFIX) else qname
def _pl_atom(value: str) -> str:
"""Quote a Python string as a Prolog atom (single-quote doubling)."""
return "'" + value.replace("'", "''") + "'"
def _extract_req_ids(decorator_text: str) -> list[str]:
"""Pull every ``REQ-...`` id out of one ``traces(..., reqs=[...])``
decorator source string (as captured verbatim by the AST serializer).
"""
match = _REQS_LIST_RE.search(decorator_text)
if not match:
return []
return _REQ_ITEM_RE.findall(match.group(1))
def _iter_symbols(modules: list[dict[str, Any]]) -> Iterator[tuple[str, list[Any]]]:
"""Yield (qualified_name, decorators) for every class, method, and
top-level function symbol across all modules."""
for mod in modules:
for cls in mod.get("classes") or []:
yield cls.get("qualified_name", ""), cls.get("decorators") or []
for meth in cls.get("methods") or []:
yield meth.get("qualified_name", ""), meth.get("decorators") or []
for fn in mod.get("functions") or []:
yield fn.get("qualified_name", ""), fn.get("decorators") or []
def _traces_facts(modules: list[dict[str, Any]]) -> list[str]:
"""Build ``traces(QName, ReqId).`` facts from every ``@traces(...)``
decorator's ``reqs=[...]`` list."""
facts: list[str] = []
seen: set[tuple[str, str]] = set()
for qname, decorators in _iter_symbols(modules):
if not qname:
continue
q = _strip_prefix(qname)
for dec in decorators:
text = str(dec).strip()
if not text.startswith("traces("):
continue
for req_id in _extract_req_ids(text):
key = (q, req_id)
if key in seen:
continue
seen.add(key)
facts.append(f"traces({_pl_atom(q)}, {_pl_atom(req_id)}).")
return facts
def _entrypoint_kind(file_path: str) -> str | None:
"""Classify a module's file path as an entrypoint-hosting file, or
None. cli_main/server_main only (see module docstring).
``server_main`` was called ``grpc_server`` until 2026-08-07. The label was
always about SHAPE -- a ``main``/``serve`` module-level function in a
server module -- never about gRPC, and keeping the old name after the
grpc.aio transport was retired implied a coupling that never existed."""
basename = Path(file_path).name
if basename in _CLI_BASENAMES:
return "cli_main"
if basename in _SERVER_BASENAMES or basename.endswith("server.py"):
return "server_main"
return None
# ---------------------------------------------------------------------------
# entrypoint/2 detector constants -- PRODUCTION entries only.
#
# Every detector below answers one question: "can this symbol be reached
# by something a DEPLOYED orchestrata-ultimate process actually calls" --
# kernel boot, the gateway RPC table, a CLI parser, the CXP gRPC surface,
# the volk dispatch loop, the tool-calling loop. Deliberately EXCLUDED:
# anything only a test harness reaches (pytest fixtures/conftest wiring,
# `tests/generated_*` stubs, scenario runners under tests/). Seeding from
# test-exercised roots would make orphan_feature/unspecced_code/
# specless_usecase (traceability_db.pl) blind to code that is built but
# never actually wired into a running process -- exactly the failure mode
# this expansion exists to close (AGENTS.md Hard Rule 8: "built is not
# done -- WIRED is done"; the 2026-07-12 audit found a 63-module
# subsystem that shipped tested-but-unreachable because nothing but its
# own tests ever called it). A false-positive entrypoint here MASKS a
# real orphan, so every detector is intentionally narrow (exact
# method-name sets, module-path prefixes, base-class names) and skips
# ``is_protocol`` classes; when a genuine production entry doesn't fit
# these seams, that is itself a finding to report, never a reason to
# loosen a detector to force a number.
# ---------------------------------------------------------------------------
_VOLK_BASE_NAME = "BaseAgent"
_TOOL_BASE_NAME = "BaseTool"
_GATEWAY_HANDLERS_MODULE_SUBSTR = "gateway.handlers"
_GATEWAY_HANDLE_METHOD_RE = re.compile(r"^handle_[a-z]")
_CXP_SERVICER_METHOD_NAMES = frozenset(
{
"send_packet",
"send_packet_stream",
"health_check",
"get_status",
"receive_packet",
"subscribe",
"acknowledge",
}
)
_CLICK_DECORATOR_RE = re.compile(r"^\w+\.(command|group)\(")
_CLI_CMD_FUNC_RE = re.compile(r"^_cmd_[a-z]")
def _build_class_index(
modules: list[dict[str, Any]],
) -> dict[str, list[dict[str, Any]]]:
"""Simple-name -> [class dict, ...] index across all modules.
``bases`` in ``plans/ast-ultimate.json`` are bare names (no module
qualification), so this bare-name index is the only resolution the
data supports -- see ``_inherits_transitively``.
"""
index: dict[str, list[dict[str, Any]]] = {}
for mod in modules:
for cls in mod.get("classes") or []:
name = cls.get("name")
if name:
index.setdefault(name, []).append(cls)
return index
def _inherits_transitively(
cls: dict[str, Any],
target_name: str,
class_index: dict[str, list[dict[str, Any]]],
_visited: set[str] | None = None,
) -> bool:
"""True if ``cls`` inherits from ``target_name``, directly or through
any number of intermediate bases (e.g. a mixin chain), resolved via
the bare-name ``class_index``.
Bare-name resolution is inherently ambiguous when two classes share a
name (see ``_build_class_index``) -- that ambiguity is accepted as
the cost of the AST serializer not qualifying ``bases``, not papered
over by guessing; it can only ever affect entrypoint/2 membership,
never the traces/2 or spec_requirement/3 facts.
"""
visited = _visited if _visited is not None else set()
for base_name in cls.get("bases") or []:
if base_name == target_name:
return True
if base_name in visited:
continue
visited.add(base_name)
for base_cls in class_index.get(base_name, ()):
if _inherits_transitively(base_cls, target_name, class_index, visited):
return True
return False
def _is_cli_command_decorator(text: str) -> bool:
"""True for a click ``@x.command(...)``/``@x.group(...)`` decorator
source string (covers ``@click.command()``, ``@cli.command()``,
``@agents.group()``, and any decorator text merely containing
``click.command`` as a defensive fallback)."""
stripped = text.strip()
return bool(_CLICK_DECORATOR_RE.match(stripped)) or "click.command" in stripped
def _emit_entrypoint(
facts: list[str], seen: set[tuple[str, str]], qname_raw: str, kind: str
) -> None:
"""Append ``entrypoint(QName, Kind).`` to ``facts`` unless ``qname_raw``
is empty or ``(QName, Kind)`` was already emitted."""
qname = _strip_prefix(qname_raw)
if not qname:
return
key = (qname, kind)
if key in seen:
return
seen.add(key)
facts.append(f"entrypoint({_pl_atom(qname)}, {kind}).")
def _entrypoint_facts(modules: list[dict[str, Any]]) -> list[str]:
"""Build every ``entrypoint(QName, Kind).`` fact -- the production
entry surface the traceability finders (orphan_feature/unspecced_code/
specless_usecase) treat as reachable roots. See the detector-constants
block above for what "production entry" means and why test-exercised
roots are excluded.
Kinds emitted (dedup'd on ``(QName, Kind)`` across the whole
function, matching the original cli_main/server_main behaviour):
- ``cli_main`` / ``server_main`` -- unchanged MVP detector, see
``_entrypoint_kind``.
- ``volk_handler`` -- ``.process`` on a concrete (non-Protocol)
class transitively inheriting ``BaseAgent``.
- ``tool_entry`` -- ``.execute`` on a concrete (non-Protocol,
non-abstract) class transitively inheriting ``BaseTool``, or on
the ``ToolExecutor`` class (native-tool dispatch).
- ``gateway_handler`` -- a ``handle_*`` method on a
``gateway.handlers.*`` class (the SPEC-014 RPC table).
- ``cxp_handler`` -- the CXP servicer method set on a ``cxp.*``
class, or ``.__call__`` on a ``*PacketHandler`` class. The
``*grpc_server*`` module-path alternative was dropped on 2026-08-07
with the grpc transport: ``CXPServicer`` now lives in
``cxp.servicer``, which the ``cxp.`` prefix already covers.
- ``cli_command`` -- a click ``@x.command()``/``@x.group()``-
decorated function or method, or an argparse ``_cmd_*``
module-level function in a ``*.cli`` module.
"""
facts: list[str] = []
seen: set[tuple[str, str]] = set()
# -- cli_main / server_main (unchanged MVP detector) --------------------
for mod in modules:
kind = _entrypoint_kind(mod.get("file_path", ""))
if kind is None:
continue
wanted_names = _CLI_FUNC_NAMES if kind == "cli_main" else _SERVER_FUNC_NAMES
for fn in mod.get("functions") or []:
if fn.get("name") in wanted_names:
_emit_entrypoint(facts, seen, fn.get("qualified_name", ""), kind)
class_index = _build_class_index(modules)
for mod in modules:
for cls in mod.get("classes") or []:
if cls.get("is_protocol"):
continue
cls_name = cls.get("name") or ""
module_path = cls.get("module_path") or ""
is_abstract = bool(cls.get("is_abstract"))
methods = cls.get("methods") or []
# -- volk_handler -----------------------------------------------
if _inherits_transitively(cls, _VOLK_BASE_NAME, class_index):
for meth in methods:
if meth.get("name") == "process":
_emit_entrypoint(
facts, seen, meth.get("qualified_name", ""), "volk_handler"
)
# -- tool_entry -------------------------------------------------
if not is_abstract and _inherits_transitively(
cls, _TOOL_BASE_NAME, class_index
):
for meth in methods:
if meth.get("name") == "execute":
_emit_entrypoint(
facts, seen, meth.get("qualified_name", ""), "tool_entry"
)
if cls_name == "ToolExecutor":
for meth in methods:
if meth.get("name") == "execute":
_emit_entrypoint(
facts, seen, meth.get("qualified_name", ""), "tool_entry"
)
# -- gateway_handler ----------------------------------------------
if _GATEWAY_HANDLERS_MODULE_SUBSTR in module_path:
for meth in methods:
if _GATEWAY_HANDLE_METHOD_RE.match(meth.get("name", "")):
_emit_entrypoint(
facts,
seen,
meth.get("qualified_name", ""),
"gateway_handler",
)
# -- cxp_handler ----------------------------------------------------
if module_path.startswith("cxp."):
for meth in methods:
if meth.get("name") in _CXP_SERVICER_METHOD_NAMES:
_emit_entrypoint(
facts, seen, meth.get("qualified_name", ""), "cxp_handler"
)
if cls_name.endswith("PacketHandler"):
for meth in methods:
if meth.get("name") == "__call__":
_emit_entrypoint(
facts, seen, meth.get("qualified_name", ""), "cxp_handler"
)
# -- cli_command (methods) ------------------------------------------
for meth in methods:
for dec in meth.get("decorators") or []:
if _is_cli_command_decorator(str(dec)):
_emit_entrypoint(
facts, seen, meth.get("qualified_name", ""), "cli_command"
)
break
# -- cli_command (module-level functions) --------------------------------
for mod in modules:
for fn in mod.get("functions") or []:
fn_module_path = fn.get("module_path") or ""
for dec in fn.get("decorators") or []:
if _is_cli_command_decorator(str(dec)):
_emit_entrypoint(
facts, seen, fn.get("qualified_name", ""), "cli_command"
)
break
if "cli" in fn_module_path and _CLI_CMD_FUNC_RE.match(fn.get("name") or ""):
_emit_entrypoint(
facts, seen, fn.get("qualified_name", ""), "cli_command"
)
return facts
def _load_parse_spec_requirements() -> Any:
"""Load scripts/parse_spec_requirements.py by path (REUSED, not
reimplemented — same pattern as scripts/ast_report.py's ``_load_tool``:
scripts/ isn't an importable package member for direct ``import``)."""
spec = importlib.util.spec_from_file_location(
"parse_spec_requirements", _PARSE_SPEC_REQ_PATH
)
if spec is None or spec.loader is None:
raise ImportError(f"cannot load {_PARSE_SPEC_REQ_PATH}")
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
def _spec_requirement_facts(specs_root: Path) -> list[str]:
"""Build ``spec_requirement(SpecId, ReqId, Desc).`` facts by running
``parse_spec_requirements.parse_requirements`` over every
``requirements.md`` checklist under ``specs_root`` (both the direct
``specs/NNN-.../requirements.md`` and the nested
``specs/NNN-.../checklists/requirements.md`` layouts are present in
this repo)."""
if not specs_root.is_dir():
return []
parser_mod = _load_parse_spec_requirements()
facts: list[str] = []
seen_paths: set[Path] = set()
paths = sorted(specs_root.glob("*/requirements.md")) + sorted(
specs_root.glob("*/checklists/requirements.md")
)
for path in paths:
resolved = path.resolve()
if resolved in seen_paths:
continue
seen_paths.add(resolved)
spec_dirname = path.relative_to(specs_root).parts[0]
match = _SPEC_DIR_RE.match(spec_dirname)
if not match:
continue
spec_id = match.group(1)
raw_facts = parser_mod.parse_requirements(spec_id, str(path))
facts.extend(f"{fact}." for fact in raw_facts)
return facts
def extract_facts(
ast_json_path: Path = AST_ULTIMATE_JSON,
specs_root: Path = SPECS_ROOT,
output_path: Path = TRACEABILITY_FACTS,
) -> tuple[int, int, int]:
"""Regenerate ``output_path`` from ``ast_json_path`` + ``specs_root``.
Returns ``(n_traces, n_spec_requirements, n_entrypoints)``.
"""
with open(ast_json_path, encoding="utf-8") as fh:
ast_data = json.load(fh)
modules = ast_data.get("modules", [])
traces_facts = _traces_facts(modules)
entrypoint_facts = _entrypoint_facts(modules)
specreq_facts = _spec_requirement_facts(specs_root)
lines = [
"% traceability_facts.pl -- GENERATED by scripts/trace_intel.py extract.",
"% Do not hand-edit; regenerate via `python scripts/trace_intel.py extract`.",
"",
":- dynamic traces/2, spec_requirement/3, entrypoint/2.",
"",
*traces_facts,
"",
*specreq_facts,
"",
*entrypoint_facts,
"",
]
output_path.parent.mkdir(parents=True, exist_ok=True)
output_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
return len(traces_facts), len(specreq_facts), len(entrypoint_facts)
def cmd_extract(_args: argparse.Namespace) -> int:
n_traces, n_specreqs, n_entrypoints = extract_facts()
print(
f"trace_intel: wrote {n_traces} traces/2, {n_specreqs} spec_requirement/3, "
f"{n_entrypoints} entrypoint/2 facts to {TRACEABILITY_FACTS}",
file=sys.stderr,
)
return 0
# ---------------------------------------------------------------------------
# Prolog session — codebase_db.pl + structural facts + traceability_db.pl
# + traceability_facts.pl, consulted together in one swipl subprocess.
# ---------------------------------------------------------------------------
def _kb_files() -> list[Path]:
return [
CODEBASE_DB,
STRUCTURAL_FACTS,
TRACEABILITY_DB,
TRACEABILITY_FACTS,
EXPLAIN_PL,
]
# The entrypoint-expansion (263 entrypoint/2 facts vs. the original 9)
# makes `report --merged` tabling `merged_reaches/2` (calls_resolved/3 UNION
# runtime_call/2, ~7K runtime edges) from every one of those 263 roots --
# measured at ~8GB of SLG table space and ~173s wall time on the real repo
# KB, against SWI's 1GB default `table_space` flag (immediate
# `private_table_space` resource error) and this module's prior 180s
# subprocess timeout (which the 173s real run would occasionally miss under
# load). Both bumped here so the pre-existing `report --merged` subcommand
# keeps working at the larger entrypoint scale; harmless for the cheaper
# non-merged queries since `--table-space` is a ceiling, not a reservation.
_SWIPL_TABLE_SPACE = "8G"
_PROLOG_TIMEOUT_SECONDS = 300
# Consulting the ~13MB generated structural-facts file dominates every
# swipl round trip (measured 1.70s of a 2.9s `report` call); the same
# clauses load from a precompiled .qlf in ~0.10s. Compilation is
# mtime-gated and BEST-EFFORT: on any failure the original .pl path is
# returned, so correctness never depends on QLF availability. The size
# floor keeps tiny fixture/rule files (where qcompile costs more than it
# saves) on the plain-consult path.
_QLF_MIN_BYTES = 1_000_000
_QLF_COMPILE_TIMEOUT_SECONDS = 120
def _qlf_for(pl_path: Path) -> Path:
"""Return a fresh precompiled ``.qlf`` twin for large ``.pl`` fact
files, compiling it when missing/stale; fall back to ``pl_path``."""
try:
if pl_path.suffix != ".pl" or pl_path.stat().st_size < _QLF_MIN_BYTES:
return pl_path
qlf_path = pl_path.with_suffix(".qlf")
if qlf_path.exists() and qlf_path.stat().st_mtime >= pl_path.stat().st_mtime:
return qlf_path
escaped = str(pl_path).replace("'", "''")
proc = subprocess.run(
["swipl", "-q", "-g", f"qcompile('{escaped}')", "-t", "halt"],
capture_output=True,
text=True,
timeout=_QLF_COMPILE_TIMEOUT_SECONDS,
)
if proc.returncode == 0 and qlf_path.exists():
return qlf_path
except OSError, subprocess.SubprocessError:
pass
return pl_path
_WORKER_BOOT_PL = """\
% GENERATED by scripts/trace_intel.py -- warm-worker bootstrap.
% ti_query/3 takes the goal as an ATOM (read at runtime) so the swipl
% toplevel's DWIM unknown-predicate scan can never abort the wrapper
% term itself; the done-file is written unconditionally so the Python
% side always unblocks, result file present or not.
:- use_module(library(http/json)).
ti_query(GoalAtom, ResFile, DoneFile) :-
catch(ignore(( read_term_from_atom(GoalAtom, G, [variable_names(Vs)]),
memberchk('R'=R, Vs),
call(G),
setup_call_cleanup(
open(ResFile, write, Out, [encoding(utf8)]),
( json_write_dict(Out, R, [width(0)]), nl(Out) ),
close(Out)) )),
_, true),
setup_call_cleanup(open(DoneFile, write, D, []), true, close(D)).
"""
class _WarmPrologWorker:
"""One persistent ``swipl`` process holding the STANDARD repo KB.
The KB (codebase_db.pl + the two generated fact files +
traceability_db.pl) is immutable between ``extract``/reindex runs,
so consulting it once and streaming goals over stdin removes the
per-query consult cost and lets SLG tables persist across queries.
The worker is keyed on ``(path, mtime_ns, size)`` of every KB file:
any change restarts it, so a stale KB can never be queried. Any
anomaly (worker death, missing result, unreadable KB) makes
``query`` return ``None`` and the caller falls back to the one-shot
subprocess path, whose semantics are unchanged.
"""
def __init__(self) -> None:
self._proc: subprocess.Popen[str] | None = None
self._key: tuple[tuple[str, int, int], ...] | None = None
self._lock = threading.Lock()
@staticmethod
def _kb_key(kb_files: list[Path]) -> tuple[tuple[str, int, int], ...]:
return tuple((str(p), p.stat().st_mtime_ns, p.stat().st_size) for p in kb_files)
def _start(self, kb_files: list[Path], key: tuple) -> None:
self._stop_locked()
SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
boot_path = SCRATCH_DIR / "ti_worker_boot.pl"
boot_path.write_text(_WORKER_BOOT_PL, encoding="utf-8")
load_paths = [str(_qlf_for(p)) for p in kb_files]
with open(SCRATCH_DIR / "ti_worker.log", "w", encoding="utf-8") as log:
self._proc = subprocess.Popen(
[
"swipl",
"-q",
f"--table-space={_SWIPL_TABLE_SPACE}",
str(boot_path),
*load_paths,
],
stdin=subprocess.PIPE,
stdout=log,
stderr=subprocess.STDOUT,
text=True,
)
self._key = key
def _stop_locked(self) -> None:
if self._proc is not None:
with contextlib.suppress(OSError):
self._proc.kill()
with contextlib.suppress(Exception):
self._proc.wait(timeout=10)
self._proc = None
self._key = None
def shutdown(self) -> None:
with self._lock:
self._stop_locked()
def query(self, goal: str, kb_files: list[Path]) -> dict[str, Any] | None:
"""Run ``goal`` on the warm worker; ``None`` => use the one-shot path."""
with self._lock:
try:
key = self._kb_key(kb_files)
except OSError:
return None
if self._proc is None or self._proc.poll() is not None or key != self._key:
try:
self._start(kb_files, key)
except OSError:
self._stop_locked()
return None
return self._query_locked(goal)
def _query_locked(self, goal: str) -> dict[str, Any] | None:
if self._proc is None or self._proc.stdin is None:
return None
result_fd, result_path = tempfile.mkstemp(suffix=".json", dir=str(SCRATCH_DIR))
os.close(result_fd)
done_fd, done_path = tempfile.mkstemp(suffix=".done", dir=str(SCRATCH_DIR))
os.close(done_fd)
for path in (result_path, done_path):
with contextlib.suppress(OSError):
os.unlink(path)
try:
escaped_goal = goal.replace("\\", "\\\\").replace("'", "''")
term = f"ti_query('{escaped_goal}', '{result_path}', '{done_path}').\n"
self._proc.stdin.write(term)
self._proc.stdin.flush()
deadline = time.monotonic() + _PROLOG_TIMEOUT_SECONDS
while not os.path.exists(done_path):
if self._proc.poll() is not None:
self._stop_locked()
return None
if time.monotonic() > deadline:
self._stop_locked()
return {"error": "timeout"}
time.sleep(0.005)
if os.path.exists(result_path):
with open(result_path, encoding="utf-8") as fh:
content = fh.read().strip()
if content:
parsed = json.loads(content)
if isinstance(parsed, dict):
return parsed
return {"error": "json_not_object", "raw": content[:500]}
return None
except OSError, ValueError:
self._stop_locked()
return None
finally:
for path in (result_path, done_path):
with contextlib.suppress(OSError):
os.unlink(path)
# The tests load this script FRESH per test via importlib, so a plain
# module global would silently fork one worker (and one KB consult) per
# test; anchoring the singleton in sys.modules keeps ONE warm engine per
# process. Benign creation race accepted: construction spawns nothing
# (the swipl process starts lazily on first query).
_WORKER_REGISTRY_MODULE = "trace_intel__warm_worker"
def _get_worker() -> _WarmPrologWorker:
registry = sys.modules.setdefault(
_WORKER_REGISTRY_MODULE, types.ModuleType(_WORKER_REGISTRY_MODULE)
)
worker = getattr(registry, "worker", None)
if worker is None:
worker = _WarmPrologWorker()
setattr(registry, "worker", worker) # noqa: B010 -- dynamic module attr
atexit.register(worker.shutdown)
return worker
def _run_prolog(goal: str, kb_files: list[Path]) -> dict[str, Any]:
"""Run ``goal`` against ``kb_files``: the standard repo KB goes to
the warm persistent worker (falling back to one-shot on any
anomaly); every non-standard KB list (fixtures, empty) always uses
the one-shot subprocess with unchanged semantics."""
if kb_files == _kb_files() and all(p.exists() for p in kb_files):
result = _get_worker().query(goal, kb_files)
if result is not None:
return result
return _run_prolog_oneshot(goal, kb_files)
def _run_prolog_oneshot(goal: str, kb_files: list[Path]) -> dict[str, Any]:
"""Consult every file in ``kb_files`` (in order) then run ``goal`` in
one standalone ``swipl`` subprocess, returning its ``R`` binding
(a ``json{}`` dict written via ``json_write_dict``) as a Python dict.
Mirrors ``CodebaseTools._run_subprocess``'s query/result scratch-file
protocol (own tempfiles, unlinked result file so a stale prior run can
never be read back) but generalized to consult N files, not just
(rules, facts).
"""
SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
query_fd, query_path = tempfile.mkstemp(suffix=".pl", dir=str(SCRATCH_DIR))
os.close(query_fd)
result_fd, result_path = tempfile.mkstemp(suffix=".json", dir=str(SCRATCH_DIR))
os.close(result_fd)
with contextlib.suppress(OSError):
os.unlink(result_path)
try:
with open(query_path, "w", encoding="utf-8") as fh:
# json_write_dict lives in library(http/json); the warm worker
# declares it explicitly (_WORKER_BOOT_PL) and the one-shot file
# must too — autoload does not resolve it on every SWI build
# (guix swipl 9.3.8: "Unknown procedure: json_write_dict/3").
fh.write(":- use_module(library(http/json)).\n")
for kb_file in kb_files:
escaped = str(_qlf_for(kb_file)).replace("'", "''")
fh.write(f":- consult('{escaped}').\n")
escaped_result = result_path.replace("'", "''")
fh.write(f"run :- {goal}, ")
fh.write("setup_call_cleanup(")
fh.write(f"open('{escaped_result}', write, Out, [encoding(utf8)]),")
fh.write("(json_write_dict(Out, R, [width(0)]), nl(Out)),")
fh.write("close(Out)).\n")
cmd = [
"swipl",
"-q",
f"--table-space={_SWIPL_TABLE_SPACE}",
"-g",
"run",
"-g",
"halt",
query_path,
]
proc = subprocess.run(
cmd, capture_output=True, text=True, timeout=_PROLOG_TIMEOUT_SECONDS
)
if os.path.exists(result_path):
with open(result_path, encoding="utf-8") as fh:
content = fh.read().strip()
if content:
parsed = json.loads(content)
if isinstance(parsed, dict):
return parsed
return {"error": "json_not_object", "raw": content[:500]}
return {
"error": "prolog_failed",
"returncode": proc.returncode,
"stderr": proc.stderr[:1000],
"stdout": proc.stdout[:1000],
}
except subprocess.TimeoutExpired:
return {"error": "timeout"}
finally:
with contextlib.suppress(OSError):
os.unlink(query_path)
with contextlib.suppress(OSError):
os.unlink(result_path)
def _needs_refresh(artifact: Path, *sources: Path) -> bool:
"""True when ``artifact`` is missing or older than ANY of ``sources``."""
if not artifact.exists():
return True
stamp = artifact.stat().st_mtime
return any(s.exists() and stamp < s.stat().st_mtime for s in sources)
def _traceability_sources() -> list[Path]:
"""Every input ``extract_facts`` reads, for the staleness gate.
``traceability_facts.pl`` used to be gated on ``plans/ast-ultimate.json``
alone, but ``extract_facts`` also reads the requirement checklists under
``specs/`` and the parser module that reads them. Editing a checklist --
or the parser -- therefore left the cached facts on disk and every
consumer (``derive_rtm``, ``trace_intel`` queries) silently answered from
the pre-edit KB. That is a false-number generator: a measurement taken
right after such an edit reports the state BEFORE it, which is
indistinguishable from "the edit had no effect".
"""
sources = [AST_ULTIMATE_JSON, _PARSE_SPEC_REQ_PATH]
if SPECS_ROOT.is_dir():
sources += sorted(SPECS_ROOT.glob("*/requirements.md"))
sources += sorted(SPECS_ROOT.glob("*/checklists/requirements.md"))
return sources
#: The UNTRACKED live snapshot. ``plans/ast-ultimate.json`` is a committed
#: artifact and nothing on a query path may write it -- an always-on writer
#: kept it permanently dirty and churned it into unrelated commits
#: (DESIGN_WAIVER 2026-08-09, backlog #284), and one refresh was measured
#: modifying two tracked files again on 2026-08-22. So the query path keeps
#: its own copy under ``.cache/`` (gitignored), seeded from the committed one.
LIVE_SNAPSHOT_DIR = REPO_ROOT / ".cache" / "ast"
LIVE_SNAPSHOT = LIVE_SNAPSHOT_DIR / "ast-ultimate.json"
_AST_SYNC = REPO_ROOT / ".venv" / "bin" / "ast-sync"
_RUNTIME_CALLGRAPH = REPO_ROOT / "plans" / "runtime-callgraph.json"
def _active_snapshot() -> Path:
"""The snapshot the structural KB is reindexed from.
The live one whenever it exists; the committed one is the cold-start seed
and the CI artifact, and is only ever as fresh as the last gate run.
"""
return LIVE_SNAPSHOT if LIVE_SNAPSHOT.exists() else AST_ULTIMATE_JSON
def _newest_source_mtime() -> float:
"""Newest mtime across the trees the snapshot is serialized from.
Imported from ``ast_pipeline`` rather than restated: that module is the
one place that knows which roots are in the snapshot (2977 modules across
src/, scripts/, tests/ and claude/), and a second copy of that list here
would drift and silently stop refreshing whichever tree it forgot.
"""
from orchestrata_ultimate.tools.ast_pipeline import source_trees
newest = 0.0
for tree in source_trees():
if not tree.exists():
continue
for path in tree.rglob("*.py"):
mtime = path.stat().st_mtime
if mtime > newest:
newest = mtime
return newest
def _refresh_snapshot() -> None:
"""Bring the LIVE snapshot up to date with the working tree.
This is the leg that was missing. ``ensure_kb`` refreshed the fact files
FROM the snapshot, but nothing refreshed the snapshot FROM disk, so an
edit left the entire chain internally consistent with bytes that no
longer existed -- and re-running the query could not fix it. The sidekick
could only refuse and name a command for a human to run, which is exactly
the manual step this removes.
Cheap check, expensive fix. Stat-walking the source trees costs
milliseconds; the refresh itself was measured at ~12s warm (reusing 2976
of 2977 modules), which is fine on demand and unaffordable per query.
``runtime_calls`` is re-applied afterwards, always. ``serialize`` knows
nothing about that field, so a bare re-serialize silently strips the
merged static+runtime call graph that ``derive_rtm``, ``code_intel``'s
merged queries and ``trace-path`` depend on -- measured twice, and the
reason gates.py pairs the two steps in a single function.
Failure is deliberately NOT fatal: an unrefreshed snapshot stays stale,
``kb_module_changed/2`` keeps reporting it, and the sidekick still answers
INVALID-with-names (SPEC-114 FR4-003). Raising here would replace a
precise refusal with a crash, and swallowing it would answer over stale
bytes -- the very thing FR4-003 exists to prevent.
"""
if not _AST_SYNC.exists():
return
snapshot = _active_snapshot()
if snapshot.exists() and snapshot.stat().st_mtime >= _newest_source_mtime():
return
print(
"trace_intel: source tree is newer than the AST snapshot -- "
"refreshing incrementally",
file=sys.stderr,
)
synced = subprocess.run(
[str(_AST_SYNC), "--output-dir", str(LIVE_SNAPSHOT_DIR)],
cwd=REPO_ROOT,
capture_output=True,
text=True,
)
if synced.returncode != 0:
print(
f"trace_intel: snapshot refresh FAILED ({synced.stderr.strip()[-400:]}) "
"-- answering from the previous snapshot, which stays reportable as "
"stale",
file=sys.stderr,
)
return
if _RUNTIME_CALLGRAPH.exists():
subprocess.run(
[
sys.executable,
str(REPO_ROOT / "scripts" / "build_runtime_callgraph.py"),
"reaugment",
"--ast",
str(LIVE_SNAPSHOT),
"--from",
str(_RUNTIME_CALLGRAPH),
],
cwd=REPO_ROOT,
capture_output=True,
text=True,
)
def ensure_kb(no_refresh: bool) -> None:
"""Auto-run ``extract`` + reindex the structural KB when either
artifact is missing or older than any of its inputs.
``--no-refresh`` skips this entirely and queries whatever is already
on disk under ``.cache/trace_intel/`` (raising if nothing is there).
"""
if not no_refresh and _needs_refresh(TRACEABILITY_FACTS, *_traceability_sources()):
print(
f"trace_intel: {TRACEABILITY_FACTS} missing/stale -- running extract",
file=sys.stderr,
)
extract_facts()
if not TRACEABILITY_FACTS.exists():
raise SystemExit(
f"trace_intel: {TRACEABILITY_FACTS} does not exist -- run "
"`extract` first, or drop --no-refresh"
)
if not no_refresh:
_refresh_snapshot()
snapshot = _active_snapshot()
if not no_refresh and _needs_refresh(STRUCTURAL_FACTS, snapshot):
print(
f"trace_intel: {STRUCTURAL_FACTS} missing/stale -- reindexing structural KB",
file=sys.stderr,
)
tools = CodebaseTools(facts_file=str(STRUCTURAL_FACTS), transport="subprocess")
result = tools.reindex(str(snapshot))
if "error" in result:
raise SystemExit(f"trace_intel: reindex failed: {result}")
if not STRUCTURAL_FACTS.exists():
raise SystemExit(
f"trace_intel: {STRUCTURAL_FACTS} does not exist -- reindex failed "
"silently, or --no-refresh was passed before a first run"
)
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def _print_result(result: dict[str, Any], json_mode: bool) -> None:
if json_mode:
print(json.dumps(result, indent=2, sort_keys=True))
return
if "error" in result:
print(f"error: {result}", file=sys.stderr)
return
for key, value in result.items():
if isinstance(value, list):
print(f"{key}: ({len(value)})")
for item in value:
print(f" - {item}")
else:
print(f"{key}: {value}")
# static-key, merged-key, display-label triples for the --merged comparison
# rows -- tool_report_merged/1's json{} carries exactly these six counters
# plus runtime_edges (see traceability_db.pl).
_MERGED_ROWS: tuple[tuple[str, str, str], ...] = (
("orphans_static", "orphans_merged", "orphans"),
("unspecced_static", "unspecced_merged", "unspecced"),
("specless_static", "specless_merged", "specless_usecases"),
)
def _print_merged_result(result: dict[str, Any]) -> None:
"""Text-mode renderer for ``report --merged``: a static-vs-merged
comparison (each row's static count, merged count, and delta) instead
of ``_print_result``'s flat key/value dump -- the default (no-flag)
``report`` output is untouched."""
if "error" in result:
print(f"error: {result}", file=sys.stderr)
return
print(f"runtime_edges: {result.get('runtime_edges', 0)}")
for static_key, merged_key, label in _MERGED_ROWS:
static_v = result.get(static_key, 0)
merged_v = result.get(merged_key, 0)
delta = merged_v - static_v
sign = "+" if delta >= 0 else ""
print(f"{label}: static={static_v} merged={merged_v} delta={sign}{delta}")
def _cmd_report(args: argparse.Namespace) -> dict[str, Any]:
goal = (
"tool_report_merged(R)" if getattr(args, "merged", False) else "tool_report(R)"
)
return _run_prolog(goal, _kb_files())
def _cmd_narrative(args: argparse.Namespace) -> dict[str, Any]:
goal = f"tool_narrative({_pl_atom(args.qname)}, R)"
return _run_prolog(goal, _kb_files())
def _cmd_orphans(args: argparse.Namespace) -> dict[str, Any]:
return _run_prolog(f"tool_orphans({int(args.limit)}, R)", _kb_files())
def _cmd_untraced_reqs(args: argparse.Namespace) -> dict[str, Any]:
return _run_prolog(f"tool_untraced_reqs({int(args.limit)}, R)", _kb_files())
def _cmd_unspecced(args: argparse.Namespace) -> dict[str, Any]:
return _run_prolog(f"tool_unspecced({int(args.limit)}, R)", _kb_files())
def _cmd_specless_usecases(args: argparse.Namespace) -> dict[str, Any]:
return _run_prolog(f"tool_specless_usecases({int(args.limit)}, R)", _kb_files())
def _cmd_why_not(args: argparse.Namespace) -> dict[str, Any]:
"""SPEC-114 FR-004 — why isn't this requirement implemented?"""
goal = f"tool_why_not_req({_pl_atom(args.req_id)}, R)"
return _run_prolog(goal, _kb_files())
_QUERY_DISPATCH: dict[str, Callable[[argparse.Namespace], dict[str, Any]]] = {
"report": _cmd_report,
"narrative": _cmd_narrative,
"orphans": _cmd_orphans,
"untraced-reqs": _cmd_untraced_reqs,
"unspecced": _cmd_unspecced,
"specless-usecases": _cmd_specless_usecases,
"why-not": _cmd_why_not,
}
def _add_common_args(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--json", action="store_true", help="Print raw JSON, not text.")
parser.add_argument(
"--no-refresh",
action="store_true",
help="Skip the auto extract/reindex freshness check; query the KB as-is.",
)
def _build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="trace_intel",
description=(
"T1 traceability / use-case-narrative engine -- semantic Prolog "
"layer over the structural code-intel KB."
),
)
sub = parser.add_subparsers(dest="command", required=True)
sub.add_parser(
"extract",
help="Regenerate traceability_facts.pl from plans/ast-ultimate.json + specs/.",
)
report_p = sub.add_parser(
"report", help="Run all five finders and print their counts."
)
report_p.add_argument(
"--merged",
action="store_true",
help=(
"Compare static vs merged (static+runtime) reachability via "
"tool_report_merged/1 instead of the default tool_report/1."
),
)
_add_common_args(report_p)
narrative_p = sub.add_parser(
"narrative", help="Modules transitively reachable from one entrypoint."
)
narrative_p.add_argument(
"qname", help="Entrypoint qualified name (see `report`'s entrypoints count)."
)
_add_common_args(narrative_p)
limited = [
("orphans", "Traced code unreachable from any known entrypoint."),
("untraced-reqs", "Spec requirements with zero @traces coverage."),
("unspecced", "Code reachable from an entrypoint but carrying no @traces."),
(
"specless-usecases",
"Specs whose requirements never chain to reachable code.",
),
]
for name, help_text in limited:
p = sub.add_parser(name, help=help_text)
p.add_argument(
"--limit", type=int, default=50, help="Max items to list (default 50)."
)
_add_common_args(p)
why_not_p = sub.add_parser(
"why-not",
help="Why isn't requirement X implemented? Names the first missing conjunct.",
)
why_not_p.add_argument(
"req_id", help="Requirement id, e.g. REQ-000-004 (see `untraced-reqs`)."
)
_add_common_args(why_not_p)
return parser
def main(argv: list[str] | None = None) -> int:
"""Parse ``argv`` and dispatch to the matching subcommand handler."""
parser = _build_parser()
args = parser.parse_args(argv)
if args.command == "extract":
return cmd_extract(args)
ensure_kb(no_refresh=args.no_refresh)
result = _QUERY_DISPATCH[args.command](args)
if args.command == "report" and getattr(args, "merged", False) and not args.json:
_print_merged_result(result)
else:
_print_result(result, json_mode=args.json)
# REQ-114-FR4-003: three-valued, matching the gate layer's own contract
# (`gate run` exits 0 pass / 1 fail / 2 invalid). `invalid` is NOT an
# error in the query — the query was well-formed and the requirement may
# be perfectly fine; the FACT BASE could not answer for the bytes on
# disk. Collapsing it into 0 is the false-green freshness.pl was written
# against; collapsing it into 1 would read as "this requirement failed".
if result.get("outcome") == "invalid":
return 2
return 1 if "error" in result else 0
if __name__ == "__main__":
sys.exit(main())