orchestrata_ast.models

Reference › orchestrata_ast › models

This module defines the data model of the serialized code AST, using standard-library dataclasses only. It was extracted from the serializer script so that the command-line tool and library packages such as orchestrata_tools share one definition. The JSON emitted from these dataclasses is the interchange format read by json_loader and compared in snapshots under plans/. It is produced by analyzer and flattened by serialize.

The model nests from the top down:

• CodebaseAST holds the name, root path, extra roots, a list of ModuleInfo, the derived inheritance_graph and dependency_graph, the totals, and a captures list naming the per-module fields the writing analyzer emits. That list is part of the warm-start cache identity.
• ModuleInfo holds ClassInfo, FunctionInfo, ImportInfo, module variables, __all__ exports, EnvAccessInfo and ArgvLiteralInfo records, and a residue hash for the file text outside top-level definitions.
• ClassInfo and FunctionInfo carry line spans, body hashes, decorators, bases, parameters (ParamInfo), and the calls a function makes. Class variables are ClassVarInfo.
• ImportInfo records the line, a scope label (module, class, function or type-checking), the owner and whether the import is guarded.
• EnvAccessInfo and ArgvLiteralInfo record located environment reads and literal program names at spawn sites, because the plain call list loses both.

Field order and meaning are frozen. New fields are added with defaults so old JSON still loads.

At a glance

Class diagram

Dependency diagram

API

Classes

ParamInfo (dataclass)

Decorators: dataclass · Line 25

FunctionInfo (dataclass)

Decorators: dataclass · Line 33

ClassVarInfo (dataclass)

Decorators: dataclass · Line 58

ClassInfo (dataclass)

Decorators: dataclass · Line 66

ImportInfo (dataclass)

Decorators: dataclass · Line 90

EnvAccessInfo (dataclass)

Decorators: dataclass · Line 137

One read of the process environment, located.

ArgvLiteralInfo (dataclass)

Decorators: dataclass · Line 167

One literal program name at a process-spawn site, located.

ModuleInfo (dataclass)

Decorators: dataclass · Line 238

CodebaseAST (dataclass)

Decorators: dataclass · Line 260

Source

Permalink: https://gitlab.com/MSamy/orchestrata-ultimate-architecture/-/blob/a4cbc2bdf/src/orchestrata_ast/models.py

"""Data model for the serialized code AST — stdlib-only.

Extracted from scripts/serialize_codebase_ast.py (2026-07-10) so that BOTH
the CLI script and library packages (orchestrata_tools) reference ONE
canonical component for AST generation. Field order and semantics are
frozen: the JSON emitted from these dataclasses is the interchange format
consumed by orchestrata_tools.parser.json_loader and every downstream
comparison snapshot in plans/.

2026-08-02 (user mandate: "the AST must capture byte-identical code — all
of the code, leave nothing behind"): the format gained byte-exact capture
fields — ``FunctionInfo.end_line``/``body_hash``, ``ClassInfo.end_line``/
``body_hash``, ``ModuleInfo.residue_hash``. Every one of them carries a
default so OLD serialized JSON keeps loading unchanged. Field ADDITION
with defaults is the sanctioned evolution path for this format; the order
and semantics of the pre-existing fields stay frozen.
"""

from __future__ import annotations

from dataclasses import dataclass, field


@dataclass
class ParamInfo:
    name: str
    type_annotation: str | None = None
    default: str | None = None
    kind: str = "positional"  # positional, varargs, keyword_only, kwargs


@dataclass
class FunctionInfo:
    name: str
    qualified_name: str  # module.Class.method or module.function
    module_path: str
    file_path: str
    line: int
    is_async: bool = False
    is_method: bool = False
    is_classmethod: bool = False
    is_staticmethod: bool = False
    is_property: bool = False
    is_abstract: bool = False
    decorators: list[str] = field(default_factory=list)
    parameters: list[ParamInfo] = field(default_factory=list)
    return_type: str | None = None
    docstring: str | None = None
    class_name: str | None = None
    calls: list[str] = field(default_factory=list)  # functions/methods this calls
    end_line: int = (
        0  # node.end_lineno; 0 only for old-format JSON predating this field
    )
    body_hash: str = ""  # sha256(ast.get_source_segment(text, node))[:16] — byte-exact


@dataclass
class ClassVarInfo:
    name: str
    type_annotation: str | None = None
    default: str | None = None
    is_class_var: bool = False


@dataclass
class ClassInfo:
    name: str
    qualified_name: str  # module.ClassName
    module_path: str
    file_path: str
    line: int
    bases: list[str] = field(default_factory=list)
    decorators: list[str] = field(default_factory=list)
    methods: list[FunctionInfo] = field(default_factory=list)
    class_vars: list[ClassVarInfo] = field(default_factory=list)
    docstring: str | None = None
    is_abstract: bool = False
    is_dataclass: bool = False
    is_pydantic: bool = False
    is_protocol: bool = False
    is_enum: bool = False
    inner_classes: list[str] = field(default_factory=list)
    end_line: int = (
        0  # node.end_lineno; 0 only for old-format JSON predating this field
    )
    body_hash: str = ""  # sha256(ast.get_source_segment(text, node))[:16] — byte-exact


@dataclass
class ImportInfo:
    module: str
    names: list[str] = field(default_factory=list)  # imported names
    alias: str | None = None
    is_from: bool = False
    is_relative: bool = False
    level: int = 0  # relative import level
    # Source line of the `import`/`from` statement. 0 only for an ImportInfo
    # built by hand rather than from a parse.
    #
    # Added for the Prolog gate fact base (plans/prolog-gates-implementation-
    # plan.md P1): a violation is reported as file:line, and the snapshot is
    # the ONLY sanctioned source of structural facts (REQ-GATE-001), so an
    # import with no line forces every import rule to either re-open the file
    # or report a location it cannot name.
    line: int = 0

    # Where the statement lives, and therefore WHEN it executes. Until this
    # existed the snapshot only carried imports that were direct children of
    # the Module node, so a lazy `import` inside a function body was invisible
    # — and `from_name/4`, the Prolog fact that resolves a call target back to
    # its module, never had the data. Measured consequence: `code_intel.py
    # callers core.redis_factory.create_sync_redis` answered 0 callers while
    # real ones existed, because every one of them imports it inside a
    # function.
    #
    #   "module"        — direct child of the Module node; runs at import time
    #   "class"         — class body; also runs at import time
    #   "function"      — function/method body; runs only when called (lazy)
    #   "type_checking" — under `if TYPE_CHECKING:`; NEVER runs
    #
    # Kept as a label rather than flattened into one list on purpose: a lazy
    # import is real runtime coupling but is NOT an import-time dependency,
    # and folding it into `dependency_graph` would resurrect exactly the
    # cycles that lazy imports are written to break.
    scope: str = "module"
    # Dotted name of the enclosing function/class, relative to the module
    # (e.g. "IntentValidator.__init__"). None at module scope.
    owner: str | None = None
    # True when the statement sits under any runtime guard — `if`/`try`/`for`/
    # `while`/`with`/`match`. A module-scope import inside `try: ... except
    # ImportError:` is the optional-dependency shape: it executes at import
    # time, but it is allowed to fail.
    conditional: bool = False


@dataclass
class EnvAccessInfo:
    """One read of the process environment, located.

    WHY THIS IS A SNAPSHOT FACT. `os.environ.get("X", default)` appears in the
    `calls` list as the bare string `os.environ.get`, usually without its
    arguments, and bare `os.environ["X"]` is a SUBSCRIPT that never appears in
    `calls` at all. So neither the KEY nor the FORM is recoverable from the
    snapshot as it stood, while RUL-003 is stated entirely in terms of both:
    required infra keys must RAISE when unset, everything else must use
    `.get` with a default.

    `form` is how the environment was reached:
        ``get``        os.environ.get("X") / os.environ.get("X", d)
        ``subscript``  os.environ["X"]  — the form RUL-003 forbids outright
        ``getenv``     os.getenv("X")
        ``other``      pop/setdefault/update and anything else

    `key` is the literal env var name, or "" when the subscript/argument is a
    computed expression rather than a literal — an unknown key is reported as
    unknown rather than guessed.
    """

    file_path: str
    line: int
    key: str
    form: str
    has_default: bool = False


@dataclass
class ArgvLiteralInfo:
    """One literal program name at a process-spawn site, located.

    WHY THIS IS A SNAPSHOT FACT. ``subprocess.run(["python", "x.py"])``
    reaches the ``calls`` list as the bare callee string ``subprocess.run``,
    WITHOUT arguments, and ``os.system("gcc -c a.c")`` the same way — so the
    program token, the one part of a spawn that decides whether ``PATH``
    resolves to this checkout's toolchain or to a foreign one, is
    unrecoverable from the snapshot as it stood. Measured 2026-10-11 on the
    host: the session ``PATH`` began with another worktree's ``.venv/bin``
    (~530 commits behind master), so a bare ``python`` in any spawned command
    silently executed that stale checkout's code, and ``gcc`` on ``PATH`` was
    a Guix build that fails the janus-swi native compile. Mohamed's directive:
    "fix bare python make it prolog and atoms" — the gate derives the
    ``bare_toolchain`` violations (Q2b) from facts extracted from this
    snapshot, and REQ-GATE-001 forbids the extractor from opening a ``.py``
    file. This is the sibling of ``EnvAccessInfo``: same shape (located,
    literal-only, recorded-not-guessed), different subject.

    THIS LAYER RECORDS AND CLASSIFIES NOTHING. Which programs count as bare
    toolchain invocations is decided by Prolog atoms (Q2b); every literal
    program token at a spawn-shaped site is recorded verbatim — absolute
    paths, non-toolchain heads, and non-spawning callees included, because a
    project wrapper like ``_run([...])`` is a spawn that no fixed callee list
    can name. Computed heads (``sys.executable``, a variable, a starred
    element) are skipped, not guessed.

    ``form`` is which spawn shape produced the record:

    | form    | AST shape captured                                              |
    |---------|-----------------------------------------------------------------|
    | ``argv``  | a literal-headed ``List``/``Tuple`` (element 0 a ``str``        |
    |           | ``Constant``) or a ``BinOp`` Add whose ``left`` is one, passed  |
    |           | as a positional argument to ANY callee — or, when the call has  |
    |           | no positional arguments, as its ``args``/``cmd``/``command``/   |
    |           | ``argv`` keyword; or assigned to a ``Name`` that a call in the  |
    |           | same scope (module body or one function body) passes as its     |
    |           | command argument                                                |
    | ``exec``  | last segment ``create_subprocess_exec``, or dotted ``os.execv*``|
    |           | /``os.execl*`` (first positional a ``str`` ``Constant``), or    |
    |           | ``os.spawnv*``/``os.spawnl*`` (SECOND positional — the first    |
    |           | is the mode)                                                    |
    | ``shell`` | dotted ``os.system``/``os.popen``, or last segment ``run``/     |
    |           | ``Popen``/``call``/``check_call``/``check_output``/``getoutput``|
    |           | /``getstatusoutput``/``create_subprocess_shell`` on base        |
    |           | ``subprocess`` (``asyncio`` too for                             |
    |           | ``create_subprocess_shell``) — first positional a ``str``       |
    |           | ``Constant`` or a ``JoinedStr`` whose leading literal chunk     |
    |           | contains whitespace; the program is that string's first         |
    |           | whitespace-separated token                                      |

    ``site`` is the dotted callee text of the spawning call (``Name`` -> its
    id, ``Attribute`` -> ``<value>.<attr>``, any other chain node -> ``?``);
    ``line`` is the line of the LITERAL that names the program — where a fix
    edits; ``program`` is the token verbatim (``python3``, ``gcc``,
    ``/usr/bin/gcc``, ``uv``, ``env``).

    KNOWN LIMITATION. A shell string prefixed by ``env`` (``"env -u X python
    y"``) records ``env``: the first token IS the program the shell execs.
    Looking past ``env`` is a later refinement (Q2b can do it in Prolog), not
    this layer's job.
    """

    file_path: str
    line: int  # line of the LITERAL that names the program (where a fix edits)
    site: str  # dotted callee text of the spawning call, e.g. "subprocess.run"
    form: str  # "argv" | "exec" | "shell"
    program: str  # the literal program token, verbatim: "python3", "/usr/bin/gcc"


@dataclass
class ModuleInfo:
    path: str  # Python import path (e.g. orchestrata_ultimate.agents.base_agent)
    file_path: str
    line_count: int = 0
    classes: list[ClassInfo] = field(default_factory=list)
    functions: list[FunctionInfo] = field(default_factory=list)
    imports: list[ImportInfo] = field(default_factory=list)
    module_vars: list[ClassVarInfo] = field(default_factory=list)
    docstring: str | None = None
    all_exports: list[str] = field(default_factory=list)  # __all__ contents
    # Every environment read in the module, at any nesting depth — see
    # EnvAccessInfo for why `calls` cannot answer this.
    env_accesses: list[EnvAccessInfo] = field(default_factory=list)
    # Every literal program name at a spawn-shaped call in the module, at any
    # nesting depth — see ArgvLiteralInfo for why `calls` cannot answer this.
    argv_literals: list[ArgvLiteralInfo] = field(default_factory=list)
    # sha256(file text minus top-level def/class line spans)[:16] — every
    # byte of the file lands in either this or a class/function body_hash.
    residue_hash: str = ""


@dataclass
class CodebaseAST:
    name: str
    root_path: str
    modules: list[ModuleInfo] = field(default_factory=list)
    # Derived data (computed after parsing)
    inheritance_graph: dict[str, list[str]] = field(default_factory=dict)
    dependency_graph: dict[str, list[str]] = field(default_factory=dict)
    total_classes: int = 0
    total_functions: int = 0
    total_methods: int = 0
    total_loc: int = 0
    #: Additional source trees this snapshot covers, repo-relative, in the
    #: order they were given. ``root_path`` remains THE root the dotted module
    #: paths are numbered from; these trees contribute modules whose path is
    #: prefixed with the tree's own directory name (``tests.``, ``scripts.``),
    #: which is what keeps the 850 pre-existing ids — and every consumer keyed
    #: on them — byte-identical when the snapshot widens.
    extra_roots: list[str] = field(default_factory=list)
    #: The per-module capture fields the analyzer that wrote this snapshot
    #: emits (``orchestrata_ast.analyzer.SNAPSHOT_CAPTURES``), and part of the
    #: warm-start cache identity: a snapshot written BEFORE a capture field
    #: existed holds module dicts without the key, and a reused dict
    #: deserializes as ``[]`` — indistinguishable from "captured, found none".
    #: ``load_or_analyze`` compares this list against the analyzer's own and
    #: forces one full reparse on any mismatch, so a snapshot can never be
    #: half-captured (2026-10-11).
    captures: list[str] = field(default_factory=list)