scripts.check_lexicon_collisions

Reference › scripts › check_lexicon_collisions

The lexicon/codebase collision gate: a work-type word must not also name a subsystem. If it did, naming the subsystem in a request would be read as a work-type signal. No module or test imports it; a gate or a human runs it. It reads the vocabulary from intent_vocabulary and uses janus_bootstrap for the Prolog side.

Invocation: main takes --list, which prints the subsystem names, the registries with their word counts and the exempt words, then exits 0, and --json, which prints collisions and exemptions as JSON. By default it prints an OK line or a FAIL line with one entry per collision. The exit code is 0 when clean, 1 on collisions and 2 when the gate cannot run, in which case it says so.

subsystem_names walks the source tree's directories to collect subsystem names with their paths. categorical_words loads the registries of categorical words, and stem reduces words to a stem so that variants match. find_collisions intersects the two sets, minus the exempt words in the allowlist.

At a glance

Class diagram

No classes; see Functions.

Dependency diagram

API

Constants

Functions

Source

Permalink: https://gitlab.com/MSamy/orchestrata-ultimate-architecture/-/blob/a4cbc2bdf/scripts/check_lexicon_collisions.py

#!/usr/bin/env python3
"""Lexicon/codebase collision gate — a work-type word must not name a subsystem.

THE DEFECT THIS EXISTS FOR, measured and shipped before it was caught.

Five words were added to the intent lexicon as work-type signals: ``memory``,
``trace``, ``integration``, ``version`` and ``dispatch``. Each of them also
names a subsystem in this repo. The result was that

    "wire the memory engine into the gateway"

classified as PERFORMANCE work, because ``memory`` was read as a performance
signal rather than as the name of the subsystem being wired. All five had to be
removed again, and the only thing that caught them was a human reading the
held-out corpus results. ``dispatch`` repeated the lesson late — mined with real
evidence, accepted by review, applied, and then rejected because ``dispatcher``
stems to it and "refactor the dispatcher into smaller functions" started
resolving to API work.

A search of the published literature found no prior art for statically checking
a controlled vocabulary against a live codebase symbol table. This gate is the
mechanism required by RUL-014 (a ruling resolves to a mechanism, never prose).

WHAT COUNTS AS A SUBSYSTEM NAME — measured, not assumed. Four discovery
strategies were run against the five known-bad words:

    strategy                     names  collisions  known-bad caught
    directories, depth 1            30           1  memory, integration
    directories, depth 2            93           6  + trace, dispatch
    dirs + modules, depth 2        394           9  + version  (ALL FIVE)
    dirs + modules, depth 3        577          16  no additional catches

Depth 2 including module files is the shipped operating point: it is the only
one that catches all five, and the two words the shallower strategies miss --
``version`` (``cxp/version.py``) and ``dispatch`` (``cxp/dispatcher.py``) -- are
both plain modules, not packages. Depth 3 is rejected: it doubles the exemption
burden and catches nothing further. The cost is the nine documented exemptions
below, 9 of 144 categorical words.

WHY NOT THE FULL SYMBOL TABLE. Almost any English word matches *some* function
or class name in a 122K-line tree — ``run``, ``add``, ``build`` and ``check``
all do — so gating on every symbol would flag most of the lexicon and be
switched off within a day. Package and module names are the high-precision
collision set: they are the vocabulary a person uses out loud when they say
"the memory engine" or "the session store".

WHAT IS AND IS NOT A COLLISION. Only *categorical* words are checked — the
work-type and generic-verb registries, which decide what KIND of work a sentence
is about. The target registry is deliberately exempt: a target word naming a
subsystem is not a bug, it is the entire point (``session``, ``gateway`` and
``kernel`` are all legitimate targets AND packages).

Usage:
    uv run python scripts/check_lexicon_collisions.py            # gate mode
    uv run python scripts/check_lexicon_collisions.py --list     # show the sets
    uv run python scripts/check_lexicon_collisions.py --json     # machine output

Exit codes: 0 clean, 1 collision found, 2 could not run the check.
"""

from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path

REPO_ROOT = Path(__file__).resolve().parents[1]
SRC_PACKAGE = REPO_ROOT / "src" / "orchestrata_ultimate"

sys.path.insert(0, str(REPO_ROOT / "src"))

# Words that name a subsystem here but are ALSO ordinary English work
# vocabulary, where the work reading is the one people actually mean. Each entry
# needs a reason naming the colliding path: an unexplained exemption is how a
# gate rots into a rubber stamp, and a stale one is how it starts lying. The
# `every exemption is a live collision` test deletes-by-failing any entry whose
# collision no longer exists.
ALLOWED: dict[str, str] = {
    "refactor": (
        "refactor/ is a package AND the canonical name of a WorkType; the "
        "collision is intentional and the work reading is correct"
    ),
    "review": (
        "verification/review/ is a package, but 'review the diff' is the "
        "dominant reading and REVIEW is a WorkType member"
    ),
    "rename": (
        "refactor/rename/ is a package; 'rename the function' is refactor work "
        "and the package is spoken of as 'the rename engine' in full"
    ),
    "requirement": (
        "verification/requirements/ is a package; 'write the requirements' is "
        "spec work, and the package is spoken of as the requirements parser"
    ),
    "contract": (
        "factory/contracts/ is a package; 'the API contract' is the work "
        "reading people use in this project's own turns"
    ),
    "audit": (
        "gateway/audit.py is a module; 'audit the specs' is work vocabulary and "
        "the module is spoken of as 'the gateway audit log' in full"
    ),
    "exception": (
        "agents/exceptions.py is a module, but nobody says 'the exception' to "
        "mean that file — they say 'the agents exceptions module'"
    ),
    "extract": (
        "drivers/extraction.py is a module; 'extract the method' is refactor "
        "work and the module is never referred to by the bare verb"
    ),
    "approve": (
        "tools/approval.py is a module, but nobody says 'the approve' to mean "
        "that file -- they say 'the approval gate'; meanwhile 'approve the "
        "escalation' is the dominant reading and names the real "
        "session-pipeline command"
    ),
    "generate": (
        "domaingen/generator.py is a module; 'generate the tests' is the "
        "dominant reading and the module is spoken of as the domain generator"
    ),
}


def subsystem_names() -> dict[str, str]:
    """Package directories and module files, two levels deep, name -> path.

    Depth 2 *including module files* — see the strategy table in the module
    docstring for the measurement behind that choice. A gate that misses the
    cases that caused it to be written is not worth running, and two of the
    five known-bad words are plain modules.

    Names are collected in sorted order and first-wins, so a collision reports a
    stable path when a name occurs more than once in the tree.
    """
    if not SRC_PACKAGE.is_dir():
        raise FileNotFoundError(f"source package not found: {SRC_PACKAGE}")

    names: dict[str, str] = {}

    def walk(directory: Path, level: int) -> None:
        if level > 2:
            return
        for entry in sorted(directory.iterdir()):
            if entry.name.startswith("_"):
                continue
            if entry.is_dir():
                rel = f"{entry.relative_to(SRC_PACKAGE)}/"
                names.setdefault(entry.name.lower(), rel)
                walk(entry, level + 1)
            elif entry.suffix == ".py":
                names.setdefault(
                    entry.stem.lower(), str(entry.relative_to(SRC_PACKAGE))
                )

    walk(SRC_PACKAGE, 1)
    return names


def categorical_words() -> dict[str, set[str]]:
    """The registries that decide WHAT KIND of work a sentence describes.

    Targets are excluded on purpose — see the module docstring.

    ``work_type_keywords`` is ``dict[WorkType, set[str]]`` and ``generic_verbs``
    is ``dict[str, WorkType]`` — different shapes. A first cut of this function
    iterated both directly and silently checked 31 items (10 WorkType members
    plus 21 verbs) instead of 144 words, which is the failure mode where a gate
    reports OK because it examined almost nothing.
    """
    from orchestrata_ultimate.orchestrator.intent_vocabulary import IntentVocabulary

    vocab = IntentVocabulary.default()
    work_type_words: set[str] = set()
    for words in vocab.work_type_keywords.values():
        work_type_words |= {w.lower() for w in words}
    return {
        "work_type": work_type_words,
        "generic_verb": {w.lower() for w in vocab.generic_verbs},
    }


def stem(word: str) -> str:
    """Porter stem, matching how the grammar actually compares tokens.

    Comparing raw strings misses the collisions that matter: the lexicon word
    ``trace`` against ``core/tracing/``, ``integration`` against
    ``integrations/``, and ``dispatch`` against ``cxp/dispatcher.py``. The
    grammar stems both sides before matching (REQ-003-US-001-050), so a collision
    check that does not stem is asking a different question than the parser
    does. Falls back to the raw lowercase word if the stemmer is unavailable,
    which degrades the gate to exact matching rather than switching it off.
    """
    try:
        from orchestrata_ultimate.core.janus_bootstrap import janus

        row = next(janus.query(f"porter_stem('{word}', S)"), None)
        return str(row["S"]) if row else word
    except Exception:  # noqa: BLE001 — degrade to exact match, never crash the gate
        return word


def find_collisions(*, ignore_allowed: bool = True) -> list[dict[str, str]]:
    """Every categorical lexicon word that also names a package or module."""
    by_stem: dict[str, tuple[str, str]] = {}
    for name, path in subsystem_names().items():
        by_stem.setdefault(stem(name), (name, path))

    collisions: list[dict[str, str]] = []
    for registry, words in categorical_words().items():
        for word in sorted(words):
            if ignore_allowed and word in ALLOWED:
                continue
            hit = by_stem.get(stem(word))
            if hit is None:
                continue
            collisions.append(
                {"word": word, "registry": registry, "name": hit[0], "path": hit[1]}
            )
    return collisions


def main(argv: list[str] | None = None) -> int:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--list", action="store_true", help="print both sets, exit 0")
    parser.add_argument("--json", action="store_true", help="machine-readable output")
    args = parser.parse_args(argv)

    try:
        names = subsystem_names()
        registries = categorical_words()
        collisions = find_collisions()
    except Exception as exc:  # noqa: BLE001 — a gate that cannot run must say so
        print(f"[lexicon-collision] CANNOT RUN: {type(exc).__name__}: {exc}")
        return 2

    if args.list:
        print(f"subsystem names ({len(names)}): {', '.join(sorted(names))}")
        for registry, words in registries.items():
            print(f"{registry} ({len(words)} words)")
        print(f"exempt ({len(ALLOWED)}): {', '.join(sorted(ALLOWED))}")
        return 0

    if args.json:
        print(
            json.dumps({"collisions": collisions, "exempt": sorted(ALLOWED)}, indent=1)
        )
        return 1 if collisions else 0

    total = sum(len(w) for w in registries.values())
    if not collisions:
        print(
            f"[lexicon-collision] OK — {total} categorical words checked against "
            f"{len(names)} subsystem names, {len(ALLOWED)} exempt, 0 collisions"
        )
        return 0

    print(f"[lexicon-collision] FAIL — {len(collisions)} collision(s)\n")
    for c in collisions:
        print(
            f"  {c['word']!r} is registered as a {c['registry']} word AND names "
            f"src/orchestrata_ultimate/{c['path']}"
        )
    print(
        "\n  A word that names a subsystem here will be read as a work-type signal\n"
        "  whenever someone names that subsystem: 'wire the memory engine into the\n"
        "  gateway' becomes performance work. Remove the word from\n"
        "  intent_vocabulary.py and regenerate, or add it to ALLOWED in this file\n"
        "  WITH the reason the work reading wins."
    )
    return 1


if __name__ == "__main__":
    raise SystemExit(main())