scripts.check_requirement_corpus

Reference › scripts › check_requirement_corpus

The requirement-corpus gate: production code must not cite a requirement that the traceability matrix cannot grade. It takes the cited requirement ids from src/ and checks them against the ratified specification corpus, using the harvest functions in generate_rtm_evidence. tests/scripts/test_check_requirement_corpus.py and tests/scripts/test_check_requirement_corpus_main.py import it.

Invocation: main takes --advisories, which also prints spec and checklist documentation drift as advisory lines. collect builds a CorpusFacts record, whose dangling view maps each unresolvable id to the files citing it, and check returns the violations with the numbers of ids and citations examined. A run that examined no ids or no citations returns the invalid code, because a pass with nothing examined is not a pass. Violations print a FAIL line and a list of files and return the violations code; otherwise an OK line reports the number of distinct ids cited and the size of the ratified corpus. checklist_advisories produces the advisory lines.

At a glance

Class diagram

Dependency diagram

API

Constants

Classes

CorpusFacts (dataclass)

Decorators: dataclass(frozen=True) · Line 75

Functions

Source

Permalink: https://gitlab.com/MSamy/orchestrata-ultimate-architecture/-/blob/a4cbc2bdf/scripts/check_requirement_corpus.py

#!/usr/bin/env python3
"""Requirement-corpus gate — production code must not cite an ungradeable requirement.

This repo had two things deciding "what requirements exist":

* ``serialize_codebase_ast._requirement_source_files`` — the RATIFIED corpus:
  ``<spec>/checklists/requirements.md`` + ``<spec>/spec.md`` + ``<spec>/*-addendum.md``.
* ``generate_rtm_evidence.load_requirements`` — what the RTM actually grades against.

They drifted, and the drift was invisible for as long as nobody compared them.
The toolkit widened its scan after finding that a checklist-only glob made 43
real ``@traces`` look dangling; the RTM generator never received that fix. So
``trace-lint`` called those citations clean while the RTM graded them GHOST,
and 68 requirement ids across 6 files — all of SPEC-033 and SPEC-034, 22 of
SPEC-110, the ``001-kernel`` and ``097`` addenda — sat outside the matrix's
universe. Not UNLINKED. Not GHOST. Absent from the denominator, so every
coverage percentage was computed against the wrong total.

The defect was never a missing checklist. It was two readers of one corpus with
nobody asserting they agree.

WHAT THIS GATE ASSERTS, and why it is this and not the obvious thing:

The obvious gate — compare ``load_requirements()`` against the corpus function
it calls — is a TAUTOLOGY. It can never fail, which is the same false green as
a checker that never looked. So the load-bearing comparison is against
``@traces`` decorators harvested from the AST of ``src/``, which know nothing
about how requirements are loaded and can therefore genuinely disagree:

``DANGLING``
    shipping code cites a requirement id the RTM cannot grade. Either the
    decorator is wrong, or the loader has been narrowed again. Both are
    defects, and this gate fails on either without needing to know which.

Documentation drift between a spec.md and its checklist is an ADVISORY, not a
failure: the loader now reads both shapes, so a lagging checklist no longer
hides requirements from the matrix. It is still worth seeing — SPEC-110's
checklist asserts "29/29 requirements implemented" against a spec.md defining
51 — but it does not break traceability, and a gate that fails on documentation
cadence gets switched off, which costs more than the drift it catches.

Exit codes: 0 pass · 1 violations · 2 INVALID. A run that examined zero
requirement ids or zero citations reports 2, never 0.
"""

from __future__ import annotations

import argparse
import sys
from dataclasses import dataclass
from pathlib import Path

_repo_root = Path(__file__).resolve().parent.parent
if str(_repo_root) not in sys.path:
    sys.path.insert(0, str(_repo_root))

from scripts.generate_rtm_evidence import (  # noqa: E402
    harvest_src_declarations,
    load_requirements,
    requirement_ids_in,
    requirement_source_files,
)

REPO_ROOT = _repo_root

EXIT_PASS = 0
EXIT_VIOLATIONS = 1
EXIT_INVALID = 2

# Directories under specs/ that are not specs and define no requirements.
NOT_SPECS: frozenset[str] = frozenset({"codegen", "spec-review"})


@dataclass(frozen=True)
class CorpusFacts:
    ratified: dict[str, str]  # id -> defining file
    universe: frozenset[str]  # ids load_requirements() returns
    cited: dict[str, list[dict]]  # id -> src/ @traces sites citing it

    @property
    def dangling(self) -> dict[str, list[dict]]:
        """src/ citations of an id the RTM cannot grade."""
        return {r: s for r, s in self.cited.items() if r not in self.universe}


def collect(repo_root: Path | None = None) -> CorpusFacts:
    """The ratified corpus, the RTM's universe, and what src/ actually cites.

    Gathered by INDEPENDENT means on purpose — see the module docstring on why
    comparing the universe against its own source function proves nothing.
    """
    root = repo_root or REPO_ROOT
    specs_dir = root / "specs"
    src_dir = root / "src"

    ratified: dict[str, str] = {}
    if specs_dir.is_dir():
        for source in requirement_source_files(specs_dir):
            try:
                text = source.read_text(encoding="utf-8")
            except OSError, UnicodeDecodeError:
                continue
            for rid in requirement_ids_in(text):
                ratified.setdefault(rid, str(source.relative_to(root)))

    universe = (
        frozenset(load_requirements(specs_dir)) if specs_dir.is_dir() else frozenset()
    )
    cited = dict(harvest_src_declarations(src_dir)) if src_dir.is_dir() else {}
    return CorpusFacts(ratified=ratified, universe=universe, cited=cited)


def check(repo_root: Path | None = None) -> tuple[list[str], int, int]:
    """(violations, requirement_ids_examined, distinct_src_citations_examined)."""
    facts = collect(repo_root)
    violations: list[str] = []

    root = repo_root or REPO_ROOT
    by_file: dict[str, set[str]] = {}
    for rid, sites in facts.dangling.items():
        for site in sites:
            raw = Path(str(site.get("file", "?")))
            # Report repo-relative: an absolute path per line makes 29
            # violations unreadable, and the reader needs the path they would
            # actually open.
            try:
                shown_path = str(raw.relative_to(root))
            except ValueError:
                shown_path = str(raw)
            by_file.setdefault(shown_path, set()).add(rid)

    for path, ids in sorted(by_file.items()):
        uniq = sorted(ids)
        shown = ", ".join(uniq[:6])
        more = f" (+{len(uniq) - 6} more)" if len(uniq) > 6 else ""
        violations.append(
            f"DANGLING {path}: @traces cites {len(uniq)} requirement id(s) the "
            f"RTM cannot grade — {shown}{more}"
        )

    return violations, len(facts.ratified), len(facts.cited)


def checklist_advisories(repo_root: Path | None = None) -> list[str]:
    """spec.md/checklist documentation drift — reported, never failing."""
    root = repo_root or REPO_ROOT
    specs_dir = root / "specs"
    out: list[str] = []
    if not specs_dir.is_dir():
        return out
    for spec_dir in sorted(p for p in specs_dir.iterdir() if p.is_dir()):
        if spec_dir.name in NOT_SPECS:
            continue
        spec_md = spec_dir / "spec.md"
        checklist = spec_dir / "checklists" / "requirements.md"
        if not spec_md.is_file() or not checklist.is_file():
            continue
        try:
            spec_ids = set(requirement_ids_in(spec_md.read_text(encoding="utf-8")))
            list_ids = set(requirement_ids_in(checklist.read_text(encoding="utf-8")))
        except OSError, UnicodeDecodeError:
            continue
        missing = sorted(spec_ids - list_ids)
        if missing:
            shown = ", ".join(missing[:6])
            more = f" (+{len(missing) - 6} more)" if len(missing) > 6 else ""
            out.append(
                f"{spec_dir.name}: checklist omits {len(missing)} id(s) its "
                f"spec.md defines — {shown}{more}"
            )
    return out


def main(argv: list[str] | None = None) -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument(
        "--advisories",
        action="store_true",
        help="Also print spec.md/checklist documentation drift",
    )
    args = ap.parse_args(argv)

    violations, ids_examined, citations_examined = check()

    if args.advisories:
        for a in checklist_advisories():
            print(f"[requirement-corpus] advisory — {a}")

    if ids_examined == 0 or citations_examined == 0:
        print(
            "[requirement-corpus] INVALID — examined "
            f"{ids_examined} requirement id(s) and {citations_examined} "
            "src/ citation(s). A pass with nothing examined is not a pass.",
            file=sys.stderr,
        )
        return EXIT_INVALID

    if violations:
        print(
            f"[requirement-corpus] FAIL — {len(violations)} file(s) cite "
            f"requirements the RTM cannot grade "
            f"({ids_examined} id(s) / {citations_examined} citation(s) examined):",
            file=sys.stderr,
        )
        for v in violations:
            print(f"  {v}", file=sys.stderr)
        return EXIT_VIOLATIONS

    print(
        f"[requirement-corpus] OK — {citations_examined} distinct requirement "
        f"id(s) cited by src/, all gradeable against the "
        f"{ids_examined}-id ratified corpus"
    )
    return EXIT_PASS


if __name__ == "__main__":
    raise SystemExit(main())