scripts.check_req_format

Reference › scripts › check_req_format

The requirement-id format gate: one canonical grammar for requirement ids, enforced permanently. tests/scripts/test_check_req_format.py imports it, and it uses the requirement parser in requirements_parser. A gate or a human runs it.

Invocation: main takes --roots (what to scan; defaults are built in), --max (fail if the number of distinct non-canonical ids exceeds this ratchet floor; default 0), --list to list non-canonical ids, --json and --specs-root (default <repo>/specs). The repository root comes from the ORCHESTRATA_ROOT variable, else the current directory.

Three checks run. First, scan counts ids in the roots and is_canonical tests each against the grammar. Second, uniqueness: find_definitions collects where each id is defined, in each spec's own requirements file only (iter_spec_requirement_files), and unbaselined_collisions reports ids defined in more than one spec, beyond a baseline. Third, story existence: check_story_existence verifies that the story an id refers to exists in its spec (story_idx, spec_dir_for_number), and unbaselined_story_violations filters out baselined ones. count_legacy_story_forms counts the old story spellings.

At a glance

Class diagram

No classes; see Functions.

Dependency diagram

API

Constants

Functions

Source

Permalink: https://gitlab.com/MSamy/orchestrata-ultimate-architecture/-/blob/a4cbc2bdf/scripts/check_req_format.py

"""Requirement-id format gate — the ONE canonical GRAMMAR, enforced forever.

Canonical requirement ids follow the structured grammar documented in
``docs/req-id-grammar.md`` (ratified 2026-08-02). This is NOT a single flat
shape: it admits the spec-scoped forms (``REQ-001-001``, ``REQ-002-US-001-001``,
``REQ-033-C1`` …), the subsystem-tag forms (``REQ-PS-001`` …), and the
cluster/task forms — the conventions the corpus and SPEC-019 actually use.
Anything OUTSIDE that grammar (``REQ-001-001-001``, prose-leaked ids, truncated
segments) is drift this gate retires. Run as a ratchet (``--max N``) while the
backlog burns down, then pin ``--max 0`` so drift can never re-enter.

Two exemptions (mirroring the existing ``FR-NNN``/``US-NNN`` section-label one),
both defined in ``docs/req-id-grammar.md``:

* **Grammar placeholders** — literal ``NNN``/``XXX``/``FRN``/``USn``/``0NN``
  tokens used in docstrings/tables to DESCRIBE the shape (``REQ-NNN-USn-NNN``).
  Prose about the format, not a citation.
* **Negative-test sentinels** — deliberately-malformed ids that exist only to
  prove the validator/contract REJECTS bad input; renaming them into valid ids
  would destroy the rejection test. Explicit allowlist below.

In addition to the grammar ceiling, this gate also asserts **cross-spec
uniqueness**: every canonical REQ id must be claimed by exactly one spec's
``requirements.md`` (see the ``UNIQUENESS_BASELINE`` section below for the
exact definition and exemptions). A future spec re-introducing an
already-claimed id fails the gate.

This script is the declared mechanism for **RUL-031** (requirement ids have
exactly one spelling): the legacy story form is not tolerated, not
ratcheted-down-over-time and not baselined — it FAILS. The ruling page is
``rulings/RUL-031-requirement-ids-have-one-spelling`` in knowledge-wiki, and
``scripts/rulings_lint.py`` verifies that pairing by requiring the literal
``RUL-031`` to appear in this file. Deleting this paragraph therefore breaks
the rulings lint rather than silently unhooking the ruling.
"""

from __future__ import annotations

import argparse
import os
import re
import sys
from collections import Counter
from pathlib import Path

# Imported FROM the installed package, never defined here and never imported
# the other way round. ``scripts/`` is not part of the distribution, so a
# ``src -> scripts`` import only resolves when the repo root happens to be on
# sys.path; running any script directly puts ``scripts/`` there instead and the
# package stops importing entirely. Re-exported so ``crf.find_story_headings``
# stays the gate's public surface for its own tests.
from orchestrata_ultimate.openspec.requirements_parser import find_story_headings

# ── The canonical grammar (closed allowlist — see docs/req-id-grammar.md) ──────
_SPEC = r"\d{3}"
_SEQ = r"\d{3}"
_IDX = r"\d{1,3}"
_SEG = r"(?:US|FR|AC|NFR|SEC|PERF|INTG|SUCC|EXT)"  # closed typed-segment vocab
# Same vocab MINUS ``US``. The un-dashed indexed form (``FR1``, ``AC2``) stays
# legal for every other segment, but a story id must be written ``US-001``:
# admitting the un-dashed story spelling here is what let the legacy form
# survive alongside the canonical one for months. The corpus was migrated
# 2026-08-19 (47,715 substitutions across 388 files); this makes the legacy
# shape fail the GRAMMAR, so a reintroduced ``REQ-NNN-USn-NNN`` is
# non-canonical and trips the --max 0 ceiling instead of being silently
# blessed. Written as a PLACEHOLDER, never a literal: a literal legacy id in
# a comment is itself a corpus violation, caught here by
# tests/scripts/test_migrate_legacy_story_ids.py on the very commit that
# introduced this line.
_SEG_NO_US = r"(?:FR|AC|NFR|SEC|PERF|INTG|SUCC|EXT)"
_TAG = r"(?:PS|INT|KG|MA|SM|DOR|PG|DEC|CLI|LIVE|GATE|STR)"  # closed subsystem vocab

CANONICAL = re.compile(
    r"^REQ-(?:"
    + "|".join(
        [
            rf"{_SPEC}-{_SEQ}",  # flat             REQ-001-001
            rf"{_SPEC}-{_SEG_NO_US}{_IDX}-{_SEQ}",  # typed indexed  REQ-002-FR1-001
            rf"{_SPEC}-{_SEG}-{_SEQ}",  # typed un-indexed REQ-012-NFR-001
            rf"{_SPEC}-US-{_IDX}-{_SEQ}",  # us dash-variant  REQ-022-US-001-001
            rf"{_SPEC}-{_SEG}{_IDX}",  # spec-level ref   REQ-007-FR2
            rf"{_SPEC}-{_SEG}",  # segment-family ref REQ-007-FR (prose REQ-007-FR*)
            rf"{_SPEC}-C{_IDX}-C{_IDX}",  # cluster range    REQ-042-C3-C5
            rf"{_SPEC}-C{_IDX}",  # cluster          REQ-033-C1
            rf"{_SPEC}-T{_IDX}",  # task cluster     REQ-033-T1
            rf"{_TAG}-{_SEQ}[A-Z]?",  # subsystem (+var) REQ-PS-001 / REQ-DEC-003B
            r"DOR-B\d{1,3}[A-Z]?",  # dor build        REQ-DOR-B15A
            rf"C{_IDX}-{_SEQ}",  # cluster-first    REQ-C4-100
        ]
    )
    + r")$"
)

# Any REQ-token: REQ- followed by one-or-more dash-separated alnum groups.
REQ_TOKEN = re.compile(r"REQ-[A-Za-z0-9]+(?:-[A-Za-z0-9]+)+")

# A requirement DECLARATION, anchored to where a registry line declares its id:
# at the START of a list item (with or without a checkbox), a table cell, or a
# markdown heading. Everything after that position is description prose, which
# may legitimately cite OTHER requirements.
#
# This distinction is the whole point. `iter_spec_requirement_files` already
# excludes spec.md / tasks.md / design.md because they are "cross-reference
# prose" -- but the same prose also lives inside a requirements.md entry's own
# description, and scanning the file WHOLE counted every citation there as a
# claim by the containing spec. Measured 2026-08-23 across all 59 registry
# files, that misread 5 tokens, one of which blocked every push:
#
#   REQ-000-004  cited 3x in SPEC-114's REQ-114-FR4-002 description as the
#                untraced example its why-not derivation was demonstrated on,
#                and declared for real at
#                specs/000-orchestrata-cli/checklists/requirements.md:8.
#                Reported as a collision between 000 and 114.
#   REQ-GATE-010 cited in the same file by REQ-114-FR1-001.
#   REQ-006-031  cited by SPEC-003's REQ-003-US-004-102 (a SPEC-001 addendum).
#   REQ-024-033 / REQ-024-044  discussed in parenthetical prose in SPEC-024's
#                registry and never declared anywhere.
#
# The checkbox class is `[^\]]?` rather than `[ xX]` deliberately: SPEC-110
# marks a superseded requirement `- [~] **REQ-110-107**`, and a narrower class
# drops a genuine declaration -- measured, before this comment was written.
#
# The alternative, baselining REQ-000-004 in UNIQUENESS_BASELINE, would have
# amended toward passing: it hides that the extractor cannot tell a citation
# from a claim, and the next spec to cite another spec's id in a description
# hits the same wall.
# The leading marker is OPTIONAL: a registry may also declare an id as the bare
# first token on its own line. What the anchor actually asserts is POSITION --
# the id opens the line -- not that any particular markdown decoration is
# present. Every one of the five misread tokens above sits MID-line, after a
# colon or inside a parenthetical, so none of them can reach this position.
DECLARATION = re.compile(
    r"^\s*(?:[-*]\s*(?:\[[^\]]?\]\s*)?|\|\s*|#+\s*)?"
    r"\*{0,2}(REQ-[A-Za-z0-9]+(?:-[A-Za-z0-9]+)+)\*{0,2}"
)

# Grammar-placeholder tokens (documentation about the shape, not real ids):
# a component is a literal placeholder — NNN / XXX / FRN / <letter>n / <digit>NN.
PLACEHOLDER = re.compile(r"(?:NNN|XXX|FRN|[A-Za-z]n|\dNN)")

# Negative-test sentinels — deliberately-malformed ids proving REJECTION.
# See docs/req-id-grammar.md. Never rewritten, never counted.
NEGATIVE_FIXTURES = frozenset(
    {
        # Rejection fixtures — fed to _validate_req_id / contract validators
        # expecting REJECTION; must stay malformed or the test is defanged.
        "REQ-999-X",
        "REQ-999-ZZ",
        "REQ-999-ZZZ",
        "REQ-999-ABCDE-001",
        # tests/scripts/test_check_feature_loss.py feeds this to the
        # feature-loss gate as a citation naming a spec that does not exist,
        # proving the gate reports it rather than resolving it. Spec 999 is
        # the established never-real sentinel above.
        "REQ-999-NOPE",
        "REQ-033-NONE",
        "REQ-033-C",  # cluster id without a digit — rejection fixture
        "REQ-033-AST-001",  # non-vocab segment — rejection fixture
        "REQ-NOT-THERE",
        "REQ-X-001",
        "REQ-1-001",  # 1-digit spec — rejection fixture
        "REQ-1-1",  # 1-digit spec AND 1-digit seq — rejection fixture
        "REQ-33-1",  # 2-digit spec / 1-digit seq — rejection fixture
        "REQ-001-A-001",
        "REQ-001-001-001",
        "REQ-001-001-001-001",
        "REQ-001-001-002",
        "REQ-001-001-01",
        "REQ-001-01",
        "REQ-001-US1",
        # Scenario fixtures — intentional test-only ids that are non-canonical
        # by design (unrelated/messy-id scenarios); not real citations.
        "REQ-DEC-999-UNRELATED",
        "REQ-DOR-CLI",
        "REQ-023-Exit-Criteria-for-Th-001",
        # tests/scripts/test_check_feature_loss.py fixtures — the
        # feature-loss gate's scenarios need requirement ids that are
        # DISTINCT from each other in a reader-legible way ("BAR", "KEEP",
        # "STALE", "DUAL"); the closed typed-segment vocab in CANONICAL
        # above has no room for that, so these are deliberately
        # non-canonical, never real citations.
        "REQ-001-A-002",
        "REQ-001-DUAL-001",
        "REQ-002-BAR-001",
        "REQ-003-KEEP-001",
        "REQ-999-STALE-001",
        # tests/scripts/test_history_intel.py — a synthetic "toy" loss event
        # for the temporal-history Prolog layer. TOY is deliberately outside
        # the closed segment vocab so it can never collide with a real spec.
        "REQ-TOY-001",
        # tests/prolog/test_requirement_whynot.py — the two ids that prove
        # `tool_why_not_req/2` distinguishes "never heard of it" from "known
        # but uncovered" (SPEC-114 FR-004). Both must stay OUTSIDE the
        # grammar: an id that parses could one day name a real requirement,
        # and then the test would assert `unknown_requirement` about
        # something the corpus knows — defanging exactly the confusion it
        # was written to prevent.
        #
        # Uncounted here since 2026-08-23. They were reported as the gate's
        # only two non-canonical ids, failing it at `--max 0` on an
        # otherwise-clean tree; measured with the fix stashed, so the failure
        # predates and is independent of the DECLARATION change above.
        "REQ-does-not-exist",  # asserted to answer error: unknown_requirement
        "REQ-never-extracted",  # asserted to be REFUSED over a stale base
    }
)

DEFAULT_ROOTS = ("specs", "src/orchestrata_ultimate", "tests")
SCAN_SUFFIXES = {".md", ".py", ".pl", ".txt", ".yaml", ".yml"}


def is_canonical(tok: str) -> bool:
    """A token conforms if it matches the grammar, is a shape-placeholder, or is
    an allowlisted negative-test sentinel."""
    return bool(
        CANONICAL.match(tok) or PLACEHOLDER.search(tok) or tok in NEGATIVE_FIXTURES
    )


def iter_files(roots: list[Path]) -> list[Path]:
    files: list[Path] = []
    for root in roots:
        if root.is_file():
            files.append(root)
        elif root.is_dir():
            files.extend(
                p for p in root.rglob("*") if p.is_file() and p.suffix in SCAN_SUFFIXES
            )
    return files


def scan(roots: list[Path]) -> tuple[Counter[str], dict[str, set[str]]]:
    """Return (non-canonical id -> occurrence count, id -> set of files).

    Skips ``LEGACY_FIXTURE_CARRIERS``: the migrator's own test file holds
    genuine legacy ids on purpose, and counting them here would make the very
    test that proves legacy is caught into a permanent gate failure.
    """
    counts: Counter[str] = Counter()
    where: dict[str, set[str]] = {}
    for path in iter_files(roots):
        if any(str(path).endswith(carrier) for carrier in LEGACY_FIXTURE_CARRIERS):
            continue
        try:
            text = path.read_text(encoding="utf-8", errors="ignore")
        except OSError:
            continue
        for tok in REQ_TOKEN.findall(text):
            if not is_canonical(tok):
                counts[tok] += 1
                where.setdefault(tok, set()).add(str(path))
    return counts, where


# ── Cross-spec REQ-id uniqueness (closes 480/481, the "each id belongs to
# exactly one spec" half of the grammar work) ───────────────────────────────
# A REQ id is DEFINED by spec S iff the canonical token appears anywhere in
# EITHER of spec S's own requirements-registry files:
#   * ``specs/<S>/requirements.md``              (2 specs use this shape)
#   * ``specs/<S>/checklists/requirements.md``   (55 specs use this shape)
# Both attribute to the SAME spec — the checklist variant's parent-of-parent
# directory name. A spec that carries both files is still ONE spec: ids
# appearing in both of its own files are not a collision with each other.
# This is a narrower scan than ``scan()`` above: it does NOT look at
# spec.md/tasks.md/design.md (cross-reference prose, not a definition site).
# v1 does not try to infer a "definition position" within the file — any
# occurrence counts as a claim.
#
# The known borrowed-namespace specs (spec-003 using the 006 number space,
# spec-021 using 004's) are NOT collisions under this definition: each id
# still lives in exactly ONE spec's requirements registry. That mismatch is
# a separate, already-tracked defect this gate does not touch.

# Accepted cross-spec id collisions, each with a one-line justification for
# why the SAME id legitimately appears in more than one spec's requirements
# registry. Expected to shrink, never grow — and it is EMPTY: the one
# collision the 2026-08-16 measurement found (spec-021's REQ-021-016
# example text citing `REQ-004-US-001-012` — a prose citation the gate's
# position-blind v1 semantics can't tell from a definition) was fixed at
# the SOURCE by rewording the citation to the placeholder shape
# `REQ-004-US001-NNN`, which the strict-canonical matcher ignores. A
# registry file that must mention a foreign id gets the same treatment:
# placeholder the digits, don't grow this dict.
UNIQUENESS_BASELINE: dict[str, str] = {}


def _spec_name_for_requirements_file(path: Path) -> str:
    """The owning spec directory name for either requirements-registry shape:
    ``<spec>/requirements.md`` -> ``<spec>``; ``<spec>/checklists/requirements.md``
    -> ``<spec>`` (parent-of-parent, since the immediate parent is "checklists")."""
    return (
        path.parent.parent.name
        if path.parent.name == "checklists"
        else path.parent.name
    )


def iter_spec_requirement_files(specs_root: Path) -> list[Path]:
    """BOTH requirements-registry shapes a spec may use — its own
    ``<specs_root>/<spec>/requirements.md`` AND/OR
    ``<specs_root>/<spec>/checklists/requirements.md`` — NOT spec.md,
    tasks.md, or design.md (cross-reference prose)."""
    if not specs_root.is_dir():
        return []
    direct = specs_root.glob("*/requirements.md")
    checklist = specs_root.glob("*/checklists/requirements.md")
    return sorted({*direct, *checklist})


def find_definitions(specs_root: Path) -> dict[str, set[str]]:
    """Map each CANONICAL req id to the set of spec directory names whose own
    requirements registry (either shape) defines it. Placeholders and
    negative-test sentinels are documentation/fixtures, never real
    definitions, so only tokens matching the strict ``CANONICAL`` grammar
    (not the looser ``is_canonical``) count."""
    definitions: dict[str, set[str]] = {}
    for path in iter_spec_requirement_files(specs_root):
        try:
            text = path.read_text(encoding="utf-8", errors="ignore")
        except OSError:
            continue
        spec_name = _spec_name_for_requirements_file(path)
        # Line by line, and only the id in DECLARATION position. Scanning the
        # whole file counted every id CITED in an entry's description as a
        # claim by the containing spec -- see DECLARATION's own comment for the
        # five tokens that misread and the push it blocked.
        for line in text.splitlines():
            match = DECLARATION.match(line)
            if match is None:
                continue
            tok = match.group(1)
            if CANONICAL.match(tok):
                definitions.setdefault(tok, set()).add(spec_name)
    return definitions


def find_uniqueness_collisions(specs_root: Path) -> dict[str, list[str]]:
    """Every canonical req id claimed by 2+ specs' requirements.md, unfiltered
    by the baseline — id -> sorted list of claiming spec directory names."""
    return {
        tok: sorted(specs)
        for tok, specs in find_definitions(specs_root).items()
        if len(specs) > 1
    }


def unbaselined_collisions(
    specs_root: Path, baseline: dict[str, str] | None = None
) -> dict[str, list[str]]:
    """Cross-spec collisions not covered by ``baseline`` (default:
    ``UNIQUENESS_BASELINE``) — what the gate actually fails on."""
    active_baseline = UNIQUENESS_BASELINE if baseline is None else baseline
    return {
        tok: specs
        for tok, specs in find_uniqueness_collisions(specs_root).items()
        if tok not in active_baseline
    }


# ── Story existence (2026-08-19: a story-segment mismatch cost real
# investigation time because nothing checked this) ─────────────────────────
# check_req_format only ever validated SHAPE: REQ-003-US-004-009 was accepted
# whether or not story US-004 actually exists in spec 003. Two forms cite a
# story -- unpadded/unhyphenated (REQ-002-US-001-001) and padded/hyphenated
# (REQ-022-US-001-001, matching every ``### US-NNN`` spec.md heading) -- and
# both must resolve to the SAME real heading in that id's own spec. ``US4``
# and ``US-004`` normalize to the same integer; that equivalence is the
# whole trick.
_STORY_ID = re.compile(r"^REQ-(?P<spec>\d{3})-US-?(?P<idx>\d{1,3})(?:-\d{3})?$")


def story_idx(tok: str) -> tuple[str, int] | None:
    """``(spec, idx)`` for a canonical REQ id that names a story, else
    ``None`` (not a story-segment id -- e.g. an FR/NFR id, or a bare
    segment-family reference with no index). ``US4`` and ``US-004`` both
    normalize to the same ``idx`` int."""
    m = _STORY_ID.match(tok)
    return (m.group("spec"), int(m.group("idx"))) if m else None


def spec_dir_for_number(specs_root: Path, spec_num: str) -> Path | None:
    """The single ``specs_root/<spec_num>-*`` directory, or ``None`` if zero
    or more than one match (ambiguous/absent spec -- "cannot determine")."""
    if not specs_root.is_dir():
        return None
    dirs = [p for p in specs_root.glob(f"{spec_num}-*") if p.is_dir()]
    return dirs[0] if len(dirs) == 1 else None


def check_story_existence(
    roots: list[Path], specs_root: Path
) -> tuple[dict[str, tuple[str, str]], int]:
    """Verify every canonical story-segment REQ id cited anywhere in
    ``roots`` against its OWN spec's real ``### US-NNN`` headings.

    Returns ``(violations, undetermined)``:

    * ``violations`` -- id -> ``(spec.md path, comma-joined stories that DO
      exist there)`` for an id naming a story with no matching heading.
    * ``undetermined`` -- count of story ids whose owning spec dir/spec.md
      could not be resolved, or whose spec.md has zero US headings (an
      FR-only spec) -- explicitly NOT violations, just unmeasurable.

    Negative-test sentinels (``NEGATIVE_FIXTURES``) are excluded even though
    some coincidentally match the strict grammar shape (e.g.
    ``REQ-001-US1`` is also a valid spec-level story reference) -- they are
    deliberately-malformed fixtures for OTHER validators, never real
    citations, and must never be checked here."""
    violations: dict[str, tuple[str, str]] = {}
    undetermined = 0
    checked: set[str] = set()
    for path in iter_files(roots):
        try:
            text = path.read_text(encoding="utf-8", errors="ignore")
        except OSError:
            continue
        for tok in REQ_TOKEN.findall(text):
            if tok in checked or tok in NEGATIVE_FIXTURES:
                continue
            if not CANONICAL.match(tok):
                continue
            resolved = story_idx(tok)
            if resolved is None:
                continue
            checked.add(tok)
            spec_num, idx = resolved
            spec_dir = spec_dir_for_number(specs_root, spec_num)
            if spec_dir is None:
                undetermined += 1
                continue
            headings = find_story_headings(spec_dir / "spec.md")
            if not headings:
                undetermined += 1
                continue
            if idx not in headings:
                existing = ", ".join(f"US-{n:03d}" for n in sorted(headings))
                violations[tok] = (str(spec_dir / "spec.md"), existing)
    return violations, undetermined


# Known pre-existing story-existence violations, measured 2026-08-19 the day
# this check was built -- mirrors UNIQUENESS_BASELINE's role: visible,
# auditable, reasoned exceptions rather than a silent exemption. NOT fixed
# here (renaming ids / adding spec.md headings is an editorial call this
# gate does not make); each is a REAL open defect surfaced to the user.
# Expected to shrink, never grow.
STORY_EXISTENCE_BASELINE: dict[str, str] = {
    # specs/035-verification/spec.md's "User Scenarios & Stories" section
    # documents only US-001..US-007. Its OWN requirements registry
    # (specs/035-verification/checklists/requirements.md) fully defines
    # US-008..US-011 (acceptance criteria, tasks, verification steps), and
    # 14 src/ + 3 tests/ files already implement/cite them -- these are
    # real, implemented stories. spec.md is simply missing the four
    # "### US-NNN:" headings that would make it self-consistent.
    **{
        f"REQ-035-US8-{n:03d}": "US-008 real per checklist; spec.md missing the heading"
        for n in range(1, 22)
    },
    **{
        f"REQ-035-US9-{n:03d}": "US-009 real per checklist; spec.md missing the heading"
        for n in range(1, 19)
    },
    **{
        f"REQ-035-US10-{n:03d}": "US-010 real per checklist; spec.md missing the heading"
        for n in range(1, 22)
    },
    **{
        f"REQ-035-US11-{n:03d}": "US-011 real per checklist; spec.md missing the heading"
        for n in range(1, 20)
    },
    # tests/scripts/test_force_trace_intel.py:97 -- a shell-command STRING
    # FIXTURE (GREP_R_QUOTED_SUBJECT) demonstrating what grepping a
    # multi-id pattern looks like for the force-trace-intel.sh hook test;
    # not a real @traces citation or requirement. The example id happens to
    # be shape-valid but names a story spec 009 never had.
    "REQ-009-US-009-013": "test-fixture shell string in test_force_trace_intel.py, not a real citation",
}


def unbaselined_story_violations(
    roots: list[Path],
    specs_root: Path,
    baseline: dict[str, str] | None = None,
) -> dict[str, tuple[str, str]]:
    """Story-existence violations not covered by ``baseline`` (default
    ``STORY_EXISTENCE_BASELINE``) -- what the gate actually fails on."""
    active_baseline = STORY_EXISTENCE_BASELINE if baseline is None else baseline
    violations, _ = check_story_existence(roots, specs_root)
    return {tok: v for tok, v in violations.items() if tok not in active_baseline}


# ── Legacy-form reporting (grammar point B: US-NNN is canonical for NEW
# ids, USn stays accepted so nothing existing breaks; legacy usage is a
# COUNT, never a failure) ───────────────────────────────────────────────────
_LEGACY_STORY_ID = re.compile(r"^REQ-\d{3}-US\d")


# Three test files MUST hold genuine legacy ids as fixtures: a test that legacy
# is RECOGNISED (this gate's own tests) and tests that legacy is REWRITTEN
# (the migrator's unit and coverage suites). A detector cannot be tested
# without a positive example, so counting their fixtures here would turn the
# tests that prove legacy is caught into permanent gate failures.
#
# These three paths are the WHOLE exception list (docs/req-id-grammar.md,
# RUL-031). All are excluded here and in the migrator's EXCLUDED_GLOBS, so a
# fixture cannot be silently canonicalised out from under its own test.
LEGACY_FIXTURE_CARRIERS: frozenset[str] = frozenset(
    {
        "tests/scripts/test_migrate_legacy_story_ids.py",
        "tests/scripts/test_migrate_legacy_story_ids_coverage.py",
        "tests/scripts/test_check_req_format.py",
    }
)


def count_legacy_story_forms(roots: list[Path]) -> int:
    """Distinct REQ ids still using a legacy story form -- a HARD FAILURE.

    Legacy is ``US`` followed immediately by digits (``US1``, ``US10``,
    ``US001``); canonical is ``US-001``. All three legacy widths were
    eliminated 2026-08-19 (47,715 substitutions across 388 files) and
    **legacy is no longer accepted** -- Mohamed's ruling, superseding the
    earlier "informational, never rewritten" tolerance.

    Deliberately NOT gated on ``CANONICAL.match``: the canonical grammar now
    REJECTS the legacy story form, so requiring both would make this counter
    structurally unable to fire -- a dead check reporting zero forever. It
    counts the legacy shape directly, which is the thing being forbidden.
    """
    ids: set[str] = set()
    for path in iter_files(roots):
        if any(str(path).endswith(carrier) for carrier in LEGACY_FIXTURE_CARRIERS):
            continue
        try:
            text = path.read_text(encoding="utf-8", errors="ignore")
        except OSError:
            continue
        for tok in REQ_TOKEN.findall(text):
            if _LEGACY_STORY_ID.match(tok) and tok not in NEGATIVE_FIXTURES:
                ids.add(tok)
    return len(ids)


def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument(
        "--roots",
        nargs="*",
        default=None,
        help=f"Roots to scan (default: {' '.join(DEFAULT_ROOTS)})",
    )
    ap.add_argument(
        "--max",
        type=int,
        default=0,
        help="Fail if the DISTINCT non-canonical id count exceeds this "
        "(ratchet floor; pin 0 once normalization completes).",
    )
    ap.add_argument("--list", action="store_true", help="List non-canonical ids.")
    ap.add_argument("--json", action="store_true", help="JSON output.")
    ap.add_argument(
        "--specs-root",
        default=None,
        help="Root containing spec dirs for the cross-spec REQ-id uniqueness "
        "scan (default: <repo>/specs; each spec's OWN requirements.md only).",
    )
    args = ap.parse_args()

    root_names = args.roots if args.roots is not None else list(DEFAULT_ROOTS)
    repo = Path(os.environ.get("ORCHESTRATA_ROOT", ".")).resolve()
    roots = [(repo / r) if not Path(r).is_absolute() else Path(r) for r in root_names]

    counts, where = scan(roots)
    distinct = len(counts)
    total = sum(counts.values())

    specs_root = (
        Path(args.specs_root).resolve()
        if args.specs_root is not None
        else repo / "specs"
    )
    collisions = unbaselined_collisions(specs_root)
    collision_count = len(collisions)

    story_violations = unbaselined_story_violations(roots, specs_root)
    story_violation_count = len(story_violations)
    legacy_story_count = count_legacy_story_forms(roots)

    if args.json:
        import json

        print(
            json.dumps(
                {
                    "distinct_non_canonical": distinct,
                    "total_occurrences": total,
                    "max": args.max,
                    "ids": sorted(counts) if args.list else None,
                    "cross_spec_collision_count": collision_count,
                    "cross_spec_collisions": collisions if args.list else None,
                    "story_existence_violation_count": story_violation_count,
                    "story_existence_violations": (
                        {
                            rid: {"spec_md": spec_md, "existing_stories": existing}
                            for rid, (spec_md, existing) in story_violations.items()
                        }
                        if args.list
                        else None
                    ),
                    "legacy_story_form_count": legacy_story_count,
                }
            )
        )
    else:
        print(
            f"Requirement-id format gate: {distinct} distinct non-canonical ids "
            f"({total} occurrences); ceiling --max {args.max}."
        )
        if args.list:
            for rid in sorted(counts):
                sample = sorted(where[rid])[0]
                print(f"  {rid}  ({counts[rid]}x)  e.g. {sample}")
        print(
            f"Cross-spec REQ-id uniqueness: {collision_count} unbaselined "
            f"collision(s) across specs/*/requirements.md."
        )
        if args.list:
            for rid in sorted(collisions):
                print(f"  {rid}  claimed by {', '.join(collisions[rid])}")
        print(
            f"Story existence: {story_violation_count} unbaselined req-id(s) "
            f"naming a story that does not exist in their own spec's spec.md."
        )
        if args.list:
            for rid in sorted(story_violations):
                spec_md, existing = story_violations[rid]
                print(f"  {rid}  spec={spec_md}  exists=[{existing or 'none'}]")
        print(
            f"Legacy story-id form (US followed directly by digits, rather "
            f"than the canonical US-NNN): {legacy_story_count} distinct id(s) "
            f"-- must be 0."
        )

    ok = True
    if legacy_story_count:
        print(
            f"FAIL: {legacy_story_count} distinct legacy story-id(s). The "
            f"legacy form is NOT accepted (docs/req-id-grammar.md, ruling "
            f"2026-08-19). Fix with "
            f"`uv run python scripts/migrate_legacy_story_ids.py --apply` -- "
            f"never by relaxing this check.",
            file=sys.stderr,
        )
        ok = False
    if distinct > args.max:
        print(
            f"FAIL: {distinct} distinct non-canonical req-ids exceeds the "
            f"allowed {args.max}. Canonical grammar: docs/req-id-grammar.md.",
            file=sys.stderr,
        )
        ok = False
    if collisions:
        ids_desc = "; ".join(
            f"{rid} claimed by {', '.join(specs)}"
            for rid, specs in sorted(collisions.items())
        )
        print(
            f"FAIL: {collision_count} canonical req-id(s) claimed by more than "
            f"one spec's requirements.md: {ids_desc}.",
            file=sys.stderr,
        )
        ok = False
    if story_violations:
        ids_desc = "; ".join(
            f"{rid} (spec {spec_md}; stories that DO exist: [{existing or 'none'}])"
            for rid, (spec_md, existing) in sorted(story_violations.items())
        )
        print(
            f"FAIL: {story_violation_count} req-id(s) name a story that does "
            f"not exist in their own spec's spec.md: {ids_desc}.",
            file=sys.stderr,
        )
        ok = False
    if not ok:
        return 1
    print("PASS: requirement-id format within ceiling.", file=sys.stderr)
    return 0


if __name__ == "__main__":
    raise SystemExit(main())