scripts.check_feature_loss

Reference › scripts › check_feature_loss

The feature-loss gate: a requirement (REQ-*) must never lose its implementation silently. It compares what the ratified specs promise with what the source and tests implement now, and with what git history shows was deleted. It is one of the largest scripts, at about 1,800 lines. tests/scripts/test_check_feature_loss.py and tests/scripts/test_check_feature_loss_corpus.py import it, and it reuses the trace scanner from scanner and the RTM evidence harvest in generate_rtm_evidence.

Invocation: main takes --repo-root (default the current directory) and --json for the full lists. The exit code is the pass code, a fail code when any loss is found, or an invalid code when the report status is invalid.

run_check builds a FeatureLossReport. collect_ratified reads the ratified requirements from the specs directory, and collect_current_implementation gathers the current @traces declarations. find_deletions and find_deleted_symbols use git history (_git, _git_show) to find symbols that carried a requirement and have since gone, producing Loss records; derive_orphaned and derive_never_implemented classify the outcome; and find_deleted_spec_docs and find_surface_losses cover deleted spec documents and public surface such as dataclass fields. A loss can be cleared only by recorded reclassification evidence (load_reclassification_evidence, load_surface_reclassification_evidence), which is validated record by record. render_report produces the text.

At a glance

Class diagram

Dependency diagram

API

Classes

DeletionEvent (dataclass)

Decorators: dataclass(frozen=True) · Line 483

One git commit that deleted path (rename-aware via -M).

Loss (dataclass)

Decorators: dataclass(frozen=True) · Line 492

A confirmed feature loss — always fails the gate.

NeverImplemented (dataclass)

Decorators: dataclass(frozen=True) · Line 505

INFO only — never affects the exit code.

SurfaceLoss (dataclass)

Decorators: dataclass(frozen=True) · Line 514

A candidate PUBLIC-SURFACE loss — HEURISTIC, not requirement- evidenced (module docstring's SURFACE-LOSS section).

FeatureLossReport (dataclass)

Decorators: dataclass(frozen=True) · Line 531

ReclassificationEvidence (dataclass)

Decorators: dataclass(frozen=True) · Line 561

Parsed, validated contents of plans/feature-loss-invalid-citations. json — see the module docstring's RECLASSIFICATION section.

SurfaceReclassificationEvidence (dataclass)

Decorators: dataclass(frozen=True) · Line 580

Parsed, validated contents of the surface_records key in plans/feature-loss-invalid-citations.json — the SURFACE-LOSS sibling of ReclassificationEvidence above (module docstring's…

Functions

Source

Permalink: https://gitlab.com/MSamy/orchestrata-ultimate-architecture/-/blob/a4cbc2bdf/scripts/check_feature_loss.py

#!/usr/bin/env python3
"""Feature-loss gate — a REQ-* must never lose its implementation silently.

User ruling, 2026-08-19, verbatim: "add checks at the gates so we never face
the lost code and features again and make them non negotiable, make them all
reference the specs and report loudly any deleted features and loss."

TWO REAL LOSSES MOTIVATE THIS GATE, neither of which any existing gate would
have caught:

* Commit ``6a186c9d`` (2026-07-09) deleted ``scripts/ast_serializer/testgen/``
  (9 files, 2,392 lines) on an audit that mistook "no importer" for "no
  consumer" — for a GENERATOR the consumers are its OUTPUTS. That incident is
  the domain of :mod:`scripts.check_generated_artifacts` (RUL-030), which is
  being extended with the GENERATOR-provenance half in parallel with this
  gate — see its module docstring's "THE OTHER DIRECTION" section. This gate
  is the OTHER half: the SPEC/REQUIREMENT half, for domain code carrying
  ``@traces(reqs=[...])`` under ``src/orchestrata_ultimate/`` rather than
  ``scripts/`` generators. The two gates cross-reference each other on
  purpose; neither replaces the other.
* A dead-code triage of 101 unreferenced symbols found 37 UNWIRED, including
  ``core/env_bootstrap.py:66 declare_owned`` — RUL-023's territory
  (:mod:`scripts.check_wiring`), not this gate's: that incident is about code
  that exists but was never CONNECTED, not code that was DELETED after being
  connected. Reported here only as background for why the project now treats
  "quietly lost" as a first-class defect class.

WHAT THIS GATE DOES THAT NOTHING ELSE DOES: walks GIT HISTORY (not just the
current tree) for evidence that a requirement's implementation, or the
document that defines a requirement, USED TO EXIST and is now gone. Every
other traceability gate in this repo answers questions about the CURRENT
tree only:

* :mod:`scripts.check_requirement_corpus` (RUL-030's traceability cousin) —
  DANGLING: does ``src/`` cite a requirement id the RTM cannot grade RIGHT
  NOW, for ANY reason. It never asks *why* — a fabricated id and a genuinely
  deleted spec look identical to it. This gate's check 4 supplies the "why"
  for the subset of dangling citations caused by a deleted spec/ADR document,
  naming the deleting commit; check_requirement_corpus.py keeps owning the
  broader "dangling for any reason" sweep. **Not duplicated: complementary.**
* :mod:`scripts.check_spec_authority` (RUL-013) — a TEST citing a requirement
  id no spec defines. Scoped to ``tests/`` and to fabrication, not deletion.
  No overlap with this gate's src/spec-history scope.
* ``scripts/trace_intel.py untraced-reqs`` — an ADVISORY (non-failing) Prolog
  query over the CURRENT tree: which ratified requirements have zero
  ``@traces`` coverage right now. It does not consult git history at all, so
  it cannot distinguish "never implemented" from "implemented, then lost" —
  exactly the distinction the user asked this gate to make. This gate's
  check 2 (never-implemented) is the same shape of finding but git-verified
  and, unlike ``untraced-reqs``, feeds a real (if deliberately non-failing —
  see below) gate output rather than an advisory report nobody has to look
  at.
* ``scripts/history_intel.py {what,when,how,why,explain}`` — the companion this
  gate hands off to. This gate answers **whether** something was lost;
  ``history_intel`` answers **what / when / how / why** for a named symbol, as
  Prolog queries over recorded evidence rather than as a diff-reading exercise.
  When a finding here needs a rationale, run ``history_intel.py why <symbol>``
  instead of reconstructing one from memory — that reconstruction has already
  produced a confidently-wrong feature-loss claim once. Its
  ``no-recorded-rationale`` answers TRUE as a FINDING: "nobody wrote down why"
  is a fact it states, not a query that failed.
* ``scripts/generate_rtm_evidence.py`` grades the CURRENT tree with the
  strongest CURRENT evidence (TESTED/CITED/DECLARED/UNLINKED). Its docstring
  explicitly warns against treating a decorator as proof — this gate obeys
  that by never treating "declared" or "cited" as its own witness of a
  requirement's *history*; every finding here is anchored to an actual git
  commit that deleted something, or to a demonstrated absence of any such
  commit. This gate REUSES ``generate_rtm_evidence``'s current-state readers
  (``harvest_src_declarations``, ``harvest_test_citations``,
  ``harvest_web_declarations``, ``WEB_SOURCE_ROOTS``, ``requirement_ids_in``,
  ``requirement_source_files``) rather than reimplementing "what counts as an
  @traces citation" a second time — exactly the divergence that module's own
  docstring documents costing 68 requirement ids for months.

POLYGLOT SCAN SCOPE — 2026-08-19 fix, two real false positives motivating it.
Every "current implementation" check in this file used to mean "a ``.py``
symbol under ``src/orchestrata_ultimate/``" ONLY. That is a structural
blind spot in a polyglot repo, not a coincidence: a requirement whose only
implementation is TypeScript/JavaScript reads, to a Python-only scan, as
"no current implementation anywhere" — full confidence, completely wrong.
Two independent 2026-08-19 audits hit this by different routes on the same
run: REQ-021-195..198 (an OpenCode ``session-status`` custom tool, fully
implemented at ``opencode/plugins/tools/session-tools.ts:17-58`` with its
own ``// @traces REQ-021-195 REQ-021-196 REQ-021-197 REQ-021-198`` marker)
and REQ-016-290..294 (VizInt Command Center gadget requirements). The fix is
WIRING, not a new parser: ``current implementation`` is now every requirement
id ``harvest_src_declarations`` finds under ``src/orchestrata_ultimate/``
UNIONED with every id ``generate_rtm_evidence.harvest_web_declarations``
finds under each root in ``generate_rtm_evidence.WEB_SOURCE_ROOTS`` (today
``opencode``, ``vizint-ui``) — see :func:`collect_current_implementation`.
This is the SAME TS/JS ``@traces`` comment-tag harvester
``generate_rtm_evidence.py`` already uses to grade DECLARED; check 3
(DELETED-SYMBOL) and check 2 (NEVER-IMPLEMENTED) both key off this union, so
a requirement whose only surviving implementation is a TS/JS ``@traces`` tag
is neither reported as lost nor as never-built — it produces no finding at
all, which is what "correctly implemented and correctly traced" means.

A scan that silently covers only ONE language of a polyglot repo will
reproduce this exact false confidence the moment anyone adds another
language (Rust, Go, ...) this gate still cannot see. So every run discloses
its own scan scope in the report (a ``SCAN SCOPE`` line, and the
``scan_scope`` field under ``--json``) — naming Python plus every
``WEB_SOURCE_ROOTS`` entry, and for each, whether it was actually scanned
or is simply ABSENT from this tree ("not present -- not scanned"). Absent
and unscanned-for-some-other-reason are deliberately reported identically
here (there is no "language exists but tooling failed to reach it" case for
this gate today) but are never conflated with "scanned, found nothing" —
the two are different facts and a reader must be able to tell them apart.

THE FOUR CHECKS
----------------
1. ORPHANED-REQUIREMENT — a REQ-* still defined in the CURRENT spec corpus
   whose every historical implementation (a ``src/orchestrata_ultimate``
   symbol that once carried ``@traces(reqs=[...])`` citing it) has been
   deleted, and no surviving symbol covers it. Derived from (3) below,
   filtered to requirements still in the ratified corpus.
2. NEVER-IMPLEMENTED-REQUIREMENT — a REQ-* with zero current implementation
   AND no git evidence it ever had one (i.e. not found among (3)'s deleted
   symbols either). Reported separately per the brief: the remedy differs
   (implement it, or accept it as backlog) from an orphaned requirement's
   remedy (restore it, or get the user's sign-off to retire it — something
   USED to satisfy this requirement and stopped).
3. DELETED-SYMBOL — a ``src/orchestrata_ultimate/**/*.py`` file that no
   longer exists, whose last committed content carried an ``@traces`` symbol
   citing a requirement id, where NO currently-existing symbol declares that
   id. Reports the deleting commit sha and date — the exact information a
   2026-08-19 session spent hours reconstructing by hand for a single
   incident.
4. DELETED-SPEC — a ``specs/**`` or ``docs/adr/**`` document that no longer
   exists, whose last committed content defined a requirement id that is
   STILL cited by a currently-existing ``src/`` symbol or test. If nobody
   currently cites the id, this is ordinary spec housekeeping, not a loss,
   and is not reported.

WHY (2) NEVER FAILS THE GATE — a deliberate, disclosed precision decision.
``generate_rtm_evidence.py``'s own docstring calls UNLINKED "the honest
majority" of this corpus's 5,500+ requirement ids. Failing on every
requirement that was simply never built would make this gate permanently
red from the day it lands, for reasons that are pre-existing backlog, not
today's regression — exactly the "noisy gate gets bypassed" failure mode the
brief warns against. So (2) is printed, counted, and NEVER affects the exit
code. (1), (3) and (4) are git-EVIDENCED deletions — rare, specific,
actionable events — and DO fail the gate at zero tolerance, every time,
unconditionally.

SCOPE LIMIT, DISCLOSED RATHER THAN HIDDEN: this gate detects WHOLE-FILE
deletions (a ``.py``/``.md`` file that no longer exists on disk) via
``git log --diff-filter=D``, which is a single, fast, git-native query
regardless of history size (measured on this repo's 3,300+ commits: well
under a second). It does NOT detect a requirement's ``@traces`` decorator
being stripped from a function that OTHERWISE SURVIVES in a file that still
exists — that would require a full historical AST diff of every commit that
ever touched every surviving source file, which is neither git-cheap (no
native git plumbing surfaces "this call's keyword argument changed") nor
reviewable (a naive ``git log -G'@traces'`` pickaxe over full history
produces megabytes of patch text an agent must never read whole per this
project's context-safety rule, and would very likely be too noisy to trust —
see CONSTRAINTS in the originating brief). Both real historical incidents
this gate was written to catch (the motivating one above, and the fixture in
the test suite) are whole-file deletions, so this scope is not a hedge
against the cases that actually happened; it is a disclosed boundary against
a detection class this repo's own rules say not to attempt blind.

FAIL-CLOSED, DELIBERATELY THE OPPOSITE OF check_wiring.py's SKIP CONVENTION.
``check_wiring.py`` treats "not a git repo" / "git unavailable" as SKIP
(exit 0) because it grades a DIFF against master that legitimately may not
exist. This gate grades HISTORY, and "I could not read history" is not
"nothing to grade" — it is exactly the situation where a loss could be
hiding. Not-a-repo, git-unavailable, or an empty ratified corpus (nothing
examined) all return INVALID (exit 2), never PASS (exit 0). Unknown is not
clean.

NO WAIVER PATH — DELIBERATE, NOT AN OVERSIGHT. This repo has a `# gate-ok:`
waiver convention (``core/gate_waivers.py``, honoured by six other
checkers). This gate does NOT read it, does not import it, and never will:
a waivable loss-detector is how the loss happens — the exact failure mode
this gate exists to close. There is likewise no ``--max``/severity/env-var
downgrade anywhere in this file; the only flags are ``--repo-root`` (test
injection seam) and ``--json`` (machine output).

RECLASSIFICATION — EVIDENCE-GATED, NOT A WAIVER (2026-08-19 ruling). A
2026-08-19 investigation found that most of check 3's DELETED-SYMBOL
findings are not lost implementations at all: they are ``@traces``
citations that were INVALID WHEN WRITTEN, almost entirely from a 2026-04
bulk-annotation pass (a systematic FR-group off-by-one in
``postman_types.py``, a single LOGGING-SCHEMA requirement stamped onto all
30 functions of ``kg_sync.py``, TUI-widget requirements attached to backend
methods in ``pipeline_runner.py``, and 86 rows citing SPEC-009, whose own
text says "superseded"). Nothing was lost when those symbols were deleted
because the citation never described the symbol, and there is no correct
place to reattach it. Reporting these as LOSS is a misclassification, not a
true positive — but "misclassification" is not licence to guess: a citation
is reclassified out of LOSS and into the pre-existing, non-failing
NEVER-IMPLEMENTED bucket **only** when an EXACT ``(req_id, deleted_symbol)``
pair has a validated, human-verified record in
``plans/feature-loss-invalid-citations.json`` (schema below). This is
deliberately NOT the ``# gate-ok:`` mechanism disavowed two paragraphs up:
- No pattern, prefix, wildcard, ``--max``, or env override — one exact pair
  per record, checked against the actual deleted-symbol finding.
- A LOSS with no matching record still fails, unconditionally.
- A malformed or unreadable evidence file makes the WHOLE gate INVALID
  (never PASS) — the same fail-closed discipline as an unreadable git repo
  (see ``load_reclassification_evidence``); a half-filled record is refused,
  not silently honoured.
- An absent evidence file is "no reclassifications" — today's behaviour,
  unchanged.
- Every run reports how many citations were reclassified and how many
  evidence records matched nothing current (stale — audit and prune),
  always naming ``plans/feature-loss-invalid-citations.json`` so a reader
  can find and audit the exclusions. Silent exclusion is the failure mode
  this guards against.
``core/gate_waivers.py`` stays un-imported; this mechanism reads its own
JSON file directly and nothing else honours it.

SURFACE-LOSS (HEURISTIC, INFO-ONLY) — A FIFTH CHECK, A DIFFERENT EVIDENCE
GRADE. Every check above is keyed on an ``@traces``/requirement-id
CITATION: if a capability was never traced, it can vanish and none of them
will ever see it. Commit ``2883703ae5ffa84c4c9794e1a6c94d67111aa3f5``
("refactor(postman): delete dead postman_types twin...") proved this
concretely: it deleted ``postman/postman_types.py``, whose ``MessageEnvelope``
carried ``correlation_id`` (traced, ``REQ-002-FR1-007`` — check 3 caught its
loss) alongside ``sender``/``recipient``/``timestamp``/``content_type``
(untraced, renamed/relocated — not a loss) and whose ``MessagePayload``
carried ``compressed``/``size_bytes`` (untraced, and — at the time this
section was written — genuinely dropped with **zero** trace of them left in
the surviving code; the class docstring's own "Supports compression and
size tracking" sentence vanished with the fields). No citation-keyed check
could ever have reported that loss, because there was no citation to key on.

WHAT IT CHECKS. For a deleted ``src/**/*.py`` file (found via the exact same
``find_deletions``/``_dedupe_latest``/``_git_show`` machinery check 3 already
uses — no second history walker), extract its PUBLIC SURFACE from the last
committed content and ask, for each name: does it exist anywhere in the
CURRENT tree? **v1 SCOPE, DELIBERATELY NARROWED (disclosed, not hidden):
public DATACLASS FIELDS only** — not class names, not method/function names.
Measured against this repo's own history (3,300+ commits): only 78 ``src/
**/*.py`` files are deleted-and-still-gone, extracting 481 public dataclass
fields, 229 public class names, 645 public methods, and 100 public top-level
functions. Classes and methods/functions are NOT attempted in v1 — a
same-owning-class check (below) is exactly as cheap for them as for fields,
but the raw candidate volume (98 classes, 280 methods, 57 functions,
measured) is dominated by common, low-signal names (``to_dict``,
``__init__``-adjacent helpers, generic class names) this session had no way
to hand-verify at that scale without guessing. Fields is where the
motivating incident actually lived, and where a same-owning-class check is
most precise (see RENAME HANDLING below); a future session can extend the
same machinery to the other three categories with the same evidence-file
mechanism, without redesigning it.

RENAME HANDLING — THE HARD PART, AND WHY A BARE NAME-EXISTS CHECK FAILS IT.
Renames are the dominant confounder: the SAME commit above renamed
``sender``→``from_``, ``recipient``→``to``, ``timestamp``→``created_at``,
and relocated ``content_type`` from the envelope to the payload class. A
naive "does this bare field name exist ANYWHERE in ``src/``, as plain text
or as any identifier" check gets BOTH directions of this wrong on this
repo's own real code, independently verified while building this section:
* It WRONGLY SUPPRESSES real losses: ``size_bytes`` is a genuinely-lost
  ``MessagePayload`` field, yet the bare token ``size_bytes`` also happens
  to be a real, unrelated public dataclass field on
  ``knowledge_graph.types_nodes.FileNode`` (file size in bytes — a
  completely different domain). A bare-name check finds that coincidence
  and calls ``size_bytes`` "still there" — a false negative on the exact
  capability this section exists to catch.
* It WRONGLY FLAGS renames as losses: ``recipient``/``timestamp`` still
  exist, unchanged, as fields on ``DeliveryReceipt`` — a DIFFERENT class in
  the SAME deleted file that this repo's own successor module
  (``postman/types.py``) still defines near-identically. A check that does
  not know WHICH class a field belonged to cannot tell "this exact capability
  persists on its own successor class" apart from "some unrelated class
  somewhere merely reuses a common word".
The fix implemented here (``_current_class_members``, ``find_surface_
losses``): existence is checked PAIRED on ``(owning class name, field
name)``, not on the field name alone. A class of the SAME NAME persisting
anywhere under ``src/orchestrata_ultimate/`` with a member (field,
``@property``, OR method — see below) of the SAME NAME clears the field;
nothing else does automatically. This is dramatically more selective than a
bare-name check (it takes a COINCIDENTAL same-CLASS-name collision, not just
a same-field-name one, to still produce a false negative) without being
scoped to file or package, which — measured on this repo's own six-name
worked example below — would have broken the ``sender`` case (its only
surviving trace is a ``core.packet.compat.PacketHeader.sender`` field and a
``kernel.pam.types.AuthorityResult.sender`` field, both in unrelated
packages). This pairing is a disclosed, accepted trade-off, not a silent
one: a same-named class in a totally unrelated part of the codebase can
still coincidentally suppress a genuine loss. It is far rarer than a
bare-field-name collision, not impossible.

The existence check is deliberately LENIENT about WHAT KIND of member
clears a field, for a reason this repo's own code demonstrates: a lost
dataclass FIELD can legitimately resurface as a plain attribute, a
``@property``, or a method on its same-named successor class, and still be
"not lost" from a caller's point of view (``envelope.sender`` reads
identically whether ``sender`` is a stored field or a computed property).
``postman/types.py``'s current ``MessageEnvelope`` documents exactly this: a
"Field migration from the pre-CXP schema" docstring plus three read-only
compatibility ``@property`` methods, literally named ``sender``,
``recipient``, ``timestamp`` — so this check's own (class, field) pairing
finds them as surviving MEMBERS of the same-named class without needing an
evidence record. ``content_type`` similarly resolves via a ``@property``
that deliberately RAISES (it points the caller at ``MessagePayload``
instead of faking a value) — this check is structural, not a runtime
behaviour verifier, so a name persisting as a class member is "found"
regardless of what its body does; that a human documented a raising
compatibility shim rather than silently returning nonsense is a code-quality
virtue this check has no way to score and does not try to.

WORKED RESULT (the six names named in this section's motivating incident, as
resolved by this mechanism when it was built): ``sender``, ``recipient``,
``timestamp`` and ``content_type`` are all cleared by the same-owning-class
member check above with ZERO evidence records needed — each survives as
either a field on its ORIGINAL class (``recipient``/``timestamp`` on
``DeliveryReceipt``, ``content_type`` on ``MessagePayload``) or a
compatibility ``@property`` of the same name on ``MessageEnvelope``
(``sender``/``recipient``/``timestamp``/``content_type``). ``compressed``
and ``size_bytes`` were, when this incident was first investigated, genuine
candidates — neither the field-loss mechanism above nor an evidence record
could clear them, because nothing in the current tree carried those names
back onto ``MessagePayload``. Independently, DURING the same session that
built this detector, both fields were restored to ``MessagePayload`` in
``postman/types.py`` (see that class's own docstring — "Restored field —
dropped by commit 2883703a..."), so this mechanism, run against the tree as
it stands today, correctly reports them as NOT a current loss: the
capability came back, and a detector that kept insisting otherwise after a
real fix would be exactly the kind of noise this project is trying to
eliminate elsewhere. The unit tests below reconstruct the ORIGINAL
pre-restoration tree shape as a synthetic fixture (the same throwaway-git-
repo pattern every other test in this suite uses) to prove this mechanism
WOULD have caught the loss at the time it existed — that is the durable,
repeatable proof; a live-tree snapshot is a moving target this file's own
history was actively moving under during development.

EVIDENCE FOR THE REST — ``surface_records``, THE SURFACE-LOSS SIBLING OF
``records`` IN THE SAME EVIDENCE FILE. When the same-owning-class check
above cannot clear a candidate (a genuine cross-class/cross-package
relocation, or a rename this repo chooses not to leave a compatibility
shim for), the same evidence file the RECLASSIFICATION section above
uses — ``plans/feature-loss-invalid-citations.json`` — gains a SECOND,
independent list: ``surface_records``. The existing ``records`` list is
keyed on ``(req_id, deleted_symbol)``, a shape a surface-loss finding
cannot use (it has no requirement id). ``surface_records`` uses the
smallest schema that keys correctly instead: each record is a JSON object
with SIX required non-empty string fields — ``class_name``, ``field_name``
(together, the key — the same ``(class, field)`` pairing check 5 checks
against), ``file`` and ``deleting_commit`` (which historical deletion this
explains), ``became`` (free text: where the capability actually went, e.g.
``"MessageEnvelope.from_ (renamed; postman/types.py)"``), and
``verified_date``. No ``mismatch_class`` taxonomy, no ``methods``
verification-narrative, no ``requirement_text``/``requirement_source``/
``symbol_did`` — those exist on the citation schema to prove a REQUIREMENT
citation was wrong, which does not apply here; the smallest addition that
still lets a human record "I checked, this became X, here is where" is all
this mechanism needs. Same fail-closed discipline as its citation sibling:
a malformed ``surface_records`` entry makes the WHOLE gate INVALID, never
silently ignored; an ABSENT ``surface_records`` key (today's real evidence
file has none yet) is "no surface reclassifications" — not an error. The
file-reading/parsing/schema-version-check step is SHARED between both
mechanisms (``_load_evidence_json``) rather than duplicated.

WHY INFO, NEVER FAIL, IN v1 — A DELIBERATE PRECISION DECISION, ARGUED, NOT
DEFAULTED. A SURFACE-LOSS finding is NEVER counted toward ``losses``,
``report.status``, or the exit code, no matter how many are found. This is
a new heuristic run for the first time against 3,300+ commits of real
history with an UNKNOWN false-positive rate — the exact same "noisy gate
gets bypassed" failure mode this project has spent this whole session
dismantling elsewhere (see the RECLASSIFICATION section above, born from
discovering the citation mechanism's OWN false-positive rate the hard way).
Turning this blocking on day one, before a single surface_records entry has
ever been human-verified, would force the identical waiver-and-ignore cycle
onto a BRAND NEW mechanism before it has earned any trust. INFO — printed
in full (subject to the same disclosed, non-silent inline-cap convention as
the never-implemented section, never truncated in ``--json``), visually
distinct (its own banner, its own ``[SURFACE-LOSS]`` tag — never
``[DELETED-SYMBOL]``/``[ORPHANED-REQUIREMENT]``/``[DELETED-SPEC]``), and
explicitly labelled heuristic and rename-prone on every single run, so
nobody mistakes it for a citation-evidenced ``Loss`` — is the version of
this mechanism that can accumulate verified ``surface_records`` entries
(or, eventually, a promotion to a failing check once its true positive rate
is established) without being disabled on week one.

Usage::

    python scripts/check_feature_loss.py            # text report, real exit code
    python scripts/check_feature_loss.py --json      # machine-readable, full lists
    python scripts/check_feature_loss.py --repo-root PATH   # gate a different tree
"""

from __future__ import annotations

import argparse
import ast
import json
import subprocess
import sys
from collections import defaultdict
from collections.abc import Mapping
from dataclasses import asdict, dataclass, field
from pathlib import Path

# Repo root on sys.path so `scripts` resolves regardless of invocation style
# (direct `python scripts/check_feature_loss.py`, `-m`, or test import) —
# the established pattern (scripts/check_wiring.py, scripts/refactor.py).
_REPO_ROOT = Path(__file__).resolve().parent.parent
if str(_REPO_ROOT) not in sys.path:
    sys.path.insert(0, str(_REPO_ROOT))

from orchestrata_ultimate.core.tracing.scanner import (  # noqa: E402
    ModuleCorpus,
    ModuleFilter,
    RequirementMentionFilter,
)
from scripts.generate_rtm_evidence import (  # noqa: E402
    WEB_SOURCE_ROOTS,
    harvest_src_declarations,
    harvest_test_citations,
    harvest_web_declarations,
    requirement_ids_in,
    requirement_source_files,
)

EXIT_PASS = 0
EXIT_FAIL = 1
EXIT_INVALID = 2

SRC_RELDIR = "src/orchestrata_ultimate"
SPECS_RELDIR = "specs"
TESTS_RELDIR = "tests"
ADR_RELDIR = "docs/adr"
EVIDENCE_RELPATH = "plans/feature-loss-invalid-citations.json"

# The exact six values a reclassification record's mismatch_class may take
# — see the module docstring's RECLASSIFICATION section. Any other value
# (typo, new taxonomy nobody ratified) makes the whole evidence file INVALID
# rather than being silently accepted.
_VALID_MISMATCH_CLASSES = frozenset(
    {
        "WRONG-FR-GROUP",
        "WRONG-SPEC",
        "WRONG-SUBSYSTEM",
        "CATEGORY-MISMATCH",
        "NEVER-BUILT-EITHER-WAY",
        "SUPERSEDED",
    }
)

# Every one of these must be present as a non-empty string on a
# reclassification record, or the record — and therefore the whole evidence
# file — is INVALID. ``methods`` is validated separately: a non-empty
# string OR a non-empty list of non-empty strings, since a verification
# NARRATIVE (the evidence agents' actual output — see
# ``_validate_evidence_record``) is more useful provenance than a list of
# bare tool names, and the field's job is proving provenance exists and is
# substantive, not dictating a serialization.
_EVIDENCE_REQUIRED_STR_FIELDS = (
    "req_id",
    "deleted_symbol",
    "file",
    "deleting_commit",
    "mismatch_class",
    "requirement_text",
    "requirement_source",
    "symbol_did",
    "verified_date",
)

# How many NEVER-IMPLEMENTED ids to print inline in the text report before
# falling back to a count + "--json" pointer. This is NOT the "never
# truncate evidence" rule being violated: every LOSS (the thing a decision
# gets made from) is always printed in full, unconditionally, below. This
# cap applies ONLY to the non-failing, disclosed-as-backlog INFO class,
# which this project's own RTM tooling already documents as routinely in
# the thousands ("UNLINKED... the honest majority").
_NEVER_IMPLEMENTED_INLINE_CAP = 30

# How many SURFACE-LOSS candidates to print inline before falling back to a
# count + "--json" pointer — the same disclosed, non-silent convention as
# _NEVER_IMPLEMENTED_INLINE_CAP above, for the same reason: SURFACE-LOSS is
# an INFO-only, non-failing class (module docstring's SURFACE-LOSS section)
# that can legitimately run into the dozens on a large history.
_SURFACE_LOSS_INLINE_CAP = 40


@dataclass(frozen=True)
class DeletionEvent:
    """One git commit that deleted *path* (rename-aware via ``-M``)."""

    commit: str
    date: str
    path: str


@dataclass(frozen=True)
class Loss:
    """A confirmed feature loss — always fails the gate."""

    kind: str  # "DELETED-SYMBOL" | "ORPHANED-REQUIREMENT" | "DELETED-SPEC"
    req_id: str
    authority: str  # the spec/ADR file naming this requirement, when known
    message: str
    remedy: str
    commit: str | None = None
    date: str | None = None


@dataclass(frozen=True)
class NeverImplemented:
    """INFO only — never affects the exit code. See module docstring."""

    req_id: str
    authority: str
    title: str


@dataclass(frozen=True)
class SurfaceLoss:
    """A candidate PUBLIC-SURFACE loss — HEURISTIC, not requirement-
    evidenced (module docstring's SURFACE-LOSS section). NEVER fails the
    gate and NEVER counts toward ``report.status``/exit code, however many
    are found. Always rendered in its own, visually distinct section with
    its own ``[SURFACE-LOSS]`` tag, so a reader never mistakes it for a
    citation-evidenced ``Loss`` above."""

    class_name: str
    field_name: str
    file: str
    commit: str
    date: str
    message: str


@dataclass(frozen=True)
class FeatureLossReport:
    status: str  # "pass" | "fail" | "invalid"
    reason: str | None
    losses: list[Loss] = field(default_factory=list)
    never_implemented: list[NeverImplemented] = field(default_factory=list)
    stats: dict[str, int] = field(default_factory=dict)
    # Audit trail for the RECLASSIFICATION mechanism (module docstring) —
    # each entry {"req_id", "deleted_symbol", "mismatch_class"}. Never used
    # to compute pass/fail; purely so a reader can see and audit exactly
    # which citations were excluded and which evidence records matched
    # nothing current, without re-deriving it from the evidence file by hand.
    reclassified: list[dict[str, str]] = field(default_factory=list)
    stale_evidence: list[dict[str, str]] = field(default_factory=list)
    # POLYGLOT SCAN SCOPE (module docstring): one entry per language/root
    # actually consulted for "current implementation" — {"language", "root",
    # "status"}. Never used to compute pass/fail; purely disclosure, so a
    # reader can see which languages were searched and never mistakes an
    # absent root ("not present in this tree -- not scanned") for a scanned
    # one that found nothing. Empty on the early "invalid" returns below,
    # which never reach the point where a scan is even attempted.
    scan_scope: list[dict[str, str]] = field(default_factory=list)
    # SURFACE-LOSS (HEURISTIC, INFO-ONLY — module docstring's SURFACE-LOSS
    # section). Never contributes to `status`/exit code, unlike `losses`
    # above — see WHY INFO, NEVER FAIL, IN v1 in the module docstring.
    surface_losses: list[SurfaceLoss] = field(default_factory=list)
    surface_reclassified: list[dict[str, str]] = field(default_factory=list)
    surface_stale_evidence: list[dict[str, str]] = field(default_factory=list)


@dataclass(frozen=True)
class ReclassificationEvidence:
    """Parsed, validated contents of ``plans/feature-loss-invalid-citations.
    json`` — see the module docstring's RECLASSIFICATION section.

    ``pairs`` is the exact-match set a DELETED-SYMBOL finding is checked
    against (``(req_id, qualname)``, both strings — ``qualname`` the dotted
    form ``_symbol_qualname`` computes on the finding side and
    ``_record_symbol_qualname`` computes, from ``file`` + ``deleted_symbol``,
    on the evidence-record side). ``records_by_pair`` keeps the full record
    so the report can cite ``mismatch_class`` etc. for audit, keyed by the
    same derived qualname pair rather than the record's raw
    ``deleted_symbol`` string.
    """

    pairs: frozenset[tuple[str, str]]
    records_by_pair: dict[tuple[str, str], dict[str, object]]


@dataclass(frozen=True)
class SurfaceReclassificationEvidence:
    """Parsed, validated contents of the ``surface_records`` key in
    ``plans/feature-loss-invalid-citations.json`` — the SURFACE-LOSS sibling
    of ``ReclassificationEvidence`` above (module docstring's SURFACE-LOSS
    section, "EVIDENCE FOR THE REST"). Keyed on ``(class_name, field_name)``
    — a surface-loss finding has no requirement id to key on."""

    pairs: frozenset[tuple[str, str]]
    records_by_pair: dict[tuple[str, str], dict[str, object]]


# --------------------------------------------------------------------------
# Git plumbing
# --------------------------------------------------------------------------


def _git(
    repo_root: Path, *args: str, timeout: int = 30
) -> subprocess.CompletedProcess[str]:
    return subprocess.run(
        ["git", "-C", str(repo_root), *args],
        capture_output=True,
        text=True,
        timeout=timeout,
    )


def is_git_repo(repo_root: Path) -> bool:
    try:
        proc = _git(repo_root, "rev-parse", "--is-inside-work-tree")
    except OSError, subprocess.TimeoutExpired:
        return False
    return proc.returncode == 0 and proc.stdout.strip() == "true"


def find_deletions(repo_root: Path, *pathspecs: str) -> list[DeletionEvent]:
    """Every commit, on HEAD's own ancestry (never ``--all`` — a loss on an
    abandoned branch that never reached this history is not a loss in it),
    that deleted a path under *pathspecs*. Rename-aware (``-M``): a rename is
    reported as ``R``, never ``D``, so it is excluded here by construction.
    """
    try:
        proc = _git(
            repo_root,
            "log",
            "--diff-filter=D",
            "-M",
            "--name-status",
            "--pretty=format:__COMMIT__%x09%H%x09%ad",
            "--date=short",
            "--",
            *pathspecs,
            timeout=120,
        )
    except OSError, subprocess.TimeoutExpired:
        return []
    if proc.returncode != 0:
        return []
    events: list[DeletionEvent] = []
    commit: str | None = None
    date: str | None = None
    for line in proc.stdout.splitlines():
        if line.startswith("__COMMIT__"):
            _, commit, date = line.split("\t")
            continue
        if not line.strip() or commit is None or date is None:
            continue
        parts = line.split("\t")
        if len(parts) >= 2 and parts[0] == "D":
            events.append(DeletionEvent(commit=commit, date=date, path=parts[1]))
    return events


def _dedupe_latest(events: list[DeletionEvent]) -> dict[str, DeletionEvent]:
    """``git log`` lists newest-first; the first occurrence of a path is its
    most recent deletion — the one that explains why it is missing NOW. A
    path deleted, restored, and deleted again keeps only the live reason."""
    out: dict[str, DeletionEvent] = {}
    for event in events:
        out.setdefault(event.path, event)
    return out


def _git_show(repo_root: Path, ref: str, path: str) -> str | None:
    try:
        proc = _git(repo_root, "show", f"{ref}:{path}", timeout=30)
    except OSError, subprocess.TimeoutExpired:
        return None
    return proc.stdout if proc.returncode == 0 else None


# --------------------------------------------------------------------------
# @traces extraction from historical blob text
# --------------------------------------------------------------------------


def traces_from_text(text: str) -> dict[str, list[dict[str, object]]]:
    """req id -> [{"name", "line"}] for ``@traces(reqs=[...])`` decorators in
    *text* — a git blob at a historical commit, never a filesystem path.

    Same decorator-shape walk as
    ``generate_rtm_evidence.harvest_src_declarations``'s inner loop, adapted
    to operate on already-read TEXT: that function reads ``Path`` objects off
    the live filesystem, which a deleted file's PAST content is not. Kept
    local rather than factored out of ``generate_rtm_evidence.py`` because
    this gate's brief scopes edits to this file, ``tests/scripts/
    test_check_feature_loss.py`` and ``scripts/gates.py`` only.
    """
    decls: dict[str, list[dict[str, object]]] = defaultdict(list)
    try:
        tree = ast.parse(text)
    except SyntaxError:
        return decls
    for node in ast.walk(tree):
        if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
            continue
        for dec in node.decorator_list:
            if not (
                isinstance(dec, ast.Call)
                and (
                    (isinstance(dec.func, ast.Name) and dec.func.id == "traces")
                    or (
                        isinstance(dec.func, ast.Attribute)
                        and dec.func.attr == "traces"
                    )
                )
            ):
                continue
            for kw in dec.keywords:
                if kw.arg != "reqs" or not isinstance(kw.value, (ast.List, ast.Tuple)):
                    continue
                for elt in kw.value.elts:
                    if (
                        isinstance(elt, ast.Constant)
                        and isinstance(elt.value, str)
                        and elt.value != "REQ-000-000"
                    ):
                        decls[elt.value].append(
                            {"name": node.name, "line": node.lineno}
                        )
    return decls


# --------------------------------------------------------------------------
# SURFACE-LOSS extraction — public dataclass fields, from historical blob
# text AND from the current tree. See module docstring's SURFACE-LOSS
# section. v1 SCOPE: dataclass FIELDS only (not classes, methods, or
# top-level functions) — narrowed deliberately, see that section.
# --------------------------------------------------------------------------


def _is_dataclass_decorated(node: ast.ClassDef) -> bool:
    """True if *node* carries a ``@dataclass`` decorator, bare or called
    (``@dataclass``, ``@dataclass()``, ``@dataclass(frozen=True)``),
    including the ``@dataclasses.dataclass`` attribute form. A class-body
    annotated assignment only counts as a "dataclass field" (module
    docstring's SURFACE-LOSS section) when its owning class actually is
    one — a Protocol/ABC class-variable annotation is a different
    declaration kind this check deliberately does not treat as a lost
    capability."""
    for dec in node.decorator_list:
        target = dec.func if isinstance(dec, ast.Call) else dec
        if isinstance(target, ast.Name) and target.id == "dataclass":
            return True
        if isinstance(target, ast.Attribute) and target.attr == "dataclass":
            return True
    return False


def _extract_public_dataclass_fields(text: str) -> list[tuple[str, str]]:
    """``[(class_name, field_name), ...]`` for every PUBLIC dataclass field
    defined in *text* — a historical git blob, never a filesystem path,
    same shape as ``traces_from_text`` above. "Public" excludes both a
    leading-underscore class and a leading-underscore field name. Multiple
    occurrences of the same field name on DIFFERENT classes in the same
    file (e.g. ``content_type`` on both ``MessageEnvelope`` and
    ``MessagePayload`` in the incident this section documents) are both
    returned — each is checked independently by ``find_surface_losses``."""
    out: list[tuple[str, str]] = []
    try:
        tree = ast.parse(text)
    except SyntaxError:
        return out
    for node in ast.walk(tree):
        if not isinstance(node, ast.ClassDef) or node.name.startswith("_"):
            continue
        if not _is_dataclass_decorated(node):
            continue
        for item in node.body:
            if (
                isinstance(item, ast.AnnAssign)
                and isinstance(item.target, ast.Name)
                and not item.target.id.startswith("_")
            ):
                out.append((node.name, item.target.id))
    return out


def _current_class_members(src_dir: Path) -> dict[str, set[str]]:
    """``class_name -> {every public field/property/method name defined on
    ANY class of that name under *src_dir*}`` — the CURRENT-tree existence
    index ``find_surface_losses`` checks a lost field against.

    Deliberately LENIENT relative to the extraction side above, in two ways
    both disclosed in the module docstring's SURFACE-LOSS / RENAME HANDLING
    sections with real worked examples from this repo's own history:

    1. NOT scoped to ``@dataclass`` classes — a lost dataclass FIELD
       legitimately resurfaces as a plain attribute, a ``@property``, or a
       method on its same-named successor class (``MessageEnvelope.sender``
       became a compatibility ``@property`` after commit ``2883703a``, not
       a dataclass field, and must still count as "not lost").
    2. NOT scoped by file or package — ANY class of the same name anywhere
       under *src_dir* counts, even in a totally unrelated package. Pairing
       on (class_name, field_name) TOGETHER, rather than field_name alone,
       removes the overwhelming majority of the resulting noise but does
       not eliminate it; a same-named class elsewhere can still
       coincidentally suppress a genuine loss. Accepted v1 trade-off, not a
       hidden one.
    """
    members: dict[str, set[str]] = defaultdict(set)
    for pyfile in src_dir.rglob("*.py"):
        try:
            text = pyfile.read_text(encoding="utf-8")
            tree = ast.parse(text)
        except OSError, UnicodeDecodeError, SyntaxError:
            continue
        for node in ast.walk(tree):
            if not isinstance(node, ast.ClassDef):
                continue
            for item in node.body:
                if (
                    isinstance(item, ast.AnnAssign)
                    and isinstance(item.target, ast.Name)
                    and not item.target.id.startswith("_")
                ):
                    members[node.name].add(item.target.id)
                elif isinstance(
                    item, (ast.FunctionDef, ast.AsyncFunctionDef)
                ) and not item.name.startswith("_"):
                    members[node.name].add(item.name)
    return members


# --------------------------------------------------------------------------
# Corpus authority
# --------------------------------------------------------------------------


def collect_ratified(specs_dir: Path) -> dict[str, dict[str, str]]:
    """id -> {"file": defining file (repo-relative when possible), "title"}.

    First definition wins, same convention
    ``check_requirement_corpus.collect()`` uses for its own ``ratified``
    mapping — built independently here (not via ``generate_rtm_evidence.
    load_requirements``, which folds multiple defining files into one entry
    without keeping the FIRST FILE as a stable "authority" pointer) from the
    same two reused primitives (``requirement_source_files``,
    ``requirement_ids_in``), so this gate's citations always resolve to one
    concrete, openable file.
    """
    ratified: dict[str, dict[str, str]] = {}
    for source in requirement_source_files(specs_dir):
        try:
            text = source.read_text(encoding="utf-8")
        except OSError, UnicodeDecodeError:
            continue
        try:
            shown = str(source.relative_to(specs_dir.parent))
        except ValueError:
            shown = str(source)
        for rid, title in requirement_ids_in(text).items():
            ratified.setdefault(rid, {"file": shown, "title": title})
    return ratified


# --------------------------------------------------------------------------
# Current implementation coverage — Python + every polyglot web root. See
# the module docstring's POLYGLOT SCAN SCOPE section for the two real false
# positives (REQ-021-195..198, REQ-016-290..294) a Python-only version of
# this produced.
# --------------------------------------------------------------------------


def collect_current_implementation(
    repo_root: Path, src_dir: Path
) -> tuple[dict[str, list[dict]], list[dict[str, str]]]:
    """Every requirement id with a CURRENT implementation, in every language
    this gate can see: Python under *src_dir* (``harvest_src_declarations``)
    UNIONED with TS/JS ``@traces`` comment tags under every
    ``generate_rtm_evidence.WEB_SOURCE_ROOTS`` entry (``opencode``,
    ``vizint-ui`` today) — the SAME harvester ``generate_rtm_evidence.py``
    already uses to grade DECLARED, reused here rather than reimplemented
    (module docstring). Every caller of this dict (checks 1-3) automatically
    treats a TS/JS-only implementation as current, exactly like a Python one.

    Returns ``(current_impl, scan_scope)``. *scan_scope* is one entry per
    language/root actually consulted — ``{"language", "root", "status"}``,
    *status* either ``"scanned"`` or ``"not present in this tree -- not
    scanned"`` — so the report can name what it searched instead of leaving
    a reader to assume "scanned, found nothing" for a root this tree simply
    does not have (the test suite's throwaway repos have no ``opencode/``;
    that must read as UNSCANNED, never as ABSENT — see module docstring).
    """
    current_impl = harvest_src_declarations(src_dir)
    scan_scope: list[dict[str, str]] = [
        {"language": "Python", "root": SRC_RELDIR, "status": "scanned"}
    ]
    for web_root in WEB_SOURCE_ROOTS:
        web_dir = repo_root / web_root
        if not web_dir.is_dir():
            scan_scope.append(
                {
                    "language": "TypeScript/JavaScript",
                    "root": web_root,
                    "status": "not present in this tree -- not scanned",
                }
            )
            continue
        for rid, occurrences in harvest_web_declarations(web_dir).items():
            current_impl[rid].extend(occurrences)
        scan_scope.append(
            {
                "language": "TypeScript/JavaScript",
                "root": web_root,
                "status": "scanned",
            }
        )
    return current_impl, scan_scope


# --------------------------------------------------------------------------
# Reclassification evidence (plans/feature-loss-invalid-citations.json) —
# see the module docstring's RECLASSIFICATION section.
# --------------------------------------------------------------------------


def _symbol_qualname(path: str, name: str) -> str:
    """``src/orchestrata_ultimate/knowledge_graph/kg_sync.py`` + ``sync_folders``
    -> ``knowledge_graph.kg_sync.sync_folders`` — the module's own matching
    key for a DELETED-SYMBOL finding, computed from the (path, name) the
    finding is already carrying."""
    rel = path.removeprefix(f"{SRC_RELDIR}/").removesuffix(".py")
    return f"{rel.replace('/', '.')}.{name}"


def _record_symbol_qualname(record: Mapping[str, object]) -> str:
    """The matching key for an evidence RECORD, derived from its own
    ``file`` + ``deleted_symbol`` fields the same way ``_symbol_qualname``
    derives one from a finding's (path, name) — so both sides land in the
    same units regardless of which shape ``deleted_symbol`` happens to use.

    In practice the evidence agents' actual output puts the BARE symbol
    name in ``deleted_symbol`` (e.g. ``"CXPMixin"``, ``"sync_folders"``),
    disambiguated by the sibling ``file`` field, not a pre-computed dotted
    qualname — so a bare name is used as-is. An already-dotted
    ``deleted_symbol`` (this schema's own illustrative form) is reduced to
    its last segment first, so either shape lands on the identical key.
    """
    bare_name = str(record["deleted_symbol"]).rsplit(".", 1)[-1]
    file = str(record["file"])
    prefix = f"{SRC_RELDIR}/"
    if file.startswith(prefix):
        return _symbol_qualname(file, bare_name)
    # A `file` that is not a src/ path at all must never silently produce a
    # key that happens to collide with a real finding — falling back to the
    # bare name alone can only make the record STAY stale, never cause a
    # false reclassification.
    return bare_name


def _validate_evidence_record(record: object) -> str | None:
    """Return a human-readable error if *record* is malformed, else None.

    Deliberately strict: a record missing ``requirement_source``, carrying
    an unrecognized ``mismatch_class``, or with an empty ``symbol_did`` must
    make the gate refuse the whole file (INVALID) rather than silently
    honour a half-filled record — an evidence mechanism nobody validates is
    a waiver with extra steps.

    ``methods`` is the one field validated on CONTENT rather than a fixed
    container shape: a non-empty string (a verification narrative — the
    shape the evidence agents actually produce, and materially more useful
    to a human re-auditing an exclusion than a bare tool-name list) OR a
    non-empty list of non-empty strings are both accepted; anything that
    fails to prove "provenance exists and is substantive" — absent, an
    empty string, an empty list, a list holding a non-string or an
    empty/whitespace-only string, or any other type — is still rejected.
    This is a relaxation of SHAPE, never of standard: every other field
    keeps exactly the strictness above.
    """
    if not isinstance(record, dict):
        return "record is not a JSON object"
    for name in _EVIDENCE_REQUIRED_STR_FIELDS:
        value = record.get(name)
        if not isinstance(value, str) or not value.strip():
            return f"record missing/empty required field {name!r}"
    if record["mismatch_class"] not in _VALID_MISMATCH_CLASSES:
        return (
            f"record has unrecognized mismatch_class "
            f"{record['mismatch_class']!r} (must be one of "
            f"{sorted(_VALID_MISMATCH_CLASSES)})"
        )
    methods = record.get("methods")
    if isinstance(methods, str):
        if not methods.strip():
            return "record's 'methods' field is an empty/whitespace-only string"
    elif isinstance(methods, list):
        if not methods:
            return "record's 'methods' field is an empty list"
        if not all(isinstance(m, str) and m.strip() for m in methods):
            return "record's 'methods' list must contain only non-empty strings"
    else:
        return (
            "record's 'methods' field must be a non-empty string or a "
            "non-empty list of non-empty strings"
        )
    return None


def _load_evidence_json(repo_root: Path) -> tuple[dict[str, object] | None, str | None]:
    """Read, parse, and schema-check the top-level JSON object at
    ``plans/feature-loss-invalid-citations.json`` — the ONE file-reading
    step shared by ``load_reclassification_evidence`` (citation-keyed
    ``records``) and ``load_surface_reclassification_evidence`` (the
    SURFACE-LOSS ``surface_records`` sibling — module docstring's "EVIDENCE
    FOR THE REST"), so the two mechanisms never duplicate "read the file,
    parse JSON, check schema_version" logic.

    Returns ``(data, None)`` on success, ``(None, error)`` on a malformed
    file, or ``(None, None)`` when the file is simply absent — a caller
    tells "absent" (fine, proceed with empty evidence) from "malformed"
    (INVALID) by checking *error* is not None.
    """
    path = repo_root / EVIDENCE_RELPATH
    if not path.exists():
        return None, None
    try:
        text = path.read_text(encoding="utf-8")
    except OSError as exc:
        return None, f"{EVIDENCE_RELPATH} could not be read: {exc}"
    try:
        data = json.loads(text)
    except json.JSONDecodeError as exc:
        return None, f"{EVIDENCE_RELPATH} is not valid JSON: {exc}"
    if not isinstance(data, dict) or data.get("schema_version") != 1:
        return None, (
            f'{EVIDENCE_RELPATH} must be a JSON object with "schema_version": 1'
        )
    return data, None


def load_reclassification_evidence(
    repo_root: Path,
) -> tuple[ReclassificationEvidence, str | None]:
    """Load + validate ``plans/feature-loss-invalid-citations.json``'s
    ``records`` list (the citation-keyed reclassification mechanism).

    Returns ``(evidence, error)``:

    - Absent file -> ``(empty evidence, None)`` — "no reclassifications",
      exactly today's behaviour (the file may not exist yet, or a sibling
      agent may still be writing it).
    - Malformed/unreadable file -> ``(empty evidence, error)`` — the caller
      turns *error* into an INVALID report. Same fail-closed discipline as
      an unreadable git repo: unknown is not clean, so this NEVER falls back
      to "just ignore the bad file and pass".
    """
    empty = ReclassificationEvidence(frozenset(), {})
    data, error = _load_evidence_json(repo_root)
    if error is not None:
        return empty, error
    if data is None:
        return empty, None
    records = data.get("records")
    if not isinstance(records, list):
        return empty, f'{EVIDENCE_RELPATH}\'s "records" must be a list'

    pairs: set[tuple[str, str]] = set()
    records_by_pair: dict[tuple[str, str], dict[str, object]] = {}
    for i, record in enumerate(records):
        error = _validate_evidence_record(record)
        if error:
            return empty, f"{EVIDENCE_RELPATH} record [{i}]: {error}"
        key = (str(record["req_id"]), _record_symbol_qualname(record))
        pairs.add(key)
        records_by_pair[key] = record
    return ReclassificationEvidence(frozenset(pairs), records_by_pair), None


# Every one of these must be present as a non-empty string on a
# surface_records entry, or the record — and therefore the WHOLE evidence
# file — is INVALID (same fail-closed discipline as
# _EVIDENCE_REQUIRED_STR_FIELDS above). Deliberately the SMALLEST schema
# that still keys correctly (module docstring's "EVIDENCE FOR THE REST"):
# no mismatch_class taxonomy, no methods verification-narrative, no
# requirement_text/requirement_source/symbol_did — those exist on the
# citation schema to prove a REQUIREMENT citation was wrong, which does not
# apply to a surface-loss finding (it has no requirement id at all).
_SURFACE_EVIDENCE_REQUIRED_STR_FIELDS = (
    "class_name",
    "field_name",
    "file",
    "deleting_commit",
    "became",
    "verified_date",
)


def _validate_surface_evidence_record(record: object) -> str | None:
    """Return a human-readable error if *record* is malformed, else None —
    the SURFACE-LOSS sibling of ``_validate_evidence_record``. Same
    fail-closed discipline, smaller schema: every field in
    ``_SURFACE_EVIDENCE_REQUIRED_STR_FIELDS`` must be a non-empty string, or
    the whole evidence file is refused rather than half-honoured."""
    if not isinstance(record, dict):
        return "surface record is not a JSON object"
    for name in _SURFACE_EVIDENCE_REQUIRED_STR_FIELDS:
        value = record.get(name)
        if not isinstance(value, str) or not value.strip():
            return f"surface record missing/empty required field {name!r}"
    return None


def load_surface_reclassification_evidence(
    repo_root: Path,
) -> tuple[SurfaceReclassificationEvidence, str | None]:
    """Load + validate ``plans/feature-loss-invalid-citations.json``'s
    ``surface_records`` list — the SURFACE-LOSS sibling of
    ``load_reclassification_evidence`` (module docstring's SURFACE-LOSS
    section, "EVIDENCE FOR THE REST").

    Returns ``(evidence, error)``:

    - Absent file, OR file present but with no ``surface_records`` key ->
      ``(empty evidence, None)`` — "no surface reclassifications yet"; the
      citation-keyed ``records`` key can (and today does) exist alone, with
      no ``surface_records`` sibling, without that being an error.
    - Malformed top-level JSON, OR a malformed ``surface_records`` entry ->
      ``(empty evidence, error)`` — the SAME fail-closed discipline as the
      citation mechanism: a broken evidence file is never silently ignored,
      whichever half of it is broken.
    """
    empty = SurfaceReclassificationEvidence(frozenset(), {})
    data, error = _load_evidence_json(repo_root)
    if error is not None:
        return empty, error
    if data is None:
        return empty, None
    records = data.get("surface_records", [])
    if not isinstance(records, list):
        return empty, f'{EVIDENCE_RELPATH}\'s "surface_records" must be a list'

    pairs: set[tuple[str, str]] = set()
    records_by_pair: dict[tuple[str, str], dict[str, object]] = {}
    for i, record in enumerate(records):
        err = _validate_surface_evidence_record(record)
        if err:
            return empty, f"{EVIDENCE_RELPATH} surface_records [{i}]: {err}"
        key = (str(record["class_name"]), str(record["field_name"]))
        pairs.add(key)
        records_by_pair[key] = record
    return SurfaceReclassificationEvidence(frozenset(pairs), records_by_pair), None


# --------------------------------------------------------------------------
# Checks 1 & 3: deleted symbols -> orphaned requirements
# --------------------------------------------------------------------------


def find_deleted_symbols(
    repo_root: Path,
    current_src: dict[str, list[dict]],
    ratified: dict[str, dict[str, str]],
    evidence: ReclassificationEvidence,
) -> tuple[list[Loss], int, frozenset[tuple[str, str]]]:
    """Check 3 — DELETED-SYMBOL. See module docstring.

    Scoped to requirement ids still in the CURRENT ratified corpus
    (*ratified*) — measured against this repo's own history: of 369 raw
    deleted-``@traces`` sites found, 186 (38 distinct ids) cited a
    requirement id that no spec anywhere defines any more, almost entirely
    from a single 2026-04-early bulk-annotation commit against a req-id
    numbering scheme that was later fully restructured. Reporting those as
    FAILURES would violate the brief's own "each failure names the SPEC/REQ
    authority it violates" requirement (there is no live authority to name)
    and would be exactly the noisy-gate-gets-bypassed failure mode the brief
    warns against. They are counted (second return value) and surfaced as a
    one-line, non-failing footnote — never silently dropped — but not
    enumerated: precision over volume for a class this repo's own tooling
    would not trust either.

    *evidence* gates the RECLASSIFICATION carve-out (module docstring): a
    site whose exact ``(rid, deleted_symbol)`` pair has a validated record
    is excluded from *losses* rather than reported — it falls through to
    ``derive_never_implemented`` instead, exactly as if it were never cited.
    The third return value is every pair actually matched, so the caller can
    report both the reclassified count and any STALE evidence records (a
    record whose pair never showed up here at all).
    """
    events = _dedupe_latest(find_deletions(repo_root, SRC_RELDIR))
    losses: list[Loss] = []
    dropped_unratified = 0
    reclassified: set[tuple[str, str]] = set()
    for path, event in sorted(events.items()):
        if not path.endswith(".py"):
            continue
        if (repo_root / path).exists():
            continue  # restored (e.g. the e39f8d16/bdafa201 window) — not a current loss
        old_text = _git_show(repo_root, f"{event.commit}^", path)
        if old_text is None:
            continue
        for rid, sites in sorted(traces_from_text(old_text).items()):
            if rid in current_src:
                continue  # covered by a surviving symbol elsewhere
            if rid not in ratified:
                dropped_unratified += len(sites)
                continue
            for site in sites:
                pair = (rid, _symbol_qualname(path, str(site["name"])))
                if pair in evidence.pairs:
                    reclassified.add(pair)
                    continue  # evidenced-bogus citation — not a loss, see docstring
                losses.append(
                    Loss(
                        kind="DELETED-SYMBOL",
                        req_id=rid,
                        authority=path,
                        message=(
                            f"{site['name']}() in {path}:{site['line']} carried "
                            f"@traces(reqs=[{rid!r}]) and was deleted in commit "
                            f"{event.commit} ({event.date}); no surviving symbol "
                            f"in {SRC_RELDIR}/ declares {rid}"
                        ),
                        remedy=(
                            f"restore an implementation of {rid}, or retire it "
                            f"from its defining spec with the user's approval"
                        ),
                        commit=event.commit,
                        date=event.date,
                    )
                )
    return losses, dropped_unratified, frozenset(reclassified)


def derive_orphaned(
    symbol_losses: list[Loss], ratified: dict[str, dict[str, str]]
) -> list[Loss]:
    """Check 1 — ORPHANED-REQUIREMENT: the requirement-centric rollup of
    check 3's findings. *symbol_losses* is already ratified-only (see
    :func:`find_deleted_symbols`); the membership check below is a defensive
    no-op, kept so this function stays correct on its own if that ever
    changes."""
    by_req: dict[str, list[Loss]] = defaultdict(list)
    for loss in symbol_losses:
        if loss.req_id in ratified:
            by_req[loss.req_id].append(loss)
    out: list[Loss] = []
    for rid, group in sorted(by_req.items()):
        info = ratified[rid]
        # A single deleted file often carried the same req id on several
        # symbols (a class AND its factory function, say) — collapse those
        # into one "N site(s)" clause instead of repeating an identical
        # (file, commit) pair once per symbol (measured: one file alone
        # produced 30 near-identical repeats before this collapse).
        site_counts: dict[tuple[str, str | None, str | None], int] = defaultdict(int)
        for g in group:
            site_counts[(g.authority, g.commit, g.date)] += 1
        sites = "; ".join(
            (
                f"{count} site(s) in {authority} (commit {commit}, {date})"
                if count > 1
                else f"{authority} (commit {commit}, {date})"
            )
            for (authority, commit, date), count in sorted(site_counts.items())
        )
        out.append(
            Loss(
                kind="ORPHANED-REQUIREMENT",
                req_id=rid,
                authority=info["file"],
                message=(
                    f"{rid} is still defined by {info['file']} ({info['title']}) "
                    f"but every symbol that ever implemented it has been "
                    f"deleted: {sites}"
                ),
                remedy=(
                    f"restore an implementation of {rid}, or retire it from "
                    f"{info['file']} with the user's approval"
                ),
                commit=group[0].commit,
                date=group[0].date,
            )
        )
    return out


# --------------------------------------------------------------------------
# Check 2: never-implemented (INFO only)
# --------------------------------------------------------------------------


def derive_never_implemented(
    ratified: dict[str, dict[str, str]],
    current_src: dict[str, list[dict]],
    orphaned_ids: set[str],
) -> list[NeverImplemented]:
    out = []
    for rid, info in sorted(ratified.items()):
        if rid in current_src or rid in orphaned_ids:
            continue
        out.append(
            NeverImplemented(req_id=rid, authority=info["file"], title=info["title"])
        )
    return out


# --------------------------------------------------------------------------
# Check 4: deleted spec/ADR docs still cited
# --------------------------------------------------------------------------


def find_deleted_spec_docs(
    repo_root: Path,
    ratified: dict[str, dict[str, str]],
    current_src: dict[str, list[dict]],
    current_tests: dict[str, list[dict]],
) -> list[Loss]:
    events = _dedupe_latest(find_deletions(repo_root, SPECS_RELDIR, ADR_RELDIR))
    losses: list[Loss] = []
    for path, event in sorted(events.items()):
        if not path.endswith(".md"):
            continue
        if (repo_root / path).exists():
            continue  # restored — not a current loss
        old_text = _git_show(repo_root, f"{event.commit}^", path)
        if old_text is None:
            continue
        for rid, title in sorted(requirement_ids_in(old_text).items()):
            if rid in ratified:
                continue  # still defined by a surviving file — not a loss
            sites = current_src.get(rid, []) + current_tests.get(rid, [])
            if not sites:
                continue  # nobody cites it any more -- ordinary retirement
            cite_desc = ", ".join(f"{s['file']}:{s['line']}" for s in sites)
            losses.append(
                Loss(
                    kind="DELETED-SPEC",
                    req_id=rid,
                    authority=path,
                    message=(
                        f"{path} defined {rid} ({title}) and was deleted in "
                        f"commit {event.commit} ({event.date}); {len(sites)} "
                        f"current citation(s) still reference it: {cite_desc}"
                    ),
                    remedy=(
                        f"restore {path} (or re-home {rid} in a surviving "
                        f"spec), or retire the citation(s) with the user's "
                        f"approval"
                    ),
                    commit=event.commit,
                    date=event.date,
                )
            )
    return losses


# --------------------------------------------------------------------------
# Check 5: SURFACE-LOSS (HEURISTIC, INFO-ONLY) — see module docstring.
# --------------------------------------------------------------------------


def find_surface_losses(
    repo_root: Path, evidence: SurfaceReclassificationEvidence
) -> tuple[list[SurfaceLoss], list[dict[str, str]], list[dict[str, str]]]:
    """Check 5 — SURFACE-LOSS. See module docstring's SURFACE-LOSS section.

    Reuses ``find_deletions``/``_dedupe_latest``/``_git_show`` exactly as
    check 3 does — no second history walker. For every ``src/**/*.py`` file
    deleted and still gone, extracts its last committed content's public
    dataclass fields (``_extract_public_dataclass_fields``), groups
    occurrences by ``(class_name, field_name)`` across the WHOLE deleted
    corpus (a field lost from the same-named class in two different
    historical files is reported once, with every site listed — the same
    site-collapsing ``derive_orphaned`` already does for citation losses),
    and checks each pair against ``_current_class_members`` (Tier 1: a
    same-named class's own member survives) then *evidence* (Tier 2: a
    human-verified ``surface_records`` entry) before calling it a candidate.

    Returns ``(losses, reclassified_records, stale_records)`` — the last two
    already reduced to the small audit-trail shape ``render_report`` prints,
    mirroring ``_evidence_audit_entry``/its call sites for the citation
    mechanism.
    """
    events = _dedupe_latest(find_deletions(repo_root, SRC_RELDIR))
    occurrences: dict[tuple[str, str], list[tuple[str, str, str]]] = defaultdict(list)
    for path, event in sorted(events.items()):
        if not path.endswith(".py"):
            continue
        if (repo_root / path).exists():
            continue  # restored — not a current loss
        old_text = _git_show(repo_root, f"{event.commit}^", path)
        if old_text is None:
            continue
        for cname, fname in _extract_public_dataclass_fields(old_text):
            occurrences[(cname, fname)].append((path, event.commit, event.date))

    current_members = _current_class_members(repo_root / SRC_RELDIR)

    losses: list[SurfaceLoss] = []
    reclassified: set[tuple[str, str]] = set()
    for (cname, fname), sites in sorted(occurrences.items()):
        if fname in current_members.get(cname, set()):
            continue  # Tier 1: a member of this name survives on a same-named class
        key = (cname, fname)
        if key in evidence.pairs:
            reclassified.add(key)
            continue  # Tier 2: evidence-gated — a human traced this to its relocation
        site_desc = "; ".join(f"{p} (commit {c}, {d})" for p, c, d in sites)
        losses.append(
            SurfaceLoss(
                class_name=cname,
                field_name=fname,
                file=sites[0][0],
                commit=sites[0][1],
                date=sites[0][2],
                message=(
                    f"public dataclass field `{fname}` on {cname} was "
                    f"deleted: {site_desc}; no member named `{fname}` exists "
                    f"on any class named {cname} anywhere under "
                    f"{SRC_RELDIR}/ today, and no surface_records entry in "
                    f"{EVIDENCE_RELPATH} clears it"
                ),
            )
        )
    stale_pairs = evidence.pairs - reclassified
    reclassified_records = [
        _surface_evidence_audit_entry(evidence.records_by_pair[pair])
        for pair in sorted(reclassified)
    ]
    stale_records = [
        _surface_evidence_audit_entry(evidence.records_by_pair[pair])
        for pair in sorted(stale_pairs)
    ]
    return losses, reclassified_records, stale_records


def _surface_evidence_audit_entry(record: dict[str, object]) -> dict[str, str]:
    """The audit-trail slice of a validated surface_records entry — the
    SURFACE-LOSS sibling of ``_evidence_audit_entry``."""
    return {
        "class_name": str(record["class_name"]),
        "field_name": str(record["field_name"]),
        "became": str(record["became"]),
    }


# --------------------------------------------------------------------------
# Orchestration
# --------------------------------------------------------------------------


def run_check(
    repo_root: Path, module_filter: ModuleFilter | None = None
) -> FeatureLossReport:
    """Injection seam: tests pass a throwaway ``tmp_path`` git repo — never
    touches the live checkout unless *repo_root* names it.

    The tests tree is read and parsed ONCE into a ``ModuleCorpus`` (filtered by
    *module_filter*, default ``RequirementMentionFilter``) and handed to the
    citation harvest, which feeds both its docstring pass and the
    ``@traces`` decorator scan from it (WI-FLOSS, tasks 578/581). Before:
    two full parses of ~2,060 test modules per run (2026-10-08: the canary
    ``test_real_opencode_session_status_ids_produce_no_finding_at_all``
    took 84.8s solo / 90.5s loaded against a 120s budget)."""
    if not is_git_repo(repo_root):
        return FeatureLossReport(
            "invalid",
            f"{repo_root} is not a git repository (or git is unavailable) — "
            "a loss detector that cannot read history must not report clean",
        )

    # Loaded before anything else derived from it is computed: a malformed
    # evidence file makes the WHOLE gate INVALID (module docstring's
    # RECLASSIFICATION section) — the same fail-closed discipline as the
    # not-a-git-repo check above, never a silent "ignore it and pass".
    evidence, evidence_error = load_reclassification_evidence(repo_root)
    if evidence_error is not None:
        return FeatureLossReport("invalid", evidence_error)

    # Same fail-closed discipline for the SURFACE-LOSS evidence sibling
    # (module docstring, "EVIDENCE FOR THE REST") — a malformed
    # surface_records entry must ALSO make the whole gate INVALID, never
    # silently ignored, even though SURFACE-LOSS itself never fails the gate.
    surface_evidence, surface_evidence_error = load_surface_reclassification_evidence(
        repo_root
    )
    if surface_evidence_error is not None:
        return FeatureLossReport("invalid", surface_evidence_error)

    specs_dir = repo_root / SPECS_RELDIR
    src_dir = repo_root / SRC_RELDIR
    tests_dir = repo_root / TESTS_RELDIR

    ratified = collect_ratified(specs_dir) if specs_dir.is_dir() else {}
    if not ratified:
        return FeatureLossReport(
            "invalid",
            f"examined 0 requirement id(s) under {SPECS_RELDIR}/ — a pass "
            "with nothing examined is not a pass",
        )
    if not src_dir.is_dir():
        return FeatureLossReport(
            "invalid",
            f"{SRC_RELDIR}/ does not exist — cannot determine current "
            "implementation coverage",
        )

    current_src, scan_scope = collect_current_implementation(repo_root, src_dir)
    if tests_dir.is_dir():
        tests_corpus = ModuleCorpus.load(
            tests_dir, module_filter or RequirementMentionFilter()
        )
        current_tests = harvest_test_citations(tests_dir, corpus=tests_corpus)
    else:
        current_tests = {}

    symbol_losses, dropped_unratified, reclassified_pairs = find_deleted_symbols(
        repo_root, current_src, ratified, evidence
    )
    orphaned = derive_orphaned(symbol_losses, ratified)
    orphaned_ids = {loss.req_id for loss in orphaned}
    never_impl = derive_never_implemented(ratified, current_src, orphaned_ids)
    spec_losses = find_deleted_spec_docs(
        repo_root, ratified, current_src, current_tests
    )

    stale_pairs = evidence.pairs - reclassified_pairs
    reclassified_records = [
        _evidence_audit_entry(evidence.records_by_pair[pair])
        for pair in sorted(reclassified_pairs)
    ]
    stale_records = [
        _evidence_audit_entry(evidence.records_by_pair[pair])
        for pair in sorted(stale_pairs)
    ]

    # Check 5 — SURFACE-LOSS (HEURISTIC, INFO-ONLY, module docstring). Runs
    # unconditionally alongside the citation-keyed checks above, but its
    # result NEVER feeds `all_losses`/`status`/the exit code — see WHY INFO,
    # NEVER FAIL, IN v1 in the module docstring.
    surface_losses, surface_reclassified, surface_stale = find_surface_losses(
        repo_root, surface_evidence
    )

    all_losses = symbol_losses + orphaned + spec_losses
    stats = {
        "ratified_requirements": len(ratified),
        "deleted_symbols": len(symbol_losses),
        "orphaned_requirements": len(orphaned),
        "deleted_specs": len(spec_losses),
        "never_implemented": len(never_impl),
        "dropped_unratified_deletions": dropped_unratified,
        "reclassified_never_implemented": len(reclassified_pairs),
        "stale_evidence_records": len(stale_pairs),
        "surface_losses": len(surface_losses),
        "surface_reclassified": len(surface_reclassified),
        "surface_stale_evidence_records": len(surface_stale),
    }
    status = "fail" if all_losses else "pass"
    return FeatureLossReport(
        status=status,
        reason=None,
        losses=all_losses,
        never_implemented=never_impl,
        stats=stats,
        reclassified=reclassified_records,
        stale_evidence=stale_records,
        scan_scope=scan_scope,
        surface_losses=surface_losses,
        surface_reclassified=surface_reclassified,
        surface_stale_evidence=surface_stale,
    )


def _evidence_audit_entry(record: dict[str, object]) -> dict[str, str]:
    """The audit-trail slice of a validated evidence record — enough for a
    reader to identify it in ``plans/feature-loss-invalid-citations.json``
    without dumping the full record into every report."""
    return {
        "req_id": str(record["req_id"]),
        "deleted_symbol": str(record["deleted_symbol"]),
        "mismatch_class": str(record["mismatch_class"]),
    }


# --------------------------------------------------------------------------
# Reporting
# --------------------------------------------------------------------------


def _render_scan_scope(scan_scope: list[dict[str, str]]) -> str:
    """One line naming exactly which language/root pairs THIS RUN scanned
    for "current implementation" — module docstring's POLYGLOT SCAN SCOPE
    section: a gate that silently covers only one language of a polyglot
    repo will reproduce the exact false-confidence defect this line exists
    to prevent the moment anyone adds a language it still cannot see. A root
    absent from this tree reads as "not present -- not scanned", never as
    "scanned, found nothing" — the two are different facts."""
    if not scan_scope:
        return "SCAN SCOPE — unavailable (no scan was attempted)"
    by_lang: dict[str, list[str]] = defaultdict(list)
    for entry in scan_scope:
        by_lang[entry["language"]].append(f"{entry['root']} ({entry['status']})")
    parts = [f"{lang}: {', '.join(roots)}" for lang, roots in by_lang.items()]
    return "SCAN SCOPE — " + " | ".join(parts)


def render_report(report: FeatureLossReport) -> str:
    lines: list[str] = ["=" * 80]
    if report.status == "invalid":
        lines.append("FEATURE-LOSS GATE — INVALID")
        lines.append("=" * 80)
        lines.append(report.reason or "unknown")
        return "\n".join(lines)

    n = len(report.losses)
    by_kind: dict[str, int] = defaultdict(int)
    for loss in report.losses:
        by_kind[loss.kind] += 1
    breakdown = ", ".join(f"{v} {k}" for k, v in sorted(by_kind.items()))
    header = f"FEATURE-LOSS GATE — {n} loss(es) detected"
    if breakdown:
        header += f" ({breakdown})"
    lines.append(header)
    lines.append("=" * 80)
    lines.append(_render_scan_scope(report.scan_scope))
    lines.append("")

    # Every LOSS is printed in full, unconditionally — never truncated. This
    # is the list a decision (restore / retire) gets made from.
    for loss in report.losses:
        lines.append(f"[{loss.kind}] {loss.req_id} (authority: {loss.authority})")
        lines.append(f"  {loss.message}")
        lines.append(f"  REMEDY: {loss.remedy}")
        lines.append("")

    lines.append("-" * 80)
    ni = len(report.never_implemented)
    lines.append(
        f"INFO — {ni} never-implemented requirement(s): ratified but no git "
        f"evidence any implementation ever existed under {SRC_RELDIR}/. NOT "
        f"a loss (nothing regressed here) and NEVER counted toward the exit "
        f"code — see the module docstring for why this class stays advisory."
    )
    if 0 < ni <= _NEVER_IMPLEMENTED_INLINE_CAP:
        for item in report.never_implemented:
            lines.append(f"  {item.req_id} ({item.authority})")
    elif ni > _NEVER_IMPLEMENTED_INLINE_CAP:
        lines.append(
            f"  ({ni} id(s) — pass --json for the full list; suppressing "
            f"the inline dump above {_NEVER_IMPLEMENTED_INLINE_CAP} entries "
            f"to keep this non-failing section from swamping the report)"
        )
    dropped = report.stats.get("dropped_unratified_deletions", 0)
    if dropped:
        lines.append(
            f"INFO — {dropped} deleted-@traces citation(s) referenced a "
            f"requirement id NO current spec defines (legacy/renumbered ids, "
            f"almost entirely from one early bulk-annotation commit). Dropped "
            f"as unverifiable — no live spec authority to name — rather than "
            f"reported; see find_deleted_symbols()'s docstring."
        )

    # RECLASSIFICATION audit trail (module docstring) — always printed, even
    # at zero, so a reader always knows where to look and how many
    # exclusions were applied. Never silent.
    reclassified_n = report.stats.get("reclassified_never_implemented", 0)
    lines.append(
        f"INFO — {reclassified_n} deleted-@traces citation(s) reclassified "
        f"NEVER-IMPLEMENTED (not LOSS) on evidence in {EVIDENCE_RELPATH}: "
        f"a validated record there proves the citation was invalid WHEN "
        f"WRITTEN, so its deletion lost nothing. This is not a waiver — see "
        f"that file and the module docstring's RECLASSIFICATION section."
    )
    for item in report.reclassified:
        lines.append(
            f"  {item['req_id']} <- {item['deleted_symbol']} "
            f"({item['mismatch_class']})"
        )
    stale_n = report.stats.get("stale_evidence_records", 0)
    if stale_n:
        lines.append(
            f"INFO — {stale_n} record(s) in {EVIDENCE_RELPATH} matched no "
            f"current DELETED-SYMBOL finding (stale — the finding may have "
            f"been fixed/restored since, or the record itself is wrong; "
            f"audit and prune):"
        )
        for item in report.stale_evidence:
            lines.append(f"  {item['req_id']} <- {item['deleted_symbol']}")

    # SURFACE-LOSS (HEURISTIC, INFO-ONLY) — module docstring's SURFACE-LOSS
    # section. Visually distinct from the LOSS list above: its own banner,
    # its own [SURFACE-LOSS] tag (never [DELETED-SYMBOL]/[ORPHANED-
    # REQUIREMENT]/[DELETED-SPEC]), and an explicit heuristic disclaimer on
    # every run — never mistakeable for citation-evidenced LOSS. NEVER
    # affects `n`/FAIL-PASS below.
    lines.append("-" * 80)
    sn = len(report.surface_losses)
    lines.append(
        f"SURFACE-LOSS (HEURISTIC, INFO ONLY — {sn} candidate(s), never "
        f"fails this gate; see module docstring's SURFACE-LOSS section)"
    )
    lines.append("-" * 80)
    lines.append(
        "HEURISTIC, NOT REQUIREMENT-EVIDENCED: a public dataclass field "
        "defined in a deleted src/ file whose (owning class, field name) "
        "pair cannot be found as a member of a same-named class anywhere "
        "in the current tree, and is not cleared by a surface_records "
        "entry. Renames are its dominant confounder — treat every finding "
        "below as a LEAD to investigate, never as proof, and never as the "
        "same grade of evidence as a LOSS finding above."
    )
    lines.append(
        f"V1 SCOPE: public DATACLASS FIELDS only, under {SRC_RELDIR}/ (not "
        f"classes, methods, or top-level functions; not specs/ or TS/JS) — "
        f"see the module docstring's SURFACE-LOSS section for why."
    )
    if 0 < sn <= _SURFACE_LOSS_INLINE_CAP:
        for sl in report.surface_losses:
            lines.append(
                f"  [SURFACE-LOSS] {sl.class_name}.{sl.field_name} (heuristic)"
            )
            lines.append(f"    {sl.message}")
    elif sn > _SURFACE_LOSS_INLINE_CAP:
        lines.append(
            f"  ({sn} candidate(s) — pass --json for the full list; "
            f"suppressing the inline dump above {_SURFACE_LOSS_INLINE_CAP} "
            f"entries to keep this non-failing section from swamping the "
            f"report)"
        )
    srn = len(report.surface_reclassified)
    lines.append(
        f"INFO — {srn} surface-loss candidate(s) reclassified (rename/"
        f"relocation, not a loss) on a surface_records entry in "
        f"{EVIDENCE_RELPATH}:"
    )
    for item in report.surface_reclassified:
        lines.append(f"  {item['class_name']}.{item['field_name']} -> {item['became']}")
    ssn = len(report.surface_stale_evidence)
    if ssn:
        lines.append(
            f"INFO — {ssn} surface_records entry(ies) in {EVIDENCE_RELPATH} "
            f"matched no current SURFACE-LOSS candidate (stale — audit and "
            f"prune):"
        )
        for item in report.surface_stale_evidence:
            lines.append(f"  {item['class_name']}.{item['field_name']}")

    lines.append("-" * 80)
    lines.append("")
    if n:
        lines.append(
            f"FAIL: {n} loss(es). This gate is unwaivable — no `# gate-ok:`, "
            f"no --max, no env override (see module docstring). Fix each "
            f"finding via its REMEDY line above."
        )
    else:
        lines.append(
            "PASS — no orphaned requirements, deleted traced symbols, "
            "or deleted spec/ADR documents still cited."
        )
    return "\n".join(lines)


def _report_to_dict(report: FeatureLossReport) -> dict[str, object]:
    return {
        "status": report.status,
        "reason": report.reason,
        "losses": [asdict(loss) for loss in report.losses],
        "never_implemented": [asdict(n) for n in report.never_implemented],
        "stats": report.stats,
        "reclassified": report.reclassified,
        "stale_evidence": report.stale_evidence,
        "scan_scope": report.scan_scope,
        # SURFACE-LOSS (HEURISTIC, INFO-ONLY) — module docstring. Always the
        # FULL, uncapped list in --json output (the text report's inline cap
        # is a rendering convenience only, never a data-loss truncation).
        "surface_losses": [asdict(sl) for sl in report.surface_losses],
        "surface_reclassified": report.surface_reclassified,
        "surface_stale_evidence": report.surface_stale_evidence,
    }


def main(argv: list[str] | None = None) -> int:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument(
        "--repo-root", default=None, help="Directory to gate (default: cwd)"
    )
    parser.add_argument(
        "--json", action="store_true", help="Machine-readable output (full lists)"
    )
    args = parser.parse_args(argv)

    repo_root = Path(args.repo_root).resolve() if args.repo_root else Path.cwd()
    report = run_check(repo_root)

    if args.json:
        print(json.dumps(_report_to_dict(report), indent=2, default=str))
    else:
        print(render_report(report))

    if report.status == "invalid":
        return EXIT_INVALID
    return EXIT_FAIL if report.losses else EXIT_PASS


if __name__ == "__main__":
    sys.exit(main())