scripts.check_external_tools
Reference › scripts › check_external_tools
The external-tools gate. It enforces the [tool.orchestrata.external-tools] table in pyproject.toml against two things: the tools actually installed, and the executables the code actually invokes. tests/scripts/test_check_external_tools.py imports it.
Invocation: main builds its parser in build_parser; it takes --repo-root (default the current directory), --path to override the search path, and --json. The exit code comes from the report, and a malformed table (GateError) returns 2.
Two checks run. The probe: load_declared_tools reads each declared ToolSpec, and probe_tool finds the executable on the path and compares its version with the declared minimum (parse_version, version_at_least), giving a ProbeResult. The scan: scan_repo walks the Python files outside the excluded directories, and scan_file finds subprocess calls whose first argument names an executable (_iter_subprocess_calls, _classify_argv0). Calls through the running interpreter are ignored (_is_sys_executable). find_undeclared then lists every invoked executable that is neither declared nor one of the project's own script names. build_report and print_text_report produce the output.
At a glance
| Source | scripts/check_external_tools.py |
| Lines of code | 668 |
| Classes | 5 |
| Functions | 20 |
| Imports (internal) | none |
| Imported by | none |
| Tests that import it | tests/scripts/test_check_external_tools.py |
Class diagram
Dependency diagram
No internal imports and no importers.
API
Constants
| Name | Type | Value |
|---|---|---|
_ALLOWED_EXECUTABLES |
dict[str, str] |
{'python': "the interpreter itself — versioned by the project's uv/pyproject pi… |
Classes
GateError
Bases: Exception · Line 124
The declaration itself cannot be read/parsed — exit 2, never a silent pass.
ToolSpec (dataclass)
Decorators: dataclass(frozen=True) · Line 134
One row of [tool.orchestrata.external-tools].
| Attribute | Type | Default |
|---|---|---|
name |
str |
|
min_version |
str |
|
install |
str |
|
purpose |
str |
'' |
homepage |
str |
'' |
license |
str |
'' |
version_stdin |
str | None |
None |
ProbeResult (dataclass)
Decorators: dataclass · Line 272
| Attribute | Type | Default |
|---|---|---|
name |
str |
|
found |
bool |
|
version |
str | None |
|
version_ok |
bool |
|
detail |
str |
Invocation (dataclass)
Decorators: dataclass · Line 348
| Attribute | Type | Default |
|---|---|---|
executable |
str |
|
file |
str |
|
line |
int |
ScanResult (dataclass)
Decorators: dataclass · Line 355
| Attribute | Type | Default |
|---|---|---|
invocations |
list[Invocation] |
field(default_factory=list) |
skipped |
int |
0 |
Functions
| Function | Signature | Summary |
|---|---|---|
_load_toml |
_load_toml(path: Path) -> dict |
Read + parse a TOML file, raising GateError on any failure — the one place every direction-1 reader below goes, so a bad pyproject.toml fails the SAME way everywhere instead of each caller… |
load_declared_tools |
load_declared_tools(pyproject_path: Path) -> dict[str, ToolSpec] |
Parse [tool.orchestrata.external-tools]; raise GateError on any read/parse/shape failure — never returns a partial or guessed table. |
load_project_script_names |
load_project_script_names(pyproject_path: Path) -> frozenset[str] |
Every key in [project.scripts] — this project's OWN console entry points, installed by the same uv sync that installs everything else, never a separate external dependency. |
load_excluded_dirs |
load_excluded_dirs(pyproject_path: Path) -> frozenset[str] |
[tool.ruff] extend-exclude, read LIVE — the same vendored/generated paths ruff already skips (scripts/_janus_patched is a patched free-threaded janus-swi fork; a Windows-only reg call… |
parse_version |
parse_version(text: str) -> tuple[int, ...] |
First dotted-number run in text, as a tuple of ints. |
version_at_least |
version_at_least(actual: tuple[int, ...], minimum: tuple[int, ...]) -> bool |
Tuple comparison, zero-padded to equal length. |
probe_tool |
probe_tool(spec: ToolSpec, path_env: str) -> ProbeResult |
Resolve spec.name against path_env and grade its version. |
_is_sys_executable |
_is_sys_executable(node: ast.expr) -> bool |
sys.executable — an explicit ignore, not a blind spot: what it resolves to is known (this interpreter), so it is never counted as skipped. |
_classify_argv0 |
_classify_argv0(node: ast.expr) -> tuple[str, str | None] |
Classify a call's first argument as ("sys_executable", None), ("literal", name), or ("unresolved", None). |
_collect_subprocess_names |
_collect_subprocess_names(tree: ast.Module) -> tuple[set[str], set[str]] |
Return (aliases bound to the subprocess module, names imported directly from it that are also one of the four call names). |
_iter_subprocess_calls |
_iter_subprocess_calls(tree: ast.Module) -> list[ast.Call] |
|
_first_arg |
_first_arg(call: ast.Call) -> ast.expr | None |
The command argument: positional args[0], or the args= keyword (subprocess.Popen commonly passes it by name). |
scan_file |
scan_file(path: Path, repo_root: Path) -> ScanResult |
Every subprocess call site in one file, classified. |
_is_excluded |
_is_excluded(rel_to_repo: Path, excluded_dirs: frozenset[str]) -> bool |
True when rel_to_repo (POSIX, relative to the repo root) is inside one of excluded_dirs — a path equal to, or nested under, an excluded entry. |
scan_repo |
scan_repo(repo_root: Path, excluded_dirs: frozenset[str] = frozenset()) -> ScanResult |
Scan every .py file under scripts/ and src/ (never tests/, never a generated build/ dir, never a path under excluded_dirs — the caller's [tool.ruff] extend-exclude, e.g.… |
find_undeclared |
find_undeclared(scan: ScanResult, declared: dict[str, ToolSpec], script_names: frozenset[str] = frozenset()) -> list[Invocation] |
Invocations naming an executable in none of: the declared table, the allowlist, or script_names (this project's own [project.scripts] console entry points) — the actual defect this direction… |
build_report |
build_report(probes: list[ProbeResult], undeclared: list[Invocation], skipped: int, declared: dict[str, ToolSpec]) -> dict |
|
print_text_report |
print_text_report(report: dict) -> None |
|
build_parser |
build_parser() -> argparse.ArgumentParser |
|
main |
main(argv: list[str] | None = None) -> int |
Source
#!/usr/bin/env python3
"""External-tools gate — enforces `[tool.orchestrata.external-tools]` against
reality, in BOTH directions (RUL-014: a declaration nothing checks is prose,
not a mechanism).
That table declares every NON-PyPI executable this project's own source shells
out to, with the minimum version it is known to work against and the command
that installs it. ``check-deps`` (``serialize_codebase_ast.py check-deps``)
gates Python IMPORTS; it is structurally blind to a Go/Rust/C binary someone
installed by hand, which is exactly how an undeclared external dependency
enters unnoticed. This gate closes that hole:
1. DECLARED-BUT-MISSING. Every tool in the table must be on PATH and its
reported version must be >= ``min_version``. Version-string SHAPE is not
uniform across tools — measured: ``sops --version`` prints
``"sops 3.13.3 (latest)"``; ``age --version`` prints bare ``"1.3.1"`` — so
parsing is defensive: the first dotted-number run anywhere in the combined
stdout+stderr is taken as the version and compared as a TUPLE OF INTS,
never as a string (a string compare gets ``"1.10.0" > "1.9.0"`` backwards).
2. INVOKED-BUT-UNDECLARED. Walks this repo's own ``scripts/`` and ``src/``
trees (never ``tests/`` — tests legitimately shell out to throwaway
binaries, and never a path under ``[tool.ruff] extend-exclude`` — read
live from pyproject.toml, same vendored/generated exclusion ruff already
owns) with the stdlib ``ast`` module for
``subprocess.{run,Popen,check_output,check_call}`` call sites, and fails
if a LITERAL ``argv[0]`` names an executable absent from all three of:
the declared table, the allowlist (POSIX baseline + venv-resolved Python
tooling + the interpreter/git/uv — see ``_ALLOWED_EXECUTABLES``), and
this project's own ``[project.scripts]`` console entry points (read live,
never hard-coded — see ``load_project_script_names``). Only a literal
``argv[0]`` — a bare string, or a list/tuple whose first element is a
string — can be resolved this way. A variable, an f-string, or any other
expression is skipped, and the COUNT of skips is reported alongside the
verdict, so the gate states its own blind spot instead of implying
completeness it does not have.
Exit 0 = every declared tool present & new enough, and no undeclared literal
invocation. Exit 1 = at least one violation, either direction. Exit 2 = the
declaration itself cannot be read/parsed at all — never a silent pass.
"""
from __future__ import annotations
import argparse
import ast
import json
import os
import re
import shutil
import subprocess
import sys
import tomllib
from dataclasses import dataclass, field
from pathlib import Path
RULE_ID = "external-tools"
# ---------------------------------------------------------------------------
# Executables this gate never expects to see declared in the table. Three
# groups, each with its own reason a future addition has to justify itself
# against.
# ---------------------------------------------------------------------------
_ALLOWED_EXECUTABLES: dict[str, str] = {
# ---- Hard prerequisites: already a documented requirement for running
# anything in this tree at all, independent of any dependency table. ----
"python": "the interpreter itself — versioned by the project's uv/pyproject pin",
"python3": "same as python — interpreter self-invocation under its POSIX name",
"git": "core VCS tooling — a hard prerequisite documented outside this table",
# ---- POSIX baseline: present on any Linux system this project runs on,
# not meaningful to version-pin, and declaring them in the external-tools
# table would be noise that trains people to ignore the gate. ----
"bash": "POSIX baseline — the shell scripts/*.sh and inline `bash -c` invocations run under",
"sh": "POSIX baseline — the portable shell some scripts invoke explicitly",
"tar": "POSIX baseline — archive extraction (e.g. agent_check.py unpacking a delivered artifact)",
"grep": "POSIX baseline — text search used by diagnostic/log-scanning scripts",
"pgrep": "POSIX baseline — process lookup by name (watchdogs, liveness checks)",
"pkill": "POSIX baseline — process termination by name, companion to pgrep",
"chmod": "POSIX baseline — permission bits on generated/throwaway files",
"cp": "POSIX baseline — file copy",
"mv": "POSIX baseline — file/directory rename-in-place",
"rm": "POSIX baseline — file/directory removal",
"mkdir": "POSIX baseline — directory creation",
"ln": "POSIX baseline — symlink/hardlink creation",
"kill": "POSIX baseline — signal delivery to a PID",
"sleep": "POSIX baseline — fixed-delay waits in polling/retry loops",
"true": "POSIX baseline — the no-op exit-0 command, used as a placeholder",
"env": "POSIX baseline — env-controlled subprocess launches (e.g. `env -u VAR cmd`)",
# ---- venv-resolved Python tooling: PyPI distributions already declared
# as project dependencies (dependencies / optional-dependencies.dev /
# dependency-groups.dev in pyproject.toml) and therefore already gated by
# `check-deps` (serialize_codebase_ast.py check-deps) against undeclared
# IMPORTS. Declaring the same package a SECOND time here — as a raw
# executable — would be a duplicate source of truth for the identical
# fact: exactly the failure mode this project's own research record names
# as its core defect class. ----
"ruff": "PyPI dev dependency (ruff, dependency-groups.dev) — already gated by check-deps",
"pyright": "PyPI dev dependency (pyright, dependency-groups.dev) — already gated by check-deps",
"black": "PyPI dev dependency (black, project.optional-dependencies.dev) — already gated by check-deps",
"mypy": "PyPI dev dependency (mypy, project.optional-dependencies.dev) — already gated by check-deps",
"pytest": "PyPI dev dependency (pytest, project.optional-dependencies.dev) — already gated by check-deps",
"coverage": "installed transitively via the declared PyPI pytest-cov dev dependency — already gated by check-deps",
"uv": "the project's own package manager — resolves from the same venv/toolchain check-deps already governs, not a second dependency to declare",
}
# Trees this gate scans for invocations. Deliberately excludes tests/: tests
# legitimately shell out to throwaway binaries that never belong in a
# production dependency table.
_TREES = ("scripts", "src")
# Directories whose contents are GENERATED, not source — same set
# check_native_build_freshness.py excludes. The vendored native-extension
# dirs (scripts/_janus_patched, _cxpt, _ctok, _codemod_fast) all gitignore a
# local `build/`; scanning it double-reports every finding already caught in
# the real source (confirmed via `git check-ignore` on
# scripts/_janus_patched/build/.../_find_swipl.py) and it would not even
# exist in a fresh clone.
_GENERATED_DIRS = frozenset({"build", ".git", "__pycache__"})
_VERSION_RE = re.compile(r"\d+(?:\.\d+)+")
_SUBPROCESS_CALLS = frozenset({"run", "Popen", "check_output", "check_call"})
class GateError(Exception):
"""The declaration itself cannot be read/parsed — exit 2, never a silent pass."""
# ---------------------------------------------------------------------------
# The declared table
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class ToolSpec:
"""One row of ``[tool.orchestrata.external-tools]``."""
name: str
min_version: str
install: str
purpose: str = ""
homepage: str = ""
license: str = ""
# Some tools have no working ``--version`` (CLIPS is a REPL that waits on
# stdin and hangs the probe); ``version_stdin`` runs the bare executable
# and feeds this string to stdin instead, then parses the output banner.
version_stdin: str | None = None
def _load_toml(path: Path) -> dict:
"""Read + parse a TOML file, raising ``GateError`` on any failure — the
one place every direction-1 reader below goes, so a bad pyproject.toml
fails the SAME way everywhere instead of each caller reimplementing its
own partial-read/parse-error handling."""
try:
raw = path.read_bytes()
except OSError as exc:
raise GateError(f"cannot read {path}: {exc}") from exc
try:
return tomllib.loads(raw.decode("utf-8"))
except (tomllib.TOMLDecodeError, UnicodeDecodeError) as exc:
raise GateError(f"cannot parse {path}: {exc}") from exc
def load_declared_tools(pyproject_path: Path) -> dict[str, ToolSpec]:
"""Parse ``[tool.orchestrata.external-tools]``; raise ``GateError`` on any
read/parse/shape failure — never returns a partial or guessed table."""
data = _load_toml(pyproject_path)
table = data.get("tool", {}).get("orchestrata", {}).get("external-tools", {})
if not isinstance(table, dict):
raise GateError("[tool.orchestrata.external-tools] is not a table")
declared: dict[str, ToolSpec] = {}
for name, entry in table.items():
if not isinstance(entry, dict):
raise GateError(f"external-tools entry {name!r} is not a table")
if "min_version" not in entry or "install" not in entry:
raise GateError(
f"external-tools entry {name!r} is missing min_version/install"
)
try:
parse_version(str(entry["min_version"]))
except ValueError as exc:
raise GateError(
f"external-tools entry {name!r} has an unparseable min_version: {exc}"
) from exc
version_stdin = entry.get("version_stdin")
if version_stdin is not None and not isinstance(version_stdin, str):
raise GateError(
f"external-tools entry {name!r} has a non-string version_stdin"
)
declared[name] = ToolSpec(
name=name,
min_version=str(entry["min_version"]),
install=str(entry["install"]),
purpose=str(entry.get("purpose", "")),
homepage=str(entry.get("homepage", "")),
license=str(entry.get("license", "")),
version_stdin=version_stdin,
)
return declared
def load_project_script_names(pyproject_path: Path) -> frozenset[str]:
"""Every key in ``[project.scripts]`` — this project's OWN console entry
points, installed by the same ``uv sync`` that installs everything else,
never a separate external dependency. Read LIVE rather than hard-coded:
two of them (``session``, ``orchestrata-tools``) were flagged undeclared
before this existed, and hard-coding those two would only rot the moment
an eighth console script is added. This is the one allowlist tier this
gate derives instead of maintaining a duplicate copy of."""
data = _load_toml(pyproject_path)
scripts = data.get("project", {}).get("scripts", {})
if not isinstance(scripts, dict):
raise GateError("[project.scripts] is not a table")
return frozenset(scripts)
def load_excluded_dirs(pyproject_path: Path) -> frozenset[str]:
"""``[tool.ruff] extend-exclude``, read LIVE — the same vendored/generated
paths ruff already skips (``scripts/_janus_patched`` is a patched
free-threaded janus-swi fork; a Windows-only ``reg`` call inside it is
upstream's code, not this project's). Deriving the exclusion from the one
table ruff already owns, rather than hard-coding a second copy, is the
same non-duplication principle as ``load_project_script_names`` above:
the two lists would otherwise drift the moment someone edits one of
them."""
data = _load_toml(pyproject_path)
excluded = data.get("tool", {}).get("ruff", {}).get("extend-exclude", [])
if not isinstance(excluded, list):
raise GateError("[tool.ruff] extend-exclude is not a list")
return frozenset(str(item).rstrip("/") for item in excluded)
# ---------------------------------------------------------------------------
# Version parsing + comparison — numeric, never string
# ---------------------------------------------------------------------------
def parse_version(text: str) -> tuple[int, ...]:
"""First dotted-number run in ``text``, as a tuple of ints.
Defensive by design: tool ``--version`` output is not uniform (``sops``
wraps its number in a sentence, ``age`` prints it bare), so this searches
the whole string for the first ``\\d+(\\.\\d+)+`` run rather than assuming
a fixed position or field count.
"""
match = _VERSION_RE.search(text)
if not match:
raise ValueError(f"no version number found in {text!r}")
return tuple(int(part) for part in match.group(0).split("."))
def version_at_least(actual: tuple[int, ...], minimum: tuple[int, ...]) -> bool:
"""Tuple comparison, zero-padded to equal length.
NEVER a string comparison: ``"1.10.0" < "1.9.0"`` lexicographically, which
is exactly backwards. ``(1, 10, 0) >= (1, 9, 0)`` is correct.
"""
length = max(len(actual), len(minimum))
a = actual + (0,) * (length - len(actual))
m = minimum + (0,) * (length - len(minimum))
return a >= m
# ---------------------------------------------------------------------------
# Direction 1: declared-but-missing
# ---------------------------------------------------------------------------
@dataclass
class ProbeResult:
name: str
found: bool
version: str | None
version_ok: bool
detail: str
def probe_tool(spec: ToolSpec, path_env: str) -> ProbeResult:
"""Resolve ``spec.name`` against ``path_env`` and grade its version.
``path_env`` is an explicit parameter (never a bare ``os.environ`` read
inside this function) so a test can point resolution at an empty
directory to prove the "absent from PATH" branch without touching the
real ``PATH``.
"""
exe = shutil.which(spec.name, path=path_env)
if exe is None:
return ProbeResult(
name=spec.name,
found=False,
version=None,
version_ok=False,
detail=f"not on PATH — install: {spec.install}",
)
try:
# ``--version`` for normal tools; a bare invocation fed
# ``version_stdin`` for REPL-style tools (CLIPS hangs on --version).
argv = [exe] if spec.version_stdin is not None else [exe, "--version"]
proc = subprocess.run(
argv,
input=spec.version_stdin,
capture_output=True,
text=True,
timeout=10,
check=False,
)
except (OSError, subprocess.TimeoutExpired) as exc:
return ProbeResult(
name=spec.name,
found=True,
version=None,
version_ok=False,
detail=f"`{spec.name} --version` failed to run: {exc} — install: {spec.install}",
)
combined = proc.stdout + "\n" + proc.stderr
try:
actual = parse_version(combined)
except ValueError:
return ProbeResult(
name=spec.name,
found=True,
version=None,
version_ok=False,
detail=(
f"could not parse a version out of `{spec.name} --version` "
f"output ({combined.strip()!r}) — install: {spec.install}"
),
)
minimum = parse_version(spec.min_version)
version_str = ".".join(str(part) for part in actual)
ok = version_at_least(actual, minimum)
detail = (
f"{version_str} >= {spec.min_version}"
if ok
else f"{version_str} < {spec.min_version} — install: {spec.install}"
)
return ProbeResult(spec.name, True, version_str, ok, detail)
# ---------------------------------------------------------------------------
# Direction 2: invoked-but-undeclared
# ---------------------------------------------------------------------------
@dataclass
class Invocation:
executable: str
file: str
line: int
@dataclass
class ScanResult:
invocations: list[Invocation] = field(default_factory=list)
skipped: int = 0
def _is_sys_executable(node: ast.expr) -> bool:
"""``sys.executable`` — an explicit ignore, not a blind spot: what it
resolves to is known (this interpreter), so it is never counted as
skipped."""
return (
isinstance(node, ast.Attribute)
and node.attr == "executable"
and isinstance(node.value, ast.Name)
and node.value.id == "sys"
)
def _classify_argv0(node: ast.expr) -> tuple[str, str | None]:
"""Classify a call's first argument as ``("sys_executable", None)``,
``("literal", name)``, or ``("unresolved", None)``.
A bare string is treated as a single command name (its first
whitespace-separated token, for the rare ``"tool --flag"`` shell-string
form); a list/tuple literal's first element is the executable proper.
Anything else — a variable, an f-string, a call — is unresolved: this is
AST pattern matching, not data-flow analysis, and does not chase a value
back to its assignment.
"""
if _is_sys_executable(node):
return "sys_executable", None
if isinstance(node, ast.Constant) and isinstance(node.value, str):
tokens = node.value.split()
return ("literal", tokens[0]) if tokens else ("unresolved", None)
if isinstance(node, (ast.List, ast.Tuple)) and node.elts:
return _classify_argv0(node.elts[0])
return "unresolved", None
def _collect_subprocess_names(tree: ast.Module) -> tuple[set[str], set[str]]:
"""Return (aliases bound to the ``subprocess`` module, names imported
directly from it that are also one of the four call names).
Restricting matches to names actually traced back to ``subprocess`` — via
``import subprocess as sp`` or ``from subprocess import run`` — is what
keeps an unrelated ``pool.run(...)`` or ``thread.Popen`` out of the scan;
matching bare attribute/method names would false-positive on those.
"""
aliases: set[str] = set()
direct: set[str] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
if alias.name == "subprocess":
aliases.add(alias.asname or alias.name)
elif isinstance(node, ast.ImportFrom):
if node.module == "subprocess":
for alias in node.names:
if alias.name in _SUBPROCESS_CALLS:
direct.add(alias.asname or alias.name)
return aliases, direct
def _iter_subprocess_calls(tree: ast.Module) -> list[ast.Call]:
aliases, direct = _collect_subprocess_names(tree)
if not aliases and not direct:
return []
calls: list[ast.Call] = []
for node in ast.walk(tree):
if not isinstance(node, ast.Call):
continue
func = node.func
if (
isinstance(func, ast.Attribute)
and func.attr in _SUBPROCESS_CALLS
and isinstance(func.value, ast.Name)
and func.value.id in aliases
):
calls.append(node)
elif isinstance(func, ast.Name) and func.id in direct:
calls.append(node)
return calls
def _first_arg(call: ast.Call) -> ast.expr | None:
"""The command argument: positional ``args[0]``, or the ``args=`` keyword
(``subprocess.Popen`` commonly passes it by name)."""
if call.args:
return call.args[0]
for kw in call.keywords:
if kw.arg == "args":
return kw.value
return None
def scan_file(path: Path, repo_root: Path) -> ScanResult:
"""Every subprocess call site in one file, classified. Unreadable or
unparseable files contribute nothing rather than aborting the scan — a
broken symlink or a stray non-UTF8 file must not take the whole gate down
with it."""
result = ScanResult()
try:
text = path.read_text(encoding="utf-8")
except OSError, UnicodeDecodeError:
return result
try:
tree = ast.parse(text, filename=str(path))
except SyntaxError:
return result
rel = str(path.relative_to(repo_root))
for call in _iter_subprocess_calls(tree):
first = _first_arg(call)
if first is None:
continue
kind, literal = _classify_argv0(first)
if kind == "sys_executable":
continue
if kind != "literal" or not literal:
result.skipped += 1
continue
exe = Path(literal).name
result.invocations.append(Invocation(exe, rel, call.lineno))
return result
def _is_excluded(rel_to_repo: Path, excluded_dirs: frozenset[str]) -> bool:
"""True when ``rel_to_repo`` (POSIX, relative to the repo root) is inside
one of ``excluded_dirs`` — a path equal to, or nested under, an excluded
entry. Prefix-only: ``"scripts/_janus_patched"`` must not also swallow an
unrelated sibling like ``"scripts/_janus_patched_notes"``."""
rel_str = rel_to_repo.as_posix()
return any(
rel_str == excl or rel_str.startswith(f"{excl}/") for excl in excluded_dirs
)
def scan_repo(
repo_root: Path, excluded_dirs: frozenset[str] = frozenset()
) -> ScanResult:
"""Scan every ``.py`` file under ``scripts/`` and ``src/`` (never
``tests/``, never a generated ``build/`` dir, never a path under
``excluded_dirs`` — the caller's ``[tool.ruff] extend-exclude``, e.g.
``scripts/_janus_patched``: vendored upstream source, not this project's
own)."""
combined = ScanResult()
for tree_name in _TREES:
base = repo_root / tree_name
if not base.is_dir():
continue
for path in sorted(base.rglob("*.py")):
if _GENERATED_DIRS.intersection(path.relative_to(base).parts):
continue
if _is_excluded(path.relative_to(repo_root), excluded_dirs):
continue
per_file = scan_file(path, repo_root)
combined.invocations.extend(per_file.invocations)
combined.skipped += per_file.skipped
return combined
def find_undeclared(
scan: ScanResult,
declared: dict[str, ToolSpec],
script_names: frozenset[str] = frozenset(),
) -> list[Invocation]:
"""Invocations naming an executable in none of: the declared table, the
allowlist, or ``script_names`` (this project's own ``[project.scripts]``
console entry points) — the actual defect this direction exists to
catch."""
return sorted(
(
inv
for inv in scan.invocations
if inv.executable not in declared
and inv.executable not in _ALLOWED_EXECUTABLES
and inv.executable not in script_names
),
key=lambda inv: (inv.file, inv.line),
)
# ---------------------------------------------------------------------------
# Report
# ---------------------------------------------------------------------------
def build_report(
probes: list[ProbeResult],
undeclared: list[Invocation],
skipped: int,
declared: dict[str, ToolSpec],
) -> dict:
missing_or_outdated = [
{"name": p.name, "detail": p.detail} for p in probes if not p.version_ok
]
undeclared_dicts = [
{"executable": inv.executable, "file": inv.file, "line": inv.line}
for inv in undeclared
]
ok = not missing_or_outdated and not undeclared_dicts
return {
"rule_id": RULE_ID,
"ok": ok,
"exit_code": 0 if ok else 1,
"declared_tools": sorted(declared),
"probes": [
{
"name": p.name,
"found": p.found,
"version": p.version,
"version_ok": p.version_ok,
"detail": p.detail,
}
for p in probes
],
"missing_or_outdated": missing_or_outdated,
"undeclared_invocations": undeclared_dicts,
"non_literal_argv_skipped": skipped,
}
def print_text_report(report: dict) -> None:
declared = report["declared_tools"]
print(
f"[external-tools] {len(declared)} declared tool(s): "
f"{', '.join(declared) or '(none)'}"
)
for probe in report["probes"]:
status = "OK " if probe["version_ok"] else "FAIL"
print(f" {status} {probe['name']:10s} {probe['detail']}")
if report["missing_or_outdated"]:
print(
f"FAIL: {len(report['missing_or_outdated'])} declared tool(s) "
"missing or below min_version."
)
for inv in report["undeclared_invocations"]:
print(
f" UNDECLARED {inv['executable']!r} invoked at "
f"{inv['file']}:{inv['line']} — add it to "
"[tool.orchestrata.external-tools] or the gate's allowlist"
)
if report["undeclared_invocations"]:
print(
f"FAIL: {len(report['undeclared_invocations'])} undeclared "
"executable invocation(s)."
)
print(
f"[external-tools] {report['non_literal_argv_skipped']} non-literal "
"argv[0] site(s) skipped — not checked in either direction."
)
if report["ok"]:
print(
"PASS: every declared tool is present & new enough; "
"no undeclared invocations."
)
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="check_external_tools.py",
description=(
f"{RULE_ID} gate — declared NON-PyPI executables must be present "
"and new enough, and every literal subprocess invocation under "
"scripts/+src/ must be declared. Both directions of RUL-014."
),
)
parser.add_argument(
"--repo-root", default=None, help="Repository to scan (default: cwd)."
)
parser.add_argument(
"--path",
default=None,
help="Override PATH used to resolve declared tools (testing seam; "
"default: the process's own PATH).",
)
parser.add_argument("--json", action="store_true", help="Machine-readable output.")
return parser
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
repo_root = Path(args.repo_root).resolve() if args.repo_root else Path.cwd()
pyproject_path = repo_root / "pyproject.toml"
try:
declared = load_declared_tools(pyproject_path)
script_names = load_project_script_names(pyproject_path)
excluded_dirs = load_excluded_dirs(pyproject_path)
except GateError as exc:
print(f"ERROR: {exc}", file=sys.stderr)
return 2
path_env = args.path if args.path is not None else os.environ.get("PATH", "")
probes = [probe_tool(spec, path_env) for spec in declared.values()]
scan = scan_repo(repo_root, excluded_dirs)
undeclared = find_undeclared(scan, declared, script_names)
report = build_report(probes, undeclared, scan.skipped, declared)
if args.json:
print(json.dumps(report, indent=2, default=str))
else:
print_text_report(report)
return int(report["exit_code"])
if __name__ == "__main__":
raise SystemExit(main())