scripts.worktree_toolchain

Reference › scripts › worktree_toolchain

scripts/worktree_toolchain.py holds toolchain pinning and locked-package provenance checks for worktrees; it is a library, not a command. tests/deploy/test_install_orchestrata_cc.py and tests/scripts/test_build_janus_ft.py cover it. declared_toolchain reads a declaration file and returns the CC, CXX and SWIPL it states. pin_system_toolchain returns a new environment whose CC, CXX and SWIPL are that declared toolchain and from which the Guix include-path variables are dropped, and prints one line naming the declaration and what was dropped; the declaration is the file named by an environment variable, else a default (ADR-0091), and an unusable declaration raises ToolchainDeclarationError. It applies unconditionally, since an inherited bare CC=gcc or SWIPL=swipl resolves from PATH and is the same defect. lock_path_packages lists the distributions uv.lock sources locally, and check_locked_packages_importable imports each from the worktree's own virtual environment, returning 1 and naming the package when the import fails, because uv sync can report success after a silent build failure. It returns 0 when uv.lock is absent.

At a glance

Class diagram

Dependency diagram

No internal imports and no importers.

API

Classes

ToolchainDeclarationError

Bases: RuntimeError · Line 60

The toolchain declaration is missing or declares something unusable.

Functions

Source

Permalink: https://gitlab.com/MSamy/orchestrata-ultimate-architecture/-/blob/a4cbc2bdf/scripts/worktree_toolchain.py

#!/usr/bin/env python3
"""Toolchain pinning + locked-package provenance for scripts/worktrees.py.

Split out of worktrees.py (RUL-021's provisioning module) purely to keep
that file under its line budget — see its own history for the split
commit. Two responsibilities, both closing the same class of failure: `uv
sync --dev` reporting SUCCESS while a native extension silently never got
built.

- ``pin_system_toolchain``: on this host (and any host with a Guix
  profile on PATH) ``~/.guix-profile/bin/gcc`` shadows ``/usr/bin/gcc``,
  and Guix's ``C_INCLUDE_PATH`` leaks its ``math.h`` ahead of the 3.14t
  headers (an ``_Float32`` redefinition clash). ``uv sync --dev`` under
  that environment silently drops every native extension it tries to
  build from source (measured 2026-09-23: janus_swi, cxpt, ctok,
  codemod-fast) while still exiting 0 — the worktree looks provisioned
  and every test importing the Prolog bridge dies with
  ``ModuleNotFoundError``. Pinning ``CC``/``CXX`` to the system compiler
  and dropping the Guix include-path vars builds all four in ~11s.
  The same declaration carries ``SWIPL`` (ADR-0091): the janus fork builds
  against the declared SWI-Prolog and bakes ITS library directory as the
  extension's RUNPATH, so a Guix profile link — never a PATH-resolved
  ``swipl`` and never the mortal ``/gnu/store`` item it points at — decides
  which libswipl the built extension loads at run time.
- ``lock_path_packages`` / ``check_locked_packages_importable``: the
  general form of the same proof — ANY package uv.lock sources locally
  (``path``, ``directory``, or ``editable``, i.e. not the registry) must
  actually import from the venv ``uv sync`` just produced. This extends
  RUL-021's existing provenance proofs (``_prove_provenance`` et al. in
  worktrees.py) rather than forking them: those proofs check the ONE
  package (``orchestrata_ultimate``) worktrees.py already knew to check;
  this walks every locally-sourced package uv.lock actually names.
"""

from __future__ import annotations

import os
import subprocess
import sys
import tomllib
from pathlib import Path

#: The ONE toolchain declaration (env/toolchain.env), read from the checkout this
#: script runs from: the declaration ships with the tool, exactly as
#: deploy/toolchain.sh reads it from the installer's own checkout.
DEFAULT_TOOLCHAIN_DECLARATION = (
    Path(__file__).resolve().parents[1] / "env" / "toolchain.env"
)
#: Names another declaration FILE (the test seam). Its values are validated the
#: same way, so it cannot reintroduce a PATH-resolved compiler.
TOOLCHAIN_ENV_VAR = "ORCHESTRATA_TOOLCHAIN_ENV"
#: ``SWIPL`` is a declared tool exactly like the compilers (ADR-0091): the
#: janus fork builds against it and bakes ITS library directory as the
#: extension's RUNPATH, so a swipl resolved from PATH is a build linked against
#: whatever the environment happened to carry.
_TOOLCHAIN_KEYS = ("CC", "CXX", "SWIPL")
_GUIX_INCLUDE_VARS = ("C_INCLUDE_PATH", "CPLUS_INCLUDE_PATH")


class ToolchainDeclarationError(RuntimeError):
    """The toolchain declaration is missing or declares something unusable."""


def declared_toolchain(declaration: Path) -> dict[str, str]:
    """``{"CC": ..., "CXX": ..., "SWIPL": ...}`` as ``declaration`` states them.

    Raises :class:`ToolchainDeclarationError` when the file is missing, omits
    a key, declares a bare name (resolved from PATH — on this host the Guix
    GCC 16.2.0 that fails the janus-swi build, or a ``swipl`` that is not the
    declared install) or a path that is not an executable. There is no
    fallback tool: a build without a usable declaration is refused, never
    guessed.

    Values are returned VERBATIM — never ``.resolve()``d. ``SWIPL`` is a Guix
    profile link on purpose: the profile is a GC root and the store item it
    points at is not, so resolving it would hand the fork's build the mortal
    path that ADR-0091 exists to remove.
    """
    try:
        lines = declaration.read_text(encoding="utf-8").splitlines()
    except OSError as exc:
        raise ToolchainDeclarationError(
            f"no toolchain declaration at {declaration}: {exc}"
        ) from exc
    declared: dict[str, str] = {}
    for line in lines:
        key, sep, value = line.partition("=")
        if sep and key in _TOOLCHAIN_KEYS:
            declared[key] = value.strip()
    for key in _TOOLCHAIN_KEYS:
        value = declared.get(key, "")
        if not value:
            raise ToolchainDeclarationError(f"{declaration} declares no {key}")
        if not Path(value).is_absolute():
            raise ToolchainDeclarationError(
                f"{declaration} declares {key}={value} — a bare name is resolved "
                "from PATH; declare an absolute path"
            )
        if not os.access(value, os.X_OK):
            raise ToolchainDeclarationError(
                f"{declaration} declares {key}={value}, which is not an executable"
            )
    return declared


def pin_system_toolchain(env: dict[str, str]) -> dict[str, str]:
    """Return a NEW ``env`` whose ``CC``/``CXX``/``SWIPL`` are the declared
    toolchain and whose Guix include-path variables are dropped; print one
    line naming the declaration and what was dropped.

    The declaration is ``env[ORCHESTRATA_TOOLCHAIN_ENV]`` when set, else
    :data:`DEFAULT_TOOLCHAIN_DECLARATION`. Raises
    :class:`ToolchainDeclarationError` when it is unusable.

    Until 2026-10-11 this pinned a hardcoded ``/usr/bin/gcc`` only when a
    different ``gcc`` led PATH, and otherwise returned ``env`` untouched —
    leaving an inherited bare ``CC=gcc`` (exported by ``.envrc``) to resolve
    from PATH. The declaration now applies unconditionally, and to ``SWIPL``
    too: an inherited bare ``SWIPL=swipl`` (what ``env/test.env`` carries) is
    the same defect, and the fork's build refuses it (ADR-0091).
    """
    declaration = Path(env.get(TOOLCHAIN_ENV_VAR) or DEFAULT_TOOLCHAIN_DECLARATION)
    pinned = dict(env)
    pinned.update(declared_toolchain(declaration))
    dropped = [v for v in _GUIX_INCLUDE_VARS if pinned.pop(v, None) is not None]
    dropped_msg = f", dropped {', '.join(dropped)}" if dropped else ""
    print(
        f"toolchain: CC={pinned['CC']} CXX={pinned['CXX']} "
        f"SWIPL={pinned['SWIPL']} (from {declaration}){dropped_msg}"
    )
    return pinned


def lock_path_packages(lock_path: Path) -> list[str]:
    """Distribution names uv.lock sources locally — ``path``, ``directory``
    or ``editable`` — rather than from a registry. These are the packages a
    silently-broken build (wrong compiler, missing headers, ...) can drop
    from the venv while ``uv sync`` still exits 0."""
    data = tomllib.loads(lock_path.read_text())
    names: list[str] = []
    for pkg in data.get("package", []):
        source = pkg.get("source")
        if isinstance(source, dict) and any(
            key in source for key in ("path", "directory", "editable")
        ):
            names.append(pkg["name"])
    return names


def check_locked_packages_importable(wt_path: Path, env: dict[str, str]) -> int:
    """FAILS LOUD (returns 1, prints to stderr naming the package and the
    venv) when any locally-sourced package uv.lock names is not importable
    from ``wt_path/.venv`` — the check `uv sync --dev` doesn't make for us.
    Returns 0 when uv.lock is absent (nothing to check yet) or every locally
    -sourced package imports cleanly."""
    lock_path = wt_path / "uv.lock"
    if not lock_path.exists():
        return 0
    venv = wt_path / ".venv"
    wt_python = venv / "bin" / "python"
    for name in lock_path_packages(lock_path):
        module = name.replace("-", "_")
        probe = subprocess.run(
            [str(wt_python), "-c", f"import {module}"],
            cwd=wt_path,
            env=env,
            capture_output=True,
            text=True,
            timeout=60,
        )
        if probe.returncode != 0:
            print(
                f"ERROR: locked path package {name!r} (module {module!r}) is "
                f"NOT importable from {venv} — uv sync reported success but "
                f"the build silently failed. stderr: "
                f"{probe.stderr.strip()[-500:]}",
                file=sys.stderr,
            )
            return 1
    print(f"locked path packages OK: all importable from {venv}")
    return 0


if __name__ == "__main__":
    # Minimal CLI so deploy/install-orchestrata.sh's post-sync stage can
    # run this proof against a plain prefix path (no worktree layout
    # required -- check_locked_packages_importable only ever needs
    # wt_path/uv.lock and wt_path/.venv, both of which a deploy prefix has
    # too). `env` is the current process's own env, i.e. exactly what the
    # calling shell exported (CC/CXX pin included, though the import probe
    # itself does not care about either).
    sys.exit(check_locked_packages_importable(Path(sys.argv[1]), dict(os.environ)))