Orchestrata Ultimate › Plans › briefs › qwen › Q1-toolchain-declaration

Q1 — One C/C++ toolchain declaration: finish, test and verify

Source: plans/briefs/qwen/Q1-toolchain-declaration.md at 8a2cb42ad · last changed 2026-10-11 · first committed 2026-10-11 · 187 lines.

Prose pending.

Plan text

Q1 — One C/C++ toolchain declaration: finish, test and verify

1. Directive and why

Mohamed (verbatim): "make the gcc path a configuration and set it".

On this host ~/.guix-profile/bin leads PATH, so a bare gcc resolves to Guix GCC 16.2.0, and it fails the vendored janus-swi native build against the system swipl headers (two worktree provisions failed this way, 2026-10-11). Before this change, four places each picked a compiler of their own: .envrc exported the bare CC=gcc CXX=g++; deploy/install-orchestrata.sh defaulted to bare gcc and re-derived /usr/bin/gcc with a PATH-shadow probe; deploy/install-tools.sh hardcoded /usr/bin/gcc; scripts/worktree_toolchain.py pinned a hardcoded /usr/bin/gcc only when shadowed. The change makes one tracked file, env/toolchain.env, the only place a compiler is named. Every consumer reads it, and every consumer refuses a missing, bare-named or non-executable compiler rather than fall back to PATH.

Most of the change is already written (by the orchestrator, before this executor model was adopted) and is uncommitted in the worktree. The new tests were written but never run. Your job: complete it, run everything, fix what fails, and stage it.

2. Rules (project law — violations are rejected)

1. Never run bare python, python3, pytest or pip. Always env -u PYTHONPATH .venv/bin/python ... from the worktree root. (A bare python on this host resolves to a stale checkout ~530 commits behind.)
2. uv only; no new dependencies; do not edit pyproject.toml / uv.lock.
3. Tests: no unittest.mock/Mock/MagicMock/patch/monkeypatch, no Fake/Stub/Dummy classes, no skip/skipif/xfail, no xdist. Function-scope fixtures; tmp_path for files. The installer tests run the REAL bash deploy/install-orchestrata.sh with an env= dict (that is the injection seam; never mutate os.environ). The existing tests use a recording uv script on a fake PATH directory: that is an existing real-subprocess seam the project already uses, keep it as is.
4. Absolute imports only. Never write the word "should" in code, comments or docstrings.
5. Never delete a function, test or helper. If one becomes unused, leave it and list it in the report as a removal candidate (only Mohamed authorises removals).
6. Git: do NOT commit, push, merge, rebase, reset, stash or switch branches. git add the files in §4 at §7 step 4.
7. Stay inside /home/mohamed/workspace/wt-BARE-PYTHON. Never touch 10.53.1.x (production). Never run the full suite; never run scripts/run_tests.py without explicit paths.
8. If the code contradicts this spec, or a test proves the shipped code wrong, fix the CODE per §3 (never weaken a test). If a design choice here looks wrong, STOP and say why in the report.

3. The design (already implemented — verify it, do not redesign)

• env/toolchain.env (NEW, untracked): CC=/usr/bin/gcc, CXX=/usr/bin/g++ plus a header comment.
• deploy/toolchain.sh (NEW, untracked, SOURCED): load_toolchain <declaration-file> reads CC/CXX, refuses (return 1, one FAIL: line on stderr) when the file is missing (FAIL: no toolchain declaration at <file>), a key is missing (FAIL: <file> declares no CXX), a value is not absolute (... a bare name is resolved from PATH; declare an absolute path) or not executable (..., which is not an executable); on success exports CC/CXX, unsets C_INCLUDE_PATH CPLUS_INCLUDE_PATH, prints toolchain: CC=<cc> CXX=<cxx> (from <file>).
• deploy/install-orchestrata.sh: the old block (bare gcc default + PATH-shadow probe + toolchain override: line) is replaced by source "$INSTALLER_ROOT/deploy/toolchain.sh" and load_toolchain "${ORCHESTRATA_TOOLCHAIN_ENV:-$INSTALLER_ROOT/env/toolchain.env}" || exit 1, where INSTALLER_ROOT is the checkout the installer script lives in (NOT $SOURCE_REPO).
• deploy/install-tools.sh: same loader; the native build passes CC="$CC" CXX="$CXX" through $SUDO env.
• .envrc: the export CC=gcc / export CXX=g++ lines are gone; dotenv "env/toolchain.env" is the LAST line of the file, so neither .env nor a profile overlay can override it.
• scripts/worktree_toolchain.py: declared_toolchain(path) (raises ToolchainDeclarationError) and pin_system_toolchain(env) (applies the declaration UNCONDITIONALLY, drops the two Guix include vars, prints toolchain: CC=<cc> CXX=<cxx> (from <decl>) plus , dropped C_INCLUDE_PATH[, CPLUS_INCLUDE_PATH] when it dropped any). Declaration = env["ORCHESTRATA_TOOLCHAIN_ENV"] when set, else DEFAULT_TOOLCHAIN_DECLARATION.
• scripts/worktrees.py: provisioning calls the above.
• scripts/build_janus_ft.py and scripts/build_ft_tree_sitter.py: their _pin_system_toolchain already delegates to worktree_toolchain.pin_system_toolchain (loaded by file path), so they ALREADY follow the declaration. Their docstrings/comments still describe the old behaviour — fix those words (§4).
• scripts/hooks/pre-push: banner text only — it describes the already-committed c1eebd019 change (an env/*.env edit resolves to the files naming its keys, never escalates). It is uncommitted: do not edit it, stage it with the rest.

4. What you change

Stage at the end: everything in §3 and §4 that is modified or new (git status --porcelain lists them), EXCEPT plans/briefs/qwen/ and any generated file you did not edit by hand (docs/CODEBASE_MAP*.md, plans/ast-*.json, context.json, reports/* — if they appear modified, leave them unstaged and list them in the report).

5. Tests

Read first: the two test modules in full, and tests/deploy/_install_orchestrata_support.py. The installer test harness already exists: run_install(tmp_path, source_repo, ...args, env_overrides={...}) runs the real installer with --bootstrap-only; _write_recording_uv(fakebin, record_file) puts a uv first on PATH that records the CC, CXX and C_INCLUDE_PATH it was called with (<unset> when unset) instead of syncing. INSTALLER_ROOT in these runs is THIS worktree, so the declaration read by default is THIS worktree's env/toolchain.env (CC=/usr/bin/gcc, CXX=/usr/bin/g++).

5.1 tests/deploy/test_install_orchestrata_cc.py

1. Replace test_sync_stage_exports_cc_and_cxx_with_gcc_default with test_sync_stage_uses_the_declared_compilers: no CC/CXX in the installer environment, PATH from _path_with_only_system_gcc(fakebin). Assert: rc 0 for the bootstrap stage reached (record_file exists), recorded CC=/usr/bin/gcc and CXX=/usr/bin/g++, and stdout contains toolchain: CC=/usr/bin/gcc CXX=/usr/bin/g++ (from followed by the path of this worktree's env/toolchain.env (compute it from the test file's location: Path(__file__).resolve().parents[2] / "env" / "toolchain.env"). Docstring: WHY (a systemd user unit has no direnv; the compiler comes from the declaration, never from PATH).
2. Replace test_sync_stage_fails_closed_without_a_compiler_on_path with test_sync_stage_refuses_an_unusable_declaration, parametrized over 3 cases, each pointing ORCHESTRATA_TOOLCHAIN_ENV (via env_overrides) at a file you write under tmp_path:
   • missing: the path does not exist → stderr/stdout contains FAIL: no toolchain declaration at;
   • bare-name: CC=gcc\nCXX=g++\n → contains a bare name is resolved from PATH;
   • not-executable: CC=<tmp_path>/no-such-cc\nCXX=/usr/bin/g++\n → contains which is not an executable. For every case assert returncode != 0 and not record_file.exists() (proves uv sync never ran). Use the real PATH plus fakebin first (the declaration, not PATH, decides).

5.2 tests/deploy/test_install_orchestrata_toolchain.py

1. Replace test_sync_stage_pins_system_gcc_when_shadowed with test_a_shadowing_gcc_on_path_does_not_change_the_compiler: _fakebin_with_shadow_gcc(fakebin) first on PATH and C_INCLUDE_PATH=/tmp/leak in env_overrides. Assert recorded CC=/usr/bin/gcc, CXX=/usr/bin/g++, C_INCLUDE_PATH=<unset>, stdout contains toolchain: CC=/usr/bin/gcc, and toolchain override: appears NOWHERE in the output (the PATH probe no longer exists).
2. Replace test_sync_stage_no_override_when_gcc_already_system with test_an_inherited_bare_cc_is_overridden_by_the_declaration: env_overrides={"PATH": _path_with_only_system_gcc(fakebin), "CC": "gcc", "CXX": "g++"}. Assert recorded CC=/usr/bin/gcc and CXX=/usr/bin/g++ (an inherited bare CC=gcc is exactly what .envrc used to export, and it is the failure this change removes). _path_with_only_system_gcc is defined in test_install_orchestrata_cc.py; reuse the existing import pattern of this module (read its import block — it already imports from that module or from _install_orchestrata_support; do not duplicate the helper).

5.3 tests/deploy/test_envrc_toolchain.py (NEW)

Text-level contract tests over the repo's real .envrc (it is a shell file, not Python; reading it is correct):

1. test_envrc_exports_no_compiler: no line matching ^\s*export\s+(CC|CXX)= exists.
2. test_envrc_loads_the_toolchain_declaration_last: the LAST line that calls dotenv (ignore comments/blank lines) is exactly dotenv "env/toolchain.env".
3. test_the_declaration_names_absolute_executables: parse env/toolchain.env with worktree_toolchain.declared_toolchain (import it the way tests/scripts/test_worktrees_provision.py does) and assert both values are absolute and executable on this host. Header docstring with WHY and the Test Standards Compliance block (copy the shape from tests/deploy/test_toolchain_sh.py).

6. Run — exact commands, in order, from the worktree root

1. Every directly affected module, fast and infrastructure-free:

   env -u PYTHONPATH .venv/bin/python -m pytest -q -p no:cacheprovider \
     tests/deploy/test_toolchain_sh.py tests/deploy/test_envrc_toolchain.py \
     tests/deploy/test_install_orchestrata_cc.py tests/deploy/test_install_orchestrata_toolchain.py \
     tests/scripts/test_worktrees_provision.py tests/scripts/test_build_janus_ft.py \
     tests/scripts/test_build_ft_tree_sitter.py
   

   All pass, 0 skipped. test_toolchain_sh.py and test_worktrees_provision.py were written but NEVER run: expect failures there and fix them (a test asserting the §3 behaviour is right; a test asserting something §3 does not say is wrong — report which and why).
2. Real behaviour, outside pytest (paste outputs into the report):
   • bash -c 'source deploy/toolchain.sh && load_toolchain env/toolchain.env && echo "CC=$CC CXX=$CXX CIP=${C_INCLUDE_PATH-unset}"'
   • env C_INCLUDE_PATH=/tmp/leak CC=gcc .venv/bin/python -c 'import os, sys; sys.path.insert(0, "scripts"); import worktree_toolchain as t; e = t.pin_system_toolchain(dict(os.environ)); print(e["CC"], e["CXX"], "C_INCLUDE_PATH" in e)' → expect a toolchain: ... dropped C_INCLUDE_PATH line, then /usr/bin/gcc /usr/bin/g++ False.
3. Lint/format the Python files you changed: env -u PYTHONPATH .venv/bin/python -m ruff check --fix <files> then env -u PYTHONPATH .venv/bin/python -m black <files>; bash -n deploy/toolchain.sh deploy/install-orchestrata.sh deploy/install-tools.sh.
4. Stage (§4 rule), then git status --porcelain and paste it.
5. Review gate exactly as the pre-commit hook runs it: env -u PYTHONPATH .venv/bin/python scripts/gate.py review-staged; echo "review rc=$?" → must be 0. rc 1: fix the findings and repeat. rc 2: report the reasons, stop.
6. Bounded run over the blast radius. Another executor works in a sibling worktree and runs its own bounded run; two at once overload the shared test fleet. So FIRST wait until no other run is in progress (bounded, 60 min): timeout 3600 bash -c 'while pgrep -f "scripts/run_tests.py" >/dev/null; do sleep 15; done'; echo "wait rc=$?" (rc 124 = still busy after 60 min: STOP and report). Then resolve, then run exactly those paths:

   env -u PYTHONPATH .venv/bin/python scripts/affected_tests.py --json --files $(git diff --cached --name-only | tr '\n' ' ') > .q1-affected.json
   env -u PYTHONPATH .venv/bin/python scripts/run_tests.py $(env -u PYTHONPATH .venv/bin/python -c 'import json; print(" ".join(json.load(open(".q1-affected.json"))["tests"]))'); echo "run rc=$?"
   rm .q1-affected.json
   

   Required before running: "escalate": false, "untested": [] (an env/*.env file resolves to the files that name its keys — CC/CXX — per commit c1eebd019; if it escalates anyway, STOP and paste the reasons). The run must end 0 failed, 0 errors, rc 0. ONE run; do not start a second while one is in progress.

7. Done when — then write plans/briefs/qwen/Q1-report.md

• [ ] §6.1 passes, 0 skipped; §6.2 outputs as expected.
• [ ] ruff/black clean; bash -n clean on the three shell files.
• [ ] only §4-allowed files staged; gate.py review-staged rc 0.
• [ ] §6.6 resolver did not escalate; the run is rc 0, 0 failed, 0 errors.

Report (plain facts, no "should"): each file changed and why; every test you had to fix in the pre-written modules and whether the code or the test was wrong; §6 outputs; any helper that became unused; anything in this spec you found wrong.

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks