Orchestrata Ultimate › Plans › briefs › qwen › Q9-janus-swipl-declared

Q9 — the janus fork links the declared SWI-Prolog (ADR-0091)

Source: plans/briefs/qwen/Q9-janus-swipl-declared.md at 8a2cb42ad · last changed 2026-10-12 · first committed 2026-10-12 · 238 lines.

Prose pending.

Plan text

Q9 — the janus fork links the declared SWI-Prolog (ADR-0091)

DESIGN_WAIVER: ADR-0091 (Accepted by Mohamed 2026-10-12: "agree with adr"). Read docs/adr/0091-the-janus-fork-links-the-declared-swi-prolog-through-a-path-guix-gc-keeps-alive.md on branch origin/docs/adr-register-0080-0085 first: git show origin/docs/adr-register-0080-0085:docs/adr/0091-the-janus-fork-links-the-declared-swi-prolog-through-a-path-guix-gc-keeps-alive.md. It is the design; this brief is the build order. ADR-0089 (Accepted, already on this branch) is the pattern you extend: env/toolchain.env and its readers.

Worktree: /home/mohamed/workspace/wt-JANUS-SWIPL, branch feat/janus-swipl-declared, head fb441c9a7 (the integration branch). Work ONLY there.

Two phases. Phase A ends in a STOP. Phase B starts only on a continuation prompt.

0. Rules (binding, both phases)

1. A red test blocks completion, whoever owns it. Every failed, errored or skipped test needs a root cause with evidence (traceback, line, timing) and either a fix or a STOP naming the cause. "Outside my files", "pre-existing", "flaky", "infra" and "unrelated" are not statuses.
2. No mocks, stubs, fakes, monkeypatch, skip, xfail, or -n/xdist. Builds are real builds of the fork; libraries are the real Guix SWI-Prolog. The existing resolver seams (dump_runtime_resolver=, ldd_resolver=, env=, path_override= in janus_bootstrap) are the sanctioned way to force a tier; use them, add no new kind of seam.
3. Do NOT commit, merge, push, checkout, restore, reset, stash or clean. git add/show/diff/status/log are allowed. The orchestrator commits.
4. Read Python through scripts/serialize_codebase_ast.py symbols|source|refs|who-imports. The AST toolkit does NOT index the vendored fork scripts/_janus_patched/ (excluded from the AST roots); read those files directly — that is the one sanctioned exception. Reading a file you are about to edit is fine.
5. Absolute imports. os.environ.get(...). Typed signatures, docstrings. Keep the fork's diff against upstream minimal: change only the lines this brief names, in the fork's existing style (the fork is excluded from ruff/black on purpose — pyproject.toml extend-exclude).
6. Never write the word "should". Every number comes from a command run in this session, quoted next to it.
7. Production is read-only to you. You may readelf -d files under /opt/orchestrata. You never run promote_env.py, install-orchestrata.sh, ship, systemctl, incus, or anything that writes outside this worktree and its .venv. You never touch the test fleet.
8. Any change outside the files listed in a phase → STOP and report why.

0.1 Running tests

• Authoritative runs (the only ones that count toward done-when) go through the host lock, forced, no reuse: bash .run_exclusive.sh --force --json <module> ... > .q9/run-<label>.json 2> .q9/run-<label>.err; echo "run rc=$?"
• Quick iteration through the lock: bash .hostlock.sh env -u PYTHONPATH -u PYTEST_ADDOPTS timeout 300 .venv/bin/python -m pytest -q -p no:cacheprovider <module>
• Always env -u PYTEST_ADDOPTS alongside env -u PYTHONPATH (defect D18: the observer plugin in PYTEST_ADDOPTS is unimportable without its PYTHONPATH, and every pytest child dies at import).
• run_tests.py --json prints [freshness] lines before the JSON document; parse from the first { (defect D17).
• The affected set: env -u PYTHONPATH -u PYTEST_ADDOPTS .venv/bin/python scripts/affected_tests.py --json <changed files>. If it reports escalate: true, STOP and report the reason.
• mkdir -p .q9 first. It is scratch and never staged.

1. Evidence (measured by the orchestrator, 2026-10-11/12)

• The fork's scripts/_janus_patched/setup.py: line 5 SWIPL=os.getenv("SWIPL") or "swipl" (bare name, PATH fallback); line 13 from _find_swipl import swipl_properties; line 15 props=swipl_properties(SWIPL) (runs <SWIPL> --dump-runtime-variables); lines 24/26 bake props["PLLIBDIR"] into RUNPATH (-Wl,-rpath={PLLIBDIR},--enable-new-dtags / -Wl,-rpath,{PLLIBDIR}); line 49 library_dirs=[props["PLLIBDIR"]]. With Guix, PLLIBDIR is the RESOLVED store path, e.g. /gnu/store/ss98pplwyxzv9jaswl5a59v8nhv16g0f-swi-prolog-9.3.8/lib/swipl/lib/x86_64-linux.
• Prod's janus (/opt/orchestrata/.venv/lib/python3.14t/site-packages/janus_swi/_swipl.cpython-314t-x86_64-linux-gnu.so): NEEDED libswipl.so.9, RUNPATH /gnu/store/n0iyzp3…-swi-prolog-9.3.8/…, a store item that no longer exists. That is the outage.
• The Guix profile path is stable and alive: /home/mohamed/.guix-profile/bin/swipl → /gnu/store/ss98ppl…/bin/swipl; /home/mohamed/.guix-profile/lib/swipl/lib/x86_64-linux/libswipl.so.9 → libswipl.so.9.3.8 (exists). The profile is a GC root.
• core/janus_bootstrap._resolve_libswipl_path today: (1) ORCHESTRATA_LIBSWIPL; (2) swipl --dump-runtime-variables via a bare swipl on PATH (_path_from_dump_runtime_variables); (3) ldd on the janus extension (_path_from_janus_ldd); (4) hardcoded _FALLBACK_LIBSWIPL = /usr/lib/swipl/lib/x86_64-linux/libswipl.so.10. Under the systemd PATH /usr/local/bin:/usr/bin, tier 3 returns the library whenever the RUNPATH exists (orchestrator ran _path_from_janus_ldd(ext_path=<dev venv .so>): returned the ss98ppl…/libswipl.so.9; on prod's: None).
• Test fleet: the app container does NOT build janus. It gets the host-built venv copied in, and bind-mounts /gnu/store and /home/mohamed/.guix-profile at the same paths (deploy/orchestrata-test/provision_app_container.sh §2, live incus config device show app: guixprofile, guixstore). Its own swipl on PATH is SWI-Prolog 10.0.2 (/usr/sbin/swipl, /usr/lib/swipl, libswipl.so.10), which §6 of that script documents as incompatible with janus: if tier 2 picks it, the process mixes .so.9 and a .so.10 home and SIGABRTs. A RUNPATH under /home/mohamed/.guix-profile resolves in the container through the mount.
• env/toolchain.env today declares CC=/usr/bin/gcc, CXX=/usr/bin/g++. Readers: scripts/worktree_toolchain.py (declared_toolchain, pin_system_toolchain, _TOOLCHAIN_KEYS), deploy/toolchain.sh (load_toolchain), .envrc (dotenv), scripts/build_janus_ft.py (_build_env → _pin_system_toolchain), deploy/install-orchestrata.sh and deploy/install-tools.sh (source deploy/toolchain.sh), scripts/worktrees.py, scripts/build_ft_tree_sitter.py, scripts/affected_tests.py.
• uv caches built wheels of path packages. A changed SWIPL does not by itself rebuild janus: rebuild with uv sync --reinstall-package janus-swi.

2. Phase A — declare SWIPL; the fork links through the profile

A.1 The declaration

• env/toolchain.env: add SWIPL=/home/mohamed/.guix-profile/bin/swipl. Extend the header comment: what SWIPL is, why the PROFILE path and never a store path or a bare name (ADR-0091 context in two sentences), its consumers.
• scripts/worktree_toolchain.py: SWIPL joins _TOOLCHAIN_KEYS; declared_toolchain validates it exactly like CC/CXX (missing, bare, non-executable → ToolchainDeclarationError); pin_system_toolchain exports it and prints it on its one line. Do NOT .resolve() the value.
• deploy/toolchain.sh load_toolchain: reads, validates and exports SWIPL with the same three refusals and messages; the echo line names it.
• .envrc: confirm by reading it that it dotenv's env/toolchain.env (so SWIPL reaches interactive builds). Change it only if it does not.

A.2 The fork's build (scripts/_janus_patched/setup.py)

• Line 5: SWIPL comes from os.getenv("SWIPL") only. Unset, empty, not absolute, or not executable → sys.exit(<message>) naming env/toolchain.env and ADR-0091. No "swipl" fallback.
• RUNPATH (lines 24/26): bake the declared install's library directory, UNRESOLVED: prefix = os.path.dirname(os.path.dirname(SWIPL)) (string operations or pathlib without .resolve()), rpath = f"{prefix}/lib/swipl/lib/{props['PLARCH']}". Before using it, refuse (sys.exit naming both paths) unless rpath contains the libswipl that PLLIBDIR contains and both resolve (os.path.realpath) to the same file — the declared path and the build's own library are one object at build time.
• library_dirs (line 49) may keep props["PLLIBDIR"] (link-time only).
• Change nothing else in the fork.

A.3 The runtime resolver (src/orchestrata_ultimate/core/janus_bootstrap.py)

• New tier order: (1) ORCHESTRATA_LIBSWIPL override (unchanged); (2) the janus extension's own ldd linkage (today's tier 3 — authoritative: it is the file the extension loads); (3) --dump-runtime-variables of the DECLARED swipl only: the SWIPL environment value when it is an absolute executable path, never a bare swipl from PATH; (4) none resolved → RuntimeError naming env/toolchain.env, ADR-0091 and the uv sync --reinstall-package janus-swi rebuild.
• Delete _FALLBACK_LIBSWIPL and the .so.10 text (both the warning and the ensure_swipl_symbols_global error). Keep _resolve_libswipl_path's signature compatible (same parameter names; update defaults/docstring).
• Update the module docstring and _resolve_libswipl_path's docstring to the new order and the reason for it (fleet: §1 evidence, one sentence).

A.4 Docs

• docs/free-threaded-prolog.md line 12: SWI-Prolog is the one declared in env/toolchain.env (on this host Guix 9.3.8, libswipl.so.9); a line that the test fleet's own /usr/lib/swipl 10.x is not janus's backend.
• §d / §f dependency matrix row for SWI-Prolog: detection is from the SWIPL declaration, not swipl on PATH. Point at ADR-0091.

A.5 Tests (red first: show each failing before its implementation)

Find the existing test modules with refs/who-imports (tests/deploy/test_toolchain_sh.py, the worktree_toolchain tests, tests/core/test_janus_bootstrap.py, tests/scripts/test_build_janus_ft.py are the expected homes; confirm). Add, red first:

1. declared_toolchain returns SWIPL; refuses a declaration missing SWIPL, with SWIPL=swipl (bare), and with a non-executable path. Real tmp declaration files; real executables (/usr/bin/gcc, /home/mohamed/.guix-profile/bin/swipl).
2. load_toolchain (shell) exports SWIPL; the same three refusals.
3. The fork's real build (the pattern test_build_janus_ft.py already uses): (a) SWIPL unset → the build fails and its output names env/toolchain.env; (b) SWIPL declared through a tmp PREFIX that is a symlink to /home/mohamed/.guix-profile → the built _swipl*.so's RUNPATH (readelf -d) equals <tmp prefix>/lib/swipl/lib/x86_64-linux exactly and contains no /gnu/store/.
4. _resolve_libswipl_path: the ldd tier wins over a dump-runtime result that names a different library (seams); no tier resolves → the RuntimeError names env/toolchain.env and ADR-0091 and does not contain .so.10; a bare SWIPL=swipl in env= is never executed (tier 3 skips it).

A.6 Measure, then STOP

1. Rebuild this worktree's janus with the declaration in the environment: env -u PYTHONPATH -u PYTEST_ADDOPTS SWIPL=/home/mohamed/.guix-profile/bin/swipl uv sync --reinstall-package janus-swi (quote the rc). Then readelf -d the built _swipl*.so (locate it with find .venv -name '_swipl*.so'): quote NEEDED and RUNPATH. RUNPATH must be /home/mohamed/.guix-profile/lib/swipl/lib/x86_64-linux.
2. Under the systemd PATH, from /tmp: env -i HOME=$HOME PATH=/usr/local/bin:/usr/bin <worktree>/.venv/bin/python -c "from orchestrata_ultimate.core import janus_bootstrap as jb; print(jb.ensure_swipl_symbols_global()); import janus_swi as j; print(j.query_once('X is 6*7')); j.consult('t','ok :- use_module(library(http/json)).'); print(j.query_once('ok'))" — quote all three lines. No ORCHESTRATA_LIBSWIPL, no SWIPL set.
3. The affected set over every changed file, then an authoritative forced run of it: quote test_totals, reused, rc. 0 failed, 0 errors, 0 skipped, reused 0.
4. ruff check, black --check, pyright on the changed non-fork .py files; bash -n deploy/toolchain.sh; scripts/gate.py review-staged after git add of exactly the Phase A files. Quote each rc.
5. Inventory, report only (do not change): every place, at build time and at run time, where the fork (scripts/_janus_patched/janus/*.py, e.g. _find_swipl.find_swipl) or our code invokes swipl, and what each does under the systemd PATH. Measure, do not guess.
6. Write plans/briefs/qwen/Q9-report.md (sections: changes per file with line refs; red-first evidence quoted per test; A.6 outputs 1–5 quoted; anything you stopped on). Then STOP.

3. Phase B — fail loudly on a changed library (continuation prompt only)

Outline, refined in the continuation brief after Phase A's report:

• B.1 The fork records the SWI-Prolog version it was compiled against (props["PLVERSION"]) in a generated, git-ignored module inside the built package.
• B.2 janus_bootstrap refuses to start when the loaded engine's version (current_prolog_flag(version, V)) differs from the recorded one, naming both and the rebuild command.
• B.3 promotion/identity.py check_identity reports a deployment whose janus extension RUNPATH does not resolve or lacks the NEEDED soname. Evidence target: run it read-only against /opt/orchestrata and quote that it names the dead n0iyzp3… RUNPATH.

4. Done-when (Phase A)

• [ ] A.1–A.4 changes in exactly these files: env/toolchain.env, scripts/worktree_toolchain.py, deploy/toolchain.sh, (.envrc only if A.1 requires), scripts/_janus_patched/setup.py, src/orchestrata_ultimate/core/janus_bootstrap.py, docs/free-threaded-prolog.md, and their test modules.
• [ ] A.5 tests: each shown red before its implementation, green after.
• [ ] A.6 1–5 quoted in plans/briefs/qwen/Q9-report.md.
• [ ] Authoritative forced run: 0 failed, 0 errors, 0 skipped, reused 0.
• [ ] Files staged, nothing committed. STOP.

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks