Home / Projects / Orchestrata Ultimate / Decisions / ADR index / ADR-0091

ADR-0091: The janus fork links the declared SWI-Prolog through a path guix gc keeps alive

← ADR-0090 · ADR-0092 →

The record

Status: Accepted (Mohamed, 2026-10-12), superseded in part by ADR-0095 (2026-10-12): the SWI source moves from the guix profile to the project-owned prefix; the declaration and its checks stand · Date: 2026-10-11

Context

The prod kernel (orchestrata-kernel.service, /opt/orchestrata) failed at the 13:00 boot on 2026-10-11 and stayed down until 22:51. Its journal at 13:03:38:

RuntimeError: Failed to RTLD_GLOBAL-preload libswipl at
'/usr/lib/swipl/lib/x86_64-linux/libswipl.so.10'

How janus finds libswipl

The kernel runs Prolog through our own fork of janus-swi, vendored at scripts/_janus_patched/ (base 1.5.3) and built in place on every uv sync (docs/free-threaded-prolog.md §d: the built .so "is a compiled artifact tied to this machine's SWI-Prolog"). Prod installed it from file:///opt/orchestrata/scripts/_janus_patched.

At build time, the fork's setup.py line 5 reads SWIPL=os.getenv("SWIPL") or "swipl": it takes swipl from PATH, runs swipl --dump-runtime-variables, and bakes the reported PLLIBDIR into the extension as RUNPATH (-Wl,-rpath=<PLLIBDIR>,--enable-new-dtags). With Guix that PLLIBDIR is a resolved store path, /gnu/store/<hash>-swi-prolog-…, which is not a GC root: once the profile moves on, guix gc may delete it.

At run time, core/janus_bootstrap._resolve_libswipl_path tries, in order: (1) ORCHESTRATA_LIBSWIPL, declared nowhere; (2) swipl on PATH; (3) ldd on the janus extension, i.e. the baked RUNPATH; (4) a hardcoded /usr/lib/swipl/lib/x86_64-linux/libswipl.so.10.

What happened

• The prod venv's janus has NEEDED libswipl.so.9 and RUNPATH /gnu/store/n0iyzp3wnkwx6496pp7znrj76s25svs5-swi-prolog-9.3.8/... (readelf -d). That store item no longer exists. The profile moved to ss98pplwyxzv9jaswl5a59v8nhv16g0f-swi-prolog-9.3.8 (same version, different build) at guix-profile-106, 2026-10-08 16:07, and a later guix gc removed the old item. guix-daemon logs no gc, so the deletion cannot be dated from the journal.
• It was gone before 13:03. Tier 3 works under the systemd PATH whenever the RUNPATH exists: run in a throwaway prod-venv process with PATH=/usr/local/bin:/usr/bin, _path_from_janus_ldd() returns the ss98ppl…/libswipl.so.9 for a dev venv's janus, and None for prod's. At 13:03 the kernel fell through to tier 4. The guix gc that another session ran during a disk cleanup at ~15:20 came after the failure.
• The 12:34 boot used an identical unit and env, and the kernel started at 12:37:24 via tier 2: "Preloaded libswipl (RTLD_GLOBAL) from /gnu/store/ss98ppl…". Something put swipl on that boot's user-manager PATH; what is not established. On the 13:00 boot the PATH was /usr/local/bin:/usr/bin.
• Every venv on the host carries the same exposure. On 2026-10-11, 30 worktree venvs' janus RUNPATH point at ss98ppl…, alive only until the next profile upgrade plus gc. Prod's points at the deleted n0iyzp3….

What the record already said

• Commit af74b4f43 (2026-09-22) measured that this host's SWI-Prolog is Guix 9.3.8 shipping libswipl.so.9, and fixed tier 2's reconstructed .so.10. The doc header (docs/free-threaded-prolog.md line 12: "SWI-Prolog: 10.x (libswipl.so.10)") and tier 4's hardcoded path were not updated and are stale.
• ADR-0089 (Accepted) moved the fork's C compiler from PATH to the declared env/toolchain.env (CC/CXX). It deliberately left swipl off its enumerated toolchain programs, and scripts/_janus_patched/ is excluded from the gate, so the fork's bare swipl was never flagged.

The restore (2026-10-11 22:51, Mohamed: "you run the restore")

systemctl --user set-environment ORCHESTRATA_LIBSWIPL=$HOME/.guix-profile/lib/swipl/lib/x86_64-linux/libswipl.so.9 plus starting the kernel and the five units earlyoom had killed (D21). The kernel preloaded libswipl from the profile, started the Prolog worker and listens on CXP 50053, 0 restarts. This lasts until the user manager restarts (logout or reboot).

Decision (proposed)

1. Declare SWI-Prolog once. env/toolchain.env gains SWIPL, the absolute path of the Guix profile's bin/swipl (~/.guix-profile/bin/swipl, the profile link, not a store path). It is read exactly like CC/CXX under ADR-0089: missing, bare-named or non-executable is refused, with no PATH fallback. build_janus_ft.py, worktree provisioning and the deploy's uv sync pass it to the fork's build.
2. The fork links through the profile, not the store. The fork's setup.py takes SWIPL from the declaration only (drops or "swipl"), and bakes the declared profile's library directory, unresolved, as RUNPATH. The profile is a GC root, so that path survives every guix gc; tier 3 then resolves on every start under any PATH, and no runtime environment variable is needed.
3. A changed library fails loudly. The fork records the SWI-Prolog version it was compiled against. janus_bootstrap refuses to start when the loaded libswipl reports a different version (a profile upgrade within the soname). verify_deployment_identity.py refuses a deployment whose janus RUNPATH does not resolve, and the next promote rebuilds.
4. Correct the stale record. docs/free-threaded-prolog.md's header names the declared SWI-Prolog (Guix 9.3.8, libswipl.so.9); tier 4's hardcoded .so.10 path is removed and the error names the declaration.
5. Rebuild what is already broken. Prod's janus is rebuilt through promote; any worktree venv whose RUNPATH does not resolve is rebuilt by provisioning.

Alternatives rejected

• Runtime declaration plus GC-rooting the exact store item (this ADR's first draft). It works, but treats the symptom at run time, needs promote to manage a GC root per deployment, and leaves every dev venv mortal. The fork is ours, so the cause can be fixed where the path is baked.
• Bake the resolved store path and GC-root it per venv. It gives an exact build for every venv, but 30-plus roots accumulate and are never collected. Item 3's version check gives the same safety by refusing loudly.
• Put Guix bin on the units' PATH. Every bare program the units spawn would then resolve from the profile, the class ADR-0089 removes, and the baked store path still dangles.
• A system swi-prolog package. OpenMandriva ships none on this host (rpm -qa), the version is not established, and the dev and test venvs use Guix.
• Rely on the session environment at login. It worked on one boot and not the next.

Consequences

• The kernel and every venv's janus load the same way on every boot, under any PATH, and survive guix gc.
• A Guix profile upgrade cannot break a running deployment silently: a new build of the same soname is caught by the version check at start, and the next promote rebuilds. A major bump (.so.10) fails at the loader and at verify.
• env/toolchain.env declares three tools (CC, CXX, SWIPL); ADR-0089's refusal semantics cover all three.
• The interim set-environment declaration becomes unnecessary once item 5 ships to prod.

Provenance

• The prod outage, raised by Mohamed 2026-10-11: "prod has been down since reboot means you hav3e no graph acess and are decieving me with the appearance of progress". Mohamed: "we have our own fork of janus prolog"; "check docs and history" / "and git".
• Evidence: journalctl --user -b -1 / -b 0 -u orchestrata-kernel.service; systemctl --user show-environment; readelf -d on the prod and worktree janus extensions; /var/guix/profiles/per-user/mohamed/guix-profile-106-link; scripts/_janus_patched/setup.py lines 5–49; serialize_codebase_ast.py source _resolve_libswipl_path / _path_from_janus_ldd; commits af74b4f43, 9a5ef5a94, b4283f2cc; docs/free-threaded-prolog.md §d–f; conversation recall orchestrata-ultimate:claude-2 #19–#23 (disk cleanup, guix gc ~15:20).
• Defect D23 in the 2026-10-11 orchestration defect log.
• Ruling: Accepted by Mohamed on 2026-10-12, after the three-line summary (problem, fix, safety): "agree with adr".

Related ADRs

• Superseded by: ADR-0095
• Also mentioned: ADR-0089

Rulings cited

None.

Specs cited

None (or no spec page exists for the ones named).