Orchestrata Ultimate › Plans › claude-surface-pipeline

Claude surface: dev → test → prod

Source: plans/claude-surface-pipeline.md at 8a2cb42ad · last changed 2026-08-23 · first committed 2026-08-23 · 292 lines.

Prose pending.

Plan text

Claude surface: dev → test → prod

Status: written 2026-08-23. Supersedes plans/claude-surface-promotion-plan.md, which rested on a false premise — that the project Claude surface had no deployed copy — and specified an elaborate source relocation to fix a problem that does not exist. That work item is deleted; see §5. Every claim below was measured against the live system.

The model this plan implements, stated by the user:

DEV = where we develop. TEST = the environment that tests the code. PROD = a copy of the dev, versioned after tests pass, copied and registered, so we can then keep updating and testing files after deployment.

1. What exists today — every row measured, not assumed

1.1 There are TWO Claude surfaces, not one duplicated one

This is the answer to "why does the claude folder have a second copy of the code?" — it is not a copy. They are different surfaces with different scopes, different deploy targets, and different lifetimes.

1.2 The three environments

DEV   ~/workspace/orchestrata-ultimate-architecture   143310ec  master    12 dirty
TEST  Incus project orchestrata-test, app @ 10.53.2.13          — NO host tree
PROD  ~/prod/orchestrata-ultimate                     143310ec  detached   0 dirty

TEST has no tree to advance, by design. promote_env.py::_promote_test_container:

"The test env has NO host tree to advance: its kernel + gateway run in the app container of the Incus orchestrata-test project against a READ-ONLY bind-mount of this checkout … ~/test/orchestrata-ultimate — a tree that has never existed on this machine (measured 2026-08-14); every promote to test was a guaranteed failure against a phantom target."

Consequence for this plan: the Incus test fleet cannot be the test stage for the Claude surface. It has no host tree and hosts no Claude session. §4 (W3) uses the seam that can.

1.3 PROD is already a copy — for both surfaces

Project surface — promoted by tree advancement. promotion/stages.py::install_assets:

"The hooks and skills travel WITH the checkout — they live under .claude/ in the tree, so advancing the tree already deployed them. What does not travel is the executable bit … and the systemd units."

Measured, dev vs the prod checkout: hooks 24/24, skills 6/6, agents 8/8. The prod checkout is at 143310ec detached, 0 dirty. install_assets FAILS the deploy if any non-_-prefixed hook arrives without +x, and refuses to repair it on the target ("fix the mode in git, not on the target"). Its glob is .claude/hooks/*.sh; all 24 project hooks are .sh, so coverage is total.

User surface — install-claude-config.sh:

• stages from git archive "$SHA" claude — a pinned ref, never the working tree;
• refuses a SHA that is not an ancestor of origin/master;
• rsync -a --delete into $PREFIX/{hooks,tools,agents,skills,commands,workflow};
• installs the plugin from the git remote, never a directory source ("a directory-sourced marketplace is deploy-in-place one layer up");
• enables the plugin and verifies enabledPlugins;
• verifies gitCommitSha == SHA;
• byte-compares every deployed file against the git blob at that SHA, and fails if zero files were checked.

That last check exists because of a measured incident: "REGISTERED IS NOT COPIED, AND A PIN IS NOT CONTENT" — the pin read c24da189 for five days while the content was 19c26470, and the ten stale files in the live cache included the context-injection machinery that runs on every prompt.

Surface parity today, dev vs plugin vs ~/.claude: hooks 42/42/42, tools 26/26/26, agents 7/7/7, skills 8/8/8, commands 14/14/14. The only difference in the whole tree is __pycache__, correctly excluded.

1.4 The four-way identity invariant holds right now

Measured 2026-08-23 — this is what ties the two surfaces to one version:

All four agree. Both surfaces are provably at one version, and that version is on the remote.

1.5 One driver runs both halves

deploy/ship.sh is the single entry point:

5a/5b verify the outcome independently rather than trusting an exit code, because "the promotion ledger recorded DEPLOYED for a tree that then sat still for four days."

1.6 Verdict on the ladder

The copy, version and register rungs the user asked for are already built and currently green. What is missing is the last rung, plus two structural defects that make it hard to prove. That is the entire remaining scope.

2. W1 — make the plugin self-contained (10 of 31 hook commands)

The defect

gen_plugin.py rewrites hook commands to ${CLAUDE_PLUGIN_ROOT}/… so the plugin is relocatable. The regex only matches the literal prefix:

CLAUDE_HOME_PREFIX = "/home/mohamed/.claude/"
_SCRIPT_PATH_RE = re.compile(r"(?<!\S)" + re.escape(CLAUDE_HOME_PREFIX) + r"(\S+)")

Measured over the 31 declared commands in claude/settings.hooks.json:

• 21 are literal /home/mohamed/.claude/… → rewritten plugin-relative ✅
• 10 are $HOME/.claude/… → pass through byte-identical ❌

The 10 break down as 8 distinct hook scripts (ast_sync_hook.py, platform_conventions_prompt.py, agent_mail_inbox.py ×2, project_context_refresh.py, workflow_context_inject.py, convention_guard.py, task_context_hook.py) plus tools/claude_turn_capture.py, registered twice.

Nothing here runs from the dev tree — ~/.claude/hooks is itself a deployed copy, rsynced from a pinned git archive. The defect is narrower and more structural: the plugin, the artifact whose whole purpose is to be self-contained and independently verifiable, silently depends on a separate rsync into a machine-specific path having already happened. One surface, two delivery paths, two verification paths.

The fix

Extend the pattern to the $HOME / ${HOME} spellings so all 31 rewrite. The _rewrite_command docstring currently documents the pass-through as intentional — that note gets replaced, not silently contradicted.

The guard

A test asserting 31 of 31 generated commands are plugin-relative and none contains .claude/. A count, not a spot check, so a hook added later in the $HOME spelling fails the gate instead of quietly taking the second path.

Ordering

W1 lands before W2 — it changes what W2 has to assert.

3. W2 — prove the executes rung

The gap

The hook tests are better than expected: tests/claude/ runs hooks as real subprocesses with real hook JSON on stdin, which is exactly how Claude invokes them. test_plugin_path_independence.py says so explicitly and builds a from-scratch environment mapping per call.

But every one of them executes claude/hooks/… — the source path — and path-independence is proven for 2 representative hooks of 42. That is the precondition for deploying anywhere. It is not evidence that the deployed copy runs. Nothing in the tree executes ~/.claude/hooks/x.py or the plugin-cache copy.

The fix

A smoke suite that reads the generated hook declaration, and for each registered command:

1. executes the command string as written (so ${CLAUDE_PLUGIN_ROOT} resolves against the real plugin root, and any surviving path is exercised as-is);
2. feeds a minimal valid payload for that hook's event type;
3. asserts the exit-code contract for its category — a gate may block (exit 2), an injector must not.

Zero commands executed is a FAILURE, not a pass. This is the recurring defect class in this repo — the serializer reported what it wrote, not what it lost; the syncer asked its cache, not its destination; /code read silence as absence. An empty smoke run must be loud.

Reuse the environment-construction helper from test_plugin_path_independence.py rather than writing a second one.

4. W3 — a real test stage for the user surface

Why the Incus fleet cannot be it

§1.2: no host tree, no Claude session. Pointing this surface at 10.53.2.x would be the phantom-target failure again, in a new place.

The seam that already exists

install-claude-config.sh already takes --prefix (default $HOME/.claude). The surface can be installed, in full, into a throwaway root. That is the test environment for this surface — the same artifact, the same installer, the same per-file byte comparison, a different destination.

The stage

dev  →  install --prefix <staging>  →  W2 smoke suite against <staging>  →  install to ~/.claude

Wire it into ship.sh between the suite and line 492. A staging install that fails the smoke suite fails the ship, and ~/.claude is never touched.

This makes dev → test → prod literal for the user surface using machinery that is already written and already verified, rather than inventing a fourth environment.

5. Deleted: the source relocation (old W5)

The superseded plan proposed moving the project Claude surface into claude/project/ and generating .claude/ from it, on the belief that the project surface had no deployed copy and therefore no dev/prod separation.

That belief was wrong. /home/mohamed/prod/orchestrata-ultimate exists, is pinned at 143310ec detached with 0 dirty, carries its own .claude/ (24 hooks, 8 agents, 6 skills — matching dev exactly), and the systemd units run from it. Checkout-level separation already provides everything the relocation was meant to provide, and install_assets already gates the one thing that does not travel with a tree.

Deleted, not deferred. There is nothing to do here.

6. Deferred — recorded, not silently dropped

• 29 hook modules under claude/hooks/ are unregistered in settings.hooks.json (error_gate, stop_audit clusters). Each is either dead code or an unwired capability; both are defects, and neither is in this plan's scope. Needs its own pass with unused / refs / who-imports.
• The user surface ships 7 agents, the project surface 8. Overlap unexamined.
• Stale KG subtree agent-a55ea2123f8449549, 865 modules.
• task_context_hook.py fires on the word "test" appearing in prose.
• 12 dirty files in dev, including this plan's own evidence (scripts/ast_live_sync.py, src/orchestrata_ast/analyzer.py, claude/commands/{ast,code}.md). These want committing before W1 opens, so W1's diff is only W1.

7. Order of work

Each lands independently and each is verified by a test that fails before the change and passes after — no item is complete on a claim.

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks