Orchestrata Ultimate › Plans › swarm-remediation-plan-2026-07-06

Swarm Remediation Plan — 2026-07-06

Source: plans/swarm-remediation-plan-2026-07-06.md at 8a2cb42ad · last changed 2026-08-10 · first committed 2026-07-05 · 333 lines.

Prose pending.

Plan text

Swarm Remediation Plan — 2026-07-06

0. Standing agent rules

NO git stash in agent tasks (added 2026-07-06 after two stash collisions between concurrent agents on this shared, non-worktree checkout — one required git-object-database recovery). For before/after baseline comparisons use git show HEAD:<path> > /tmp/baseline.py (read-only, collision-free). NO backgrounded gate runs: run test suites in the FOREGROUND with a generous timeout — a stopped agent's background task orphans the wait (four agents stalled this way).

Executable wave plan derived from plans/analysis-2026-07-06-findings.md. Owner: Opus orchestrator. Executors: roster agents (.claude/agents/): auditor / builder / wirer / refactorer / test-writer / lint-fixer / reviewer. Tracking: this file's checkboxes (flip [ ]→[x] with commit hash) + session CLI work items (added S200) + one scoped commit per task. Status legend: each task line carries id | agent | prio | deps.

0. Standing rules for EVERY agent brief (paste verbatim into prompts)

1. Context safety (also in each agent's .md): size files before reading (wc -c); NEVER read whole: docs/rtm.json (7.5M), docs/rtm.md (1.9M), docs/CODEBASE_MAP.md (1.2M), plans/ast-.json, coverage/ci dumps, reviews/. Aggregate via python3 -c/jq; cap output | head -40; ranged Reads ≤250 lines.
2. Doctrine: no mocks, no skips, no xdist (scripts/run_tests.py, -j4 max), no pip (uv only), absolute imports, function-scope fixtures, no AI attribution in commits, specs authoritative over code AND tests.
3. Gates before declaring done (per touched scope): uv run ruff check <files> · uv run black --check <files> · uv run pyright <files> (no NEW errors) · uv run python scripts/test-review.py <test files> · targeted pytest for the touched modules · uv run python scripts/serialize_codebase_ast.py standards clean on touched files.
4. Scope discipline: touch ONLY the files your brief names. Pre-existing debt you bump into → report, don't fix (wirer rule generalized), UNLESS your brief says otherwise. One scoped commit per task, message <type>(<area>): <what>.
5. Report format back to orchestrator: {task-id, files touched, tests added/passing counts, gate outputs (1 line each), anything skipped, commit hash}.

Concurrency & conflict rules

• Waves 1, 2 are parallel-safe internally (disjoint file sets listed per task).
• NEVER run two agents on the same package concurrently unless file-disjoint.
• Shared-infra ceiling: max 4 concurrent test-running agents (ONE Redis/Neo4j).
• Waves 3+ tasks marked after:DECISION are blocked until the named decision is recorded in the Decision Register (§D) — do not let builders guess.

Wave 0 — Unblock master (orchestrator, no swarm) — DO FIRST

• [x] W0.1 (5803cef6 + 043dd93e + 7d6fc9be) | orchestrator | P0 | deps:none — Commit S198 in-flight work as 2 scoped commits: (a) feat(session): SessionStart codebase-map auto-injection (.claude/settings.json, .claude/hooks/inject-codebase-map.sh, scripts/gen_codebase_map.py, docs/CODEBASE_MAP.md, CLAUDE.md, .zed/*); (b) docs(test-runner): document shared-service worker ceiling (scripts/run_tests.py, scripts/run_tests_threadpool.py). Verify: hook runs (bash .claude/hooks/inject-codebase-map.sh; exit 0).
• [x] W0.2 (06e964ff + b2fb39e1) | orchestrator | P0 | deps:none — Commit chore(agents): context-safety rules for MB-scale files (.claude/agents/*.md ×7) + this plan + findings doc (docs(plans): 2026-07-06 analysis + swarm plan). BONUS FIX: .gitignore ignored .claude/agents+skills entirely (roster wasn't in git at all, contradicting plans/agent-roster.md) — whitelisted + tracked.
• [x] W0.3 (f279a801) | orchestrator | P1 | deps:none — Two-tier codebase map IMPLEMENTED under fire: @import tier now 31K tokens (header: runners/ entrypoints/packages/test inventory); full skeleton (263K) moved to docs/CODEBASE_MAP_FULL.md (gitignored, on-demand grep/ranged reads). EVIDENCE that forced it: 4 subagent deaths in one night (haiku prompt-too- long; 3× sonnet autocompact thrash) + a 12-agent wave tripping server rate limits on tokens/min. D6 = ratify or revert (one-line change, see hook header).

Wave 1 — Fix the truth instruments (parallel-safe) 🎯 highest leverage

• [x] W1.1 (b3ec4962; agent completed edit, died verifying; orchestrator verified) | builder | P0 | deps:none — Reformat specs/000-orchestrata-cli/checklists/requirements.md to canonical - [ ] **REQ-000-NNN**: desc bold-checkbox format (~112 REQs currently invisible). Files: that one file only. Done-when: check-traces REQ-000 dangling count → 0; uv run python scripts/generate_rtm.py parses them (verified count rises).
• [x] W1.2a-e COMPLETE (f3c1c0dd + b3ec4962 + 130d9523) | builder ×5 (parallel) | P0 | deps:none — Author missing checklists/requirements.md for specs 009, 050, 097, 103, 104 by extracting REQ IDs already cited in code @traces + the spec.md bodies. One agent per spec, files disjoint. Do NOT invent requirements: every line must trace to spec.md text or an existing in-code REQ id (list them via check-traces dangling output for that spec). Done-when: check-traces dangling for that spec = 0.
• [x] W1.3 (9db8a6b9 + 38871ed2; orchestrator-executed: id-grammar widen was the real defect) | wirer | P0 | deps:W1.1,W1.2 — Regenerate RTM (scripts/generate_rtm.py), sanity-check docs/rtm.json validation block (coverage must jump from 2.8% to a real number; orphans from 4,585 to ~the 241-untraced neighborhood). Wire check-traces --max <baseline> into CI + pre-commit as a ratchet (mirror the untyped --max pattern, pyproject/CI config already has precedent). Commit rtm.json/md regen.
• [x] W1.4 (891bab7e; wired BOTH pre-commit+CI per untyped-ratchet real footprint) | wirer | P0 | deps:none — Commit a pyrightconfig.json (venv-bound, src scope) so full-tree runs are reproducible; record full-tree baseline (631 @ 1.1.410) in the config comment or a BASELINE file; add scripts/ ratchet gate pyright-ratchet (fail if count > baseline) to CI. Files: pyrightconfig.json, ci config, one small script.
• [x] W1.5 (4a516f1f; haiku agent died on prompt-too-long pre-fix, orchestrator did it directly) | lint-fixer | P2 | deps:none — Retire projectstatus.md: replaced body with pointers to context.json/session CLI + CHANGELOG + the 2026-07-06 snapshot. 69 -> 12 lines.
• [x] W1.6 (e3b4317b; 144 tests, tools files 100% cov, +166-orphan visibility, "36" mystery solved) | builder | P1 | deps:none — AST toolkit gauge fixes (all in scripts/serialize_codebase_ast.py + src tools twin): (a) snapshot-backed queries print a staleness warning when snapshot mtime < newest src mtime (105 files newer today); (b) complexity --top N footer reports TRUE total (194), not truncated count; (c) new unused --include-traced mode that reports @traces-decorated zero-reference classes (would have caught both known orphans); (d) ast_query_cli largest accepts --top N flag form for CLI consistency. Each with tests (tools pkg is test-light: 0.69 ratio — raise it). Done-when: 4 behaviors tested; gates pass.

Wave 2 — Hygiene (single agent, mechanical)

• [x] W2.1 (no commit needed; haiku succeeded post-two-tier, 28s) | lint-fixer | P1 | deps:W0.1 — Delete ignored artifacts (~130 MB, no commit needed): htmlcov/ coverage.html/ coverage_html/ ci_.txt coverage_.txt full_coverage*.txt final_verify.txt .coverage* context.json.bak non_empty test_value/ pycache/ node_modules/ logs/ + rmdir queue_data. EXCEPTION: keep pipeline_output/traces/ (live pipeline evidence) — delete only non-trace pipeline_output content. Done-when: git status unchanged; du -sh repo shrinks ~130M.
• [x] W2.2+D5 (0234af71; orchestrator-executed: 433 removals incl reviews/ --cached untrack, Read-ux deleted after diff showed corrupted stale concierge draft, port_all.sh deleted) | lint-fixer | P1 | deps:W0.1 — git rm tracked junk + .gitignore additions, ONE commit chore(repo): remove tracked build/debris artifacts: instructions-.md, kimi-export-.md, hunt_report.md hunt_results.json hunt_stubs.py, test_.txt ×4, json (12M PostScript blob), empty spec.md, stub class_diagram.mmd module_diagram.mmd, 2026-04-19-.txt, redundancy_report.md, original_test_file.txt, tmp_scanner.py, features/'.py, src/orchestrata_ultimate/session/context.py,cover. .gitignore add: queue_data/, coverage.html/, coverage_html/, 2026--.txt, kimi-export-*.md, hunt*. DO NOT touch: port_all.sh, Read ux/, reviews/ (→ D5). Done-when: python scripts/run_tests.py --collect-only still clean (grep confirms none referenced) + gates.
• [x] W2.3 (orchestrator-executed inline; D5 ruled + executed) | auditor | P2 | deps:none — 30-min evidence pass for D5: diff Read ux/volk.py vs canonical volk (which one? find by class name); check port_all.sh for anything scripts/ lacks; propose reviews/ untracking (git rm --cached keeps files). Output: recommendation memo for D5.

Wave 3 — Consolidation (decision-gated; auditor evidence → decide → refactor)

Evidence packs first (parallel, read-only): — ALL FOUR LANDED 2026-07-06, persisted under plans/evidence/ (d1-pipeline-runners, d2-traceability, d3-duplicate-twins, d4-di-container), each verified by the orchestrator (one D2 claim refuted: PEP 758 syntax misread as Python 2). Decisions D1-D4 are now UNBLOCKED and waiting on Mohamed.

• [x] W3.E1 | auditor | P1 | deps:none — PipelineRunner ownership pack: exact responsibilities/consumers/tests of kernel/pipeline.py vs orchestrator/pipeline_runner.py vs kernel/workflow_engine.py; who constructs each (composition roots); overlap matrix; recommendation. → D1
• [x] W3.E2 | auditor | P1 | deps:none — Traceability quintuple pack: exact API diff of the 5 impls (core/base/traces.py, core/traceability/init.py, core/traceability_utils.py, core/tracing/traceability.py, verification/traceability.py); which RTM/tooling reads which; consumer counts. → D2
• [x] W3.E3 | auditor | P2 | deps:none — Router/policy/types twins pack: CXPRouter ×2 + RoutingEngine; 3 policy stacks; postman/types vs postman_types; error_handler pair; LRU pair; architect mixin pairs. Classify each: mechanical-collapse vs needs-decision. → D3
• [x] W3.E4 | auditor | P2 | deps:none — DI reality pack: every construction path for gateway/kernel/postman services (container vs setters vs hand-built); what plugin SDK actually needs from the container. → D4

Execution (after decisions recorded in §D):

• [x] W3.1 | refactorer | P1 | after:D1 — Enact PipelineRunner decision (merge or demote+rename loser; remove # mypy: ignore-errors from both kernel files, fix what surfaces). Suite green before+after; architecture_checker stays ≤1.
• [x] W3.2 | refactorer | P1 | after:D2 — Traceability twin-collapse (repeat the proven interfaces→protocols pattern: one canonical, others thin aliases, then delete aliases in a follow-up). Done-when: gen-serde --check 0 drift, RTM regen byte-comparable modulo timestamps, 444 importers unbroken (refs + suite).
• [x] W3.3 | refactorer | P2 | after:D3 — Mechanical twin collapses (rename kernel/router.py class, merge postman types, single LRU, single error handler, architect mixin dedup + delete flat analyst mixins per D3; also resolves backlog #35 AnalystStorageMixin).
• [x] W3.4 | wirer | P2 | after:D4 — Enact DI decision (route composition roots through container OR shrink container to plugin scope + bless setters; encode the rule in architecture_checker so the loser pattern fails CI).
• [x] W3.5 | builder | P2 | after:D1 — Write the real architecture-standards.md: layer model (from checker LAYER_ASSIGNMENTS), exception process, protocols-placement policy (core/protocols vs per-package interfaces.py), lazy-import policy (per D7). Checker header points at it; delete the dead reference to architecture/layer-dependency-rules.md.

Wave 4 — Feature levers (builders/wirers; independent of Wave 3)

• [x] W4.1 | builder | P1 | deps:none — AnthropicAdapter tool forwarding (backlog #33, SPEC-011/103): request-side OpenAI→Anthropic tool-schema translation ({name,description,input_schema}), multi-turn tool_result message translation, tool_choice mapping. Tests: injected httpx transport asserting real payloads (pattern exists in tests/providers/adapters/test_anthropic_adapter.py). Also kills backlog #32 weak tests while there.
• [x] W4.2 | builder | P1 | deps:W4.1 — Tool-loop coverage test (backlog #13): drive real tool_call → ToolExecutor → tool message → loop → final response (real executor + workspace; scripted call_with_tools responses).
• [x] W4.3a–c | builder ×3 | P1 | after:D8 — Route volk fleet through _call_llm_with_tools (backlog #88 'biggest lever'): batch volks 3 per agent (Ar+Cd+Qa first — personas assume tools), single-shot→multi-turn behavioral change with test updates + harness scenario per volk. Gate: harness/cli.py scenario run per changed volk.
• [x] W4.4 | wirer | P2 | after:D9 — VolkContextInjector: wire or delete (backlog #10). Auditor overlap memo is the D9 input (assign with W3 evidence wave if capacity allows).
• [x] W4.5 | builder | P2 | deps:none — Memory Phase-2 (backlog #14): route the 5 direct module-level call_llm/call_reasoner callers through self._call_llm (codedirector/review.py:164, junior/volk.py:295, qa/_testgen.py:171, qa/volk.py:870, documentor/volk.py:103).

Wave 5 — Spec-gap closure (after W1 makes the RTM trustworthy)

• [x] W5.1a–e | auditor ×5 (parallel, read-only) | P1 | deps:W1.3 — For each of spec-102 (96 untraced), 105 (40), 007 (23), 003 (22), 005 (18): classify every untraced REQ as (a) implemented-unstamped → emit exact @traces stamp list (file:symbol), (b) partial, (c) missing → backlog item draft. Context-safety: work from check-traces output slices, not rtm.json.
• [x] W5.2 | lint-fixer ×N | P1 | deps:W5.1 — Apply the (a) stamp lists mechanically (bounded, verifiable: check-traces count drops per spec); builders pick up (c) items as new scoped tasks the orchestrator files.
• [x] W5.3 | auditor | P2 | deps:W5.1 — Sweep the tail (022:13, 028:13, 017:10, 009:5, 004:1) same protocol. SPEC-028 items overlap backlog #29/#31 (sentinel fallback + model-assignment divergence → D10).

Wave 6 — Debt paydown (continuous background, low priority)

• [x] W6.1 | refactorer ×N | P2 — Top-10 CC≥10 offenders (start: _attribute_from_junit 35, build_kernel_packet_handler 29) + the 21 >800-LOC modules, biggest-first (review_checks.py 1,664). One module per task.
• [x] W6.2 | builder | P2 — Verification adapters: implement or remove the ignored context param ×8 sites (interface promise; spec 035-verification says which).
• [x] W6.3 | refactorer | P3 — Break the 2 import cycles (kg anomaly↔synchronizer; session.cli achievements→main→archive).
• [x] W6.4 | lint-fixer | P3 — Pyright-debt sweeps by cluster from the 631 baseline (qa/_testgen 27, kernel/queue_manager 25, kg_sync/_stage_code 25, qa/_dep_install 24) — ratchet (W1.4) records progress; mixin host-Protocol fixes (backlog #34/#43) are builder tasks, not lint-fixer.
• [x] W6.5 | test-writer | P2 — Raise test-light packages: tools (0.69 — the toolkit guards everything else), governance (0.74), cxp (0.98).

Wave 7 — DOGFOOD on incus (user directive 2026-07-06; AFTER all waves)

• [x] W7.1 (session 204: all four services green — Redis .10, Neo4j .11 + gw-fleet .13/.14/.19, gateway :8765, kernel :50053 via kernel-launch.sh; topology recorded in plans/dogfood-notes-2026-07-06.md) | orchestrator | P1 | deps:Waves1-6 — Deploy orchestrata on incus: bring up Redis/Neo4j/kernel/gateway via incus/ launch scripts (redis-launch, neo4j-launch, kernel-launch, launch.sh); verify with scripts/start_test_services.py; record the working topology.
• [ ] W7.2 | orchestrator+agents | P1 | deps:W7.1 — End-to-end pipeline runs, ONE MODULE AT A TIME: pick a small real module intent -> requirements -> spec -> design -> code -> tests through the live pipeline (scripts/e2e_run.py or gateway path per D1). For each run: capture the full trace (pipeline_output/traces), take NOTES in plans/dogfood-notes-.md, file/fix bugs found (fix-code-not-test doctrine), and UPDATE the design docs where reality diverges. Repeat per module until the chain is clean.
• [ ] W7.3 | orchestrator | P2 | deps:W7.2 — Synthesize: dogfood report (what broke, what was fixed, design updates made), fold learnings into specs/ plans, update session tracking.

Wave 8 — Context semantic layer (user directive 2026-07-06; design-first)

GOAL: maximize context-injection effectiveness and end context thrashing. Build an ontology/semantic layer over the existing assets (AST serialization, Neo4j KG + code intelligence, domaingen OWL tooling, spec/requirement parsers, memory engine) so each task's context is SELECTED as graph nodes (typed, ranked, k-hop bounded), PACKED under an explicit token budget, and emitted as an EDITABLE context pack the orchestrator can trim — for volk injection AND for Claude-side agent briefs (replacing static map tiers with task-aware packs).

• [x] W8.1 | auditor (read-only) | P1 | deps:none — Ground-truth inventory: KG node/edge types actually populated today (sync_ast_to_neo4j, kg_sync, pipeline_kg writers), ContextInjector/VolkContextInjector query shapes (post-W4.4 merge), domaingen ontology surface reusable for a code/spec ontology, token-counting utilities available, memory-engine retrieval API. Output: evidence pack for the design.
• [x] W8.2 (specs/106-context-semantic-layer/ spec.md v0.1.0 "Design-ratified" + checklists/requirements.md REQ-106-001..044; verified present 2026-07-06 session 204 — checkbox was stale) | orchestrator (Opus design) | P1 | deps:W8.1 — Design doc + spec draft (specs/106-context-semantic-layer/): ontology (Spec/Requirement/Module/ Class/Function/Test/Doc/Decision/Volk/Task nodes; TRACES/IMPLEMENTS/ IMPORTS/TESTS/DEPENDS_ON edges), task->subgraph selection (type filters + relevance ranking + k-hop bounds), token-budget packing, pack format (editable file artifact), trim/edit + feedback loop (which injected nodes were actually used), integration points (volk ContextInjector; agent-brief CLI scripts/context_pack.py; SessionStart hook tier replacement).
• [x] W8.3 (c295195b) | builder(s) | P1 | deps:W8.2 — Ontology + sync hardening: ensure the KG holds the design's node/edge set (extends SPEC-008 remediation batch); constraints + provenance; incremental sync.
• [x] W8.4 (50f43d9b, merged session 204: knowledge_graph/context_pack/ frame/selector/packer + scripts/context_pack.py, 40 real-Neo4j tests, byte-stable, p50 ~5ms) | builder | P1 | deps:W8.2 — Selector + packer (DETERMINISTIC per D11: request-analysis classifier + entity linking -> parameterized per-task-type query templates -> deterministic ranking [graph distance/trace degree/ recency] -> reproducible budgeter; NO LLM in the selection loop; byte-stable packs): scripts/context_pack.py CLI (task-type, target, budget -> pack file); tests with real Neo4j.
• [x] W8.5 (0e883da1, merged session 204: inject_for_task → analyze_request/select/pack, VolkContext.metadata measurement + degradation reason; pipeline_kg dispatch was ALREADY wired ab920ad9 — evidence-pack "0 call sites" claim was stale; 12 enshrining tests fixed to contract, 1 deleted w/ justification; 101 green) | wirer | P1 | deps:W8.3,W8.4 — Integrations: volk execution path (merged ContextInjector task-type layer consumes the selector); agent-brief generation for swarm tasks; measure: context sizes before/after, zero thrash deaths in a full wave run.

D. Decision Register (Mohamed / Opus-with-Mohamed; blocks marked tasks)

ALL TEN RULED 2026-07-06 per Mohamed's directive (evidence recommendations, spec-favouring, consolidate-don't-delete complete implementations) — full text in plans/decisions-2026-07-06.md. One-liners: D1 orchestrator engine survives+absorbs, workflow_engine deleted; D2 base.traces canon, engines consolidated, utils stays copy-target; D3 mechanical 1-5 execute, PolicyEvaluator+RoutingEngine WIRED (staged), kernel router consolidate-or-delete by @traces, LRU promoted; D4 option (b) + honest SPEC-024 rewrite; D5 untrack reviews/, delete port_all.sh + Read-ux-after-diff; D6 two-tier RATIFIED; D7 lazy imports codified + INFO count; D8 Ar/Cd/Qa staged; D9 VolkContextInjector consolidated into live path; D10 sentinel implemented, profiles authoritative.

Progress protocol

1. Orchestrator flips checkboxes here WITH commit hashes; session CLI log-work per completed task; complete-work for the matching backlog items (this plan subsumes backlog #4, 9, 10, 13, 14, 17(part), 21(prereq), 24, 27, 28, 29, 31, 32, 33, 34, 35, 36, 43, 44 — mark them via plan-task IDs).
2. Wave completion = all tasks checked + full scripts/run_tests.py green + architecture_checker ≤1 + check-traces/pyright ratchets not regressed + wave summary logged to session CLI.
3. Any agent that STOPs on ambiguity → orchestrator resolves or escalates to a D-item; never let a builder guess a design decision.
4. Re-run bash .claude/hooks/inject-codebase-map.sh after structural waves so the map stays truthful.

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks