Orchestrata Ultimate › Plans › implementation-plan
Implementation Plan: Port Orchestrata → Orchestrata-Ultimate
Source: plans/implementation-plan.md at 8a2cb42ad · last changed 2026-08-10 · first committed 2026-04-11 · 801 lines.
Prose pending.
Plan text
Implementation Plan: Port Orchestrata → Orchestrata-Ultimate
Executive Summary
Port ~119,000 LOC (net) from orchestrata's 26 packages into orchestrata-ultimate's single-package structure. Keep ultimate's core/ infrastructure (observability, error handling, threading, exceptions). Replace all stubs with real implementations.
This plan survived 13 rounds of adversarial review (124 issues raised, all resolved).
Source: /home/mohamed/workspace/orchestrata/packages/orchestrata-*/src/orchestrata/
Target: /home/mohamed/workspace/orchestrata-ultimate/src/orchestrata_ultimate/
Python: >=3.14 (all orchestrata packages require it)
Pin: Orchestrata frozen at commit recorded in plans/ORCHESTRATA_PIN.txt
1. Current State
Ultimate: KEEP (real implementations)
| Module | LOC | What |
|---|---|---|
core/logging/ |
1,063 | structlog, audit, context |
core/metrics/ |
780 | Prometheus, decorators |
core/tracing/ |
583 | OpenTelemetry |
core/patterns/error_handling.py |
729 | Retry, circuit breaker, bulkhead |
core/threading/ |
892 | Thread pool, task tracking |
core/domain/ |
214 | Node, DomainEntity |
exceptions/ |
154 | Exception hierarchy |
Ultimate: REPLACE (stubs)
| Module | LOC | Replaced By |
|---|---|---|
agents/*.py |
538 | orchestrata-agents + 11 role packages |
integrations/openrouter.py |
80 | orchestrata-provider |
integrations/todoist.py |
75 | Real Todoist API implementation |
integrations/vizint.py |
76 | orchestrata-gateway |
integrations/cheetahclaws.py |
136 | Real CheetahClaws integration |
knowledge_graph/graph.py |
110 | orchestrata-knowledge |
governance/pam.py |
145 | orchestrata-governance |
core/orchestrator.py |
279 | orchestrata-kernel |
cli.py |
290 | Functional CLI with real commands |
Orchestrata: PORT (26 packages, 122,121 LOC)
| Package | LOC | Tier | Key Content |
|---|---|---|---|
| orchestrata-core | 6,439 | 0 | BaseEntity, interfaces, traceability, config, adapters |
| orchestrata-cxp | 9,525 | 1 | CXPPacket, serializer, validator, threads |
| orchestrata-provider | 1,936 | 1 | ProviderManager, 6 adapters, cost tracking |
| orchestrata-llm | 722 | 1 | call_llm(), call_reasoner(), VolkConfig |
| orchestrata-di | 856 | 1 | Dependency injection container |
| orchestrata-agents | 7,338 | 1+2 | BaseVolk, runtime, pool, coordinator, hierarchy |
| orchestrata-analyst | 729 | 1 | AnalystVolk (requirements) |
| orchestrata-architect | 2,028 | 1 | ArchitectVolk (design) |
| orchestrata-codedirector | 4,228 | 1 | CodeDirectorVolk (code gen) |
| orchestrata-compliance | 766 | 1 | ComplianceVolk (RTM) |
| orchestrata-concierge | 381 | 1 | ConciergeVolk (user interaction) |
| orchestrata-junior | 623 | 1 | JuniorVolk (design review) |
| orchestrata-qa | 3,039 | 1 | QaVolk (testing, auto-fixers) |
| orchestrata-ux | 260 | 1 | UXVolk (wireframes) |
| orchestrata-pm | 1,492 | 1 | PMVolk (project management) |
| orchestrata-verification | 6,458 | 1+2 | review/ subset in Tier 1 (~800 LOC, needed by Junior); remainder in Tier 2 |
| orchestrata-governance | 5,736 | 2 | PolicyEngine, escalation, Prolog, constitutional |
| orchestrata-postman | 12,674 | 2 | Postman broker, routing, queues, guarantees |
| orchestrata-kernel | 15,272 | 2 | EventBus, pipeline, workflow, PAM, audit |
| orchestrata-orchestrator | 11,332 | 2 | Intent parsing, facts, policy, Redis state |
| orchestrata-knowledge | 8,449 | 2+3 | Neo4j, code parser, sync, context injection |
| orchestrata-session | 10,595 | 3 | Session lifecycle, goals, checkpoints |
| orchestrata-memory | 1,145 | 3 | Hot/cold store, compressor, embeddings |
| orchestrata-gateway | 4,429 | 4 | WebSocket, RPC, auth, handlers |
| orchestrata-plugin | 3,599 | 4 | Plugin loader, registry, SDK |
| orchestrata-drivers | 2,070 | 1 | Claude Code, Aider, DeepSeek, browser drivers (needed by Junior+Cd volks) |
2. Module Mapping
Package Mapping
| Orchestrata | Ultimate | Notes |
|---|---|---|
orchestrata.core.* |
orchestrata_ultimate.core.* |
Distributed: see Phase 0 table |
orchestrata.cxp.* |
orchestrata_ultimate.cxp.* |
New package, skip gRPC |
orchestrata.provider.* |
orchestrata_ultimate.providers.* |
Replace openrouter stub |
orchestrata.llm.* |
orchestrata_ultimate.llm.* |
New package |
orchestrata.di.* |
orchestrata_ultimate.di.* |
New package |
orchestrata.agents.* |
orchestrata_ultimate.agents.* |
Replace stubs |
orchestrata.volks.<name>.* |
orchestrata_ultimate.agents.volks.<name>.* |
11 role packages |
orchestrata.governance.* |
orchestrata_ultimate.governance.* |
Replace pam.py |
orchestrata.postman.* |
orchestrata_ultimate.postman.* |
New package |
orchestrata.kernel.* |
orchestrata_ultimate.kernel.* |
Replace orchestrator.py |
orchestrata.orchestrator.* |
orchestrata_ultimate.orchestrator.* |
New package |
orchestrata.knowledge.* |
orchestrata_ultimate.knowledge_graph.* |
Replace graph.py stub |
orchestrata.session.* |
orchestrata_ultimate.session.* |
New package |
orchestrata.memory.* |
orchestrata_ultimate.memory.* |
New package |
orchestrata.gateway.* |
orchestrata_ultimate.gateway.* |
Replace vizint stub |
orchestrata.plugin.* |
orchestrata_ultimate.plugin.* |
New package |
orchestrata.drivers.* |
orchestrata_ultimate.drivers.* |
New package |
Import Translation
from orchestrata.core.interfaces import X → from orchestrata_ultimate.core.protocols import X
from orchestrata.core.traceability import X → from orchestrata_ultimate.core.tracing import X
from orchestrata.core.infra_config import X → from orchestrata_ultimate.core.config import X
from orchestrata.core.base import X → from orchestrata_ultimate.core.domain import X
from orchestrata.<pkg>.* import X → from orchestrata_ultimate.<pkg>.* import X
Three-pass translation: (1) bulk sed, (2) manual audit of dynamic imports, (3) runtime fix.
Dynamic Import Translation Table
String-based module references that bulk sed will NOT catch (15 locations):
| File | Line(s) | String | Correct Translation |
|---|---|---|---|
agents/task_executor.py |
60-68 | "orchestrata.volks.<name>.volk" (9 entries) |
"orchestrata_ultimate.agents.volks.<name>.volk" |
governance/constitutional.py |
302 | "orchestrata.core.enums" |
"orchestrata_ultimate.core.enums" |
plugin/loader.py |
28 | "orchestrata.plugins." (allowlist) |
"orchestrata_ultimate.plugins." |
plugin/config.py |
13 | "orchestrata.plugins." (allowlist) |
"orchestrata_ultimate.plugins." |
plugin/config.py |
~15 | "orchestrata_plugin_" (entry point) |
"orchestrata_ultimate_plugin_" |
plugin/context.py |
~20 | f-string logger name | Update to orchestrata_ultimate |
Note: orchestrata.volks.* maps to orchestrata_ultimate.agents.volks.* (extra agents segment).
This is a compound mapping — naive sed of orchestrata. → orchestrata_ultimate. produces wrong paths.
Class Mapping
| Orchestrata | Ultimate | Change |
|---|---|---|
BaseVolk(BaseEntity) |
BaseAgent(CXPMixin, BaseModel) |
Pydantic + CXP mixin |
AnalystVolk |
AnalystAgent |
Rename |
SeniorDevVolk |
SeniorDevAgent |
Rename |
JuniorVolk |
JuniorAgent |
Rename |
CodeDirectorVolk |
CodeDirectorAgent |
Rename |
QaVolk |
QualityAssuranceAgent |
Rename |
ArchitectVolk |
ArchitectAgent |
Rename |
UXVolk |
UserExperienceAgent |
Rename |
PMVolk |
ProjectManagerAgent |
Rename |
ConciergeVolk |
ConciergeAgent |
Rename |
ComplianceVolk |
ComplianceAgent |
Rename |
OrchestrationKernel |
OrchestrataKernel |
Replace ultimate's stub |
PolicyEngine |
GovernanceEngine |
Replace ultimate's stub |
Neo4jClient |
Neo4jGraph |
Replace ultimate's stub |
ProviderManager |
ProviderManager |
Replace ultimate's stub |
VolkRuntime/Pool/Registry |
AgentRuntime/Pool/Registry |
Rename |
ABC interfaces (I*) |
Protocol classes (*Protocol) |
PEP 544 |
Architecture: BaseAgent Unification
class CXPMixin:
"""CXP communication. Injected at runtime via set_postman()."""
_postman: ClassVar[Any] = None
_kg_context: ClassVar[Any] = None
def set_postman(self, postman): self._postman = postman
def set_kg(self, kg): self._kg_context = kg
async def send(self, packet: CXPPacket) -> CXPPacket | None:
if self._postman is None:
raise RuntimeError(f"{self.alias}: postman not set — cannot send")
return await self._postman.aroute(packet)
class BaseAgent(CXPMixin, BaseModel):
"""Unified: Pydantic validation + CXP communication."""
model_config = ConfigDict(arbitrary_types_allowed=True)
agent_id: str = Field(default_factory=lambda: str(uuid.uuid4()))
alias: str # "U:Cd", "U:Qa", etc.
role: str
model: str = "claude-sonnet-4-20250514"
status: str = "idle"
@abstractmethod
async def process(self, packet: CXPPacket) -> CXPPacket | None:
"""Process incoming CXP packet. All subclasses MUST be async."""
...
# Pydantic replaces BaseEntity for AGENT classes only:
# - UUID identity → agent_id field with default_factory
# - to_dict() → model_dump()
# - from_dict() → model_validate()
#
# All other @dataclass types (CXPPacket, TaskDefinition, MessageEnvelope,
# AuditEntry, Goal, Intent, node types, etc.) stay as dataclasses.
# Their to_dict()/from_dict() methods port unchanged.
# This is intentional: only agent identity needs Pydantic validation.
MRO: CXPMixin (plain class) first, BaseModel second. No metaclass conflict.
3. Phased Implementation
Phase 0: Foundation
Goal: Fix bugs, port orchestrata-core, set up test infrastructure. LOC: ~4,500 net (skip ultimate's existing logging/tracing)
0.1: Fix Ultimate Bugs
| Bug | File | Fix |
|---|---|---|
| Wrong import path | logging_config.py:71 |
core.context → core.logging.context |
| Deprecated API (6 locations) | error_handling.py |
asyncio.iscoroutinefunction → inspect.iscoroutinefunction |
| Race condition | error_handling.py:307,348 |
Await circuit breaker state tasks |
| Race condition | thread_pool.py:430,441 |
Protect _active_task_ids with lock |
| Memory leak | thread_pool.py |
Cleanup completed tasks > 1000 |
| Pydantic v1 | node.py:40 |
Config → model_config = ConfigDict(...) AND replace json_encoders with @field_serializer for datetime fields |
| Pydantic v1 | domain_entity.py |
.dict() → .model_dump() (exclude param uses set, compatible with v2) |
| Duration unrecorded | decorators.py:144-149 |
Record duration in metric |
| No-op cleanup | metrics_config.py:304-307 |
Implement stop function |
| Hardcoded state | audit.py:376-379 |
Capture real after_state |
0.2: Port orchestrata-core (6,439 LOC → 4,500 net)
| Source Module | LOC | Destination | Action |
|---|---|---|---|
interfaces/ |
1,231 | core/protocols.py |
ABC → Protocol (PEP 544) |
traceability/ + utils |
1,020 | core/tracing/traceability.py |
Direct port (@traces decorator) |
config/ |
827 | core/config/ |
New subpackage |
adapters/claude/ |
889 | adapters/claude/ |
New package |
monitoring/ |
544 | Merge into core/metrics/ |
Ultimate has better base |
base/ |
380 | core/domain/ |
Merge with Node/DomainEntity |
infra_config.py |
177 | core/config/infra.py |
Fix os.environ[] → .get() |
redis_factory.py |
181 | core/redis.py |
Add optional fallback |
state/ |
170 | core/state.py |
Direct port |
standards.py |
190 | core/standards.py |
Direct port |
lru_cache.py |
126 | core/cache.py |
Direct port |
enums.py, paths.py, etc. |
~230 | core/ |
Direct port |
logging.py |
167 | Skip | Ultimate's structlog is better |
tracing.py |
307 | Skip | Ultimate's OpenTelemetry is better |
Critical fix — infra_config.py lines 112-116:
# Before (crashes without env vars):
REDIS_HOST = os.environ["REDIS_HOST"]
# After (safe defaults):
REDIS_HOST = os.environ.get("REDIS_HOST", "localhost")
REDIS_PORT = int(os.environ.get("REDIS_PORT", "6379"))
NEO4J_URI = os.environ.get("NEO4J_URI", "bolt://localhost:7687")
NEO4J_USER = os.environ.get("NEO4J_USERNAME", "neo4j")
NEO4J_PASSWORD = «redacted» "")
0.3: Port Test Infrastructure
- Port root
tests/conftest.py(338 LOC) — merge with ultimate's existing - Port
tests/fixtures/(redis.py, neo4j.py, tempdir.py, prolog.py, init.py) - Port
tests/factories/(8 modules: agents.py, cxp.py, governance.py, intent.py, knowledge.py, postman.py, session.py, init.py) — imported by 86+ test files - Port
tests/infrastructure.py— centralized test config (REDIS_HOST, NEO4J_URI, etc.). Apply sameos.environ.get()fix as production infra_config.py - Port
tests/conftest_infra_gate.py(154 LOC) — health checks for Redis/Neo4j. Fix bareos.environ["REDIS_HOST"]reads (lines 30-34) to use.get()with defaults - Port all 18 per-directory conftest.py files (agents/, cxp/, kernel/, etc.)
- Prolog infrastructure: stack limit (4GB), GC every 50 tests, lazy pyswip import
- Auto-markers:
pytest_collection_modifyitemshook for requires_redis, requires_neo4j, etc. - xdist: per-worker Redis DB isolation (gw0→db0, gw1→db1)
0.4: Update pyproject.toml
requires-python = ">=3.14" # was >=3.11
# Bump:
"httpx>=0.28.0" # was >=0.25.0
# Add:
"watchfiles>=0.21" # file watching for KG sync
# Dev add:
"pytest-xdist>=3.5.0"
"pytest-timeout>=2.1.0"
0.5: Pin Orchestrata
cd /home/mohamed/workspace/orchestrata && git rev-parse HEAD > \
/home/mohamed/workspace/orchestrata-ultimate/plans/ORCHESTRATA_PIN.txt
Tier 1: Agents That Talk and Think (~37,000 LOC)
Goal: Agents make real LLM calls, coordinate via CXP.
Done when: ultimate intent "write tests for X" produces real LLM output in <30s.
1.1: CXP Protocol (9,525 LOC, skip gRPC)
Port: packet.py, json_serializer.py, exceptions.py, headers.py, payload.py, threads.py, validator.py.
Skip: proto/, grpc_client.py, grpc_server.py, binary_serializer.py, prolog_bridge.py.
Add: cxp/dispatcher.py — InMemoryDispatcher (~50 LOC):
class InMemoryDispatcher:
def __init__(self): self._handlers: dict[str, Callable] = {}
def register(self, alias: str, handler: Callable): self._handlers[alias] = handler
async def aroute(self, packet: CXPPacket) -> CXPPacket | None:
handler = self._handlers.get(packet.to_agent)
return await handler(packet) if handler else None
Replaced by PostmanDispatcher in Tier 2 via factory in get_dispatcher().
1.2: Provider + LLM (2,658 LOC)
Port: provider_manager.py, 6 adapters, cost_tracker.py, rate_limiter.py,
circuit_breaker.py, health_monitor.py, router.py, registry.py.
Port: llm/client.py (call_llm, call_reasoner), config.py (VolkConfig), exceptions.py.
Delete: integrations/openrouter.py (stub).
1.3: DI Container (856 LOC)
Port: orchestrata-di entire package. Direct port, minimal changes.
1.4: Drivers (2,070 LOC)
Port: orchestrata-drivers entire package. Required by JuniorVolk (top-level import) and CodeDirectorVolk (lazy import) for code generation. Includes: claude_code_driver, aider_driver, deepseek_coder_driver, qwen_driver, browser_driver, driver_factory. These are standalone — no internal dependencies beyond stdlib + httpx.
1.5: Verification Review Subset (~800 LOC)
Port ONLY the review subpackage from orchestrata-verification:
verification/review/checks_correctness.pyverification/review/helpers.pyverification/review/types.pyverification/review/engine.pyverification/review/checks_quality.py,checks_safety.py,checks_structure.pyRequired by JuniorVolk (top-level import). Self-contained within verification/review/. Full verification framework (6,458 LOC) remains in Tier 2.
1.6: Agent Implementations — VOLKS ONLY (~21,000 LOC)
Port agents base: volks/base.py → merge into BaseAgent (CXPMixin pattern).
Source clarification: Some volks exist in BOTH orchestrata-agents/volks/ (inline)
AND as separate packages (orchestrata-<role>/). Port from the separate packages —
they're the canonical, most complete versions. senior_dev.py is an exception: it only
exists in orchestrata-agents/volks/senior_dev.py (no separate package).
Port 10 role packages + 1 inline volk into agents/volks/:
- analyst (729), architect (2,028), codedirector (4,228), compliance (766), concierge (381), junior (623), qa (3,039), ux (260), pm (1,492), senior_dev (from agents/volks/, ~500)
- Plus agent base framework: communication.py, hierarchy.py, constraints.py, types.py, constants.py, exceptions.py (~5,338)
Also needed for CodeDirectorVolk: agents/swarm/coordinator.py (imports
driver_factory at top level). Port SwarmCoordinator into agents/swarm/.
NOT Tier 1 (deferred to Tier 2):
- runtime.py, pool.py, task_executor.py, coordinator.py (agent coordinator), escalation.py, lifecycle.py, prolog_bridge.py, intent_adapter.py (~5,000 LOC) — depend on kernel + orchestrator.
Delete stubs after 90%+ test pass rate: base_agent.py, architect.py, code_director.py, quality_assurance.py, user_experience.py.
Tier 2: Coordination and Governance (~50,000 LOC)
Goal: Multi-agent coordination, message routing, governance enforcement. Done when: Submit intent → routes through Postman → agents collaborate → results aggregated.
2.1: Knowledge Subset (1,755 LOC)
Port only modules needed by Tier 2: context_injector.py (369), synchronizer.py (789), cxp_auditor.py (~200), neo4j_client.py (297), config.py + constants.py (~100).
2.2: Agent Infrastructure (5,000 LOC)
Port deferred Tier 1 files: runtime.py (AgentRuntime), pool.py (AgentPool), task_executor.py, coordinator.py (AgentCoordinator), escalation.py, lifecycle.py, prolog_bridge.py, intent_adapter.py.
2.2b: Verification Framework Remainder (~5,658 LOC)
Port orchestrata-verification minus the review/ subset already ported in Tier 1.5.
Includes: constraints, checkpoints, proofs, autofix, edge cases, agents, escalation,
orchestration, session verification, types. NOT a volk — test traceability infrastructure.
Lives at orchestrata_ultimate/verification/.
2.3: Governance (5,736 LOC + prolog_gate.py)
Port: policy_engine.py, escalation_handler.py, constitutional.py, ethical.py,
prolog_client.py, prolog_gate.py (174 LOC, required by kernel/pipeline.py),
resource_access.py, safe_mode.py, cache.py, monitoring.py,
audit.py, violations.py, dynamic_policy.py, types.py, interfaces.py, constants.py, exceptions.py.
Port: configs/ YAML files and prolog/ rule files as package data.
Prolog optional — factory returns PythonPolicyEngine if pyswip unavailable.
Delete: governance/pam.py (stub).
2.4: Postman (12,674 LOC)
Port: postman.py, router.py, delivery.py, queue.py, priority.py, envelope.py, payload.py, guarantees.py, message_store.py, receipt_store.py, replay_engine.py, governance.py, spike_router.py (152 LOC, required by kernel/pipeline.py + agents/base.py), types.py, config.py, interfaces.py (keep as Protocols), exceptions.py. In-memory queues default, Redis optional via factory.
2.5: Kernel (15,272 LOC)
Port: kernel.py, event_bus.py, dispatcher.py (with conditional gRPC import),
router.py, pipeline.py, pipeline_events.py (338 LOC, required by pipeline.py),
workflow_engine.py, queue_manager.py, volk_registry.py,
config_manager.py, policy_evaluator.py, pam_enforcer.py, performance_manager.py,
audit_logger.py, session_manager.py, session_context.py, resource_monitor.py,
error_handler.py. CLI files adapted to typer subcommands.
EventBus: add os.environ.get() defaults, InMemoryEventBus fallback.
Delete: core/orchestrator.py (stub).
Update: __init__.py — change from orchestrata_ultimate.core.orchestrator import UltimateOrchestrator
to from orchestrata_ultimate.kernel import OrchestrataKernel as UltimateOrchestrator (backward compat alias).
2.6: Orchestrator (11,332 LOC)
Port: intent parsing, fact management, policy engine, Redis state, Neo4j audit, impact analysis, Prolog bridge, NLP, metrics, workflow. Governance ↔ Orchestrator circular: TYPE_CHECKING + runtime lazy import.
Tier 3: Knowledge and Memory (~18,000 LOC)
Goal: Code understanding, persistent memory, session management.
3.1: Knowledge Remainder (6,694 LOC)
Port: code_parser.py, anomaly_detector.py, kg_design.py, types.py, types_nodes.py,
search, query engine, dependency analyzer, file watcher, etc.
Delete: knowledge_graph/graph.py (stub).
3.2: Session (10,595 LOC)
Port: session lifecycle, WorkItem, Goal, GoalProgress, Redis store, Neo4j store,
checkpoint, KG context, hook system.
Prolog optional in session hooks (guard with if self._prolog_bridge).
3.3: Memory (1,145 LOC)
Port: memory_engine.py, hot_store.py, cold_store.py, compressor.py, embedding.py, token_counter.py.
Tier 4: External Surface (~8,000 LOC)
Goal: WebSocket API, plugin system, external integrations.
4.1: Gateway (4,429 LOC)
Adapt to FastAPI. Keep JSON-RPC 2.0 and JWT auth. Port RPC handlers.
Add pyjwt>=2.8.0 to dependencies.
Delete: integrations/vizint.py (stub).
4.2: Plugin SDK (3,599 LOC)
Direct port. Adapt entry point discovery to orchestrata_ultimate_plugin_*.
Also update ALLOWED_PLUGIN_PREFIXES in loader.py and config.py:
"orchestrata.plugins." → "orchestrata_ultimate.plugins."
4.3: Integrations (build new)
CheetahClaws: build real TUI integration → orchestrata_ultimate/integrations/cheetahclaws/
Todoist: build real API client → orchestrata_ultimate/integrations/todoist/
Delete remaining stubs: cheetahclaws.py, todoist.py.
The integrations/ directory stays — it hosts the real implementations replacing the stubs.
4. External Dependencies
Optional Infrastructure (factory pattern)
All external services use factory functions with in-memory fallbacks:
def create_event_bus(redis_url=None):
return RedisEventBus(redis_url) if redis_url else InMemoryEventBus()
def get_policy_engine():
try:
from pyswip import Prolog
return PrologPolicyEngine()
except ImportError:
return PythonPolicyEngine()
| Service | Required For | Default | Production |
|---|---|---|---|
| Redis | EventBus, sessions, queues | In-memory | Redis |
| Neo4j | Knowledge graph, audit | Skip (empty context) | Neo4j |
| Prolog (pyswip) | Governance rules | Python rule engine | SWI-Prolog |
| gRPC | CXP transport (optional) | JSON/HTTP | gRPC (conditional import) |
Dependency Changes to pyproject.toml
# Bump
requires-python = ">=3.14"
"httpx>=0.28.0"
# Add
"watchfiles>=0.21"
# Add (dev)
"pytest-xdist>=3.5.0"
"pytest-timeout>=2.1.0"
# Add (Tier 4 only)
"pyjwt>=2.8.0"
# Remove (unused)
"asyncio>=3.4.3" # stdlib, no-op shim
Package Data
[tool.hatch.build]
packages = ["src/orchestrata_ultimate"]
[tool.hatch.build.targets.wheel.force-include]
"configs" = "orchestrata_ultimate/configs"
"prolog" = "orchestrata_ultimate/prolog"
5. Test Strategy
Phase 0: Test Infrastructure First
Port BEFORE any code: root conftest.py, 18 per-directory conftest.py files, tests/fixtures/ (5 modules), conftest_infra_gate.py. Fix 31 failing generated tests. Result: 244 tests, 244 pass.
Per-Tier Pass Rate Targets
| Tier | New Tests | Target Pass Rate | Cumulative |
|---|---|---|---|
| Phase 0 | 0 new (fix existing) | 100% | 244 |
| Tier 1 | ~3,000 | 90% (2,700+) | ~3,000 |
| Tier 2 | ~5,000 | 85% (4,250+) | ~8,000 |
| Tier 3 | ~2,000 | 85% (1,700+) | ~10,000 |
| Tier 4 | ~1,400 | 90% (1,260+) | ~11,500 |
| Final | Debug remaining | 95%+ | ~11,900 |
Test Infrastructure Details
- Prolog: Stack limit 4GB, GC every 50 tests, lazy pyswip import (prevents 3.14t segfault)
- Redis: Per-worker DB isolation for xdist (gw0→db0, gw1→db1), test_id-based cleanup
- Neo4j: test_id-tagged nodes, surgical cleanup (not global flush)
- Markers: Auto-applied via pytest_collection_modifyitems: requires_redis, requires_neo4j, requires_prolog
- Parallel: pytest-xdist -n auto, --timeout=600 per test
- No mocks: Zero mock usage in production tests. 4 files import mock for meta-testing (mock detector enforcement). Consistent with no-mock policy.
6. Communication Patterns
Three patterns coexist (no bridging needed):
| Pattern | Used For | Mechanism |
|---|---|---|
| EventBus | System lifecycle events (agent spawned, config changed) | Redis pub/sub (or in-memory) — fire and forget |
| Postman | Agent-to-agent CXP messages with delivery guarantees | Routing + queues + retries + receipts |
| Direct calls | Synchronous within orchestrator | agent.execute_edit() method calls |
Tier 1 uses InMemoryDispatcher (direct async routing). Tier 2 adds Postman (full guarantees).
7. Circular Dependency Resolution
Cross-Tier: Agents → Kernel (broken by splitting agents)
- Tier 1a: Volk implementations (volks/) — depend only on core + cxp + llm
- Tier 2: Agent infrastructure (runtime, pool, coordinator) — depends on kernel + orchestrator
Within Tier 2: Kernel ↔ Governance ↔ Orchestrator
- Handled with
TYPE_CHECKINGimports + runtime lazy loading kernel/volk_registry.pyTYPE_CHECKING-imports agent typesagents/imports postman at runtime only (in send() method)postman/imports governance lazily (in route() method)
Tier 2 → Tier 3: Kernel → Knowledge (broken by knowledge subset)
- Tier 2 ports only 1,755 LOC of knowledge (context_injector, synchronizer, neo4j_client)
- Tier 3 ports remaining 6,694 LOC (code_parser, anomaly_detector, search, etc.)
8. Runtime Concerns
8.1: Environment Variable Hardcoding (95 locations)
Orchestrata has 95 os.environ["X"] reads (no defaults) across all packages. 20 are in
kernel alone. Each tier port must include a bulk conversion:
# Per-tier audit during port:
grep -rn 'os\.environ\[' src/orchestrata_ultimate/<tier_pkg>/ --include="*.py"
# Replace each with os.environ.get("X", "<sensible_default>")
Defaults: REDIS_HOST=localhost, REDIS_PORT=6379, NEO4J_URI=bolt://localhost:7687, NEO4J_USERNAME=neo4j, NEO4J_PASSWORD="" (empty = skip connection).
8.2: Initialization Sequencing
Boot order for the full system (Tier 2+):
- Config — load environment, YAML configs
- Logging/Metrics/Tracing — observability must be first (ultimate's
initialize_observability()) - EventBus — lazy-init on first access (property in kernel), in-memory fallback if no Redis
- Dispatcher/Postman — create InMemoryDispatcher (Tier 1) or Postman (Tier 2)
- Agents — instantiate BaseAgent subclasses, call
set_postman(dispatcher)on each - Kernel — register agents as components, wire EventBus subscriptions
- Gateway — start WebSocket server (Tier 4 only)
For Tier 1 (no kernel): steps 1-2-4-5 only. Orchestrator creates agents and injects dispatcher.
8.3: Orchestration-Level Error Handling
When call_llm() raises LLMClientError after exhausting retries:
- Agent's
process()method catches it, setswork_status = WorkStatus.FAILEDon CXP packet - Sends failure packet back to requester via
self.send() - Postman (Tier 2) handles dead-letter queueing for undeliverable failure packets
- Kernel logs the failure via AuditLogger
This is how orchestrata already works — the pattern ports with the agent code. No new design needed.
8.4: Trace Context Through CXP
CXPPacket has correlation_id field. Propagation rules:
- Agent creates outbound packet → copies
correlation_idfrom inbound packet (thread continuity) - New intent (no parent) → generates new UUID as
correlation_id @tracesdecorator metadata is compile-time (spec/story/req IDs), NOT runtime correlation- Ultimate's
set_request_id()from core/logging/context.py can be wired to CXPcorrelation_id
Wire during Tier 2: in Postman.aroute(), call set_request_id(packet.correlation_id) before dispatching.
8.5: Agent Authentication (Deferred)
Not needed for MVP (Tier 1). All agents run in-process, single-user. CXP signature field
exists in the protocol but enforcement is a Tier 4 concern (gateway authenticates external clients).
Internal agent-to-agent trust is implicit for local deployment.
8.6: VolkConfig Thread Safety
Fix during Tier 1.2 port:
import threading
_config_lock = threading.Lock()
_CONFIG: VolkConfig | None = None
def _get_config() -> VolkConfig:
global _CONFIG
if _CONFIG is None:
with _config_lock:
if _CONFIG is None:
_CONFIG = VolkConfig()
return _CONFIG
Double-checked locking. Standard pattern.
8.7: Development Workflow
Tier 1 agents are instantiated per-request (no persistent pool). Change code, re-run CLI command.
Tier 2 agent pool supports terminate() + spawn() for replacing agents without full restart.
Knowledge FileWatcher (Tier 3) is a background thread — watches source files, syncs to Neo4j.
No special hot-reload mechanism needed beyond Python's normal module reloading.
8.8: Data Migration (Stubs → Real)
No migration needed. Stubs don't write to Redis or Neo4j (they return hardcoded data). When real implementations activate, they create fresh state. No legacy data to clean up.
8.9: File Locking for Concurrent Agent Edits
Multiple agents (CodeDirector, QA, Architect) can write to the same file in parallel.
No fcntl/aiofiles locking exists in orchestrata. Fix during Tier 1 port:
import asyncio
_file_locks: dict[str, asyncio.Lock] = {}
async def acquire_file_lock(path: str) -> asyncio.Lock:
if path not in _file_locks:
_file_locks[path] = asyncio.Lock()
return _file_locks[path]
Add to execute_multi_agent_edit() — acquire lock before parallel agent calls, release after.
Prevents silent data loss from overlapping writes.
8.10: InMemoryDispatcher Backpressure
Tier 1 InMemoryDispatcher has no queue bounds. Fix: add asyncio.Queue with max size:
class InMemoryDispatcher:
def __init__(self, max_pending: int = 1000):
self._queue: asyncio.Queue = asyncio.Queue(maxsize=max_pending)
# ...
Raises asyncio.QueueFull when backpressure threshold hit. Callers must handle.
Postman (Tier 2) replaces this with BoundedQueue + overflow policies.
8.11: Agent Crash Isolation
If an agent's process() raises unhandled exception, the runtime marks it BUSY forever.
Fix during Tier 2 agent infrastructure port:
- Wrap
process()in try/except inAgentRuntime._execute_task() - On exception: set instance status to FAILED, log via AuditLogger, release from pool
- Postman sends failure CXP packet back to requester
8.12: CXP Packet TTL Enforcement
Orchestrata's Postman.route() does NOT check packet.is_expired() before queueing.
Fix during Tier 2 Postman port: add TTL check at routing entry point:
async def aroute(self, packet: CXPPacket) -> CXPPacket | None:
if hasattr(packet, 'is_expired') and packet.is_expired():
logger.warning(f"Dropping expired packet {packet.message_id}")
return None
# ... normal routing
8.13: KG Context Size Limits
VolkContextInjector injects full Neo4j subgraph into every LLM prompt. With 11 agents this can produce 1-5MB of context per batch. Fix during Tier 3 knowledge port:
- Add
max_context_tokensparameter toinject_context()(default: 4000 tokens) - Truncate context to fit within budget
- Use token_counter from memory package for estimation
8.14: Graceful Shutdown for In-Flight Work
Kernel stop() doesn't wait for in-flight handlers. Fix during Tier 2 kernel port:
- Track active tasks in a set:
_active_tasks: set[asyncio.Task] - On
stop(): set_shutting_down = True, await all active tasks with timeout - After timeout: cancel remaining tasks, log which were abandoned
9. Stub Lifecycle
Stubs are NOT fallbacks for the factory pattern. The factory's in-memory implementation IS the permanent fallback. Stubs are temporary code returning fake data.
- Port real code alongside stubs (both exist in codebase)
- Wire orchestrator to use real code via factory
- Run tests — if 90%+ pass, delete stubs
- If <90% pass, debug ported code (stubs serve as reference for expected behavior)
10. Risks and Mitigations
| Risk | Mitigation |
|---|---|
| grpcio won't build on Python 3.14t | Skip gRPC, conditional import in kernel dispatcher |
| Prolog (pyswip) unavailable | Factory: PythonPolicyEngine fallback |
| Redis not available | Factory: InMemoryEventBus, InMemoryQueue, InMemoryRegistry |
| Neo4j not available | Graceful degradation: agents receive empty context |
| 95 hard os.environ[] reads across all packages | Bulk sed during each tier port: os.environ["X"] → os.environ.get("X", default) |
| 119K LOC import translation | Three-pass: bulk sed → manual audit → runtime fix |
| Circular deps in flat package | TYPE_CHECKING + runtime lazy imports |
| Orchestrata moves during port | Pin to commit, re-pin only after tier completion |
| Stubs deleted too early | Keep until 90%+ test pass rate per tier |
11. Success Criteria
Functional
- Tier 1:
ultimate intent "write tests for X"→ real LLM output via CodeDirector, <30s - Tier 2: Multi-agent coordination: intent → Postman routing → CXP collaboration → aggregated result
- Tier 3: Agents receive real KG context in prompts; sessions persist across restarts
- Tier 4: External clients connect via WebSocket; plugins extend agent capabilities
Quantitative
| Metric | Before | After |
|---|---|---|
| Source LOC | 1,985 | ~121,000 |
| Packages with real code | 7 | 17+ |
| Stub modules | 9 | 0 |
| Test functions | 213 passing | ~11,500+ passing |
| Agents with LLM calls | 0 | 11 |
| External deps unused | 8 | 0 |
12. Duplicate Spec Resolution
| Pair | Keep | Archive |
|---|---|---|
| 000-orchestrata-cli / 000-ultimate-cli | 000-ultimate-cli | 000-orchestrata-cli |
| 003-intent / 003-intent-broker | 003-intent-broker | 003-intent |
| 001-kernel / 001-ultimate-architecture | 001-ultimate-architecture | 001-kernel |
| 007-prolog-governance / 035-verification | Both (different scopes) | Neither |
Review History
13 rounds of adversarial review. 124 issues raised across build, architecture, QA, scope, runtime, edge-case, and implementation-feasibility perspectives. All resolved.
Round summary: R1-4 (structure, deps, architecture), R5-6 (verification, math), R7 (runtime init, error handling, env vars), R8 (dynamic imports, missing files), R9 (entry points, stub lifecycle, section numbering), R10 (first approval), R11 (async process(), Pydantic v2, dataclass scope), R12 (test factories, test infra), R13 (final sign-off).
Key corrections:
- Total LOC: 83K → 122K (11 agent role packages + orchestrata-core were uncounted)
- Tier 1: 12K → 42K (accurate scoping)
- orchestrata-core: 1,763 → 6,439 LOC (subdirectories missed)
- 2 missing packages found: orchestrata-di, orchestrata-orchestrator
- Python: >=3.11 → >=3.14 (all orchestrata packages require it)
- "No new dependencies" claim removed (was false)
- Knowledge subset moved to Tier 2 (forward dependency from kernel/postman)
- Agents split into Tier 1a (volks) + Tier 2 (runtime) to break circular dep
- infra_config.py os.environ crash identified and fixed
- 18 conftest.py files identified (was 1)
- Zero mock usage confirmed (4 meta-test files are enforcement tests)
- All interfaces converted to Protocol classes (PEP 544), not deleted
Created: 2026-04-10
-
Drivers moved from Tier 4 to Tier 1 (JuniorVolk + CodeDirectorVolk depend on them)
-
Verification review/ subset moved to Tier 1 (JuniorVolk top-level import)
-
CXPMixin.send() fixed: raises RuntimeError (matches real BaseVolk), not silent None
-
DI container confirmed not needed by volks directly (only by Tier 2 runtime wiring)
-
Hatch package data syntax corrected (force-include, not shared-data)
-
Tier LOC recalculated: Tier 1=37K, Tier 2=50K, Tier 3=18K, Tier 4=8K
-
95 os.environ[] hardcoded reads identified across all packages (was 1 file)
-
Runtime init sequencing specified (config → observability → dispatcher → agents → kernel)
-
Orchestration-level error handling documented (agent catches LLMClientError, sends failure CXP)
-
Trace context propagation via CXP correlation_id specified
-
Agent auth deferred to Tier 4 (not needed for in-process MVP)
-
VolkConfig thread safety: double-checked locking added
-
Data migration: confirmed unnecessary (stubs don't write persistent state)
-
Round 8: 3 missing files found (pipeline_events.py, spike_router.py, prolog_gate.py)
-
Round 8: Dynamic Import Translation Table added (15 string-based module refs)
-
Round 8: Plugin security allowlist prefix translation specified
-
Round 9: init.py entry point update specified for Tier 2.5
-
Round 9: base_agent.py stub added to delete list
-
Round 9: Section numbering fixed (was duplicate section 9)
-
Round 9: integrations/ directory fate clarified (stays for real implementations)
-
Round 10: Plan approved. Confidence 9/10. Biggest risk: compound import translation.
-
Round 11: BaseAgent must declare
async def process()as abstractmethod (sync base + async overrides = bug) -
Round 11: node.py json_encoders deprecated in Pydantic v2 — use @field_serializer
-
Round 11: Dataclass vs Pydantic scope clarified (only agents become Pydantic; CXPPacket etc stay dataclass)
-
Round 12: tests/factories/ (8 modules, 86+ test files depend on them) added to Phase 0.3
-
Round 12: tests/infrastructure.py added to Phase 0.3 with os.environ.get() fix
-
Round 12: conftest_infra_gate.py os.environ[] crash fix added
-
Round 13: Bookkeeping updated. Plan re-approved at confidence 9/10.
Last updated: 2026-04-10 (post round 13 — FINAL APPROVAL)