Orchestrata Ultimate › Plans › implementation-plan

Implementation Plan: Port Orchestrata → Orchestrata-Ultimate

Source: plans/implementation-plan.md at 8a2cb42ad · last changed 2026-08-10 · first committed 2026-04-11 · 801 lines.

Prose pending.

Plan text

Implementation Plan: Port Orchestrata → Orchestrata-Ultimate

Executive Summary

Port ~119,000 LOC (net) from orchestrata's 26 packages into orchestrata-ultimate's single-package structure. Keep ultimate's core/ infrastructure (observability, error handling, threading, exceptions). Replace all stubs with real implementations.

This plan survived 13 rounds of adversarial review (124 issues raised, all resolved).

Source: /home/mohamed/workspace/orchestrata/packages/orchestrata-*/src/orchestrata/ Target: /home/mohamed/workspace/orchestrata-ultimate/src/orchestrata_ultimate/ Python: >=3.14 (all orchestrata packages require it) Pin: Orchestrata frozen at commit recorded in plans/ORCHESTRATA_PIN.txt


1. Current State

Ultimate: KEEP (real implementations)

Module LOC What
core/logging/ 1,063 structlog, audit, context
core/metrics/ 780 Prometheus, decorators
core/tracing/ 583 OpenTelemetry
core/patterns/error_handling.py 729 Retry, circuit breaker, bulkhead
core/threading/ 892 Thread pool, task tracking
core/domain/ 214 Node, DomainEntity
exceptions/ 154 Exception hierarchy

Ultimate: REPLACE (stubs)

Module LOC Replaced By
agents/*.py 538 orchestrata-agents + 11 role packages
integrations/openrouter.py 80 orchestrata-provider
integrations/todoist.py 75 Real Todoist API implementation
integrations/vizint.py 76 orchestrata-gateway
integrations/cheetahclaws.py 136 Real CheetahClaws integration
knowledge_graph/graph.py 110 orchestrata-knowledge
governance/pam.py 145 orchestrata-governance
core/orchestrator.py 279 orchestrata-kernel
cli.py 290 Functional CLI with real commands

Orchestrata: PORT (26 packages, 122,121 LOC)

Package LOC Tier Key Content
orchestrata-core 6,439 0 BaseEntity, interfaces, traceability, config, adapters
orchestrata-cxp 9,525 1 CXPPacket, serializer, validator, threads
orchestrata-provider 1,936 1 ProviderManager, 6 adapters, cost tracking
orchestrata-llm 722 1 call_llm(), call_reasoner(), VolkConfig
orchestrata-di 856 1 Dependency injection container
orchestrata-agents 7,338 1+2 BaseVolk, runtime, pool, coordinator, hierarchy
orchestrata-analyst 729 1 AnalystVolk (requirements)
orchestrata-architect 2,028 1 ArchitectVolk (design)
orchestrata-codedirector 4,228 1 CodeDirectorVolk (code gen)
orchestrata-compliance 766 1 ComplianceVolk (RTM)
orchestrata-concierge 381 1 ConciergeVolk (user interaction)
orchestrata-junior 623 1 JuniorVolk (design review)
orchestrata-qa 3,039 1 QaVolk (testing, auto-fixers)
orchestrata-ux 260 1 UXVolk (wireframes)
orchestrata-pm 1,492 1 PMVolk (project management)
orchestrata-verification 6,458 1+2 review/ subset in Tier 1 (~800 LOC, needed by Junior); remainder in Tier 2
orchestrata-governance 5,736 2 PolicyEngine, escalation, Prolog, constitutional
orchestrata-postman 12,674 2 Postman broker, routing, queues, guarantees
orchestrata-kernel 15,272 2 EventBus, pipeline, workflow, PAM, audit
orchestrata-orchestrator 11,332 2 Intent parsing, facts, policy, Redis state
orchestrata-knowledge 8,449 2+3 Neo4j, code parser, sync, context injection
orchestrata-session 10,595 3 Session lifecycle, goals, checkpoints
orchestrata-memory 1,145 3 Hot/cold store, compressor, embeddings
orchestrata-gateway 4,429 4 WebSocket, RPC, auth, handlers
orchestrata-plugin 3,599 4 Plugin loader, registry, SDK
orchestrata-drivers 2,070 1 Claude Code, Aider, DeepSeek, browser drivers (needed by Junior+Cd volks)

2. Module Mapping

Package Mapping

Orchestrata Ultimate Notes
orchestrata.core.* orchestrata_ultimate.core.* Distributed: see Phase 0 table
orchestrata.cxp.* orchestrata_ultimate.cxp.* New package, skip gRPC
orchestrata.provider.* orchestrata_ultimate.providers.* Replace openrouter stub
orchestrata.llm.* orchestrata_ultimate.llm.* New package
orchestrata.di.* orchestrata_ultimate.di.* New package
orchestrata.agents.* orchestrata_ultimate.agents.* Replace stubs
orchestrata.volks.<name>.* orchestrata_ultimate.agents.volks.<name>.* 11 role packages
orchestrata.governance.* orchestrata_ultimate.governance.* Replace pam.py
orchestrata.postman.* orchestrata_ultimate.postman.* New package
orchestrata.kernel.* orchestrata_ultimate.kernel.* Replace orchestrator.py
orchestrata.orchestrator.* orchestrata_ultimate.orchestrator.* New package
orchestrata.knowledge.* orchestrata_ultimate.knowledge_graph.* Replace graph.py stub
orchestrata.session.* orchestrata_ultimate.session.* New package
orchestrata.memory.* orchestrata_ultimate.memory.* New package
orchestrata.gateway.* orchestrata_ultimate.gateway.* Replace vizint stub
orchestrata.plugin.* orchestrata_ultimate.plugin.* New package
orchestrata.drivers.* orchestrata_ultimate.drivers.* New package

Import Translation

from orchestrata.core.interfaces import X   → from orchestrata_ultimate.core.protocols import X
from orchestrata.core.traceability import X → from orchestrata_ultimate.core.tracing import X
from orchestrata.core.infra_config import X → from orchestrata_ultimate.core.config import X
from orchestrata.core.base import X         → from orchestrata_ultimate.core.domain import X
from orchestrata.<pkg>.* import X           → from orchestrata_ultimate.<pkg>.* import X

Three-pass translation: (1) bulk sed, (2) manual audit of dynamic imports, (3) runtime fix.

Dynamic Import Translation Table

String-based module references that bulk sed will NOT catch (15 locations):

File Line(s) String Correct Translation
agents/task_executor.py 60-68 "orchestrata.volks.<name>.volk" (9 entries) "orchestrata_ultimate.agents.volks.<name>.volk"
governance/constitutional.py 302 "orchestrata.core.enums" "orchestrata_ultimate.core.enums"
plugin/loader.py 28 "orchestrata.plugins." (allowlist) "orchestrata_ultimate.plugins."
plugin/config.py 13 "orchestrata.plugins." (allowlist) "orchestrata_ultimate.plugins."
plugin/config.py ~15 "orchestrata_plugin_" (entry point) "orchestrata_ultimate_plugin_"
plugin/context.py ~20 f-string logger name Update to orchestrata_ultimate

Note: orchestrata.volks.* maps to orchestrata_ultimate.agents.volks.* (extra agents segment). This is a compound mapping — naive sed of orchestrata. → orchestrata_ultimate. produces wrong paths.

Class Mapping

Orchestrata Ultimate Change
BaseVolk(BaseEntity) BaseAgent(CXPMixin, BaseModel) Pydantic + CXP mixin
AnalystVolk AnalystAgent Rename
SeniorDevVolk SeniorDevAgent Rename
JuniorVolk JuniorAgent Rename
CodeDirectorVolk CodeDirectorAgent Rename
QaVolk QualityAssuranceAgent Rename
ArchitectVolk ArchitectAgent Rename
UXVolk UserExperienceAgent Rename
PMVolk ProjectManagerAgent Rename
ConciergeVolk ConciergeAgent Rename
ComplianceVolk ComplianceAgent Rename
OrchestrationKernel OrchestrataKernel Replace ultimate's stub
PolicyEngine GovernanceEngine Replace ultimate's stub
Neo4jClient Neo4jGraph Replace ultimate's stub
ProviderManager ProviderManager Replace ultimate's stub
VolkRuntime/Pool/Registry AgentRuntime/Pool/Registry Rename
ABC interfaces (I*) Protocol classes (*Protocol) PEP 544

Architecture: BaseAgent Unification

class CXPMixin:
    """CXP communication. Injected at runtime via set_postman()."""
    _postman: ClassVar[Any] = None
    _kg_context: ClassVar[Any] = None

    def set_postman(self, postman): self._postman = postman
    def set_kg(self, kg): self._kg_context = kg

    async def send(self, packet: CXPPacket) -> CXPPacket | None:
        if self._postman is None:
            raise RuntimeError(f"{self.alias}: postman not set — cannot send")
        return await self._postman.aroute(packet)

class BaseAgent(CXPMixin, BaseModel):
    """Unified: Pydantic validation + CXP communication."""
    model_config = ConfigDict(arbitrary_types_allowed=True)
    agent_id: str = Field(default_factory=lambda: str(uuid.uuid4()))
    alias: str          # "U:Cd", "U:Qa", etc.
    role: str
    model: str = "claude-sonnet-4-20250514"
    status: str = "idle"

    @abstractmethod
    async def process(self, packet: CXPPacket) -> CXPPacket | None:
        """Process incoming CXP packet. All subclasses MUST be async."""
        ...

    # Pydantic replaces BaseEntity for AGENT classes only:
    # - UUID identity → agent_id field with default_factory
    # - to_dict() → model_dump()
    # - from_dict() → model_validate()
    #
    # All other @dataclass types (CXPPacket, TaskDefinition, MessageEnvelope,
    # AuditEntry, Goal, Intent, node types, etc.) stay as dataclasses.
    # Their to_dict()/from_dict() methods port unchanged.
    # This is intentional: only agent identity needs Pydantic validation.

MRO: CXPMixin (plain class) first, BaseModel second. No metaclass conflict.


3. Phased Implementation

Phase 0: Foundation

Goal: Fix bugs, port orchestrata-core, set up test infrastructure. LOC: ~4,500 net (skip ultimate's existing logging/tracing)

0.1: Fix Ultimate Bugs
Bug File Fix
Wrong import path logging_config.py:71 core.context → core.logging.context
Deprecated API (6 locations) error_handling.py asyncio.iscoroutinefunction → inspect.iscoroutinefunction
Race condition error_handling.py:307,348 Await circuit breaker state tasks
Race condition thread_pool.py:430,441 Protect _active_task_ids with lock
Memory leak thread_pool.py Cleanup completed tasks > 1000
Pydantic v1 node.py:40 Config → model_config = ConfigDict(...) AND replace json_encoders with @field_serializer for datetime fields
Pydantic v1 domain_entity.py .dict() → .model_dump() (exclude param uses set, compatible with v2)
Duration unrecorded decorators.py:144-149 Record duration in metric
No-op cleanup metrics_config.py:304-307 Implement stop function
Hardcoded state audit.py:376-379 Capture real after_state
0.2: Port orchestrata-core (6,439 LOC → 4,500 net)
Source Module LOC Destination Action
interfaces/ 1,231 core/protocols.py ABC → Protocol (PEP 544)
traceability/ + utils 1,020 core/tracing/traceability.py Direct port (@traces decorator)
config/ 827 core/config/ New subpackage
adapters/claude/ 889 adapters/claude/ New package
monitoring/ 544 Merge into core/metrics/ Ultimate has better base
base/ 380 core/domain/ Merge with Node/DomainEntity
infra_config.py 177 core/config/infra.py Fix os.environ[] → .get()
redis_factory.py 181 core/redis.py Add optional fallback
state/ 170 core/state.py Direct port
standards.py 190 core/standards.py Direct port
lru_cache.py 126 core/cache.py Direct port
enums.py, paths.py, etc. ~230 core/ Direct port
logging.py 167 Skip Ultimate's structlog is better
tracing.py 307 Skip Ultimate's OpenTelemetry is better

Critical fix — infra_config.py lines 112-116:

# Before (crashes without env vars):
REDIS_HOST = os.environ["REDIS_HOST"]
# After (safe defaults):
REDIS_HOST = os.environ.get("REDIS_HOST", "localhost")
REDIS_PORT = int(os.environ.get("REDIS_PORT", "6379"))
NEO4J_URI = os.environ.get("NEO4J_URI", "bolt://localhost:7687")
NEO4J_USER = os.environ.get("NEO4J_USERNAME", "neo4j")
NEO4J_PASSWORD = «redacted» "")
0.3: Port Test Infrastructure
0.4: Update pyproject.toml
requires-python = ">=3.14"           # was >=3.11
# Bump:
"httpx>=0.28.0"                      # was >=0.25.0
# Add:
"watchfiles>=0.21"                   # file watching for KG sync
# Dev add:
"pytest-xdist>=3.5.0"
"pytest-timeout>=2.1.0"
0.5: Pin Orchestrata
cd /home/mohamed/workspace/orchestrata && git rev-parse HEAD > \
  /home/mohamed/workspace/orchestrata-ultimate/plans/ORCHESTRATA_PIN.txt

Tier 1: Agents That Talk and Think (~37,000 LOC)

Goal: Agents make real LLM calls, coordinate via CXP. Done when: ultimate intent "write tests for X" produces real LLM output in <30s.

1.1: CXP Protocol (9,525 LOC, skip gRPC)

Port: packet.py, json_serializer.py, exceptions.py, headers.py, payload.py, threads.py, validator.py. Skip: proto/, grpc_client.py, grpc_server.py, binary_serializer.py, prolog_bridge.py. Add: cxp/dispatcher.py — InMemoryDispatcher (~50 LOC):

class InMemoryDispatcher:
    def __init__(self): self._handlers: dict[str, Callable] = {}
    def register(self, alias: str, handler: Callable): self._handlers[alias] = handler
    async def aroute(self, packet: CXPPacket) -> CXPPacket | None:
        handler = self._handlers.get(packet.to_agent)
        return await handler(packet) if handler else None

Replaced by PostmanDispatcher in Tier 2 via factory in get_dispatcher().

1.2: Provider + LLM (2,658 LOC)

Port: provider_manager.py, 6 adapters, cost_tracker.py, rate_limiter.py, circuit_breaker.py, health_monitor.py, router.py, registry.py. Port: llm/client.py (call_llm, call_reasoner), config.py (VolkConfig), exceptions.py. Delete: integrations/openrouter.py (stub).

1.3: DI Container (856 LOC)

Port: orchestrata-di entire package. Direct port, minimal changes.

1.4: Drivers (2,070 LOC)

Port: orchestrata-drivers entire package. Required by JuniorVolk (top-level import) and CodeDirectorVolk (lazy import) for code generation. Includes: claude_code_driver, aider_driver, deepseek_coder_driver, qwen_driver, browser_driver, driver_factory. These are standalone — no internal dependencies beyond stdlib + httpx.

1.5: Verification Review Subset (~800 LOC)

Port ONLY the review subpackage from orchestrata-verification:

1.6: Agent Implementations — VOLKS ONLY (~21,000 LOC)

Port agents base: volks/base.py → merge into BaseAgent (CXPMixin pattern).

Source clarification: Some volks exist in BOTH orchestrata-agents/volks/ (inline) AND as separate packages (orchestrata-<role>/). Port from the separate packages — they're the canonical, most complete versions. senior_dev.py is an exception: it only exists in orchestrata-agents/volks/senior_dev.py (no separate package).

Port 10 role packages + 1 inline volk into agents/volks/:

Also needed for CodeDirectorVolk: agents/swarm/coordinator.py (imports driver_factory at top level). Port SwarmCoordinator into agents/swarm/.

NOT Tier 1 (deferred to Tier 2):

Delete stubs after 90%+ test pass rate: base_agent.py, architect.py, code_director.py, quality_assurance.py, user_experience.py.


Tier 2: Coordination and Governance (~50,000 LOC)

Goal: Multi-agent coordination, message routing, governance enforcement. Done when: Submit intent → routes through Postman → agents collaborate → results aggregated.

2.1: Knowledge Subset (1,755 LOC)

Port only modules needed by Tier 2: context_injector.py (369), synchronizer.py (789), cxp_auditor.py (~200), neo4j_client.py (297), config.py + constants.py (~100).

2.2: Agent Infrastructure (5,000 LOC)

Port deferred Tier 1 files: runtime.py (AgentRuntime), pool.py (AgentPool), task_executor.py, coordinator.py (AgentCoordinator), escalation.py, lifecycle.py, prolog_bridge.py, intent_adapter.py.

2.2b: Verification Framework Remainder (~5,658 LOC)

Port orchestrata-verification minus the review/ subset already ported in Tier 1.5. Includes: constraints, checkpoints, proofs, autofix, edge cases, agents, escalation, orchestration, session verification, types. NOT a volk — test traceability infrastructure. Lives at orchestrata_ultimate/verification/.

2.3: Governance (5,736 LOC + prolog_gate.py)

Port: policy_engine.py, escalation_handler.py, constitutional.py, ethical.py, prolog_client.py, prolog_gate.py (174 LOC, required by kernel/pipeline.py), resource_access.py, safe_mode.py, cache.py, monitoring.py, audit.py, violations.py, dynamic_policy.py, types.py, interfaces.py, constants.py, exceptions.py. Port: configs/ YAML files and prolog/ rule files as package data. Prolog optional — factory returns PythonPolicyEngine if pyswip unavailable. Delete: governance/pam.py (stub).

2.4: Postman (12,674 LOC)

Port: postman.py, router.py, delivery.py, queue.py, priority.py, envelope.py, payload.py, guarantees.py, message_store.py, receipt_store.py, replay_engine.py, governance.py, spike_router.py (152 LOC, required by kernel/pipeline.py + agents/base.py), types.py, config.py, interfaces.py (keep as Protocols), exceptions.py. In-memory queues default, Redis optional via factory.

2.5: Kernel (15,272 LOC)

Port: kernel.py, event_bus.py, dispatcher.py (with conditional gRPC import), router.py, pipeline.py, pipeline_events.py (338 LOC, required by pipeline.py), workflow_engine.py, queue_manager.py, volk_registry.py, config_manager.py, policy_evaluator.py, pam_enforcer.py, performance_manager.py, audit_logger.py, session_manager.py, session_context.py, resource_monitor.py, error_handler.py. CLI files adapted to typer subcommands. EventBus: add os.environ.get() defaults, InMemoryEventBus fallback. Delete: core/orchestrator.py (stub). Update: __init__.py — change from orchestrata_ultimate.core.orchestrator import UltimateOrchestrator to from orchestrata_ultimate.kernel import OrchestrataKernel as UltimateOrchestrator (backward compat alias).

2.6: Orchestrator (11,332 LOC)

Port: intent parsing, fact management, policy engine, Redis state, Neo4j audit, impact analysis, Prolog bridge, NLP, metrics, workflow. Governance ↔ Orchestrator circular: TYPE_CHECKING + runtime lazy import.


Tier 3: Knowledge and Memory (~18,000 LOC)

Goal: Code understanding, persistent memory, session management.

3.1: Knowledge Remainder (6,694 LOC)

Port: code_parser.py, anomaly_detector.py, kg_design.py, types.py, types_nodes.py, search, query engine, dependency analyzer, file watcher, etc. Delete: knowledge_graph/graph.py (stub).

3.2: Session (10,595 LOC)

Port: session lifecycle, WorkItem, Goal, GoalProgress, Redis store, Neo4j store, checkpoint, KG context, hook system. Prolog optional in session hooks (guard with if self._prolog_bridge).

3.3: Memory (1,145 LOC)

Port: memory_engine.py, hot_store.py, cold_store.py, compressor.py, embedding.py, token_counter.py.


Tier 4: External Surface (~8,000 LOC)

Goal: WebSocket API, plugin system, external integrations.

4.1: Gateway (4,429 LOC)

Adapt to FastAPI. Keep JSON-RPC 2.0 and JWT auth. Port RPC handlers. Add pyjwt>=2.8.0 to dependencies. Delete: integrations/vizint.py (stub).

4.2: Plugin SDK (3,599 LOC)

Direct port. Adapt entry point discovery to orchestrata_ultimate_plugin_*. Also update ALLOWED_PLUGIN_PREFIXES in loader.py and config.py: "orchestrata.plugins." → "orchestrata_ultimate.plugins."

4.3: Integrations (build new)

CheetahClaws: build real TUI integration → orchestrata_ultimate/integrations/cheetahclaws/ Todoist: build real API client → orchestrata_ultimate/integrations/todoist/ Delete remaining stubs: cheetahclaws.py, todoist.py. The integrations/ directory stays — it hosts the real implementations replacing the stubs.


4. External Dependencies

Optional Infrastructure (factory pattern)

All external services use factory functions with in-memory fallbacks:

def create_event_bus(redis_url=None):
    return RedisEventBus(redis_url) if redis_url else InMemoryEventBus()

def get_policy_engine():
    try:
        from pyswip import Prolog
        return PrologPolicyEngine()
    except ImportError:
        return PythonPolicyEngine()
Service Required For Default Production
Redis EventBus, sessions, queues In-memory Redis
Neo4j Knowledge graph, audit Skip (empty context) Neo4j
Prolog (pyswip) Governance rules Python rule engine SWI-Prolog
gRPC CXP transport (optional) JSON/HTTP gRPC (conditional import)

Dependency Changes to pyproject.toml

# Bump
requires-python = ">=3.14"
"httpx>=0.28.0"

# Add
"watchfiles>=0.21"

# Add (dev)
"pytest-xdist>=3.5.0"
"pytest-timeout>=2.1.0"

# Add (Tier 4 only)
"pyjwt>=2.8.0"

# Remove (unused)
"asyncio>=3.4.3"    # stdlib, no-op shim

Package Data

[tool.hatch.build]
packages = ["src/orchestrata_ultimate"]

[tool.hatch.build.targets.wheel.force-include]
"configs" = "orchestrata_ultimate/configs"
"prolog" = "orchestrata_ultimate/prolog"

5. Test Strategy

Phase 0: Test Infrastructure First

Port BEFORE any code: root conftest.py, 18 per-directory conftest.py files, tests/fixtures/ (5 modules), conftest_infra_gate.py. Fix 31 failing generated tests. Result: 244 tests, 244 pass.

Per-Tier Pass Rate Targets

Tier New Tests Target Pass Rate Cumulative
Phase 0 0 new (fix existing) 100% 244
Tier 1 ~3,000 90% (2,700+) ~3,000
Tier 2 ~5,000 85% (4,250+) ~8,000
Tier 3 ~2,000 85% (1,700+) ~10,000
Tier 4 ~1,400 90% (1,260+) ~11,500
Final Debug remaining 95%+ ~11,900

Test Infrastructure Details


6. Communication Patterns

Three patterns coexist (no bridging needed):

Pattern Used For Mechanism
EventBus System lifecycle events (agent spawned, config changed) Redis pub/sub (or in-memory) — fire and forget
Postman Agent-to-agent CXP messages with delivery guarantees Routing + queues + retries + receipts
Direct calls Synchronous within orchestrator agent.execute_edit() method calls

Tier 1 uses InMemoryDispatcher (direct async routing). Tier 2 adds Postman (full guarantees).


7. Circular Dependency Resolution

Cross-Tier: Agents → Kernel (broken by splitting agents)

Within Tier 2: Kernel ↔ Governance ↔ Orchestrator

Tier 2 → Tier 3: Kernel → Knowledge (broken by knowledge subset)


8. Runtime Concerns

8.1: Environment Variable Hardcoding (95 locations)

Orchestrata has 95 os.environ["X"] reads (no defaults) across all packages. 20 are in kernel alone. Each tier port must include a bulk conversion:

# Per-tier audit during port:
grep -rn 'os\.environ\[' src/orchestrata_ultimate/<tier_pkg>/ --include="*.py"
# Replace each with os.environ.get("X", "<sensible_default>")

Defaults: REDIS_HOST=localhost, REDIS_PORT=6379, NEO4J_URI=bolt://localhost:7687, NEO4J_USERNAME=neo4j, NEO4J_PASSWORD="" (empty = skip connection).

8.2: Initialization Sequencing

Boot order for the full system (Tier 2+):

  1. Config — load environment, YAML configs
  2. Logging/Metrics/Tracing — observability must be first (ultimate's initialize_observability())
  3. EventBus — lazy-init on first access (property in kernel), in-memory fallback if no Redis
  4. Dispatcher/Postman — create InMemoryDispatcher (Tier 1) or Postman (Tier 2)
  5. Agents — instantiate BaseAgent subclasses, call set_postman(dispatcher) on each
  6. Kernel — register agents as components, wire EventBus subscriptions
  7. Gateway — start WebSocket server (Tier 4 only)

For Tier 1 (no kernel): steps 1-2-4-5 only. Orchestrator creates agents and injects dispatcher.

8.3: Orchestration-Level Error Handling

When call_llm() raises LLMClientError after exhausting retries:

This is how orchestrata already works — the pattern ports with the agent code. No new design needed.

8.4: Trace Context Through CXP

CXPPacket has correlation_id field. Propagation rules:

Wire during Tier 2: in Postman.aroute(), call set_request_id(packet.correlation_id) before dispatching.

8.5: Agent Authentication (Deferred)

Not needed for MVP (Tier 1). All agents run in-process, single-user. CXP signature field exists in the protocol but enforcement is a Tier 4 concern (gateway authenticates external clients). Internal agent-to-agent trust is implicit for local deployment.

8.6: VolkConfig Thread Safety

Fix during Tier 1.2 port:

import threading
_config_lock = threading.Lock()
_CONFIG: VolkConfig | None = None

def _get_config() -> VolkConfig:
    global _CONFIG
    if _CONFIG is None:
        with _config_lock:
            if _CONFIG is None:
                _CONFIG = VolkConfig()
    return _CONFIG

Double-checked locking. Standard pattern.

8.7: Development Workflow

Tier 1 agents are instantiated per-request (no persistent pool). Change code, re-run CLI command. Tier 2 agent pool supports terminate() + spawn() for replacing agents without full restart. Knowledge FileWatcher (Tier 3) is a background thread — watches source files, syncs to Neo4j. No special hot-reload mechanism needed beyond Python's normal module reloading.

8.8: Data Migration (Stubs → Real)

No migration needed. Stubs don't write to Redis or Neo4j (they return hardcoded data). When real implementations activate, they create fresh state. No legacy data to clean up.

8.9: File Locking for Concurrent Agent Edits

Multiple agents (CodeDirector, QA, Architect) can write to the same file in parallel. No fcntl/aiofiles locking exists in orchestrata. Fix during Tier 1 port:

import asyncio
_file_locks: dict[str, asyncio.Lock] = {}

async def acquire_file_lock(path: str) -> asyncio.Lock:
    if path not in _file_locks:
        _file_locks[path] = asyncio.Lock()
    return _file_locks[path]

Add to execute_multi_agent_edit() — acquire lock before parallel agent calls, release after. Prevents silent data loss from overlapping writes.

8.10: InMemoryDispatcher Backpressure

Tier 1 InMemoryDispatcher has no queue bounds. Fix: add asyncio.Queue with max size:

class InMemoryDispatcher:
    def __init__(self, max_pending: int = 1000):
        self._queue: asyncio.Queue = asyncio.Queue(maxsize=max_pending)
    # ...

Raises asyncio.QueueFull when backpressure threshold hit. Callers must handle. Postman (Tier 2) replaces this with BoundedQueue + overflow policies.

8.11: Agent Crash Isolation

If an agent's process() raises unhandled exception, the runtime marks it BUSY forever. Fix during Tier 2 agent infrastructure port:

8.12: CXP Packet TTL Enforcement

Orchestrata's Postman.route() does NOT check packet.is_expired() before queueing. Fix during Tier 2 Postman port: add TTL check at routing entry point:

async def aroute(self, packet: CXPPacket) -> CXPPacket | None:
    if hasattr(packet, 'is_expired') and packet.is_expired():
        logger.warning(f"Dropping expired packet {packet.message_id}")
        return None
    # ... normal routing

8.13: KG Context Size Limits

VolkContextInjector injects full Neo4j subgraph into every LLM prompt. With 11 agents this can produce 1-5MB of context per batch. Fix during Tier 3 knowledge port:

8.14: Graceful Shutdown for In-Flight Work

Kernel stop() doesn't wait for in-flight handlers. Fix during Tier 2 kernel port:


9. Stub Lifecycle

Stubs are NOT fallbacks for the factory pattern. The factory's in-memory implementation IS the permanent fallback. Stubs are temporary code returning fake data.

  1. Port real code alongside stubs (both exist in codebase)
  2. Wire orchestrator to use real code via factory
  3. Run tests — if 90%+ pass, delete stubs
  4. If <90% pass, debug ported code (stubs serve as reference for expected behavior)

10. Risks and Mitigations

Risk Mitigation
grpcio won't build on Python 3.14t Skip gRPC, conditional import in kernel dispatcher
Prolog (pyswip) unavailable Factory: PythonPolicyEngine fallback
Redis not available Factory: InMemoryEventBus, InMemoryQueue, InMemoryRegistry
Neo4j not available Graceful degradation: agents receive empty context
95 hard os.environ[] reads across all packages Bulk sed during each tier port: os.environ["X"] → os.environ.get("X", default)
119K LOC import translation Three-pass: bulk sed → manual audit → runtime fix
Circular deps in flat package TYPE_CHECKING + runtime lazy imports
Orchestrata moves during port Pin to commit, re-pin only after tier completion
Stubs deleted too early Keep until 90%+ test pass rate per tier

11. Success Criteria

Functional

Quantitative

Metric Before After
Source LOC 1,985 ~121,000
Packages with real code 7 17+
Stub modules 9 0
Test functions 213 passing ~11,500+ passing
Agents with LLM calls 0 11
External deps unused 8 0

12. Duplicate Spec Resolution

Pair Keep Archive
000-orchestrata-cli / 000-ultimate-cli 000-ultimate-cli 000-orchestrata-cli
003-intent / 003-intent-broker 003-intent-broker 003-intent
001-kernel / 001-ultimate-architecture 001-ultimate-architecture 001-kernel
007-prolog-governance / 035-verification Both (different scopes) Neither

Review History

13 rounds of adversarial review. 124 issues raised across build, architecture, QA, scope, runtime, edge-case, and implementation-feasibility perspectives. All resolved.

Round summary: R1-4 (structure, deps, architecture), R5-6 (verification, math), R7 (runtime init, error handling, env vars), R8 (dynamic imports, missing files), R9 (entry points, stub lifecycle, section numbering), R10 (first approval), R11 (async process(), Pydantic v2, dataclass scope), R12 (test factories, test infra), R13 (final sign-off).

Key corrections:

Created: 2026-04-10

Last updated: 2026-04-10 (post round 13 — FINAL APPROVAL)