Orchestrata Ultimate › Plans › spec-audit › A-core-runtime

Cluster A — Core runtime & protocol (SPEC-000…005)

Source: plans/spec-audit/A-core-runtime.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-04 · 274 lines.

Prose pending.

Plan text

Cluster A — Core runtime & protocol (SPEC-000…005)

Investigation agent A. Read-only audit of src/orchestrata_ultimate/ against the six specs, evidence-based (AST symbols/refs + targeted reads). Checklist - [ ] markers ignored per _METHOD.md. Builds on docs/spec-gap-register.md (002/003/004 already audited there as "clean" — confirmed/updated below with added orphan nuance).

Recurring theme across the cluster: the system evolved a fixed live orchestration path (gateway → PipelineRunner → CXP gRPC server → kernel_handler → 9 volks, plus SwarmCoordinator for junior fan-out). Many of the generic subsystems the specs describe (a do-everything Kernel, an IntentBroker facade, an ACP driver protocol, competitive delegation) exist and are unit-tested but are not on that live path — they are orphaned duplicates. That is where the real (mostly consolidation/spec-update) work sits, not in missing functionality — except SPEC-000, which is genuinely a stub CLI.

SPEC-000 Orchestrata CLI — PARTIAL (large remaining work)

• Specifies: a unified orchestrata Click command tree (init / start·stop·status / intent·respond / agents·packets / review·approve·reject / governance / verify / knowledge / session) delegating to the live subsystems, with an orchestrata console entry point.
• Verified:
  • ✅ Command-tree scaffold exists as a Click group — kernel/cli_main.py:302 (cli()), with start, version, health, init, stop, status, config team, agents list/health/packets, reject, governance check/log/policies, verify, knowledge query, session groups all declared.
  • ✅ start is real — boots kernel+postman+state+registry+Prolog+CXP volk kernel (cli_main.py:250 _boot_monolithic, 339 start). version/health real.
  • ✅ Config load with ${VAR:-default} env resolution — cli_main.py:36 _load_config, 53 _resolve_env_vars. JSON/text logging (73 _setup_logging).
  • ❌ No orchestrata console-script entry point. pyproject.toml [project.scripts] (lines 71-79) defines ast-sync, orchestrata-tools, session, ast-query, … but no orchestrata = "...cli_main:main" — the spec calls this out as required for uv workspace compat (spec.md:267-271). The CLI is unreachable as orchestrata ….
  • ❌ Four spec-required commands entirely absent: intent, respond, review, approve (US3/US5). Only reject exists — grep of cli_main.py shows no def intent, def respond, def review, def approve.
  • ⚠️ Stubs that print "not yet implemented" and/or raise SystemExit(1): init (cli_main.py:396), stop (419), config team/team_remove (474/487), agents packets (533), reject (558), governance check (582), verify (626).
  • ⚠️ Hardcoded empty responses (no live introspection): status always prints "Runtime state: stopped / Active volks: none" (cli_main.py:440), agents list/health "No volks registered" (505/514), governance log/policies empty (591/600), knowledge query "Knowledge graph not available" (654), session prints a redirect string with no subcommands (679; spec US9 wants status/list/end/resume/facts).
• Remaining work / plan:
  • Scope: (1) add the orchestrata console entry point; (2) implement the 4 missing verbs (intent, respond, review, approve) by delegating to the live path; (3) replace the stub/hardcoded commands with real introspection (status/agents query the running kernel/registry; governance query the Prolog/PolicyEngine; knowledge query Neo4j; session wrap the existing session CLI); (4) init/stop real project + runtime lifecycle.
  • Target files: pyproject.toml ([project.scripts]); src/.../kernel/cli_main.py (all stub commands); likely thin new delegators to orchestrator/pipeline_runner.py, gateway/ (intent submit), governance/policy_engine.py, knowledge_graph/neo4j_client.py, session/cli.py.
  • Approach: intent/respond/review/approve/reject should send CXP packets into the live path (gateway WS or in-proc PipelineRunner) rather than re-implement routing. status/agents need a runtime handle — either query a running daemon over the health/WS port or connect to Redis/registry directly. session * should shell to the existing session console script (already installed) instead of a parallel impl.
  • Dependencies: gateway (SPEC-014, other cluster) or PipelineRunner (SPEC-005) for intent; governance (SPEC-007) for governance; knowledge (SPEC-008) for knowledge; session (SPEC-006) for session; verification module for verify. This CLI is the human seam over most other clusters — highest cross-cluster coupling in the audit.
  • Effort: L (>2d) — 4 new commands + ~8 stub replacements each needing a live handle. Risk: med — needs a decision on CLI↔runtime transport (daemon/WS vs direct), and whether to keep the fixed-pipeline model (no free-form intent routing exists yet).
• Evidence log: read cli.py (it is the FastAPI app, not the CLI); read full kernel/cli_main.py; grep -n '\[project.scripts\]' pyproject (no orchestrata); refs OrchestrationKernel → imported by cli_main:137.

SPEC-001 Orchestration Kernel (U:KR) — PARTIAL (orphaned generic subsystems)

• Specifies: central kernel doing Volk registry, message routing, Prolog governance, session lifecycle, escalation, workflow orchestration, resource mgmt, audit, config hot-reload — the "central nervous system."
• Verified: the spec's monolith is decomposed into sibling modules; the live kernel is a thin coordinator, and several capability modules are implemented + unit-tested but wired to nothing in src/:
  • ⚠️ OrchestrationKernel (kernel/kernel.py:47) is only a component registry + service discovery + EventBus (capabilities list literally ["component_registry","service_discovery","event_bus"], kernel.py:72). It does not itself route/govern/escalate. Wired: imported by cli_main.py:137, cli.py:24.
  • ✅ Volk registry — VolkRegistry wired (used by agents/runtime.py:82, kernel/router.py:216, cli_main.py:172). refs VolkRegistry → 4 uses.
  • ✅ Audit trail — AuditLogger (kernel/audit/logger.py:81) comprehensive (routing/policy/ escalation/session/resource/workflow/volk log methods + Neo4j/file + query/report).
  • ✅ Governance on the live path is governance/policy_engine.py PolicyEngine (refs PolicyEngine → used by gateway/server.py:270) + ConstitutionalEnforcer (refs → used by policy_engine.py:130), on Prolog/janus — not the kernel's own module.
  • ❌ WorkflowEngine (US-006, kernel/workflow_engine.py:313) — 0 imports, 0 uses. Orphaned. Live orchestration is the fixed PipelineRunner, not a generic DAG engine.
  • ❌ ResourceMonitor (US-007, kernel/resource_monitor.py:124) — 0 imports, 0 uses. Orphaned; no per-volk CPU/mem quota enforcement on the live path.
  • ❌ PolicyEvaluator (kernel/policy_evaluator.py:289) — 0 refs; kernel/router.py CXPRouter (:204) — 0 src uses (the live CXPRouter is cxp/router.py:53, used by cxp/dispatcher.py:72); kernel/session_manager.py:217 SessionManager — 0 refs (live session is the session/ controller); pam/EscalationManager (pam/escalation.py:70) — exported in pam/__init__ but 0 uses. All tested (tests/kernel/…) but unwired.
  • ✅ Config — ConfigManager (kernel/config_manager.py:251) with yaml/env/redis load + schema; hot-reload machinery present.
• Remaining work / plan:
  • Scope: a wire-or-delete decision for the orphaned generic subsystems (WorkflowEngine, ResourceMonitor, PolicyEvaluator, kernel-level CXPRouter, SessionManager, EscalationManager). Either wire them into the running kernel/boot, or delete + update the spec to state the live model is the fixed PipelineRunner + CXP router + governance PolicyEngine.
  • Target files: kernel/workflow_engine.py, kernel/resource_monitor.py, kernel/policy_evaluator.py, kernel/router.py, kernel/session_manager.py, kernel/pam/escalation.py; boot wiring in kernel/cli_main.py if wiring rather than deleting; specs/001-kernel/spec.md for the divergence.
  • Approach: recommend documenting the divergence (spec is out of date: the "do-everything kernel" is realized as decomposed + a fixed pipeline) and deleting the truly-dead generic routers/engines, keeping VolkRegistry/AuditLogger/ConfigManager. US-006 workflow DAG and US-007 resource quotas are genuinely unbuilt-on-the-live-path capabilities if the product wants them.
  • Dependencies: couples to SPEC-003 (orphaned IntentBroker, same "generic vs pipeline" theme), SPEC-005 (PipelineRunner is the de-facto kernel), and the register's SPEC-023 goal-system / SPEC-010 memory orphan-consolidation theme — one shared "what is the canonical engine" call.
  • Effort: M (0.5–2d, mostly decision + deletion + spec edit; L if US-006/007 must be built for real). Risk: med — deleting tested code needs sign-off it is not a planned surface.
• Evidence log: refs OrchestrationKernel|WorkflowEngine|ResourceMonitor|EscalationManager| MessageDispatcher|PolicyEvaluator|VolkRegistry|SessionManager|CXPRouter; read kernel/kernel.py; ls kernel/; grepped tests/ (orphans are covered by tests/kernel/* — tested, not wired).

SPEC-002 Postman Message Broker (U:PS) — DONE (confirms register)

• Specifies: reliable delivery, topics/subscriptions, queues (FIFO/priority/DLQ/TTL), priorities, filtering/routing, persistence/replay, delivery guarantees, metrics, config.
• Verified:
  • ✅ Postman wired into the live path — imported by orchestrator/pipeline_runner.py:120, agents/volks/base.py:21, kernel/pipeline.py:33, cli_main.py:138.
  • ✅ Delivery guarantees — DeliveryGuaranteeManager (postman/guarantees.py:91) implements AT_MOST_ONCE / AT_LEAST_ONCE / EXACTLY_ONCE with dedup (guarantees.py:159,176-177).
  • ✅ Full subsystem set present: topic_registry.py, subscription_manager.py, queue_manager.py/queue.py, priority.py, filter.py/routing_rules.py, message_store.py/replay_engine.py, metrics.py, delivery.py, receipt_store.py, config.py, health.py, event_emitting_postman.py.
• Remaining work: none material. (Perf targets — 5000 msg/s etc. — are runtime SLAs, not code gaps.) Matches the register's "clean" verdict.
• Evidence log: ls postman/; refs Postman; grep guarantees.py for DeliveryMode/dedup.

SPEC-003 Intent Broker (U:Intents) — DONE, with an orphaned facade (updates register)

• Specifies: parse NL → structured intent, Prolog-validate, route to volks, lifecycle, history, templates, multi-intent workflows, metrics, security.
• Verified: every named subcomponent exists in orchestrator/:
  • ✅ intent_parser.py (IntentParser used by orchestrator/engine.py:276), intent_validator.py, intent_router.py, intent_lifecycle.py, intent_history.py, intent_template.py, intent_workflow.py, intent_metrics.py, intent_security.py, intent_enums.py (IntentAction/IntentTarget), parameter_extractor.py, nlp.py, ambiguity_resolver.py.
  • ❌ IntentBroker facade (orchestrator/intent_broker.py:83) — 0 imports, 0 uses. Its IntentValidator/IntentRouter are used only inside intent_broker.py (:63/:64). The live intent handling is the concierge/pipeline path (SPEC-032 ConciergeVolk → PipelineRunner), not this broker → orphaned facade.
• Remaining work / plan: not a functional gap (capabilities delivered via the pipeline). Consolidation call: delete IntentBroker (+ note the live path is concierge/pipeline) or wire it if a standalone broker API is wanted (e.g. for the SPEC-000 orchestrata intent command). Target: orchestrator/intent_broker.py, specs/003-intent/spec.md. Depends on the SPEC-000 intent command decision and SPEC-032 concierge. Effort: S. Risk: low.
• Evidence log: ls orchestrator/; refs IntentBroker (0 use), refs IntentParser/ IntentValidator/IntentRouter.

SPEC-004 CXP Protocol — DONE (confirms register)

• Specifies: 11-field CXPPacket, header mgmt, payload (JSON/binary), delivery-status & work-status state machines, JSON+MessagePack (de)serialization, validation/schema, thread correlation, priority/ordering, versioning, and the fix_issues review subject.
• Verified:
  • ✅ CXPPacket + enums with enforced transitions — cxp/delivery_status.py:63 uses DELIVERY_TRANSITIONS, cxp/work_status.py:43 uses WORK_TRANSITIONS (validated in packet.py). is_*/history methods present.
  • ✅ Serialization — json_serializer.py, binary_serializer.py (to_bytes/from_bytes MessagePack, :151/:191), serializer.py.
  • ✅ Threads — threads.py:199 get_thread_packets, thread_store.py. Versioning — version_compat.py:72 check_compatibility, version.py. Validation — validator.py, schema.py.
  • ✅ fix_issues subject works: subject is a free-form string ≤200 chars (validator.py:294 _check_subject, schema.py:67), so fix_issues (and review_code, review_complete) are valid by construction; consumed by codedirector/volk.py:94,277.
  • ⚠️ Divergence (not a gap): the spec's US-011 "subject validation includes new fix_issues value" implies an enum; the code uses free strings (no subject enum). Functionally equivalent; update the spec wording if desired.
• Remaining work: none material. Confirms register "clean."
• Evidence log: symbols check_compatibility|to_bytes|from_bytes|get_thread_packets; grep subject validation; grep fix_issues across cxp/core (only free-string usage, no enum).

SPEC-005 Multi-Agent Coordination — DONE, with divergences + orphan duplicates

• Specifies: lightweight agent spawn, escalation paths, lifespan/cleanup, monitoring, constitutional constraints, junior-volk drivers (CLI/API/Browser via an ACPDriver protocol), senior→junior delegation over CXP, parallel delegation + synthesis, browser free-tier LLMs, and the U:Qa↔U:Vz code-review loop (fix_issues, MAX_REVIEW_CYCLES).
• Verified:
  • ✅ Live coordination path = PipelineRunner (orchestrator/pipeline_runner.py:101) run via build_kernel_packet_handler (agents/kernel_handler.py:35, wired into cli_main.py:213 and runtime_main.py:49) driving the 9 volks.
  • ✅ Constitutional constraints wired — ConstitutionalConstraints (agents/constraints.py:126, 3 uses) + governance ConstitutionalEnforcer. Hierarchy wired — AgentHierarchy (used by agents/lifecycle.py:117, 4 uses).
  • ✅ Junior volks + drivers exist — JuniorVolk (agents/volks/junior/volk.py:87); drivers/: claude_code_driver.py, aider_driver.py, deepseek_coder_driver.py, qwen_driver.py, browser_driver.py (Playwright, US-010), claude_api_driver.py, driver_factory.py.
  • ✅ Parallel delegation — SwarmCoordinator (agents/swarm/coordinator.py, 3 uses) fans shards to parallel juniors (codedirector/implementation.py:81 get_fan_out_decision, 115 execute_swarm_path; codedirector/volk.py:122).
  • ✅ Code-review loop — codedirector/volk.py:277 _handle_fix_issues; QA bounce-back with escalating context (agents/volks/qa/_context.py:297 G227/G229); capability routing (TESTING, IMPLEMENTATION) → "fix_issues" (orchestrator/pipeline_router.py:141).
  • ⚠️ Divergence 1 — driver interface. drivers/base.py is AsyncCodeDriver with a single execute(task, files, is_fix) -> (success, output) — not the spec's ACPDriver (initialize/send_task(CXPPacket)/get_response/cleanup). Simpler, real, but the spec text is stale.
  • ⚠️ Divergence 2 — delegation model. Implemented as file-sharded swarm (each junior gets different files), not the spec's competitive "send the same task to N juniors, compare quality, select best" (US-009). No competitive synthesis of redundant outputs.
  • ⚠️ Divergence 3 — review-cycle mechanics. No MAX_REVIEW_CYCLES constant; cycles are capped per-volk (codedirector/prompts.py:6 MAX_SELF_REVIEW_CYCLES=3, architect MAX_DESIGN_CYCLES=99, junior MAX_DESIGN_REVIEW_CYCLES=3) and routed via the capability graph, not the spec's "kernel tracks review_cycle in packet metadata & escalates to U:Ox at MAX." (Also see register SPEC-029: 3-vs-99 is an open product call.)
  • ❌ Orphan duplicates (0 src uses): AgentCoordinator.fan_out (agents/coordinator.py:297), AgentEscalation (agents/escalation.py:85), EscalationEngine, verification/escalation_handler.py:28 EscalationHandler. VolkRuntime/AgentRuntime (agents/runtime.py:61/500) are used inside the agents subsystem but the live boot deliberately bypasses them (cli_main comment: the old in-proc VolkRuntime "drove an empty registry and could spawn nothing (gap W6)").
• Remaining work / plan: no functional hole. (1) Update specs/005-multi-agent/spec.md for the three divergences (driver interface, sharded-vs-competitive, cycle mechanics). (2) Delete the orphaned duplicate coordinator/escalation classes (or wire one as canonical). If the product actually wants competitive "select-best" delegation, that is net-new (M/L). Target files: the spec; agents/coordinator.py, agents/escalation.py, verification/escalation_handler.py. Dependencies: SPEC-004 (fix_issues subject), SPEC-029 (codedirector cycle cap), SPEC-013 QA (other cluster). Effort: S for spec-update + orphan cleanup; M/L only if competitive delegation is built. Risk: low.
• Evidence log: symbols CLIDriver|APIDriver|BrowserDriver|JuniorVolk; refs fan_out| SwarmCoordinator|ConstitutionalConstraints|AgentHierarchy|AgentEscalation|VolkRuntime| PipelineRunner|build_kernel_packet_handler; read drivers/base.py, codedirector/implementation.py; grep MAX_*CYCLE, swarm coordinator (sharding).

Cluster summary table

Real remaining work: SPEC-000 (substantial — genuine stub CLI); SPEC-001 (orphan wire-or-delete + spec update); minor consolidation for SPEC-003/005. SPEC-002/004 clean.

Cross-cluster deps noticed:

• SPEC-000 is the human seam over almost everything: its intent/respond/review/approve need the gateway (SPEC-014, cluster w/ gateway) or PipelineRunner; governance→SPEC-007; knowledge→SPEC-008; session→SPEC-006 (a session console script already exists); verify→verification module.
• SPEC-001/003/005 share one "what is the canonical orchestration engine" decision (the live fixed PipelineRunner vs the generic Kernel/IntentBroker/AgentCoordinator duplicates) — the same consolidation theme the register already flags for SPEC-023 (goal system) and SPEC-010 (memory). Resolve these together, not piecemeal.
• SPEC-005 review loop couples to SPEC-004 (fix_issues subject) and SPEC-029 (codedirector MAX_SELF_REVIEW_CYCLES 3-vs-99, already an open call in the register) and SPEC-013 QA.

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks