Orchestrata Ultimate / Specs / 005-multi-agent
005-multi-agent: Multi-Agent Coordination
The Multi-Agent Coordination system manages the lifecycle, communication, and escalation of Volks (agents) in Orchestrata. It supports lightweight Prolog agents, hierarchical agent organization, constitutional constraints, and the Junior Volk delegation system for parallel task execution.
At a glance
| Item | Value |
|---|---|
| Spec directory | specs/005-multi-agent/ |
| Status line in the spec | Complete |
| Requirement IDs defined | 48 |
With a @traces citation in src/ |
25 (52.1%) |
With a @traces citation in tests/ |
0 (0.0%) |
| Tracked files | 8 |
| RTM page | 005-multi-agent |
Reading this spec
Multi-Agent Coordination manages the lifecycle, communication and escalation of Volks (agents). It supports lightweight Prolog agents, hierarchical agent organization, constitutional constraints, and the Junior Volk delegation system for parallel task execution.
The scope is a set of user stories: lightweight agent spawning, escalation paths, agent lifespan management, agent monitoring, constitutional constraints and agent escalation, followed by the Junior Volk stories: spawning junior Volks, senior-to-junior delegation, parallel delegation and synthesis, and browser-based free-tier access. Requirements use the REQ-005- prefix; the spec defines 48.
The design in the overview rests on a Volk Registry for discovery, hierarchical parent-child relationships, Prolog-driven constitutional validation, the CXP protocol for all inter-agent communication, and Junior Volk drivers for the CLI, an API and a browser. The aims are to coordinate specialized agents, resolve escalations and conflicts, enforce constraints across agents and manage the spawn, monitor and cleanup cycle.
In the code, the package is orchestrata_ultimate/agents. The largest tracers are orchestrata_ultimate/agents/lifecycle, orchestrata_ultimate/agents/escalation, orchestrata_ultimate/agents/runtime, orchestrata_ultimate/agents/hierarchy and orchestrata_ultimate/agents/prolog_bridge. The per-requirement view is on the RTM page.
The spec's status line reads Complete. It tracks 8 files and defines 48 requirement IDs. The full text follows below.
Measured coverage is 25 of 48 requirements cited by at least one @traces in src/ (52.1%), spread over 12 modules; the module citing the most of them is orchestrata_ultimate/agents/lifecycle (13 requirements). In tests/, 0 requirements are cited by a test-side @traces.
Coverage gaps: 23 requirements have no citing symbol. The largest untraced groups are REQ-005-NFR2 (1), REQ-005-NFR3 (1), REQ-005-US-007 (1). Untraced means no @traces names the ID; the behaviour may still exist in code that was never annotated. Test-side citations are thinner than code-side ones (0 against 25).
Requirements
Checkbox is the state recorded in the requirements checklist; Traced by is the count of src/ symbols whose @traces names the ID.
| ID | Done | Requirement | Traced by |
|---|---|---|---|
REQ-005-US-001-001 |
yes | As a developer, I can spawn lightweight Prolog agents that handle specialized tasks without creating process overhead | 10 |
REQ-005-US-002-001 |
yes | As a system architect, I can define agent escalation paths that automatically promote issues requiring broader context | 3 |
REQ-005-US-003-001 |
yes | As a developer, I can configure agent lifespans to automatically terminate agents when their task is complete | 6 |
REQ-005-US-004-001 |
yes | As a system operator, I can monitor agent spawning metrics and performance to ensure system health | 3 |
REQ-005-US-005-001 |
yes | As a developer, I can define constitutional constraints that prevent agents from violating system invariants | 3 |
REQ-005-US-006-001 |
yes | As an agent, I can escalate to a higher-level agent when I encounter situations outside my constitutional bounds | 4 |
REQ-005-FR1-001 |
yes | Agent spawn time < 100ms (measured from request to first query execution) | 3 |
REQ-005-FR2-001 |
yes | Escalation path resolution < 10ms (measured from escalation trigger to parent agent identification) | 3 |
REQ-005-FR3-001 |
yes | Agent lifecycle management with automatic cleanup when task completes | 22 |
REQ-005-FR4-001 |
yes | Constitutional constraint validation at agent spawn and during execution | 13 |
REQ-005-FR5-001 |
yes | Hierarchical agent organization with parent-child relationships | 18 |
REQ-005-FR6-001 |
yes | Agent context isolation with controlled fact visibility | 2 |
REQ-005-FR7-001 |
yes | Escalation mechanism with automatic parent notification | 6 |
REQ-005-FR8-001 |
yes | Performance metrics collection for agent spawn, query, and escalation operations | 4 |
REQ-005-NFR1-001 |
yes | System maintains performance with 100+ concurrent lightweight agents | 4 |
REQ-005-NFR2-001 |
yes | Agent spawn overhead < 1MB memory per agent | none |
REQ-005-NFR3-001 |
yes | Constitutional validation adds < 5ms to spawn time | none |
REQ-005-NFR4-001 |
yes | Escalation paths are deterministic and reproducible | 3 |
REQ-005-NFR5-001 |
yes | System remains responsive during agent cleanup operations | 9 |
REQ-005-AC1-001 |
yes | Agent spawn time verified < 100ms in benchmark tests | 1 |
REQ-005-AC2-001 |
yes | Escalation path resolution verified < 10ms in benchmark tests | 2 |
REQ-005-AC3-001 |
yes | Constitutional constraints validated at spawn and runtime | 9 |
REQ-005-AC4-001 |
yes | Parent-child relationships maintained correctly during escalation | 14 |
REQ-005-AC5-001 |
yes | Automatic cleanup verified for completed agents | 5 |
REQ-005-AC6-001 |
yes | Performance metrics collected and accessible via monitoring interface | 13 |
REQ-005-US-007-001 |
no | As a senior volk, I can spawn junior volks with configurable ACP drivers so that I can delegate implementation tasks to specialized coding tools | none |
REQ-005-US-008-001 |
no | As a senior volk, I can delegate tasks to junior volks via CXP packets and receive their output through the same protocol | none |
REQ-005-US-009-001 |
no | As a senior volk, I can fan out the same task to multiple junior volks in parallel, evaluate all outputs, and synthesize the best elements into a final deliverable | none |
REQ-005-US-010-001 |
no | As the system, I can drive browser-based free-tier LLM chat interfaces via automation so that junior volks can use Qwen Chat, DeepSeek, and Kimi at zero cost | none |
REQ-005-FR9-001 |
no | Junior volk spawns with driver initialization — CLI driver <500ms, API driver <200ms, Browser driver <5s | none |
REQ-005-FR10-001 |
no | Senior-to-junior delegation uses standard CXP packets routed through Postman, identical to senior-to-senior communication | 2 |
REQ-005-FR11-001 |
no | Parallel delegation sends same CXP task packet to N junior volks simultaneously (max 5 per senior volk) | none |
REQ-005-FR12-001 |
no | Senior volk receives all junior outputs, compares quality, selects best elements, and produces synthesized final output | none |
REQ-005-FR13-001 |
no | Browser driver manages Playwright sessions with login persistence, timeout handling, and session recovery | none |
REQ-005-FR14-001 |
no | Free-tier quota tracking per ACP — system tracks remaining quota and falls back to alternative ACPs when exhausted | none |
REQ-005-FR15-001 |
no | Driver interface is uniform — senior volk code is identical regardless of whether junior uses CLI, API, or browser backend | none |
REQ-005-FR16-001 |
no | Junior volk output is always a CXP packet with work_status and payload — driver converts ACP-native output to CXP format | none |
REQ-005-NFR6-001 |
no | CLI/API driver task kickoff response < 500ms (p95) | none |
REQ-005-NFR7-001 |
no | Browser driver task kickoff response < 5s (p95) including page load | none |
REQ-005-NFR8-001 |
no | Senior volk synthesis decision < 30s (p95) for comparing up to 5 junior outputs | none |
REQ-005-NFR9-001 |
no | Browser sessions remain stable for up to 30 minutes of interaction | none |
REQ-005-NFR10-001 |
no | System degrades gracefully when all free-tier quotas are exhausted — falls back to paid ACPs or alerts human | none |
REQ-005-AC7-001 |
no | Junior volk spawned with CLI driver produces output within 60s for a simple coding task | none |
REQ-005-AC8-001 |
no | Junior volk spawned with Browser driver produces output within 120s for a simple coding task | none |
REQ-005-AC9-001 |
no | Comparison delegation with 3 junior volks produces 3 distinct outputs and senior selects/synthesizes | none |
REQ-005-AC10-001 |
no | Quota exhaustion on one ACP triggers automatic fallback to alternative ACP | none |
REQ-005-AC11-001 |
no | All junior volk communication uses CXP — no direct ACP-native protocol exposed to senior volk | 2 |
REQ-005-AC12-001 |
no | Browser driver handles session timeout and re-authenticates without data loss | none |
Code that traces this spec
Checklists
checklists/code_review.md
Code Review Checklist — Google Python Style Guide
Reference: https://google.github.io/styleguide/pyguide.html
1. Formatting
- [ ] Line length: Maximum 80 characters (Black allows 88)
- [ ] Indentation: 4 spaces, no tabs
- [ ] No semicolons at end of lines
- [ ] Blank lines: 2 between top-level definitions, 1 between methods
2. Imports
- [ ] Order: stdlib → third-party → local
- [ ] No relative imports (
from . importprohibited) - [ ] Use aliases only for standard abbreviations (
numpy as np) - [ ] **No import *** (wildcard imports)
3. Naming Conventions
- [ ] Modules:
snake_case.py - [ ] Classes:
PascalCase - [ ] Functions/variables:
snake_case - [ ] Constants:
UPPER_SNAKE_CASE - [ ] Private:
_leading_underscore - [ ] No
l,O,Ias single-char names
4. Type Hints (Required for Public APIs)
- [ ] Return type annotations on all functions/methods
- [ ] Parameter type annotations on all functions/methods
- [ ] Use
Optional[T]instead ofT | Nonefor compatibility - [ ] Avoid
Anyunless absolutely necessary - [ ] Use type aliases for complex types
5. Docstrings (Google Style)
- [ ] Module-level docstring on all
.pyfiles - [ ] Class docstrings explaining purpose and usage
- [ ] Function docstrings with:
- [ ] One-line summary (≤80 chars)
- [ ] Blank line after summary
- [ ] Args: section with types
- [ ] Returns: section with types
- [ ] Raises: section for exceptions
- [ ] Example: section if helpful
6. Code Quality
- [ ] No mutable default arguments (
def foo(a=[])→ useNone) - [ ] No bare
except:— catch specific exceptions - [ ] Use implicit False:
if foo:notif foo != [] - [ ] No power features: no metaclasses,
__del__, etc. - [ ] Max function length: ≤40 lines (split if longer)
- [ ] Max cyclomatic complexity: ≤10
7. Whitespace
- [ ] No trailing whitespace
- [ ] Binary operators spaced:
x == 1notx==1 - [ ] No space before comma/semicolon/colon
- [ ] No space inside parentheses/brackets/braces:
spam(ham[1]) - [ ] No vertical alignment of tokens (
:,#,=)
8. Strings
- [ ] Use f-strings for formatting
- [ ] No line continuation with backslashes (use parentheses)
- [ ] Logging strings: Use lazy formatting
9. Testing Requirements
- [ ] No mocking in tests (use real infrastructure)
- [ ] Tests use
uv run pytest tests/ - [ ] Coverage ≥90%
10. Project-Specific Rules
- [ ] Use
uvfor package management (notpip) - [ ] Absolute imports only (no
from ..) - [ ] Run formatters:
black,isortbefore commit - [ ] Run linter:
flake8 - [ ] Run type checker:
mypy
Quick Reference
# Format code
uv run black packages/ tests/
uv run isort packages/ tests/
# Lint
uv run flake8 packages/ tests/
# Type check
uv run mypy packages/
# Run tests (use local-ci, not raw pytest)
ci/local-ci.sh test
checklists/constitutional.md
Constitutional Constraints Checklist
Article I: Agent Identity and Scope
- [ ] 1.1: Each agent has a unique identifier assigned at spawn time
- [ ] 1.2: Agent scope is defined by constitutional constraints at creation
- [ ] 1.3: Agent identity is immutable after spawn
- [ ] 1.4: Agent scope constraints are validated before first query execution
Article II: Fact Visibility and Isolation
- [ ] 2.1: Agent can only access facts within its constitutional scope
- [ ] 2.2: Agent cannot modify facts outside its constitutional scope
- [ ] 2.3: Agent context is isolated from other agents unless explicitly shared
- [ ] 2.4: Parent agent facts are visible to child agents (read-only)
Article III: Escalation Rights and Obligations
- [ ] 3.1: Agent must escalate when encountering queries outside constitutional scope
- [ ] 3.2: Agent cannot suppress or ignore escalation requirements
- [ ] 3.3: Escalation path is determined by constitutional hierarchy
- [ ] 3.4: Agent must preserve context during escalation
Article IV: Lifecycle Management
- [ ] 4.1: Agent lifespan is defined at spawn (task-bound or session-bound)
- [ ] 4.2: Agent must complete cleanup before termination
- [ ] 4.3: Agent cannot extend its own lifespan beyond constitutional limits
- [ ] 4.4: Parent agent is notified when child agent terminates
Article V: Performance Constraints
- [ ] 5.1: Agent spawn time must not exceed 100ms
- [ ] 5.2: Escalation path resolution must not exceed 10ms
- [ ] 5.3: Agent query execution must respect timeout constraints
- [ ] 5.4: Agent must yield to higher-priority operations
Article VI: Resource Limits
- [ ] 6.1: Agent memory usage must not exceed constitutional limits
- [ ] 6.2: Agent cannot spawn child agents beyond depth limit
- [ ] 6.3: Agent must respect system-wide resource constraints
- [ ] 6.4: Agent must release resources upon termination
Article VII: Error Handling
- [ ] 7.1: Agent must report errors to parent agent or monitoring system
- [ ] 7.2: Agent cannot silently fail or suppress errors
- [ ] 7.3: Agent must preserve error context for debugging
- [ ] 7.4: Agent must attempt graceful degradation before escalation
Article VIII: Query Constraints
- [ ] 8.1: Agent can only execute queries within constitutional scope
- [ ] 8.2: Agent cannot modify global state without escalation
- [ ] 8.3: Agent must validate query parameters against constitutional constraints
- [ ] 8.4: Agent must log all queries for audit purposes
Article IX: Inter-Agent Communication
- [ ] 9.1: Agent can communicate with parent and child agents only
- [ ] 9.2: Agent cannot directly communicate with sibling agents
- [ ] 9.3: Agent must use escalation mechanism for cross-hierarchy communication
- [ ] 9.4: Agent communication must be asynchronous and non-blocking
Article X: Monitoring and Observability
- [ ] 10.1: Agent must expose performance metrics (spawn time, query count, escalations)
- [ ] 10.2: Agent must log lifecycle events (spawn, escalate, terminate)
- [ ] 10.3: Agent must support health checks from monitoring system
- [ ] 10.4: Agent metrics must be accessible without blocking execution
Article XI: Constitutional Amendment
- [ ] 11.1: Agent constitutional constraints are immutable after spawn
- [ ] 11.2: System-wide constitutional updates require explicit approval
- [ ] 11.3: Constitutional changes must be versioned and auditable
- [ ] 11.4: Agent must terminate if constitutional constraints become invalid
Validation Requirements
- [ ] V1: All constitutional constraints validated at agent spawn time
- [ ] V2: Runtime constitutional violations trigger automatic escalation
- [ ] V3: Constitutional compliance verified in integration tests
- [ ] V4: Constitutional audit logs maintained for all agents
Junior Volk Constitutional Amendments
Article IX Addendum: Senior-Junior Communication
- [ ] 9.5: Junior volks communicate exclusively with their parent senior volk
- [ ] 9.6: Junior volks cannot initiate communication with the human or other senior volks
- [ ] 9.7: All junior volk output is routed through the parent senior volk before delivery
- [ ] 9.8: Senior volk may communicate with multiple junior volks simultaneously via CXP
Article XII: Junior Volk Constraints
- [ ] 12.1: Junior volks are transient — spawned for a task, terminated on completion
- [ ] 12.2: Junior volks cannot spawn child volks (max depth = 1 below senior)
- [ ] 12.3: Junior volks cannot access the knowledge graph directly — all knowledge requests go through parent
- [ ] 12.4: Junior volks cannot modify governance rules or Prolog facts
- [ ] 12.5: Junior volk output must be validated by parent senior volk before it becomes an artifact
- [ ] 12.6: Junior volks have no persistence — no session state survives termination
- [ ] 12.7: A senior volk may have at most 5 concurrent junior volks
- [ ] 12.8: Junior volks inherit the session scope of their parent but cannot expand it
Article XIII: ACP Driver Safety
- [ ] 13.1: Browser drivers run in sandboxed browser contexts — no access to host filesystem
- [ ] 13.2: ACP credentials (API keys, browser cookies) are isolated per driver instance
- [ ] 13.3: Browser automation cannot navigate outside approved ACP domains
- [ ] 13.4: Free-tier quota limits are enforced — driver refuses tasks when quota is exhausted
- [ ] 13.5: Driver sessions have a maximum lifetime (30 minutes) with forced termination
- [ ] 13.6: No ACP credentials are stored in the knowledge graph or session state
- [ ] 13.7: Browser driver screenshots and DOM state are ephemeral — destroyed on termination
- [ ] 13.8: Driver failures are non-fatal — senior volk continues with remaining junior volks
Junior Volk Validation Requirements
- [ ] JV1: All junior volk constitutional constraints validated at spawn time
- [ ] JV2: Runtime violations in junior volks trigger immediate termination (not escalation)
- [ ] JV3: Driver safety constraints verified in integration tests per driver type
- [ ] JV4: Quota enforcement verified with exhaustion scenarios
- [ ] JV5: Senior-junior communication isolation verified — no cross-talk between juniors
checklists/requirements.md
The requirement list above comes from this file; 25 of 48 checkbox items are ticked.
checklists/testing.md
Testing Checklist
Performance Tests
Agent Spawn Performance
- [ ] T1.1: Measure agent spawn time (target: < 100ms)
- [ ] T1.2: Verify spawn time remains < 100ms with 10 concurrent spawns
- [ ] T1.3: Verify spawn time remains < 100ms with 100 concurrent spawns
- [ ] T1.4: Measure memory overhead per agent (target: < 1MB)
- [ ] T1.5: Verify spawn performance with constitutional constraint validation
Escalation Performance
- [ ] T2.1: Measure escalation path resolution time (target: < 10ms)
- [ ] T2.2: Verify escalation time remains < 10ms with 3-level hierarchy
- [ ] T2.3: Verify escalation time remains < 10ms with 5-level hierarchy
- [ ] T2.4: Measure escalation overhead in concurrent scenarios
- [ ] T2.5: Verify escalation maintains context correctly
Query Performance
- [ ] T3.1: Measure query execution time within agent scope
- [ ] T3.2: Verify query performance with constitutional constraint checking
- [ ] T3.3: Measure query timeout handling
- [ ] T3.4: Verify query performance under concurrent load
Functional Tests
Agent Lifecycle
- [ ] T4.1: Verify agent spawns with correct identity and scope
- [ ] T4.2: Verify agent executes queries within constitutional scope
- [ ] T4.3: Verify agent terminates correctly after task completion
- [ ] T4.4: Verify automatic cleanup of agent resources
- [ ] T4.5: Verify parent notification on child termination
Constitutional Constraints
- [ ] T5.1: Verify constitutional constraints validated at spawn
- [ ] T5.2: Verify runtime constitutional violation triggers escalation
- [ ] T5.3: Verify agent cannot access facts outside scope
- [ ] T5.4: Verify agent cannot modify facts outside scope
- [ ] T5.5: Verify parent facts visible to child (read-only)
Escalation Mechanism
- [ ] T6.1: Verify escalation triggered when query outside scope
- [ ] T6.2: Verify escalation path follows constitutional hierarchy
- [ ] T6.3: Verify context preserved during escalation
- [ ] T6.4: Verify parent agent receives escalation notification
- [ ] T6.5: Verify escalation resolution returns to original agent
Hierarchy Management
- [ ] T7.1: Verify parent-child relationships established correctly
- [ ] T7.2: Verify multi-level hierarchy (3+ levels) works correctly
- [ ] T7.3: Verify agent depth limits enforced
- [ ] T7.4: Verify sibling agent isolation
- [ ] T7.5: Verify hierarchy cleanup when parent terminates
Integration Tests
System Integration
- [ ] T8.1: Verify agent system integrates with existing orchestrator
- [ ] T8.2: Verify agent metrics collected and exposed
- [ ] T8.3: Verify agent logging works correctly
- [ ] T8.4: Verify agent health checks work correctly
- [ ] T8.5: Verify agent system performance under load (100+ agents)
Error Handling
- [ ] T9.1: Verify graceful handling of spawn failures
- [ ] T9.2: Verify graceful handling of query failures
- [ ] T9.3: Verify graceful handling of escalation failures
- [ ] T9.4: Verify graceful handling of cleanup failures
- [ ] T9.5: Verify error context preserved for debugging
Concurrency
- [ ] T10.1: Verify correct behavior with concurrent spawns
- [ ] T10.2: Verify correct behavior with concurrent queries
- [ ] T10.3: Verify correct behavior with concurrent escalations
- [ ] T10.4: Verify correct behavior with concurrent terminations
- [ ] T10.5: Verify no race conditions in hierarchy management
Regression Tests
Performance Regression
- [ ] T11.1: Agent spawn time < 100ms (baseline maintained)
- [ ] T11.2: Escalation path resolution < 10ms (baseline maintained)
- [ ] T11.3: Memory overhead per agent < 1MB (baseline maintained)
- [ ] T11.4: System responsive with 100+ concurrent agents
Functional Regression
- [ ] T12.1: All constitutional constraints still enforced
- [ ] T12.2: Escalation mechanism still works correctly
- [ ] T12.3: Agent lifecycle still works correctly
- [ ] T12.4: Hierarchy management still works correctly
Acceptance Tests
User Story Validation
- [ ] T13.1: US1 - Lightweight agent spawning verified
- [ ] T13.2: US2 - Escalation paths verified
- [ ] T13.3: US3 - Agent lifespans verified
- [ ] T13.4: US4 - Monitoring metrics verified
- [ ] T13.5: US5 - Constitutional constraints verified
- [ ] T13.6: US6 - Agent escalation verified
End-to-End Scenarios
- [ ] T14.1: Complete task lifecycle (spawn → query → escalate → resolve → terminate)
- [ ] T14.2: Multi-level hierarchy scenario (3+ levels)
- [ ] T14.3: Concurrent agent scenario (10+ agents)
- [ ] T14.4: Long-running session scenario (100+ operations)
- [ ] T14.5: Error recovery scenario (simulated failures)
Test Data Requirements
- [ ] TD1: Benchmark data for spawn time measurements
- [ ] TD2: Benchmark data for escalation time measurements
- [ ] TD3: Sample constitutional constraints for testing
- [ ] TD4: Sample fact sets for scope testing
- [ ] TD5: Test scenarios for each user story
Test Environment
- [ ] TE1: Test environment configured with Prolog backend
- [ ] TE2: Performance monitoring tools configured
- [ ] TE3: Test data loaded and verified
- [ ] TE4: Baseline performance metrics established
- [ ] TE5: CI/CD pipeline configured for automated testing
Junior Volk Tests
Junior Volk Lifecycle
- [ ] T15.1: Verify junior volk spawns with CLI driver and produces output
- [ ] T15.2: Verify junior volk spawns with API driver and produces output
- [ ] T15.3: Verify junior volk spawns with Browser driver and produces output
- [ ] T15.4: Verify junior volk terminates cleanly after task completion
- [ ] T15.5: Verify junior volk resources released on termination (subprocess, session, browser)
- [ ] T15.6: Verify junior volk cannot outlive parent senior volk
- [ ] T15.7: Verify max 5 concurrent junior volks per senior volk enforced
ACP Driver Tests
CLI Driver
- [ ] T16.1: Verify CLI driver starts Aider subprocess correctly
- [ ] T16.2: Verify CLI driver starts OpenCode subprocess correctly
- [ ] T16.3: Verify CLI driver sends task and receives output
- [ ] T16.4: Verify CLI driver handles subprocess crash gracefully
- [ ] T16.5: Verify CLI driver terminates subprocess on cleanup
API Driver
- [ ] T16.6: Verify API driver initializes HTTP session
- [ ] T16.7: Verify API driver sends task via API and receives response
- [ ] T16.8: Verify API driver handles API rate limits gracefully
- [ ] T16.9: Verify API driver handles network timeout gracefully
- [ ] T16.10: Verify API driver tracks quota remaining
Browser Driver
- [ ] T16.11: Verify Browser driver launches Playwright session
- [ ] T16.12: Verify Browser driver navigates to approved ACP domain only
- [ ] T16.13: Verify Browser driver sends message and receives response
- [ ] T16.14: Verify Browser driver handles session timeout and recovers
- [ ] T16.15: Verify Browser driver enforces 30-minute max session lifetime
- [ ] T16.16: Verify Browser driver sandbox — no host filesystem access
- [ ] T16.17: Verify Browser driver cleans up on termination (no orphan processes)
Delegation Pattern Tests
Chat-to-Completion
- [ ] T17.1: Verify senior volk delegates task to one junior and iterates to completion
- [ ] T17.2: Verify senior volk sends followup messages to junior via CXP
- [ ] T17.3: Verify senior volk terminates junior after acceptable output received
- [ ] T17.4: Verify senior volk escalates to human if junior fails after max retries
Kick-Off
- [ ] T17.5: Verify senior volk uses junior for initial research then takes over
- [ ] T17.6: Verify junior output is available to senior after junior termination
- [ ] T17.7: Verify senior volk produces final artifact from kick-off draft
Comparison
- [ ] T17.8: Verify senior volk fans out same task to 3 junior volks in parallel
- [ ] T17.9: Verify all 3 junior outputs collected before synthesis begins
- [ ] T17.10: Verify senior volk evaluates and selects best elements from each output
- [ ] T17.11: Verify senior volk discards rejected elements
- [ ] T17.12: Verify final synthesized output is a new CXP packet (not copy of any single junior)
Synthesis Tests
- [ ] T18.1: Verify synthesis with 2 junior outputs produces merged result
- [ ] T18.2: Verify synthesis with 3 junior outputs produces merged result
- [ ] T18.3: Verify synthesis with 1 failed junior and 2 successful juniors still works
- [ ] T18.4: Verify synthesis with all juniors failed escalates to human
- [ ] T18.5: Verify synthesis decision time < 30s for up to 5 outputs
Browser Automation Tests
- [ ] T19.1: Verify Qwen Chat browser automation end-to-end
- [ ] T19.2: Verify DeepSeek Chat browser automation end-to-end
- [ ] T19.3: Verify Kimi Chat browser automation end-to-end
- [ ] T19.4: Verify browser session recovery after network interruption
- [ ] T19.5: Verify quota tracking for free-tier services
Junior Volk Integration Tests
- [ ] T20.1: Full flow — senior spawns junior, delegates, collects, synthesizes, terminates
- [ ] T20.2: Comparison flow — senior spawns 3 juniors, parallel delegation, synthesis
- [ ] T20.3: Quota exhaustion — junior quota runs out, system falls back to alternative
- [ ] T20.4: Mixed driver flow — CLI junior + Browser junior working on same task set
- [ ] T20.5: Senior volk completes may_complete/2 gate using junior-produced artifacts
Other tracked files
specs/005-multi-agent/compliance-audit.mdspecs/005-multi-agent/prompt.mdspecs/005-multi-agent/review-2026-03-25.md
Full spec text
Verbatim from specs/005-multi-agent/spec.md; headings are pushed down two levels.
SPEC-005: Multi-Agent Coordination
Metadata
- Spec ID: 005-multi-agent
- Title: Multi-Agent Coordination
- Author: Mohamed
- Status: Complete
- Created: 2026-01-15
- Last Updated: 2026-03-07
- Priority: P0
- Components: Agent Runtime, Volk Registry, Escalation Manager
Overview
What
The Multi-Agent Coordination system manages the lifecycle, communication, and escalation of Volks (agents) in Orchestrata. It supports lightweight Prolog agents, hierarchical agent organization, constitutional constraints, and the Junior Volk delegation system for parallel task execution.
Why
- Coordinate multiple specialized agents
- Handle agent escalation and conflict resolution
- Enforce constitutional constraints across agents
- Support parallel delegation to junior volks
- Manage agent lifecycle (spawn, monitor, cleanup)
How
- Volk Registry for agent discovery
- Hierarchical parent-child relationships
- Prolog-driven constitutional validation
- CXP protocol for all inter-agent communication
- Junior Volk drivers (CLI, API, Browser) for delegation
User Scenarios & Stories
US-001: Lightweight Agent Spawning (Critical)
As a developer I want to spawn lightweight Prolog agents So that I can handle specialized tasks without process overhead
Acceptance Criteria:
- Spawn agent in < 100ms
- Agent handles specialized tasks
- Minimal process overhead
- Automatic cleanup when task completes
US-002: Agent Escalation Paths (Critical)
As a system architect I want to define agent escalation paths So that issues requiring broader context are automatically promoted
Acceptance Criteria:
- Define escalation paths
- Automatic promotion for out-of-scope issues
- Escalation path resolution < 10ms
- Parent agent notification
US-003: Agent Lifespan Management (High)
As a developer I want to configure agent lifespans So that agents automatically terminate when tasks complete
Acceptance Criteria:
- Configurable agent lifespans
- Automatic termination on task completion
- Lifecycle management with cleanup
- Resource reclamation
US-004: Agent Monitoring (High)
As a system operator I want to monitor agent spawning metrics So that I can ensure system health
Acceptance Criteria:
- Monitor spawn metrics and performance
- Performance metrics collection
- System health indicators
- Alert on anomalies
US-005: Constitutional Constraints (Critical)
As a developer I want to define constitutional constraints So that agents cannot violate system invariants
Acceptance Criteria:
- Define constraints that prevent invariant violations
- Constraint validation at spawn and runtime
- Agent blocked from violating constraints
- Violation logging and alerts
US-006: Agent Escalation (Critical)
As an agent I want to escalate to higher-level agents So that situations outside my bounds are handled
Acceptance Criteria:
- Escalate when outside constitutional bounds
- Automatic parent agent identification
- Escalation notification
- Hierarchical escalation support
US-007: Junior Volk Spawning (P1)
As a senior volk I want to spawn junior volks with configurable drivers So that I can delegate implementation tasks
Acceptance Criteria:
- Spawn junior volks with CLI, API, or Browser drivers
- Driver initialization times: CLI <500ms, API <200ms, Browser <5s
- Uniform driver interface
- Junior volk configuration via ACP (Agent Communication Protocol)
US-008: Senior-to-Junior Delegation (P1)
As a senior volk I want to delegate tasks to junior volks via CXP So that work is distributed
Acceptance Criteria:
- Delegation via standard CXP packets
- Communication through Postman (same as senior-to-senior)
- Junior output received through CXP
- Task assignment and tracking
US-009: Parallel Delegation and Synthesis (P1)
As a senior volk I want to fan out tasks to multiple junior volks So that I can select the best output
Acceptance Criteria:
- Send same CXP task to N junior volks (max 5)
- Receive all outputs
- Compare quality and select best elements
- Synthesize final deliverable
- Senior synthesis decision < 30s (p95)
US-010: Browser-Based Free-Tier LLM (P1)
As the system I want to drive browser-based LLM chat interfaces So that junior volks can use free tiers at zero cost
Acceptance Criteria:
- Browser driver manages Playwright sessions
- Support Qwen Chat, DeepSeek, Kimi
- Login persistence
- Timeout handling
- Session recovery
- Stable sessions up to 30 minutes
US-011: Code Review Loop (Critical)
As a U:Vz (VizInt) agent I want to participate in an iterative review-fix cycle with U:Qa So that code quality is ensured before completion
Acceptance Criteria:
- U:Qa reviews code against 8-item checklist (Google Python style)
- If issues found, U:Qa sends
fix_issuesCXP packet to U:Vz - U:Vz applies fixes and resubmits for re-review
- Loop continues for maximum MAX_REVIEW_CYCLES (default: 3)
- Escalation to U:Ox when MAX_REVIEW_CYCLES exceeded with remaining issues
- Review cycle tracking via packet metadata
- Auto-fix capability for single issues when AUTO_FIX_ON_SINGLE_ISSUE enabled
Functional Requirements
Agent Management
| Requirement | Target |
|---|---|
| Agent spawn time | < 100ms |
| Escalation path resolution | < 10ms |
| Concurrent lightweight agents | 100+ |
| Memory overhead per agent | < 1MB |
Junior Volk Requirements
| Requirement | Target |
|---|---|
| CLI driver initialization | < 500ms |
| API driver initialization | < 200ms |
| Browser driver initialization | < 5s |
| Parallel delegation max | 5 per senior |
| Senior synthesis decision | < 30s (p95) |
| Browser session stability | 30 minutes |
Code Review Loop Requirements
| Requirement | Target |
|---|---|
| Review cycle time | < 300s (5 minutes) |
| MAX_REVIEW_CYCLES | 3 iterations |
| Review checklist items | 8-item Google Python style |
| Auto-fix single issues | Enabled by default |
| Escalation on max cycles | Automatic to U:Ox |
| Issue categorization | style, type, complexity, docstring, security, test |
Constitutional Constraints
- Validation at agent spawn
- Runtime constraint checking
- Violation blocking
- Audit logging
Technical Design
Architecture
┌─────────────────────────────────────────────────────────┐
│ Multi-Agent Coordination │
├─────────────────────────────────────────────────────────┤
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Volk │ │ Escalation │ │ Constitutional│ │
│ │ Registry │ │ Manager │ │ Validator │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ ┌──────┴──────┐ ┌──────┴──────┐ ┌──────┴──────┐ │
│ │ Agent │ │ Junior │ │ ACP │ │
│ │ Runtime │ │ Volk Mgr │ │ Drivers │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
├─────────────────────────────────────────────────────────┤
│ CLI Driver │ API Driver │ Browser Driver (Playwright) │
└─────────────────────────────────────────────────────────┘
Volk Hierarchy
U:Ox (Orchestrator)
├── U:PS (Postman)
├── U:Vz (VizInt)
│ └── Junior Volks (CLI/API/Browser)
├── U:Qa (QA)
├── U:Ar (Architect)
├── U:Ux (UX)
├── U:PM (PM)
└── U:Cx (Concierge)
ACP Driver Interface
class ACPDriver(Protocol):
async def initialize(self) -> bool: ...
async def send_task(self, task: CXPPacket) -> str: ...
async def get_response(self, task_id: str) -> CXPPacket: ...
async def cleanup(self) -> None: ...
class CLIDriver(ACPDriver): ...
class APIDriver(ACPDriver): ...
class BrowserDriver(ACPDriver): ...
Pipeline Sequence
U:Cx → [U:Ar] → U:Vz → U:Qa → HUMAN
Code Review Loop Architecture
The U:Qa (QA Volk) implements an iterative review-fix cycle to ensure code quality before completion. This section documents the complete flow including CXP subjects, routing, and escalation.
CXP Subjects for Review Flow
| Subject | From | To | Payload |
|---|---|---|---|
review_code |
U:Cx, U:Ox, U:Vz | U:Qa | {file_path, review_type, standards} |
fix_issues |
U:Qa | U:Vz | {review_id, issues[], summary} |
review_complete |
U:Qa | U:Ox | {review_id, final_status, issues_remaining} |
Complete Flow
1. Task Request → U:Cx receives user requirement
2. U:Cx → U:Vz: "build feature X"
3. U:Vz generates code → ready for review
4. U:Vz → U:Qa: "review_code" (subject: review_code)
5. U:Qa analyzes code against standards
- Style (flake8, black)
- Type hints (mypy)
- Complexity
- Documentation
6. IF issues found:
- U:Qa → U:Vz: "fix_issues" with issue list
- U:Vz applies fixes
- Increment cycle count
- IF cycle_count < MAX_REVIEW_CYCLES:
- GOTO step 5 (re-review)
- ELSE:
- U:Qa → U:Ox: escalation (MAX_CYCLES_REACHED)
7. ELSE (no issues):
- U:Qa → U:Ox: "review_complete" (approved: true)
8. U:Ox routes to next step in workflow
MAX_REVIEW_CYCLES Configuration
# In orchestrata-qa configuration
MAX_REVIEW_CYCLES: int = 3 # Maximum iterations
REVIEW_TIMEOUT_SECONDS: int = 300 # 5 minutes per cycle
ESCALATE_ON_MAX_CYCLES: bool = True
AUTO_FIX_ON_SINGLE_ISSUE: bool = True # Auto-fix if only 1 issue
Review Checklist (Google Python Style)
- Naming conventions (snake_case, PascalCase)
- Type hints for all public functions
- Error handling with proper exception types
- Documentation with Google-style docstrings
- Code complexity within bounds (<40 lines per function)
- Security considerations (no hardcoded secrets)
- Test coverage targets (≥90%)
- Code quality metrics (no code smells, duplications)
Kernel Routing for Review Cycles
The Kernel must support recursive routing:
- Detect
fix_issuessubject - Extract cycle count from packet metadata
- Route back to U:Qa for re-review
- Track cycle count - increment on each fix_issues round-trip
- Escalate to U:Ox when cycle_count >= MAX_REVIEW_CYCLES
# In orchestrata-kernel routing logic
def route_packet(packet: CXPacket) -> str:
if packet.subject == "fix_issues":
cycle_count = packet.metadata.get("review_cycle", 0) + 1
packet.metadata["review_cycle"] = cycle_count
if cycle_count >= MAX_REVIEW_CYCLES:
return "U:Ox" # Escalate
return "U:Qa" # Re-review
# ... other routing
Issue Types
| Type | Source | Example | Severity |
|---|---|---|---|
| style | flake8 | E501 line too long | warning |
| type | mypy | Missing return type | error |
| complexity | AST | Function > 40 lines | warning |
| docstring | Google style | Missing docstring | info |
| security | bandit | Hardcoded password | error |
| test | Coverage | < 90% coverage | warning |
Escalation to U:Ox
When MAX_REVIEW_CYCLES is reached with remaining issues:
# U:Qa sends escalation to U:Ox
escalation = CXPacket(
subject="escalation",
from_agent="U:Qa",
to_agent="U:Ox",
payload={
"reason": "MAX_REVIEW_CYCLES_EXCEEDED",
"review_id": "rev_123",
"file_path": "packages/.../kernel.py",
"issues_remaining": 3,
"issue_summary": {
"errors": 1,
"warnings": 2,
"infos": 0
},
"cycles_completed": 3
}
)
U:Vz Fix Handler
U:Vz must handle fix_issues packets:
# In orchestrata-vizint volk handler
async def handle_fix_issues(packet: CXPacket) -> Dict[str, Any]:
issues = packet.payload["issues"]
file_path = packet.payload["file_path"]
fixed = []
failed = []
for issue in issues:
try:
apply_fix(file_path, issue)
fixed.append(issue)
except Exception as e:
failed.append({**issue, "error": str(e)})
return {
"fixed": len(fixed),
"failed": len(failed),
"issues": failed # Send back for re-review
}
Code Review Loop Architecture
The U:Qa (QA Volk) implements an iterative review-fix cycle to ensure code quality before completion. This section documents the complete flow including CXP subjects, routing, and escalation.
CXP Subjects for Review Flow
| Subject | From | To | Payload |
|---|---|---|---|
review_code |
U:Cx, U:Ox, U:Vz | U:Qa | {file_path, review_type, standards} |
fix_issues |
U:Qa | U:Vz | {review_id, issues[], summary} |
review_complete |
U:Qa | U:Ox | {review_id, final_status, issues_remaining} |
Complete Flow
1. Task Request → U:Cx receives user requirement
2. U:Cx → U:Vz: "build feature X"
3. U:Vz generates code → ready for review
4. U:Vz → U:Qa: "review_code" (subject: review_code)
5. U:Qa analyzes code against standards
- Style (flake8, black)
- Type hints (mypy)
- Complexity
- Documentation
6. IF issues found:
- U:Qa → U:Vz: "fix_issues" with issue list
- U:Vz applies fixes
- Increment cycle count
- IF cycle_count < MAX_REVIEW_CYCLES:
- GOTO step 5 (re-review)
- ELSE:
- U:Qa → U:Ox: escalation (MAX_CYCLES_REACHED)
7. ELSE (no issues):
- U:Qa → U:Ox: "review_complete" (approved: true)
8. U:Ox routes to next step in workflow
MAX_REVIEW_CYCLES Configuration
# In orchestrata-qa configuration
MAX_REVIEW_CYCLES: int = 3 # Maximum iterations
REVIEW_TIMEOUT_SECONDS: int = 300 # 5 minutes per cycle
ESCALATE_ON_MAX_CYCLES: bool = True
AUTO_FIX_ON_SINGLE_ISSUE: bool = True # Auto-fix if only 1 issue
Review Checklist (Google Python Style)
- Naming conventions (snake_case, PascalCase)
- Type hints for all public functions
- Error handling with proper exception types
- Documentation with Google-style docstrings
- Code complexity within bounds (<40 lines per function)
- Security considerations (no hardcoded secrets)
- Test coverage targets (≥90%)
- Code quality metrics (no code smells, duplications)
Kernel Routing for Review Cycles
The Kernel must support recursive routing:
- Detect
fix_issuessubject - Extract cycle count from packet metadata
- Route back to U:Qa for re-review
- Track cycle count - increment on each fix_issues round-trip
- Escalate to U:Ox when cycle_count >= MAX_REVIEW_CYCLES
# In orchestrata-kernel routing logic
def route_packet(packet: CXPacket) -> str:
if packet.subject == "fix_issues":
cycle_count = packet.metadata.get("review_cycle", 0) + 1
packet.metadata["review_cycle"] = cycle_count
if cycle_count >= MAX_REVIEW_CYCLES:
return "U:Ox" # Escalate
return "U:Qa" # Re-review
# ... other routing
Issue Types
| Type | Source | Example | Severity |
|---|---|---|---|
| style | flake8 | E501 line too long | warning |
| type | mypy | Missing return type | error |
| complexity | AST | Function > 40 lines | warning |
| docstring | Google style | Missing docstring | info |
| security | bandit | Hardcoded password | error |
| test | Coverage | < 90% coverage | warning |
Escalation to U:Ox
When MAX_REVIEW_CYCLES is reached with remaining issues:
# U:Qa sends escalation to U:Ox
escalation = CXPacket(
subject="escalation",
from_agent="U:Qa",
to_agent="U:Ox",
payload={
"reason": "MAX_REVIEW_CYCLES_EXCEEDED",
"review_id": "rev_123",
"file_path": "packages/.../kernel.py",
"issues_remaining": 3,
"issue_summary": {
"errors": 1,
"warnings": 2,
"infos": 0
},
"cycles_completed": 3
}
)
U:Vz Fix Handler
U:Vz must handle fix_issues packets:
# In orchestrata-vizint volk handler
async def handle_fix_issues(packet: CXPacket) -> Dict[str, Any]:
issues = packet.payload["issues"]
file_path = packet.payload["file_path"]
fixed = []
failed = []
for issue in issues:
try:
apply_fix(file_path, issue)
fixed.append(issue)
except Exception as e:
failed.append({**issue, "error": str(e)})
return {
"fixed": len(fixed),
"failed": len(failed),
"issues": failed # Send back for re-review
}
Integration Points
| Component | Integration |
|---|---|
| orchestrata-kernel | Volk registry and routing |
| orchestrata-postman | CXP packet delivery |
| orchestrata-governance | Constitutional validation |
| orchestrata-cxp | Inter-agent communication |
Revision History
| Date | Version | Author | Changes |
|---|---|---|---|
| 2026-03-07 | 1.0 | Mohamed | Initial specification |
| 2026-03-07 | 1.1 | Mohamed | Added U:Qa code review loop functionality: review-fix cycle flow, fix_issues CXP subject, MAX_REVIEW_CYCLES configuration, recursive routing |
| 2026-03-07 | 1.1 | Mohamed | Added U:Qa code review loop functionality: review-fix cycle flow, fix_issues CXP subject, MAX_REVIEW_CYCLES configuration, recursive routing |
Success Criteria
- Spawn: <100ms agent spawn time verified
- Escalation: <10ms path resolution verified
- Constraints: Constitutional validation at spawn and runtime
- Delegation: Junior volk produces output within 60s (CLI) / 120s (Browser)
- Parallel: Comparison delegation produces synthesized output
- Quota: Automatic fallback when free-tier quotas exhausted
Dependencies
- orchestrata-core (enums, interfaces)
- orchestrata-kernel (Volk registry, recursive routing)
- orchestrata-postman (CXP routing)
- orchestrata-governance (Prolog constraints)
- orchestrata-cxp (packet format, fix_issues subject)
- orchestrata-qa (code review, MAX_REVIEW_CYCLES config)
- orchestrata-vizint (fix_issues handler)
- Playwright (browser driver)
Gap Findings
FR-MA-SPIKE-1: U:Qa in Pipeline
- Pipeline sequence: U:Cx → [U:Ar] → U:Vz → U:Qa → HUMAN
- U:Vz sets
escalation_target = "U:Qa" - U:Qa receives files_created and task_summary
- U:Qa generates pytest test files and reports results
FR-MA-SPIKE-2: Deployment Context Awareness
- U:Cx and U:Vz system prompts include deployment platform context
- Default: Incus containers (Linux)
- Generated artifacts match specified deployment target
FR-MA-SPIKE-3: Concierge Fast Path
- U:Cx detects sufficiently specified intents
- Skip clarification if language/framework, features, and scope present
- Go directly to REQUIREMENTS_COMPLETE output