Knowledge Wiki Orchestrata Ultimate

Orchestrata Ultimate / Specs / 019-intent-impact-analyzer

019-intent-impact-analyzer: Intent Impact Analyzer

The Intent Impact Analyzer is an intelligent analysis system that processes submitted intents to determine the scope of work, affected code areas, and creates parallel execution plans for multiple agents.

At a glance

Item Value
Spec directory specs/019-intent-impact-analyzer/
Status line in the spec Implemented
Requirement IDs defined 47
With a @traces citation in src/ 40 (85.1%)
With a @traces citation in tests/ 0 (0.0%)
Tracked files 6
RTM page 019-intent-impact-analyzer

Reading this spec

The Intent Impact Analyzer processes a submitted intent to work out the scope of the work, the code areas it affects, and a parallel execution plan for several agents. Before it, intents went straight to agents without any understanding of their full scope, of cascading changes through dependencies, or of how to split work across agents.

The scope is six user stories: intent classification, dependency impact analysis, test impact analysis, interface verification and parallel execution planning (critical), and refactoring plan generation (high). Ten functional groups run from classification, entity extraction and target identification through dependency analysis, impact severity ranking, test discovery, interface contract extraction, breaking change detection, backward compatibility verification and parallel task identification. Requirement IDs use the REQ-019- prefix.

The approach in the overview is to classify the intent text (the terminology table names the classes NEW_REQUIREMENT, NEW_SPEC, FEATURE, BUGFIX and REFACTOR), query the knowledge graph for affected packages, modules and files, follow dependency relationships to find all affected code, identify test files that need updates, create a parallel plan with agent assignments, and verify interface compatibility before execution.

In the code, the work is done by the analyst Volk: orchestrata_ultimate/agents/volks/analyst/agent and orchestrata_ultimate/agents/volks/analyst/parsers, with single-requirement citations in orchestrata_ultimate/orchestrator/impact/planner, orchestrata_ultimate/orchestrator/impact/dependency_analyzer and orchestrata_ultimate/orchestrator/impact/classifier. The per-requirement view is on the RTM page.

The spec's status line reads Implemented. It tracks 6 files and defines 47 requirement IDs. The full text follows below.

Measured coverage is 40 of 47 requirements cited by at least one @traces in src/ (85.1%), spread over 6 modules; the module citing the most of them is orchestrata_ultimate/agents/volks/analyst/agent (23 requirements). In tests/, 0 requirements are cited by a test-side @traces.

Coverage gaps: 7 requirements have no citing symbol. The largest untraced groups are REQ-019-* (7). Untraced means no @traces names the ID; the behaviour may still exist in code that was never annotated. Test-side citations are thinner than code-side ones (0 against 40).

Requirements

Checkbox is the state recorded in the requirements checklist; Traced by is the count of src/ symbols whose @traces names the ID.

ID Done Requirement Traced by
REQ-019-001 no AnalystVolk must define alias U:BA and role "Business Analyst" 3
REQ-019-002 no AnalystVolk must extend BaseVolk and comply with all BaseVolk interface contracts 2
REQ-019-003 no process() must route messages with subject "implement" to _handle_analysis() 1
REQ-019-004 no process() must route messages with subject "analyze_requirements" to _handle_analysis() 1
REQ-019-005 no _handle_analysis() must extract raw requirements from the incoming CXP message payload 2
REQ-019-006 no _handle_analysis() must assemble a full context string prior to invoking the LLM, including agent alias, role, and session context 1
REQ-019-007 no _handle_analysis() must include conversation history in the LLM context when history is present in the session 1
REQ-019-008 no _handle_analysis() must call the configured LLM provider with ANALYST_SYSTEM_PROMPT as the system prompt 2
REQ-019-009 no _handle_analysis() must pass the assembled context and raw requirements as the user message to the LLM 2
REQ-019-010 no _handle_analysis() must invoke all six parser functions on the raw LLM output string 2
REQ-019-011 no _handle_analysis() must store parsed user stories and requirements to the Knowledge Graph via KGDesignStore 2
REQ-019-012 no _handle_analysis() must escalate to U:PM after successful analysis regardless of KG storage outcome 1
REQ-019-013 no parse_project_type() must accept the raw LLM output string as its sole argument 1
REQ-019-014 no parse_project_type() must use a regex to locate and return the value of the PROJECT_TYPE field 1
REQ-019-015 no parse_project_type() must return an empty string when the PROJECT_TYPE field is absent from the output 1
REQ-019-016 no parse_technology() must extract all key-value pairs within the TECHNOLOGY block 1
REQ-019-017 no parse_technology() must return a dict mapping each technology key (language, framework, packaging, testing, deployment) to its value 1
REQ-019-018 no parse_technology() must return an empty dict when the TECHNOLOGY block is absent 1
REQ-019-019 no parse_spec_metadata() must extract all key-value pairs within the SPEC_METADATA block 1
REQ-019-020 no parse_spec_metadata() must return a dict of spec metadata fields 1
REQ-019-021 no parse_spec_metadata() must return an empty dict when the SPEC_METADATA block is absent 1
REQ-019-022 no parse_user_stories() must extract every US-NNN entry from the USER_STORIES block 3
REQ-019-023 no parse_user_stories() must capture the acceptance criteria associated with each US-NNN entry 3
REQ-019-024 no parse_user_stories() must return a list of dicts, each containing "id", "description", and "acceptance_criteria" keys 2
REQ-019-025 no parse_requirements() must extract every REQ-NNN-USn-NNN entry from the REQUIREMENTS block 3
REQ-019-026 no parse_requirements() must return a list of dicts, each containing "id" and "description" keys 2
REQ-019-027 no parse_nfrs() must extract every NFR-NNN entry from the NON_FUNCTIONAL block 2
REQ-019-028 no parse_nfrs() must capture the category associated with each NFR-NNN entry 2
REQ-019-029 no parse_nfrs() must return a list of dicts, each containing "id", "category", and "description" keys 2
REQ-019-030 no KGDesignStore must be initialised lazily — instantiated only on first use, not at volk startup 1
REQ-019-031 no _handle_analysis() must store each parsed user story as a node in the KG via KGDesignStore 2
REQ-019-032 no _handle_analysis() must store each parsed requirement as a node in the KG, linked to its parent user story 2
REQ-019-033 no KG storage failures must be caught, logged, and must not prevent escalation to U:PM 1
REQ-019-034 no _handle_analysis() must store structured requirements under the active session identifier 1
REQ-019-035 no _handle_analysis() must send the escalation CXP message to U:PM (not U:Ox or any other volk) 1
REQ-019-036 no The escalation CXP subject must be "scope_review" 1
REQ-019-037 no The escalation payload must include the original raw requirements text under the key "requirements" 1
REQ-019-038 no The escalation payload must include a "task_summary" field summarising the analyst output 1
REQ-019-039 no The escalation payload must include a "prolog_facts" field containing Prolog-compatible fact assertions derived from the parsed output 1
REQ-019-040 no The escalation payload must include "project_type", "technology", "spec_metadata", "user_stories", "structured_requirements", "nfrs", and "analyst_output" fields 1
REQ-019-041 no ANALYST_SYSTEM_PROMPT must instruct the LLM to classify the project type and emit it under the PROJECT_TYPE section header none
REQ-019-042 no ANALYST_SYSTEM_PROMPT must instruct the LLM to determine and emit technology decisions (language, framework, packaging, testing, deployment) under the TECHNOLOGY section header none
REQ-019-043 no ANALYST_SYSTEM_PROMPT must instruct the LLM to emit spec metadata under the SPEC_METADATA section header none
REQ-019-044 no ANALYST_SYSTEM_PROMPT must require user story IDs in the format US-NNN (zero-padded three-digit integer) none
REQ-019-045 no ANALYST_SYSTEM_PROMPT must require each user story to include at least one acceptance criterion none
REQ-019-046 no ANALYST_SYSTEM_PROMPT must require functional requirement IDs in the format REQ-NNN-USn-NNN none
REQ-019-047 no ANALYST_SYSTEM_PROMPT must require NFR IDs in the format NFR-NNN and mandate a category label on every NFR none

Code that traces this spec

Module Symbols Requirements
orchestrata_ultimate/agents/volks/analyst/agent 9 23
orchestrata_ultimate/agents/volks/analyst/parsers 11 17
orchestrata_ultimate/orchestrator/impact/analyzer 1 1
orchestrata_ultimate/orchestrator/impact/classifier 1 1
orchestrata_ultimate/orchestrator/impact/dependency_analyzer 1 1
orchestrata_ultimate/orchestrator/impact/planner 1 1

Checklists

checklists/constitutional.md

SPEC-019 Constitutional Checklist — Intent Impact Analyzer (Business Analyst Volk)

checklists/requirements.md

The requirement list above comes from this file; 0 of 47 checkbox items are ticked.

checklists/testing.md

SPEC-019 Testing Checklist — Intent Impact Analyzer (Business Analyst Volk)

Volk Identity & Routing
Analysis Pipeline
Parsers — parse_project_type
Parsers — parse_technology
Parsers — parse_spec_metadata
Parsers — parse_user_stories
Parsers — parse_requirements
Parsers — parse_nfrs
Knowledge Graph Storage
Escalation
ANALYST_SYSTEM_PROMPT

Other tracked files

Full spec text

Verbatim from specs/019-intent-impact-analyzer/spec.md; headings are pushed down two levels.

SPEC-019: Intent Impact Analyzer

Metadata

Overview

What

The Intent Impact Analyzer is an intelligent analysis system that processes submitted intents to determine the scope of work, affected code areas, and creates parallel execution plans for multiple agents.

Why
How

Terminology

Term Definition
Intent Classification Categorization of intent as NEW_REQUIREMENT, NEW_SPEC, FEATURE, BUGFIX, REFACTOR
Impact Scope The set of packages/modules/files affected by the intent
Dependency Graph Neo4j graph showing imports, calls, inherits relationships
Parallel Plan Execution plan with independent tasks that can run concurrently
Agent Task A unit of work assigned to a specific agent type
Interface Contract The public API of a module (functions, classes)
Breaking Change An interface change that breaks existing callers

User Scenarios

US-001: Intent Classification (Critical)

As a developer I want the system to classify my intent So that I know if it's a new requirement, spec, feature, or bugfix

Acceptance Criteria:

US-002: Dependency Impact Analysis (Critical)

As a developer I want to know all files affected by my intent So that I understand the full scope of changes

Acceptance Criteria:

US-003: Test Impact Analysis (Critical)

As a developer I want to know which tests will be affected So that I can plan test updates

Acceptance Criteria:

US-004: Interface Verification (Critical)

As a developer I want the system to verify interface compatibility So that I avoid breaking changes

Acceptance Criteria:

US-005: Parallel Execution Planning (Critical)

As a developer I want a plan that multiple agents can execute in parallel So that I can speed up development

Acceptance Criteria:

US-006: Refactoring Plan Generation (High)

As a developer I want a comprehensive refactoring plan So that I can execute changes systematically

Acceptance Criteria:

Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                    INTENT IMPACT ANALYZER                              │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                          │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐            │
│  │   Intent     │    │   Impact     │    │  Dependency  │            │
│  │ Classifier   │───▶│  Analyzer    │───▶│    Graph      │            │
│  └──────────────┘    └──────────────┘    └──────────────┘            │
│         │                   │                   │                      │
│         ▼                   ▼                   ▼                      │
│  ┌────────────────────────────────────────────────────────────────┐   │
│  │                    Impact Report                                │   │
│  │  • Classification      • Affected files                        │   │
│  │  • Confidence          • Dependency chain                    │   │
│  │  • Entities            • Test coverage                        │   │
│  └────────────────────────────────────────────────────────────────┘   │
│                               │                                           │
│                               ▼                                           │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐            │
│  │  Interface   │    │    Test      │    │   Parallel   │            │
│  │   Verifier   │    │   Analyzer   │    │    Planner    │            │
│  └──────────────┘    └──────────────┘    └──────────────┘            │
│         │                   │                   │                      │
│         ▼                   ▼                   ▼                      │
│  ┌────────────────────────────────────────────────────────────────┐   │
│  │                    Execution Plan                               │   │
│  │  • Agent assignments     • Task dependencies                  │   │
│  │  • Parallel branches     • Verification checkpoints           │   │
│  └────────────────────────────────────────────────────────────────┘   │
│                                                                          │
└─────────────────────────────────────────────────────────────────────────┘

Data Models

IntentClassification
@dataclass
class IntentClassification:
    type: IntentType  # NEW_REQUIREMENT, NEW_SPEC, FEATURE, BUGFIX, REFACTOR
    confidence: float
    entities: list[str]  # File names, function names, class names
    target_package: Optional[str]
    target_module: Optional[str]
ImpactScope
@dataclass
class ImpactScope:
    target_package: str
    target_module: str
    affected_packages: list[str]
    affected_modules: list[str]
    affected_files: list[str]
    upstream_dependencies: list[str]  # Files that import this
    downstream_dependencies: list[str]  # Files this imports
    test_files: list[str]
    severity: ImpactSeverity  # LOW, MEDIUM, HIGH, CRITICAL
InterfaceContract
@dataclass
class InterfaceContract:
    module_path: str
    classes: list[ClassInterface]
    functions: list[FunctionInterface]
    
@dataclass
class ClassInterface:
    name: str
    public_methods: list[str]
    attributes: list[str]
    
@dataclass
class FunctionInterface:
    name: str
    parameters: list[str]
    return_type: Optional[str]
BreakingChange
@dataclass
class BreakingChange:
    type: BreakingType  # REMOVED, RENAMED, SIGNATURE_CHANGE
    entity: str
    old_contract: Optional[InterfaceContract]
    severity: ImpactSeverity
ParallelPlan
@dataclass
class ParallelPlan:
    total_tasks: int
    parallel_branches: list[TaskBranch]
    estimated_speedup: float
    critical_path: list[str]
    
@dataclass  
class TaskBranch:
    agent_type: str  # U:Cd, U:Qa, U:Ar
    tasks: list[AgentTask]
    dependencies: list[tuple[str, str]]  # (task_id, depends_on_task_id)
    
@dataclass
class AgentTask:
    task_id: str
    description: str
    target_files: list[str]
    agent_type: str
    priority: int

Integration Points

With Intent Broker
With Knowledge Graph
With Agents

Query Patterns

Find Affected Files
MATCH (target:Module {name: $module_name})
OPTIONAL MATCH (target)<-[:IMPORTS]-(upstream:File)
OPTIONAL MATCH (target)-[:IMPORTS]->(downstream:File)
RETURN upstream, downstream
Find Tests
MATCH (test:File)-[:DEFINES]->(tc:Class)
WHERE tc.name CONTAINS 'Test' OR tc.name ENDS WITH 'Test'
OPTIONAL MATCH (test)-[:IMPORTS]->(target:File {path: $target_path})
RETURN test
Find Interface
MATCH (m:Module {name: $module_name})
MATCH (m)-[:HAS_FILE]->(f:File)-[:DEFINES]->(c:Class)
RETURN c.name as class, collect(DISTINCT c.methods) as methods

Functional Requirements

FR-001: Intent Classification

The system MUST classify intents into one of: NEW_REQUIREMENT, NEW_SPEC, FEATURE, BUGFIX, REFACTOR with a confidence score >= 0.85.

FR-002: Entity Extraction

The system MUST extract file names, function names, and class names from intent text with >= 90% accuracy.

FR-003: Target Identification

The system MUST identify target package and module from intent context with >= 90% accuracy.

FR-004: Dependency Analysis

The system MUST query the dependency graph to find all upstream and downstream dependencies within 3 hops.

FR-005: Impact Severity Ranking

The system MUST rank affected files by severity: CRITICAL (direct), HIGH (1 hop), MEDIUM (2 hops), LOW (3 hops).

FR-006: Test Discovery

The system MUST identify all test files affected by the impact scope, including unit tests and integration tests.

FR-007: Interface Contract Extraction

The system MUST extract current interface contracts including public methods, parameters, and return types.

FR-008: Breaking Change Detection

The system MUST detect breaking changes including: REMOVED, RENAMED, SIGNATURE_CHANGE with >= 95% accuracy.

FR-009: Backward Compatibility Verification

The system MUST verify backward compatibility and suggest additions vs modifications.

FR-010: Parallel Task Identification

The system MUST identify independent tasks that can execute in parallel with zero dependencies between them.

FR-011: Agent Assignment

The system MUST assign tasks to agent types: U:Cd (Code), U:Qa (Quality), U:Ar (Architecture) based on task requirements.

FR-012: Execution Order Generation

The system MUST generate execution order with parallel branches and define task dependencies.

FR-013: Parallelization Efficiency

The system MUST calculate estimated speedup factor >= 2.0 for typical multi-agent scenarios.

FR-014: Impact Report Generation

The system MUST generate a complete ImpactReport within 5 seconds for standard intents.

FR-015: Circular Dependency Detection

The system MUST detect and highlight circular dependencies in the impact scope.

Error Conditions

ID Condition Handling
ERR-001 Intent text is blank Return error with message "Intent text is required"
ERR-002 Intent text exceeds 10000 characters Truncate and warn user, proceed with analysis
ERR-003 Target module not found in knowledge graph Return partial impact analysis with warning "Module not indexed"
ERR-004 Knowledge graph connection unavailable Return error with message "Knowledge graph unavailable"
ERR-005 Classification confidence < 0.5 Flag as LOW_CONFIDENCE, suggest manual review
ERR-006 Circular dependency detected Include in report, mark CRITICAL severity, suggest refactoring
ERR-007 Breaking change detected Block execution, require explicit override from user
ERR-008 No parallelizable tasks found Generate sequential plan, explain why parallelization not possible
ERR-009 Agent pool unavailable Queue tasks, notify user of delay
ERR-010 Task execution timeout Log timeout, attempt retry with exponential backoff (max 3 retries)
ERR-011 Interface verification fails Generate detailed change report, suggest migration path
ERR-012 Test files not found Warn user, proceed with file changes only
ERR-013 Insufficient permissions for target files Report authorization error, do not proceed
ERR-014 Neo4j query timeout Retry query once, then return partial results with warning
ERR-015 Invalid intent type classification Default to FEATURE, flag for manual review

API Contracts

ImpactAnalyzer
class ImpactAnalyzer:
    def analyze_intent(intent: str, context: dict) -> ImpactReport:
        """
        Analyze an intent and generate impact report.
        
        Args:
            intent: Natural language intent description
            context: Additional context (optional)
            
        Returns:
            ImpactReport with classification, scope, and plan
        """
        
    def classify_intent(intent: str) -> IntentClassification:
        """
        Classify intent type and extract entities.
        
        Returns:
            IntentClassification with type, confidence, entities
        """
        
    def analyze_dependencies(target_module: str) -> ImpactScope:
        """
        Analyze dependency impact for target module.
        
        Returns:
            ImpactScope with affected packages, modules, files
        """
ParallelPlanner
class ParallelPlanner:
    def create_plan(impact_scope: ImpactScope) -> ParallelPlan:
        """
        Create parallel execution plan from impact scope.
        
        Returns:
            ParallelPlan with tasks, branches, dependencies
        """
        
    def identify_independent_tasks(impact_scope: ImpactScope) -> list[AgentTask]:
        """
        Identify tasks that can run in parallel.
        
        Returns:
            List of independent AgentTask objects
        """
        
    def assign_agents(tasks: list[AgentTask]) -> list[TaskBranch]:
        """
        Assign tasks to appropriate agent types.
        
        Returns:
            List of TaskBranch objects grouped by agent type
        """
InterfaceVerifier
class InterfaceVerifier:
    def extract_contract(module_path: str) -> InterfaceContract:
        """
        Extract interface contract from module.
        
        Returns:
            InterfaceContract with classes and functions
        """
        
    def predict_changes(intent: str, current: InterfaceContract) -> list[BreakingChange]:
        """
        Predict interface changes from intent.
        
        Returns:
            List of BreakingChange objects
        """
        
    def verify_compatibility(changes: list[BreakingChange]) -> CompatibilityResult:
        """
        Verify backward compatibility of proposed changes.
        
        Returns:
            CompatibilityResult with is_compatible flag and suggestions
        """

Non-Functional Requirements

Performance
Scalability
Reliability

Technical Design

Processing Pipeline
  1. Intent Parsing: Parse intent text, extract entities
  2. Classification: Classify intent type with confidence score
  3. Target Resolution: Identify target package/module
  4. Dependency Query: Query knowledge graph for affected code
  5. Impact Calculation: Calculate severity and scope
  6. Test Discovery: Find affected test files
  7. Interface Analysis: Extract and analyze contracts
  8. Breaking Change Detection: Identify incompatible changes
  9. Parallel Planning: Generate execution plan
  10. Report Generation: Compile final ImpactReport
Data Flow
Intent Text → Classifier → ImpactAnalyzer → Knowledge Graph
                              ↓
                        ImpactScope ← Dependency Graph
                              ↓
                    InterfaceVerifier → TestAnalyzer
                              ↓
                        ParallelPlanner
                              ↓
                        ImpactReport

Examples

Example 1: Feature Implementation

Intent: "Add authentication middleware to the user service"

Classification: FEATURE (confidence: 0.92)

Impact Scope:

Parallel Plan:

Example 2: Breaking Change

Intent: "Remove deprecated getUser method from UserService"

Breaking Change: REMOVED (severity: CRITICAL)

Verification: FAILS backward compatibility

Action: Block execution, suggest deprecation path

Security

Authentication
Authorization
Input Validation
Audit Logging

Governance

Change Management
Quality Gates

Visual Architecture

Class Diagram

Diagram source shown as text: Mermaid could not parse it (Error: Parse error on line 11:).

classDiagram-direction LR
    class GovernanceEngine {
        +__init__()
        #_load_default_policies(): dict[str, Any]
        +async check(action: str, context: dict[str, Any]): bool
        #async _check_edit_file(context: dict[str, Any]): bool
        #async _check_create_project(context: dict[str, Any]): bool
        #async _check_execute_command(context: dict[str, Any]): bool
    }
Unit Tests
Integration Tests
Coverage Targets

Success Criteria

Core Functionality
Performance
Quality

Observability

Logging Requirements
  1. Component Logging
    • Log all major operations with appropriate log levels
    • Include correlation IDs for request tracing
    • Log performance metrics for critical operations
    • Log errors with full context for debugging
Metrics Requirements
  1. Performance Metrics
    • Operation duration histograms
    • Request/error rate counters
    • Resource utilization gauges
    • Business metrics relevant to domain
Tracing Requirements
  1. Distributed Tracing
    • Trace propagation across component boundaries
    • Span attributes for domain-specific context
    • Trace sampling based on operational needs
Alerting Requirements
  1. Critical Alerts
    • Error rate spikes
    • Performance degradation
    • Resource exhaustion
    • Business metric anomalies
Dashboard Requirements
  1. Operational Dashboard
    • Real-time system health
    • Performance trends
    • Error analysis
    • Resource monitoring

Revision History

Version Date Changes Author
1.0.0 2026-03-29 Initial spec Mohamed
1.0.1 2026-04-09 Added missing sections per SPEC_STANDARD Mohamed