Orchestrata Ultimate / RTM / 035-verification

RTM: 035-verification

Requirement to symbol to test rows for 035-verification: Verification Agent (U:Vfy).

At a glance

Item Value
Requirement IDs 230
Traced from src/ 188 (81.7%)
Untraced 42
Cited from tests/ 8 (3.5%)
Traced in src/ but not cited from tests/ 182

Reading this matrix

This matrix lists each of the 230 requirement IDs of spec 035-verification with the src/ symbols that cite it and the tests that cite it. 188 of them (81.7%) have at least one citing symbol and 42 have none.

8 requirements are also cited from tests/, and 182 are traced in code but never named by a test. A test citation is a @traces on a test function; it shows intent to cover, not that the test runs or reaches the symbol.

Rows showing none in the symbols column are the gaps to close first. The densest untraced groups are REQ-035-US-011 (15), REQ-035-US-010 (14), REQ-035-US-003 (4).

Requirement to symbols to tests

Requirement Symbols in src/ Tests citing it
REQ-035-US-001-001 Parse simple requirement statements ("create a file named test.py") orchestrata_ultimate/verification/config/verification_config ModelConfig L15
orchestrata_ultimate/verification/config/verification_config VerificationConfig L27
orchestrata_ultimate/verification/constraints/definitions filter_by_type L36
+11 more
none
REQ-035-US-001-002 Parse compound requirements ("read config.yaml and update database") orchestrata_ultimate/verification/requirements/requirements_parser parse L71 none
REQ-035-US-001-003 Extract action verbs (create, read, update, delete, execute) orchestrata_ultimate/verification/requirements/requirements_parser extract_functional L160 none
REQ-035-US-001-004 Extract target nouns (file, database, api, system) orchestrata_ultimate/verification/requirements/requirements_parser extract_functional L160 none
REQ-035-US-001-005 Identify constraints (permissions, limits, conditions) orchestrata_ultimate/verification/requirements/requirements_parser RequirementsParser L59
orchestrata_ultimate/verification/requirements/requirements_parser extract_quality L119
none
REQ-035-US-001-006 Handle ambiguous requirements with clarification questions orchestrata_ultimate/verification/requirements/requirements_parser parse L71 none
REQ-035-US-001-007 Support multi-language input (English primary) none none
REQ-035-US-001-008 Generate valid Prolog predicates from requirements orchestrata_ultimate/verification/requirements/prolog_generator PrologGenerator L13
orchestrata_ultimate/verification/requirements/prolog_generator generate_predicate L48
none
REQ-035-US-001-009 Validate generated Prolog syntax orchestrata_ultimate/verification/requirements/prolog_generator PrologGenerator L13
orchestrata_ultimate/verification/requirements/prolog_generator validate_syntax L73
none
REQ-035-US-001-010 Store constraints for session persistence orchestrata_ultimate/orchestrator/verification_gate _persist L741
orchestrata_ultimate/orchestrator/verification_gate open_session L518
orchestrata_ultimate/verification/integrations/redis_manager VerificationRedisManager L20
+4 more
none
REQ-035-US-001-011 Create RequirementsParser class orchestrata_ultimate/verification/requirements/requirements_parser RequirementsParser L59 none
REQ-035-US-001-012 Implement NLP parsing for requirements orchestrata_ultimate/verification/requirements/requirements_parser parse L71 none
REQ-035-US-001-013 Create action verb dictionary (CREATE, READ, UPDATE, DELETE, EXECUTE, VALIDATE) orchestrata_ultimate/verification/requirements/requirements_parser extract_functional L160 none
REQ-035-US-001-014 Create target noun dictionary (FILE, DATABASE, API, SYSTEM, SERVICE) orchestrata_ultimate/verification/requirements/requirements_parser extract_functional L160 none
REQ-035-US-001-015 Implement constraint extraction using pattern matching orchestrata_ultimate/verification/requirements/requirements_parser RequirementsParser L59
orchestrata_ultimate/verification/requirements/requirements_parser extract_performance L93
orchestrata_ultimate/verification/requirements/requirements_parser extract_quality L119
+1 more
none
REQ-035-US-001-016 Create Prolog predicate generator orchestrata_ultimate/verification/requirements/prolog_generator PrologGenerator L13 none
REQ-035-US-001-017 Implement Prolog syntax validator orchestrata_ultimate/verification/requirements/prolog_generator PrologGenerator L13
orchestrata_ultimate/verification/requirements/prolog_generator validate_syntax L73
none
REQ-035-US-001-018 Create constraint storage backend (Redis) orchestrata_ultimate/verification/integrations/redis_manager VerificationRedisManager L20
orchestrata_ultimate/verification/integrations/redis_manager available L113
none
REQ-035-US-001-019 Add requirement session management none none
REQ-035-US-001-020 Implement requirement versioning none none
REQ-035-US-001-021 Unit test: Parse "create file test.py" → constraint(file_exists, ['test.py']) orchestrata_ultimate/verification/requirements/requirements_parser parse L71 none
REQ-035-US-001-022 Unit test: Parse "API must respond in 200ms" → constraint(response_time_max, [200]) orchestrata_ultimate/verification/requirements/requirements_parser extract_performance L93 none
REQ-035-US-001-023 Integration test: Parse complex requirement generates valid Prolog orchestrata_ultimate/verification/requirements/prolog_generator PrologGenerator L13
orchestrata_ultimate/verification/requirements/prolog_generator generate L31
orchestrata_ultimate/verification/requirements/requirements_parser parse L71
none
REQ-035-US-001-024 Performance test: Parse 100 requirements in <500ms average orchestrata_ultimate/verification/requirements/requirements_parser parse L71 none
REQ-035-US-002-001 Define functional constraints (file exists, API responds) orchestrata_ultimate/verification/agents/execution_agent ExecutionAgent L17
orchestrata_ultimate/verification/requirements/constraint_definitions get_constraints_by_type L107
orchestrata_ultimate/verification/types Constraint L99
none
REQ-035-US-002-002 Define quality constraints (test coverage, documentation) orchestrata_ultimate/verification/requirements/constraint_definitions get_constraints_by_type L107
orchestrata_ultimate/verification/types ConstraintType L28
none
REQ-035-US-002-003 Define performance constraints (response time, throughput) orchestrata_ultimate/verification/requirements/constraint_definitions get_constraints_by_type L107
orchestrata_ultimate/verification/types ConstraintType L28
none
REQ-035-US-002-004 Define security constraints (no vulnerabilities, authentication) orchestrata_ultimate/verification/requirements/constraint_definitions get_constraints_by_type L107
orchestrata_ultimate/verification/types ConstraintType L28
none
REQ-035-US-002-005 Support constraint priorities (required vs. optional) orchestrata_ultimate/verification/constraints/definitions sort_by_priority L26
orchestrata_ultimate/verification/types Constraint L99
none
REQ-035-US-002-006 Support constraint dependencies (A must satisfy before B) orchestrata_ultimate/verification/constraints/evaluator eval_one L194
orchestrata_ultimate/verification/constraints/evaluator evaluate_with_dependencies L177
none
REQ-035-US-002-007 Support constraint groups (AND/OR logic) orchestrata_ultimate/verification/constraints/definitions group_by_type L15 none
REQ-035-US-002-008 Validate constraint syntax before activation orchestrata_ultimate/verification/constraints/definitions is_blocking L9
orchestrata_ultimate/verification/requirements/prolog_generator PrologGenerator L13
orchestrata_ultimate/verification/requirements/prolog_generator validate_syntax L73
none
REQ-035-US-002-009 Persist constraint definitions orchestrata_ultimate/verification/integrations/redis_manager VerificationRedisManager L20
orchestrata_ultimate/verification/integrations/redis_manager save_state L45
none
REQ-035-US-002-010 Create constraint definition schema orchestrata_ultimate/verification/types Constraint L99
orchestrata_ultimate/verification/types ConstraintType L28
none
REQ-035-US-002-011 Implement constraint parser (YAML/JSON → Prolog) orchestrata_ultimate/verification/requirements/prolog_generator PrologGenerator L13
orchestrata_ultimate/verification/requirements/prolog_generator generate_predicate L48
none
REQ-035-US-002-012 Create constraint priority system orchestrata_ultimate/verification/constraints/definitions sort_by_priority L26 none
REQ-035-US-002-013 Implement constraint dependency resolution orchestrata_ultimate/verification/constraints/evaluator eval_one L194
orchestrata_ultimate/verification/constraints/evaluator evaluate_with_dependencies L177
none
REQ-035-US-002-014 Create constraint validation rules orchestrata_ultimate/verification/constraints/definitions is_blocking L9
orchestrata_ultimate/verification/requirements/prolog_generator PrologGenerator L13
orchestrata_ultimate/verification/requirements/prolog_generator validate_syntax L73
none
REQ-035-US-002-015 Implement constraint persistence orchestrata_ultimate/verification/integrations/redis_manager VerificationRedisManager L20
orchestrata_ultimate/verification/integrations/redis_manager load_state L67
none
REQ-035-US-002-016 Add constraint versioning none none
REQ-035-US-002-017 Create constraint import/export orchestrata_ultimate/verification/types Constraint L99
orchestrata_ultimate/verification/types from_dict L133
orchestrata_ultimate/verification/types to_dict L121
+3 more
none
REQ-035-US-002-018 Unit test: Constraint with all fields validates orchestrata_ultimate/verification/types Constraint L99 none
REQ-035-US-002-019 Unit test: Invalid constraint syntax rejected orchestrata_ultimate/verification/requirements/prolog_generator PrologGenerator L13
orchestrata_ultimate/verification/requirements/prolog_generator validate_syntax L73
none
REQ-035-US-002-020 Unit test: Priority ordering works correctly orchestrata_ultimate/verification/constraints/definitions sort_by_priority L26 none
REQ-035-US-002-021 Integration test: Constraint dependencies resolve correctly orchestrata_ultimate/verification/constraints/evaluator eval_one L194
orchestrata_ultimate/verification/constraints/evaluator evaluate_with_dependencies L177
none
REQ-035-US-002-022 Integration test: Constraint persistence works orchestrata_ultimate/verification/integrations/redis_manager VerificationRedisManager L20
orchestrata_ultimate/verification/integrations/redis_manager save_state L45
none
REQ-035-US-003-001 Evaluate single constraint satisfaction orchestrata_ultimate/orchestrator/verification_gate ArtifactContextBuilder L290
orchestrata_ultimate/orchestrator/verification_gate _fold_qa_payload_signals L383
orchestrata_ultimate/orchestrator/verification_gate _parses L361
+8 more
tests/orchestrator/test_verification_gate.py::test_artifact_context_builder_computes_actuals
REQ-035-US-003-002 Evaluate all constraints (AND logic) orchestrata_ultimate/verification/constraints/evaluator ConstraintEvaluator L66
orchestrata_ultimate/verification/constraints/evaluator evaluate_all L161
none
REQ-035-US-003-003 Evaluate constraint groups (OR logic) none none
REQ-035-US-003-004 Handle constraint dependencies correctly orchestrata_ultimate/verification/constraints/evaluator eval_one L194
orchestrata_ultimate/verification/constraints/evaluator evaluate_with_dependencies L177
none
REQ-035-US-003-005 Return specific failure reasons orchestrata_ultimate/orchestrator/verification_gate ArtifactContextBuilder L290
orchestrata_ultimate/orchestrator/verification_gate VerificationVerdict L132
orchestrata_ultimate/verification/constraints/evaluator evaluate L113
+4 more
none
REQ-035-US-003-006 Cache evaluation results orchestrata_ultimate/verification/integrations/redis_manager VerificationRedisManager L20 none
REQ-035-US-003-007 Support partial satisfaction orchestrata_ultimate/verification/constraints/evaluator evaluate_all L161
orchestrata_ultimate/verification/types CheckpointResult L215
none
REQ-035-US-003-008 Handle constraint timeouts none none
REQ-035-US-003-009 Log evaluation results orchestrata_ultimate/verification/verification/engine VerificationEngine L28
orchestrata_ultimate/verification/verification/engine verify L49
none
REQ-035-US-003-010 Support evaluation replay orchestrata_ultimate/verification/session/verification_session update_results L120 none
REQ-035-US-003-011 Create constraint evaluator orchestrata_ultimate/verification/constraints/evaluator ConstraintEvaluator L66
orchestrata_ultimate/verification/constraints/evaluator _bridge L95
none
REQ-035-US-003-012 Implement evaluation result aggregation orchestrata_ultimate/verification/constraints/evaluator evaluate_all L161 none
REQ-035-US-003-013 Add result caching (Redis) orchestrata_ultimate/verification/integrations/redis_manager VerificationRedisManager L20
orchestrata_ultimate/verification/integrations/redis_manager save_state L45
none
REQ-035-US-003-014 Create failure reason generator orchestrata_ultimate/orchestrator/verification_gate ArtifactContextBuilder L290
orchestrata_ultimate/verification/verification/feedback FeedbackGenerator L18
orchestrata_ultimate/verification/verification/feedback suggest_fixes L62
none
REQ-035-US-003-015 Implement timeout handling none none
REQ-035-US-003-016 Add evaluation logging orchestrata_ultimate/verification/verification/engine VerificationEngine L28
orchestrata_ultimate/verification/verification/engine verify L49
none
REQ-035-US-003-017 Create evaluation replay mechanism orchestrata_ultimate/verification/session/verification_session update_results L120 none
REQ-035-US-003-018 Unit test: Satisfied constraint returns SATISFIED orchestrata_ultimate/verification/types VerificationStatus L51 none
REQ-035-US-003-019 Unit test: Failed constraint returns NOT_MET with reason orchestrata_ultimate/verification/types VerificationResult L148
orchestrata_ultimate/verification/types VerificationStatus L51
none
REQ-035-US-003-020 Unit test: Multiple constraints aggregate correctly orchestrata_ultimate/verification/constraints/evaluator evaluate_all L161 none
REQ-035-US-003-021 Integration test: Evaluation with timeouts works none none
REQ-035-US-003-022 Integration test: Caching improves performance orchestrata_ultimate/verification/integrations/redis_manager VerificationRedisManager L20 none
REQ-035-US-004-001 Define 5 standard checkpoints (implementation, tests, integration, docs, final) orchestrata_ultimate/orchestrator/verification_gate checkpoint_for L508
orchestrata_ultimate/verification/checkpoints/definitions get_checkpoint_predicates L60
orchestrata_ultimate/verification/types CheckpointName L62
none
REQ-035-US-004-002 Verify at each checkpoint orchestrata_ultimate/orchestrator/pipeline_runner _apply_verification_verdict L703
orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/orchestrator/verification_gate after_stage L555
+3 more
none
REQ-035-US-004-003 Support custom checkpoint definitions orchestrata_ultimate/orchestrator/verification_gate checkpoint_for L508
orchestrata_ultimate/verification/checkpoints/definitions get_checkpoint_predicates L60
none
REQ-035-US-004-004 Track checkpoint progress orchestrata_ultimate/orchestrator/verification_gate after_stage L555
orchestrata_ultimate/verification/checkpoints/verifier CheckpointVerifier L24
orchestrata_ultimate/verification/checkpoints/verifier transition_state L86
+1 more
none
REQ-035-US-004-005 Allow checkpoint skipping (with approval) orchestrata_ultimate/verification/checkpoints/definitions get_next_checkpoint L44 none
REQ-035-US-004-006 Provide checkpoint-specific feedback orchestrata_ultimate/verification/checkpoints/verifier CheckpointVerifier L24
orchestrata_ultimate/verification/checkpoints/verifier get_checkpoint_criteria L79
orchestrata_ultimate/verification/verification/feedback FeedbackGenerator L18
+1 more
none
REQ-035-US-004-007 Support checkpoint rollback orchestrata_ultimate/verification/checkpoints/verifier CheckpointVerifier L24
orchestrata_ultimate/verification/checkpoints/verifier transition_state L86
none
REQ-035-US-004-008 Maintain checkpoint history orchestrata_ultimate/knowledge_graph/temporal get_history L133
orchestrata_ultimate/verification/session/verification_session update_results L120
none
REQ-035-US-004-009 Allow concurrent verification at checkpoints orchestrata_ultimate/verification/checkpoints/concurrent ConcurrentCheckpointEvaluator L22
orchestrata_ultimate/verification/checkpoints/concurrent eval_one L56
orchestrata_ultimate/verification/checkpoints/concurrent evaluate_all L41
none
REQ-035-US-004-010 Create checkpoint definitions orchestrata_ultimate/verification/checkpoints/definitions get_checkpoint_predicates L60
orchestrata_ultimate/verification/checkpoints/definitions get_next_checkpoint L44
none
REQ-035-US-004-011 Implement checkpoint verification logic orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/orchestrator/verification_gate after_stage L555
orchestrata_ultimate/verification/checkpoints/verifier CheckpointVerifier L24
+1 more
none
REQ-035-US-004-012 Create checkpoint progress tracker orchestrata_ultimate/verification/checkpoints/verifier CheckpointVerifier L24
orchestrata_ultimate/verification/checkpoints/verifier transition_state L86
none
REQ-035-US-004-013 Add checkpoint history storage none none
REQ-035-US-004-014 Implement concurrent verification orchestrata_ultimate/verification/checkpoints/concurrent ConcurrentCheckpointEvaluator L22
orchestrata_ultimate/verification/protocols/verification_cxp create_verification_request L52
none
REQ-035-US-004-015 Create custom checkpoint API orchestrata_ultimate/verification/checkpoints/definitions get_checkpoint_predicates L60 none
REQ-035-US-004-016 Add checkpoint rollback support orchestrata_ultimate/verification/checkpoints/verifier CheckpointVerifier L24
orchestrata_ultimate/verification/checkpoints/verifier transition_state L86
none
REQ-035-US-004-017 Unit test: Checkpoint criteria evaluate correctly orchestrata_ultimate/verification/checkpoints/verifier CheckpointVerifier L24
orchestrata_ultimate/verification/checkpoints/verifier get_checkpoint_criteria L79
none
REQ-035-US-004-018 Unit test: Checkpoint progress tracking works orchestrata_ultimate/verification/checkpoints/verifier CheckpointVerifier L24
orchestrata_ultimate/verification/checkpoints/verifier transition_state L86
none
REQ-035-US-004-019 Integration test: Full checkpoint sequence passes orchestrata_ultimate/verification/checkpoints/definitions get_next_checkpoint L44
orchestrata_ultimate/verification/checkpoints/verifier CheckpointVerifier L24
none
REQ-035-US-004-020 Integration test: Concurrent verification works orchestrata_ultimate/verification/checkpoints/concurrent ConcurrentCheckpointEvaluator L22
orchestrata_ultimate/verification/checkpoints/concurrent eval_one L56
orchestrata_ultimate/verification/checkpoints/concurrent evaluate_all L41
+1 more
none
REQ-035-US-004-021 Performance test: 5 checkpoints in <5 seconds orchestrata_ultimate/verification/checkpoints/concurrent ConcurrentCheckpointEvaluator L22
orchestrata_ultimate/verification/checkpoints/concurrent eval_one L56
orchestrata_ultimate/verification/checkpoints/concurrent evaluate_all L41
none
REQ-035-US-005-001 Support Claude Opus 4 for verification orchestrata_ultimate/verification/agents/supervisor_agent generate_proof_via_llm L77
orchestrata_ultimate/verification/models/opus ClaudeOpusModel L13
orchestrata_ultimate/verification/models/opus analyze L56
+3 more
none
REQ-035-US-005-002 Support Deepseek R1 for verification orchestrata_ultimate/verification/models/deepseek_r1 DeepseekR1Model L13
orchestrata_ultimate/verification/models/deepseek_r1 analyze L56
orchestrata_ultimate/verification/models/deepseek_r1 capabilities L35
+2 more
none
REQ-035-US-005-003 Support Qwen Max for verification orchestrata_ultimate/verification/models/qwen_max QwenMaxModel L13
orchestrata_ultimate/verification/models/qwen_max analyze L56
orchestrata_ultimate/verification/models/qwen_max capabilities L35
+2 more
none
REQ-035-US-005-004 Support Grok4 MiniMax for verification orchestrata_ultimate/verification/models/grok4_minimax GrokMiniMaxModel L13
orchestrata_ultimate/verification/models/grok4_minimax analyze L54
orchestrata_ultimate/verification/models/grok4_minimax capabilities L33
+2 more
none
REQ-035-US-005-005 Select appropriate model based on task orchestrata_ultimate/verification/agents/verification_agent __init__ L55
orchestrata_ultimate/verification/agents/verification_agent analyze_via_llm L84
orchestrata_ultimate/verification/models/base_model ModelRegistry L65
+3 more
none
REQ-035-US-005-006 Handle model unavailability gracefully orchestrata_ultimate/verification/agents/supervisor_agent generate_proof_via_llm L77
orchestrata_ultimate/verification/models/base_model ModelFallbackChain L109
orchestrata_ultimate/verification/models/base_model analyze L54
+3 more
none
REQ-035-US-005-007 Maintain verification consistency across models orchestrata_ultimate/verification/agents/verification_agent VerificationAgent L37
orchestrata_ultimate/verification/agents/verification_agent analyze_via_llm L85
orchestrata_ultimate/verification/models/base_model LLMModel L28
+1 more
none
REQ-035-US-005-008 Track model usage for metrics orchestrata_ultimate/verification/models/base_model ModelRegistry L65 none
REQ-035-US-005-009 Support model fallback chain orchestrata_ultimate/verification/models/base_model ModelFallbackChain L109
orchestrata_ultimate/verification/models/base_model generate_proof L57
orchestrata_ultimate/verification/models/base_model generate_proof L171
+3 more
none
REQ-035-US-005-010 Create model abstraction layer orchestrata_ultimate/verification/models/base_model LLMModel L28
orchestrata_ultimate/verification/models/base_model name L37
none
REQ-035-US-005-011 Implement Opus integration orchestrata_ultimate/verification/agents/supervisor_agent generate_proof_via_llm L77
orchestrata_ultimate/verification/models/opus ClaudeOpusModel L13
orchestrata_ultimate/verification/models/opus analyze L56
+3 more
none
REQ-035-US-005-012 Implement Deepseek R1 integration orchestrata_ultimate/verification/models/deepseek_r1 DeepseekR1Model L13
orchestrata_ultimate/verification/models/deepseek_r1 analyze L56
orchestrata_ultimate/verification/models/deepseek_r1 capabilities L35
+2 more
none
REQ-035-US-005-013 Implement Qwen Max integration orchestrata_ultimate/verification/models/qwen_max QwenMaxModel L13
orchestrata_ultimate/verification/models/qwen_max analyze L56
orchestrata_ultimate/verification/models/qwen_max capabilities L35
+2 more
none
REQ-035-US-005-014 Implement Grok4 MiniMax integration orchestrata_ultimate/verification/models/grok4_minimax GrokMiniMaxModel L13
orchestrata_ultimate/verification/models/grok4_minimax analyze L54
orchestrata_ultimate/verification/models/grok4_minimax capabilities L33
+2 more
none
REQ-035-US-005-015 Create model selection logic orchestrata_ultimate/verification/models/base_model ModelRegistry L65
orchestrata_ultimate/verification/models/base_model get L85
none
REQ-035-US-005-016 Add fallback handling orchestrata_ultimate/verification/models/base_model ModelFallbackChain L109
orchestrata_ultimate/verification/models/base_model generate_proof L57
orchestrata_ultimate/verification/models/base_model generate_proof L171
+2 more
none
REQ-035-US-005-017 Implement usage tracking orchestrata_ultimate/verification/models/base_model ModelRegistry L65 none
REQ-035-US-005-018 Create cost optimization orchestrata_ultimate/verification/models/base_model ModelFallbackChain L109 none
REQ-035-US-005-019 Unit test: Model abstraction works orchestrata_ultimate/verification/models/base_model LLMModel L28
orchestrata_ultimate/verification/models/base_model list_models L91
none
REQ-035-US-005-020 Unit test: Fallback triggers correctly orchestrata_ultimate/verification/models/base_model ModelFallbackChain L109
orchestrata_ultimate/verification/models/base_model verify L40
orchestrata_ultimate/verification/models/base_model verify L133
none
REQ-035-US-005-021 Integration test: All models respond correctly orchestrata_ultimate/verification/models/deepseek_r1 DeepseekR1Model L13
orchestrata_ultimate/verification/models/deepseek_r1 verify L46
orchestrata_ultimate/verification/models/grok4_minimax GrokMiniMaxModel L13
+5 more
none
REQ-035-US-005-022 Integration test: Verification consistent across models orchestrata_ultimate/verification/agents/verification_agent VerificationAgent L37
orchestrata_ultimate/verification/agents/verification_agent analyze_via_llm L85
orchestrata_ultimate/verification/protocols/verification_cxp create_verification_request L52
none
REQ-035-US-005-023 Performance test: Model response times acceptable orchestrata_ultimate/verification/models/base_model ModelFallbackChain L109
orchestrata_ultimate/verification/models/base_model list_models L91
orchestrata_ultimate/verification/models/base_model verify L40
+1 more
none
REQ-035-US-006-001 Identify common edge cases for task type orchestrata_ultimate/verification/edge_cases/detector EdgeCaseDetector L375
orchestrata_ultimate/verification/edge_cases/detector _init_common_edge_cases L114
orchestrata_ultimate/verification/edge_cases/detector get_edge_cases_for_task L337
none
REQ-035-US-006-002 Verify edge cases are tested orchestrata_ultimate/verification/edge_cases/detector EdgeCaseOccurrence L66
orchestrata_ultimate/verification/edge_cases/detector _is_edge_case_tested L533
none
REQ-035-US-006-003 Suggest edge cases to test orchestrata_ultimate/verification/edge_cases/detector suggest_edge_cases L504 none
REQ-035-US-006-004 Detect untested edge cases orchestrata_ultimate/verification/edge_cases/detector EdgeCaseOccurrence L66 none
REQ-035-US-006-005 Categorize edge cases by severity orchestrata_ultimate/verification/edge_cases/detector EdgeCaseCategory L33
orchestrata_ultimate/verification/edge_cases/detector EdgeCaseSeverity L23
orchestrata_ultimate/verification/types EdgeCase L306
none
REQ-035-US-006-006 Track edge case coverage orchestrata_ultimate/verification/edge_cases/database EdgeCaseDatabase L13
orchestrata_ultimate/verification/edge_cases/database count L64
orchestrata_ultimate/verification/edge_cases/detector EdgeCaseCoverage L83
+2 more
none
REQ-035-US-006-007 Provide edge case examples orchestrata_ultimate/verification/edge_cases/database EdgeCaseDatabase L13
orchestrata_ultimate/verification/edge_cases/database get_by_constraint L46
orchestrata_ultimate/verification/edge_cases/detector generate_test_suggestions L588
none
REQ-035-US-006-008 Support custom edge case definitions orchestrata_ultimate/verification/edge_cases/database EdgeCaseDatabase L13
orchestrata_ultimate/verification/edge_cases/database add L34
orchestrata_ultimate/verification/edge_cases/detector EdgeCase L47
+1 more
none
REQ-035-US-006-009 Create edge case database orchestrata_ultimate/verification/edge_cases/database EdgeCaseDatabase L13
orchestrata_ultimate/verification/edge_cases/detector EdgeCaseDatabase L96
none
REQ-035-US-006-010 Implement edge case detection orchestrata_ultimate/verification/edge_cases/detector EdgeCaseDetector L375
orchestrata_ultimate/verification/edge_cases/detector detect L391
orchestrata_ultimate/verification/edge_cases/detector detect_boundary_cases L425
+2 more
none
REQ-035-US-006-011 Create edge case suggestion engine none none
REQ-035-US-006-012 Add edge case coverage tracking orchestrata_ultimate/verification/edge_cases/database EdgeCaseDatabase L13
orchestrata_ultimate/verification/edge_cases/database count L64
orchestrata_ultimate/verification/edge_cases/detector calculate_coverage L556
none
REQ-035-US-006-013 Implement edge case severity scoring orchestrata_ultimate/verification/types EdgeCase L306 none
REQ-035-US-006-014 Create edge case reporting orchestrata_ultimate/verification/edge_cases/database EdgeCaseDatabase L13
orchestrata_ultimate/verification/edge_cases/database list_all L51
orchestrata_ultimate/verification/edge_cases/detector EdgeCaseReporter L680
+4 more
none
REQ-035-US-006-015 Unit test: Edge cases identified for task none tests/verification/edge_cases/test_detector.py::test_detect_edge_cases_in_code
REQ-035-US-006-016 Unit test: Edge case coverage calculated orchestrata_ultimate/verification/edge_cases/database EdgeCaseDatabase L13 none
REQ-035-US-006-017 Integration test: Edge case suggestions improve coverage none tests/verification/edge_cases/test_detector.py::test_calculate_coverage_with_tested
REQ-035-US-006-018 Integration test: Edge case tracking works orchestrata_ultimate/verification/edge_cases/database EdgeCaseDatabase L13 none
REQ-035-US-007-001 Accept exhaustive search proofs orchestrata_ultimate/verification/proofs/verifier verify_exhaustive_search L43 none
REQ-035-US-007-002 Accept logical deduction proofs orchestrata_ultimate/verification/proofs/verifier verify_logical_deduction L76 none
REQ-035-US-007-003 Accept mathematical proofs orchestrata_ultimate/verification/proofs/verifier verify_mathematical L100 none
REQ-035-US-007-004 Accept practical explanations orchestrata_ultimate/verification/proofs/verifier verify_practical_explanation L124 none
REQ-035-US-007-005 Verify proof completeness orchestrata_ultimate/verification/proofs/verifier verify L19 none
REQ-035-US-007-006 Verify proof soundness orchestrata_ultimate/verification/proofs/verifier verify L19 none
REQ-035-US-007-007 Verify alternatives exhausted orchestrata_ultimate/verification/proofs/verifier verify_exhaustive_search L43 none
REQ-035-US-007-008 Reject invalid proofs orchestrata_ultimate/verification/proofs/verifier verify L19
orchestrata_ultimate/verification/types ProofResult L203
none
REQ-035-US-007-009 Request additional proof if needed orchestrata_ultimate/verification/orchestration/coordinator build_constraint_message L157
orchestrata_ultimate/verification/orchestration/coordinator build_escalation_message L185
orchestrata_ultimate/verification/orchestration/coordinator build_feedback_message L191
+7 more
none
REQ-035-US-007-010 Create proof type handlers orchestrata_ultimate/verification/proofs/verifier ProofVerifier L15 none
REQ-035-US-007-011 Implement exhaustive search verification orchestrata_ultimate/verification/proofs/verifier verify_exhaustive_search L43 none
REQ-035-US-007-012 Implement logical deduction verification orchestrata_ultimate/verification/proofs/verifier verify_logical_deduction L76
orchestrata_ultimate/verification/protocols/verification_cxp create_verification_request L52
none
REQ-035-US-007-013 Implement mathematical proof verification orchestrata_ultimate/verification/proofs/verifier verify_mathematical L100
orchestrata_ultimate/verification/protocols/verification_cxp create_verification_request L52
none
REQ-035-US-007-014 Implement practical explanation verification orchestrata_ultimate/verification/proofs/verifier verify_practical_explanation L124
orchestrata_ultimate/verification/protocols/verification_cxp create_verification_request L52
none
REQ-035-US-007-015 Create proof evaluation criteria orchestrata_ultimate/verification/types ProofResult L203 none
REQ-035-US-007-016 Add proof rejection handling orchestrata_ultimate/verification/agents/verification_agent VerificationAgent L37
orchestrata_ultimate/verification/agents/verification_agent validate_proof L120
none
REQ-035-US-007-017 Implement proof appeal process orchestrata_ultimate/verification/integrations/intent_broker processed_count L76
orchestrata_ultimate/verification/orchestration/coordinator build_proof_submission L204
orchestrata_ultimate/verification/protocols/verification_cxp VerificationCXPAdapter L19
+1 more
none
REQ-035-US-007-018 Unit test: Valid exhaustive proof accepted orchestrata_ultimate/verification/proofs/verifier verify_exhaustive_search L43 none
REQ-035-US-007-019 Unit test: Invalid proof rejected orchestrata_ultimate/verification/proofs/verifier verify L19 none
REQ-035-US-007-020 Integration test: Full proof workflow works orchestrata_ultimate/verification/agents/supervisor_agent SupervisorAgent L27
orchestrata_ultimate/verification/agents/supervisor_agent generate_proof L60
orchestrata_ultimate/verification/agents/supervisor_agent submit_proof L110
+2 more
none
REQ-035-US-007-021 Integration test: Proof appeal process works orchestrata_ultimate/verification/protocols/verification_cxp VerificationCXPAdapter L19
orchestrata_ultimate/verification/protocols/verification_cxp create_proof_request L141
none
REQ-035-US-008-001 Track verification attempts orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/orchestrator/verification_gate finalize L624
orchestrata_ultimate/verification/session/verification_session increment_attempt L98
none
REQ-035-US-008-002 Enforce max attempts (5) orchestrata_ultimate/orchestrator/verification_gate VerificationGateConfig L94
orchestrata_ultimate/verification/session/verification_session is_max_exceeded L110
orchestrata_ultimate/verification/types max_exceeded L288
none
REQ-035-US-008-003 Provide feedback after each attempt orchestrata_ultimate/orchestrator/verification_gate VerificationVerdict L132
orchestrata_ultimate/orchestrator/verification_gate finalize L624
orchestrata_ultimate/verification/verification/feedback FeedbackGenerator L18
+1 more
none
REQ-035-US-008-004 Support automatic retry orchestrata_ultimate/orchestrator/verification_gate VerificationGateConfig L94
orchestrata_ultimate/orchestrator/verification_gate from_env L111
orchestrata_ultimate/verification/orchestration/coordinator VerificationCoordinator L43
+1 more
tests/orchestrator/test_verification_gate.py::test_gate_config_from_env_defaults_audit_only
REQ-035-US-008-005 Support manual intervention orchestrata_ultimate/orchestrator/verification_gate VerificationGateConfig L94
orchestrata_ultimate/verification/escalation_handler EscalationHandler L18
orchestrata_ultimate/verification/escalation_handler build_escalation_report L55
+3 more
none
REQ-035-US-008-006 Escalate after max attempts orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/orchestrator/verification_gate finalize L624
orchestrata_ultimate/verification/escalation_handler EscalationHandler L18
+1 more
none
REQ-035-US-008-007 Track attempt history orchestrata_ultimate/verification/session/verification_session update_results L120 none
REQ-035-US-008-008 Support attempt reset orchestrata_ultimate/verification/session/verification_session update_status L142 none
REQ-035-US-008-009 Log all attempts orchestrata_ultimate/verification/verification/engine VerificationEngine L28
orchestrata_ultimate/verification/verification/engine verify L49
none
REQ-035-US-008-010 Create attempt tracking system orchestrata_ultimate/verification/session/verification_session increment_attempt L98 none
REQ-035-US-008-011 Implement max attempt enforcement orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/verification/session/verification_session is_max_exceeded L110
none
REQ-035-US-008-012 Create feedback generation orchestrata_ultimate/verification/orchestration/coordinator build_feedback_message L192
orchestrata_ultimate/verification/protocols/verification_cxp create_feedback_message L126
orchestrata_ultimate/verification/verification/feedback FeedbackGenerator L18
none
REQ-035-US-008-013 Add automatic retry support orchestrata_ultimate/verification/orchestration/coordinator VerificationCoordinator L43
orchestrata_ultimate/verification/orchestration/coordinator advance_step L81
none
REQ-035-US-008-014 Implement escalation handler orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/verification/escalation_handler EscalationHandler L18
none
REQ-035-US-008-015 Create attempt history storage none none
REQ-035-US-008-016 Add attempt reset functionality orchestrata_ultimate/verification/session/verification_session update_status L142 none
REQ-035-US-008-017 Implement attempt logging orchestrata_ultimate/verification/verification/engine VerificationEngine L28
orchestrata_ultimate/verification/verification/engine verify L49
none
REQ-035-US-008-018 Unit test: Max attempts enforced orchestrata_ultimate/verification/session/verification_session is_max_exceeded L110 none
REQ-035-US-008-019 Unit test: Feedback generated correctly orchestrata_ultimate/verification/verification/feedback FeedbackGenerator L18
orchestrata_ultimate/verification/verification/feedback generate_feedback L36
none
REQ-035-US-008-020 Integration test: Full retry loop works orchestrata_ultimate/orchestrator/pipeline_runner _apply_verification_verdict L703
orchestrata_ultimate/verification/orchestration/coordinator VerificationCoordinator L43
tests/orchestrator/test_verification_gate.py::test_pipeline_runner_default_constructs_gate_nonfatally
REQ-035-US-008-021 Integration test: Escalation triggers correctly orchestrata_ultimate/verification/escalation_handler EscalationHandler L18
orchestrata_ultimate/verification/escalation_handler should_escalate L34
none
REQ-035-US-009-001 Receive intents from Intent Broker orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/orchestrator/verification_gate open_session L518
orchestrata_ultimate/verification/integrations/intent_broker IntentBrokerAdapter L12
+2 more
none
REQ-035-US-009-002 Extract constraints from intents orchestrata_ultimate/orchestrator/verification_gate WorkTypeConstraintSource L226
orchestrata_ultimate/orchestrator/verification_gate constraints_for L247
orchestrata_ultimate/verification/integrations/intent_broker IntentBrokerAdapter L12
+2 more
tests/orchestrator/test_verification_gate.py::test_worktype_constraint_source_total_over_worktypes
REQ-035-US-009-003 Generate verification requirements from intents orchestrata_ultimate/orchestrator/verification_gate WorkTypeConstraintSource L226
orchestrata_ultimate/orchestrator/verification_gate constraints_for L247
orchestrata_ultimate/verification/integrations/intent_broker IntentBrokerAdapter L12
+1 more
none
REQ-035-US-009-004 Return verification results to Intent Broker orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/orchestrator/verification_gate VerificationVerdict L132
orchestrata_ultimate/orchestrator/verification_gate finalize L624
+11 more
tests/orchestrator/test_verification_gate.py::test_verdict_packet_built_via_coordinator_cxp
REQ-035-US-009-005 Support intent lifecycle integration orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/verification/orchestration/coordinator VerificationCoordinator L43
orchestrata_ultimate/verification/orchestration/coordinator start_workflow L69
none
REQ-035-US-009-006 Track verification history per intent orchestrata_ultimate/orchestrator/verification_gate _persist L741
orchestrata_ultimate/verification/session/verification_session update_results L120
none
REQ-035-US-009-007 Support intent replay with verification orchestrata_ultimate/verification/orchestration/coordinator VerificationCoordinator L43
orchestrata_ultimate/verification/orchestration/coordinator advance_step L81
none
REQ-035-US-009-008 Coordinate with Prolog validation orchestrata_ultimate/verification/agents/verification_agent VerificationAgent L37
orchestrata_ultimate/verification/agents/verification_agent check_via_prolog L77
none
REQ-035-US-009-009 Create Intent Broker integration API orchestrata_ultimate/verification/integrations/intent_broker IntentBrokerAdapter L12
orchestrata_ultimate/verification/session/verification_session create_session_from_intent L67
none
REQ-035-US-009-010 Implement intent constraint extraction orchestrata_ultimate/orchestrator/verification_gate WorkTypeConstraintSource L226
orchestrata_ultimate/orchestrator/verification_gate constraints_for L247
orchestrata_ultimate/verification/integrations/intent_broker IntentBrokerAdapter L12
+2 more
tests/orchestrator/test_verification_gate.py::test_constraint_source_delegates_to_intent_broker_adapter
REQ-035-US-009-011 Create result forwarding orchestrata_ultimate/verification/orchestration/coordinator build_constraint_message L158
orchestrata_ultimate/verification/orchestration/coordinator build_verification_result L174
orchestrata_ultimate/verification/protocols/verification_cxp VerificationCXPAdapter L19
+1 more
none
REQ-035-US-009-012 Add lifecycle coordination orchestrata_ultimate/orchestrator/verification_gate PipelineVerificationGate L419
orchestrata_ultimate/verification/orchestration/coordinator VerificationCoordinator L43
none
REQ-035-US-009-013 Implement history tracking orchestrata_ultimate/knowledge_graph/temporal get_history L133
orchestrata_ultimate/verification/session/verification_session update_results L120
none
REQ-035-US-009-014 Create replay support orchestrata_ultimate/verification/orchestration/coordinator VerificationCoordinator L43
orchestrata_ultimate/verification/orchestration/coordinator advance_step L81
none
REQ-035-US-009-015 Unit test: Intent parsing works orchestrata_ultimate/verification/integrations/intent_broker IntentBrokerAdapter L12
orchestrata_ultimate/verification/integrations/intent_broker intent_to_constraints L32
none
REQ-035-US-009-016 Unit test: Constraint extraction works orchestrata_ultimate/verification/integrations/intent_broker IntentBrokerAdapter L12
orchestrata_ultimate/verification/integrations/intent_broker intent_to_constraints L32
none
REQ-035-US-009-017 Integration test: Full integration with Intent Broker orchestrata_ultimate/verification/integrations/intent_broker IntentBrokerAdapter L12 none
REQ-035-US-009-018 Integration test: Intent replay works orchestrata_ultimate/verification/orchestration/coordinator VerificationCoordinator L43
orchestrata_ultimate/verification/orchestration/coordinator advance_step L81
none
REQ-035-US-010-001 Detect hardcoded secrets none none
REQ-035-US-010-002 Detect SQL injection vulnerabilities none none
REQ-035-US-010-003 Detect XSS vulnerabilities none none
REQ-035-US-010-004 Verify authentication implementation orchestrata_ultimate/verification/requirements/constraint_definitions get_constraints_by_type L107
orchestrata_ultimate/verification/types ConstraintType L28
none
REQ-035-US-010-005 Verify authorization implementation orchestrata_ultimate/verification/requirements/constraint_definitions get_constraints_by_type L107
orchestrata_ultimate/verification/types ConstraintType L28
none
REQ-035-US-010-006 Check dependency vulnerabilities none none
REQ-035-US-010-007 Verify encryption usage none none
REQ-035-US-010-008 Check compliance requirements orchestrata_ultimate/verification/requirements/constraint_definitions get_constraints_by_type L107 none
REQ-035-US-010-009 Support custom security rules orchestrata_ultimate/verification/edge_cases/database EdgeCaseDatabase L13
orchestrata_ultimate/verification/edge_cases/database add L34
none
REQ-035-US-010-010 Create security constraint library orchestrata_ultimate/verification/requirements/constraint_definitions get_constraints_by_type L107
orchestrata_ultimate/verification/types ConstraintType L28
none
REQ-035-US-010-011 Implement secret detection none none
REQ-035-US-010-012 Implement vulnerability scanning none none
REQ-035-US-010-013 Create authentication verifier none none
REQ-035-US-010-014 Create authorization verifier none none
REQ-035-US-010-015 Add dependency auditing none none
REQ-035-US-010-016 Implement compliance checking none none
REQ-035-US-010-017 Create custom rule support orchestrata_ultimate/verification/requirements/constraint_definitions get_constraint_by_id L127 none
REQ-035-US-010-018 Unit test: Secret detection works none none
REQ-035-US-010-019 Unit test: Vulnerability detection works none none
REQ-035-US-010-020 Integration test: Security scan completes none none
REQ-035-US-010-021 Integration test: Custom rules work orchestrata_ultimate/verification/requirements/constraint_definitions get_constraint_by_id L127 none
REQ-035-US-011-001 Measure response time none none
REQ-035-US-011-002 Measure throughput none none
REQ-035-US-011-003 Measure memory usage none none
REQ-035-US-011-004 Measure cold start time none none
REQ-035-US-011-005 Support benchmark execution none none
REQ-035-US-011-006 Support performance profiling none none
REQ-035-US-011-007 Handle performance variability none none
REQ-035-US-011-008 Support percentile requirements none none
REQ-035-US-011-009 Generate performance reports orchestrata_ultimate/verification/verification/feedback FeedbackGenerator L18
orchestrata_ultimate/verification/verification/feedback generate_feedback L36
none
REQ-035-US-011-010 Create performance constraint library orchestrata_ultimate/verification/requirements/constraint_definitions get_constraints_by_type L107
orchestrata_ultimate/verification/types ConstraintType L28
none
REQ-035-US-011-011 Implement benchmarking none none
REQ-035-US-011-012 Create profiling support none none
REQ-035-US-011-013 Add statistical analysis none none
REQ-035-US-011-014 Implement variability handling none none
REQ-035-US-011-015 Create report generation orchestrata_ultimate/verification/verification/feedback FeedbackGenerator L18
orchestrata_ultimate/verification/verification/feedback generate_feedback L36
none
REQ-035-US-011-016 Unit test: Response time measurement works none none
REQ-035-US-011-017 Unit test: Throughput measurement works none none
REQ-035-US-011-018 Integration test: Performance report generated orchestrata_ultimate/verification/verification/feedback FeedbackGenerator L18
orchestrata_ultimate/verification/verification/feedback generate_feedback L36
none
REQ-035-US-011-019 Performance test: Measurements accurate none none