Orchestrata Ultimate › Plans › evidence › w5-spec102-classification
W5.1a — spec-102 classification (96 REQs: 65 stamp / 20 partial / 11 missing)
Source: plans/evidence/w5-spec102-classification.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-06 · 79 lines.
Prose pending.
Plan text
W5.1a — spec-102 classification (96 REQs: 65 stamp / 20 partial / 11 missing)
Stamp list below is machine-actionable (REQ -> file:line Symbol). PARTIAL/MISSING details live in the session log + backlog items filed 2026-07-06. Key real bugs: aggregate-score weight drift (runner.py:383), enforce_whitelist never called (executor.py:167), scenario-YAML abort (scenario.py:122/162), junit never generated (reporting.py:301), no temp-dir fallback, neo4j prefix unused, silent snapshot-read swallow. Spec updates: REQ-102-128 xdist contradiction, -111 retry policy, -012/-092 worker wording, -126/129/130 unstampable meta.
Stamp list (65)
REQ-102-001 -> src/orchestrata_ultimate/harness/scenario.py:93 load_scenarios_for_volk
REQ-102-002 -> src/orchestrata_ultimate/harness/models.py:30 Scenario
REQ-102-003 -> src/orchestrata_ultimate/harness/models.py:30 Scenario
REQ-102-004 -> src/orchestrata_ultimate/harness/scenario.py:72 load_scenario
REQ-102-006 -> src/orchestrata_ultimate/harness/cli.py:172 _cmd_list
REQ-102-008 -> src/orchestrata_ultimate/harness/cli.py:114 _run_selected_scenarios
REQ-102-009 -> src/orchestrata_ultimate/harness/cli.py:114 _run_selected_scenarios
REQ-102-010 -> src/orchestrata_ultimate/harness/runner.py:78 HarnessRunner.run_scenario
REQ-102-011 -> src/orchestrata_ultimate/harness/runner.py:78 HarnessRunner.run_scenario
REQ-102-013 -> src/orchestrata_ultimate/harness/runner.py:78 HarnessRunner.run_scenario
REQ-102-014 -> src/orchestrata_ultimate/harness/models.py:30 Scenario
REQ-102-017 -> src/orchestrata_ultimate/harness/cli.py:145 _cmd_run
REQ-102-018 -> src/orchestrata_ultimate/harness/cli.py:37 _print_results_table
REQ-102-022 -> src/orchestrata_ultimate/harness/history.py:70 JSONHistoryTracker.record
REQ-102-025 -> src/orchestrata_ultimate/harness/history.py:140 Neo4jHistoryTracker.record
REQ-102-027 -> src/orchestrata_ultimate/harness/runner.py:78 HarnessRunner.run_scenario
REQ-102-029 -> src/orchestrata_ultimate/harness/reporting.py:152 generate_report
REQ-102-030 -> src/orchestrata_ultimate/harness/reporting.py:152 generate_report
REQ-102-031 -> src/orchestrata_ultimate/harness/reporting.py:152 generate_report
REQ-102-032 -> src/orchestrata_ultimate/harness/reporting.py:152 generate_report
REQ-102-033 -> src/orchestrata_ultimate/harness/reporting.py:152 generate_report
REQ-102-035 -> src/orchestrata_ultimate/harness/healing.py:102 SelfHealingLoop.__init__
REQ-102-036 -> src/orchestrata_ultimate/harness/runner.py:326 HarnessRunner._maybe_self_heal
REQ-102-038 -> src/orchestrata_ultimate/harness/healing.py:190 SelfHealingLoop._run_hop
REQ-102-040 -> src/orchestrata_ultimate/harness/healing.py:190 SelfHealingLoop._run_hop
REQ-102-041 -> src/orchestrata_ultimate/harness/healing.py:190 SelfHealingLoop._run_hop
REQ-102-042 -> src/orchestrata_ultimate/harness/healing.py:48 _extract_strategy
REQ-102-043 -> src/orchestrata_ultimate/harness/healing.py:190 SelfHealingLoop._run_hop
REQ-102-044 -> src/orchestrata_ultimate/harness/healing.py:190 SelfHealingLoop._run_hop
REQ-102-045 -> src/orchestrata_ultimate/harness/healing.py:190 SelfHealingLoop._run_hop
REQ-102-047 -> src/orchestrata_ultimate/harness/healing.py:106 SelfHealingLoop.heal
REQ-102-048 -> src/orchestrata_ultimate/harness/healing.py:425 SelfHealingLoop._assert_under_worktree
REQ-102-051 -> src/orchestrata_ultimate/harness/snapshots.py:84 SnapshotManager._snapshot_path
REQ-102-052 -> src/orchestrata_ultimate/harness/snapshots.py:89 SnapshotManager._build_header
REQ-102-054 -> src/orchestrata_ultimate/harness/runner.py:175 HarnessRunner._run_single_trial
REQ-102-056 -> src/orchestrata_ultimate/harness/snapshots.py:154 _transform_value
REQ-102-060 -> src/orchestrata_ultimate/harness/runner.py:46 _COMPOSITE_PASS_THRESHOLD
REQ-102-068 -> src/orchestrata_ultimate/harness/runner.py:283 HarnessRunner._run_evaluators_stage
REQ-102-069 -> src/orchestrata_ultimate/harness/runner.py:345 HarnessRunner._run_evaluator
REQ-102-071 -> src/orchestrata_ultimate/harness/evaluators/llm_judge.py:29 _build_user_message
REQ-102-074 -> src/orchestrata_ultimate/harness/evaluators/llm_judge.py:86 LLMJudgeEvaluator.evaluate
REQ-102-075 -> src/orchestrata_ultimate/harness/runner.py:362 HarnessRunner._compute_composite_score
REQ-102-076 -> src/orchestrata_ultimate/harness/models.py:21 EvaluatorConfig
REQ-102-077 -> src/orchestrata_ultimate/harness/models.py:21 EvaluatorConfig
REQ-102-078 -> src/orchestrata_ultimate/harness/runner.py:175 HarnessRunner._run_single_trial
REQ-102-087 -> src/orchestrata_ultimate/harness/scenario.py:174 run_setup_hook
REQ-102-088 -> src/orchestrata_ultimate/harness/scenario.py:174 run_setup_hook
REQ-102-089 -> src/orchestrata_ultimate/harness/runner.py:225 HarnessRunner._run_setup_stage
REQ-102-090 -> src/orchestrata_ultimate/harness/runner.py:225 HarnessRunner._run_setup_stage
REQ-102-091 -> src/orchestrata_ultimate/harness/runner.py:78 HarnessRunner.run_scenario
REQ-102-093 -> src/orchestrata_ultimate/harness/isolation.py:226 get_neo4j_prefix
REQ-102-098 -> src/orchestrata_ultimate/harness/executor.py:42 CXPacketExecutor.execute
REQ-102-101 -> src/orchestrata_ultimate/harness/executor.py:82 DirectCallExecutor.execute
REQ-102-103 -> src/orchestrata_ultimate/harness/executor.py:82 DirectCallExecutor.execute
REQ-102-104 -> src/orchestrata_ultimate/harness/config.py:19 load_config
REQ-102-105 -> src/orchestrata_ultimate/harness/config.py:73 _apply_env
REQ-102-106 -> src/orchestrata_ultimate/harness/config.py:73 _apply_env
REQ-102-107 -> src/orchestrata_ultimate/harness/models.py:103 HarnessConfig
REQ-102-108 -> src/orchestrata_ultimate/harness/config.py:19 load_config
REQ-102-114 -> src/orchestrata_ultimate/harness/history.py:140 Neo4jHistoryTracker.record
REQ-102-116 -> src/orchestrata_ultimate/harness/runner.py:175 HarnessRunner._run_single_trial
REQ-102-117 -> src/orchestrata_ultimate/harness/runner.py:345 HarnessRunner._run_evaluator
REQ-102-120 -> src/orchestrata_ultimate/harness/models.py:103 HarnessConfig
REQ-102-123 -> src/orchestrata_ultimate/harness/healing.py:425 SelfHealingLoop._assert_under_worktree
REQ-102-124 -> src/orchestrata_ultimate/harness/evaluators/llm_judge.py:29 _build_user_message