Knowledge Wiki Orchestrata Ultimate

orchestrata_volks.harness

Reference › components › volks › orchestrata_volks › harness

Volk Test Harness — scenario-driven evaluation system for autonomous agents.

The Volk Test Harness of orchestrata_volks: a scenario-driven evaluation system for autonomous agents. scenario loads, validates and sets up scenarios; harness_runner (474 lines) runs them through an executor (direct_call_executor or c_x_packet_executor); evaluators in evaluators score the results into scenario_result and trial_result. self_healing_loop retries with healing; history is kept by json_history_tracker and graph_history_tracker with trend_queries. Isolation is incus_isolation, worktree_isolation, isolation_manager and snapshot_manager. reporting formats results, config loads configuration, and cli (429 lines) is the argparse-based command line. See the harness narrative.

42 modules, 4,373 lines of code.

Subpackages

Package Modules LOC
evaluators 7 710

Modules

Module Summary LOC Lifted from
__init__ Volk Test Harness — scenario-driven evaluation system for autonomous agents. 25 lifted from orchestrata_ultimate.harness
c_x_packet_executor 69
cli CLI interface for the Volk Test Harness — argparse-based, minimal dependencies. 429 lifted from orchestrata_ultimate.harness.cli
config Configuration loader for the Volk Test Harness. 134 lifted from orchestrata_ultimate.harness.config
direct_call_executor 104
evaluator 31
evaluator_config 19
evaluator_result 17
evaluators Whole-module re-export — see packages/volks/design/decisions.md (clean-target split). 13 lifted from orchestrata_ultimate.harness.evaluators
executor 58 lifted from orchestrata_ultimate.harness.executor
graph_history_tracker 303
harness_config 63
harness_context 25
harness_runner 474
healing 4 lifted from orchestrata_ultimate.harness.healing
healing_result 19
history 6 lifted from orchestrata_ultimate.harness.history
history_tracker 17
hop_diagnostic 20
incus_isolation 120
isolation Residue-only lift of frozen harness.isolation (ADR-0103 P2-F). 24 lifted from orchestrata_ultimate.harness.isolation
isolation_manager 22
j_s_o_n_history_tracker 136
models 40 lifted from orchestrata_ultimate.harness.models
reporting Residue-only lift of frozen harness.reporting (ADR-0103 P2-F). 354 lifted from orchestrata_ultimate.harness.reporting
runner Harness runner — whole-module re-export (clean-target split). 5 lifted from orchestrata_ultimate.harness.runner
scenario Scenario loading, validation, and setup hook execution. 245 lifted from orchestrata_ultimate.harness.scenario
scenario_result 23
security_error 10
self_healing_loop 479
snapshot_manager 191
snapshots 4 lifted from orchestrata_ultimate.harness.snapshots
trend_queries 43
trial_result 22
worktree_isolation 115

Dependency diagram

Package dependency diagram orchestrata_volks.harness