Orchestrata Ultimate › Tests › Reference › scripts › test_benchmark_code_intel
Tests reference — tests/scripts/test_benchmark_code_intel.py
Module of tests/scripts, split out because the directory page would be too large.
Prose pending.
tests/scripts/test_benchmark_code_intel.py
source · 25 tests · 311 lines
Tests for scripts/benchmark_code_intel.py — the Step 3.4 SLO harness.
| Kind | Name | Line | Description |
|---|---|---|---|
| function | test_plan_scans_flags_a_whole_graph_scan |
78 | |
| function | test_plan_scans_flags_a_label_scan |
82 | |
| function | test_plan_scans_accepts_an_index_lookup |
86 | ScanAllByLabelProperties is the INDEX operator — its name merely |
| function | test_plan_scans_is_false_for_an_empty_plan |
92 | |
| function | test_every_slo_has_a_budget_and_a_reason |
101 | |
| function | test_slo_ids_are_unique |
107 | |
| function | test_the_plan_ratified_budgets_are_the_ones_declared |
112 | The numbers come from the migration plan, not from what the graph |
| function | test_only_the_deliberate_scan_slo_waives_the_index_gate |
123 | |
| function | test_whole_graph_passes_are_measured_once_not_as_a_p99 |
132 | Repeating a whole-graph MAGE pass would measure the result cache. |
| function | test_repeated_slo_is_judged_on_p99_not_max |
157 | |
| function | test_one_shot_slo_is_judged_on_its_single_duration |
163 | |
| function | test_over_budget_fails_and_says_by_how_much |
170 | |
| function | test_a_fast_query_that_scans_still_fails |
176 | The whole point of the plan gate: fast today because the graph is |
| function | test_a_scan_is_allowed_where_the_slo_waives_the_gate |
185 | |
| function | test_report_payload_is_json_serializable_and_counts_passes |
191 | |
| function | test_resolve_prefers_the_explicit_override |
206 | |
| function | test_resolve_prefers_the_cutover_key_when_it_is_set |
210 | Phase 4.2 fills KG_ROUTING_URI; the harness must follow it there |
| function | test_resolve_falls_back_while_the_cutover_key_is_still_blank |
217 | |
| function | test_resolve_treats_changeme_sentinels_as_unset |
222 | An unsubstituted .env template leaves CHANGEME_* values that are NOT |
| function | test_resolve_raises_rather_than_guessing_an_address |
231 | |
| function | test_sample_ids_returns_real_canonical_ids |
241 | Hardcoded ids would let the benchmark measure lookups that match |
| function | test_method_lookup_by_canonical_id_uses_an_index |
258 | Regression for the gap this step found: Method is out of |
| function | test_every_index_gated_slo_plans_an_index_lookup |
268 | |
| function | test_measure_returns_real_timings_for_a_live_slo |
277 | |
| function | test_main_measures_every_slo_and_writes_a_report |
288 | End to end against the live cluster. Few iterations: this asserts the |