Knowledge Wiki Orchestrata Ultimate

Orchestrata Ultimate › Tests › Reference › scripts › test_run_tests

Tests reference — tests/scripts/test_run_tests.py

Module of tests/scripts, split out because the directory page would be too large.

Prose pending.

tests/scripts/test_run_tests.py

source · 75 tests · 1868 lines

Tests for scripts/run_tests.py — THE canonical unified test runner.

Kind Name Line Description
function test_expand_paths_explicit_file 87
function test_expand_paths_directory_rglob 92
function test_expand_paths_nonexistent_kept_as_is 107
function test_expand_paths_dedupes_preserving_order 113
function test_partition_package_paths_groups_by_package 123
function test_partition_package_paths_ignores_root_paths 140
function test_partition_package_paths_requires_package_depth 147
function test_check_infra_success_sets_skip_env 174
function test_check_infra_failure_returns_false 185 A throwaway profile whose graph is a dead address (see the
function test_run_gates_runs_real_static_analysis 208 Real gate subprocesses against a small fixture tree.
function test_run_one_gate_returns_gate_name_passed_and_stdout 238 Direct test of the subprocess wrapper for a single gate.
function test_run_one_gate_reports_failure_when_gate_fails 250 Gate returning non-zero should be reported as failed.
function test_run_suite_no_modules_found 267
function test_run_suite_all_passing 300
function test_run_suite_with_failure_json_keeps_all_modules 323
function test_run_suite_solo_retry_recovers_order_flake 374 One batch (workers=1, multiplier=1) → B fails beside A; the solo
function test_failure_excerpt_keeps_assertions_and_drops_noise 427
function test_failure_excerpt_is_capped 445 A whole log in an append-only NDJSON record helps nobody.
function test_failure_excerpt_keeps_captured_output_sections 451 pytest's Captured stdout/stderr sections carry the diagnostic detail
function test_failure_excerpt_keeps_pytest_timeout_stack_dump 475 The all-threads dump between pytest-timeout's +++ Timeout +++
function test_failure_excerpt_timeout_dump_keeps_the_tail 506 The main thread prints LAST in pytest-timeout's dump — truncation
function test_failure_excerpt_is_empty_when_there_is_no_result 521 Degrade to the old name-only record rather than raising mid-run.
function test_log_flake_recoveries_without_batch_results_still_records 527
function test_log_flake_recoveries_records_the_batch_peers 543 A flake record must name WHO shared the process with the victim.
function test_log_flake_recoveries_omits_peers_when_the_plan_is_unavailable 574 Degrade to a peer-less record rather than failing the run.
function test_strip_cov_args_removes_all_coverage_flags 591 The solo retry must run UNINSTRUMENTED: a --cov retry re-triggers the
function test_run_suite_solo_retry_cap_zero_disables 602
function test_run_suite_solo_retry_cap_exceeded_not_retried 622 Two batch failures against a cap of 1: nothing is retried, and the
function test_main_skip_infra_and_gates_all_passing 663
function test_main_failure_prints_failed_module_detail 688
function test_main_gates_failure_returns_1 708 --skip-infra but not --skip-gates: the "gate failed -> return 1"
function test_main_batch_error_and_stderr_tail_printed 742 A module that outlives the batch timeout AND writes to real stderr:
function test_main_json_output 774
function test_main_debug_flag_sets_log_level_and_pytest_args 797
function test_main_custom_workers_timeout_batch_multiplier_and_pytest_arg 816
function test_main_solo_retry_recovers_and_reports_both_rounds 846 End-to-end through main(): a real order flake fails the batch round,
function test_main_solo_retry_cap_zero_flag_keeps_batch_failure 874 --solo-retry-cap 0 disables the retry: the order flake stays a
function test_resolve_affected_bad_rev_returns_none 920
function test_resolve_affected_head_returns_structured_dict 929 Against the real repo + real HEAD: content depends on the live
function test_resolve_affected_missing_script_returns_none 942
function test_resolve_affected_empty_stdout_reports_no_output_not_invalid_json 963 Regression pin for the authoritative-ship-run defect: ``json.loads
function test_resolve_affected_invalid_json_reports_stdout_excerpt 995 Non-empty, genuinely unparseable stdout is a DIFFERENT failure mode
function test_resolve_affected_success_subset_structured 1076 A deterministic non-escalate resolution: a non-.py change under
function test_main_affected_escalates_through_full_cli 1095 End-to-end through main(): a change to an UNREFERENCED code-adjacent
function test_main_affected_bad_rev_falls_back_to_requested_paths 1178 A bad --affected REV can't resolve -- main() must fall back to
function test_affected_flag_registered_with_optional_rev_in_help 1219 A real subprocess --help invocation (the actual argparse registration
function test_slow_test_candidates_filters_by_threshold_and_recency 1251
function test_slow_test_candidates_separates_known_slow 1273 A durations record carrying the observer-recorded known_slow
function test_slow_test_candidates_missing_file_returns_empty 1301
function test_slow_test_candidates_uses_latest_duration_per_test 1310
function test_slow_test_candidates_sorted_descending 1328
function test_main_json_output_includes_slow_tests_and_affected_keys 1346 The --json report carries slow_tests (Phase 3.3) and affected
function test_main_json_stdout_is_parseable_whole 1366 Under --json, stdout is the JSON document and nothing else.
function test_supersede_module_totals_replaces_failed_attempt_counts 1394 The exact live shape (2026-07-29): a recovered module's rerun attempt
function test_supersede_module_totals_handles_missing_counts 1407 A timeout-killed batch leaves no JUnit entry (old side None); a module
function test_prune_stale_basetemps_removes_old_keeps_new 1424 A dir older than the cutoff is removed and its size counted; a dir
function test_prune_stale_basetemps_skips_non_directory_entries 1448 A stray regular file directly under root (not a pytest basetemp dir)
function test_prune_stale_basetemps_nonexistent_root_returns_zero 1464
function test_main_prunes_stale_basetemps_and_prints_summary 1469 The step-0.5 preflight call site: with a real stale pytest-of-<user>
function test_prune_stale_basetemps_no_stale_dirs_is_a_noop 1511 Only fresh dirs present -> nothing removed, nothing reclaimed.
function test_main_rejects_nonexistent_test_target 1536 An unknown positional target fails BEFORE any infra or gate work.
function test_main_names_the_valueless_flag_trap_in_its_error 1549 The exact invocation that caused the incident is rejected and explained.
function test_main_zero_executed_tests_is_not_a_pass 1573 Zero collected tests fails even when no module reports a failure.
function test_scoped_run_yields_to_current_full_record 1620
function test_scoped_run_overwrites_stale_full_record 1624
function test_scoped_run_overwrites_scoped_or_missing_record 1633
function test_full_run_always_writes 1640
function test_abort_document_yields_to_current_full_record_on_disk 1663
function test_abort_document_replaces_stale_or_scoped_record_on_disk 1678
function test_abort_document_is_written_when_no_record_exists 1690
function test_main_second_invocation_reuses_via_ledger 1708 A second invocation of the SAME module, once the outer report-level
function test_main_mixed_reuse_and_run_folds_totals 1757 One proven module is reused, one new module actually runs -- the
function test_main_force_bypasses_ledger_reuse 1804
function test_affected_all_inert_true_for_pure_non_code 1825 GATE-AFFECTED: every changed file inert -> short-circuit eligible.
function test_affected_all_inert_false_when_any_code_or_empty 1847 A single non-inert row (code/refused/named_full), or an empty set, is not all-inert.