Orchestrata Ultimate › Plans › briefs › pcap-cap-tests
P-CAP-TESTS — real tests for the adopted test-phase cap work — brief for one qwen3-coder-next run
Source: plans/briefs/pcap-cap-tests.md at 8a2cb42ad · last changed 2026-10-20 · first committed 2026-10-19 · 119 lines.
Prose pending.
Plan text
P-CAP-TESTS — real tests for the adopted test-phase cap work — brief for one qwen3-coder-next run
Context: the previous session left an uncommitted, ruled implementation in the tree (RUL-038 cap ruling, Mohamed 2026-10-19: the test phase is SELECTED, COUNTED and CAPPED; plus the ADR-0076 graphs-stage change, which is NOT your packet). Mohamed adopted this work 2026-10-19. The implementation ships as-is; your packet closes its DoD gap: ~347 new source lines have ZERO tests. You write them.
Model-routing note (RUL-037): .py-fenced brief on qwen3-coder-next by Mohamed's explicit session
instruction (stated override; M2 not landed). Coordinator verifies before commit. Workers do not commit.
The source under test is DIRTY BY DESIGN — never edit it
Fence: you may edit/create ONLY these test files:
tests/core/threading/test_pytest_live_reader.py (EXTEND)
tests/core/threading/test_live_observer.py (EXTEND)
tests/scripts/test_observe_pytest.py (EXTEND)
tests/scripts/test_run_tests_cap.py (CREATE — focused sibling, pattern: test_run_tests_container_cpu.py)
Source files src/orchestrata_ultimate/core/threading/pytest_live_reader.py, pytest_live_observer.py,
scripts/observe_pytest.py, scripts/run_tests.py contain the adopted uncommitted work. READ them
(live-source AST toolkit works on dirty files: branches/source/symbols with --source <file>),
never write them. If a test exposes a source bug: keep the test asserting the CORRECT behaviour, and
report the bug in your reply (file, function, evidence). Do not fix source, do not weaken the test.
Targets (measured new/changed defs in the dirty diff)
pytest_live_reader.py: read_collection_counts, count_selected, run_progress, format_progress,
class ProgressTicker (__init__, start, line, tick, stop, _loop).
pytest_live_observer.py: multi_file_refusal, pytest_collection_finish, _deadline, _hard_stop,
arm_deadline, pytest_configure (changed).
scripts/observe_pytest.py: _format_progress, _render_event (new collection_finished + capped
branches), _format_session_line (new progress= field).
scripts/run_tests.py: _test_cap_seconds, _run_deadline, _child_run_env (load run_tests.py via the
importlib pattern already used by tests/scripts/test_run_tests*.py — read one first).
Mandated workflow (Mohamed 2026-10-19: branches → gen-tests → real assertions)
For EVERY target function, before writing its tests:
.venv/bin/python scripts/serialize_codebase_ast.py branches --name <func> --source <the source file>
The branch listing IS your test plan: one real test per decision point minimum. Worked example
(already measured): count_selected has 6 branches — if not paths (L356), for enumerate(groups)
(L363), for procs (L394), except subprocess.TimeoutExpired (L397), except ProcessLookupError
(L402), if counts is None or total is None (L408) — a complete set covers all six. Optionally seed
scaffolds with gen-tests --name <func> --source <file>; every scaffold stub must become a real
assertion before you finish. In your reply, include the branches→test mapping table per function.
Test standards (AGENTS.md — non-negotiable)
- Real everything: no mocks/MagicMock/patch/monkeypatch, no Fake/Stub/Dummy, no skips, function-scope
fixtures, no
os.environ[...] =(pass env dicts explicitly to subprocesses — that is allowed and is the T2-compliant way to test env-reading helpers). ProgressTicker/run_progress/read_collection_counts: real ndjson files under tmp_path (or theisolationfixture), real short intervals; join threads in teardown (never leak a ticker)._hard_stop/arm_deadline/_deadline: spawn a REAL.venv/bin/pythonchild that arms a tiny deadline and observe it self-terminate (rc/signal + elapsed); never arm a deadline in the test process itself.count_selected: realpytest --collect-onlysubprocesses against a tiny committed module (tests/core/test_thread_pool.py, 1 test) — cheap, real, no infra.pytest_collection_finish/pytest_configure: drive with a real minimal pytest run in a tmp dir (a 1-test file you write under tmp_path) using the observer plugin via-p/PYTHONPATH the way test_live_observer.py already does — read its existing fixtures first and reuse them.- Follow each target file's EXISTING structure, fixtures and naming (extend, don't reorganize).
Coverage doctrine (session priority ruling, 2026-10-19)
Finishing outranks perfect coverage: 90% is the goal but a second pass. Measure honestly:
--cov=src/orchestrata_ultimate/core/threading/pytest_live_reader.py (etc.) -o addopts="",
report the % per touched source module and, for run_tests.py, the term-missing status of ONLY the
cap-helper lines. Gaps → named list in your reply, no test padding.
VERIFY (run exactly, report real outputs)
.venv/bin/pytest tests/core/threading/test_pytest_live_reader.py tests/core/threading/test_live_observer.py tests/scripts/test_observe_pytest.py tests/scripts/test_run_tests_cap.py -o addopts="" -q
.venv/bin/ruff check tests/core/threading/test_pytest_live_reader.py tests/core/threading/test_live_observer.py tests/scripts/test_observe_pytest.py tests/scripts/test_run_tests_cap.py
.venv/bin/black --check tests/core/threading/test_pytest_live_reader.py tests/core/threading/test_live_observer.py tests/scripts/test_observe_pytest.py tests/scripts/test_run_tests_cap.py
No commits, no git add, no other files, no prod hosts (10.53.1.*), no full-suite runs.
Done
Reply with: the four test files + test counts added per file; the branches→test mapping table; the pytest/ruff/black outputs; coverage numbers per source module (honest); any SOURCE BUG found (file, function, failing evidence) — reported, not fixed; and the named second-pass coverage gaps if any.
REWORK — run 1 was REJECTED on these measured failures (2026-10-20, coordinator verdict). Read before starting; repeating any one fails run 2.
- PRIMARY TARGETS SKIPPED: run 1 wrote ZERO tests for
pytest_live_reader.py(read_collection_counts, count_selected, run_progress, format_progress, ProgressTicker) andpytest_live_observer.py(multi_file_refusal, pytest_collection_finish, _deadline, _hard_stop, arm_deadline, pytest_configure) — the packet's reason to exist. It never createdtests/scripts/test_run_tests_cap.pyand never touchedtests/core/threading/. These are YOUR targets. - WROTE TO FENCED-OUT SOURCE: run 1 edited
scripts/run_tests.py(+50 lines) and claimed edits toscripts/observe_pytest.py. The brief says READ the source, NEVER write it. In your run the source is already complete (the coordinator adopted run 1's helper definitions after review:_test_cap_seconds,_run_deadline,_child_run_envnow exist in run_tests.py) — you TEST them, you do not touch them. Source bug found → report it (file, function, evidence), keep the test asserting correct behaviour. - FALSE ATTRIBUTION: run 1 described the PRE-EXISTING dirty diff of
scripts/observe_pytest.py(a prior session's adopted work, 28 lines, untouched by it) as functions it "added". Report ONLY what you create in THIS run; every file you claim must match your edit-tool calls. - WRONG TEST FILE: run 1 added tests to
tests/scripts/test_run_tests.py(not in the fence). The fence is exactly: EXTENDtests/core/threading/test_pytest_live_reader.py, EXTENDtests/core/threading/test_live_observer.py, EXTENDtests/scripts/test_observe_pytest.py(run 1 already landed 39 green tests there — do NOT rewrite or duplicate them; add only what is missing for the new functions it did not cover), CREATEtests/scripts/test_run_tests_cap.py(for_test_cap_seconds,_run_deadline,_child_run_env— the importlib loading pattern lives in the existing tests/scripts/test_run_tests*.py files). - NO branches→test MAPPING, NO coverage numbers, NO source-bug section in the reply — all mandatory.
Known open design question you must NOT fix, only test around/observe:
_test_cap_secondsdefaults to 120 (worker-invented in run 1; the ruling's real default is pending) — your tests must pass env values explicitly (T2-compliant) and never depend on the default staying 120.