Orchestrata Ultimate › Plans › evidence › d1-pipeline-runners
D1 Evidence Pack — PipelineRunner Ownership (kernel vs orchestrator vs workflow_engine)
Source: plans/evidence/d1-pipeline-runners.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-05 · 60 lines.
Prose pending.
Plan text
D1 Evidence Pack — PipelineRunner Ownership (kernel vs orchestrator vs workflow_engine)
Source: auditor agent W3.E1 (2026-07-06). Decision: plan §D, D1. Blocks W3.1/W3.5. Orchestrator note: agent self-reported 6 explicit not-verified gaps (listed at bottom) — spot-check those during W3.1 execution. Trace-count discrepancy resolved: 344 = all trace files, 93 = non-_TEST subset; both correct.
1. Responsibility matrix
| kernel/pipeline.py:83 | orchestrator/pipeline_runner.py:109 | kernel/workflow_engine.py | |
|---|---|---|---|
| Docstring claim | "Sole orchestrator" (FALSE today) | "dynamic agent stages through Postman routing" | generic DAG/step engine |
| Volk dispatch | Static _register_volks() (:146) hardcodes 9 built-ins + plugin adapter (:36-79) | Dynamic PipelineRouter.route(work_type, has_ui)->StageSpec list; every pipeline ends U:Cp | zero volk/CXP/Postman refs |
| Checkpoint | PipelineCheckpoint wired in init; save site implicated at kernel/pipeline_session.py:277 _checkpoint_stage() (not body-confirmed) | Docstring claims Redis checkpoint per stage (lines 272/280) but grep hits ONLY those 2 docstring lines — backlog #38 CONFIRMED: dead promise | n/a |
| Tracing | PipelineEventBus (Redis/UI push), no JSONL | pipeline_trace() at :177 — the ONLY call site in src/ => ALL pipeline_output/traces JSONL came from THIS engine | none |
| Escalation/KG | PipelineEscalationMixin + PipelineKGMixin in class decl | not found in lines 1-160 (absence not exhaustively verified) | n/a |
| Non-test consumers | 3 | 6 incl. gateway | 0 (grep -rl empty) |
2. Consumers
- Kernel engine: kernel/pipeline_cli.py:10,275; scripts/e2e_run.py:145,171 (dogfood); agents/kernel_handler.py::build_kernel_packet_handler -> (a) agents/runtime_main.py CXPServer (out-of-process), (b) kernel/cli_main.py:_boot_agent_runtime (:200-225, monolithic boot).
- Orchestrator engine: orchestrator/pipeline_kernel.py::PipelineKernelClient._run_pipeline (lazy import ~:131) -> gateway/server.py:388-403 as the IN-PROCESS FALLBACK FOR THE WEBSOCKET GATEWAY ("PipelineKernelClient connected as fallback", :401) — the most product-facing live path; session/cli/pipeline.py:56-68 (pipeline run/status/resume); integrations/cheetahclaws/init.py:369 (UltimateTUI); scripts/pipeline_quick_test.py, verify_pipeline.py:52, test_pipeline_smoke.py.
- scripts/benchmark_pipeline.py target: unresolved.
- Trace provenance: 356 JSONL files (263 TEST-prefixed, 93 real: pipeline* x86, exp_thread_* x2, cli_exp_* x2, +3 probes) — ALL emitted by the orchestrator engine.
3. Test evidence
- Orchestrator: tests/orchestrator/test_pipeline_e2e.py (only *_e2e.py pipeline test).
- Kernel: 14 tests/kernel/test_pipeline*.py — all unit/component (checkpoint, escalation, kg, events, cli...). No kernel e2e.
- workflow_engine: 3 unit test files, 0 e2e, 0 product consumers => dead code corroborated.
4. Divergence
- Checkpointing real (kernel) vs documented-but-absent (orchestrator) — #38 confirmed drift.
- Static hardcoded dispatch vs dynamic capability routing — architectural, not a rename.
- Tracing formats structurally incompatible (Redis event bus vs JSONL tracer).
5. Recommendation (auditor)
Bifurcation by deployment mode, no clean winner today. SURVIVOR: orchestrator/ pipeline_runner.py (owns the gateway live path + session CLI + TUI), AFTER absorbing kernel's checkpoint persistence (implement #38 for real), KG-sync, escalation. Kernel's _register_volks stays as a thin bootstrap shim for the out-of-process agent-runtime. Fix kernel's false "Sole orchestrator" docstring immediately. workflow_engine.py: DELETE (992 LOC + its 3 unit-test files; zero product blast radius). Blast radius: kernel-side ~3 call sites + 14 unit-test files whose BEHAVIOR ports over; orchestrator-side additive only.
Agent-declared not-verified gaps
(1) kernel/pipeline_session.py:277 body unread (checkpoint call site inferred); (2) test_pipeline_e2e.py header unopened; (3) orchestrator escalation/KG absence not exhaustively grepped; (4) benchmark_pipeline.py target unresolved; (5) trace-count discrepancy (resolved by orchestrator: 344 total vs 93 non-test); (6) used grep not the AST toolkit for consumer discovery.