Pipeline-Event Convergence — One Taxonomy from Kernel to UI

Status: SHIPPED. Commit bf8cf695 on feat/cxp-direct-transport, deployed to the test-fleet app container (10.53.2.13). Post-deploy Playwright acceptance against the redeployed kernel+gateway pair: 160 passed / 0 failed / 4 skipped (the 4 were filed conditional-infra guards, not dead specs; commit ac9b763f later made those specs self-provisioning — the suite now executes 164/164 with 0 skipped). Certifying gate on the commit: 30,782 / 0 / 0 / 0.

Design digest of plans/pipeline-event-convergence-design.md (+ contract plans/contracts/pipeline-event-convergence.mmd) in the orchestrata-ultimate repo.

The defect — a two-producer fork

Two producers published onto the same UI topic pipeline.<thread_id> with incompatible payload shapes:

• Producer B (ratified): kernel/pipeline_events.py::PipelineEventBus — the SPEC-001 typed taxonomy (event_type, thread_id, volk, data, timestamp, iteration), written to a Redis stream and broadcast by the gateway's pipeline forwarder (gateway/server.py).
• Producer A (ad-hoc): the fallback orchestrator/pipeline_kernel.py:: PipelineKernelClient._emit_event invented its own flat {"event": ...} vocabulary six days after the kernel bus was wired to the same topic. All three UI gadgets (cxp-feed, debug-panel, orchestrator) and the e2e spec were built against the fallback's shape — so UI pipeline progress was silently dead whenever the required kernel (the shape that actually ships) was serving. Producer A also carried a dead Redis fallback stream (pipeline:events) with zero readers — deleted.

The fork was spec-vs-spec: SPEC-003's REQ-003-US-004-102 certified the fallback shape in direct contradiction of SPEC-001's "enum is the source of truth" requirement.

The one-taxonomy design

• 13-member PipelineEventType StrEnum (was 9): added STAGE, PAUSED, RESUMED, CANCELLED. No new START/ERROR members — pipeline.start maps onto INTENT and pipeline.error onto COMPLETE with data.success=False, per the addendum's own glosses.
• PipelineEvent.to_payload() — the ONE shared serializer (exactly asdict), so the REQ-006-032 golden-wire-bytes test pins it. Byte-for-byte wire compat preserved.
• Six build_*_event module factories (build_intent/stage/complete/paused/ resumed/cancelled_event) — both producers construct events through the same factories, making data-shape drift structurally impossible. Four new emit_* bus helpers keep the drift-guard invariant (emitted set == enum set exactly).
• All 10 fallback emission sites in pipeline_kernel.py fold onto the factories; start_workflow's spawned task now emits a failed COMPLETE on an escaping exception (previously swallowed — the thread hung at "running" with no terminal event).
• Forwarder decode casts iteration to int, so both delivery paths (direct broadcast and stream→forwarder) hand gadgets identical types.
• A new shipping-shape proof test (tests/gateway/test_pipeline_forwarder_canonical_shape.py) drives the real bus → Redis stream → production forwarder and asserts the canonical payload — the suite now proves the shape production ships, not the compat shim.

Compat window

_EMIT_LEGACY_COMPAT = True (module flag in orchestrator/pipeline_kernel.py): only the fallback client dual-emits the legacy flat fields, merged additively into the canonical payload, so stale browser tabs keep rendering for exactly one deploy cycle. The kernel bus never emits legacy fields (would break the golden-bytes guard and poison the durable replay streams). Removal = Packet 5, the NEXT deploy cycle — delete _legacy_compat_fields + the flag, gated on pipeline-e2e.spec.js green while asserting canonical-only. Legacy-shape test checks are guarded behind the same flag so they degrade to no-op at Packet 5.

Gadget dispatch contract

All three gadgets now dispatch on payload.event_type (never payload.event) with fields under payload.data.*. Completion failure is pipeline.complete + data.success === false (the bespoke pipeline.error branches are gone); paused/cancelled render as their own hold/terminal states instead of "Running..."; the orchestrator gadget renders data.error when present.

Requirement renumbering

The pipeline-event-type addendum's requirement ids were renumbered REQ-006-010/011/012 → REQ-006-031/032/033: the old ids collided with SPEC-001's pre-existing US-006 workflow-engine requirements, conflating RTM rows for unrelated requirements. All live @traces and citations updated; a renumbering note lives in specs/001-kernel/pipeline-event-type-addendum.md (historical documents retain the old ids and must be read through that note). REQ-003-US-004-102 was rewritten to reference the SPEC-001 taxonomy with a dated supersession note — the cross-spec contradiction is closed.

Driver provenance — resolve on output, never mutate the caller's event

Status: SHIPPED (session 262, 2026-08-12), on top of the convergence above. PipelineEvent gained a driver field (str = "", default unattributed — empty is legal, never an error) so a UI consumer can show which execution produced an event ("inprocess" today; "headless:<harness>" at VolkDriver Phase 2).

The design that shipped supersedes an earlier one that was refuted before any code landed: making driver default to "inprocess" on the dataclass would have made every unattributed producer look attributed, destroying the one observable signal (driver == "" on the wire) that found this gap in the first place — a third, un-audited producer would silently read as "inprocess" forever.

The contract instead: both production producers of the canonical payload carry their own producer identity as a constructor default (driver: str = "inprocess"), and resolve it onto the serialized output, never onto the PipelineEvent object itself:

• kernel/pipeline_events.py::PipelineEventBus.emit() — event_dict = event.to_payload(); event_dict["driver"] = event_dict["driver"] or self._driver. Callbacks still receive the original, unmutated event; only the dict that goes to Redis/the UI stream carries the resolved value.
• orchestrator/pipeline_kernel.py::PipelineKernelClient._emit_event() — the identical pattern: payload["driver"] = payload["driver"] or self._driver.
• to_payload() == asdict(self) is preserved (the REQ-006-032 golden-bytes guard still pins it) — resolution happens strictly after serialization, not inside it.
• A caller-supplied driver is never overridden by either producer.
• orchestrata watch's renderer (kernel/cli_watch.py::render_event_line) shows driver=<value> only when non-empty — exactly why a fabricated non-empty default would have been the wrong fix: it would make every line claim attribution whether or not the field actually meant anything.

The structural guard. A third producer is not caught by a behavioural test — it has to be named to be constructed. Instead, tests/orchestrator/test_pipeline_kernel.py::test_every_production_payload_producer_resolves_driver walks every to_payload() call site under src/ via ast and asserts the set equals exactly {pipeline_events.py::emit, pipeline_kernel.py::_emit_event} — a new production call site changes that set and turns the test red until it either resolves driver too or the list is amended deliberately. Paired with test_cross_producer_driver_attribution_matches_for_same_event and test_no_producer_emits_unattributed_driver_on_the_wire in the same module.

The XSS lesson

Event-derived strings reach innerHTML in gadget renderers. The reviewer wave on this epic caught un-escaped event data; the rule is now: every event-derived string must go through escapeHtml before rendering (e.g. the orchestrator gadget escapes thread.error, stage names, statuses — vizint-ui/gadgets/orchestrator/gadget.js). The security-xss e2e specs (TaskBoard/ProjectExplorer escape proofs) are green against the real gateway.

Repo pointers

• Design: plans/pipeline-event-convergence-design.md · contract: plans/contracts/pipeline-event-convergence.mmd
• Spec: specs/001-kernel/pipeline-event-type-addendum.md (13-member REQ-006-031, amendment + renumbering notes)
• Code: src/orchestrata_ultimate/kernel/pipeline_events.py, src/orchestrata_ultimate/orchestrator/pipeline_kernel.py, src/orchestrata_ultimate/gateway/server.py, vizint-ui/gadgets/*/gadget.js, src/orchestrata_ultimate/kernel/cli_watch.py (driver rendering)
• Driver-provenance tests: tests/orchestrator/test_pipeline_kernel.py (test_emit_event_attributes_driver_when_unset, test_emit_event_preserves_explicit_driver, test_no_producer_emits_unattributed_driver_on_the_wire, test_cross_producer_driver_attribution_matches_for_same_event, test_every_production_payload_producer_resolves_driver), tests/kernel/test_cli_watch.py
• Commit: bf8cf695 (convergence + fix wave); parent epic 9914d662. Driver provenance lands uncommitted on feat/session-262-gate-and-provenance (2026-08-12) — update this pointer once it ships.