Orchestrata Ultimate › Plans › spec-audit › G-gateway-ui

Cluster G — Gateway, UI, TUI & plugins (spec audit + plan)

Source: plans/spec-audit/G-gateway-ui.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-04 · 137 lines.

Prose pending.

Plan text

Cluster G — Gateway, UI, TUI & plugins (spec audit + plan)

Investigation agent G. READ-ONLY audit of src/orchestrata_ultimate/gateway/, orchestrator/, integrations/cheetahclaws/, plugin/, kernel/ CLIs, and the vizint-ui/ web app. Evidence by code only (spec - [ ] markers ignored per _METHOD.md). Builds on docs/spec-gap-register.md (which audited 014, 015, 016).

Specs: 014, 015, 016, 022, 050, 051.

SPEC-014 WebSocket Gateway — PARTIAL (core DONE; auth divergence + HTTP-observability gap)

• Specifies: Real-time control plane — WS server, JSON-RPC 2.0 dispatch, pluggable auth, pub/sub topics, ring-buffer replay, heartbeat, PAM++ authorization, HTTP health/metrics, audit.
• Verified:
  • ✅ WS server + lifecycle — WebSocketGateway gateway/server.py:182 (constructed by gateway/cli.py serve; live per memory sessions 165/193).
  • ✅ JSON-RPC dispatch + full method surface — RPCHandler.dispatch gateway/rpc.py:213; registry gateway/rpc.py:275-373 implements every FR-015–FR-020 method (cxp.*, volk.*, session.*, governance.*, knowledge.*, gateway.*) plus extras (state.*, memory.*, projects.*, orchestrator.intent, tui.*, fireflies.*, tests.run).
  • ✅ Ring buffer + replay (FR-012/US-008) — RedisRingBuffer + GatewayHandler.handle_replay gateway/handlers/gateway.py:205.
  • ✅ Heartbeat (FR-014) — ConnectionManager.update_heartbeat gateway/session.py:124.
  • ✅ Audit trail (FR-024) — AuditWriter logged on every dispatch gateway/rpc.py:238.
  • ✅ Serve CLI (US-010) — gateway/cli.py:44 (orchestrata-gateway serve|version).
  • ⚠️ Auth is a single JWT TokenAuthenticator gateway/auth.py:16, not the spec's pluggable IAuthStrategy shipping StaticTokenAuth/HMACAuth/PrologPolicyAuth (FR-005–008). Grep for IAuthStrategy|StaticTokenAuth|HMACAuth|PrologPolicyAuth → 0 hits. (Confirms register's divergence row.)
  • ⚠️ No IGatewayServer ABC / ComponentType.GATEWAY DI registration (FR-001) — grep IGatewayServer → 0 hits; the class is WebSocketGateway, constructed directly. Structural, not functional.
  • ❌ No HTTP /health /ready /metrics and no Prometheus metrics (FR-021/022, US-009). Grep for /health|/ready|/metrics|prometheus|start_http_server under gateway/ → 0 hits. gateway/config.py has no metrics_port/health_enabled/metrics_enabled fields. (VizIntWebUI gateway/server.py:675 is just an empty WS alias, serves no HTTP.)
• Remaining work / plan:
  • Scope: add an HTTP surface alongside the WS server — /health (200/503), /ready (Redis/Neo4j/Prolog/kernel probes), /metrics (Prometheus). Add metrics_port/health_enabled to GatewayConfig. (Auth-strategy abstraction: recommend update the spec to record the JWT divergence rather than build 3 strategies, unless HMAC/Prolog auth is actually required — surface as a decision.)
  • Target files: gateway/server.py (bind an aiohttp/asgi HTTP listener), gateway/config.py (new fields), new gateway/http_health.py; spec update to specs/014-websocket-gateway/spec.md for the auth divergence.
  • Approach: reuse the existing health-probe pattern already used for the gateway↔engine fallback; export counters the handlers already track (connected_clients, rpc latency, auth success/failure).
  • Dependencies: SPEC-032 concierge/pipeline (kernel readiness the /ready probe reports), Redis/Neo4j/Prolog infra, governance (Prolog auth if the strategy work is chosen). Loose coupling to SPEC-011 provider events (already forwarded).
  • Effort: M (HTTP health/metrics). Risk: low. Auth-strategy build (if chosen) is M/L, med risk (security surface).
• Evidence log: grep -rnE '/health|/ready|/metrics|prometheus|aiohttp' src/…/gateway/ → 0; grep metrics_port|health_enabled gateway/config.py → 0; grep IAuthStrategy|StaticTokenAuth|HMACAuth → 0; rpc.py registry read; symbols WebSocketGateway.

SPEC-015 Plugin SDK — PARTIAL / ORPHANED (full SDK built, disconnected from the running system)

• Specifies: Runtime extension framework — IPluginManager (DI ComponentType.PLUGIN_MANAGER), discovery, validation, load/lifecycle, SDK registration (volk/tool/provider/channel/rules/kg), capability enforcement, PAM++ governance, resource limits, orchestrata plugins CLI, plugins.list RPC, audit.
• Verified:
  • ✅ SDK + manager fully implemented — PluginManager plugin/manager.py:48; IPluginManager ABC plugin/interfaces.py:350+ (discover/list/enable/disable/install/uninstall); module-level install_plugin/uninstall_plugin/enable_plugin/disable_plugin plugin/discovery.py:61/179/219/267; PluginContext, loader, registry, manifest, capability governance, PAM++ predicates all present under plugin/.
  • ❌ The subsystem is orphaned. refs PluginManager → 1 def + 2 imports/uses, all inside plugin/ (plugin/sdk.py). Grep PluginManager\(|discover_plugins|load_plugin|list_plugins outside plugin/ → 0 hits. It is never instantiated at kernel bootstrap.
  • ❌ Not DI-registered (FR-001). Grep PLUGIN_MANAGER across src → 0 hits; ComponentType.PLUGIN_MANAGER does not exist.
  • ❌ No CLI surface (FR-014/015/016). No add_parser("plugins"); the SPEC-000 Click CLI (kernel/cli_main.py:302 cli group) exposes only start/version/health. No orchestrata console-script entry in pyproject.toml either.
  • ❌ No plugins.list gateway RPC (US-007 last criterion). Absent from gateway/rpc.py registry.
• Remaining work / plan:
  • Scope: (1) construct + start PluginManager in the monolithic boot (_boot_monolithic) and register it in the DI container under a new ComponentType.PLUGIN_MANAGER; (2) hang a plugins subcommand group (list/info/install/uninstall/enable/disable) on the SPEC-000 Click CLI, delegating to the ready-made plugin/discovery.py functions + manager.list_plugins(); (3) add a PluginsHandler with plugins.list/plugins.install/… to the gateway RPC registry.
  • Target files: kernel/cli_main.py (new @cli.group() plugins), core/* (add ComponentType.PLUGIN_MANAGER + DI register in boot), new gateway/handlers/plugins.py + wire in gateway/rpc.py.
  • Approach: additive — the manager/discovery/registry code is complete and tested-shaped; this is wiring + thin CLI/RPC adapters, not new logic.
  • Dependencies: SPEC-000 (CLI) — needs the orchestrata Click entry to be a real console-script + a place to add subcommands; SPEC-014 (gateway) for the plugins.* RPC; kernel/DI for ComponentType.
  • Effort: M. Risk: med — starting the plugin manager at boot means real plugin loading/governance runs in the live process (error-isolation matters).
• Evidence log: refs PluginManager; grep PluginManager\(|discover_plugins|PLUGIN_MANAGER outside plugin/ → 0; symbols install_plugin|enable_plugin → discovery.py; cli_main.py cli Click group read; pyproject [project.scripts] read (no orchestrata).

SPEC-016 VizInt Command Center — DONE (confirm register)

• Specifies: Browser command center over the SPEC-014 gateway (JSON-RPC/WS), 10 P1 orchestration gadgets + backend Session/Projects handlers.
• Verified:
  • ✅ All 10 P1 gadgets present — vizint-ui/gadgets/: project-explorer, session-manager, cxp-feed, volk-status, claude-chat, task-board, qa-dashboard, audit-trail, kg-explorer, notification-center (+ extras: orchestrator, debug-panel, calendar, todoist).
  • ✅ Portal + bridge framework present — vizint-ui/lib/{portal,bus,registry,chrome,shared}.js, vizint-ui/bridge/{websocket-bridge,gadget-bridge,state-sync,notification-manager}.js.
  • ✅ FR-027 SessionHandler — gateway/handlers/session.py wired (session.active/list/get/open/close/events/facts.subscribe in rpc.py:300-306).
  • ✅ FR-028 ProjectsHandler — gateway/handlers/projects.py wired (projects.list/get/tasks/taskCreate/taskUpdate rpc.py:317-321).
  • ✅ Pipeline→UI wiring done — orchestrator.intent + state.* RPC live (memory: session 193 SPEC-016 wiring; gRPC health-probe fallback + broadcast delivery).
  • ⚠️ Spec-accuracy nit (not a gap): spec claims 31 gadgets; repo ships 14 (10 P1 + 4). The 21 "personal" gadgets (flashcards/prayer/iqama/etc.) are upstream-portal inventory, not vendored here — the inventory in the spec is aspirational. No functional impact on the orchestration surface.
• Remaining work / plan: none required for classification DONE. Optional: correct the gadget-inventory count in the spec. Individual per-US - [ ] sub-items in the spec (inline task edit, CSV export, cypher autocomplete, etc.) are frontend polish, JS-only, out of the Python audit scope.
• Evidence log: ls vizint-ui/gadgets (14); ls vizint-ui/{lib,bridge}; rpc.py registry rows for session/projects; register row (SPEC-016 clean).

SPEC-022 Terminal User Interface — GAP (Design Phase; essentially unimplemented)

• Specifies: A Textual three-panel app (tui_app.py + panels/ + widgets/ + tui.tcss) launched via orchestrata tui, replacing the pipeline_cli.py REPL; real-time thread/chat/status panels, command palette, approval workflow.
• Verified:
  • ❌ No Textual anywhere. grep -rl 'import textual|from textual|Textual' src/ → 0 files; textual is not in pyproject.toml deps (only rich, click).
  • ❌ No orchestrata tui command and no tui_app.py/panels//widgets//tui.tcss. The only "tui" token is the gateway RPC prefix in gateway/handlers/ultimatetui.py:38 (SPEC-050 browser bridge, unrelated to a terminal UI).
  • ⚠️ The REPL the spec intends to replace still exists — kernel/pipeline_cli.py (basic input()/print() loop over PipelineRunner). Spec status is literally "Design Phase", so this is planned scope, not a regression.
• Remaining work / plan:
  • Scope: build the Textual app per the spec — tui_app.py (3-panel grid: ThreadList | Chat | Status + input + header/footer), panels/{thread_list,chat,status}.py, widgets/{input_area,command_palette}.py, tui.tcss; register an orchestrata tui subcommand; drive it via the TUIKernelInterface (submit_intent/respond/approve/reject/get_status/subscribe) against PipelineRunner/PipelineKernel. Add textual>=0.90 to deps.
  • Target files: new src/orchestrata_ultimate/tui/ package; kernel/cli_main.py (@cli.command() tui); pyproject.toml (textual dep + declare it).
  • Approach: greenfield but well-scoped; reuse existing pipeline_cli command semantics (intent/respond/approve/status) as the kernel adapter; the gateway tui.*/orchestrator.intent RPC path is an alternative backend if a WS-based TUI is preferred over direct kernel calls.
  • Dependencies: SPEC-000 (CLI) for the orchestrata tui subcommand + console entry; kernel PipelineRunner/PipelineKernel (approve/reject surface — overlaps the SPEC-023 goal/approval area); new Textual dependency (must be declared — check-deps gate).
  • Effort: L (>2d, full app). Risk: med — approval workflow couples to governance (Article I) and the human-in-the-loop semantics.
• Evidence log: grep -rl textual src/ → 0; grep 'add_parser(.tui|orchestrata tui' → only ultimatetui.py:38 RPC prefix; grep textual pyproject.toml → 0; pipeline_cli.py head read.

SPEC-050 CheetahClaws Integration — PARTIAL (multi-agent edit wired; checkpoint/40+-commands/live-diff/path-traversal missing)

Register/orchestrator marked "clean", but SPEC-050 was not in the register's audited set (002…032). Code shows real remaining work.

• Specifies: CheetahClaws Rich TUI + 40+ slash commands routed through the kernel, multi-agent (Cd/Qa/Ar/Ux) coordinated edits, PAM++ governance, checkpoint create/restore, multi-provider model switching, live diff, audit.
• Verified:
  • ✅ FR-001 multi-agent edit — MultiAgentCoordinator.dispatch_review integrations/cheetahclaws/__init__.py:119 (parallel Cd/Qa/Ar[/Ux], BLOCK rejects); UltimateTUI.handle_edit:281.
  • ✅ FR-002 governance pre-edit — GovernanceBridge.can_edit:221 (10 MB cap, null-byte binary detect, secret-name + blocked-path patterns).
  • ✅ Live surface wired — UltimateTUIHandler exposes tui.edit/intent/agents/models/model_switch/read gateway/handlers/ultimatetui.py, registered in rpc.py:325-344 and consumed by the VizInt orchestrator gadget.
  • ✅ Intent pipeline (FR-009 partial) — handle_intent:353 → PipelineRunner.run.
  • ❌ FR-003/FR-004 checkpoint create/restore + US-002 NOT wired to the CC surface — no /checkpoint//rewind; grep CheckpointManager under integrations/ + gateway/ → 0 hits. (A real CheckpointManager exists at session/checkpoint.py:21 and kernel/pipeline_checkpoint.py, but it is a separate session subsystem, not exposed as a CC command.)
  • ❌ FR-005 path-traversal (../) not enforced — GovernanceBridge.can_edit checks size/binary/secrets/blocked-list but never rejects ... The EditRequest JSON-schema declares the pattern; the code does not.
  • ⚠️ FR-008 multi-provider model switch is profile-based (tui.model_switch → select_profile), not the spec's per-agent /model switch U:Cd gpt-4o with cost/fallback. Divergence — consistent with the SPEC-028 "profiles-only, no env overrides" design (register).
  • ❌ FR-009 "40+ commands": only ~6 tui.* RPC methods exist (edit/intent/agents/models/model_switch/read); the 40-command catalog is not routed.
  • ❌ FR-012 live diff rendering — absent.
  • 🐞 Dead code + latent bug (surface, don't fix): UltimateTUI.route_command cheetahclaws/__init__.py:379 has 0 refs (refs route_command → 1 def / 0 uses) and would raise AttributeError if ever called — it does asyncio.iscoroutinefunction(handler.__wrapped__) on plain lambdas that have no __wrapped__. Candidate for deletion, or fix if the 40-command router is revived.
• Remaining work / plan:
  • Scope: (1) add tui.checkpoint/tui.rewind/tui.list RPC + CC handlers delegating to the existing session/checkpoint.py CheckpointManager (or the pipeline checkpoint) — capturing files+session+KG per FR-003; (2) enforce .. rejection in GovernanceBridge.can_edit (FR-005); (3) decide the 40-command scope (route what's real, drop the aspirational catalog + delete/fix dead route_command); (4) optionally live-diff (FR-012). Reconcile FR-008 model spec to the profile model or build per-agent switch.
  • Target files: integrations/cheetahclaws/__init__.py (path check, checkpoint methods, remove/repair route_command), gateway/handlers/ultimatetui.py + gateway/rpc.py (new tui.checkpoint/rewind methods), spec update for FR-008/FR-009 scope.
  • Approach: wiring over the existing CheckpointManager + a 4-line path guard is small; the 40-command catalog + live diff is a scope decision, not required for the core edit flow.
  • Dependencies: session subsystem (CheckpointManager, SPEC-024/session), SPEC-014 gateway RPC, SPEC-016 VizInt orchestrator gadget (consumer), SPEC-028 LLM profile design (for FR-008 reconciliation), pipeline (PipelineRunner).
  • Effort: M (checkpoint wiring + path guard + dead-code cleanup). Risk: low–med (checkpoint restore touches files/session/KG atomically).
• Evidence log: read integrations/cheetahclaws/__init__.py + gateway/handlers/ultimatetui.py; refs route_command → 0 uses; symbols Checkpoint (session/checkpoint.py:21, kernel/pipeline_checkpoint.py); grep CheckpointManager integrations/ gateway/ → 0.

SPEC-051 VizInt UI — DIVERGENCE (deprecated; superseded by SPEC-016 — no action)

• Specifies: A FastAPI + HTMX + TailwindCSS web dashboard (VizIntWebUI, REST /api/projects, Jinja templates, OAuth2, CSRF). The spec.md header (line 1) itself declares it DEPRECATED — superseded by SPEC-016.
• Verified:
  • ✅ Deprecation is authoritative — specs/051-vizint-ui/spec.md:1 "⚠️ DEPRECATED … superseded by SPEC-016", Status = "Deprecated".
  • ✅ The FastAPI/HTMX app was never built — no htmx/tailwind FastAPI app, no /api/projects REST routes, no app/routes/*, no Jinja templates/. Grep VizIntWebUI|htmx|/api/projects in src → the only VizIntWebUI is an empty WS alias gateway/server.py:675 (class VizIntWebUI(WebSocketGateway): pass), not the HTMX server the spec describes.
  • ✅ The real UI is the SPEC-016 zero-build ES-module vizint-ui/ talking to the SPEC-014 WS gateway (JSON-RPC), an intentionally different architecture than SPEC-051's REST/HTMX.
• Remaining work / plan: none. Action = keep the spec retired (already marked). Do not build the SPEC-051 FastAPI/HTMX stack — it was consciously replaced.
• Evidence log: spec header read; grep VizIntWebUI|htmx|/api/projects src/ → only the empty alias at server.py:675.

Cluster summary table

Cross-cluster dependencies noticed

• SPEC-015 plugin CLI ↔ SPEC-000 CLI, and SPEC-022 orchestrata tui ↔ SPEC-000 CLI: both need a real orchestrata console entry + subcommand hooks on the Click group kernel/cli_main.py:302 (cli). Today pyproject.toml [project.scripts] has no orchestrata entry, and the group exposes only start/version/health. Sequencing: land the SPEC-000 console entry + subcommand pattern first; then 015 and 022 hang off it.
• SPEC-014 gateway ↔ SPEC-032 concierge/pipeline: the gateway /ready probe (015 remaining work) must report kernel/pipeline readiness — the same live volk↔pipeline path resolved in sessions 165/193.
• SPEC-015 plugins ↔ SPEC-014 gateway: plugins.list RPC (US-007) is a gateway handler addition.
• SPEC-016 & SPEC-050 ↔ pipeline: the VizInt orchestrator gadget and the CC tui.* bridge both consume orchestrator.intent/PipelineRunner — any pipeline contract change ripples to both UIs.
• SPEC-050 ↔ session subsystem: checkpoint wiring reuses session/checkpoint.py CheckpointManager (session cluster) rather than building new.
• SPEC-050 FR-008 ↔ SPEC-028: model switching is profile-based per the SPEC-028 "profiles-only" decision — reconcile the SPEC-050 per-agent-model text there, don't add env/per-agent overrides.

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks