Knowledge Wiki Orchestrata Ultimate

Orchestrata Ultimate › Plans › test_coverage_plan_revised

Revised Test Coverage Plan: 104 Untested Methods

Source: plans/test_coverage_plan_revised.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-05-25 · 288 lines.

Prose pending.

Plan text

Revised Test Coverage Plan: 104 Untested Methods

Generated by orchestrata_tools — deterministic analysis with exact parity to legacy ast_explorer.py

Executive Summary

Gap Analysis by Category

Category Count Strategy Difficulty
Simple getters (get_*, is_*) 15 Direct assertion Easy
Property-like (no args beyond self) 10 State verification Easy
Async I/O methods 46 asyncio.run() + mocks Medium
gRPC/proto generated 8 Integration with test server Hard
Complex business logic 25 Multi-step setup + assertions Medium-Hard

Workstream 1: CXP Layer (23 methods) — HIGH PRIORITY

1.1 gRPC Client/Server (11 methods)

Module: cxp.grpc_client, cxp.grpc_server, cxp.proto.cxp_pb2_grpc

These methods wrap gRPC calls. Testing requires:

Method Test Approach Stub Generated
GRPCClient.disconnect Connect then disconnect, assert closed Yes
GRPCClient.subscribe Subscribe to channel, assert callback called Yes
GRPCClient.acknowledge Send ack, assert packet removed from pending Yes
CXPServicer.send_packet Call via gRPC stub, assert packet queued Yes
CXPServicer.send_packet_stream Stream multiple packets, assert all queued Yes
CXPServicer.receive_packet Receive via gRPC stub, assert packet returned Yes
CXPServicer.subscribe Subscribe, assert stream opened Yes
CXPServicer.acknowledge Acknowledge, assert ack stored Yes
CXPServer.start Start server, assert port bound Yes
CXPServer.serve_forever Start in background, assert running Yes
CXPWebSocketHandler.handle Send WS message, assert handled Yes

Test file: tests/cxp/test_grpc_integration.py

@pytest.fixture
def grpc_server(tmp_path):
    server = grpc.aio.server()
    port = server.add_insecure_port("[::]:0")
    # add servicer
    yield server, port
    server.stop(None)

1.2 WebSocket Server (3 methods)

Module: cxp.ws_server

Method Test Approach
WSServer.start Start, assert port listening
WSServer.serve_forever Start in background task, assert running
CXPWebSocketHandler.handle Connect client, send message, assert handled

Test file: tests/cxp/test_websocket_server.py

1.3 Dispatcher & Session (3 methods)

Module: cxp.dispatcher, cxp.session_adapter

Method Test Approach
CXPDispatcher.get_dispatch_count Dispatch packets, assert count matches
SessionAdapter.is_bound Create adapter, assert is_bound state
VersionChecker.check_compatibility Pass compatible/incompatible versions

Test file: tests/cxp/test_dispatcher.py (add to existing)

1.4 Proto Generated (8 methods)

Module: cxp.proto.cxp_pb2_grpc

These are generated gRPC stub classes. They don't need direct tests — they're tested indirectly via the servicer tests in 1.1. Mark as tested-by-proxy once 1.1 is complete.

Workstream 2: Kernel Performance & Pipeline (20 methods) — HIGH PRIORITY

2.1 PerformanceManager (15 methods)

Module: kernel.performance_manager

This is the largest single class gap. PerformanceManager has 15 untested public methods that delegate to internal sub-components.

Method Sub-component Test Approach
get_routing_p95 LatencyTracker Record samples, assert percentile
get_routing_p99 LatencyTracker Record samples, assert percentile
get_policy_p95 LatencyTracker Record samples, assert percentile
get_policy_p99 LatencyTracker Record samples, assert percentile
get_cache_stats LRUCache Insert items, assert hit/miss stats
get_node_for_session ConsistentHash Register nodes, assert consistent mapping
get_node_for_volk ConsistentHash Register nodes, assert consistent mapping
mark_system_down ClusterManager Mark down, assert node excluded
mark_system_up ClusterManager Mark up, assert node included
get_scaling_decisions AutoScaler Set thresholds, assert decisions
should_accept_request BackpressureController Set load, assert accept/reject
update_load BackpressureController Update load, assert state transition
enable_chaos_testing ChaosController Enable, assert flag set
disable_chaos_testing ChaosController Disable, assert flag cleared
add_chaos_fault ChaosController Add fault, assert in fault list

Test file: tests/kernel/test_performance_manager_methods.py

Strategy: Create PerformanceManager with real sub-components (no mocking). Each test exercises one delegation path.

2.2 Pipeline Runner (5 methods)

Module: kernel.pipeline_runner, kernel.pipeline

Method Test Approach
PipelineRunner.submit_intent Submit intent, assert queued
PipelineRunner.respond Send response, assert delivered
PipelineEscalationMixin.approve Approve escalation, assert state changed
ConnectionPool.get_stats Create pool, assert stats dict
BatchProcessor.add Add items, assert batch flushed

Test file: tests/kernel/test_pipeline_runner.py (add to existing)

Workstream 3: Plugin System (14 methods) — MEDIUM PRIORITY

3.1 PluginSDK (7 methods)

Module: plugin.sdk

Method Test Approach
load Load plugin, assert registered
unload Load then unload, assert removed
reload Load, modify, reload, assert updated
enable Enable disabled plugin, assert active
disable Disable enabled plugin, assert inactive
install Install from path, assert loaded
uninstall Uninstall, assert removed from filesystem

Test file: tests/plugin/test_sdk.py

3.2 PluginManager (2 methods)

Module: plugin.manager

Method Test Approach
unload_plugin Load then unload by name
reload_plugin Load, modify config, reload

Test file: tests/plugin/test_manager.py (add to existing)

3.3 PluginContext (5 methods)

Module: plugin.context

Method Test Approach
register_volk Register, assert in registry
register_tool Register tool, assert callable
register_channel Register channel, assert available
subscribe_event Subscribe, emit, assert callback called
emit_event Emit with subscribers, assert all called

Test file: tests/plugin/test_context.py

Workstream 4: Knowledge Graph (6 methods) — MEDIUM PRIORITY

4.1 Neo4jSyncer (6 methods)

Module: knowledge_graph.sync

Method Test Approach
connect Connect to Neo4j, assert driver created
close Connect then close, assert driver closed
full_sync Sync AST, assert nodes created in Neo4j
map_classes Map class pair, assert relationship created
get_stats After sync, assert node/edge counts
verify After sync, assert all expected nodes exist

Test file: tests/knowledge_graph/test_sync.py

Requirements: Neo4j container running (configured via .env). Use test_id prefix for nodes.

Workstream 5: Orchestrator (6 methods) — MEDIUM PRIORITY

5.1 PrologWorker / PrologProxy (6 methods)

Module: orchestrator.prolog_worker

Method Test Approach
PrologWorker.process Assert consults Prolog, returns bindings
PrologWorker.query Query with known fact, assert result
PrologWorker.assert_fact Assert fact, query back
PrologProxy.query Proxy to worker, assert same result
PrologProxy.assert_fact Proxy assert, worker verifies
RedisStateManager.save Save state, assert in Redis

Test file: tests/orchestrator/test_prolog_worker.py

Workstream 6: Providers (4 methods) — LOW PRIORITY

6.1 LLM Provider Adapters (4 methods)

Module: providers.adapters.*

Method Provider Test Approach
complete ClaudeCLIAdapter Mock HTTP, assert request format
complete DeepSeekAdapter Mock HTTP, assert request format
complete OllamaAdapter Mock HTTP, assert request format
complete OpenAIAdapter Mock HTTP, assert request format

Note: These are I/O adapters. Per project standards, no mocking allowed. These should be tested with:

  1. A local HTTP test server (using http.server or pytest-httpx if available)
  2. OR marked as integration tests requiring real API keys

Decision: Move to tests/integrations/ with pytest.mark.requires_api_key.

Workstream 7: Agents & Misc (5 methods) — LOW PRIORITY

Module Method Test Approach
agents.volks.pm.goal_report GoalReportGenerator.generate Create goal, assert report structure
agents.volks.ux.ux_js_scaffold JSScaffoldBuilder.build_architecture_prompt Pass requirements, assert prompt contains keywords
core.prolog_adapter PrologEngine.consult Consult file, assert predicates available
core.threading.thread_pool CustomFuture.get Submit work, assert result via get()
governance.escalation_handler LogNotificationChannel.send Send notification, assert log output

Test Stub Files Generated

The CombinedTestGenerator produced 44 test files with stubs for all 104 methods.

Location: Generated on-demand via:

python -m orchestrata_tools discover --gaps --test-ast plans/ast-tests.json

Or programmatically:

from orchestrata_tools.test_generator import CombinedTestGenerator
gen = CombinedTestGenerator(index, matcher)
files = gen.generate_for_gaps()

Implementation Order

  1. Week 1: Workstreams 1.3, 1.4, 2.1, 2.2 (CXP + Kernel — highest impact)
  2. Week 2: Workstreams 3, 4, 5 (Plugin + KG + Orchestrator)
  3. Week 3: Workstreams 6, 7 (Providers + Misc)

Verification

After each workstream:

uv run pytest tests/<workstream>/ -n auto --timeout=120
python -m orchestrata_tools discover --coverage --test-ast plans/ast-tests.json

Target: 100% class coverage, >98% method coverage.