Orchestrata Ultimate › Plans › spec-audit › reverify-2026-10-18 › I-tools

Cluster I — Tools suite re-verification (061–072)

Source: plans/spec-audit/reverify-2026-10-18/I-tools.md at 8a2cb42ad · last changed 2026-10-18 · first committed 2026-10-18 · 327 lines.

Prose pending.

Plan text

Cluster I — Tools suite re-verification (061–072)

Read-only re-audit against CURRENT code on branch components (HEAD de9e996f2), 337 commits past the session-195 audit. Method: AST toolkit only for Python source; checklists/requirements.md boxes ignored (stale); every claim has file:line from serialize_codebase_ast.py or the AST-derived docs/CODEBASE_MAP_FULL.md skeleton.

Headline finding

Session 195 classed 063–072 as all GAP (greenfield). That verdict is now obsolete. The "tool wave" landed the entire tool suite. The unified framework (K2 — BaseTool, ToolMetadata, ToolRegistry) is built and wired, and 11 of the 15 tools SPEC-062 enumerates are implemented and registered through one central site agents/tool_registry.py:2061 register_native_tools. Two specs are ARCHIVED by ruling (066 PowerShell, 070 Team — session 227, plans/tool-wave-plan.md), and three tools remain PARTIAL (065 LSP, 068 ScheduleCron, 071 MCP). 061 (CheetahClaws Volk) is PARTIAL (a real, gateway-wired integration exists but lacks checkpoint/rollback).

SPEC-061 cheetahclaws-volk — PARTIAL

• Specifies: CheetahClaws TUI volk (U:Cd) — terminal code editing, checkpoint/rollback, CXP multi-agent coordination, permission modes ask/bypass/strict, OpenRouter provider.
• Verified:
  • UltimateTUI ✅ (renamed from spec's CheetahClawsTUI) — integrations/cheetahclaws/__init__.py:332; production-wired: gateway/handlers/ultimatetui.py:34 imports it. Methods: handle_edit(file_path, old_string, new_string, strict) (350), handle_intent (431), route_command (457), _handle_write (497), _handle_model (514).
  • MultiAgentCoordinator ✅ — __init__.py:99; dispatch_review (144) routes edits to agents via CXP, _build_edit_prompt (194), _parse_agent_response (212).
  • GovernanceBridge ✅ — __init__.py:263; can_edit (283), _is_binary (305), _contains_secrets (315) → REQ-061-017/018/019 path/binary/secret guards.
  • EditRequest/EditResult/AgentVerdict/AgentFeedback ✅ — 73/83/52/62.
  • Checkpoint/rollback ❌ — no create_checkpoint/rewind/rollback anywhere (symbols --pattern Checkpoint → 0 in this module; REQ-061-005/010/011 unmet).
  • query_memory/search_files/execute_command/_load_commands ❌ — absent (REQ-061-037 unmet).
  • CheetahClawsConfig + permission_mode enum ❌ — no ask/bypass/strict config object.
  • CodeDirectorVolk (the U:Cd agent) ✅ — agents/volks/codedirector/volk.py:53, distinct from the CheetahClaws TUI integration.
• Remaining work / plan:
  • Scope: checkpoint/rollback subsystem (create/restore file snapshots with persisted metadata) + the missing TUI commands (query_memory, search_files, execute_command, _load_commands) + a CheetahClawsConfig/permission-mode object. Confirm whether the spec's "TUI launch" (FR-001) is satisfied by the gateway handler or deferred.
  • Target files: src/orchestrata_ultimate/integrations/cheetahclaws/__init__.py (extend); gateway/handlers/ultimatetui.py.
  • Approach: add a checkpoint store keyed by session, then the command set, then config object.
  • Dependencies: none external; session/workspace state for snapshots.
  • Effort: M. Risk: low — additive; the multi-agent + governance core already exists.

SPEC-062 cheetahclaws-tools-gap — DONE (framework) / roadmap residual under 066/070/071

• Specifies: parent gap-analysis + the unified tool framework (Tool Definition Schema, category taxonomy, BaseTool, ToolRegistry) + a 15-tool roadmap.
• Verified:
  • ToolMetadata ✅ — tools/base.py:101 (frozen dataclass: name/description/category/ parameters/priority/requires_approval/governance_article/estimated_hours/dependencies/ tags/version) → REQ-062-006 schema.
  • ToolCategory ✅ — tools/base.py:72 (file/shell/search/web/task/memory/planning/agent/ config/mcp/utility) → REQ-062-006 taxonomy.
  • ToolPriority ✅ — tools/base.py:88 (P0–P3).
  • BaseTool(ABC) ✅ — tools/base.py:136; ToolResult (29), Tool(Protocol) (47), FunctionToolAdapter (174) bridges legacy function-tools.
  • ToolRegistry ✅ — tools/registry.py:43 (register/unregister/get/has/names/list/metadata/ to_function_schemas/execute with fails-closed ApprovalGate); default_registry (192), register_tool (197).
  • Wiring ✅ — agents/tool_registry.py:2061 register_native_tools registers all wave tools; agents/tool_registry.py:40 imports ToolRegistry.
  • Roadmap: 11/15 implemented; PowerShellTool (066) + TeamCreate/Delete (070) ARCHIVED; McpAuthTool open (071).
• Remaining work / plan: none for the framework (K2 done, b1b9380b). Residual roadmap items are tracked under 066/070 (archived) and 071 (McpAuth gap).
• Effort: — (framework). Risk: n/a.

SPEC-063 repl-tool — DONE

• Specifies: multi-language (Python/JS/Bash) REPL with persistent sessions, resource limits, sandbox modes, malicious-code detection; tool id repl_execute; optional REST/WS/CLI.
• Verified:
  • REPLTool(BaseTool) ✅ — tools/repl_tool.py:692; metadata (710), execute (787); registered in register_native_tools (tool_registry.py:2107). requires_approval=True (registry.py:151-159 docstring names repl as one of two approval tools).
  • REPLSession ✅ — 193 (expression_count, touch, is_idle_expired idle timeout, to_info_dict).
  • Language backends ✅ — _PythonRunner (475), _JavaScriptRunner (516), _BashRunner (550), _runner_for (644). Stateful subprocess-per-session (_RunningProcess 286, _LineReader 254).
  • Resource limits ✅ — ResourceLimits (150), _enforce_output_limit (666), _ExecutionTimeout (141).
  • Malicious-code detection + sandbox mode ✅ — _check_malicious(code, language, sandbox_mode) (652); MaliciousCodeDetectedError (137); ConcurrentSessionLimitError (127).
  • Error taxonomy ✅ — 9 REPLError subclasses (101–137) + _ExecutionTimeout.
  • REST/WS/CLI ❌ (optional per prior audit) — no FastAPI/websocket/CLI module in tools/; the integration surface is the registry tool. Not a blocker.
• Remaining work / plan: none required. REST/WS/CLI (FR-021..032) remain unbuilt if that surface is ever wanted; otherwise scope-complete.
• Effort: —. Risk: n/a (was L/med in session 195).

SPEC-064 git-worktree-tools — DONE

• Specifies: EnterWorktreeTool + ExitWorktreeTool (enter existing / create + branch, path validation, branch protection, uncommitted-change detection, CWD context).
• Verified:
  • EnterWorktreeTool(BaseTool) ✅ — tools/worktree_tools.py:445; _enter_existing (602), _create (654) runs git worktree add via _run_git (184).
  • ExitWorktreeTool(BaseTool) ✅ — 786; _anchor_is_main idempotence (967).
  • WorktreeInfo ✅ — 110; _SessionWorktreeState (129) stores/restores previous CWD.
  • Path validation ✅ — _resolve_path (335), _check_bounds (360), _validate_branch_name (378).
  • Branch protection ✅ — _is_protected (395); uncommitted-change detection _uncommitted_changes (310).
  • Registered ✅ — both in register_native_tools (tool_registry.py:2108-2109).
  • ListWorktreesTool ⚠️ — no BaseTool class (symbols --pattern ListWorktrees → 0); the capability exists as module helper list_worktrees (267) + _split_porcelain_blocks (204), used internally. US-004/FR-044..048 not a registered tool.
• Remaining work / plan: optional — promote list_worktrees (267) to a ListWorktreesTool if US-004 needs first-class exposure.
• Effort: — (S to add ListWorktreesTool). Risk: n/a.

SPEC-065 lsp-tool — PARTIAL

• Specifies: full multi-language LSP over JSON-RPC stdio (completion, hover, diagnostics, goto-definition, references, code actions, formatting, rename) with server lifecycle.
• Verified:
  • LSPTool(BaseTool) ✅ — tools/lsp_tool.py:774; execute (822) dispatches a fixed action set: definition/references/diagnostics/hover/document_symbols.
  • PythonSymbolIndex ✅ — 438 (find_definition 468, find_references 498, document_symbols 523, hover 546) — AST-based, Python-only (_ensure_python_file 277).
  • PyrightDiagnosticsBackend ✅ — 601 (spawns pyright subprocess for diagnostics).
  • Models ✅ — Position/Range/LSPLocation/LSPHover/LSPDiagnostic(s)/DocumentSymbolInfo (129–227).
  • Registered ✅ — tool_registry.py:2110.
  • Multi-language ❌ — Python only; no typescript/rust/go/cpp servers (spec §Dependencies list).
  • Completion ❌, code actions ❌, formatting ❌, rename ❌, goto type-definition ❌, signature help ❌, JSON-RPC stdio client + server lifecycle ❌ — none present.
  • DIVERGENCE: real JSON-RPC LSP client (spec) → hand-rolled AST index + pyright subprocess.
• Remaining work / plan:
  • Scope: either (a) generalize to a real JSON-RPC LSP client (pyright/ruff/etc.) with completion/code-actions/rename/format, or (b) formally rescope to "Python LSP-lite" (definition/references/hover/diagnostics/document-symbols) and archive the rest.
  • Target files: tools/lsp_tool.py (+ new tools/lsp/jsonrpc.py if (a)).
  • Approach: decision-first. (a) is L; (b) is a doc/spec update.
  • Dependencies: external LSP server binaries (pyright already used).
  • Effort: M–L. Risk: med — largest real gap; needs a rescope decision.

SPEC-066 powershell-tool — ARCHIVED (GAP, out of scope)

• Specifies: sandboxed pwsh execution (allowlist, timeout, sanitization, binary detection).
• Verified:
  • PowerShellTool ❌ absent — symbols --pattern PowerShell → 0, --pattern owerShell → 0, --pattern pwsh → 0.
  • Spec self-archived 2026-07-12 (session 227, plans/tool-wave-plan.md): Windows-only target; fleet is Linux/CachyOS+Incus. Shell coverage = run_shell volk tool + SPEC-063 bash REPL.
• Remaining work / plan: none — archived by ruling. Revisit only if a Windows host enters fleet.
• Effort: —. Risk: n/a (was M/med in session 195).

SPEC-067 remote-trigger-tool — DONE

• Specifies: HTTP(S) webhook/API trigger — auth (API key/Bearer/Basic/OAuth2/HMAC), retry w/ backoff, response validation/transform, TLS verify, webhook signature verify, SSRF guard.
• Verified:
  • RemoteTriggerTool(BaseTool) ✅ — tools/remote_trigger_tool.py:1037; execute (1056), execute_simple (1065), verify_signature (1112), parse_response (1119).
  • Auth ✅ — AuthConfig (157), AuthManager.apply (717), _fetch_oauth2_token (755), _hmac_digest (466), _verify_hmac (478).
  • Retry ✅ — RetryConfig (221), RetryExecutor.run (823), _compute_delay (809), _parse_retry_after (799).
  • Validation ✅ — ValidationConfig (240), ResponseValidator.validate (874), _validate_schema (601), _extract_fields (554).
  • SSRF ✅ — SSRFGuard (658), _is_blocked_address (637), _sanitize_url (494).
  • Registered ✅ — register_remote_trigger_tool (1275); requires_approval=True.
  • DIVERGENCE (spec-vs-spec, unchanged): SPEC-062 FR-006 describes SSH remote exec; SPEC-067 is HTTP. Code follows 067 (HTTP/httpx). 062's SSH framing remains unreconciled.
• Remaining work / plan: none for 067. Reconcile 062↔067 mechanism note only.
• Effort: —. Risk: n/a (was M/low-med).

SPEC-068 schedule-cron-tool — PARTIAL

• Specifies: cron scheduling w/ timezone, SQLite persistence, async background scheduler, missed-job recovery, distributed-lock concurrency protection, retry.
• Verified:
  • ScheduleCronTool(BaseTool) ✅ — tools/schedule_cron_tool.py:799; schedule_job (877), unschedule_job (959), update_job (978), list_jobs (1040), get_job_status (1057), pause_job/resume_job (1077/1082), get_next_runs (1115), validate_cron_expression (1139), run_job_now (1218), cancel_job (1324), get_execution_history (1357), clear_history (1376).
  • Cron parsing ✅ — CronExpression.parse (358) hand-rolled (5/6-field, * , - / L W # via _parse_field 287 + _resolve_token 277, named months/weekdays), next_fire_time tz-aware (403).
  • Timezone ✅ — next_fire_time(..., tz: ZoneInfo) (403).
  • Persistence ⚠️ DIVERGENCE — ScheduleCronStore (511) uses redis_asyncio.Redis (spec says SQLite; store has _lock_key 546). Spec also permits Redis for locking, but FR-003 "SQLite database" is unmet; no croniter/aiosqlite dep (hand-rolled parser).
  • Distributed lock ✅ — acquire_lock (665), release_lock (672), is_locked (676).
  • Models ✅ — MissedJobPolicy (170), ConcurrencyPolicy (180), JobStatus (190), CronJob (210), JobExecution (235).
  • Registered ✅ — register_schedule_cron_tool (1431).
  • Background scheduler loop ❌ — no automatic fire-at-cron-time (no scheduler task/loop); only manual run_job_now. FR-006 "execute at scheduled times" unmet.
  • Missed-job recovery ❌ — MissedJobPolicy enum + miss_policy field exist, but no detect-on-startup/recover executor (grep recover|missed|scheduler in module → only the enum).
• Remaining work / plan:
  • Scope: a background scheduler loop (asyncio task that fires due jobs) + missed-job recovery executor honoring MissedJobPolicy. Optionally swap store to SQLite per FR-003, or rescope to Redis.
  • Target files: tools/schedule_cron_tool.py (+ tools/cron/scheduler.py).
  • Approach: scheduler first (fire due jobs via next_fire_time), then recovery on startup, then the SQLite-vs-Redis decision.
  • Dependencies: none new (Redis present; hand-rolled cron already in-tree).
  • Effort: M. Risk: med — long-running loop ownership + tz/recovery edge cases (was L/med).

SPEC-069 task-management-tools — DONE

• Specifies: TaskStopTool (graceful/forced, by id/name-pattern/batch, governance-gated) + TaskOutputTool (full/tail/lines/stream, status metrics), over the sibling task subsystem.
• Verified:
  • TaskManager ✅ — tools/task_tools.py:392; create (453), get (501), list (506), update (521), append_output (554), stop (596), stop_by_pattern (744), check_rate_limit (791), output_of (824), status_of (927), purge_expired_output (980), audit_log (441).
  • TaskStopTool(BaseTool) ✅ — 1041; graceful SIGTERM→SIGKILL _signal_and_wait (696), _await_exit (723); TaskStopMode (95), PatternType (129).
  • TaskOutputTool(BaseTool) ✅ — 1208; OutputMode full/tail/lines/stream (108), TaskOutput (312), TaskStatusResult (292), StopOutcome/BatchStopOutcome (328/340).
  • Registered ✅ — register_task_tools (1344).
  • DIVERGENCE: TaskManager is in-memory + subprocess (_TaskRecord 363, subprocess.Popen in create 453), not Redis-backed as spec SEC-REQ-010/T7 implies; no WebSocket streaming (stream mode is stream_position-based, 824). Functionally covers stop/output/status.
• Remaining work / plan: optional — Redis persistence + WebSocket streaming if multi-node task state is required; otherwise scope-complete.
• Effort: —. Risk: n/a (was M–L).

SPEC-070 team-tools — ARCHIVED (GAP, out of scope)

• Specifies: TeamCreate/Delete/Update — team CRUD, RBAC roles, hierarchy, quota, soft/hard delete.
• Verified:
  • Team tools ❌ absent — symbols --pattern TeamTool → 0, --pattern Team → only unrelated tests/kernel/test_cli_main_coverage.py hits; --pattern Teammate → 0.
  • Spec self-archived 2026-07-12 (session 227): generic user/team-CRUD + RBAC boilerplate disjoint from the volk agent model; governance RBAC tracked as a SPEC-007 gap instead.
• Remaining work / plan: none — archived by ruling. Revisit only if multi-tenant human collaboration becomes a product goal.
• Effort: —. Risk: n/a (was L, blocked on auth/JWT in session 195).

SPEC-071 mcp-resource-tools — PARTIAL

• Specifies: ListMcpResourcesTool + ReadMcpResourceTool + McpAuthTool — MCP client, server registry + connection pool, encrypted credential vault, TTL cache, version+partial reads, Prolog/PAM++ governance.
• Verified:
  • ListMcpResourcesTool(BaseTool) ✅ — tools/mcp_resource_tools.py:203; ReadMcpResourceTool ✅ — 357. Partial reads _slice_text/_slice_bytes (134/142). Models MCPResourceInfo (52), MCPServerStatus (73), ListResourcesOutput (93), ReadResourceOutput (110).
  • McpStdioClient ✅ — tools/mcp_stdio_client.py:89; hand-rolled JSON-RPC stdio client: connect (126), close (153), list_resources (169), read_resource (184), _request (205), _notify (216), _send/_receive (223/229). DIVERGENCE: stdio subprocess client, not the spec's network MCP server (mcp:// HTTP + TLS + connection pool).
  • Registered ✅ — both in register_native_tools (tool_registry.py:2111-2112).
  • McpAuthTool ❌ — symbols --pattern McpAuth → 0 (spec FR-009/010/011, REQ-062-024/081).
  • TTL/LRU cache ❌ — no cache class in module (spec FR-004/007, SR-002).
  • Server registry / connection pool ❌ — per-call stdio spawn; no registry class.
  • Encrypted credential vault ❌ — no vault; no auth plumbing.
  • PAM++/Prolog access control ❌ — not in this module.
• Remaining work / plan:
  • Scope: McpAuthTool (credential configure/test/remove/refresh), a TTL+LRU resource cache, an MCP server registry, and governance hooks; decide whether to retain stdio transport or add the network MCP path.
  • Target files: tools/mcp_resource_tools.py, tools/mcp_stdio_client.py (+ tools/mcp/registry.py, cache).
  • Approach: cache + registry first, then McpAuthTool, then governance.
  • Dependencies: none strictly new (mcp SDK optional; stdio client hand-rolled); vault backend.
  • Effort: M. Risk: med — credential/vault + governance pieces under-provisioned.

SPEC-072 tool-search-tool — DONE (capstone no longer blocked)

• Specifies: semantic + keyword(TF-IDF) + fuzzy search over an in-memory index of all registered tools, filters/facets/pagination, suggestions, TTL+LRU cache, index rebuild/incremental, embeddings w/ graceful degradation.
• Verified:
  • ToolSearchTool(BaseTool) ✅ — tools/search.py:1480; ToolSearchEngine (1210) with search (1252), suggest (1393), _apply_filters (1414), _paginate (1432), _facets (1439), _record_history (1465).
  • Index over registry ✅ — ToolIndex(registry: ToolRegistry, ...) (937); rebuild (952), sync (961), remove (1007), stats (1029), save/load disk persistence (1044/1052). FR-011 satisfied.
  • Strategies ✅ — SemanticMatchStrategy (645, optional EmbeddingService + available() → graceful degradation), KeywordMatchStrategy (681, TF-IDF fit/_idf_of), FuzzyMatchStrategy (767), RelevanceRanker (832).
  • Cache ✅ — SearchCache (1085) TTL+LRU (key_for, get/put, invalidate_tool, warm, stats).
  • Parser/validator ✅ — QueryParser (411), QueryValidator.validate/sanitize (499/564).
  • Registered LAST ✅ — register_native_tools (tool_registry.py:2125-2136) registers ToolSearchTool.create(registry) so its corpus covers all prior tools; also register_search_tool (1596).
• Remaining work / plan: none — the session-195 hard blocker (no ToolRegistry) is resolved. Embeddings remain optional/degradable per spec.
• Effort: —. Risk: n/a (was M–L, registry-blocked).

Cluster summary table

Evidence log (commands run + results)

• git rev-parse --abbrev-ref HEAD → components; git log --oneline -5 → HEAD de9e996f2 (wiki tooling wave; branch 337 commits ahead of audit baseline).
• grep -n "tools/" docs/CODEBASE_MAP.md → tools/ now 20 modules; class map lists all 061–072 symbols (lines 1939–2078).
• symbols --pattern PowerShell|owerShell|pwsh → 0 (066 archived). --pattern TeamTool → 0, --pattern Team → only tests/kernel/test_cli_main_coverage.py (070 archived).
• symbols --pattern CheetahClaws|cheetah|claws → 0 by name; but grep integrations/cheetahclaws → UltimateTUI/MultiAgentCoordinator/GovernanceBridge (061 real, renamed).
• symbols --pattern McpAuth → 0; --pattern ListWorktrees → 0 (capability is a helper, not a tool class).
• symbols --pattern ToolRegistry → tools/registry.py:43 + plugin/registries.py:135 ToolRegistryAdapter.
• refs ToolRegistry → def + IMP in agents/tool_registry.py:40, tools/* (8 modules), plugin, refactor; USE in agents/tool_registry.py (11 sites). Framework wired.
• source tools.base → full framework (ToolResult/Tool/ToolCategory/ToolPriority/ToolMetadata/ BaseTool/FunctionToolAdapter), lines 29–196.
• source tools.registry → full registry (register/discover/dispatch, fails-closed ApprovalGate, default_registry/register_tool), lines 1–199.
• source agents.tool_registry.register_native_tools → registers REPL/Enter/Exit/LSP/ListMcp/ReadMcp/ task/remote/cron/context/codebase-query + ToolSearchTool.create(registry) LAST (lines 2061–2137).
• source tools.lsp_tool (ranged 805–860) → LSPTool dispatches exactly definition/references/ diagnostics/hover/document_symbols; _ensure_python_file (Python-only); @traces(spec="065",…).
• grep on docs/CODEBASE_MAP_FULL.md (AST-derived skeleton) per module → full method inventories for repl/worktree/lsp/remote/schedule/task/mcp/search/cheetahclaws (line ranges cited inline above).
• refs UltimateTUI → def integrations/cheetahclaws/__init__.py:332, IMP gateway/handlers/ultimatetui.py:34 (production wiring). refs MultiAgentCoordinator → def + 11 use.
• who-imports integrations.cheetahclaws → 1 site: gateway/handlers/ultimatetui.py:34.
• symbols --pattern BaseTool sanity → framework present (toolkit working).
• Spec 066 + 070 spec.md headers read → both self-archived 2026-07-12 (session 227, plans/tool-wave-plan.md).

(End of file)

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks