tools package

Reviewed against commit 8a2cb42ad on 2026-10-23.

Reference: generated package page orchestrata_ultimate.tools (20 modules, 13,457 lines of code, of which only seven modules were described when this page was first written); key modules base, registry, search, approval, repl_tool. The package is in the "operations" group of the orchestrata overview; the AST library behind ast_query is orchestrata_ast.

The story

Every volk in orchestrata needs tools — governed capabilities it can invoke mid-conversation, from messaging another volk to validating a Mermaid diagram or searching the codebase. Before this package existed, those tools lived as ad-hoc function-call dictionaries in agents/tool_registry.py, dispatched through a hardcoded if/elif chain. Adding a tool meant editing dispatch code by hand, and nothing in the system could enumerate, describe, or search the tools that existed. The tools package (SPEC-062) replaces that with a single unified framework: one BaseTool contract, a metadata taxonomy, and a ToolRegistry that every first-class tool (SPEC-063…072) registers into dynamically.

The contract lives in base.py. A tool is either a BaseTool subclass or a plain function wrapped by FunctionToolAdapter; either way it carries a frozen ToolMetadata record — a ToolCategory and ToolPriority taxonomy plus a description and a JSON-Schema parameter block. That metadata is what makes a tool describable to an LLM (rendered as an OpenAI/DeepSeek-style function schema by to_function_schema) and discoverable by search. Execution is uniform: every tool implements async execute(arguments) -> ToolResult, so the registry can route any tool without special-casing it. registry.py is the hub — ToolRegistry is a thread-safe index of tool instances keyed by name that supports register, discover, filter, describe, and dispatch. A process-wide singleton comes from default_registry(), and register_tool() is the one-line registrar. When a volk's LLM emits a tool call, dispatch is now a registry lookup rather than a branch in an if/elif ladder.

On top of the registry sits search.py, the SPEC-072 ToolSearchTool — itself a BaseTool, so the ability to find tools is exposed to volks exactly like any other capability. Internally it is composition-over-inheritance: the tool delegates to a ToolSearchEngine that builds a ToolIndex from the registry, validates and sanitizes the query (QueryValidator), parses AND/OR/wildcard expressions (QueryParser), then runs a Strategy-pattern list of MatchStrategy scorers — semantic (embedding cosine similarity, backed by memory.embedding), TF-IDF keyword, and fuzzy — before a RelevanceRanker orders the hits and a SearchCache (built on core.lru_cache) memoizes them. Every result flows back through typed, frozen dataclasses (SearchQuery, SearchResultItem, SearchResponse), and failures never propagate: they surface as a ToolSearchError folded into a failed ToolResult. If the embedding service is unavailable, semantic scoring simply contributes nothing and keyword/fuzzy carry the match — graceful degradation by composition.

The package also hosts a second, quieter concern: the AST query engine (ast_query.py + its CLI) and the ast_pipeline.py serialize-and-compare pipeline. ASTQuery loads the serialized codebase AST (plans/ast-ultimate.json) and answers structural questions programmatically — summaries, stub listings, inheritance trees, porting-coverage by tier — while ast_pipeline.main() (invoked as uv run ast-sync) serializes both this package and the frozen orchestrata monorepo and deep-compares them. This tooling was the authoritative measure of porting progress throughout the migration, and it remains the way to regenerate the AST artifacts the query engine consumes.

Key responsibilities

• Define the canonical tool contract (BaseTool ABC + Tool Protocol) and the metadata taxonomy (ToolMetadata, ToolCategory, ToolPriority) every first-class tool implements.
• Render any tool as an LLM function-calling schema (to_function_schema) so volks can invoke it, and return results as a uniform ToolResult.
• Provide a thread-safe registry (ToolRegistry) for dynamic register / discover / filter / describe / dispatch — replacing the old hardcoded if/elif dispatch.
• Adapt pre-existing plain-function tools into the framework without rewriting them (FunctionToolAdapter).
• Expose tool search (SPEC-072 ToolSearchTool) — semantic / keyword / fuzzy retrieval over all registered tools, as itself just another registered tool.
• Host the first-class tools built on the framework (REPL, LSP, MCP resources, tasks, worktrees, cron scheduling, remote triggers, codebase and context queries, conversation capture) behind a Prolog approval gate (approval.py).
• Offer a programmatic AST query engine (ASTQuery + ast-query CLI) and the serialize/deep-compare pipeline (ast_pipeline) that measured porting progress.

Class diagram

The framework — contract, metadata, registry (base.py + registry.py)

The BaseTool ABC (and its thinner sibling, the Tool Protocol) is the one abstraction boundary. Each tool exposes an immutable ToolMetadata and an async execute; FunctionToolAdapter bridges legacy function-tools in; ToolRegistry indexes instances by name and dispatches through them.

The search engine — ToolSearchTool and its pipeline (search.py)

ToolSearchTool is itself a BaseTool; it composes a ToolSearchEngine that wires together the index, the query validator/parser, a list of MatchStrategy scorers (Strategy pattern), a ranker, and a cache.

The search data model (search.py)

The request/response shapes are all frozen dataclasses. A SearchQuery bundles its SearchFilters and SearchOptions; the engine returns a SearchResponse of ranked SearchResultItems, each wrapping the matched ToolIndexEntry.

The AST query engine (ast_query.py)

ASTQuery loads the serialized codebase AST and answers structural questions, projecting classes and methods into the lightweight ClassInfo / MethodInfo summaries.

Module dependency diagram

Internal module imports within orchestrata_ultimate.tools (generated from the AST, render-verified):

Module inventory

Notes on the principal classes

• base.py defines what it means to be a tool. The metadata taxonomy (ToolCategory / ToolPriority, both StrEnums) is not decorative — it is the substrate that makes tools describable to LLMs and filterable by the SPEC-072 search facets, so every first-class tool carries it. FunctionToolAdapter lets an existing plain executor(name, args) -> ToolResult participate without being rewritten as a class.
• registry.py's ToolRegistry guards its _tools dict with a threading.Lock. register raises ValueError on a duplicate name unless replace=True; execute on an unknown name returns a failed ToolResult rather than raising. default_registry() supplies the process-wide singleton and doubles as the corpus ToolSearchTool indexes.
• search.py's ToolSearchTool.create(registry) is the Factory that wires the whole pipeline: it builds and rebuilds the ToolIndex, fits the keyword strategy's IDF, and warms the cache with each tool's own name (the most likely first lookup). register_search_tool(...) then registers the search tool back into the registry — closing the loop so search is itself discoverable, searchable, and dispatchable. ToolSearchTool.execute never raises; a ToolSearchError becomes a failed ToolResult.
• ast_query.py + ast_pipeline.py are the porting-progress tooling. ASTQuery warns via staleness_warning() when the serialized AST has drifted from the source tree; ast_pipeline.main() (uv run ast-sync) regenerates the AST artifacts and deep-compares this package against the frozen orchestrata monorepo.

Class map (generated)

Generated from the live source AST by scripts/gen-module-class-diagrams.py — do not hand-edit. Abstract classes have a green border; members listed are those defined on the class itself (inherited members excluded). Solid empty-triangle arrows = in-package inheritance; dashed arrows into dashed boxes = external/stdlib/cross-package bases; filled diamond = composition (constructed in __init__); open diamond = aggregation (annotated/injected reference); diamond edge labels name the owning attribute.