llm package

Reviewed against commit 8a2cb42ad on 2026-10-23.

Reference: generated package page orchestrata_ultimate.llm (5 modules, 1,894 lines of code); key modules client, config, _cassette, exceptions. For how the package fits into the whole system, see the orchestrata overview; its transport is the sibling providers package.

The story

Every agent ("volk") in the orchestrata pipeline — the analyst, the architect, the QA reviewer, the junior coder — ultimately needs to think, and thinking means calling a large language model. orchestrata_ultimate.llm exists so that none of those agents has to know which model, which provider, or how to talk to it. It is a small, deliberately thin async facade: a volk calls await call_llm(system_prompt, user_message, volk="U:BA") and gets text back. Everything else — provider selection, retries, adapter wiring, failure semantics — is this package's problem. The public surface is only four names of substance: call_llm, call_reasoner, call_with_tools, and the config helpers around VolkConfig.

The defining design decision lives in config.py: there are no hardcoded provider or model defaults anywhere. The single source of truth is configs/spike-profiles.yaml, and the active profile is chosen by the SPIKE_PROFILE environment variable (falling back to the "default" profile). Profiles map each agent alias to a provider and a model. Until 2026-07-14 the default profile ran everything on the native DeepSeek API; since then default mirrors openrouter-minimax (orchestration volks on MiniMax M3, coding and QA volks on Qwen3-coder-next, both via OpenRouter), because the native DeepSeek account was unfunded. Other profiles in configs/spike-profiles.yaml are deepseek (the old default, still selectable), claude (Anthropic), kimi and qwen (OpenRouter), budget (local ollama), mix (Claude for design-heavy volks, DeepSeek models through OpenRouter for the rest), claude_cli (the only profile the test environment runs, with no default sentinel on purpose), openrouter-cheap, openrouter-minimax and claude-hybrid. The design_authority alias always resolves to the strongest Anthropic model whatever the profile. Two agents are reasoning-heavy — the Architect (U:Ar) and the SeniorDev (U:SD), captured in the REASONING_AGENTS frozenset — and are wired to reasoning-tier models; everyone else gets standard tier. Each profile entry may carry a default sentinel so an unknown or unstamped volk still resolves (SPEC-028 REQ-028-021/028); if even that is missing, the package refuses to guess and raises LLMConfigError — a deliberate fail-loud choice so a misconfigured deployment never silently burns tokens on the wrong model.

When a call arrives, client.py resolves the calling volk's provider and model from the active profile at call time, then routes the request through a lazily-built, process-wide singleton ProviderManager borrowed from the neighboring providers package. On first use the client registers one adapter per supported backend — claude-cli, anthropic, deepseek, openai, openrouter, ollama — loading credentials from a hand-parsed dotenv so registration and credential loading happen exactly once per process. Each request runs inside _execute_with_retry (which also applies the cassette modes described under _cassette.py when LLM_CASSETTE_DIR is set), which classifies transient errors (_is_transient) and retries them with backoff; every request/response pair is recorded via _capture_exchange for observability. Beyond plain call_llm(), the client offers call_reasoner() for reasoning-designated agents and call_with_tools() for the tool-calling loop that powers agentic behavior — the latter takes a full message list and returns the whole ProviderResponse so callers can read .tool_calls as well as .content.

When things go wrong past the retry budget, failures surface through the tiny hierarchy in exceptions.py: LLMClientError as the base and LLMProviderUnavailableError for provider outages (traced to SPEC-028 and SPEC-017 requirements). Callers — the volks and the pipeline runner — catch these to trigger escalation or fallback, and escalate_agent_model() lets the system bump a struggling agent from its standard model up to the provider's reasoning tier for the rest of the session. In the wider architecture, this package sits squarely between agents (its consumers) and providers (its transport), leaning on core.protocols for the request/response types and on core.base.traces for requirement traceability.

Key responsibilities

• Give every volk a single async entry point (call_llm / call_reasoner / call_with_tools) that hides provider and model choice entirely.
• Resolve provider + model per agent alias from configs/spike-profiles.yaml, driven by the SPIKE_PROFILE env var — with zero hardcoded defaults.
• Lazily build and cache a process-wide ProviderManager, registering one adapter per backend (claude-cli, anthropic, deepseek, openai, openrouter, ollama) with dotenv-loaded credentials.
• Retry transient failures with backoff and capture each request/response exchange for observability.
• Inject pre-formatted knowledge-graph context directly into the prompt tokens (not as an optional tool).
• Escalate a struggling agent from standard- to reasoning-tier model at runtime (escalate_agent_model).
• Let tests replay or synthesize exchanges from cassettes (_cassette.py) so the suite never places a billed provider call by accident.
• Fail loud with a stable exception vocabulary (LLMConfigError, LLMClientError, LLMProviderUnavailableError) so callers can escalate or fall back without touching provider SDKs.

Class diagram

The package is intentionally function-first: client.py is a module of coroutines, not a class, and its only real class is the VolkConfig dataclass that holds the resolved profile. The first diagram shows that config core alongside the client facade module and its two external collaborators (ProviderManager, LLMConfigError); the second shows the exception hierarchy. Both are generated from the real source and render-verified.

Module dependency diagram

Internal module imports within orchestrata_ultimate.llm and its immediate collaborators (generated from the AST, render-verified):

Module inventory

client.py

This is the package's engine room and the call path every volk uses. call_llm(system_prompt, user_message, volk=...) resolves the volk's provider and model from config at call time, then dispatches through a lazily-built singleton ProviderManager (_get_provider_manager()), so adapter registration and credential loading happen exactly once per process. When a graph_context string is supplied it is prepended to the user message — knowledge-graph context lives in the tokens, not in an optional tool the model can ignore. The variants call_reasoner() and call_with_tools() share the same plumbing: the former targets reasoning-tier agents, the latter carries a full message list and returns the entire ProviderResponse so the caller can read .tool_calls for the SPEC-103 refactor loop.

Internally the module is a collection of quiet workhorses: dotenv loading (_parse_dotenv_file, _load_dotenv); one builder + registration helper per backend (_build_claude_cli/_register_claude_cli, and the same pattern for anthropic, deepseek, openai, openrouter, ollama), aggregated by _register_all_providers / _register_adapters and described by the private _ProviderSpec NamedTuple; transient-error classification (_is_transient) feeding the retry executor (_execute_with_retry); and _capture_exchange, which records each request/response pair. When retries are exhausted, the failure is mapped onto the exceptions module rather than leaking provider-specific errors upward.

config.py

Where all provider/model policy lives — and pointedly, where no defaults do. The module reads configs/spike-profiles.yaml and exposes the active profile through the VolkConfig dataclass, whose get_provider(agent_alias) and get_model(agent_alias) are the lookups client.py performs on every call. Both fall back to the profile's default sentinel entry when an alias is unknown or None, and raise LLMConfigError only when neither the alias nor a default exists. Profile management is available via the module functions get_active_profile(), list_profiles(), and select_profile(name); escalate_agent_model(agent_alias) consults PROVIDER_MODELS to bump a struggling agent to its provider's reasoning tier in the in-memory singleton. Because selection is purely SPIKE_PROFILE env var → "default" profile, switching the entire agent fleet between vendors is a one-line environment change rather than a code edit.

exceptions.py

A small, traceability-decorated hierarchy that gives the rest of the system a stable failure vocabulary. LLMClientError is the base, raised when a call fails after all retries (tracing REQ-028-054/056); LLMProviderUnavailableError subclasses it for the case where no provider is registered for the requested volk or the whole fallback chain has failed (tracing REQ-028-055 and REQ-017-012/013). LLMConfigError — defined over in config.py — is the configuration-time sibling. Because client.py funnels every terminal failure through these types, volks and the pipeline runner can implement escalation and fallback without knowing anything about the underlying provider SDKs.

Class map (generated)

Generated from the live source AST by scripts/gen-module-class-diagrams.py — do not hand-edit. Abstract classes have a green border; members listed are those defined on the class itself (inherited members excluded). Solid empty-triangle arrows = in-package inheritance; dashed arrows into dashed boxes = external/stdlib/cross-package bases; filled diamond = composition (constructed in __init__); open diamond = aggregation (annotated/injected reference); diamond edge labels name the owning attribute.