Knowledge Wiki Orchestrata Ultimate

Orchestrata Ultimate / Specs / 028-llm-client

028-llm-client: LLM Client Facade

The LLM Client Facade is the single async entry point through which every Volk makes LLM calls in Orchestrata. It routes each request through the ProviderManager using per-volk provider/model assignments, injects Knowledge Graph context into prompts, validates that the LLM used the injected entities, and retries transient errors with exponential backoff.

At a glance

Item Value
Spec directory specs/028-llm-client/
Status line in the spec Implemented
Requirement IDs defined 56
With a @traces citation in src/ 34 (60.7%)
With a @traces citation in tests/ 1 (1.8%)
Tracked files 6
RTM page 028-llm-client

Reading this spec

The LLM Client Facade is the single async entry point through which every Volk makes LLM calls. It routes each request through the Provider Manager of spec 011 using per-Volk provider and model assignments, injects knowledge graph context into prompts, validates that the LLM used the injected entities, and retries transient errors. A lightweight facade keeps provider selection and retry logic out of individual Volk implementations.

The scope is sixteen functional requirements: the single entry point, reasoning tier access, provider configuration and registration, transient error retry, permanent error propagation, knowledge graph context injection, compliance validation, environment variable overrides, the exception hierarchy, Volk resolution, lazy adapter initialization, adapter replacement, compliance metadata, logging and provider lifecycle. Requirement IDs use the REQ-028- prefix.

The key decisions are these. call_llm() is the one entry point and every other function delegates to it. A VolkConfig maps each Volk alias to a provider, a model and a reasoning model, and environment variable overrides take precedence, so Volks never hardcode provider or model choices. The named providers are claude_cli, anthropic, deepseek, openrouter and ollama. Transient errors (HTTP 429, 500, 502, 503, timeouts, connection refusals) are retried with exponential backoff and jitter up to MAX_RETRIES; permanent errors (400, 401, 403, 404, malformed responses) propagate. Graph context is a structured prefix on the first user message, and non-compliant responses are retried up to MAX_VALIDATION_RETRIES before being passed through with a warning. Reasoning-tier tasks (U:Ar and U:SD) use a distinct high-capability model tier.

In the code, the package is orchestrata_ultimate/llm: orchestrata_ultimate/llm/client holds most citations, with orchestrata_ultimate/llm/exceptions, orchestrata_ultimate/llm/config and orchestrata_ultimate/core/infra_config. The per-requirement view is on the RTM page.

The spec's status line reads Implemented. It tracks 6 files and defines 56 requirement IDs. The full text follows below.

Measured coverage is 34 of 56 requirements cited by at least one @traces in src/ (60.7%), spread over 6 modules; the module citing the most of them is orchestrata_ultimate/llm/client (26 requirements). In tests/, 1 requirement is cited by a test-side @traces.

Coverage gaps: 22 requirements have no citing symbol. The largest untraced groups are REQ-028-* (22). Untraced means no @traces names the ID; the behaviour may still exist in code that was never annotated. Test-side citations are thinner than code-side ones (1 against 34).

Requirements

Checkbox is the state recorded in the requirements checklist; Traced by is the count of src/ symbols whose @traces names the ID.

ID Done Requirement Traced by
REQ-028-001 no call_llm is an async function with signature call_llm(volk_id: str, messages: list[dict], *, model: str | None, provider: str | None, temperature: float | None, max_tokens: int | None, graph_context: dict | None, reasoning: bool) -> str none
REQ-028-002 no call_llm resolves the effective provider via get_volk_provider(volk_id) unless provider override is supplied none
REQ-028-003 no call_llm resolves the effective model via get_volk_model(volk_id, reasoning=reasoning) unless model override is supplied 1
REQ-028-004 no call_llm delegates the resolved request to ProviderManager.complete(provider, model, messages) none
REQ-028-005 no call_llm injects graph context into the first user message before dispatching when graph_context is non-empty 1
REQ-028-006 no call_llm validates graph context compliance on the response and retries up to MAX_VALIDATION_RETRIES on failure 10
REQ-028-007 no call_llm returns the plain assistant response text as a str none
REQ-028-008 no call_llm raises LLMClientError for permanent provider errors (HTTP 400, 401, 403, 404) none
REQ-028-009 no call_llm raises LLMProviderUnavailableError after exhausting all retry attempts on transient errors none
REQ-028-010 no call_llm logs each attempt at DEBUG level and each retry at WARNING level with attempt index and backoff delay none
REQ-028-011 no call_reasoner is an async function with the same signature as call_llm minus the reasoning parameter 1
REQ-028-012 no call_reasoner delegates to call_llm with reasoning=True 1
REQ-028-013 no Volks in REASONING_VOLKS calling call_reasoner receive the reasoning_model from their VolkConfig entry 1
REQ-028-014 no Volks not in REASONING_VOLKS calling call_reasoner receive their standard model (no error raised) 1
REQ-028-015 no REASONING_VOLKS is defined as the frozen set {"U:Ar", "U:SD"} 1
REQ-028-016 no VolkConfig is a dataclass with fields provider: str, model: str, reasoning_model: str none
REQ-028-017 no Default assignments are defined for U:Cx → claude/claude-sonnet-4-5 none
REQ-028-018 no Default assignments are defined for U:Ar → claude/claude-opus-4-5 (standard) and claude/claude-opus-4-5 (reasoning) none
REQ-028-019 no Default assignments are defined for J:Junior → claude/claude-sonnet-4-5 none
REQ-028-020 no Default assignments are defined for U:Ox, U:PS, U:Cd, U:Ux, U:PM, U:Qa, U:SD covering all system Volks none
REQ-028-021 no A sentinel default key in VolkConfig map applies to any unrecognised volk_id 8
REQ-028-022 no ORCHESTRATA_LLM_PROVIDER env var globally overrides the provider field for all Volks 1
REQ-028-023 no ORCHESTRATA_LLM_MODEL env var globally overrides the model field for all Volks 1
REQ-028-024 no ORCHESTRATA_LLM_REASONING_MODEL env var globally overrides the reasoning_model field for all Volks none
REQ-028-025 no Per-volk env var overrides (pattern ORCHESTRATA_LLM_MODEL_<VOLK_SANITISED>) take precedence over global overrides none
REQ-028-026 no get_volk_provider(volk_id: str) -> str returns the effective provider after applying env overrides 1
REQ-028-027 no get_volk_model(volk_id: str, reasoning: bool = False) -> str returns reasoning_model when reasoning=True, else model none
REQ-028-028 no Both functions fall back to the default config entry when volk_id is not in the map 4
REQ-028-029 no HTTP status codes 429, 500, 502, 503 are classified as transient errors 1
REQ-028-030 no asyncio.TimeoutError and concurrent.futures.TimeoutError are classified as transient errors 1
REQ-028-031 no ConnectionRefusedError, ConnectionResetError, and aiohttp.ClientConnectorError are classified as transient errors 1
REQ-028-032 no HTTP status codes 400, 401, 403, 404 are classified as permanent errors 1
REQ-028-033 no A helper is_transient(exc) -> bool function encapsulates all transient error detection logic 1
REQ-028-034 no Maximum retry attempts is MAX_RETRIES = 5 (6 total attempts including the first) 2
REQ-028-035 no Backoff delay formula is min(base_delay * 2 ** attempt + random.uniform(0, jitter), max_delay) 2
REQ-028-036 no Default base_delay = 1.0 second, jitter = 0.5 seconds, max_delay = 60.0 seconds 2
REQ-028-037 no On permanent error, the exception is re-raised immediately without any retry 2
REQ-028-038 no After MAX_RETRIES transient failures, LLMProviderUnavailableError is raised with the final underlying exception attached as __cause__ 2
REQ-028-039 no When graph_context is a non-empty dict, its content is serialised to a human-readable block none
REQ-028-040 no The serialised block is prepended to the content of the first message where role == "user" 1
REQ-028-041 no The injected block is delimited by ### Knowledge Graph Context ### header and ### End Graph Context ### footer none
REQ-028-042 no The original user message content follows the injected block verbatim without modification 1
REQ-028-043 no When graph_context is None or an empty dict, no prefix is added and the messages list is unchanged 1
REQ-028-044 no After receiving a response, the facade extracts entity names from graph_context none
REQ-028-045 no Compliance passes if at least one entity name appears (case-insensitive substring match) in the response text none
REQ-028-046 no On compliance failure, a nudge message "Please reference the provided graph context entities in your response." is appended to messages and the call retried none
REQ-028-047 no Maximum validation retries is MAX_VALIDATION_RETRIES = 3 none
REQ-028-048 no After MAX_VALIDATION_RETRIES compliance failures, the last response is returned with compliance_warning=True in call metadata; no exception is raised none
REQ-028-049 no Compliance validation is entirely skipped when graph_context is None or empty 2
REQ-028-050 no ProviderManager is accessed via a module-level get_provider_manager() function returning a singleton 1
REQ-028-051 no Default adapter registration covers claude_cli, anthropic, deepseek, openrouter, ollama 1
REQ-028-052 no Each adapter is lazily initialised on first use (constructor not called at import time) 1
REQ-028-053 no Registering a provider name that already exists silently replaces the previous adapter 1
REQ-028-054 no LLMClientError(Exception) is the base exception for all LLM client failures 1
REQ-028-055 no LLMProviderUnavailableError(LLMClientError) signals exhausted retries or no available provider 1
REQ-028-056 no Both exception classes are importable from the orchestrata_llm package top-level __init__.py 1

Code that traces this spec

Module Symbols Requirements
orchestrata_ultimate/core/infra_config 2 3
orchestrata_ultimate/drivers/deepseek_coder_driver 3 1
orchestrata_ultimate/knowledge_graph/graph_context_validator 9 2
orchestrata_ultimate/llm/client 6 26
orchestrata_ultimate/llm/config 5 2
orchestrata_ultimate/llm/exceptions 2 3

Checklists

checklists/constitutional.md

LLM Client Facade Constitutional Checklist

Security: API Key Handling
Privacy: Prompt and Response Confidentiality
Reliability: Retry Safety
Reliability: Provider Isolation
Correctness: Graph Context Integrity
Performance
Ethical and Responsible AI

checklists/requirements.md

The requirement list above comes from this file; 0 of 56 checkbox items are ticked.

checklists/testing.md

LLM Client Facade Testing Checklist

call_llm Interface Tests
call_reasoner Interface Tests
VolkConfig Tests
get_volk_provider / get_volk_model Tests
Retry Logic: Transient Error Detection Tests
Retry Logic: Backoff and Limits Tests
Graph Context Injection Tests
Graph Context Compliance Validation Tests
Provider Registration Tests
Exception Tests

Other tracked files

Full spec text

Verbatim from specs/028-llm-client/spec.md; headings are pushed down two levels.

SPEC-028: LLM Client Facade

Metadata

Overview

What

The LLM Client Facade is the single async entry point through which every Volk makes LLM calls in Orchestrata. It routes each request through the ProviderManager using per-volk provider/model assignments, injects Knowledge Graph context into prompts, validates that the LLM used the injected entities, and retries transient errors with exponential backoff.

Why
How

Terminology

Term Definition
Volk A named agent role in Orchestrata (e.g. U:Ar, U:Cx, J:Junior)
VolkConfig Data structure that maps a Volk alias to its LLM provider, standard model, and reasoning model
Provider An LLM backend adapter: claude_cli, anthropic, deepseek, openrouter, or ollama
ProviderManager SPEC-011 singleton that owns adapter lifecycle and routes requests to the correct provider
Transient error A recoverable failure: HTTP 429, 500, 502, 503, Timeout, or connection-refused
Permanent error A non-recoverable failure: HTTP 400, 401, 403, 404, or malformed response
Exponential backoff Retry delay strategy: delay = min(base_delay * 2^attempt * jitter, max_delay); caps at a configured maximum
Graph context A structured block of KG entities (classes, functions, relationships) prepended to the first user message
Compliance validation Post-response check that the LLM response references at least one of the injected KG entities
Reasoning tier High-capability model assignment (e.g. claude-opus) reserved for volks in REASONING_VOLKS
REASONING_VOLKS Set of Volk aliases that receive reasoning-tier models: {U:Ar, U:SD}
MAX_RETRIES Maximum number of retry attempts for transient errors (default: 5)
MAX_VALIDATION_RETRIES Maximum retries when LLM response fails graph context compliance check (default: 3)

Data Models

VolkConfig
{
  "type": "object",
  "properties": {
    "volk_alias": {
      "type": "string",
      "description": "Unique identifier for the Volk (e.g., 'U:Ar', 'U:Cx', 'J:Junior')",
      "pattern": "^[A-Z]:[A-Za-z]+$"
    },
    "provider": {
      "type": "string",
      "description": "LLM provider backend identifier",
      "enum": ["claude_cli", "anthropic", "deepseek", "openrouter", "ollama"],
      "default": "anthropic"
    },
    "model": {
      "type": "string",
      "description": "Standard model for this Volk",
      "examples": ["claude-3-5-sonnet-latest", "deepseek-chat", "ollama/llama3.1"]
    },
    "reasoning_model": {
      "type": "string",
      "description": "High-capability model for reasoning tasks",
      "examples": ["claude-3-5-opus-latest", "deepseek-reasoner"]
    }
  },
  "required": ["volk_alias", "provider", "model", "reasoning_model"],
  "additionalProperties": false
}
LLMRequest
{
  "type": "object",
  "properties": {
    "volk_alias": {
      "type": "string",
      "description": "Volk making the request",
      "required": true
    },
    "messages": {
      "type": "array",
      "description": "List of messages in the conversation",
      "items": {
        "type": "object",
        "properties": {
          "role": {
            "type": "string",
            "enum": ["system", "user", "assistant"],
            "required": true
          },
          "content": {
            "type": "string",
            "description": "Message content",
            "required": true
          }
        },
        "required": ["role", "content"]
      },
      "minItems": 1
    },
    "knowledge_graph_context": {
      "type": "array",
      "description": "Entities from knowledge graph to inject",
      "items": {
        "type": "object",
        "properties": {
          "name": {"type": "string"},
          "type": {"type": "string"},
          "description": {"type": "string"},
          "relationships": {
            "type": "array",
            "items": {"type": "string"}
          }
        },
        "required": ["name", "type", "description"]
      }
    },
    "provider_override": {
      "type": "string",
      "description": "Override default provider for this call"
    },
    "model_override": {
      "type": "string",
      "description": "Override default model for this call"
    },
    "reasoning_model_override": {
      "type": "string",
      "description": "Override default reasoning model for this call"
    },
    "max_retries_override": {
      "type": "integer",
      "description": "Override MAX_RETRIES for this call",
      "minimum": 0,
      "maximum": 10
    },
    "metadata": {
      "type": "object",
      "description": "Additional metadata for the request",
      "additionalProperties": true
    }
  },
  "required": ["volk_alias", "messages"]
}
LLMResponse
{
  "type": "object",
  "properties": {
    "content": {
      "type": "string",
      "description": "The assistant's response text",
      "required": true
    },
    "usage": {
      "type": "object",
      "properties": {
        "prompt_tokens": {"type": "integer"},
        "completion_tokens": {"type": "integer"},
        "total_tokens": {"type": "integer"}
      },
      "required": ["prompt_tokens", "completion_tokens", "total_tokens"]
    },
    "model_used": {
      "type": "string",
      "description": "The actual model that generated the response"
    },
    "provider": {
      "type": "string",
      "description": "The provider that handled the request"
    },
    "compliance_verified": {
      "type": "boolean",
      "description": "Whether KG context compliance was verified",
      "default": false
    },
    "kg_entities_referenced": {
      "type": "array",
      "items": {"type": "string"},
      "description": "KG entity names found in the response"
    },
    "retries_attempted": {
      "type": "integer",
      "description": "Number of retry attempts made",
      "minimum": 0
    },
    "metadata": {
      "type": "object",
      "description": "Additional response metadata",
      "additionalProperties": true
    }
  },
  "required": ["content", "usage", "model_used", "provider"]
}
LLMClientError
{
  "type": "object",
  "properties": {
    "error_type": {
      "type": "string",
      "enum": ["TRANSIENT", "PERMANENT", "PROVIDER_UNAVAILABLE", "VALIDATION_FAILED"],
      "required": true
    },
    "message": {
      "type": "string",
      "description": "Human-readable error message",
      "required": true
    },
    "provider": {
      "type": "string",
      "description": "Provider that raised the error"
    },
    "model": {
      "type": "string",
      "description": "Model being used when error occurred"
    },
    "http_status": {
      "type": "integer",
      "description": "HTTP status code if applicable"
    },
    "retry_count": {
      "type": "integer",
      "description": "Number of retries attempted before failure"
    },
    "original_error": {
      "type": "object",
      "description": "Original exception from provider SDK"
    }
  },
  "required": ["error_type", "message"]
}

Functional Requirements

FR-001: Single Entry Point

The LLM Client Facade MUST provide a single async function call_llm() that accepts all LLM requests and routes them through the appropriate provider, model, and retry logic without exposing implementation details to callers.

FR-002: Reasoning Tier Access

The facade MUST provide call_reasoner() function that delegates to call_llm() with automatic selection of the reasoning-tier model based on the Volk's configuration.

FR-003: Provider Configuration

The system MUST support per-Volk provider and model assignments via VolkConfig, with environment variable overrides taking precedence over default assignments.

FR-004: Provider Registration

The ProviderManager MUST maintain a registry of LLM provider adapters, lazily initializing them on first use and supporting registration of new providers at runtime.

FR-005: Transient Error Retry

The facade MUST implement exponential backoff retry for transient errors (HTTP 429, 500-503, timeouts, connection refused) with up to MAX_RETRIES attempts before raising an exception.

FR-006: Permanent Error Propagation

Permanent errors (HTTP 400, 401, 403, 404) MUST be raised immediately without retry attempts.

FR-007: Knowledge Graph Context Injection

When knowledge_graph_context is provided, the facade MUST serialize entities as a structured prefix and prepend them to the first user message with clear delimiters.

FR-008: Compliance Validation

The facade MUST validate that LLM responses reference at least one injected KG entity, retrying up to MAX_VALIDATION_RETRIES times with compliance nudges when validation fails.

FR-009: Environment Variable Overrides

The system MUST support global overrides via ORCHESTRATA_LLM_PROVIDER, ORCHESTRATA_LLM_MODEL, and ORCHESTRATA_LLM_REASONING_MODEL environment variables, with per-Volk overrides taking precedence.

Deliberate divergence (2026-07): The live configuration model was redesigned so that spike-profiles.yaml is the single source of truth for provider/model assignments (see src/orchestrata_ultimate/llm/config.py). The ORCHESTRATA_LLM_* environment variable overrides described above are not evaluated by the current resolver; profile selection is via SPIKE_PROFILE (default profile when unset). This spec section is retained as the original requirement; the operational behavior follows the profile-based model.

FR-010: Exception Hierarchy

The system MUST define LLMClientError as the base exception and LLMProviderUnavailableError as a specialized subclass for provider unavailability.

FR-011: Volk Resolution

The system MUST provide resolve_provider() and resolve_model() functions that return effective (post-override) values for a given Volk.

FR-012: Lazy Adapter Initialization

Provider adapters MUST be lazily initialized on first use, avoiding startup costs for unused providers.

FR-013: Adapter Replacement

Registering a provider with an existing name MUST replace the previous registration atomically.

FR-014: Compliance Metadata

Non-compliant responses MUST be returned with compliance_verified: false and kg_entities_referenced list in the response metadata.

FR-015: Logging

Each retry attempt MUST be logged at WARNING level with attempt number and calculated delay.

FR-016: Provider Lifecycle

Unregistering all providers MUST cause LLMProviderUnavailableError on the next call requiring a provider.

FR-017: No Direct SDK Imports

No Volk package MAY import provider SDKs directly; all LLM access MUST route through call_llm() or call_reasoner().

Error Conditions

ID Condition HTTP Status Handling
ERR-028-001 Rate limit exceeded (429) 429 Retry with exponential backoff up to MAX_RETRIES
ERR-028-002 Provider internal server error (500) 500 Retry with exponential backoff up to MAX_RETRIES
ERR-028-003 Bad gateway (502) 502 Retry with exponential backoff up to MAX_RETRIES
ERR-028-004 Service unavailable (503) 503 Retry with exponential backoff up to MAX_RETRIES
ERR-028-005 Request timeout - Retry with exponential backoff up to MAX_RETRIES
ERR-028-006 Connection refused - Retry with exponential backoff up to MAX_RETRIES
ERR-028-007 Bad request (400) 400 Raise LLMClientError immediately (permanent)
ERR-028-008 Unauthorized (401) 401 Raise LLMClientError immediately (permanent)
ERR-028-009 Forbidden (403) 403 Raise LLMClientError immediately (permanent)
ERR-028-010 Not found (404) 404 Raise LLMClientError immediately (permanent)
ERR-028-011 Maximum retries exceeded - Raise LLMClientError with TRANSIENT error type
ERR-028-012 Provider not registered - Raise LLMProviderUnavailableError
ERR-028-013 Model not available - Raise LLMClientError with provider-specific message
ERR-028-014 Invalid Volk alias - Raise LLMClientError before provider selection
ERR-028-015 Missing messages list - Raise LLMClientError with validation error
ERR-028-016 Malformed response from provider - Retry up to MAX_RETRIES, then raise LLMClientError
ERR-028-017 All providers unregistered - Raise LLMProviderUnavailableError
ERR-028-018 Compliance validation failed - Retry with compliance nudge up to MAX_VALIDATION_RETRIES
ERR-028-019 KG context injection failed - Proceed without context, log WARNING
ERR-028-020 Provider adapter initialization failed - Retry up to MAX_RETRIES, then raise LLMProviderUnavailableError

API Contracts

call_llm()

Signature:

async def call_llm(
    volk_alias: str,
    messages: List[Dict[str, str]],
    knowledge_graph_context: Optional[List[Dict]] = None,
    provider_override: Optional[str] = None,
    model_override: Optional[str] = None,
    reasoning_model_override: Optional[str] = None,
    max_retries_override: Optional[int] = None,
    metadata: Optional[Dict] = None
) -> LLMResponse

Parameters:

Returns:

Raises:

Example:

response = await call_llm(
    volk_alias="U:Cx",
    messages=[
        {"role": "user", "content": "Explain the architecture"}
    ],
    knowledge_graph_context=[
        {
            "name": "ProviderManager",
            "type": "class",
            "description": "Manages LLM provider adapters",
            "relationships": ["implements SPEC-011"]
        }
    ]
)
print(response.content)
call_reasoner()

Signature:

async def call_reasoner(
    volk_alias: str,
    messages: List[Dict[str, str]],
    knowledge_graph_context: Optional[List[Dict]] = None,
    **overrides
) -> LLMResponse

Parameters:

Returns:

Example:

# U:Ar will use reasoning_model from VolkConfig
response = await call_reasoner(
    volk_alias="U:Ar",
    messages=[{"role": "user", "content": "Design a system architecture"}]
)
resolve_provider()

Signature:

def resolve_provider(volk_alias: str) -> str

Parameters:

Returns:

resolve_model()

Signature:

def resolve_model(volk_alias: str, use_reasoning: bool = False) -> str

Parameters:

Returns:

Non-Functional Requirements

Performance
Scalability
Reliability
Observability
Maintainability

Technical Design

Architecture Overview

The LLM Client Facade follows a layered architecture:

┌─────────────────────────────────────┐
│          Volk Implementations         │
│   (U:Ar, U:Cx, J:Junior, etc.)       │
└──────────────┬──────────────────────┘
               │ call_llm() / call_reasoner()
┌──────────────▼──────────────────────┐
│         LLM Client Facade            │
│  • Context injection                 │
│  • Compliance validation            │
│  • Retry logic                      │
└──────────────┬──────────────────────┘
               │
┌──────────────▼──────────────────────┐
│        ProviderManager (SPEC-011)    │
│  • Provider registry                 │
│  • Adapter lifecycle                 │
│  • Load balancing                    │
└──────────────┬──────────────────────┘
               │
    ┌──────────┼──────────┐
    │          │          │
┌───▼───┐  ┌──▼──┐  ┌──▼──┐
│Claude │  │Deep │  │Open │
│Adapter│  │Seek │  │Router│
└───────┘  └─────┘  └─────┘
Provider Adapters

Each provider implements a standard interface:

class LLMProviderAdapter(ABC):
    @abstractmethod
    async def initialize(self) -> None: ...
    
    @abstractmethod
    async def generate(
        self,
        model: str,
        messages: List[Dict],
        **kwargs
    ) -> Dict: ...
    
    @abstractmethod
    async def cleanup(self) -> None: ...

Supported providers:

Retry Strategy

Exponential backoff formula:

delay = min(base_delay * (2 ^ attempt) * jitter, max_delay)

Where:

Knowledge Graph Context Format
<knowledge_graph_context>
<entities>
<entity name="ProviderManager" type="class">
<description>Manages LLM provider adapters</description>
<relationships>implements SPEC-011</relationships>
</entity>
</entities>
</knowledge_graph_context>
Compliance Validation Algorithm
  1. Extract all entity names from knowledge_graph_context
  2. Tokenize response text (case-insensitive)
  3. Check for substring matches of entity names in response
  4. If no matches found after MAX_VALIDATION_RETRIES, mark as non-compliant

Examples

Example 1: Basic LLM Call
from llm_client_facade import call_llm

response = await call_llm(
    volk_alias="U:Cx",
    messages=[
        {"role": "user", "content": "What is the purpose of ProviderManager?"}
    ]
)

print(f"Response: {response.content}")
print(f"Model: {response.model_used}")
print(f"Provider: {response.provider}")
Example 2: Reasoning Tier Call
from llm_client_facade import call_reasoner

# U:Ar automatically uses reasoning-tier model
response = await call_reasoner(
    volk_alias="U:Ar",
    messages=[
        {"role": "user", "content": "Design a microservices architecture"}
    ]
)
Example 3: With Knowledge Graph Context
context = [
    {
        "name": "call_llm",
        "type": "function",
        "description": "Main entry point for LLM requests",
        "relationships": ["used by all Volks", "injects KG context"]
    },
    {
        "name": "ProviderManager",
        "type": "class",
        "description": "Routes requests to provider adapters",
        "relationships": ["implements SPEC-011", "contains registry"]
    }
]

response = await call_llm(
    volk_alias="J:Junior",
    messages=[
        {"role": "user", "content": "How does call_llm integrate with ProviderManager?"}
    ],
    knowledge_graph_context=context
)

print(f"Compliance verified: {response.compliance_verified}")
print(f"Entities referenced: {response.kg_entities_referenced}")
Example 4: Error Handling
try:
    response = await call_llm(
        volk_alias="U:Cx",
        messages=[{"role": "user", "content": "Hello"}]
    )
except LLMClientError as e:
    if e.error_type == "TRANSIENT":
        print(f"Temporary error, retries exhausted: {e.retry_count}")
    elif e.error_type == "PERMANENT":
        print(f"Permanent error: {e.message}")
    elif e.error_type == "PROVIDER_UNAVAILABLE":
        print("No providers available")
Example 5: Environment Variable Overrides
# Override provider globally
export ORCHESTRATA_LLM_PROVIDER=deepseek

# Override model for specific Volk
export ORCHESTRATA_LLM_MODEL_U:AR=claude-3-5-opus-latest

# Override reasoning model globally
export ORCHESTRATA_LLM_REASONING_MODEL=claude-3-5-opus-latest

# Resolves to: provider=deepseek, model=claude-3-5-opus-latest (override)
provider = resolve_provider("U:Cx")
model = resolve_model("U:Cx", use_reasoning=True)
Example 6: Provider Registration
from provider_manager import ProviderManager
from my_custom_provider import MyCustomAdapter

pm = ProviderManager.get_instance()

# Register new provider
pm.register_provider("my_custom", MyCustomAdapter())

# Now available to all Volks
response = await call_llm(
    volk_alias="U:Cx",
    messages=[{"role": "user", "content": "Test"}],
    provider_override="my_custom"
)

Security

Input Validation
Authentication
Authorization
Audit Logging

All LLM calls MUST log:

Example log format:

2026-04-09T10:15:23.456789 INFO [call_llm] volk=U:Cx provider=anthropic model=claude-3-5-sonnet-latest tokens=1250/300 success=true retries=0
Data Privacy
Secure Configuration

Governance

Version Compatibility
Deprecation Policy
Provider Support
Change Management
Review Process

Visual Architecture

Class Diagram

Diagram projects-orchestrata-ultimate-specs-028-llm-client-m67d1b75df9

Unit Tests
Integration Tests
Test Coverage
Quality Gates

Integration Points

SPEC-011: Provider Manager

Integration:

from provider_manager import ProviderManager

pm = ProviderManager.get_instance()
adapter = pm.get_adapter(provider_name)
orchestrata-knowledge: Knowledge Graph

Integration:

from orchestrata_knowledge import KnowledgeGraph

kg = KnowledgeGraph()
entities = kg.get_relevant_entities(query)
orchestrata-core: Base Framework

Integration:

from orchestrata_core import get_config, setup_logging

config = get_config()
logger = setup_logging()
Volk Implementations

Integration:

# In any Volk implementation
from llm_client_facade import call_llm, call_reasoner

response = await call_llm("U:Cx", messages)
Monitoring Systems

Integration:

from orchestrata_core.metrics import Counter, Histogram

llm_requests = Counter("llm_requests_total")
response_time = Histogram("llm_response_time_seconds")

User Stories

US-028-001: Single Entry Point for LLM Calls

As a Volk implementation, I want a single call_llm() async function, So that I never need to know which provider or model I am using and all retry/context logic is handled for me.

Acceptance Criteria:

US-028-002: Reasoning Tier via call_reasoner

As a Volk that performs architectural or strategic reasoning (U:Ar, U:SD), I want a call_reasoner() wrapper that automatically selects the reasoning-tier model, So that I receive the highest-capability model without manually specifying it in every call site.

Acceptance Criteria:

US-028-003: Per-Volk Provider and Model Assignment

As an operator deploying Orchestrata, I want each Volk to have a default provider/model assignment that I can override via environment variables, So that I can switch models for specific Volks in production without changing code.

Acceptance Criteria:

US-028-004: Retry with Exponential Backoff on Transient Errors

As a system operator, I want transient provider errors to be retried automatically with exponential backoff, So that short-lived rate limits or provider blips do not surface as errors to the calling Volk.

Acceptance Criteria:

US-028-005: Knowledge Graph Context Injection

As a Volk performing code-aware reasoning, I want relevant KG entities automatically prepended to my prompt, So that the LLM has grounded, accurate knowledge of the codebase without me manually constructing context blocks.

Acceptance Criteria:

US-028-006: Graph Context Compliance Validation

As a system architect requiring grounded LLM responses, I want the facade to verify that the LLM actually referenced the injected KG entities in its response, So that Volks acting on LLM output can trust it is grounded in the current codebase state.

Acceptance Criteria:

US-028-007: Provider Registration and Adapter Lifecycle

As a developer adding a new LLM provider, I want to register a provider adapter once and have it available system-wide via the ProviderManager, So that adding a new backend requires no changes to call sites.

Acceptance Criteria:

Acceptance Criteria Summary

Success Criteria

Implementation Success
Functional Success
Performance Success
Observability Success
Testing Success

Observability

Logging Requirements
  1. Component Logging
    • Log all major operations with appropriate log levels
    • Include correlation IDs for request tracing
    • Log performance metrics for critical operations
    • Log errors with full context for debugging
Metrics Requirements
  1. Performance Metrics
    • Operation duration histograms
    • Request/error rate counters
    • Resource utilization gauges
    • Business metrics relevant to domain
Tracing Requirements
  1. Distributed Tracing
    • Trace propagation across component boundaries
    • Span attributes for domain-specific context
    • Trace sampling based on operational needs
Alerting Requirements
  1. Critical Alerts
    • Error rate spikes
    • Performance degradation
    • Resource exhaustion
    • Business metric anomalies
Dashboard Requirements
  1. Operational Dashboard
    • Real-time system health
    • Performance trends
    • Error analysis
    • Resource monitoring

Revision History

Version Date Author Changes
1.0.0 2026-04-04 Mohamed Initial specification

(End of file - total 1007 lines)