orchestrata_ultimate.providers

Reference › orchestrata_ultimate › providers

Provider Manager — unified LLM provider abstraction.

The providers package (specification 011) hides the operational reality of LLM backends behind one class, ProviderManager. APIs rate-limit, fail and cost money, so the manager picks a healthy, affordable and capable backend for each request and falls back when one fails. The LLM client (llm) is its caller. The hand-written story is providers.

Parts by role:

• Entry point: provider_manager, with ProviderError and BudgetExceededError.
• Selection: registry (known providers, pricing, model lookup) and router (filtering and ranking by cost, quality, latency or round-robin).
• Protection: circuit_breaker and rate_limiter, both able to persist state in Redis.
• Accounting: cost_tracker (pricing and atomic budgets) and health_monitor (passive latency and error metrics).
• Backends: the adapters subpackage.

A request enters ProviderManager.execute; the router filters out providers with open breakers or exhausted rate limits and ranks the rest; the manager checks the budget, calls the adapter, records cost, health and breaker state, and on failure moves down the router's fallback chain. Start reading at provider_manager, then router.

Curated narrative

The hand-written story of this package: orchestrata_ultimate.providers.

16 modules, 2,722 lines of code.

Subpackages

Modules

Dependency diagram