Continuum-AI-Corp/OrcaRouter-Lite

Self-hosted LLM router with a managed safety net. OpenAI-compatible. BYOK. Single-workspace. Streaming. For more advanced routing choose hosted OrcaRouter

What it solves

OrcaRouter Lite is a self-hosted LLM router that allows users to manage multiple AI providers (like OpenAI, Anthropic, and Google) through a single, OpenAI-compatible API. It eliminates the need to write complex routing logic or manual cost-optimization code in applications by providing a unified interface and an automated way to select the most cost-effective model for a given task.

How it works

The system acts as a proxy server. Users provide their own API keys (BYOK) for various providers, and the server routes requests to the appropriate model. A key feature is the model="auto" setting, which automatically selects the cheapest available model that meets the specific requirements of the request (such as vision capabilities or JSON mode). It also supports native protocol endpoints for Anthropic and Gemini, meaning clients using those specific SDKs can connect directly without needing an OpenAI-compatible wrapper.

Who it’s for

It is designed for developers and teams who want to maintain control over their LLM infrastructure by self-hosting their router, while still benefiting from multi-provider support, automated cost optimization, and a managed fallback to a hosted service if local providers fail.

Highlights

  • Automated Routing: The model="auto" feature picks the cheapest capable model based on request requirements.
  • Multi-Protocol Support: Native ingress for OpenAI, Anthropic, and Gemini wire formats.
  • Cross-Provider Prompt Caching: Reduces costs and latency by caching deterministic requests across different providers.
  • Self-Hosted Dashboard: Includes a built-in dashboard for managing provider keys, routing strategies, and viewing local analytics on spend and latency.
  • Managed Fallback: Can be configured to use a hosted version of OrcaRouter as a safety net for providers the user doesn't manage keys for.

Related

  • Project
    tashfeenahmed/freellmapi

    A unified OpenAI-compatible API gateway that aggregates free tiers from dozens of LLM providers into a single endpoint with smart routing and automatic failover.

  • Project
    mozilla-ai/otari

    Otari is an open‑source, self‑hosted gateway that provides a single OpenAI‑compatible API endpoint for dozens of LLM providers, with built‑in authentication, budget enforcement, routing policies, usage tracking, and optional guard‑rail services.

  • Project
    BlockRunAI/ClawRouter

    An open-source smart LLM router that enables autonomous agents to pay for API usage via USDC micropayments and reduces costs through local, high-speed routing.

  • Project
    yym68686/uni-api

    uni‑api is an open‑source, Docker‑based gateway that lets you expose a single OpenAI‑compatible API while routing requests to many different LLM back‑ends (OpenAI, Anthropic, Gemini, Vertex, Azure, AWS, etc.). It offers YAML‑only configuration, weighted/round‑robin load balancing, automatic retries, per‑model timeouts, rate limiting, moderation, and tool/image support.

  • Project
    weave-os/router

    Weave‑OS Router is an open‑source Go service that acts as a universal proxy for many LLM APIs. It decides per request which model (Anthropic, OpenAI, Gemini, or any OpenAI‑compatible endpoint) is best, forwards the call, and returns the result, while keeping API keys local and providing OTLP tracing.