mozilla-ai/otari

Open-source, OpenAI-compatible LLM gateway you run yourself. One endpoint for 40+ providers, with virtual keys, budgets, and usage tracking.

What it solves

Otari is an OpenAI-compatible LLM gateway that allows users to manage multiple model providers through a single endpoint. It solves the problem of managing fragmented provider credentials, tracking usage and budgets across different services, and implementing routing policies for failover and weighting without modifying application code.

How it works

Otari acts as a proxy between applications and LLM providers. It authenticates requests, resolves the necessary provider credentials, checks budgets before dispatching the request, and records usage after the completion. It uses the any-llm library to handle the actual provider calls.

Who it’s for

It is designed for developers and organizations that use multiple LLM providers and need a centralized way to handle authentication, cost control, and routing policies.

Highlights

  • Unified Endpoint: Provides a single OpenAI and Anthropic-compatible endpoint for 40+ providers.
  • Credential Management: Keeps provider credentials secure behind the gateway.
  • Capped Spending: Enforces budgets and tracks usage records after settlement.
  • Flexible Routing: Supports local routing policies for failover, weighting, and learned selection.
  • Access Control: Offers revocable API keys with scopes for users, workspaces, and models.
  • Extended Capabilities: Optional support for code execution, web search, MCP, guardrails, and file understanding.

Related

  • Project
    Continuum-AI-Corp/OrcaRouter-Lite

    A self-hosted, OpenAI-compatible LLM router that optimizes costs and reliability using automatic model selection and a managed fallback safety net.

  • Project
    theopenco/llmgateway

    LLM Gateway is an open‑source API gateway that provides a single OpenAI‑compatible endpoint for routing requests to multiple LLM providers, centralising API‑key management and offering built‑in usage and performance analytics. It can be used as a hosted SaaS service or self‑hosted via Docker.

  • Project
    BerriAI/litellm

    An open-source AI gateway and Python SDK that provides a unified OpenAI-compatible interface for calling 100+ different LLM providers.

  • Project
    yym68686/uni-api

    uni‑api is a config‑only server that presents a single OpenAI‑compatible API while routing calls to many LLM providers (OpenAI, Anthropic, Gemini, Vertex, Azure, AWS, etc.). It supports weighted and round‑robin load balancing, automatic retries, per‑model timeouts, rate limits, tool‑call passthrough, content moderation, and fine‑grained request overrides—all defined in an `api.yaml` file or via a `CONFIG_URL`. Deployable as a Docker container (one‑click Fugue button provided).

  • Project
    Mirrowel/LLM-API-Key-Proxy

    A self-hosted universal LLM API proxy that provides OpenAI and Anthropic compatible endpoints for multiple providers with built-in key rotation and failover.