theopenco/llmgateway

Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.

LLM Gateway – A Unified API Layer for Large Language Models

What it is

  • An open‑source middleware that sits between your application and multiple LLM providers (OpenAI, Anthropic, Google Vertex AI, etc.).
  • It exposes a single API that mimics the OpenAI chat‑completion endpoint, so you can switch providers without changing your client code.

Core capabilities

  • Unified API – One endpoint (/v1/chat/completions) works for any configured provider.
  • Multi‑provider routing – Choose the target model per request or let the gateway balance traffic across providers.
  • Central key management – Store API keys for all providers in one place, secured by a secret.
  • Usage analytics – Logs token counts, response times and cost per request; useful for budgeting and performance tuning.
  • Performance monitoring – Compare latency and price of different models side‑by‑side.
  • Enterprise extensions (in the ee/ folder) – Billing, full payload retention, organization management, etc., under a commercial license.

How to run it

  • Hosted SaaS – Sign up at https://llmgateway.io, get an API key and start sending requests.
  • Self‑hosted – Deploy the Docker‑based stack yourself. The README provides two patterns:
    1. Use the provided run-unified-container.sh script with generated secrets.
    2. Run a single docker run command, mounting named volumes for PostgreSQL and Redis, and exposing the gateway’s ports (3002‑3007, 4001‑4002).
  • Development workflow: pnpm i && pnpm run setup to install deps, start containers, and seed the DB; pnpm dev to launch the services; pnpm build for a production build.

Project layout

  • apps/ – Several Next.js front‑ends (dashboard, playground, provider portal) and the Hono backend.
  • apps/gateway – The actual request‑routing service.
  • packages/ – Shared code: database schema (drizzle), model/provider definitions, utilities.
  • ee/ – Enterprise‑only admin UI and features, gated by a commercial license.

Who might use it

  • Teams building products that need to call LLMs but want to avoid vendor lock‑in.
  • Companies that need to monitor and control LLM spend across multiple providers.
  • Developers who want a local test environment that behaves like the OpenAI API while experimenting with alternative models.

License

  • Core code: AGPL‑v3 (open source, requires source distribution of any modified version).
  • Enterprise add‑ons: proprietary license; commercial use requires a separate agreement.

All details are taken directly from the repository’s README.

Related

  • Project
    BerriAI/litellm

    An open-source AI gateway that provides a unified, OpenAI-compatible interface for over 100 LLM providers, simplifying integration and management for production applications.

  • Project
    tbphp/gpt-load

    GPT‑Load is an open‑source Go service that acts as a single, self‑hosted gateway for many LLM APIs. It lets you configure multiple providers, manage API keys, schedule traffic, handle retries, and view unified logs and cost dashboards—all behind a familiar OpenAI‑compatible HTTP interface.

  • Project
    mnfst/llm-gateway

    Manifest LLM Gateway is an open-source tool that unifies access to hundreds of LLMs from various providers into a single, reliable, OpenAI-compatible endpoint with built-in routing, cost tracking, and automatic failover.

  • Project
    QuantumNous/new-api

    New API is an open‑source, self‑hosted gateway that unifies access to many LLM providers behind a single OpenAI‑compatible API. It handles routing, usage/cost tracking, fine‑grained access control, and offers a multilingual web console plus extensible JavaScript task plugins. Deployable via Docker or Docker‑Compose, it’s built with Go (backend) and React/Bun (frontend) and is licensed under AGPL‑v3.

  • Project
    ENTERPILOT/GoModel

    GoModel is an open‑source Go‑written AI gateway that provides a single OpenAI‑compatible endpoint for dozens of LLM providers. It adds caching, cost‑tracking, budgets, rate‑limits, fail‑over, observability (Prometheus/OpenTelemetry), a web dashboard, and extensible plugins, helping teams control spend and reliability while keeping existing SDKs unchanged.