theopenco/llmgateway

Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.

TL;DR

LLM Gateway is an open‑source API gateway that sits between your app and multiple large‑language‑model providers (OpenAI, Anthropic, Google Vertex AI, …). It gives you a single, OpenAI‑compatible endpoint, centralised API‑key management, and built‑in analytics for token usage, latency and cost.

What it does

Unified API – Your code talks to one endpoint that mimics the OpenAI /v1/chat/completions shape, while the gateway forwards the request to whichever provider you choose.
Multi‑provider routing – Configure several back‑ends (OpenAI, Anthropic, Vertex AI, etc.) and route calls per model, per request, or per organization.
Key vault – Store all provider API keys in one place; the gateway injects the correct key when forwarding.
Usage & cost tracking – Every request is logged; the system records token counts, response times and the monetary cost reported by each provider.
Performance monitoring – Compare latency and price across models, helping you pick the most efficient one for a given workload.

Who might need it

Product teams that want to switch between LLM providers without rewriting client code.
Enterprises that need visibility into LLM spend across many services or teams.
Developers building SaaS platforms that expose LLM capabilities to end‑users and want a single, auditable gateway.
Self‑hosters who must keep data on‑premise and cannot rely on a third‑party gateway.

How to get started

Hosted (quick start)

Sign up at llmgateway.io.
Create an API key.
Use the key exactly like an OpenAI key (see the curl example in the README).

Self‑hosted (full control)

Prerequisites – Docker, Docker Compose, and pnpm.

Set two secret env vars (random base‑64 strings) for internal auth:

export LLM_GATEWAY_SECRET="$(openssl rand -base64 32 | tr -d '\n')"
export GATEWAY_API_KEY_HASH_SECRET="$(openssl rand -base64 32 | tr -d '\n')"

Run the provided script which creates PostgreSQL & Redis volumes and starts the unified container:
```
./scripts/run-unified-container.sh
```
Or launch a container manually with the docker run command shown in the README.
The gateway’s HTTP ports (3002‑3007, 4001‑4002) become available; point your app at http://localhost:3002/v1/....
For development, clone the repo, run pnpm i && pnpm run setup, then pnpm dev to start the UI, API and other services locally.

Core components (folder overview)

Path	Role
`apps/ui`	Next.js dashboard for admins to view analytics, manage keys, etc.
`apps/playground`	Consumer‑facing chat UI (a simple “Lounge” demo).
`apps/code`	Landing page & tools for developers (coding‑assistant style UI).
`apps/airside`	Provider‑self‑service portal where new model providers register – no external PRs for new providers.
`apps/api`	Hono (a lightweight Node.js web framework) backend that handles auth and routing logic.
`apps/gateway`	The actual request‑forwarding service that talks to the various LLM APIs.
`apps/docs`	Documentation site (likely built with Next.js).
`packages/db`	Drizzle ORM schema + migrations for PostgreSQL.
`packages/models`	Type definitions for LLM models and provider‑specific configuration.
`packages/shared`	Common TypeScript types and utility helpers used across the monorepo.

Licensing & commercial add‑ons

Core – AGPL‑v3, meaning you can run, modify and redistribute the software as long as you also share your source under the same license.
Enterprise (ee/) – Proprietary extensions (advanced billing, multi‑org admin, unlimited data retention, etc.) that require a paid white‑label license.

Quick takeaways

Real, production‑grade gateway – not just a demo; includes DB, Redis, analytics, and a UI.
Open‑source core – you can run it yourself and extend the open parts.
Enterprise layer – the project monetises via a commercial add‑on, a common model for SaaS‑oriented open‑source tools.
Target audience – teams that need a managed, observable way to consume multiple LLM APIs.

How to evaluate further

Clone the repo and spin up the Docker container; try the /v1/chat/completions endpoint against a provider you already have a key for.
Browse the UI (apps/ui) to see the analytics dashboards.
Look at packages/models to understand how new providers would be described (even though external PRs are blocked, the code shows the pattern).
Review the AGPL license file if you plan to embed the gateway in a product you’ll distribute.

All information above is taken directly from the repository’s README; no additional features have been assumed.