cheaper-inference/OmniRoute

OmniRoute – a free “AI gateway” that lets you call dozens of LLM providers through a single endpoint

What it is – OmniRoute is a self‑hosted service (available as an npm package, Docker image, or Electron app) that aggregates 375+ AI model providers (OpenAI, Claude, Gemini, etc.) behind one HTTP API (/v1/chat/completions‑compatible). You register your own API keys for any of the supported providers, and OmniRoute routes each request to the best‑available backend, automatically falling back when a provider is down or hits its quota.

Why it matters – The landscape of LLM APIs is fragmented: each vendor has its own URL, auth scheme, rate limits and pricing. OmniRoute removes that friction, and it also stacks free‑tier quotas from many providers, giving users an estimated ~1.6 billion free tokens per month across 150+ free‑tier services. It also applies a compression layer (RTK + Caveman) that can shave 15‑95 % off token usage, further extending free usage.

Key features

• Massive provider catalog – 375 providers, 1 320 unique model IDs, with a live dashboard showing free‑tier budgets and remaining quotas.
• Automatic routing & fallback – Choose model: "auto" and OmniRoute picks a healthy provider; if one fails it seamlessly retries another.
• Quota‑aware scheduling – Tracks each provider’s token limits and spreads traffic to avoid exhausting any single free tier.
• Token compression – Built‑in RTK + Caveman compression saves up to 95 % of tokens on eligible calls.
• Multi‑modality – Supports text, vision, audio and video models (via the “Modality Bridge”).
• CLI & agent integrations – 37+ command‑line tools and coding agents can be pointed at the same OmniRoute endpoint.
• Headless / server mode – Run as a background service (serve --headless) or as a desktop Electron app.
• Privacy‑first – All routing happens locally; your API keys never leave your machine unless you explicitly share them.

How to get started

1. Install via npm (npm i -g omniroute), Docker, or download the Electron app.
2. Run the service, open the web dashboard, and add the API keys for the providers you want to use.
3. Use the generated OmniRoute key in any client (cURL, Python, Node, etc.) and call the standard OpenAI‑compatible endpoint.

Typical use case – A developer building a chatbot wants to avoid paying for OpenAI usage. By installing OmniRoute, they can tap into dozens of free‑tier LLMs, automatically switch to a paid provider only when all free quotas are exhausted, and reduce token costs thanks to built‑in compression.

Where it runs – Works on any platform that can run Node.js or Docker, and also ships as a cross‑platform Electron desktop app.

Community & support – Active Discord, Telegram, WhatsApp groups, plus a public roadmap and sponsorship options.

---

TL;DR – OmniRoute is a free, open‑source gateway that unifies hundreds of LLM APIs behind one endpoint, auto‑routes, saves tokens, and lets you exploit free‑tier quotas across providers.