mnfst/llm-gateway

What it solves

Manifest provides a unified, OpenAI-compatible endpoint for AI agents and applications, eliminating provider lock-in. It solves the problem of managing multiple API keys, subscriptions, and local models across different providers, while ensuring reliability through automated fallbacks and self-healing for failed requests.

How it works

It acts as a gateway layer that connects to over 300 models via 35 built-in provider connections (including OpenAI, Anthropic, Google, and local runners like Ollama). Users can send queries to a single endpoint using "model": "auto", and Manifest routes the request to the appropriate model based on custom routing rules. It also records full request/response bodies for logging and tracks spending and limits.

Who it’s for

Developers building AI agents and apps who need to switch between different LLM providers seamlessly, manage costs across multiple subscriptions, or integrate local models into their production workflows.

Highlights

• Unified Endpoint: Connects various API keys, subscriptions, and local models to one OpenAI-compatible API.
• Broad Provider Support: Integrates with 300+ models across 35 providers, including major labs and local tools like llama.cpp and LM Studio.
• Reliability Tools: Features automated fallbacks to different models when queries fail and self-healing capabilities for bad requests.
• Cost Management: Tracks spending in real-time and allows for the setup of notifications and limits.
• Flexible Deployment: Supports Docker, cloud platforms (Railway, Render, AWS, GCP), and local hosting.