fruitbars/simple-one-api

What it solves

It provides a unified gateway to connect multiple Large Language Model (LLM) providers, allowing users to manage various API keys and models through a single interface. It specifically addresses the problem of "hitting the wall" with API rate limits (429 errors) by implementing an intelligent key pool and capacity-aware scheduling, ensuring that requests are distributed across available keys to avoid repeated failures.

How it works

The project acts as a proxy between clients and LLM providers. It supports three major client protocols: OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. It uses a 60-second rolling window to estimate remaining Tokens Per Minute (TPM) for each API key and model combination. When a request arrives, it filters out disabled or circuit-broken keys, estimates the cost of the request, and selects the key with the most available capacity. If a provider returns a 429 error, the system automatically cools down that specific key-model pair and attempts to route the request to another healthy key in the pool.

Who it’s for

• Developers who use multiple LLM providers and want a single API endpoint.
• Users of local LLM clients (like Codex or Claude Code) who need a managed gateway for their API keys.
• Teams needing a robust way to handle rate limiting and load balancing across multiple API accounts.

Highlights

• Capacity-Aware Scheduling: Intelligent key selection based on estimated TPM and real-time 429 cooldowns.
• Multi-Protocol Support: Compatible with OpenAI and Anthropic API formats.
• Unified Management: Includes a visual configuration dashboard, a built-in React web chat interface, and a Wails-based desktop application.
• Granular Rate Limiting: Supports four layers of limits (Provider, Provider-Model, Key, and Key-Model) across QPS, QPM, RPM, TPM, and concurrency.
• Observability: Real-time log views and lightweight usage statistics (latency, TTFT, token counts) without storing prompt content.