tashfeenahmed/freellmapi

What it solves

FreeLLMAPI provides a unified, OpenAI-compatible endpoint to aggregate multiple free-tier LLM providers. It eliminates the need to manage dozens of different SDKs, rate limits, and API keys manually, allowing users to stack free quotas from various AI labs to create a large pool of inference capacity.

How it works

The project acts as a local router that sits between your AI applications and various LLM providers. You add your provider keys to the router, and it exposes a single /v1 API. It automatically handles routing, failover to the next available model when rate-limited, and tracks per-key usage to stay under free-tier caps. The router also syncs a signed model catalog from a central feed to keep up with changes in the models and quotas available from providers.

Who it’s for

Developers and AI enthusiasts who want to use a wide variety of free LLM models without the same repetitive setup and a complex management of multiple API keys and rate limits.

Highlights

• Unified API: Supports /v1/chat/completions, /v1/embeddings, and media generation endpoints (image, video, audio).
• Smart Routing: Features six routing strategies with automatic failover and key rotation to maximize uptime.
• Multi-Model Synthesis (Fusion): Can fan out prompts to multiple free models in parallel and synthesize a final answer using a judge model.
• Broad Compatibility: Works with any OpenAI-compatible client, including coding agents like Claude Code, Aider, and Cursor.
• Local-First Security: Provider keys are AES-256-GCM encrypted in a local SQLite database.
• Self-Updating Catalog: Automatically updates its model list and quota information without requiring manual updates.
• Admin Dashboard: Includes a React UI for managing keys, monitoring analytics, and testing models in a playground.