caidaoli/ccLoad

What it solves

ccLoad is an AI API gateway designed to eliminate the operational complexity of managing multiple AI API providers. It solves common issues like manual channel switching, rate limits, upstream failures (including "soft errors" where a 200 OK response contains an error message), and opaque request status. It also prevents cost drift by implementing per-channel and per-token limits.

How it works

The system acts as a transparent proxy between clients (such as Claude Code, Codex, Gemini, and OpenAI) and various upstream AI providers. It uses smart routing based on priority and weighted round-robin selection, with automatic failover and model-aware cooldowns to skip failed keys or models without disabling entire channels. It supports multi-URL scheduling and protocol transformation to ensure requests are routed to the compatible upstream provider.

Who it’s for

Developers and teams who use multiple AI API keys and providers across different clients and need a centralized way to manage routing, failover, cost control, and monitoring.

Highlights

• Smart Routing & Failover: Priority-based selection with automatic failover for keys, models, and channels using exponential backoff.
• Model-Aware Cooldown: Specifically cools down only the affected model or key rather than the entire channel.
• Comprehensive Monitoring: A web dashboard for real-time tracking of active requests, token usage, TTFB, and cost.
• Protocol Transformation: Supports multiple client protocols and can automatically detect or explicitly route based on upstream capabilities.
• Cost & Rate Control: Implements per-channel daily cost limits, RPM caps, and concurrency limits.
• Broad Provider Support: Integrated support for OAuth channels (Codex, Anthropic, xAI) and various API-key based providers.