labring/aiproxy

AI Proxy is a high performance AI gateway using OpenAI / Claude / Gemini protocol as the entry point. It features intelligent error handling, multi-channel management, and comprehensive monitoring. With support for multiple models, rate limiting, and multi-tenant isolation.

What it solves

AI Proxy acts as a production-ready gateway (middleware) between client applications and various AI providers. It solves the complexity of managing multiple AI APIs by providing a single, OpenAI-compatible entry point that handles routing, monitoring, multi-tenancy, and protocol translation, reducing the need for clients to implement provider-specific logic.

How it works

It intercepts requests from clients and passes them through a pipeline consisting of authentication, an intelligent router, and a plugin system before forwarding them to providers like OpenAI, Anthropic, or Gemini. It can automatically convert protocols (e.g., converting a Claude request to an OpenAI format) and uses a management panel for configuration and analytics. It also integrates with the Model Context Protocol (MCP) to extend AI capabilities with external tools.

Who it’s for

It is designed for developers and organizations building AI applications that require high reliability, scalability, and the ability to manage multiple AI models and tenants under a unified interface.

Highlights

  • Intelligent Routing: Features smart retry logic, load balancing, and priority-based channel selection.
  • Protocol Conversion: Seamlessly translates between OpenAI, Claude, and Gemini protocols.
  • Multi-tenant Management: Provides organization isolation, token-based access control, and per-group resource quotas (RPM/TPM).
  • Extensible Plugin System: Includes built-in plugins for caching (Redis/memory), web search (Google, Bing, Arxiv), and reasoning model support (splitting <think> tags).
  • MCP Support: Integrates public, private, and embedded MCP servers, including automatic OpenAPI-to-MCP conversion.
  • Comprehensive Monitoring: Offers real-time alerts, detailed audit logs, and cost analysis.

Related

  • Project
    BenedictKing/ccx

    A high-performance AI API proxy and gateway that unifies Claude, OpenAI, and Gemini APIs into a single entry point with built-in channel management and failover.

  • Project
    ZhiYi-R/moon-bridge

    A Go-based protocol conversion and model routing proxy that unifies various AI provider APIs (Anthropic, Gemini, OpenAI) under a single OpenAI Responses API interface.

  • Project
    yym68686/uni-api

    uni‑api is an open‑source, Docker‑based gateway that lets you expose a single OpenAI‑compatible API while routing requests to many different LLM back‑ends (OpenAI, Anthropic, Gemini, Vertex, Azure, AWS, etc.). It offers YAML‑only configuration, weighted/round‑robin load balancing, automatic retries, per‑model timeouts, rate limiting, moderation, and tool/image support.

  • Project
    kittors/CliRelay

    A unified proxy server that consolidates AI CLI subscriptions and API keys into a single managed API layer with routing, quotas, and multi-tenant governance.

  • Project
    router-for-me/CLIProxyAPI

    CLI Proxy API is an open‑source proxy server that presents a unified OpenAI‑compatible API for many LLM providers (OpenAI, Gemini, Claude, Grok, Kimi, Meta, etc.). It handles OAuth logins, multi‑account load‑balancing, streaming, function calls, and multimodal inputs, and offers optional dashboards for usage and cost tracking. Ideal for developers with multiple AI subscriptions who want a single local endpoint for their CLI tools.