looplj/axonhub

⚡️ Open-source AI Gateway — Use any SDK to call 100+ LLMs. Built-in failover, load balancing, cost control & end-to-end tracing.

The one-paragraph explanation

AxonHub is a model-agnostic AI gateway — a server that sits between your application and the AI providers (OpenAI, Anthropic, Gemini, DeepSeek, etc.). Your app keeps using its existing SDK (say, the OpenAI SDK), but instead of calling OpenAI directly, it calls AxonHub. AxonHub then translates your request into whatever format the actual backend provider expects and forwards it. Want to switch from GPT-4o to Claude 3.5? You don't touch your app code — you just change a config entry in AxonHub. It's a classic "one interface, many backends" pattern, tailored for the LLM API zoo.

What problem does it actually solve?

  • Vendor lock-in — your code is coupled to one provider's SDK; AxonHub decouples it
  • Integration sprawl — instead of writing adapters for 10+ providers, you write one against AxonHub's API, which speaks OpenAI, Anthropic, and Gemini protocols natively
  • Zero observability — it gives you full request tracing out of the box (thread-level timelines for debugging)
  • Runaway costs — real-time per-request cost breakdowns (input/output/cache tokens) plus budget management

Key features (from the README)

  • Any SDK → any model: call Claude via the OpenAI SDK, or GPT via the Anthropic SDK
  • Request tracing: full timeline per request for fast debugging
  • Enterprise RBAC: fine-grained access control, usage quotas, data isolation
  • Smart load balancing: routes to the healthiest channel, auto-failover in <100ms
  • Cost tracking per request

API types supported

Text generation (chat/completions), image generation, rerank, and embeddings. Realtime conversation is listed as upcoming.

Providers supported (each through multiple protocols)

OpenAI, Anthropic, Zhipu (GLM), Moonshot (Kimi), DeepSeek, ByteDance Doubao, Gemini, Jina AI, OpenRouter, ZAI, NanoGPT — with AWS Bedrock and Google Cloud in testing. So it's not just “10 providers” — most accept requests in OpenAI, Anthropic, or Gemini formats interchangeably.

Tech stack & shape

  • Written in Go (HTTP via Gin, ORM via ent, GraphQL via gqlgen)
  • Ships a web dashboard (screenshots show system dashboard, channel management, model pricing, trace viewer, request monitor)
  • Docker-ready; there's a live demo instance and a demo account
  • Deployment flow: init wizard → add provider channels with API keys → create an API key → call it like a normal OpenAI-compatible endpoint (base_url + your key)

Who it's for

Teams that build on top of LLM APIs and want to (a) avoid being stuck with one vendor, (b) monitor and control spend centrally, or (c) offer multiple models behind a single consistent API — e.g., internal platform teams, SaaS builders, or anyone managing many API keys across providers.

Verdict

Genuine, substantial infrastructure project — not a wrapper or a list. It's a well-scoped solution to a real pain point (API fragmentation and vendor lock-in in the LLM space), with a plausible architecture, multiple completed API translations, and visible operational tooling. The heavy sponsor section (paid API resellers) is a bit unusual for an OSS repo, but it doesn't change what the software itself does.

Rating: 4/5 — solid and pragmatic; the value is clear the moment you've managed even two LLM providers.

Related

  • Project
    astaxie/TokenHub

    TokenHub is an open‑source, Go‑based gateway that adds enterprise‑level governance (routing, permissions, budgeting, and bill reconciliation) to any OpenAI‑compatible LLM provider. It offers role‑aware consoles, OpenAI‑compatible APIs, SQLite/PostgreSQL deployment options, and Docker‑Compose or systemd installers.

  • Project
    ENTERPILOT/GoModel

    GoModel is an open‑source Go‑written AI gateway that provides a single OpenAI‑compatible endpoint for dozens of LLM providers. It adds caching, cost‑tracking, budgets, rate‑limits, fail‑over, observability (Prometheus/OpenTelemetry), a web dashboard, and extensible plugins, helping teams control spend and reliability while keeping existing SDKs unchanged.

  • Project
    pleaseai/shunt

    A transparent LLM gateway that allows you to route Claude Code requests to alternative AI providers, enabling you to use different models while keeping your agent's tools and workflow intact.

  • Project
    ogx-ai/ogx

    OGX is an open‑source, model‑agnostic API server that mimics OpenAI’s REST endpoints. It lets you run any LLM (Llama, GPT, Gemini, etc.) locally or in the cloud, while keeping the same client code. In addition to standard chat/completion/embedding calls, OGX offers a “Responses” API for tool‑calling and RAG, versioned skill bundles, and native routes for Anthropic and Google GenAI SDKs. Install with a one‑line script, start the server, and point any OpenAI‑compatible client at it.

  • Project
    BerriAI/litellm

    An open-source AI gateway that provides a unified, OpenAI-compatible interface for over 100 LLM providers, simplifying integration and management for production applications.