ENTERPILOT/GoModel

AI gateway / AI control plane / AI proxy written in Go. Unified OpenAI-compatible and Anthropic-compatible API for OpenAI, Anthropic, Gemini, Groq, xAI, Ollama, vLLM and more. A LiteLLM alternative with observability, guardrails, streaming, cost tracking, intelligent routing, sticky sessions, failover, real-time logs and usage tracking. Prod ready.

What is GoModel?

GoModel is an open‑source AI gateway written in Go. It sits between your application (or any OpenAI/Anthropic‑compatible SDK) and a large list of LLM providers, handling the HTTP calls for you. Its main goal is to make using multiple models cheap, reliable, and observable.

Core ideas

Idea Why it matters
Single endpoint – expose a /v1 (OpenAI) or /v1/messages (Anthropic) API that forwards to any configured provider. You can keep your existing code unchanged while swapping or mixing back‑ends.
Caching & prompt compression – store exact or semantic responses and optionally shrink prompts before they hit the provider. Reduces token usage and therefore cost.
Cost & budget tracking – per‑request cost estimates, dashboards, hard spend limits. Prevents surprise bills when using expensive models.
Fail‑over & key rotation – automatic rerouting, circuit‑breakers, round‑robin over multiple API keys. Improves reliability and bypasses per‑key rate limits.
Observability – Prometheus metrics, OpenTelemetry traces, live request streaming in a web UI. Gives you the data you need to monitor performance and debug issues.
Extensible plugins & workflows – custom guardrails, response filters, routing logic, loaded as Go plugins (.so). Lets teams enforce policies (e.g., profanity filters) without changing application code.

Main features (as listed in the README)

  • Caching – exact and semantic response caching.
  • Cost tracking & budgets – real‑time spend analytics, hard limits per user/team/key.
  • Rate limits – caps on requests, tokens, concurrency.
  • Virtual models – stable model aliases that can load‑balance across providers.
  • Session keeping – pins a client session to a specific provider to keep its prompt cache warm.
  • Fail‑over & resilience – retries, circuit‑breakers, backup providers.
  • Labelling & user paths – hierarchical scoping for keys, budgets, and audit logs.
  • MCP gateway – aggregates multiple MCP servers behind one endpoint.
  • Passthrough API – direct access to provider‑native endpoints under /p/{provider}/….
  • Audio & image APIs – TTS, transcription, image generation/editing, all with the same access controls.
  • Guardrails & plugins – request/response policies, custom filters, routing strategies.
  • Workflows – versioned per‑request policy bundles.
  • Provider key rotation – round‑robin over several API keys.
  • Observability – Prometheus, OpenTelemetry, audit logs, live streaming.
  • Playground UI – try any model from the dashboard and see raw JSON.

Who might use it?

Use case How GoModel helps
Start‑ups or SaaS platforms that need to switch between OpenAI, Anthropic, Gemini, etc., without rewriting code. One configurable gateway replaces many SDK changes.
Teams with strict budgets that want to cap spend per project or per user. Built‑in cost tracking, budgets, and rate limits.
Enterprises that require audit logs, SSO, and custom guardrails. Dashboard, OpenTelemetry, OIDC SSO (Pro), plugin system.
Developers running self‑hosted LLMs (e.g., Ollama, vLLM). Supports “all OpenAI‑compatible providers” and can proxy to local models.
Ops teams that need metrics and alerts. Prometheus metrics, circuit‑breaker logic, health checks.

Getting started (quick recap)

  1. Install – one‑liner script for macOS/Linux, PowerShell script for Windows, or Docker image.
  2. Configure – .env, config.yaml, or dashboard UI; a template lists all env vars.
  3. Run – gomodel (or docker run …). The admin UI is at http://localhost:8080/admin/dashboard.
  4. Call the API – send a POST to /v1/responses (OpenAI style) or use any OpenAI/Anthropic SDK pointing at http://localhost:8080/v1.

License & community

The repository is open source (the README does not specify a license, but the presence of CI, Docker images, and a public repo suggests an OSS licence). The project is actively maintained – CI badge, Docker pulls, a Discord community, and a Hacker News discussion are linked.

TL;DR

GoModel is a Go‑based, self‑hosted AI gateway that lets you route, cache, monitor, and control requests to dozens of LLM providers behind a single OpenAI‑compatible endpoint. It’s aimed at teams that need cost control, reliability, and observability while keeping their existing SDKs untouched.

Related

  • Project
    theopenco/llmgateway

    LLM Gateway is an open‑source API layer that lets you call many LLM providers through a single OpenAI‑compatible endpoint, with built‑in key management, cost tracking, and performance monitoring. It can be used as a hosted SaaS or self‑hosted via Docker.

  • Project
    open-webui/open-webui

    Open WebUI is a self-hosted AI platform that provides a user-friendly interface for interacting with AI models, supporting local models via Ollama and any OpenAI-compatible API. It offers features like plugins, role-based access control, persistent memory, RAG, and more, and can run entirely offline.

  • Project
    tbphp/gpt-load

    GPT‑Load is an open‑source Go service that acts as a single, self‑hosted gateway for many LLM APIs. It lets you configure multiple providers, manage API keys, schedule traffic, handle retries, and view unified logs and cost dashboards—all behind a familiar OpenAI‑compatible HTTP interface.

  • Project
    QuantumNous/new-api

    New API is an open‑source, self‑hosted gateway that unifies access to many LLM providers behind a single OpenAI‑compatible API. It handles routing, usage/cost tracking, fine‑grained access control, and offers a multilingual web console plus extensible JavaScript task plugins. Deployable via Docker or Docker‑Compose, it’s built with Go (backend) and React/Bun (frontend) and is licensed under AGPL‑v3.

  • Project
    router-for-me/CLIProxyAPI

    CLI Proxy API is an open‑source proxy server that presents a unified OpenAI‑compatible API for many LLM providers (OpenAI, Gemini, Claude, Grok, Kimi, Meta, etc.). It handles OAuth logins, multi‑account load‑balancing, streaming, function calls, and multimodal inputs, and offers optional dashboards for usage and cost tracking. Ideal for developers with multiple AI subscriptions who want a single local endpoint for their CLI tools.