agentgateway/agentgateway

What it solves

Agentgateway provides a unified connectivity and security layer for Agentic AI. It solves the problem of managing complex communications between agents, LLMs, and external tools, providing a centralized way to handle security, observability, and governance without needing to rebuild these features into every individual agent framework.

How it works

It acts as an open-source proxy based on AI-native protocols like MCP (Model Context Protocol) and A2A (Agent-to-Agent). It routes traffic across three primary gateways:

• LLM Gateway: A unified OpenAI-compatible API that routes traffic to various providers (OpenAI, Anthropic, Gemini, Bedrock) with built-in budget controls and load balancing.
• MCP Gateway: Connects LLMs to tools and data sources using various transports (stdio, HTTP, SSE) and OpenAPI integration.
• A2A Gateway: Facilitates secure communication and task collaboration between different AI agents.

Additionally, it includes an inference routing system for self-hosted models on Kubernetes, multi-layered guardrails for content filtering, and a robust security suite including RBAC and OpenTelemetry integration.

Who it’s for

Developers and organizations building agentic workflows that need a centralized way to manage API keys, spend, security policies, and tool connectivity across different LLM providers and agent frameworks.

Highlights

• Unified API: Access multiple LLM providers through a single OpenAI-compatible interface.
• Tool Federation: Connects LLMs to external data and tools via MCP.
• Agent Interoperability: Enables secure agent-to-agent communication using the A2A protocol.
• Inference Optimization: Intelligent routing for self-hosted models based on GPU utilization and KV cache.
• Enterprise Security: Built-in RBAC, JWT/OAuth authentication, and content guardrails.