agentgateway: what it is, what problem it solves & why it's gaining traction

What it solves

Agentgateway addresses the complexity of connecting AI agents to LLMs, tools, and other agents. It provides a centralized proxy to handle security, observability, and governance, eliminating the need for developers to build these infrastructure components manually for every agentic workflow.

How it works

It operates as an open-source proxy based on AI-native protocols like MCP (Model Context Protocol) and A2A. It routes traffic through several specialized gateways:

• LLM Gateway: Unifies multiple LLM providers under one OpenAI-compatible API with spend controls and load balancing.
• MCP Gateway: Connects LLMs to external data and tools using various transports (stdio, HTTP, SSE) and OAuth.
• A2A Gateway: Facilitates secure communication and task collaboration between different agents.
• Inference Routing: Directs traffic to self-hosted models based on real-time metrics like GPU utilization and queue depth.

Who it’s for

It is designed for developers and organizations building agentic AI systems who need a production-ready way to manage connectivity, enforce guardrails, and monitor communications across different frameworks and environments.

Highlights

• Multi-layered Guardrails: Content filtering via regex, OpenAI moderation, AWS Bedrock, and Google Model Armor.
• Enterprise Security: Includes JWT, API keys, OAuth, and fine-grained RBAC using a CEL policy engine.
• Observability: Integrated OpenTelemetry for metrics, logs, and tracing.
• Deployment Flexibility: Supports standalone installations and Kubernetes deployments via a built-in controller and Gateway API.