Azure-Samples/ai-hub-gateway-solution-accelerator

Reference architecture that provides a set of guidelines and best practices for implementing a central AI API gateway to empower various line-of-business units in an organization to leverage Azure AI services

What it solves

Citadel Governance Hub provides a centralized, enterprise-grade "landing zone" for AI deployments. It solves the problem of scaling AI innovation across large organizations while maintaining strict security, compliance, and observability. Instead of each team deploying AI services in silos, this hub acts as a governed gateway that enforces consistent policies, manages costs, and ensures safety guardrails across all AI consumption.

How it works

The project implements a hub-and-spoke architecture where a central control plane (the Governance Hub) manages the runtime enforcement of AI policies. It uses Azure API Management as a unified AI gateway to route requests to various LLM backends.

Governance is managed through a "contract-driven" approach using infrastructure-as-code (Bicep/Terraform), consisting of three contract types:

  • Backend Contracts: Define the available LLM backends and models (e.g., Azure OpenAI, Amazon Bedrock).
  • Access Contracts: Define which agents or business units can access specific models and what policies (like PII detection or content safety) apply to them.
  • Publish Contracts: Allow the publication of AI tools (MCP) and agents (A2A) through the gateway.

Who it’s for

Enterprise platform teams and AI architects who need to deploy a secure, scalable, and observable AI infrastructure on Azure that supports multiple business units and diverse AI workloads.

Highlights

  • Unified AI Gateway: Centralized routing, rate limiting, and identity validation using Azure API Management.
  • Contract-Driven Governance: Declarative, version-controlled onboarding of models and use cases via IaC.
  • Integrated Safety: Built-in support for PII detection and content safety protections.
  • Enterprise Observability: Centralized logging and real-time usage analytics using Event Hub and Cosmos DB.
  • Flexible Networking: Supports both dedicated spoke VNet and integrated enterprise hub VNet deployment patterns.

Related

  • Project
    Azure-Samples/AI-Gateway

    An enterprise-grade AI Gateway framework powered by Azure API Management that provides centralized security, cost control, and observability for managing AI models, tools, and agents.

  • Project
    Azure/AI-Landing-Zones

    Azure AI‑Landing‑Zones is a Microsoft‑provided reference architecture (Bicep, Terraform, Portal) that lets enterprises quickly stand up secure, production‑grade AI app and agent workloads on Azure, with built‑in alignment to the Cloud Adoption and Well‑Architected frameworks.

  • Project
    agentic-community/mcp-gateway-registry

    A governed control plane and gateway for AI assets, providing centralized discovery, access control, and audit trails for MCP servers, agents, and model inference endpoints.

  • Project
    devchilll/scope

    A production-ready governance framework for enterprise AI agents that implements safety guardrails, compliance rules, IAM, and human-in-the-loop escalation.

  • Project
    astaxie/TokenHub

    TokenHub is an open‑source, Go‑based gateway that adds enterprise‑level governance (routing, permissions, budgeting, and bill reconciliation) to any OpenAI‑compatible LLM provider. It offers role‑aware consoles, OpenAI‑compatible APIs, SQLite/PostgreSQL deployment options, and Docker‑Compose or systemd installers.