Azure-Samples/ai-hub-gateway-solution-accelerator

Reference architecture that provides a set of guidelines and best practices for implementing a central AI API gateway to empower various line-of-business units in an organization to leverage Azure AI services

What it solves

Citadel Governance Hub provides a centralized, enterprise-grade "landing zone" for managing AI deployments. It solves the problem of scaling AI across a large organization while maintaining strict security, compliance, and observability. Instead of each team deploying AI models in silos, this hub acts as a unified gateway that enforces runtime policies, manages costs, and ensures consistent security guardrails across all AI workloads.

How it works

The project implements a hub-and-spoke architecture where a central governance layer (the Hub) manages the runtime control plane, while individual business units operate their own agent execution planes (the Spokes).

Key components include:

  • Unified AI Gateway: Powered by Azure API Management, it handles routing, token rate limiting, and identity validation.
  • Contract-Driven Governance: Uses Infrastructure-as-Code (Bicep/Terraform) to manage the "supply side" (Backend Contracts for LLM capacity) and the "demand side" (Access Contracts for use-case specific policies).
  • Security Layer: Integrates Azure Content Safety for prompt shielding and Azure Language Service for PII detection and anonymization.
  • Observability Stack: Combines Log Analytics, Application Insights, Event Hub, and Cosmos DB to stream and analyze usage data in near real-time.

Who it’s for

Enterprise platform teams and AI architects who need to deploy a governed, secure, and repeatable foundation for AI applications and agents on Azure at scale.

Highlights

  • Declarative Governance: Every change to model access or policies is version-controlled via .bicepparam files rather than manual portal configuration.
  • Multi-Provider Support: Routes to various backends including Microsoft Foundry, Azure OpenAI, and external OpenAI-compatible providers.
  • Automated Guardrails: Built-in support for PII detection, content filtering, and layered authentication (API key + JWT).
  • Integrated Registry: Uses Azure API Center for the discovery of available AI tools, agents, and services.

Related

  • Project
    Azure-Samples/AI-Gateway

    An enterprise-grade AI Gateway framework powered by Azure API Management that provides centralized security, cost control, and observability for managing AI models, tools, and agents.

  • Project
    devchilll/scope

    A production-ready governance framework for enterprise AI agents that implements safety guardrails, compliance-as-code, IAM, and human-in-the-loop escalation.

  • Project
    Azure/AI-Landing-Zones

    An enterprise-scale reference architecture and IaC implementation for deploying secure and resilient AI applications and agents on Azure.

  • Project
    ThinkWatchProject/ThinkWatch

    An enterprise-grade secure gateway that centralizes the governance, auditing, and cost tracking of AI API calls and MCP tool invocations across an organization.

  • Project
    cordum-io/cordum

    An agent control plane for governance, safety, and trust that provides a deterministic layer to define, enforce, and audit the behavior of autonomous AI agents.