mckinsey/agents-at-scale-ark

What it solves

Ark addresses the challenge of vendor lock-in and the operational complexity of deploying AI agents at scale. It provides a way to build agentic applications that are portable and provider-agnostic, meaning they can be moved between different AI models or cloud environments without rewriting the underlying code.

How it works

Ark is a Kubernetes-native framework that uses a declarative approach. Instead of writing proprietary code to define how an agent behaves, developers define agents as Kubernetes custom resources. These specifications include the agent's prompts, tools, and model references. Because it is built on Kubernetes, it leverages existing infrastructure for security, monitoring, and role-based access control (RBAC), allowing it to run on a single machine or across multi-cloud and on-prem environments.

Who it’s for

It is designed for developers and operations teams who need to build, host, and scale distributed AI agent systems while maintaining the flexibility to swap AI providers or re-platform use cases with minimal overhead.

Highlights

• Provider Agnostic: Easily switch between providers like OpenAI, Anthropic, Google, Azure, or local Ollama.
• Multi-Agent Orchestration: Supports various team strategies including sequential, graph, selector, and round-robin.
• Declarative Configuration: Agents are defined as resources rather than hard-coded logic.
• Extensible Tooling: Integrates with HTTP APIs, MCP servers, and other agents.
• Persistent Memory: Features pluggable backends to maintain conversation context across sessions.
• A2A Protocol: Enables interoperability with external agent systems.