vllm-project/agentic-api

What it solves

Agentic API provides a stateful, server-side layer for LLM inference engines like vLLM. It removes the burden of managing conversation history, tool-call loops, and multi-turn orchestration from the client application and moves it to the server, allowing clients to make a single API call for complex agentic workflows.

How it works

Written in Rust, it acts as a gateway sitting in front of a stateless inference engine (such as vLLM, SGLang, or NVIDIA Dynamo). It implements OpenAI-compatible Responses and Anthropic-compatible Messages APIs. The gateway manages state hydration via previous_response_id, executes server-side tools (like web search, MCP tools, and a code interpreter), and handles persistence using SQLite or PostgreSQL.

Who it’s for

Developers building agentic applications who want to run OpenAI-grade workloads (such as Codex or Claude Code) on their own GPUs using open models.

Highlights

• Stateful Conversations: Server-side history management eliminates the need for clients to track and replay full transcripts.
• Server-side Tool Execution: Built-in support for web search (via You.com, Brave, or Tavily), MCP tools, and an embedded code interpreter.
• Multi-agent Orchestration: Ability to spawn and coordinate subagents server-side over HTTP.
• Broad Transport Support: Supports non-streaming HTTP, Server-Sent Events (SSE), and WebSockets with stream_id multiplexing.
• Compatibility: Validated against the Open Responses compatibility suite and ready for Codex and Claude Code traffic.