CopilotKit/aimock

aimock – Deterministic mock server for AI‑application testing

What it is – aimock (formerly llmock) is a single‑package, zero‑dependency Node.js server that pretends to be every major LLM, vision, speech, video, embedding, vector‑db, and agent‑to‑agent endpoint your app might call. By pointing your SDK at the local port it provides, you get fully deterministic responses without any real API keys or network traffic, which eliminates surprise billing and makes end‑to‑end tests reliable.

Why it matters – Real AI services are costly, rate‑limited, and can change over time. aimock lets developers record real interactions once and then replay them forever, while also supporting chaos testing, drift detection, and fine‑grained fixture routing. This makes CI pipelines fast, cheap, and reproducible.

---

Core components (the “aimock suite”)

Component	What it mocks	Typical use‑case
LLMock	OpenAI (Chat, Realtime, Images, Audio, Video), Claude, Gemini, Bedrock, Azure, Vertex AI, Ollama, Cohere, OpenRouter, ElevenLabs TTS, BytePlus Ark, Grok, etc.	Replace any LLM or multimodal provider in unit/integration tests.
MCPMock	Copilot‑Kit MCP tools, resources, prompts with session handling	Test prompt‑management back‑ends without a live server.
A2AMock	Agent‑to‑agent SSE streaming protocol	Verify multi‑agent coordination logic.
AGUIMock	AG‑UI event streams (agent‑to‑UI)	Front‑end tests for UI‑driven agents.
VectorMock	Pinecone, Qdrant, ChromaDB‑compatible vector‑store APIs	Test retrieval‑augmented generation pipelines.
Services	Tavily search, Cohere rerank, OpenAI moderation, ElevenLabs TTS	Mock auxiliary services that many agents rely on.

All of these can run on one port (npx @copilotkit/aimock --config aimock.json) or be started programmatically for a tighter test harness.

---

Highlighted features

• Record & Replay – Proxy real APIs, store the exact JSON payloads (including per‑frame timestamps), then replay them forever. Replay speed can be scaled with --replay-speed.
• Timing‑aware playback – Preserves first‑token latency (ttft) and overall token‑per‑second cadence, useful for testing UI loading states.
• Accurate token‑usage & cost – Recorded usage frames are replayed so billing‑related code sees realistic prompt_tokens, completion_tokens, and provider‑specific cost breakdowns.
• Multi‑turn conversation support – Fixtures can be scoped to specific turns, tool calls, system messages, or custom predicates, enabling realistic tool‑use scenarios.
• Chaos testing – Random 500 errors, malformed JSON, mid‑stream disconnects, and configurable rate‑limit headers let you verify retry/back‑off logic.
• Strict vs. lenient matching – Global --strict flag or per‑request X‑AIMock‑Strict header controls whether unmatched requests error out.
• Context‑based fixture routing – X‑AIMock‑Context header isolates fixture sets per integration, avoiding cross‑test contamination.
• Drift detection – CI jobs can automatically compare recorded fixtures against live APIs to catch provider changes.
• Streaming physics – Adjustable ttft, tokens‑per‑second, and jitter to simulate realistic streaming behavior.
• Metrics – Prometheus‑compatible endpoint (/metrics) reports request counts, latency, and fixture hit‑rates.
• Docker & Helm – Official container image and Helm chart for easy deployment in CI/CD pipelines.
• Test‑framework plugins – Vitest and Jest helpers (useAimock()) automatically start/stop the server and patch environment variables.

---

Typical workflow

1. Install – npm i @copilotkit/aimock.
2. Record (optional) – Run npx @copilotkit/aimock llmock --record --provider-openai https://api.openai.com while your app talks to the real provider. Fixtures are saved as JSON.
3. Replay – Start the mock server with those fixtures: npx @copilotkit/aimock --config aimock.json (or programmatically via new LLMock({port:0})).
4. Point SDKs – Set OPENAI_BASE_URL (or the equivalent for other providers) to http://localhost:<port>/v1 and give any dummy API key.
5. Run tests – Your test suite now hits the mock server, receives deterministic responses, and can also exercise error/latency scenarios via headers or CLI flags.

---

Installation & quick start (code snippet from README)

npm install @copilotkit/aimock

import { LLMock } from "@copilotkit/aimock";

const mock = new LLMock({ port: 0 });
mock.onMessage("hello", { content: "Hi there!" });
await mock.start();

process.env.OPENAI_BASE_URL = `${mock.url}/v1`;
process.env.OPENAI_API_KEY = "mock"; // required by most SDKs

// … run your application / tests …
await mock.stop();

---

Integration points

• LangChain, CrewAI, LlamaIndex, Mastra, Google ADK, Microsoft Agent Framework – dedicated guides show how to swap the provider URL for aimock.
• GitHub Action – CopilotKit/aimock@v1 can preload fixtures and expose the mock URL to subsequent steps.
• CLI utilities – llmock (compat alias) for flag‑driven runs, aimock convert to import fixtures from other mock tools, and Docker command for containerised CI.

---

Who uses it?

The AG‑UI project cites aimock in its end‑to‑end test suite to verify agent behavior across multiple LLM providers using fixture‑driven responses.

---

License

MIT – free for commercial and open‑source use.

---

Bottom line – aimock is a comprehensive, zero‑dependency mock server that covers the full stack of modern generative‑AI services. It enables cheap, fast, and deterministic testing of any application that talks to LLMs, multimodal models, vector stores, or agent protocols.