AI System Design Guide

What it is: A free, continuously updated reference for people who design and build AI systems in production — and for engineers preparing for AI system design job interviews. It's maintained by Om Bharatiya and hosted on GitHub under an MIT license, meaning anyone can use it and contribute improvements.

The core idea

The guide's premise is that printed books about AI go stale quickly — new models, pricing, and engineering patterns appear faster than books can be updated. This project is a "living document": it tracks new model releases, protocol changes, and emerging patterns as they ship, so readers get current information rather than last year's.

What's inside

The repository is organized into 19 numbered topic folders, plus standalone guides. Major areas include:

• Foundations — how LLMs work internally (transformers, attention, embeddings)
• Model landscape — specific models (Claude, GPT, Gemini, Grok, DeepSeek, and others), how to pick between them, and real pricing data
• Retrieval systems — RAG (retrieval-augmented generation), chunking strategies, vector databases, reranking
• Agentic systems — AI agents that take actions, tool use, the MCP and A2A protocols, durable execution
• Voice and audio agents — real-time voice AI, including latency and cost considerations
• Evaluation and observability — how to measure whether AI systems actually work (RAGAS, LangSmith, benchmarks)
• Security, reliability, and compliance — access control, guardrails, AI governance (EU AI Act, NIST)
• Infrastructure and MLOps — GPU clusters, AI gateways, cost control (FinOps)
• Case studies — 20 real-world architecture examples with diagrams, including a multi-tenant SaaS system, a fraud detection system, a voice healthcare agent, and a coding agent

There are also two bonus deep-dive guides (3,000+ lines each) on AI evaluation, covering platforms like Phoenix, Langfuse, and LangWatch.

Interview preparation

A significant portion of the guide targets AI engineering interviews. It includes:

• A 147-question interview bank with real questions (including a Voice AI set)
• Answer frameworks with a worked mock transcript
• Ten whiteboard exercises
• A job market trends document (October 2026)
• A FAQ and a glossary defining every term

Example interview questions the guide helps you answer: "Design a multi-tenant RAG system where competitors cannot see each other's data" and "Your agent takes 15 steps for a 3-step task. How do you debug it?"

What it is NOT

The README is explicit about boundaries:

• Not a tutorial for Python, PyTorch, or basic ML — readers should start with a course first
• Not vendor-neutral — it names specific models, prices, and frameworks because real systems require real choices
• Not a replacement for hands-on building — meant to be read alongside a project
• Not a research paper digest — it cites papers only when they change practice

Extras

The author offers paid 1:1 mock interviews and mentorship (via EngineBogie and Topmate), and the guide is readable online at aidaddy.tech with instant search. There are also companion documents for course recommendations, career transition guidance (from backend/QA/PM roles into AI), and a research radar tracking frontier papers.

In short: a practical, opinionated, up-to-date reference for building production AI systems and preparing for senior-level AI engineering interviews — free to read and open to community contributions.