bojieli/ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
| Field | What the README actually says |
|---|---|
| What it is | An open-source book, titled “深入理解 AI Infra：量化分析与系统设计” (roughly: Understanding AI Infra in Depth: Quantitative Analysis and System Design). It is the companion volume to the author's earlier book 深入理解 AI Agent (Understanding AI Agent), which has over 50,000 GitHub stars. The manuscript is still a draft and being revised continuously. |
| Author | Bojie Li (李博杰, GitHub: @bojieli). Owns the repo; author of the original Simplified Chinese text. The book is licensed under Apache License 2.0 (original text, figures, and code). Copyright © 2026 Bojie Li. |
| Why it exists | The author's stated motivation: building model-based applications means understanding the infrastructure underneath them, just as software engineers learn operating systems and compilers even if they never build one — because every memory allocation, file read, and function call has resource and time costs. For LLM apps, latency and cost differences change products and business models. The author's framing: the LLM has become the “operating system” of the LLM era, and AI Infra is its “computer architecture.” The book aims to fill a gap — there was no book deriving system design quantitatively from hardware constraints and model architecture. The author also cites a PhD advisor's advice: optimize up to the physical limit; the book first computes the hardware's theoretical upper bound, then compares against measured results. |
| Core method (the “five questions”) | Throughout the book the recurring theme is data movement (数据搬移). Every case repeatedly asks five questions: what to move, how much, how many times, through where (which path), and who must wait for it — i.e., what is moved, its quantity, the number of transfers, the route, and the dependency/waits. The method throughout is “derive design from constraints”: state the task and quality requirements, list compute/storage/communication/dependency needs, then do order-of-magnitude estimates against hardware capacity, bandwidth, and compute power. The README stresses this is error-prone: forgetting KV cache, checking peak compute but not bandwidth, splitting work across GPUs while missing inter-card communication — any miss can skew conclusions by several times or orders of magnitude. |
| Contents — 12 chapters | 1) Intro to AI Infra — how much memory, compute, and data I/O a single generation needs. 2) Model architecture — how attention, history state, and expert structures change system requirements. 3) Inference & training workloads — how task phases, arrival patterns, and state lifetime affect resource needs. 4) Accelerator architecture — trade-offs among compute, storage, bandwidth, power, cost. 5) Operators & runtime — how fusion, reuse, concurrency, scheduling cut execution overhead. 6) Super-nodes — how multi-device collaboration balances capacity, throughput, sync cost. 7) Data-center networks — how bandwidth, communication patterns, congestion affect compute efficiency. 8) Inference optimization — when batching, KV management, offloading, speculative decoding help. 9) Distributed inference — how to place compute and state, handle scaling and recovery. 10) Training systems — how to arrange memory, communication, recomputation for efficiency. 11) Resource scheduling & runtime env — how model serving and tool environments share resources and reduce waiting. 12) Edge-cloud-device collaboration — where to put tasks (local, edge, cloud) balancing quality, latency, cost. |
| Who it's for | People who have called model APIs or run models locally and want to understand why models are slow and how to cut cost; also engineers in systems, networking, chips, plus researchers and students in those areas. Prerequisites: some Python, linear algebra, and computer systems basics. The README suggests reading chapters 1–3 first, then chapter-specific paths: app/inference readers → ch. 8–9 then 11–12; systems/network readers → ch. 5–7 then 9–10; chip/architecture readers → ch. 4–7 with later chapters' worked examples. |
| Companion calculation tool | A command-line tool in calculations/ to recompute the book's numbers or estimate resource needs with different models and inputs. Static calculations only need Python 3.10+ standard library — no GPU or model weights required. Example commands: python3 calculations/calc.py models (list supported models); python3 calculations/calc.py forward --model qwen3-8b --tokens 8192 --format md (estimate per-operator resource needs for Qwen3-8B at 8192-token prefill). Results index and model configs are included. |
| Companion experiments | Organized in experiments/chXX/XX-YY/ per chapter; each has run method, input conditions, and result explanation. GPU-required experiments note hardware and dependency requirements; readers without that hardware can still read existing records. To reproduce or cite, the README says to note the manuscript version, model, hardware, and input parameters. |
| Repository size & Git LFS | Large files (papers, big inputs, measurement records) are stored via Git LFS, about 20 GB total. The repo's .lfsconfig defaults to skipping all LFS files — clones get only pointer files; the main text, figures, and static calculations don't need them. To fetch only specific experiments: git lfs pull --include="experiments/ch05/05-01/**" --exclude="". |
| Available formats & languages | Original: Simplified Chinese (author's own). Community translations: English (by @tg1482), Traditional Chinese (by @edward821220), Russian (community edition organized by @ilkruglov). Each version is offered as PDF, EPUB, and an online reading site. Online site offers chapter navigation, full-text search, dark mode, and language switching at top right (Simplified Chinese at root; en/, zh-tw/, ru/ subpaths). PDFs are typeset with XeLaTeX; EPUB reflows for phones/readers. Releases are auto-built and published on every push to main — the download links always point to the latest. |
| Building locally | Website: Python 3.10+, create venv, pip install -r website/requirements.txt, then python scripts/build_site.py, optionally python scripts/check_site.py, preview with --serve at http://127.0.0.1:8000. Full PDF: needs Pandoc, XeLaTeX, fonts — scripts book/build_pdf.sh (zh), book-en/build_pdf.sh (en), book-zh-tw/build_pdf.sh (zh-TW). EPUB: python3 book/build_epub.py --edition zh (edition can be zh, en, zh-tw, ru) needs only Pandoc and Poppler. Translation image assets live in Git LFS — fetch with git lfs pull --include="book-en/images/**,book-zh-tw/images/**" --exclude="" before building. CI (GitHub Actions) checks PR builds of site and PDF/EPUB in all languages. |
| Contribution | Welcomed; the README lists: fix factual errors (wrong numbers, units/orders of magnitude, mismatched citations, bad formulas/figures) via erratum issue template or PR; rewrite unclear passages (skipped derivation steps, undefined terms at first mention, hard-to-follow examples); add missing important mechanisms/architectures/trade-offs; fix bugs in the companion code (experiments/ and calculations/ CLI); improve the web version (layout, navigation, search, mobile experience); translate into more languages (English and Traditional Chinese already exist as reference structures; open an issue before starting to coordinate). The single source of truth for the text is the Markdown in manuscripts/ — web and PDF are built from it, so changes go there. |
| Structure | Repo directories: manuscripts/ (preface, 12 chapters, figures, plotting scripts); experiments/ (per-chapter experiments and records); calculations/ (calculation tool, fixed inputs, recomputed results); case-studies/ (model/hardware/system analyses, indexed by chapter); references/ (citations, source lists, version snapshots); research/ (supporting investigations and revision records); book/ (PDF template, build/check tools); book-en/, book-zh-tw/ (translation editions with own scripts); website/, scripts/ (site assets and build/check); archive/ (historical outlines, review, coordination notes). |
Related
- Project
- Project
- Project
- Project