datawhalechina/zero-to-sglang

zero‑to‑sglang

What it is – An open‑source, bilingual (Chinese & English) tutorial that walks you through the fundamentals of large‑language‑model (LLM) inference and then guides you to implement a tiny version of the SGLang inference engine (called mini‑sglang) from scratch. After you have a working mini‑engine, the material shows how the real SGLang codebase implements advanced optimisations and even teaches you how to contribute a pull‑request.

Why it matters – Inference cost and latency are the biggest bottlenecks for deploying LLMs. Modern inference engines (e.g., SGLang) can handle many more requests per GPU, but there are few hands‑on resources that explain how they work. This project fills that gap by:

• Explaining core concepts such as KV‑Cache, prefill vs. decode, compute‑bound vs. memory‑bound performance.
• Providing a step‑by‑step code‑first construction of a functional mini inference engine.
• Bridging the gap to the production‑grade SGLang code, covering topics like Continuous Batching, Paged KV‑Cache, RadixAttention, quantisation, and distributed attention.
• Guiding readers through profiling, tracing, and the full PR workflow for SGLang.

Who should use it

• Developers or researchers with basic Python and PyTorch knowledge who want to understand LLM serving at a low level.
• Anyone interested in building or optimising inference back‑ends, even if they don’t have a GPU (the early parts run on CPU).
• Contributors who want to help improve SGLang or the surrounding ecosystem.

Prerequisites

• Python programming skills and familiarity with PyTorch.
• Basic linear‑algebra / probability background to follow attention derivations.
• (Optional) CUDA basics – not required for Part 0‑I, helpful for later GPU‑focused sections.

How to get started

# Clone the repo
git clone https://github.com/datawhalechina/zero-to-sglang.git
cd zero-to-sglang
# Follow the “快速开始” section – install the dependencies required for the chapter you are reading.
# Recommended learning path:
# 1️⃣ Read Part I (concepts only) – no GPU needed.
# 2️⃣ Follow Part II to implement mini‑sglang (most code runs on CPU, GPU optional for performance tests).
# 3️⃣ Dive into Part III to compare your implementation with the real SGLang source.
# 4️⃣ Optionally complete Part IV to learn profiling and submit a PR.

Repository layout

• course-material/ – the textbook, split into Chinese (ch/) and English (eng/) directories, further divided into Parts 0‑IV.
• community/ – space for community contributions such as implementation notes, bug reports, and translations.
• docs/.vitepress/ – configuration for the online VitePress site that renders the tutorial.
• Standard project files (README.md, .gitignore).

Current status

• Part 0 (setup & ethics) is complete.
• Part I is largely finished, with a few chapters still being updated.
• Part II (building mini‑sglang) and Part III (advanced inference techniques) are in active development, with most chapters marked “🚧”.
• The repository is open for contributions: fixing content, adding translations, sharing implementation experiences, or helping finish the remaining chapters.

License – Creative Commons BY‑NC‑SA 4.0 (non‑commercial, share‑alike).

---

Zero‑to‑sglang is a collaborative effort by Datawhale and RadixArk (the company behind SGLang), aiming to make LLM inference engineering accessible to a broader audience.