datawhalechina/easy-rl

What it solves

Reinforcement learning (RL) is often taught through dense math and scattered resources, which makes it hard for newcomers — especially Chinese-speaking learners — to get started. This project assembles a complete, beginner-friendly Chinese tutorial that combines well-known lecture notes (Li Hongyi's Deep Reinforcement Learning, Zhou Bolei's RL Outline, Li Kejiao's RL course, and other classic materials) into one coherent book, known as the "Mushroom Book." It also provides exercises, code notebooks, and paper readings so learners can go from theory to practice in one place.

How it works

The tutorial is structured into 13+ chapters that cover core RL topics step by step: starting with fundamentals, Markov decision processes, tabular methods, then policy gradients, PPO, DQN and its variants, actor-critic, sparse rewards, imitation learning, DDPG, and even a deep dive into AlphaStar. Each chapter pairs with exercises and, where relevant, practical code notebooks (e.g., Q-learning, Sarsa, Monte Carlo, Policy Gradient, PPO, DQN, Double DQN, Dueling DQN, PER DQN, Noisy DQN, A2C, DDPG). The content is openly available online (readable at any time) and also released as a PDF for download. A printed book version exists (published by Posts & Telecom Press), refined by editors beyond the initial draft.

Who it's for

This project is designed for anyone who wants to learn reinforcement learning in Chinese — whether you are a student, a practitioner, or a self-learner. The humorous and example-driven teaching style of Li Hongyi's lectures, plus the structured exercises and code, make it accessible even if you are new to RL. The book's nickname "Mushroom Book" reflects the hope that readers will gain energy and power from it, just like Mario after eating a mushroom.

Highlights

• Structured around well-known Chinese RL lecture content, making classic theory approachable.
• Includes exercises for most chapters, with an errata page for corrections.
• Provides runnable code notebooks for many algorithms, plus two project walkthroughs (Q-learning and DQN, DDPG) and links to JoyRL (offline and online versions).
• Contains a section of classic RL paper interpretations, including an ICLR'25 Oral on RL playing Minecraft and a visual RL paper list.
• Available as a free online version, downloadable PDF (including a compressed low-resolution edition), and a polished printed book (with purchase links).