walkinglabs/hands-on-modern-rl

What it solves

Reinforcement learning (RL) is often split into separate classical and modern topics, making it hard to see how they connect. This course provides a single path from basic interactions (like CartPole) through policy optimization (PPO) to modern methods for language models (RLHF, DPO, GRPO) and agentic AI, so that learners can understand how the same ideas power both classic games and cutting-edge AI systems.

How it works

The material is organized into two parts: classical and deep RL (chapters 1–12) and modern RL & agentic AI (chapters 13–26). Each chapter follows a problem–method–experiment–reflection rhythm: a concrete task reveals the difficulty, the needed concepts and equations are introduced, runnable code and training curves test the argument, and the chapter ends by examining assumptions and failure modes. The project also provides 14 self-contained chapter notebooks and 12 Studio companions on ModelScope, allowing learners to run experiments in the browser without local setup.

Who it’s for

Designed for learners who want to go from understanding RL basics to applying them to modern AI, including students, researchers, and practitioners. It assumes some programming (Python shown in examples) and a willingness to work through math and code.

Highlights

• A single learning path from CartPole to agentic AI, spanning 26 chapters.
• Equations sit beside implementations, with every tensor accounted for (PPO, DPO, GRPO).
• Real training curves, ablations, and failure signals show when algorithms work and when they don't.
• Hands-on examples include CartPole, Atari Pong, LunarLander, DPO fine-tuning, code generation with RL, a Deep Research agent, and visual reasoning (GeoQA).
• Online Studio environments and notebooks on ModelScope let you train classic RL experiments in a browser.
• Topics cover RLHF, DPO, GRPO, reinforcement learning with verifiable rewards (RLVR), agentic AI, and multimodal/embodied RL.