datawhalechina/deep-learning-notes

What it is

A collection of study notes that walk from the very basics of neural networks all the way to modern large language models. The author started these notes to solve a common problem: deep learning material is scattered across papers, blog posts, courses, and code, making it hard to connect individual ideas into one clear picture. The notes aim to tie together formulas, code, and real projects.

Who it's for

People learning deep learning on their own who want a structured path from fundamentals to LLM-scale topics, with working code they can actually run.

What's covered

Topic area	Specifics mentioned in the README
Neural network basics	Network fundamentals, PyTorch, MLPs, optimization algorithms
Classical models	CNNs, regularization, normalization, RNNs, LSTMs, GRUs, Seq2Seq
Attention era	Attention mechanisms, Transformers, FlashAttention
Building LLMs from scratch	MiniGPT implementation, tokenization, GPT-2 from scratch
Real-world LLM engineering	Large-scale training engineering, data processing, scaling laws, model evaluation, post-training, inference
Inference frameworks	vLLM and SGLang

How it's built and published

Written in Quarto Markdown (a plain-text format based on Markdown, good for version control) and rendered as a static website.
A companion repository (jshn9515/dnnl-notebooks) offers the same content as Jupyter Notebooks, kept in sync. You can also convert the source files yourself with quarto convert file.qmd if you install Quarto locally.

Environment and setup

Tested with Python 3.14 and PyTorch 2.14.
Uses Transformers v5 — the README warns that this has significant API differences from v4 (e.g., tokenizers and quantization), so other tutorials written for v4 may need adjusting.
Requires a custom helper library called dnnlpy, which contains custom implementations and utilities used throughout the notes. Many examples won't run without it. Installed via:

uv pip install dnnlpy

or directly from the repository:

uv pip install "git+https://github.com/datawhalechina/deep-learning-notes.git#subdirectory=dnnlpy"

Honesty about AI assistance

The author states that LLMs were used to help draft text, but every draft is reviewed, revised, and double-checked by the author before publication. They acknowledge the notes may still contain omissions or errors and welcome corrections.

How to contribute

You can contribute via Issues or Pull Requests by pointing out errors, suggesting clearer explanations or code comments, proposing structural improvements, or recommending new topics. For larger changes, the author asks you to open an Issue first to discuss.

Influences

Acknowledges Dive into Deep Learning (Zhang, Lipton, Li, Smola) and Professor Hung-yi Lee's lecture series as major influences. The book cover design is inspired by Understanding Deep Learning.

License

The notes themselves: CC BY-NC 4.0 (non-commercial).
The dnnlpy library: MIT.