datawhalechina/base-llm

What it solves

Many developers jump straight into calling LLM APIs or fine-tuning models without understanding the underlying NLP foundations. This project fills the gap between pure academic theory and shallow API usage by offering a structured path from classic NLP (word vectors, RNNs, Transformers) to modern large language models (BERT, GPT, Llama) — with both theory and hands-on code.

How it works

The tutorial is organized into six main parts: theoretical foundations, practical projects (text classification, named entity recognition), parameter-efficient fine-tuning and quantization (PEFT, LoRA, QLoRA, RLHF), deployment (FastAPI, Docker, Jenkins), LLM safety, and multimodal frontiers. It uses a "question → iterate → refactor" teaching style, showing how simple scripts evolve into industrial-grade implementations. Readers write core architectures like Transformer and Llama2 from scratch, then apply them to real tasks such as fine-tuning Qwen2.5 on private data or deploying a model service with Docker Compose.

Who it’s for

This project is aimed at students, AI algorithm engineers transitioning from traditional ML/DL to LLMs, LLM enthusiasts wanting deep architectural understanding, and researchers needing quick reviews of NLP classics. Prerequisites include solid Python, basic PyTorch, fundamental deep learning knowledge, and basic math (linear algebra, probability, gradient descent).

Highlights

• Systematically covers the evolution from NLP basics to advanced LLM techniques (RLHF, quantization, MoE)
• Hand-writes core architectures (Transformer, Llama2) instead of just calling libraries
• Includes complete practical projects: text classification, NER, fine-tuning Qwen2.5 with QLoRA, and model deployment
• Covers the full engineering lifecycle: training, quantization, inference acceleration, Docker containerization, and service deployment
• Uses abundant illustrations and minimizes heavy math to keep learning accessible