datawhalechina/all-in-rag

What it is

All-in-RAG is a free, open-source tutorial (written mainly in Chinese, with an English version available) that teaches developers how to build RAG applications — systems that let a large language model answer questions by first pulling relevant information from your own documents or knowledge base, then generating a response grounded in that retrieved content. The project calls itself a "full-stack guide": it walks you from basic concepts through to production-ready systems.

Who it's for

The README names four target audiences:

• Developers with some Python experience who want to learn RAG
• AI engineers seeking a structured way to learn RAG
• Product developers building intelligent Q&A systems
• Researchers interested in retrieval-augmented generation

Before starting, the project assumes you can: use basic Python syntax and common libraries, run simple Docker commands, and operate a basic Linux command line. Knowing basic LLM concepts is recommended but not required.

What it covers

The tutorial is organized into 10 chapters across 5 parts:

1. RAG foundations — what RAG is, how to set up your environment, and a "four-step" quick-start to build your first RAG system. Also covers data preparation: loading documents in various formats and splitting text into manageable chunks.

2. Index construction & optimization — turning text into vector embeddings (including multimodal embeddings that handle both text and images), building and using vector databases (with a hands-on practice using Milvus), and tuning index performance.

3. Advanced retrieval — hybrid search (combining dense and sparse retrieval), intelligent query construction, converting natural language to SQL queries (Text2SQL), query rewriting and routing strategies, and other advanced retrieval algorithms.

4. Generation & evaluation — formatting the model's output and, importantly, how to evaluate a RAG system: methodology, tools, and metrics.

5. Advanced applications & practice — an extra chapter on knowledge-graph-based RAG, followed by two full hands-on projects. The first project (Chapter 8) builds a complete RAG application — environment setup, data preparation, index building, retrieval, and generation. Chapter 9 is an optional "Graph RAG" optimization of that same project, and Chapter 10 (a second project) is still being planned. There are also community-contributed "Extra-chapter" topics, such as a simple Neo4J application and a multimodal embedding practice.

What makes it stand out

The README lists five selling points:

• Systematic learning path — from basics to advanced, forming a complete mental model of RAG
• Theory + practice combined — every chapter includes both explanation and code you can run
• Multimodal support — not just text search; it covers embedding and retrieving images alongside text
• Engineering-focused — pays attention to real-world concerns like performance optimization and system evaluation
• Multiple hands-on projects — from beginner to advanced, to consolidate what you learn

How the repository is laid out

The repo contains: docs/ (the tutorial documents), code/ (code examples), data/ (sample data), models/ (pretrained models), and Extra-chapter/ (community contributions).

Community & contribution

The project is maintained by Datawhale, a Chinese open-source learning community, with a lead contributor plus several others. The README invites contributions of several kinds: bug reports (via Issues), ideas (via Discussions), and — currently limited to the Extra-chapter section — new standalone topics from the community, following a review process.

License

All-in-RAG is released under the CC BY-NC-SA 4.0 license (Creative Commons Attribution-NonCommercial-ShareAlike). In plain terms: you may share and adapt the material, but you must credit the source, may not use it commercially, and any derivative work must be released under the same license.

Bottom line

If you want to learn RAG in a structured, practical way — and are comfortable with Python and basic DevOps tools — this is a complete, free resource that takes you from "what is RAG?" to "I built and evaluated my own RAG system."