GiovanniPasq/agentic-rag-for-dummies

Agentic RAG for Dummies

What it is – A minimal‑code example of a modular, agent‑driven Retrieval‑Augmented Generation (RAG) system built on LangGraph. It shows how to combine hierarchical document indexing, conversation memory, query clarification, and multi‑agent map‑reduce reasoning into a single, extensible pipeline.

---

Core ideas

Step	What happens
Document preparation	PDFs are turned into Markdown, then split twice: large parent chunks (by Markdown headers) for context and small child chunks for precise retrieval.
Conversation memory	A rolling summary + recent turns keep the dialogue coherent without blowing up the LLM context.
Query clarification	The system rewrites ambiguous questions, splits multi‑part queries, and can pause to ask the user for missing details.
Multi‑agent map‑reduce	Each sub‑query runs in its own LangGraph sub‑graph (parallel agents). Each agent searches child chunks, pulls the corresponding parent chunk, self‑corrects if the result is weak, and compresses context to stay within token limits.
Aggregation	All agent answers are merged into a single, fluent response.

---

Main features (as listed in the README)

• Hierarchical indexing – precision from tiny chunks, richness from large parent sections.
• Conversation memory – context persists across turns.
• Human‑in‑the‑loop clarification – the system can ask the user for clarification before proceeding.
• Agent orchestration – LangGraph coordinates the whole workflow.
• Map‑reduce parallelism – multiple sub‑queries are handled concurrently.
• Self‑correction – agents re‑query automatically when confidence is low.
• Context compression – keeps the working memory lean.
• Observability – integrates with Langfuse to log LLM calls, tool usage, and graph execution.
• Evaluation – provides a notebook that scores answers with the RAGAS metric suite.

---

Tech stack

• Python ≥ 3.11
• LangGraph (v1.2+) for graph‑based agent orchestration
• Qdrant vector store with hybrid dense + sparse search (dense via HuggingFace embeddings, sparse via FastEmbed BM25)
• LLM providers – default is Ollama (local model granite4.1:8b), but the code works with OpenAI, Anthropic, Google Gemini, or any LangChain‑compatible chat model.
• Chunky (optional) – a helper toolkit for PDF‑to‑Markdown conversion and chunk inspection.
• Langfuse – optional observability dashboard.
• RAGAS – for quantitative evaluation of retrieval and answer quality.

---

Getting started (high‑level)

1. Install dependencies – pip install -r requirements.txt (includes LangGraph, LangChain, Qdrant client, etc.).
2. Set up a vector DB – the repo creates a local Qdrant instance (client = QdrantClient(path="qdrant_db")).
3. Add documents – place PDFs in docs/, run the notebook or script to convert them to Markdown (pdfs_to_markdowns).
4. Index – the index_documents() function builds parent/child chunks and stores child chunks in Qdrant; parent metadata is saved as JSON files.
5. Choose an LLM – edit the llm = ChatOllama(...) line or swap in ChatOpenAI, ChatAnthropic, etc., after setting the appropriate API key.
6. Run the demo – open notebooks/agentic_rag.ipynb (or launch the Colab badge) and interact with the chat interface.
7. Evaluate – use notebooks/evaluation.ipynb to compute RAGAS scores on the generated answers.

---

Who it’s for

• Learners who want a step‑by‑step notebook that explains each component of an agentic RAG pipeline.
• Developers looking for a ready‑made, modular architecture they can fork and adapt to their own data, LLM, or tooling.
• Researchers interested in experimenting with multi‑agent retrieval, self‑correction, or hybrid dense‑sparse search.

---

Strengths

• Modular – each piece (embedding model, vector store, PDF conversion, agent graph) is isolated, making swaps straightforward.
• Provider‑agnostic – works with any LangChain chat model, not locked to a single vendor.
• Observability – optional Langfuse integration helps debug and monitor LLM calls.
• Evaluation built‑in – RAGAS notebook gives quantitative feedback.
• Human‑in‑the‑loop – query clarification reduces hallucinations caused by ambiguous prompts.

---

Limitations / open points

• Requires a local Qdrant instance and a GPU‑capable environment for the default Ollama model; cloud alternatives need extra setup.
• The hierarchical indexing logic is fairly involved; newcomers may need to read the helper functions to understand the merging/rebalancing steps.
• Self‑correction thresholds (RETRIEVAL_SCORE_THRESHOLD = 0.4) are hard‑coded; tuning may be needed for different corpora.
• The demo focuses on PDF/Markdown sources; other document types would need additional preprocessing code.

---

Quick start snippet

from langchain_ollama import ChatOllama
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_qdrant.fastembed_sparse import FastEmbedSparse
from qdrant_client import QdrantClient

# LLM (local Ollama)
llm = ChatOllama(model="granite4.1:8b", temperature=0, seed=42)

# Embeddings (dense + sparse)
emb_dense = HuggingFaceEmbeddings(model_name="Qwen/Qwen3-Embedding-0.6B")
emb_sparse = FastEmbedSparse(model_name="Qdrant/bm25")

# Vector store
client = QdrantClient(path="qdrant_db")
# … create collection, index docs, then start the LangGraph chat loop as shown in the notebook.

---

Bottom line – Agentic RAG for Dummies is a concrete, runnable reference implementation of a sophisticated RAG system that combines hierarchical chunking, memory, clarification, and multi‑agent reasoning, all built on open‑source tools. It is a solid learning resource and a solid starting point for building production‑grade agentic retrieval pipelines.