NirDiamant/Controllable-RAG-Agent

This repository provides an advanced Retrieval-Augmented Generation (RAG) solution for complex question answering. It uses sophisticated graph based algorithm to handle the tasks.

Controllable-RAG-Agent

A retrieval-augmented generation (RAG) system designed for complex, non-trivial questions that simple similarity search can't handle. Instead of just pulling the most similar text snippets and generating an answer, this project builds a deterministic decision graph that acts as the agent's 'brain'.

Core Idea

Traditional RAG does: find similar text → generate answer. This project does much more:

  1. Load & prepare: PDFs are split into chapters, cleaned, and summarized by an LLM.
  2. Index: Book quotes and chapter summaries are encoded into vector stores (FAISS) for fast retrieval.
  3. Plan: The question is anonymized (named entities replaced with placeholders) so the planner reasons generally without bias. A high-level plan is generated, then de-anonymized and broken into specific tasks.
  4. Execute: Each task either retrieves information (then distills it down to what matters) or answers directly using chain-of-thought reasoning.
  5. Verify & re-plan: The agent verifies its generated content is grounded in the retrieved context (not from the model's pre-trained memory), and updates its plan if needed.
  6. Final answer: Produced from the accumulated, verified context.

Why This Is Different

  • Hallucination prevention: Answers must be traceable to the provided data. Verification loops check this, echoing ideas from the Self-RAG paper.
  • Multi-step reasoning: Breaks complex queries into sub-tasks (e.g., for "How did the protagonist defeat the villain's assistant?", it first identifies protagonist, villain, assistant, then finds confrontations, then deduces the reason).
  • Adaptive planning: The plan is a living document, updated as new information arrives (inspired by Plan-and-Solve and babyagi).

What's Included

  • A step-by-step tutorial notebook (sophisticated_rag_agent_harry_potter.ipynb) using the first Harry Potter book as a test case
  • A Streamlit app (simulate_agent.py) for real-time visualization of the agent's reasoning
  • Docker support (docker-compose up --build)
  • Evaluation via Ragas metrics: answer correctness, faithfulness, answer relevancy, context recall, answer similarity

Technologies

  • LangChain (for orchestration)
  • FAISS (vector store)
  • Streamlit (visualization)
  • Ragas (evaluation)
  • Flexible LLM support: OpenAI (default gpt-4o), Groq, or any LangChain-compatible provider

Getting Started

Requires Python 3.8+ and an LLM API key. Clone the repo, add your key to a .env file, install with pip install -r requirements.txt, then open the notebook or run the Streamlit app. Full README instructions are in the project.

License

Apache 2.0 — open source, free commercial use.

Author

Nir Diamant, who also maintains related repositories (RAG Techniques, GenAI Agents, Agents Towards Production) and a newsletter.

Related

  • Project
    iMoonLab/Hyper-RAG

    A hypergraph-driven Retrieval-Augmented Generation (RAG) framework that reduces LLM hallucinations by capturing complex, beyond-pairwise relationships in domain-specific knowledge.

  • Project
    java-up-up/nexus-agent

    An enterprise-grade AI agent platform that implements a complete loop from document governance and hybrid RAG retrieval to ReAct-based agentic reasoning and MCP tool integration.

  • Project
    aigc-apps/PAI-RAG

    An open-source Agentic RAG framework that combines multimodal understanding, ReAct-based agents, and enterprise-grade knowledge management for complex intelligent Q&A.

  • Project
    yolo-hyl/medical-rag

    A professional medical RAG system using LangChain and Milvus that employs hybrid retrieval and intelligent agents to provide high-accuracy medical question-answering.

  • Project
    Alibaba-NLP/VRAG

    A multi-turn multimodal RAG framework that uses agentic reinforcement learning to enable VLMs to reason and retrieve information across text, images, and videos.