NirDiamant/Controllable-RAG-Agent

This repository provides an advanced Retrieval-Augmented Generation (RAG) solution for complex question answering. It uses sophisticated graph based algorithm to handle the tasks.

What it solves

This project provides a controllable autonomous agent designed to handle complex, non-trivial questions that standard semantic similarity-based retrieval (simple RAG) cannot solve. It prevents hallucinations by ensuring answers are strictly grounded in the provided data and uses multi-step reasoning to break down intricate queries into manageable tasks.

How it works

The agent uses a deterministic graph as its "brain" to orchestrate a multi-step process:

  1. Data Processing: PDFs are loaded, split into chapters, cleaned, and summarized. Both the full content, chapter summaries, and specific quotes are encoded into vector stores.
  2. Question Processing: The agent anonymizes the question to remove bias, generates a high-level plan, and then de-anonymizes it into specific retrievable or answerable tasks.
  3. Task Execution: The agent decides whether to retrieve information from the vector stores or generate an answer using chain-of-thought reasoning.
  4. Verification: Generated content is verified for grounding in the original context, and the plan is adaptively updated based on new information.
  5. Final Answer: A final response is produced using the accumulated context and chain-of-thought reasoning.

Who it’s for

Developers and AI researchers looking to build production-grade RAG systems that require high controllability, strict grounding to avoid hallucinations, and the ability to solve complex reasoning tasks over custom datasets.

Highlights

  • Deterministic Graph Brain: Uses a structured graph to manage complex reasoning and agentic behavior.
  • Anonymized Planning: Creates general plans without relying on the model's pre-trained knowledge.
  • Hallucination Prevention: Implements content verification and distillation of retrieved content to ensure accuracy.
  • Adaptive Planning: Continuously updates the steps required to solve a query based on new findings.
  • Ragas Evaluation: Integrates Ragas metrics (correctness, faithfulness, relevancy, recall, and similarity) for quality assessment.