jiangxinke/Agentic-RAG-R1

Agentic RAG R1 Framework via Reinforcement Learning

What it solves

Agentic RAG-R1 aims to improve the reasoning and search capabilities of language models in Retrieval-Augmented Generation (RAG) systems. It addresses the limitation where models may struggle to decide when to search for information, what to retrieve, and how to integrate that evidence into a final answer.

How it works

The project uses reinforcement learning—specifically the GRPO (Generalized Relevance Policy Optimization) algorithm—to train a base language model to autonomously handle the search and reasoning loop. The system employs an agent memory stack that allows the model to perform actions such as reasoning, backtracking, summarizing, and using search tools (like Wikipedia via ArtSearch). It uses a combined reward model consisting of accuracy, format, and RAG-specific accuracy (via the RAGAS framework) to reinforce successful behaviors.

Who it’s for

Developers and researchers interested in building autonomous AI agents that can perform complex, multi-step retrieval and reasoning tasks, particularly in specialized domains like medicine (as evidenced by the MedQA test set results).

Highlights

  • Reinforcement Learning Integration: Uses GRPO to boost search depth and answer quality.
  • Multi-step Reasoning: Supports reasoning, backtracking, and summarization within a deliberation loop.
  • Custom Tool Integration: Allows users to integrate their own tools and personal RAG datasets.
  • Resource Efficient: Supports models up to 32B parameters using only two A100 GPUs through LoRA and quantization (nf4).
  • Distributed Training: Compatible with DeepSpeed Zero 2 and Zero 3 stages.