NovaSearch-Team/RAG-Retrieval

Unify Efficient Fine-tuning of RAG Retrieval, including Embedding, ColBERT, ReRanker.

What it solves

RAG-Retrieval provides a comprehensive toolkit for optimizing the retrieval stage of Retrieval-Augmented Generation (RAG) pipelines. It addresses the difficulty of training, fine-tuning, and deploying different types of retrieval models—such as embedding models, late interaction models, and rerankers—using a unified framework.

How it works

The project implements a full lifecycle for retrieval models through three primary functions:

  • Training and Fine-tuning: Supports end-to-end fine-tuning of BERT-based and LLM-based embedding models, ColBERT-style late interaction models, and rerankers. It is compatible with popular open-source models like BGE, BCE, and GTE.
  • Distillation: Allows users to transfer knowledge from larger, high-performance models (like large LLM-based rerankers) into smaller, more efficient models (such as BERT-base or 0.5B LLMs).
  • Inference: Includes a lightweight Python library (rag-retrieval) that provides a unified interface to call various reranker models, handling long documents via truncation or splitting to optimize scoring.

Who it’s for

Developers and researchers working on RAG systems who need to improve the accuracy and efficiency of their retrieval components through custom training or model compression.

Highlights

  • Unified Framework: One codebase for embedding, late interaction, and reranker models.
  • Model Distillation: Built-in support for distilling large models into smaller, deployable versions.
  • Advanced Algorithms: Implements the MRL algorithm for reducing output vector dimensionality and the Stella distillation method.
  • Scalable Training: Integrated support for multi-GPU strategies using DeepSpeed and FSDP.