texttron/tevatron

What it solves

Tevatron provides a unified toolkit for building and training large-scale neural retrievers. It addresses the challenge of training billion-scale LLM-based retrievers across different languages and modalities (text and images) while maintaining efficiency in terms of compute and memory.

How it works

The toolkit allows users to fine-tune pre-trained models (such as Mistral-7B, BGE-Embedding, or Instruct-E5) using Parameter-Efficient Fine-Tuning (PEFT) via LoRA. It supports both PyTorch (GPU) and JAX (TPU/GPU) backends. To optimize performance, it integrates with high-efficiency libraries like vLLM, DeepSpeed, FlashAttention, and GradCache. The workflow typically involves training a retriever on query-passage pairs, encoding the corpus into embeddings, and performing a similarity search to retrieve relevant documents.

Who it’s for

AI researchers and engineers working on dense retrieval, open-domain question answering, and multi-modal search systems who need to scale their models to billions of documents.

Highlights

• Multi-Backend Support: Compatible with PyTorch for GPUs and JAX for TPUs and GPUs.
• Scale and Modality: Supports billion-scale retrieval across multiple languages and both text and image modalities.
• Efficiency: Integrates LoRA for parameter-efficient tuning and DeepSpeed/FlashAttention for faster training.
• Ready-to-use Data: Includes self-contained HuggingFace datasets for multi-modal and multilingual retrieval tasks.