castorini/rank_llm

RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking.

What it solves

RankLLM provides a comprehensive suite of rerankers to improve the precision of search and retrieval systems. It allows users to take a set of candidates retrieved by a first-stage retriever (like BM25) and re-order them more accurately using advanced language models, ensuring the most relevant documents are at the top.

How it works

The project implements several reranking strategies:

  • Pointwise: Models like MonoT5 and MonoELECTRA score documents individually.
  • Pairwise: Models like DuoT5 compare documents in pairs.
  • Listwise: Models like RankZephyr and RankVicuna (including proprietary options like RankGPT and RankGemini) process multiple documents at once to determine the best order.

It supports various backends including vLLM for open-source models and APIs for hosted providers. To increase efficiency, it can perform reranking using only the first-token logits of a model.

Who it’s for

This tool is designed for developers and researchers working on information retrieval (IR) and RAG pipelines who need to high-quality reranking to improve their retrieval accuracy.

Highlights

  • Diverse Model Zoo: Includes a wide range of pointwise, pairwise, and listwise models (e.g., RankZephyr, LiT5, MonoT5).
  • Flexible Integration: Supports both local Hugging Face models and cloud APIs (OpenAI, OpenRouter, Gemini).
  • End-to-End Workflow: Provides a CLI and Python API for retrieval, reranking, and evaluation (using metrics like NDCG).
  • Efficiency Optimizations: Supports vLLM for faster inference and first-token logit reranking to reduce latency.

Related

  • Project
    Raudaschl/rag-fusion

    A search methodology and implementation that uses LLM-generated query variations and Reciprocal Rank Fusion to improve document retrieval, especially when user terminology differs from the indexed text.

  • Project
    DataScienceUIBK/Rankify

    A comprehensive Python toolkit for retrieval, re-ranking, and RAG, providing a unified interface to experiment with and deploy state-of-the-art models.

  • Project
    srbhr/Resume-Matcher

    An AI-powered tool that tailors resumes and cover letters to specific job descriptions using LLMs to improve match scores and keyword alignment.

  • Project
    FlagOpen/FlagEmbedding

    A comprehensive retrieval toolkit for search and RAG that provides state-of-the-art embedding models and rerankers for text and multimodal data.

  • Project
    Farzad-R/LLM-Zero-to-Hundred

    A collection of LLM projects and tutorials covering RAG, agentic workflows, multimodal chatbots, and fine-tuning pipelines to move from basic to advanced AI application development.