PrithivirajDamodaran/FlashRank

Lite & Super-fast re-ranking for your search & retrieval pipelines. Supports SoTA Listwise and Pairwise reranking based on LLMs and cross-encoders and more. Created by Prithivi Da, open for PRs & Collaborations.

What it solves

FlashRank is a lightweight Python library designed to improve the quality of search and retrieval results by re-ranking them before they are passed to a Large Language Model (LLM). It addresses the overhead and cost associated with using heavy re-ranking models, providing a fast, CPU-based alternative that maintains competitive performance.

How it works

The library provides a set of state-of-the-art (SoTA) rerankers, including pairwise/pointwise cross-encoders and listwise LLM-based rerankers. It allows users to integrate these models into existing lexical, semantic, or hybrid search pipelines. The system is optimized for speed and memory footprint, running on CPU without requiring Torch or Transformers libraries.

Who it’s for

Developers building search engines, RAG (Retrieval-Augmented Generation) pipelines, and serverless deployments (like AWS Lambda) who need a high-performance, low-latency re-ranking step without the need for specialized hardware.

Highlights

  • Ultra-lite footprint: Includes a default model as small as ~4MB, allowing for fast cold starts in serverless environments.
  • CPU-optimized: Runs on CPU without dependencies on Torch or Transformers.
  • Diverse model support: Supports a variety of models including TinyBERT, MiniLM, RankT5, and 4-bit quantized GGUF models like Rank Zephyr.
  • Flexible integration: Works with any search pipeline, including lexical, semantic (VectorDBs), and hybrid search.