castorini/anserini

Anserini is a Lucene toolkit for reproducible information retrieval research

What it solves

Anserini addresses the gap between academic information retrieval (IR) research and the practical implementation of real-world search applications. It specifically focuses on providing a toolkit that ensures IR research is reproducible, preventing the common issue where academic baselines are difficult to replicate in practice.

How it works

Anserini is built on top of Apache Lucene, leveraging its powerful indexing and search capabilities to provide a standardized environment for retrieval experiments. It allows researchers to perform reproductions using either prebuilt indexes or raw document collections. For those who prefer Python, most of its features are accessible via the Pyserini interface.

Who it’s for

It is designed for researchers and developers in the field of information retrieval who need a reliable, reproducible way to build and test search baselines and experimental results.

Highlights

  • Built on Apache Lucene for industry-standard performance.
  • Supports reproducible IR research through standardized toolkits.
  • Offers multiple setup options, including a self-contained fatjar for quick starts and a full development environment.
  • Provides a Python interface via Pyserini for easier accessibility.

Related

  • Project
    castorini/pyserini

    Pyserini is a Python toolkit that wraps the Anserini Lucene search engine and FAISS to provide easy, reproducible first‑stage retrieval (sparse, dense, or hybrid) for information‑retrieval research and RAG pipelines. It includes pre‑built indexes, evaluation scripts, and a REST server, and can be installed via `pip`.

  • Project
    terrier-org/pyterrier

    A framework for building and evaluating information retrieval pipelines, supporting sparse and dense retrieval, neural reranking, and RAG use-cases.

  • Project
    DataScienceUIBK/Rankify

    A comprehensive Python toolkit for retrieval, re-ranking, and RAG, providing a unified interface to experiment with and deploy state-of-the-art models.

  • Project
    PrithivirajDamodaran/FlashRank

    A lightweight, CPU-based Python library for re-ranking search results using state-of-the-art cross-encoders and LLMs to improve retrieval precision for LLM pipelines.

  • Project
    nipy/nipype

    A Python project that provides a uniform interface to various neuroimaging software packages, allowing researchers to create integrated and reproducible processing pipelines.