carrot2/carrot2

Carrot2: Text Clustering Algorithms and Applications

What it solves

Carrot2 is a programming library designed to organize large sets of text documents into groups of related items. It solves the problem of making search results or document collections more navigable by automatically discovering clusters of related content and assigning them descriptive labels.

How it works

The library analyzes text documents to identify patterns and group them together based on similarity. It provides a Java-based API for developers to integrate clustering into their applications, and also offers a Document Clustering Server (DCS) with a REST API for integration with other programming languages. It can also be implemented as a plugin for search engines like Apache Solr and Elasticsearch.

Who it’s for

It is intended for software developers who need to add automated text clustering and categorization capabilities to their software, particularly those building search interfaces or document retrieval systems.

Highlights

  • Automated discovery of document groups based on text content.
  • Automatic generation of short key terms or phrases to label clusters.
  • REST API available via the Document Clustering Server (DCS).
  • Native plugins for Apache Solr and Elasticsearch.

Related

  • Project
    Marker-Inc-Korea/AutoRAG

    A self-evolving librarian agent that searches across local documents and external datasources to provide curated, synthesized answers instead of raw search results.

  • Project
    huggingface/datatrove

    A library for processing, filtering, and deduplicating massive text datasets at scale, specifically optimized for LLM training data preparation.

  • Project
    deepdoctection/deepdoctection

    A Python library for document understanding that orchestrates layout analysis, OCR, and token classification to extract structured data from PDFs and scans.

  • Project
    GiulioRossetti/cdlib

    A meta-library for community detection in complex networks that provides a standardized way to implement, evaluate, and visualize community discovery algorithms.

  • Project
    facebookresearch/stopes

    A library for preparing machine translation data, providing tools for cleaning monolingual data, mining bitext, and running knowledge distillation pipelines.