lmcinnes/pynndescent

A Python nearest neighbor descent for approximate nearest neighbors

What it solves

PyNNDescent solves the problem of efficiently finding the nearest neighbors of a data point in high-dimensional spaces. It provides a fast, approximate nearest neighbor (ANN) search that maintains high accuracy (80%-100%) while avoiding the computational cost of an exact search.

How it works

The library implements the Nearest Neighbor Descent algorithm for constructing k-neighbor graphs. To improve performance, it uses random projection trees for initialization and performs graph diversification by pruning the longest edges of any triangles in the graph. It supports a wide variety of distance metrics, including Minkowski, angular, correlation, and probability metrics, as well as custom user-defined metrics.

Who it’s for

It is designed for developers and data scientists who need a high-performance ANN search library that is easy to install (via pip or conda) and integrates seamlessly with Scikit-learn.

Highlights

  • Fast approximate nearest neighbor queries with competitive performance on ANN benchmarks.
  • Support for a vast array of built-in distance metrics (e.g., cosine, Euclidean, Hamming, Wasserstein).
  • Ability to use custom user-defined distance metrics without sacrificing performance.
  • Simple interface for index construction and querying, similar to Scikit-learn's KDTrees and BallTrees.

Related

  • Project
    microsoft/SPTAG

    A library for fast, large-scale approximate nearest neighbor search using space partition trees and relative neighborhood graphs.

  • Project
    NGT-labs/NGT

    A high-speed approximate nearest neighbor search library for high-dimensional vector data, supporting graph, tree, and quantized indexing methods.

  • Project
    datastax/jvector

    A graph-based approximate nearest neighbor search index that combines HNSW and DiskANN to enable fast, memory-efficient vector search over massive datasets.

  • Project
    microsoft/DiskANN

    A composable vector indexing library that enables scalable and cost-effective nearest-neighbor search across various database storage backends.

  • Project
    jlblancoc/nanoflann

    A high-performance, header-only C++ library for building KD-trees and performing fast nearest-neighbor searches on point clouds and N-dimensional data.