zju3dv/MatchAnything

Code for "MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training", TPAMI 2026.

MatchAnything – Universal Cross‑Modality Image Matching

What it does

  • Learns a single model that can find correspondences between images that come from different visual modalities (e.g., RGB photos, infrared, sketches, depth maps, etc.).
  • The model is pre‑trained on a very large, diverse dataset so it can “match anything” without task‑specific fine‑tuning.

Why it matters

  • Traditional image‑matching pipelines (SIFT, SuperPoint, etc.) work well only when the two images share the same appearance domain. Cross‑modality matching is needed in robotics, remote sensing, medical imaging, and AR where sensors produce heterogeneous data.
  • A universal matcher reduces the engineering effort of building separate pipelines for each modality pair.

Key components (as described in the README)

  1. Large‑scale pre‑training – The authors train the network on a massive collection of paired images across many modalities, letting it learn modality‑agnostic features.
  2. Universal matcher architecture – While the README does not detail the network, the name suggests a single backbone that outputs dense descriptors usable for any modality pair.
  3. Pre‑trained weights – Ready‑to‑use models are hosted on Hugging Face, so users can run inference immediately.
  4. Demo – An interactive Hugging Face Space lets you upload two images of different types and see the matching results.

How to get started

  1. Visit the Hugging Face demo linked in the README to try the model without installing anything.
  2. Clone the repository and follow the (future) training instructions once they are released.
  3. Use the provided pre‑trained weights (available in the third_party/MatchAnything folder on Hugging Face) to run inference on your own image pairs.

Intended users

  • Researchers and engineers who need a plug‑and‑play solution for cross‑modality correspondence.
  • Developers building perception stacks for robots, drones, or AR devices that fuse data from multiple sensors.

Paper & citation

  • The method is described in the paper MatchAnything: Universal Cross‑Modality Image Matching with Large‑Scale Pre‑Training (TPAMI 2026). The README provides a BibTeX entry for citation.

Current status

  • Pre‑trained models are publicly available.
  • Training code is promised for a later release.

All information above is taken directly from the repository’s README; no additional features have been inferred.

Related

  • Project
    facebookresearch/map-anything

    An open-source research framework for universal metric 3D reconstruction that uses a single transformer model to handle multiple 3D tasks from various input combinations.

  • Project
    gmberton/vismatch

    A unified API for quickly testing and comparing over 50 image matching models, automating weight downloads and providing a consistent interface for matching and keypoint extraction.

  • Project
    Mengqi-Lei/count-anything

    Count Anything is a research model that counts objects in images based on a natural‑language query. It works across six domains (scenes, satellite, medical, etc.) by combining a sparse region counter and a dense pixel counter, merging their results with a parameter‑free fusion step. The repo provides a pretrained checkpoint, training/evaluation scripts, and instructions for preparing the large CLOC dataset.

  • Dispatch
    Image Similarity with Hugging Face Datasets and Transformers

    Hugging Face demonstrates how to build an image similarity system using the Transformers and Datasets libraries by computing dense vector embeddings and measuring cosine similarity.

  • Dispatch
    Modelmap: Interactive Visualizations of Hugging Face Model Architectures

    Modelmap is a tool that generates interactive, animated network maps of any Hugging Face model architecture without requiring the user to download model weights.