Mengqi-Lei/count-anything

Code and implementation guidelines for the paper ✨Counting Anything. Project Page: https://mengqi-lei.github.io/count-anything-projectpage/

📊 Count Anything – Text‑guided object counting

What it is – A research‑grade vision‑language model that, given an image and a natural‑language query (e.g., "airplanes" or "cells with nuclei"), returns the exact locations of every object that matches the query. The number of returned points is the count. It works across very different domains such as everyday scenes, satellite imagery, medical microscopy, agriculture, and microbiology.

Why it matters – Traditional counting methods either output a density map (hard to interpret) or are limited to a fixed set of categories. Count Anything unifies category‑conditioned counting and spatial grounding, and it is trained on the CLOC (Cross‑domain Large‑scale Object Counting) benchmark (≈220 k images, 619 categories, 15 M instances). The paper shows it outperforms recent open‑world models like LocateAnything‑3B on all six domains.

Key ideas

Component What it does
Region‑level Sparse Counter (RSC) Detects and anchors large, sparsely distributed objects (e.g., airplanes, trees).
Pixel‑level Dense Counter (PDC) Predicts dense point clouds for tiny or crowded objects (e.g., cells, grains).
Complementary Count Fusion (CCF) Merges RSC and PDC outputs without extra parameters, removing duplicate points while keeping complementary detections.
Point‑centric supervision All training annotations (boxes, masks, polygons, etc.) are converted to a single point per instance, simplifying the learning signal.

Quick start (for a user who just wants to run the model)

  1. Create the environment
    conda create -n countanything python=3.12 -y
conda activate countanything
pip install -r requirements.txt   # minimal deps; install matching CUDA build of PyTorch if needed
  2. Download the model checkpoint
    • Go to the Hugging Face repo linked in the README and download count_anything.pt.
    • Place it at checkpoints/count_anything.pt.
  3. Run inference on a single image
    from count_anything import CountAnything

model = CountAnything("checkpoints/count_anything.pt")
results = model("path/to/image.jpg", "airplanes")
print("Count:", results[0].count)
results[0].save()   # saves overlay image and JSON with points
    The output folder will be exp/count_anything_inference/<image>__<query>__<timestamp>/.

Training / evaluation (research use)

  • Data – The repository expects the CLOC dataset. Only annotation files are provided; you must download the original source images yourself (license‑restricted). A helper script in data/README.md explains how to assemble the full dataset or request a ready‑made copy from the authors.
  • Weights – Training starts from the public SAM3 backbone. Download sam3.pt from the official SAM3 Hugging Face page and put it under pretrained/.
  • Scripts –
    • train.sh – launches multi‑GPU training (default config config/count_anything_train_cloc.yaml).
    • val.sh – runs validation on the CLOC validation split.
    • test.sh – evaluates on the CLOC test split and writes predictions.json.
  • Configuration – All paths, batch sizes, learning‑rate schedule, etc., are in the three YAML files under config/. Adjust them if you change the dataset layout or hardware.

Dataset – CLOC

  • Scope – 6 visual domains (general scenes, remote sensing, histopathology, cellular microscopy, agriculture, microbiology).
  • Size – ~220 k images, 619 textual categories, 15 M annotated object instances.
  • Annotation format – JSON files list image_path, the textual query, and point (or optional box) annotations for the target category.
  • Version 1.1 – Released July 2026 with a manual clean‑up of noisy labels; see data/README.md for download links.

Where to find more

TL;DR

Count Anything is a cross‑domain, text‑driven object counting model that outputs explicit point locations, trained on a large, multi‑domain benchmark (CLOC). The repo supplies the model checkpoint, training scripts, and clear instructions for inference, training, and dataset preparation. It is a genuine AI research project, not a simple demo or curated list.

Related

  • Project
    zju3dv/MatchAnything

    MatchAnything is a research project that trains a single neural network to find correspondences between images from any visual modality (e.g., RGB, infrared, sketches). Pre‑trained weights are hosted on Hugging Face, and an online demo lets users try cross‑modality matching instantly. The approach aims to replace many specialized matching pipelines with one universal model, useful for robotics, remote sensing, medical imaging, and AR.

  • Project
    facebookresearch/sam3

    SAM 3 (Segment Anything with Concepts) is Meta’s 848 M‑parameter foundation model that lets you prompt an image or video with free‑form text (or visual exemplars) and receive masks, boxes, and scores for *all* matching objects. It combines a DETR‑style detector and a SAM 2‑style tracker, introduces a presence token for fine‑grained prompt discrimination, and is trained on >4 M auto‑annotated concepts. The repo provides installation steps, example notebooks, and a new SA‑CO benchmark (270 K concepts) for evaluation.

  • Dispatch
    Hugging Face Accelerating Document AI

    Hugging Face provides a comprehensive guide to using open-source multimodal models to automate document classification, parsing, and visual question answering for enterprise workflows.

  • Dispatch
    Image search with Hugging Face datasets

    Hugging Face demonstrates how to build an image search application by combining the datasets library, CLIP embeddings, and FAISS for efficient similarity search.

  • Dispatch
    The State of Computer Vision at Hugging Face

    Hugging Face has expanded its ecosystem to support 8 core computer vision tasks, over 3,000 models, and 100+ datasets, integrating both Transformer and convolutional architectures.