lartpang/SAMs-CDConcepts-Eval

Inspiring the Next Generation of Segment Anything Models: Comprehensively Evaluate SAM and SAM 2 with Diverse Prompts Towards Context-Dependent Concepts under Different Scenes

What it solves

This project provides a comprehensive evaluation framework to test how well Segment Anything Models (SAM and SAM 2) handle "context-dependent" (CD) concepts. While these models excel at segmenting common objects like cars or people, they often struggle with concepts that rely on surrounding context, such as medical lesions, industrial defects, or camouflaged objects.

How it works

The framework evaluates SAM and SAM 2 across 11 different CD concepts using 2D and 3D images and videos. It employs three prompting strategies—manual, automatic, and intermediate self-prompting—to test the models' discriminative capabilities. Additionally, it includes prompt robustness testing to simulate real-world scenarios where prompts might be imperfect.

Who it’s for

This tool is designed for computer vision researchers and developers working on image and video segmentation, specifically those looking to understand the performance boundaries of foundation models in specialized domains like medicine and industry.

Highlights

  • Evaluates both SAM and SAM 2 across natural, medical, and industrial scenes.
  • Supports multiple modalities, including 2D/3D images and videos.
  • Implements a unified framework for manual, automatic, and intermediate self-prompting.
  • Includes robustness testing for imperfect prompts to simulate real-world usage.

Related

  • Project
    rameau-fr/MC-Calib

    A generic and robust calibration toolbox for multi-camera systems that supports both perspective and fisheye lenses in hybrid configurations.

  • Project
    naver/anny

    Anny is an open‑source, PyTorch‑based differentiable human‑body mesh model covering all ages. It offers multiple rigs (compact 104‑bone default, full MakeHuman), several mesh topologies (including SMPL‑X and SOMA), and fine‑grained shape, local‑change, and facial‑action parameters. Install via pip, run a quick Python example to output a .ply mesh, or launch an interactive Gradio demo. The library is Apache‑2.0 licensed, builds on MakeHuman assets, and is intended for research/production tasks such as pose estimation, animation, and cross‑age body modeling.

  • Project
    deepfates/memery

    A natural language image search tool that allows users to find specific images in local folders using text or image queries powered by OpenAI's CLIP.

  • Project
    xrdevrob/QuestCameraKit

    A collection of Unity samples for Meta Quest 3/3S that enables mixed reality experiences to detect objects, track QR codes, and use multimodal AI to understand the surroundings.