tianrun-chen/SAM-Adapter-PyTorch

Adapting Meta AI's Segment Anything to Downstream Tasks with Adapters and Prompts

What it solves

SAM-Adapter addresses the performance drops seen in the Segment Anything Model (SAM) when applied to challenging "underperformed scenes," such as camouflaged object detection, shadow detection, and medical image segmentation (e.g., polyp segmentation).

How it works

It adapts the pre-trained SAM backbone (supporting SAM, SAM2, and SAM3) to downstream tasks by incorporating an adapter mechanism. This allows the model to maintain the general segmentation capabilities of SAM while specializing in specific, difficult visual environments where the original model might fail.

Who it’s for

Computer vision researchers and developers working on specialized image segmentation tasks, particularly those involving low contrast, camouflage, or medical imaging.

Highlights

  • Support for SAM, SAM2, and SAM3 backbones.
  • Proven effectiveness on camouflaged object detection (COD10K, CAMO, CHAMELEON) and shadow detection (ISTD).
  • Applicable to medical imaging tasks like polyp segmentation (Kvasir dataset).
  • Provides configuration files for different ViT versions (ViT-H, ViT-L, ViT-B) to accommodate different GPU memory constraints.