SimonZeng7108/efficientsam3
EfficientSAM3 compresses SAM3 into lightweight, edge-friendly models via progressive knowledge distillation for fast promptable concept segmentation and tracking.
What it solves
EfficientSAM3 addresses the high computational cost of the Segment Anything Model 3 (SAM3), making it more accessible for deployment on resource-constrained devices. It reduces the model size of the vision and text encoders while maintaining competitive performance on segmentation benchmarks.
How it works
The project uses progressive hierarchical knowledge distillation from SAM1, 2, and 3 to compress the original heavy encoders into lightweight student models. It offers two main configurations:
- EfficientSAM3 Full Models: Replaces both the vision encoder (using EfficientViT, RepViT, or TinyViT) and the text encoder (using MobileCLIP) to achieve up to 90% reduction in parameter count compared to ImageSAM3.
- SAM3-LiteText: Keeps the original SAM3 vision encoder but replaces the heavy text encoder with a lightweight MobileCLIP variant, reducing the text encoder size by 88%.
Who it’s for
Researchers and developers building vision-language segmentation tools that require high efficiency and low latency, particularly for mobile or edge device deployment.
Highlights
- Massive Compression: Reduces model size by up to 90% (e.g., EV-M model is 89.2M params vs 861.5M for ImageSAM3).
- Multiple Backbone Options: Supports various lightweight vision backbones including RepViT, TinyViT, and EfficientViT.
- Multiple Text Encoders: Integrates distilled MobileCLIP variants for efficient text-prompted segmentation.
- End-to-End Fine-tuning: Includes models fine-tuned on SA1B data and SACap labels for improved performance.