OschAI/VisioFirm

What it solves

VisioFirm is an AI-powered image and video annotation tool that reduces the manual effort required to label datasets for computer vision. It eliminates the need to start labeling from scratch by providing semi-automated pre-annotations for various tasks, significantly speeding up the process for researchers and ML engineers.

How it works

The tool provides a browser-based interactive canvas where users can refine AI-generated labels. It integrates several state-of-the-art models to automate the initial labeling phase:

• Detection & Grounding: Uses YOLO (v5-v12), Grounding DINO (for zero-shot detection via text prompts), and YOLOv8-world for open-vocabulary pre-annotation.
• Segmentation: Employs SAM2 for precise segmentation and click-to-segment functionality.
• Classification: Uses OpenAI CLIP to suggest image classes.
• Video Tracking: The "SmartPropagator" uses SAM2 to propagate labels across frames, while other options include OpenCV trackers and linear interpolation.
• Cross-Domain Workflow: Users can generate segmentation masks from detection models or bounding boxes from segmentation models.

Who it’s for

It is designed for researchers, data scientists, and machine learning engineers who need to create high-quality annotated datasets for training computer vision models (such as YOLO or SAM).

Highlights

• Multi-Task Support: Handles classification, axis-aligned bounding boxes, oriented bounding boxes (OBB), and polygon segmentation.
• AI-Driven Speed: Leverages SAM2, YOLO, and Grounding DINO to potentially save up to 80% of manual effort.
• Video Annotation: Features frame-to-frame label propagation using SAM2 and various OpenCV trackers.
• Flexible Export: Supports exporting annotations in YOLO, COCO, and custom formats.
• Integration: Includes a Python API for pipeline integration and supports cloud/SSH image downloading.