Sompote/DINOV3-YOLOV12

What it solves

This project addresses the challenge of data-efficient object detection, particularly for small or complex datasets where standard CNN-based detectors like YOLOv12 may struggle. By integrating DINOv3's self-supervised Vision Transformer (ViT) features, it enables models to achieve higher accuracy (mAP) and faster convergence with significantly fewer training images.

How it works

It creates a hybrid architecture that combines the speed of YOLOv12 with the rich feature extraction capabilities of DINOv3. The system supports four integration types:

• Single: Uses DINOv3 for input preprocessing (P0).
• Dual: Integrates DINOv3 at the P3 and P4 backbone levels.
• Triple: Combines P0 input preprocessing with P3 and P4 backbone enhancements.
• DualP0P3: A balanced approach combining P0 preprocessing and P3 backbone enhancement.

The project allows users to mix and match five YOLOv12 sizes with official DINOv3 variants (e.g., ViT-S, ViT-B, ViT-L) to create over 40 different model combinations.

Who it’s for

• AI Researchers and Developers working on object detection for niche domains with limited labeled data.
• Autonomous Driving Engineers needing high-precision detection in complex scenes (e.g., KITTI dataset).
• Production ML Engineers seeking a balance between inference speed and detection accuracy.

Highlights

• Massive Performance Gains: Up to 88.6% mAP improvement on complex datasets like KITTI and 15-25% on small datasets.
• Data Efficiency: Reduces the number of epochs needed for optimal performance by 50-70%.
• Flexible Integration: Supports multiple integration levels (Single, Dual, Triple, DualP0P3) to balance VRAM usage and accuracy.
• Exact State Restoration: Features a robust training resume system that restores all 61 hyperparameters and loss values perfectly to avoid training spikes.
• Custom Weight Support: Ability to load local .pth, .pt, and .safetensors DINO weight files.