lightly-ai/lightly-train

All-in-one training for vision models (YOLO, ViTs, RT-DETR, DINOv3): pretraining, fine-tuning, distillation.

What it solves

LightlyTrain is a framework designed to streamline the creation of state-of-the-art computer vision models. It addresses the difficulty of moving from raw, unlabeled data to deployed models by providing tools for pretraining, fine-tuning, and distillation across various vision tasks.

How it works

The framework supports the entire model development lifecycle:

  • Pretraining: Users can pretrain vision foundation models (like DINOv2/v3) on unlabeled data.
  • Fine-tuning: It provides specialized methods for training models on specific tasks, including object detection, instance segmentation, semantic segmentation, and panoptic segmentation.
  • Distillation: It includes methods to create smaller, more efficient models (e.g., Distillationv3) that maintain high performance across different architectures (ViTs, CNNs, hybrids).
  • Deployment: The framework supports exporting models to ONNX and TensorRT (including FP16 precision) for faster inference on edge devices.

Who it’s for

It is built for machine learning engineers and researchers who need to train high-performance computer vision models for production or edge deployment, particularly those working with transformer-based backbones like DINOv2/v3 and EdgeCrafter ECViT.

Highlights

  • Comprehensive Task Support: Built-in workflows for object detection, instance segmentation, semantic segmentation, panoptic segmentation, and monocular depth estimation.
  • Edge-Ready: Direct support for ONNX and TensorRT export for low-power embedded devices.
  • SOTA Architectures: Integration with DINOv2, DINOv3, and the EoMT method from CVPR 2025.
  • Efficient Model Scaling: Tools for distilling large foundation models into tiny, high-performance models (e.g., PicoDet).