lightly-ai/lightly-train

All-in-one training for vision models (YOLO, ViTs, RT-DETR, DINOv3): pretraining, fine-tuning, distillation.

What it solves

LightlyTrain is a framework designed to simplify the creation of state-of-the-art computer vision models. It addresses the difficulty of managing the full model development lifecycle, from pretraining foundation models on unlabeled data to fine-tuning them for specific tasks like object detection and segmentation, and finally optimizing them for edge deployment.

How it works

The framework provides high-level APIs to handle various computer vision workflows. It supports pretraining and distillation from vision foundation models (such as DINOv2 and DINOv3) and fine-tuning for specialized tasks. It also integrates with optimization tools like ONNX and TensorRT for faster inference on embedded devices.

Who it’s for

It is intended for machine learning engineers and developers who need to build, train, and deploy high-performance computer vision models, particularly those targeting edge devices or proprietary datasets.

Highlights

  • Comprehensive Task Support: Supports object detection, instance segmentation, panoptic segmentation, semantic segmentation, and monocular depth estimation.
  • SOTA Architectures: Integrates latest models like LTDETRv2, DINOv3, and EoMT (CVPR 2025).
  • Edge Optimization: Out-of-the-box support for ONNX and TensorRT export in FP16 precision for low-power embedded devices.
  • Flexible Backbones: Compatible with various backbones including ViTs, hybrids, and CNNs, and supports custom teachers for distillation.

Related

  • Project
    Intellindust-AI-Lab/DEIMv2

    DEIMv2 is a real-time object detection framework leveraging DINOv3 features to provide scalable, state-of-the-art performance across various model sizes for edge and server deployment.

  • Project
    ultralytics/ultralytics

    A high-performance computer vision framework implementing YOLO models for real-time object detection, segmentation, classification, and pose estimation.

  • Project
    huggingface/pytorch-image-models

    A comprehensive PyTorch library providing a vast collection of state-of-the-art computer vision backbones and pretrained weights for easy integration into AI projects.

  • Project
    bghira/SimpleTuner

    A comprehensive training framework for fine-tuning image, video, and audio generative models, featuring a user-friendly web UI and enterprise-grade orchestration.

  • Project
    luohongk/SuperVINS

    A real-time Visual-Inertial SLAM framework that replaces traditional handcrafted features with SuperPoint and LightGlue deep learning models to handle challenging imaging conditions.