lightly-ai/lightly-train

All-in-one training for vision models (YOLO, ViTs, RT-DETR, DINOv3): pretraining, fine-tuning, distillation.

What it solves

LightlyTrain simplifies the end-to-end development of computer vision models. It provides a unified framework to handle the entire lifecycle, from pretraining foundation models on unlabeled data to fine-tuning specialized models for tasks like object detection and segmentation, including deployment-ready optimizations.

How it works

The framework offers high-level Python functions to train and fine-tune state-of-the-art architectures, such as DINOv2, DINOv3, and LTDETRv2. It supports various vision tasks by allowing users to plug in their own datasets and backbones. Once trained, models can be exported to industry-standard formats like ONNX and TensorRT for high-performance inference on edge devices or production servers.

Who it’s for

It is designed for machine learning engineers and researchers building computer vision applications who need to move from raw data to high-accuracy, deployable models quickly. It is particularly useful for those targeting edge deployment or requiring state-of-the-art performance in detection and segmentation tasks.

Highlights

  • Supports advanced pretraining, fine-tuning, and distillation methods.
  • Includes state-of-the-art architectures like DINOv3 and LTDETRv2.
  • Provides out-of-the-box export to ONNX and TensorRT for fast inference.
  • Optimized for diverse tasks including object detection, instance segmentation, semantic segmentation, and depth estimation.
  • Enables efficient training with features like gradient accumulation and activation checkpointing.

Related

  • Project
    isl-org/Open3D-ML

    An extension of Open3D that provides tools, pretrained models, and pipelines for 3D machine learning tasks like semantic segmentation and object detection.

  • Project
    EvolvingLMMs-Lab/lmms-engine

    A unified, high-performance training engine for multimodal models that integrates distributed training, kernel fusion, and advanced optimizers to enable scalable pretraining and fine-tuning.

  • Project
    lightly-ai/lightly

    LightlySSL is a modular computer vision framework for self-supervised learning and world models, providing the building blocks to train models on raw data without manual labels.

  • Project
    dexmal/dexbotic

    Dexbotic is a PyTorch-based, one-stop toolbox for developing Vision-Language-Action (VLA) models in embodied intelligence. It unifies pretraining, fine-tuning, inference, and evaluation for models like pi0, CogACT, and OFT, with modular configuration, cloud/local training support, and compatibility with multiple robots.

  • Project
    Lightning-AI/pytorch-lightning

    A deep learning framework that abstracts away the engineering boilerplate of PyTorch, enabling researchers to pretrain and finetune AI models at scale across various hardware.