zhiqwang/yolort

yolort is a runtime stack for yolov5 on specialized accelerators such as tensorrt, libtorch, onnxruntime, tvm and ncnn.

What it solves

yolort is a runtime stack designed to simplify the deployment of YOLOv5 object detection models. It addresses the complexity of integrating training and inference, specifically by reducing the need for complex external libraries during the deployment phase on specialized accelerators.

How it works

The project implements the YOLOv5 model structure but introduces a dynamic shape mechanism. This allows the project to embed both pre-processing (letterbox) and post-processing (Non-Maximum Suppression or NMS) directly into the model graph. By doing so, the model becomes a self-contained unit that is easier to export and run on various backends.

Who it’s for

Developers and AI engineers who want to deploy YOLOv5 models to production environments using backends like LibTorch, ONNX Runtime, TVM, or TensorRT, and those who prefer a simpler, torchvision-like implementation of object detection.

Highlights

  • Integrated Pipeline: Embeds pre- and post-processing into the model graph to simplify deployment.
  • Multi-Backend Support: Provides native support and examples for exporting to and inferencing with LibTorch, ONNX Runtime, TVM, and TensorRT.
  • Dynamic Shape Support: Supports dynamic shape and batch inference.
  • Compatibility: Ability to load checkpoints trained with the official Ultralytics YOLOv5 implementation.
  • Visualization: Includes tools for visualizing the model graph.