cyrusbehr/YOLOv8-TensorRT-CPP

YOLOv8 TensorRT C++ Implementation

What it solves

This project provides a high-performance C++ implementation for running YOLOv8 models. It allows developers to move away from Python-based inference to a compiled language for better speed and efficiency, specifically targeting NVIDIA GPUs using the TensorRT API.

How it works

The project converts PyTorch models (.pt) to ONNX format and then uses the TensorRT C++ API to generate an optimized engine file for the GPU. It handles the necessary preprocessing and post-processing (such as bounding box decoding and Non-Maximum Suppression) externally in C++ rather than inside the model itself. It supports multiple precision modes, including FP32, FP16, and INT8 (which requires calibration data to maintain accuracy).

Who it’s for

It is designed for developers and engineers who need to deploy YOLOv8 for real-time object detection, semantic segmentation, or body pose estimation on Ubuntu systems with NVIDIA hardware.

Highlights

  • Multi-task Support: Works with object detection, segmentation, and pose estimation models.
  • Hardware Optimization: Leverages TensorRT for GPU-accelerated inference.
  • C++ Implementation: Reduces overhead compared to Python implementations.
  • Precision Flexibility: Supports FP32, FP16, and INT8 quantization for varying speed-accuracy trade-offs.
  • Real-time Capability: Includes executables for image processing and live webcam feeds.

Related

  • Project
    triple-mu/YOLOv8-TensorRT

    A high-performance TensorRT acceleration wrapper for YOLOv8, enabling fast inference for detection, segmentation, pose, OBB, and classification in both Python and C++.

  • Project
    pytorch/TensorRT

    Torch‑TensorRT is an open‑source library that lets you compile any PyTorch model to run with TensorRT speed on NVIDIA GPUs. Use a single line (`torch.compile(..., backend="tensorrt")`) or an export workflow to produce a serialized module that can be loaded in Python or C++. It supports Linux, Windows, and Jetson, works with CUDA 13.4 + TensorRT 11.3, and is installable via pip or NVIDIA containers. Ideal for developers who want fast inference for vision, diffusion, or LLM models without leaving the PyTorch ecosystem.

  • Project
    Geekgineer/YOLOs-CPP

    A production-ready C++ inference engine that provides a unified API for the entire YOLO family, supporting tasks from object detection to metric depth estimation.

  • Project
    yzfzzz/depth-detect

    depth-detect is a C++14 framework that combines YOLO object detection with monocular or stereo depth estimation to identify approaching hazards for a bike‑mounted rear‑view mirror. It supports TensorRT and ONNX inference, runs on x86 and Jetson devices, offers three scheduling modes (sync, overlap, stagger), tracks objects with BYTETracker, and sends JSON alerts over TCP. Benchmarks show ~19× speed‑up on Jetson TX2 with TensorRT FP16 and ~13.5 fps in a staggered configuration.

  • Project
    onnx/onnx-tensorrt

    A TensorRT backend for ONNX that parses ONNX models for high-performance execution on NVIDIA GPUs.