NVIDIA/nvidia-container-toolkit

Build and run containers leveraging NVIDIA GPUs

What it solves

The NVIDIA Container Toolkit removes the complexity of setting up containers to work with NVIDIA GPUs. It ensures that software running inside a container can automatically detect and utilize the host's GPU hardware without requiring manual configuration for every environment.

How it works

It acts as a bridge between your container runtime (like Docker) and the NVIDIA GPU hardware. By providing a specialized runtime library and utility tools, it automatically configures the container environment to access the necessary GPU drivers installed on the host system. This allows you to run GPU-accelerated applications inside containers while only needing the NVIDIA driver installed on the host machine.

Who it’s for

This toolkit is designed for developers, system administrators, and engineers working in cloud-native environments who need to deploy GPU-accelerated applications using container technology on Linux systems.

Highlights

  • Enables seamless execution of GPU-accelerated containers.
  • Includes a dedicated container runtime library for hardware interaction.
  • Automates the configuration process for GPU-enabled containers.
  • Eliminates the need to install the full CUDA Toolkit on the host system.

Related

  • Project
    xlite-dev/lite.ai.toolkit

    A lightweight C++ toolkit providing a unified API to deploy over 300 AI models using backends like ONNXRuntime, MNN, and TensorRT.

  • Project
    openvinotoolkit/open_model_zoo

    A collection of optimized pre-trained deep learning models and tools for accelerating the development and deployment of high-performance inference applications.

  • Project
    PINTO0309/PINTO_model_zoo

    A curated collection of pre‑converted deep‑learning models (image classification, object detection, etc.) in many deployment formats (TensorFlow Lite, ONNX, OpenVINO, CoreML, EdgeTPU, etc.). The repo provides the artefacts, licence info, and links to conversion tools, helping developers deploy models on diverse hardware without doing the conversion themselves.

  • Project
    openxla/xla

    XLA is an open-source machine learning compiler that optimizes models from PyTorch, TensorFlow, and JAX for high-performance execution on GPUs, CPUs, and ML accelerators.