stereolabs/zed-sdk

⚡️The spatial perception framework for rapidly building smart robots and spaces

What it solves

The ZED SDK provides a comprehensive spatial perception platform that enables developers to integrate human-like sensing capabilities into their applications. It simplifies the process of extracting 3D information from ZED cameras, allowing for real-time depth sensing, object detection, and positional tracking without requiring the developer to build these complex computer vision algorithms from scratch.

How it works

The SDK is a cross-platform library that interfaces with ZED cameras. It uses neural depth models to compute depth maps and 3D point clouds. It also includes modules for SLAM (Simultaneous Localization and Mapping) for positional tracking and global localization, and specialized AI models for body tracking and object detection. The SDK can be integrated into various programming languages (C++, Python, C#, C) and environments like ROS, ROS 2, Unity, and Unreal Engine 5.

Who it’s for

Developers building robotics, AR/VR, and spatial AI applications who use ZED cameras and need real-time 3D perception, spatial mapping, and environment understanding.

Highlights

  • Real-time Performance: Optimized for NVIDIA GPUs to ensure all algorithms run in real-time.
  • Neural Depth Sensing: High-performance depth inference with support for custom depth algorithms via DEPTH_MODE::CUSTOM.
  • Spatial Perception: Includes built-in capabilities for object detection, body tracking, and plane detection.
  • ** uma a comprehensive suite of tools for positional tracking, global localization, and 3D mesh capture (spatial mapping).
  • Broad Integration: Supports multiple languages and third-party libraries like PyTorch, YOLO, and OpenCV.

Related

  • Project
    stereolabs/zed-unity

    A Unity plugin that integrates ZED cameras' depth sensing, spatial mapping, and object detection capabilities into the Unity engine for AR/MR development.

  • Project
    realsenseai/librealsense

    RealSense SDK 2.0 (librealsense) – open‑source C++ library for Intel RealSense depth cameras, offering cross‑platform colour‑depth streaming, calibration data, and wrappers (Python, ROS 2, C#, Unity). Provides pre‑built binaries or source builds, supports AMD GPU acceleration, and includes sample code for easy start‑up.

  • Project
    3DiVi/nuitrack-sdk

    A 3D body and skeletal tracking library that provides a unified API for various depth sensors across multiple platforms, enabling human-centric spatial understanding.

  • Project
    stereolabs/zed-ros2-wrapper

    Official ROS 2 wrapper for Stereolabs ZED depth cameras, providing image, depth, point‑cloud, SLAM, object detection, body tracking, GNSS fusion and simulation support. Installable on Ubuntu with ZED SDK and CUDA, launchable via `ros2 launch zed_wrapper zed_camera.launch.py`.

  • Project
    orbbec/OrbbecSDK

    A software development kit for Orbbec 3D cameras that enables developers to capture data streams and control hardware across various device series.