stereolabs/zed-sdk

What it solves

The ZED SDK provides a comprehensive spatial perception platform that enables developers to integrate human-like sensing capabilities into their applications. It simplifies the process of extracting 3D information from ZED cameras, allowing for real-time depth sensing, object detection, and positional tracking without requiring the developer to build these complex computer vision algorithms from scratch.

How it works

The SDK is a cross-platform library that interfaces with ZED cameras. It utilizes NVIDIA GPUs to run optimized neural depth models and computer vision pipelines. It supports multiple programming languages (C++, Python, C#, C) and integrates with popular frameworks like ROS, ROS 2, OpenCV, and PyTorch. Recent updates have introduced a custom depth mode that allows developers to feed their own externally computed depth maps into the SDK's full pipeline (point cloud, spatial mapping, etc.).

Who it’s for

It is designed for developers building robotics, augmented reality, and computer vision applications who use ZED cameras. It supports a wide range of platforms from desktop PCs to embedded systems like NVIDIA Jetson.

Highlights

• Real-time Spatial Perception: Optimized for real-time performance across depth sensing, object detection, and Body Tracking.
• Advanced Tracking: Includes positional tracking, global localization (fusing positional tracking with GNSS), and spatial mapping for 3D mesh capture.
• Neural Depth Models: High-performance neural depth inference with optimized paths for NVIDIA GPUs.
• Extensible Pipeline: Experimental support for custom depth algorithms via DEPTH_MODE::CUSTOM.
• Broad Integration: Compatible with Unity, Unreal Engine 5, and various AI frameworks like YOLO and PyTorch.