nutonomy/nuscenes-devkit

What it solves

This project provides a software development kit (SDK) to interact with and evaluate models using the nuScenes and nuImages datasets. It simplifies the process of loading, parsing, and analyzing multimodal sensor data—including LiDAR, radar, and cameras—essential for developing driverless vehicle software.

How it works

The devkit acts as a common interface for both the nuScenes (3D multimodal) and nuImages (2D image) datasets. It provides tools for:

• Data Access: Loading metadata, samples, and sweeps from JSON tables.
• Specialized Expansions: Support for semantic map layers, CAN bus vehicle data (IMU, steering, throttle), and panoptic/semantic LiDAR segmentation.
• Evaluation: Tools for scoring object detection, tracking, and trajectory prediction challenges.
• Tutorials: Jupyter notebooks that guide users through the dataset's schema and usage.

Who it’s for

It is designed for researchers and engineers working on autonomous driving, perception, and motion prediction systems.

Highlights

• Multimodal Support: Handles a wide range of sensors including LiDAR, radar, and camera images.
• Comprehensive Tooling: Includes evaluation code for detection, tracking, and panoptic segmentation.
• C-Bus Integration: Provides access to low-level vehicle telemetry via the CAN bus expansion.
• Semantic Mapping: Includes a map expansion with 11 semantic layers for better environmental understanding.