NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation

What it solves

It provides NVIDIA-accelerated 3D object pose estimation for robots, allowing them to determine the position and orientation of objects in their environment. This data is essential for tasks like navigation and manipulation, where a robot needs to know exactly where an object is and how far away it is to interact with it.

How it works

The project consists of three ROS 2 packages, each utilizing GPU acceleration via Triton or TensorRT for deep learning inference:

• isaac_ros_foundationpose: Uses the FoundationPose model to estimate and track the pose of novel, unseen objects without requiring fine-tuning. It employs a refine model to iteratively improve pose hypotheses and a score model to finalize the estimation.
• isaac_ros_dope: Uses the Deep Object Pose Estimation (DOPE) model to estimate the pose of known objects. It requires a pre-trained model and uses a DNN decoder to output poses based on belief maps.
• isaac_ros_centerpose: Uses the CenterPose model to perform object detection, generate 2D keypoints, and estimate the 6-DoF pose for known object classes (e.g., detecting any chair rather than a specific instance).

Who it’s for

Robotics developers building perception systems for mobile robots or robotic arms that require precise 3D object localization and tracking in real-time.

Highlights

• Support for Novel Objects: FoundationPose allows for pose estimation and tracking of objects the system has never seen before without retraining.
• High-Speed Tracking: Tracking in FoundationPose can exceed 120 FPS on the Jetson Thor platform.
• Flexible Model Options: Offers a range of nodes tailored for different needs, balancing speed, quality, and maturity.
• Hardware Acceleration: Fully optimized for NVIDIA GPUs and Jetson platforms using TensorRT and Triton.