NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation

Deep learned, NVIDIA-accelerated 3D object pose estimation

What it solves

Isaac ROS Pose Estimation provides tools to predict the 3D pose of objects in an environment, which is essential for robots to navigate and manipulate objects. It solves the problem of accurately identifying the position and orientation of items—whether they are known objects with pre-trained models or novel objects that the robot has never seen before.

How it works

The project consists of three specialized ROS 2 packages that use GPU-accelerated deep learning models for inference:

  • FoundationPose: Estimates the pose of novel (unseen) objects using 3D bounding cuboid dimensions without requiring fine-tuning. It uses a refine model to iteratively improve pose hypotheses and a score model to finalize the estimation. It also supports efficient tracking of objects across frames.
  • DOPE (Deep Object Pose Estimation): Predicts the pose of known objects using pre-trained models. It uses belief maps produced by DNN inference and a decoder to output the final pose.
  • CenterPose: Performs object detection, generates 2D keypoints, and estimates 6-DoF pose and relative 3D bounding cuboid dimensions for known object classes (e.g., detecting any chair regardless of the specific instance).

These packages rely on NVIDIA Triton or TensorRT for accelerated inference and the Isaac ROS DNN Image Encoder for preprocessing.

Who it’s for

Robotics developers and engineers building autonomous systems that require precise 3D object perception, navigation, and manipulation capabilities on NVIDIA hardware (such as Jetson Thor or RTX GPUs).

Highlights

  • Support for Novel Objects: FoundationPose allows for pose estimation and tracking of unseen objects without retraining.
  • GPU Acceleration: Optimized for NVIDIA hardware using TensorRT and Triton, achieving high frame rates (e.g., over 120 FPS for tracking on Jetson Thor).
  • Versatile Model Options: Offers a choice between high-quality estimation (FoundationPose), established reliability (CenterPose), and maximum speed (DOPE).
  • Integration with ROS 2: Designed to work within a ROS 2 node graph for seamless integration into robotic perception pipelines.