zju3dv/OnePose_Plus_Plus

What it solves

OnePose++ addresses the challenge of object pose estimation—determining an object's position and orientation in 3D space—without requiring pre-existing CAD models or the manual definition of keypoints, which are typically necessary for traditional methods.

How it works

The system uses a one-shot approach to estimate the pose of an object. It leverages Structure-from-Motion (SfM) via COLMAP to reconstruct semi-dense object point clouds and establish 2D-3D correspondences from training data. It integrates components like LoFTR for feature matching and DeepLM for optimization to refine the pose estimation process.

Who it’s for

This project is designed for researchers and developers working in computer vision, robotics, and augmented reality who need to track or locate objects in a scene without having access to their digital 3D blueprints (CAD models).

Highlights

• CAD-Free: Eliminates the need for 3D CAD models of the target objects.
• Keypoint-Free: Removes the requirement for manually defined keypoints.
• One-Shot: Capable of estimating pose using minimal training data.
• Mobile Integration: Includes a companion iOS app (OnePose Cap) for capturing custom training and test data.