pypose/pypose

What it solves

PyPose addresses the gap between data-centric deep learning and physics-based optimization in robotics. While deep learning excels at perception but struggles with generalization, physics-based optimization generalizes well but lacks high-level semantic understanding and requires manual tuning. PyPose provides a unified framework to combine these two approaches, making robot learning more efficient, interpretable, and adaptable to new environments.

How it works

Built on PyTorch, PyPose uses an imperative style interface to integrate deep perceptual models with physics-based optimization. It provides specialized tools for handling geometric transformations and system dynamics:

• LieTensor: A dedicated system for Lie groups (like SO3, SE3, Sim3) and Lie algebras, enabling efficient computation of rotations and transformations.
• Modules: Pre-built components for system modeling (LTI, LTV, NLS), filtering (EKF, UKF, PF), and control (LQR) and solvers (EPnP).
• Optimizers: Second-order optimizers such as Gauss-Newton and Levenberg-Marquardt, including support for sparse Jacobian tracing to accelerate complex tasks like bundle adjustment.

Who it’s for

It is designed for robotics researchers and developers who need to integrate deep learning with classical physics-based optimization for real-world robotic applications.

Highlights

• PyTorch-based: Seamless integration with the deep learning ecosystem.
• High Efficiency: Supports parallel computing for Jacobian calculations of Lie groups and Lie algebras.
• Sparse Jacobian Tracing: Significantly accelerates second-order optimization for large-scale problems.
• Comprehensive Toolset: Includes a wide range of Lie group operations, filters, and second-order optimizers.