openxrlab/xrmocap

What it solves

XRMoCap is a PyTorch-based toolbox designed to simplify multi-view motion capture (mocap), allowing users to estimate 3D human poses and shapes from multiple calibrated cameras. It provides a unified framework for both single-person and multi-person scenarios, supporting various state-of-the-art (SOTA) methods.

How it works

The project integrates both optimization-based and learning-based motion capture algorithms into a modular framework. It supports two primary motion representations: 3D keypoints and parametric human models like SMPL(-X). The system can handle an arbitrary number of calibrated cameras (more than two) and includes tools for automatic camera selection and conversion between different motion representations.

Who it’s for

This toolkit is for researchers and engineers working on human perception, 3D pose estimation, and motion capture, specifically those needing a standardized benchmark and a flexible pipeline for prototyping new mocap algorithms.

Highlights

• Comprehensive Method Support: Re-implements SOTA methods such as SMPLify, MvP, and HuMMan MoCap.
• Modular Architecture: Allows users to customize mocap pipelines by selecting different components via configuration files.
• Dual Representation: Supports both keypoint-based and parametric human model-based (SMPL/SMPL-X) estimation.
• Multi-Person Capability: Handles both single and multiple people in a scene using multi-view setups.
• Integrated Benchmarking: Includes support for several standard datasets like Campus, Shelf, and CMU Panoptic.