perfanalytics/pose2sim

Markerless kinematics with any cameras — From 2D Pose estimation to 3D OpenSim motion

What it solves

Pose2Sim provides a markerless 3D kinematics workflow for humans and animals, serving as a low-cost alternative to traditional marker-based motion capture (MoCap) systems. It allows users to capture high-accuracy 3D joint angles and skeletal analysis using standard hardware like smartphones, webcams, or GoPros, while maintaining data privacy by running locally.

How it works

The system converts 2D pose estimations from multiple camera views into 3D kinematics through a multi-stage pipeline:

  1. Pose Estimation: Extracts 2D keypoints using models like RTMPose (default), DeepLabCut, or legacy models (OpenPose, AlphaPose).
  2. Calibration & Sync: Calibrates camera positions and synchronizes video feeds.
  3. Triangulation: Associates persons across views and triangulates 2D keypoints into 3D coordinates.
  4. Kinematics: Filters the 3D data and uses OpenSim to compute full-body 3D joint angles and skeletal analysis.

Who it’s for

  • Sports Analysts: For field-based 3D motion capture.
  • Clinicians: For gait analysis and clinical assessments in a doctor's office.
  • Animators: For outdoor 3D capture of fully clothed subjects.
  • Researchers: For biomechanics studies involving multiple participants.

Highlights

  • Research-Grade Accuracy: Validated joint angle error of 2-6°.
  • Hardware Agnostic: Works with any combination of phones, webcams, or GoPros.
  • Multi-Person Support: Capable of tracking multiple people simultaneously with occlusion handling.
  • Local Processing: Respects GDPR and privacy rules by avoiding cloud uploads.
  • OpenSim Integration: Provides complete skeletal analysis and exports results for visualization in OpenSim or Blender.

Related

  • Project
    opencap-org/opencap-core

    A pipeline that estimates 3D human movement kinematics and marker positions from multiple smartphone videos, outputting data in OpenSim format.

  • Project
    phongdaot/MocapAnything

    An end-to-end motion capture system that converts monocular videos into 3D joint rotations for arbitrary skeletons, enabling movement transfer to any rigged character.

  • Project
    freemocap/freemocap

    A free and open-source motion capture platform that provides research-grade tracking without requiring expensive, proprietary hardware.

  • Project
    cuevhv/mamma

    MAMMA is a markerless multi-person 3D motion acquisition system that reconstructs accurate human poses from multi-view video footage.

  • Project
    DeepLabCut/DeepLabCut

    DeepLabCut is a markerless pose estimation toolbox that allows researchers to track animals and objects in video without physical markers using deep learning.