AmmarkoV/SAM3DBody-cpp

Real-time 3D full-body reconstruction from a single camera, Multiperson BVH output, Pure C++ runtime, ONNX + ggml, 70-joint skeleton with hands.

What it solves

This project provides a high-performance C++ engine for 3D human body pose estimation from monocular video. It eliminates the need for Python at runtime, allowing developers to extract 3D body parameters, camera translation, and full body meshes directly from standard 2D images without depth sensors or multi-camera setups.

How it works

The pipeline uses a deep learning approach to regress 3D body parameters directly from image features. First, a YOLO model detects people in the frame. A DINOv2-ViT-H backbone then processes these crops into spatial feature maps, which a transformer decoder compresses into pose tokens. Finally, specialized heads compute pose, shape, and camera parameters, which are used to generate a 3D mesh via Linear Blend Skinning (LBS).

Who it’s for

It is designed for developers and researchers who need to integrate 3D motion capture or avatar animation into C++ applications, ROS environments, or performance-sensitive pipelines. It is particularly useful for those who need to export motion data to formats like BVH for animation software (e.g., Blender) or MPEG ARF for avatar representation.

Highlights

  • Zero Python dependency at runtime for the core C++ inference engine.
  • Multi-person tracking and motion capture export to standard BVH files.
  • Support for MPEG Avatar Representation Format (ARF) for full mesh and animation streaming.
  • Flexible hardware support, including CUDA-accelerated GPU inference and CPU-only fallback modes.
  • Includes Python frontends and bindings for easy integration into existing workflows.

Related

  • Project
  • Project
  • Project
  • Project
  • Project