AmmarkoV/SAM3DBody-cpp

Real-time 3D full-body reconstruction from a single camera, Multiperson BVH output, Pure C++ runtime, ONNX + ggml, 70-joint skeleton with hands.

What it solves

This project provides a high-performance, standalone C++ implementation for reconstructing 3D human body poses and meshes from a single BGR image or video. It eliminates the need for Python dependencies at runtime, making it suitable for efficient deployment in production environments or embedded systems.

How it works

The pipeline processes raw images through several stages:

  1. Detection: YOLO detects person bounding boxes.
  2. Feature Extraction: A DINOv3-ViT-H backbone generates spatial feature maps from the detected crops.
  3. Pose Decoding: A transformer decoder compresses features into a pose token.
  4. Parameter Regression: Small FFN heads (running via ggml) decode the token into 519 pose parameters (including body, hand, and facial expression) and camera translation.
  5. Mesh Generation: Linear Blend Skinning (LBS) converts these parameters into full 3D body meshes and 70 keypoints.

Who it’s for

  • Developers needing high-speed, C++ based 3D human pose estimation without Python overhead.
  • Motion Capture Artists looking to export multi-person motion data to standard formats like BVH for use in Blender or other DCC tools.
  • Computer Vision Engineers requiring robust 3D body reconstruction from monocular video without depth sensors.

Highlights

  • Zero Python Dependency: Standalone C++ inference engine using ONNX Runtime and ggml.
  • Motion Capture Export: Generates standard BVH files per detected person with stable identities via IoU tracking.
  • Multi-Modal Output: Produces 3D mesh vertices, 70 keypoints, and camera parameters.
  • Temporal Smoothing: Includes a Butterworth low-pass filter to reduce jitter in joint and pose vectors.
  • Flexible Deployment: Supports CUDA-accelerated GPU inference or CPU-only execution via FP32 models.