AmmarkoV/SAM3DBody-cpp

Real-time 3D full-body reconstruction from a single camera, Multiperson BVH output, Pure C++ runtime, ONNX + ggml, 70-joint skeleton with hands.

What it solves

SAM3DBody-cpp provides a high-performance, standalone C++ inference engine for the SAM-3D-Body model. It enables the extraction of 3D human body pose, shape, and camera parameters from standard 2D images or video without requiring a Python runtime, making it suitable for integration into production software or real-time applications.

How it works

The pipeline processes raw BGR images through several stages:

  1. Detection: YOLO detects person bounding boxes.
  2. Feature Extraction: A DINOv3-ViT-H backbone generates a spatial feature map from each crop.
  3. Pose Tokenization: A 6-layer transformer decoder compresses the feature map into a 1024-dimensional pose token, conditioned on camera focal length and ray directions.
  4. Parameter Decoding: Small FFN heads (running via ggml) decode the token into 519 pose parameters (global orientation, joint angles, shape betas, hand pose, and facial expressions) and 3 camera parameters.
  5. Mesh Generation: These parameters drive a native C Linear Blend Skinning (LBS) process to produce a full 3D body mesh (18,439 vertices) and 70 keypoints.

Who it’s for

  • Developers needing a zero-Python dependency for 3D human pose estimation.
  • Motion Capture Artists who want to export multi-person motion to standard BVH files for use in Blender or other DCC tools.
  • Avatar Creators looking to generate MPEG ARF (.arfz) containers that include skeleton, skin weights, and personalized meshes.
  • Robotics/CV Engineers requiring fast, C++ based 3D body parameter regression from monocular video.

Highlights

  • Zero Python Runtime: Fully standalone C++ implementation using ONNX Runtime and ggml.
  • Multi-Person Tracking: Built-in 2D-bbox IoU tracker to maintain stable identities across video frames.
  • Industry Standard Exports: Supports BVH motion-capture files and MPEG ARF avatar containers.
  • Flexible Hardware Support: Optimized for CUDA GPUs but includes FP32/FP16 fallback models for CPU-only inference.
  • Comprehensive Output: Provides full 3D mesh vertices, 70 body/hand keypoints, and camera translation.

Related

  • Project
    localai-org/depth-anything.cpp

    A dependency-free C++ port of Depth Anything 3 and V2 for fast, monocular metric depth and camera pose inference on CPU and GPU.

  • Project
    FORTH-ModelBasedTracker/MocapNET

    A real-time 2D-to-3D human pose estimator that converts RGB video feeds into standard BVH motion capture files for 3D animation.

  • Project
    Geekgineer/YOLOs-CPP

    A production-ready C++ inference engine that provides a unified API for the entire YOLO family, supporting tasks from object detection to metric depth estimation.

  • Project
    facebookresearch/MHR

    A high-fidelity parametric 3D human body model that uses identity, pose, and facial expression parameters to generate realistic, differentiable human meshes.

  • Project
    mudler/locate-anything.cpp

    A C++/ggml inference port of NVIDIA's LocateAnything-3B that enables fast, open-vocabulary object detection on CPU and GPU without a Python runtime.