DeepLabCut/DeepLabCut

What it solves

DeepLabCut provides a way to perform markerless pose estimation of animals and objects. It allows researchers to track specific body parts or points of interest in videos without needing to attach physical markers to the subject, making it animal and object agnostic.

How it works

The toolbox uses deep learning backbones (such as ResNet, HRNet, and RTMPose) to detect features in images. It leverages transfer learning, which significantly reduces the amount of training data required to achieve high accuracy. The system supports both PyTorch and TensorFlow backends and can be used via a graphical user interface (GUI) or an API. It also supports 3D pose estimation using either a single camera or multiple cameras with triangulation.

Who it’s for

It is designed for computer vision researchers and life scientists working in fields such as neuroscience, ecology, and medicine who need to track animal behavior.

Highlights

• Animal Agnostic: Works on any subject as long as the user can label the points of interest.
• Low Data Requirement: Uses transfer learning to minimize the need for large labeled datasets.
• Pretrained Models: Offers foundation models like SuperAnimal-Quadruped and SuperAnimal-TopViewMouse.
• Flexible Backend: Supports PyTorch (primary) and TensorFlow.
• Multi-Dimensional Tracking: Capable of 2D and 3D pose estimation.