talmolab/sleap

A deep learning framework for multi-animal pose tracking.

What it solves

SLEAP provides a deep-learning framework for tracking the poses of multiple animals in videos. It eliminates the need for manual, frame-by-frame labeling of every animal's position, which is traditionally a tedious process in behavioral research.

How it works

SLEAP uses customizable neural network architectures and supports both top-down and bottom-up training strategies for pose estimation. It features a human-in-the-loop workflow where users can rapidly label small amounts of data and use the provided GUI to proofread and correct predictions, which the model then learns from (active learning).

Who it’s for

Researchers in neuroscience and biology who need to quantify animal behavior by tracking body parts across multiple animals in video data.

Highlights

  • Active Learning GUI: A purpose-built interface for rapid labeling and proofreading of datasets.
  • High Performance: Capable of batch inference at 600+ FPS and real-time latency under 10ms.
  • Flexible Training: Fast training times (15-60 minutes on a single GPU) and support for remote training workflows for users without local GPUs.
  • Broad Compatibility: Supports any type or number of animals and works across all major operating systems.

Related

  • Project
    sgoldenlab/simba

    SimBA is a GUI-based toolkit that lets researchers train supervised machine-learning classifiers to automatically detect and quantify animal behaviors from pose-estimation video data, without needing programming skills. It turns raw tracking data into validated behavioral classifications and rich downstream analyses for behavioral neuroscience.

  • Project
    dluvizon/deephar

    A multi-task deep learning framework for real-time 2D and 3D human pose estimation and action recognition.

  • Project
    DeepLabCut/DeepLabCut

    DeepLabCut is a markerless pose estimation toolbox that allows researchers to track animals and objects in video without physical markers using deep learning.

  • Project
    SwanHubX/SwanLab

    An AI training analysis and metric observation platform that provides training visualization, automatic logging, and experiment comparison for model training teams.

  • Project
    paninski-lab/lightning-pose

    An end-to-end animal pose estimation package that uses transformer architectures to robustly track movements in single-view and multi-view videos, especially during occlusions.