radekd91/inferno

🔥🔥🔥 Set the world of 3D faces on fire with INFERNO 🔥🔥🔥

What it solves

INFERNO is a library designed for deep-learning-based face reconstruction, animation, and related tasks. It provides tools to process face video datasets, train reconstruction networks, and generate 3D faces that can be subsequentlly used for applications like speech-driven animation.

How it works

Built on PyTorch, PyTorch Lightning, and the FLAME model, the library is divided into a core research library (inferno) containing models, layers, and dataset handlers, and a set of application prototypes (inferno_apps). These applications implement specific tasks such as:

  • TalkingHead: Official release of EMOTE for speech-driven 3D avatar animation.
  • FaceReconstruction: Combines DECA, EMOCA, SPECTRE, and MICA (via EMICA) for high-quality in-the-wild face reconstruction.
  • MotionPrior: Implements FLINT, a facial motion prior used in EMOTE.
  • EmotionRecognition: Tools for single-image and video-based emotion recognition.

Who it’s for

Researchers and developers working with 3D facial modeling, facial animation, and emotion-driven face reconstruction.

Highlights

  • Integrated tools for the entire pipeline from dataset processing to 3D face generation.
  • Support for state-of-the-art reconstruction models like EMICA.
  • Ability to fine-tune and run speech-driven 3D avatars.
  • Includes dedicated modules for both image and video-based emotion recognition.

Related

  • Project
    tomas-gajarsky/facetorch

    A Python library for facial detection and analysis that curates open-source models and packages them as portable torch.export models for efficient inference.

  • Project
    deepinv/deepinv

    DeepInverse is an open-source PyTorch-based library designed to solve imaging inverse problems using deep learning through a modular framework of operators and algorithms.

  • Project
    frgfm/holocron

    A library of high-quality PyTorch implementations of recent deep learning tricks and model architectures for computer vision tasks.

  • Project
    facebookresearch/pytorch3d

    A PyTorch-based library for 3D Computer Vision research providing differentiable rendering and efficient tools for manipulating 3D meshes and point clouds.

  • Project
    nschaetti/EchoTorch

    A PyTorch-based module for implementing and testing Echo State Networks and Reservoir Computing, designed for research purposes and integration into deep architectures.