facebookresearch/eb_jepa

An open source library designed to provide community examples of Joint Embedding Predictive Architectures (JEPAs). It contains code and examples for learning representations from images, video, and action-conditioned video, as well as planning using JEPA-based models.

What it solves

EB-JEPA is a library designed to help researchers and developers learn representations for prediction and planning. It implements Energy-Based Joint-Embedding Predictive Architectures (JEPA), which allow models to predict future states or representations of data (like images or video) without needing to generate every single pixel, focusing instead on the underlying structure of the data.

How it works

The library provides a framework for joint-embedding predictive architectures. It includes implementations for various modalities:

  • Image JEPA: Learns self-supervised representations from unlabeled images (e.g., CIFAR-10) and evaluates them on classification tasks.
  • Video JEPA: Predicts the next image representation in a sequence of frames.
  • AC Video JEPA: Combines JEPA with world modeling and planning in a simulated environment (Two Rooms).

Who it’s for

It is primarily intended for AI researchers and developers working on self-supervised learning, world models, and predictive architectures for vision and robotics/planning tasks.

Highlights

  • Self-supervised learning: Learns from unlabeled data across images and video.
  • World modeling and planning: Includes examples for planning in a simulated environment.
  • Conda/uv package management: Supports modern Python package management for easy installation.
  • H100 GPU optimization: Default configurations are tuned for high-end GPUs, with flexibility for older hardware.

Related

  • Project
    ginwind/VLA-JEPA

    VLA-JEPA is a Vision-Language-Action model that integrates a latent world model to enhance robotic task performance and environment understanding.

  • Project
    ZhiChengAIR/Chengling-PWM

    A JEPA-inspired Physical World Model for robots that uses an Energy-Based Transformer to simulate and refine action trajectories based on physical consistency.

  • Project
    reservoirpy/reservoirpy

    A flexible Python library for implementing Reservoir Computing architectures like Echo State Networks for time-series prediction and classification.

  • Project
    google-research/scenic

    A JAX and Flax-based library for computer vision research that provides scalable libraries and project templates for developing attention-based models across images, video, and audio.

  • Project
    datawhalechina/every-embodied

    A comprehensive learning library for Embodied AI that provides a structured path from basic robotics simulation to the reproduction of state-of-the-art VLA and world models.