habitat-lab: a modular framework for training and evaluating embodied AI agents in indoor environments

What it solves

Habitat-Lab provides a modular framework for the end-to-end development of embodied AI. It addresses the challenge of training agents to perform complex tasks in indoor environments—such as navigation, object rearrangement, and following human instructions—while providing the tools to evaluate their performance and allow humans to interact with the simulation.

How it works

Built on top of the Habitat-Sim core simulator, the library allows developers to:

• Define Tasks: Create flexible single or multi-agent tasks including question answering, human following, and navigation.
• Configure Agents: Instantiate various embodied agents, ranging from humanoids to commercial robots, by specifying their sensors and capabilities.
• Train and Evaluate: Use provided algorithms for reinforcement learning (including PPO baselines), imitation learning, or non-learning pipelines (SensePlanAct) to train agents and benchmark them using standard metrics.
• Human Interaction: Use a framework that lets humans interact with the simulator to collect data or test trained agents.

Who it’s for

This project is designed for AI researchers and developers working on embodied AI, robotics simulation, and the interaction between autonomous agents and indoor environments.

Highlights

• Modular Design: Supports a wide variety of task definitions and agent configurations.
• Diverse Agent Support: Compatible with various robot types and humanoid models.
• Integrated Baselines: Includes reinforcement learning baselines via PPO.
• Human-in-the-Loop: Enables direct human interaction with the simulated environment for data collection.