habitat-sim: a high-performance physics-enabled 3D simulator for embodied AI research

What it solves

Habitat-Sim is a high-performance 3D simulator designed for embodied AI research. It solves the need for a fast, physics-enabled environment where agents can be trained and tested in realistic indoor and outdoor spaces without the overhead of slow rendering or complex setup.

How it works

It provides a simulation engine that supports 3D scans of real-world spaces and CAD models of objects. It integrates with Bullet Physics for rigid-body mechanics and supports configurable sensors, such as RGB-D cameras and egomotion sensing. The simulator is optimized for extreme speed, prioritizing frames-per-second (FPS) and steps-per-second (SPS) to enable rapid iteration in AI training.

Who it’s for

It is primarily for researchers and developers working on embodied AI, specifically those training agents for tasks like navigation, instruction following, and question answering using reinforcement learning or imitation learning.

Highlights

• Extreme Performance: Capable of rendering thousands of frames per second on a single GPU.
• Broad Dataset Support: Built-in support for major 3D datasets including HM3D, HSSD, Matterport3D, Gibson, and Replica.
• Physics Integration: Supports rigid-body mechanics and articulated dynamics via Bullet Physics.
• Robot Versatility: Supports various robot configurations described via URDF, including mobile manipulators, fixed-base arms, and quadrupeds.