zju3dv/habitat-gs

[ECCV 2026] Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting

What it solves

Habitat-GS is a high-fidelity navigation simulator designed for embodied AI research. It solves the problem of achieving photo-realistic rendering and high-fidelity dynamic avatars within a simulation environment, moving beyond the limitations of traditional mesh-based simulators while maintaining compatibility with standard navigation frameworks.

How it works

It acts as a non-intrusive extension of Habitat-Sim, replacing the rendering backbone with 3D Gaussian Splatting (3DGS). This allows the simulator to render photo-realistic scenes and drive dynamic humanoid Gaussian avatars (using modules like GaussianAvatar and AnimatableGaussians) without needing a neural forward pass at runtime. It integrates with Habitat-Lab for task definition and training, and includes a web-based tool for creating NavMeshes (walkable areas) directly from 3DGS scenes.

Who it’s for

It is primarily for researchers in embodied AI and robotics who need highly realistic visual environments for training and evaluating navigation agents (such as PointNav, ImageNav, and ObjectNav) and vision-and-language navigation (VLN) tasks.

Highlights

  • Photo-realistic Rendering: Uses 3D Gaussian Splatting for high-fidelity visuals.
  • Dynamic Gaussian Avatars: Supports humanoid avatars with efficient CUDA LBS rendering.
  • Habitat Integration: Maintains compatibility with Habitat-Sim's scene dataset abstraction, NavMesh pathfinding, and Habitat-Lab integration.
  • Comprehensive Tooling: Includes a NavMesh editor, an interactive viewer, and an LLM-powered agent system called HabitatAgent for natural-language exploration.
  • Extensive Dataset: Provides a dataset of 129 high-quality GS scenes.

Related

  • Project
    xArm-Developer/xarm_ros

    A ROS-based development and simulation framework for the xArm series of robotic arms, providing hardware interfaces, MoveIt motion planning, and Gazebo simulation.

  • Project
    jolibrain/colette

    Colette is a self‑hosted RAG platform that indexes PDFs as images, preserving tables, figures and layout. It combines vision embeddings, BM25 lexical search, and optional HyDE prompting, and runs on high‑end GPUs via Docker. Ideal for confidential technical documentation.

  • Project
    OpenSimulationInterface/open-simulation-interface

    Open Simulation Interface (OSI) is a generic, protocol-buffer-based interface for the environmental perception of automated driving functions in virtual simulation scenarios.

  • Project
    jiupinjia/rocket-recycling

    A 2D rocket simulator using PPO reinforcement learning to train agents for stable hovering and precision landing, inspired by SpaceX rocket recovery.