zsibot/matrix

MATRiX is an advanced simulation platform that integrates MuJoCo, Unreal Engine 5, and CARLA to provide high-fidelity, interactive environments for robotics research.

What it solves

MATRiX is a high-fidelity robot simulation platform designed to bridge the gap between simulated environments and physical robot deployment. It solves the problem of slow robot onboarding and the lack of realistic sensor data, providing a unified workflow for testing perception, SLAM, and motion control without requiring a full ROS 2 installation for the core simulator.

How it works

The platform combines Unreal Engine 5 for high-fidelity rendering and MuJoCo for physics dynamics. It allows users to load MJCF/XML robot models at runtime via JSON configuration, avoiding the need to rebuild the simulator. It uses a dedicated MuJoCo worker running at 500 Hz for high-rate physics and control. For data communication, it employs Zenoh to stream ROS 2-compatible CDR messages (like PointCloud2 and compressed images) over a network, making it ROS-compatible without being dependent on ROS.

Who it’s for

  • Robotics Researchers: Those developing perception, SLAM, sensor-fusion, and synthetic data.
  • Control Engineers: Those working on locomotion, whole-body control, and reinforcement learning.
  • System Integrators: Developers integrating external controllers before deploying to physical hardware.
  • Demo Developers: Users needing multi-scene testing and browser-based visualization via Pixel Streaming 2.

Highlights

  • Runtime Robot Onboarding: Load and configure robots and sensors via JSON without rebuilding the simulator.
  • Calibrated Perception: Support for RGB, infrared, GPU depth, fisheye lenses, and LiDAR with motion-distortion modeling.
  • Open Data Plane: Zenoh-based communication for ROS 2-compatible data streaming.
  • Flexible Control: Built-in motion control or integration with external controllers via a shared-memory hardware interface.
  • Expandable Environments: Dynamic map catalog with DLC-style downloads and support for Gaussian-splatting rendering.

Related

  • Project
    URLab-Sim/UnrealRoboticsLab

    An Unreal Engine 5 plugin that integrates the MuJoCo physics engine to provide high-fidelity robotics simulation with photorealistic rendering.

  • Project
    ARISE-Initiative/robosuite

    A MuJoCo-powered simulation framework and benchmark suite for robot learning, providing modular tools to create environments and test AI control algorithms.

  • Project
    google-deepmind/mujoco_playground

    A GPU-accelerated suite of robot learning environments built with MuJoCo MJX, designed for efficient reinforcement learning and sim-to-real research.

  • Project
    google-deepmind/mujoco

    MuJoCo is a general-purpose physics engine from Google DeepMind for simulating articulated structures that interact with their environment. It provides a high-performance C API, Python bindings, and interactive 3D visualization, making it a popular tool in robotics and machine learning research.

  • Project
    robfiras/loco-mujoco

    An imitation learning benchmark for whole-body control featuring diverse humanoid and quadruped environments with extensive motion capture datasets.