zsibot/matrix

MATRiX is an advanced simulation platform that integrates MuJoCo, Unreal Engine 5, and CARLA to provide high-fidelity, interactive environments for robotics research.

What it solves

MATRiX is a high-fidelity robot simulation platform designed to bridge the gap between virtual testing and physical deployment. It eliminates the need to rebuild the simulator for every new robot model or environment, allowing developers to load robots and configure sensors via JSON at runtime. It also provides a high-rate physics engine to ensure that control algorithms are tested against realistic dynamics and sensor noise.

How it works

The platform combines Unreal Engine 5 for high-fidelity visual rendering and MuJoCo for precise physics dynamics. It uses a dedicated worker to run MuJoCo simulations at a target rate of 500 Hz. Data is streamed via Zenoh using ROS 2-compatible message layouts, allowing the simulator to run standalone without requiring a full ROS 2 installation. The system supports dynamic map loading (DLCs) and a wide array of calibrated sensors, including LiDAR, RGB, and fisheye cameras.

Who it’s for

  • Perception and SLAM developers: Those building sensor-fusion and synthetic data pipelines.
  • Control engineers: Those working on locomotion, whole-body control, and reinforcement learning.
  • Robotics researchers: Users needing high-fidelity environments for controller regression and simulated-hardware integration.

Highlights

  • Runtime Onboarding: Load MJCF/XML robot models and configure sensor suites via JSON without recompiling.
  • High-Rate Physics: Dedicated MuJoCo worker targeting 500 Hz for realistic control loops.
  • Calibrated Perception: Support for GPU depth, fisheye lens models, and LiDAR motion-distortion modeling.
  • ROS 2 Compatibility: Uses Zenoh to publish ROS 2-compatible CDR messages while remaining ROS-independent.
  • Expandable Environments: Supports Gaussian-splatting rendering and on-demand map DLC downloads.

Related

  • Project
    URLab-Sim/UnrealRoboticsLab

    An Unreal Engine 5 plugin that integrates the MuJoCo physics engine to provide high-fidelity robotics simulation with photorealistic rendering.

  • Project
    ARISE-Initiative/robosuite

    A MuJoCo-powered simulation framework and benchmark suite for robot learning, providing modular tools to create environments and test AI control algorithms.

  • Project
    vikashplus/robohive

    A unified framework for robot learning that provides a collection of MuJoCo-simulated environments and tasks exposed via the OpenAI-Gym API.

  • Project
    google-deepmind/mujoco

    A high-performance physics engine for the fast and accurate simulation of articulated structures, widely used in robotics and machine learning research.

  • Project
    google-deepmind/mujoco_playground

    A GPU-accelerated suite of robot learning environments built with MuJoCo MJX, designed for efficient reinforcement learning and sim-to-real research.