sunyitong/DeepMetricEye

Research code for metric depth estimation in periocular VR imagery using UE MetaHuman-generated data.

What it solves

DeepMetricEye addresses the limitation of 2D eye-tracking cameras in VR headsets, which cannot provide precise metric measurements of the eye region. It enables the reconstruction of 3D metric geometry—such as pupil diameter and periocular deformation—from a single 2D image.

How it works

The project consists of two main components:

  1. Depth-Estimation Model: A lightweight PyTorch-based monocular depth-estimation model that predicts depth maps from periocular RGB images.
  2. DPDG Environment: A synthetic data-generation tool built in Unreal Engine 5.2 using MetaHumans. This environment allows researchers to generate synchronized RGB images and ground-truth depth maps to train and validate the model.

Who it’s for

It is designed for researchers in VR eye-health, XR sensing, and computer vision who need to extract precise physical measurements from headset-based eye imagery.

Highlights

  • Synthetic Data Pipeline: Uses Unreal Engine and MetaHumans to create high-fidelity training pairs.
  • Metric Accuracy: Focuses on real-world metric depth rather than relative depth.
  • Research-Ready: Includes a reproducible PyTorch implementation and a data-generation framework.