Tobias-Fischer/rt_gene

What it solves

This project provides tools for real-time eye gaze and blink estimation in natural environments. It addresses the challenge of accurately tracking where a person is looking and whether they are blinking using standard camera images, which is particularly useful for human-robot interaction and vision-based monitoring.

How it works

The system consists of two primary components: RT-GENE for gaze direction estimation and RT-BENE for blink estimation. It processes face and eye images to determine gaze direction and blink probability. The current implementation is a ROS 2 runtime port using PyTorch and OpenCV, organized into core Python logic for inference and ROS 2 nodes for integration into robotic systems. It supports multiple hardware backends including CUDA, Apple Metal (MPS), and CPU.

Who it’s for

This software is designed for researchers and developers working in computer vision, robotics, and human-computer interaction who need to integrate real-time eye tracking and blink detection into their applications.

Highlights

• Dual Functionality: Combines gaze estimation (RT-GENE) and blink detection (RT-BENE).
• ROS 2 Integration: Provides a modern runtime port with dedicated ROS 2 interfaces and nodes for easy integration into robotic pipelines.
• Cross-Platform Support: Compatible with Linux, macOS (arm64), and Windows, with optimized device selection for different GPUs.
• Standalone Demos: Includes scripts to run inference on single images without requiring a full ROS 2 environment.