facebookresearch/projectaria_tools

projectaria_tools is an C++/Python open-source toolkit to interact with Project Aria data.

What it solves

It provides a standardized set of tools for researchers to process and analyze data from Project Aria glasses (Gen1 and Gen2). This simplifies the workflow for those working on augmented reality (AR), machine perception, and artificial intelligence by providing unified APIs to handle complex sensor data and on-device machine learning outputs.

How it works

The project offers a suite of C++ and Python utilities that interface with Aria VRS files. It allows users to load sensor streams, handle device calibration (intrinsics and extrinsics), align timestamps across multiple devices, and access on-device perception signals. For Gen2 data, it includes specific tools for exporting machine perception CSVs and interactive 3D visualization via the aria_rerun_viewer.

Who it’s for

Researchers in the fields of machine perception, computer vision, and AR/AI who are utilizing Project Aria hardware and its associated open datasets to build and train new models.

Highlights

  • Unified API: A single Python/C++ interface that supports both Gen1 and Gen2 hardware data.
  • Comprehensive Sensor Support: Access to RGB cameras, CV cameras, eye tracking, hand tracking, proximity, PPG health, and GNSS sensors.
  • On-Device Perception: Tools to utilize on-device VIO/SLAM, 21-keypoint hand tracking, and gaze tracking.
  • Data Alignment: Specialized utilities for multi-device timestamp alignment and sequential multi-sensor access.

Related

  • Project
    voxel51/fiftyone

    An open-source tool for building high-quality computer vision datasets and models by providing visual data exploration, labeling, and model evaluation.

  • Project
    luxonis/depthai-core

    A C++ and Python library for interfacing with Luxonis DepthAI hardware to build spatial AI and computer vision applications.

  • Project
    lycorp-jp/sim-use

    A cross-platform CLI that enables AI agents to observe and act on iOS and Android screens using token-efficient UI outlines and alias-based interactions.

  • Project
    spear-sim/spear

    A Python library for photorealistic embodied AI research that provides deep programmatic control over Unreal Engine applications with high-speed rendering and extensive API access.

  • Project
    ailia-ai/ailia-models

    A collection of 419 pre-trained AI models across various domains like vision, audio, and text, all runnable via a unified CLI with automatic weight downloads.