mkalten/reacTIVision

computer vision framework for tangible interactive surfaces

What it solves

reacTIVision is a computer vision framework designed for the fast and robust tracking of fiducial markers on physical objects and multi-touch finger tracking. It specifically targets the development of table-based tangible user interfaces (TUI) and multi-touch interactive surfaces, such as the reacTable modular synthesizer.

How it works

The application processes real-time video from a supported camera to detect the ID, position, and rotation of fiducial markers (using sets like amoeba and yamaarashi). It also identifies "blobs"—round white regions interpreted as fingers or untagged objects—by analyzing image contrast and size. To handle wide-angle lens distortion, it includes a calibration system that corrects the found positions of markers and fingers.

Data is transmitted to client applications via the TUIO protocol using UDP, TCP, or Websockets (OSC messages), allowing the users to build interactive applications in languages like C++, C#, Java, Processing, Pure Data, and Max/MSP.

Who it’s for

Developers and artists creating tangible user interfaces, interactive installations, and multi-touch tabletop surfaces.

Highlights

  • Multi-modal tracking: Supports fiducial markers, multi-touch finger tracking, and general blob tracking.

  • TUIO Protocol: Implements the TUIO protocol for standardized communication with client applications.

  • Lens Distortion Correction: Built-in tools to calibrate and correct for wide-angle lens distortion.

  • Cross-platform: Runs on Windows, Mac OS X, and Linux with broad camera support (WDM, UVC, Video4Linux2).

  • Image Processing Tools: Includes image equalization and adjustable thresholding to improve recognition in various lighting conditions.

Related

  • Project
    oculix-org/SikuliX1

    SikuliX1 is a historical, read-only archive of a computer vision-based automation tool that allows users to interact with graphical interfaces by identifying elements visually on the screen.

  • Project
    ok-oldking/ok-script

    ok-script is a Python-based computer vision automation framework that enables developers to create cross-platform scripts for Windows applications, games, and Android devices.

  • Project
    jaredrhod/barehands

    A webcam-based hand-tracking interface that lets you manipulate notes, images, and 3D models spatially, designed to be wired into an AI agent as a visual and interactive body.

  • Project
    roboflow/supervision

    A computer vision toolkit that provides model-agnostic building blocks for data loading, visualization, and dataset management to accelerate the development of vision applications.

  • Project
    RizwanMunawar/yolov7-object-tracking

    A computer vision project that integrates YOLOv7 and YOLOv8 for real-time object detection and tracking across video streams from various sources.