Maelic/RelateAnything

What it solves

RelateAnything provides a way to predict the relations between objects in an image in real-time. Unlike traditional models, it is "open-vocabulary," meaning you can specify the relations you are looking for (e.g., "riding," "beside," "in front of") at inference time without needing to retrain the model.

How it works

The system uses a DINOv3 backbone to process an image and convert object regions into visual tokens. A pair sampler identifies candidate relations, and a transformer refines these. Finally, a vocabulary head scores these pairs against text embeddings of the user-provided relation phrases, allowing the system to distinguish between spatial and semantic relations.

Who it’s for

This tool is designed for developers and researchers working on scene graph generation, computer vision, and real-time object relation prediction.

Highlights

• Open-Vocabulary: Define custom relation labels at runtime without retraining.
• Real-Time Performance: Optimized for speed, with the ViT-S+ model taking approximately 17.2ms per image on an RTX 3080 Laptop GPU using TensorRT.
• Flexible Inputs: Accepts object regions as boxes or masks from any detector or segmenter.
• Multi-Platform Deployment: Supports PyTorch, TensorRT, OpenVINO, and even browser-based inference via WebGPU or WASM.