junhong-3dv/s2m2

What it solves

S²M² provides a high-performance solution for stereo matching, which is the process of calculating depth information from two offset camera images. It addresses the need for reliable, scalable depth estimation that can simultaneously predict disparity, occlusion, and confidence levels.

How it works

It uses a deep learning architecture designed for stereo matching that operates under the pinhole camera model. The model takes a pair of rectified stereo images as input and processes them through a scalable network to output depth maps. It includes an iterative refinement module to improve accuracy and supports various precision modes (FP16/FP32) to balance speed and performance. The architecture is optimized for deployment, allowing for conversion to ONNX and TensorRT formats for real-time inference.

Who it’s for

This project is for researchers and computer vision engineers working on 3D reconstruction, robotics, or autonomous systems who need state-of-the-art depth estimation capabilities. It is particularly useful for those looking to deploy high-accuracy stereo matching models in production environments using NVIDIA hardware.

Highlights

• State-of-the-art performance on major benchmarks including ETH3D, Middlebury V3, and Booster.
• Jointly estimates disparity, occlusion, and confidence for more robust depth maps.
• Highly optimized for deployment with support for TorchScript, ONNX, and TensorRT.
• Offers multiple model sizes (S, M, L, XL) to allow users to trade off between computational speed and accuracy.