facebookresearch/OrienterNet

Source Code for Paper "OrienterNet Visual Localization in 2D Public Maps with Neural Matching"

What it solves

OrienterNet addresses the challenge of visual localization—determining the exact position and orientation of a camera image within a large area—without relying on expensive and complex 3D point clouds. Instead, it uses globally available 2D semantic maps (like OpenStreetMap) to orient images.

How it works

The system uses a deep neural network to estimate a position and orientation by matching a neural Bird's-Eye-View (BEV) representation of the input image with 2D map data. This allows the model to localize an image by comparing visual features to the semantic layout of the environment as humans do.

Who it’s for

This project is designed for researchers and developers working on computer vision, robotics, and autonomous navigation who need a way to localize images using public 2D map data.

Highlights

  • 2D Map Integration: Uses OpenStreetMap data for localization rather than 3D reconstructions.
  • Neural Matching: Employs a neural Bird's-Eye-View approach to match images to maps.
  • Pre-trained Models: Provides pre-trained weights for immediate use via web and Colab demos.
  • Cross-Dataset Evaluation: Evaluated on Mapillary Geo-Localization (MGL) and KITTI datasets.

Related

  • Project
    rsasaki0109/lidar_localization_ros2

    A map-based 3D LiDAR localization system for ROS 2 and Nav2 that uses NDT and GICP algorithms to align sensor scans with 3D point cloud maps.

  • Project
    layumi/University1652-Baseline

    A multi-view, multi-source benchmark and baseline for drone-based geo-localization, enabling the matching of drone, satellite, and street-view images to locate buildings.

  • Project
    nianticlabs/map-free-reloc

    A reference implementation for map-free visual relocalization that enables metric pose estimation relative to a single image, removing the need for large 3D maps.

  • Project
    facebookresearch/boxer

    Boxer is a system that lifts open-world 2D object detections into 3D oriented bounding boxes for indoor scenes, utilizing posed images and semi-dense point clouds.

  • Project
    PeterFWS/Structure-PLP-SLAM

    A visual SLAM system that uses points, lines, and planes for efficient sparse mapping and localization across monocular, RGB-D, and stereo cameras.