zju3dv/NeuralRecon-W

Code for "Neural 3D Reconstruction in the Wild", SIGGRAPH 2022 (Conference Proceedings)

What it solves

NeuralRecon-W addresses the challenge of reconstructing 3D scenes from images taken "in the wild" (uncontrolled environments), which often include transient objects like people or cars that can interfere with the 3D reconstruction process.

How it works

The project implements a neural 3D reconstruction pipeline that uses a Signed Distance Function (SDF) to represent the same scene. It integrates with COLMAP for camera pose estimation and provides tools to generate semantic maps to help the system distinguish between the static scene and transient objects. The process involves training a neural network to represent the 3D geometry of a scene from a set of images, and then extracting a 3D mesh from the same network's learned representation.

Who it’s for

Researchers and developers working in computer vision, 3D reconstruction, and neural rendering, specifically those dealing with outdoor or indoor scenes captured in uncontrolled environments.

Highlights

  • Wild Scene Reconstruction: Specifically designed for handling images captured in uncontrolled environments.
  • Mesh Extraction: Includes tools to extract high-quality 3D meshes from trained checkpoints.
  • Custom Data Support: Supports automatic and manual data preparation from standard COLMAP workspace formats.
  • Heritage-Recon Dataset: Provides a toolkit and pipeline to reproduce results on a specialized dataset for heritage sites.

Related

  • Project
    zcablii/LSKNet

    A lightweight foundation backbone for remote sensing that uses a Large Selective Kernel mechanism to dynamically adjust receptive fields for better object detection and segmentation.

  • Project
    hzxie/CityDreamer

    CityDreamer is a research‑grade PyTorch framework that generates unlimited, photorealistic 3‑D cityscapes by combining a layout generator, a background‑stuff generator, and a building instance generator. It includes code for training on large OSM/Google‑Earth datasets, pre‑trained checkpoints, a web demo, and a CLI for rendering videos.

  • Project
    microsoft/BiomedParse

    A foundation model for joint segmentation, detection, and recognition of biomedical objects across nine imaging modalities, supporting both 2D and 3D volumetric inference.

  • Project
    OCR4all/OCR4all

    An open-source tool providing a semi-automatic OCR workflow specifically designed for high-quality text recognition of historical printings and early printed books.