zju3dv/manhattan_sdf

Manhattan‑SDF – Neural 3D Reconstruction with a Manhattan‑world Prior

What it is – A research codebase that reconstructs indoor 3‑D scenes from RGB‑D images by learning a signed‑distance‑function (SDF) representation. It extends recent neural rendering approaches (NeRF, VolSDF, NeuS, etc.) with a Manhattan‑world assumption, i.e., most surfaces are aligned with three orthogonal axes, which helps regularise the geometry and improve reconstruction quality on cluttered indoor scans.

Key components

• Neural SDF model built on the VolSDF framework, modified to enforce axis‑aligned planar priors.
• Training pipeline that ingests multi‑view ScanNet data (or custom datasets) and optimises the SDF together with camera poses.
• Mesh extraction script that converts the learned SDF into a watertight .obj mesh.
• Evaluation utilities for quantitative comparison against baselines such as COLMAP, ACMP, NeRF, UNISURF, NeuS, and VolSDF.

Getting started

1. Environment – Run conda env create -f environment.yml and activate with conda activate manhattan.
2. Data – Download the pre‑processed ScanNet scenes from the provided Google‑Drive link and place them under data/ as required by the config files.
3. Training – Example command:

   python train_net.py --cfg_file configs/scannet/0050.yaml gpus 0, exp_name scannet_0050
   

4. Mesh extraction – After training, run:

   python run.py --type mesh_extract --output_mesh result.obj \
       --cfg_file configs/scannet/0050.yaml gpus 0, exp_name scannet_0050
   

5. Evaluation – Use the same run.py with --type evaluate to reproduce the paper’s numbers.

Custom data – The repository includes a docs/CUSTOM.md guide that explains how to adapt the pipeline to your own image sets (camera intrinsics, pose format, etc.).

Why it matters – By leveraging the Manhattan‑world prior, the method achieves sharper, more accurate indoor reconstructions than vanilla neural SDF/NeRF approaches, especially in scenes with many axis‑aligned walls and furniture. This makes it useful for applications such as indoor mapping, AR/VR content creation, and robotics perception.

Citation – If you use the code, cite the CVPR‑2022 paper:

@inproceedings{guo2022manhattan,
  title={Neural 3D Scene Reconstruction with the Manhattan-world Assumption},
  author={Guo, Haoyu and Peng, Sida and Lin, Haotong and Wang, Qianqian and Zhang, Guofeng and Bao, Hujun and Zhou, Xiaowei},
  booktitle={CVPR},
  year={2022}
}

Acknowledgements – Builds on VolSDF (Lior Yariv), COLMAP, and several open‑source implementations referenced in the README.