zju3dv/AutoRecon

AutoRecon – Automated 3D Object Discovery & Reconstruction

What it is – AutoRecon is an open‑source pipeline that takes ordinary RGB image sequences (or datasets in standard formats) and automatically discovers individual objects, segments them, and reconstructs their 3‑D geometry as neural surfaces. It builds on the NeRFStudio and SDFStudio frameworks, re‑using their neural rendering and signed‑distance‑function (SDF) capabilities.

Key components

• AutoDecomp (in a separate repo) – coarse scene decomposition that finds and isolates objects from a casual capture.
• Neural surface reconstruction – the code in this repository fits an SDF‑based neural representation to each discovered object and can export a mesh.

Typical workflow

1. Prepare data – place a folder of sequential images (or a dataset in IDR/BlendedMVS/CO3D format) under data/.
2. Run the pipeline – execute one of the provided shell scripts, e.g. exps/code-release/run_pipeline_demo_low-res.sh. The script runs structure‑from‑motion, object discovery, and SDF training automatically.
3. Extract a mesh – after training, call ns-extract-mesh with the generated config.yml and model checkpoint to produce a .ply mesh. Optional post‑processing (ambient‑occlusion‑based internal‑geometry removal) can be done with MeshLab.

Installation – Detailed steps are in docs/INSTALL.md. The project depends on the Python packages from nerfstudio, sdfstudio, nerfacc, and tyro, and on a recent PyTorch build.

Why it matters – Traditional 3‑D reconstruction pipelines require manual object segmentation and careful camera calibration. AutoRecon automates the discovery step, making it possible to reconstruct individual objects from everyday video captures without hand‑labeling, which is useful for robotics, AR/VR content creation, and research on neural scene representations.

Citation – If you use the code in academic work, cite the CVPR 2023 paper:

@inproceedings{wang2023autorecon,
  title={AutoRecon: Automated 3D Object Discovery and Reconstruction},
  author={Wang, Yuang and He, Xingyi and Peng, Sida and Lin, Haotong and Bao, Hujun and Zhou, Xiaowei},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={21382--21391},
  year={2023}
}

Acknowledgements – The implementation re‑uses substantial code from nerfstudio, sdfstudio, nerfacc, and tyro.