zju3dv/InfiniDepth

InfiniDepth

InfiniDepth is a research‑grade codebase for single‑image 3D perception. It uses neural implicit fields to produce depth maps at any resolution and to synthesize 3D Gaussian‑splatting (3DGS) representations, optionally fusing sparse depth from a sensor to obtain metric‑scale results.

What it does

Capability	Input	Output
Arbitrary‑resolution depth	RGB image	Depth map at any resolution (relative or metric)
Monocular view synthesis	RGB image	3D Gaussian‑splatting model + optional novel‑view video
Depth‑sensor augmentation	RGB + sparse depth	Metric depth + 3DGS

Quick start

1. Install – follow the step‑by‑step guide in INSTALL.md (Python, PyTorch, required checkpoints).
2. Try the demo – a public Gradio app is hosted on Hugging Face:

   python app.py   # runs the same interface locally
   

   Upload an RGB image (and a depth map if you have one) and choose Depth or 3DGS.
3. Run inference from the command line – example scripts are provided under example_scripts/:

   # relative depth from a single RGB image
   bash example_scripts/infer_depth/courtyard_infinidepth.sh
   
   # 3D Gaussian from a single RGB image
   bash example_scripts/infer_gs/courtyard_infinidepth_gs.sh
   
   # metric depth using a depth sensor
   bash example_scripts/infer_depth/eth3d_infinidepth_depthsensor.sh
   

   The scripts call inference_depth.py or inference_gs.py with sensible defaults; you can change output resolution, enable sky‑masking, or export point clouds via the command‑line flags documented in the README.

Multi‑view / video support

inference_multi_view_depth.py can process a folder of images or a video, producing per‑frame depth maps, aligned point clouds, and a merged global point cloud. It can either rely on the DA3 depth‑anything model for scale alignment or use explicit camera intrinsics/extrinsics (Waymo‑style txt files).

Training & fine‑tuning

The repository includes a full training pipeline (main.py). After setting the environment variables workspace (where experiment outputs are stored) and commonspace (shared datasets / pretrained weights), you can:

• Fine‑tune an existing checkpoint:

  python3 main.py \
    --cfg_file training/exp_configs/exps/infinidepth.yaml \
    --include training/exp_configs/components/data/train/infinidepth_train_hypersim.yaml \
    ckpt_path=checkpoints/depth/infinidepth.ckpt \
    exp_name=finetune_infinidepth_on_hypersim \
    pl_trainer.devices=8
  

• Train from scratch by omitting ckpt_path.
• Validation is performed automatically on a mixed real‑data benchmark after each epoch.

Resources

• Project page: https://zju3dv.github.io/InfiniDepth/
• Paper: https://arxiv.org/abs/2601.03252 (CVPR 2026)
• Hugging Face demo: https://huggingface.co/spaces/ritianyu/InfiniDepth
• Dataset used for demos: https://huggingface.co/datasets/ritianyu/game_4k_data

---

InfiniDepth is aimed at researchers and developers who need high‑quality depth or 3D‑Gaussian reconstructions from a single image, with optional sensor fusion for metric accuracy.