xibushu/UniForge3D

UniForge3D – Joint 3D Reconstruction & Generation

What it does

• Takes a handful of unaligned, sparse photos of an object and produces a complete 3D asset.
• The model learns to reconstruct the observed geometry while generating missing parts, all inside a shared “anchor‑guided” latent space.
• A structure‑guided view‑selection step keeps fine details that appear in the input images.

Why it matters

• Traditional 3D reconstruction needs many calibrated views; UniForge3D works with just a few unordered shots, lowering the barrier for creating 3D assets.
• By coupling reconstruction with generative completion, it can fill in occluded regions plausibly, useful for content creation, AR/VR, and robotics simulation.

Installation

# Clone the repo and run the provided setup script
git clone https://github.com/xibushu/UniForge3D.git
cd UniForge3D
bash setup.sh   # installs dependencies, builds environment
conda activate uniforge3d

The script creates a Conda environment named uniforge3d with the required Python packages and downloads the pretrained weights from Hugging Face.

Running inference

1. Organise each object's input images in its own folder under assets/:

assets/
├── case01/
│   ├── 001.png
│   ├── 002.png
│   └── ...
└── case02/ …

2. Execute the inference script:

python infer.py

The script loads the pretrained model, selects informative views, and writes out the generated 3D mesh (and optionally texture) for each case.

Key components & influences

• Built on top of Microsoft’s TRELLIS.2 and Facebook‑Research’s VGGT codebases, inheriting their multi‑view geometry handling and large‑scale generative transformer back‑bone.
• Uses the BRIA RMBG‑2.0 background remover as a preprocessing step (requires separate commercial licence).

License

• The repository’s own source code is released under the MIT License.
• Model weights are not MIT‑licensed: they inherit the original licenses (e.g., VGGT‑1B is CC BY‑NC 4.0, BRIA RMBG‑2.0 has its own commercial‑use restrictions).

Citation If you use UniForge3D in academic work, cite the arXiv paper:

@article{chen2026uniforge3d,
  title   = {UniForge3D: Joint 3D Reconstruction and Generation in a Shared Anchor-Guided Space},
  author  = {Chen, Wenjue and Chen, Wenyue and Zheng, Heliang and Jia, Rongfei and Liu, Yuan and Li, Meng},
  journal = {arXiv preprint arXiv:2610.15357},
  year    = {2026}
}

Where to find more

• Project page with visual results: https://xishuxishu.github.io/uniforge3d-project/
• Pretrained models on Hugging Face: https://huggingface.co/xibushu/UniForge3D
• Full paper (arXiv): https://arxiv.org/abs/2610.15357