Pointcept/Utonia

[ICML'26] Official repository of Utonia: Toward One Encoder for All Point Clouds

📦 Project: Utonia – One Encoder for All Point Clouds

TL;DR – Utonia ships a cross‑domain, pre‑trained Point Transformer V3 model for 3D point‑cloud tasks. It provides ready‑to‑use weights (tiny, small, full), inference code, and a set of visualisation demos. The repo can be installed as a quick‑start environment or as a Python package to embed the encoder in your own pipelines.

🎯 What is it?

Domain: 3D point‑cloud representation learning (self‑supervised pre‑training, downstream inference).
Core idea: A single encoder – the Point Transformer V3 – trained on many 3D datasets, aiming to work well on indoor, outdoor, and object‑level clouds without task‑specific finetuning.
Outputs: Hierarchical point features that can be visualised, fed to downstream heads (e.g., segmentation), or exported for downstream tasks.

✨ Highlights (from the README)

Pre‑trained weights (tiny, small, full) hosted on Hugging Face (Oct 2026).
Demo suite reproducing the visualisations from the Sonata and Concerto papers – PCA, similarity heatmaps, semantic segmentation, batch forwarding, video‑to‑point‑cloud demos.
Two installation modes – a stand‑alone Conda environment (includes CUDA 12.4, PyTorch 2.5, FlashAttention) for fast demo runs, and a package mode for integration into other codebases.
Docker images for RTX 4090‑class hardware (CUDA 12.8) provided by the community.

🛠️ Installation

1️⃣ Stand‑alone (quick demo)

conda env create -f environment.yml   # creates env "utonia" with CUDA 12.4, PyTorch 2.5, FlashAttention
conda activate utonia

(If you already have a CUDA install, unset CUDA_PATH first.)

2️⃣ Package mode (embed in your project)

# prerequisites: matching CUDA and PyTorch versions already installed
pip install spconv-cu${CUDA_VERSION}
pip install torch-scatter -f https://data.pyg.org/whl/torch-${TORCH_VERSION}+cu${CUDA_VERSION}.html
pip install git+https://github.com/Dao-AILab/flash-attention.git
pip install huggingface_hub timm
python setup.py install   # optional – copies the utonia package into your site‑packages

Additional runtime deps for the demo scripts:

pip install open3d fast_pytorch_kmeans psutil addict scipy camtools natsort opencv-python trimesh gradio numpy==1.26.4

(Open3D currently requires NumPy 1.x.)

🚀 Quick Start (demo & inference)

Visualisation demos

export PYTHONPATH=./
python demo/0_pca_indoor.py            # PCA on indoor scan
python demo/1_similarity.py           # similarity heat‑map
python demo/2_sem_seg.py              # linear‑probe segmentation on ScanNet (full encoder only)
python demo/3_batch_forward.py         # batch processing example
python demo/4_pca_hk.py                # PCA on HK dataset
python demo/5_pca_manipulation.py     # interactive manipulation demo
python demo/6_pca_object.py           # object‑level PCA
python demo/7_pca_outdoor.py           # outdoor scene PCA

Swap encoder size with --model utonia_small or --model utonia_tiny (most demos support it, except the ScanNet head).

Video‑to‑point‑cloud demo (requires VGGT)

git clone https://github.com/facebookresearch/vggt.git && cd vggt && pip install -e . && cd ..
python demo/8_pca_video.py \
  --input_video YOUR_VIDEO.mp4 \
  --conf_thres 0 \
  --prediction_mode "Depthmap and Camera Branch" \
  --if_TSDF \
  --pca_start 1 \
  --pca_brightness 1.2

Inference on your own data

import utonia, torch, numpy as np

# 1️⃣ Prepare a dict matching the expected schema
point = {
    "coord": np.random.rand(10000, 3).astype(np.float32),   # XYZ
    "color": np.random.rand(10000, 3).astype(np.float32),   # RGB (optional)
    "normal": np.random.rand(10000, 3).astype(np.float32),  # normals (optional)
}

# 2️⃣ Build the transform pipeline (same as used for the pre‑trained model)
transform = utonia.transform.default(scale=1.0, apply_z_positive=True, normalize_coord=False)
point = transform(point)

# 3️⃣ Move tensors to GPU
for k, v in point.items():
    if isinstance(v, torch.Tensor):
        point[k] = v.cuda(non_blocking=True)

# 4️⃣ Load the encoder (full, small, or tiny)
model = utonia.model.load("utonia", repo_id="Pointcept/Utonia").cuda()
# model = utonia.model.load("utonia_small").cuda()   # alternative

# 5️⃣ Forward pass – output is a hierarchical point feature dict
out = model(point)

# 6️⃣ (Optional) Collapse hierarchical features back to the original point count
for _ in range(2):
    parent = out.pop("pooling_parent")
    inv = out.pop("pooling_inverse")
    parent.feat = torch.cat([parent.feat, out.feat[inv]], dim=-1)
    out = parent
while "pooling_parent" in out:
    parent = out.pop("pooling_parent")
    inv = out.pop("pooling_inverse")
    parent.feat = out.feat[inv]
    out = parent
feat_original = out.feat[out.inverse]   # (N, C) aligned with the input cloud

The resulting feat_original can be fed to downstream heads (e.g., a linear classifier) or visualised with the provided demo scripts.

📄 License

Code: Apache 2.0 (inherited from Meta’s Sonata code).
Pre‑trained weights: CC‑BY‑NC 4.0 (non‑commercial use only, due to dataset restrictions such as HM3D, ARKitScenes).

📚 Citation

If you use Utonia in research, please cite the ICML 2026 paper:

@inproceedings{zhang2026utonia,
  title={Utonia: Toward One Encoder for All Point Clouds},
  author={Yujia Zhang and Xiaoyang Wu and Yunhan Yang and Xianzhe Fan and Han Li and Yuechen Zhang and Zehao Huang and Naiyan Wang and Hengshuang Zhao},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2026}
}

Additional references to the underlying Point Transformer V3, Sonata, Concerto, and related works are listed in the README for context.

Bottom line: Utonia gives you a ready‑to‑run, high‑performance 3‑D point‑cloud encoder with pre‑trained weights and a suite of visual demos, plus a clean Python package for integration into custom pipelines.