pwilkin/trellis.cpp

What it solves

It provides a standalone, high-performance C++ implementation of the TRELLIS image-to-3D pipeline. It eliminates the need for Python at runtime, allowing users to generate UV-textured 3D models (GLB files) from a single image without the heavy dependencies of the original reference implementation.

How it works

Built on the GGML library, the project ports the TRELLIS.2-4B pipeline into native C++. The process follows several stages:

1. Preprocessing: Removes backgrounds using BiRefNet/RMBG and extracts image features via a DINOv3 ViT-L extractor.
2. Neural Generation: Uses three flow-based Diffusion Transformers (DiT) to generate sparse structure, shape, and texture latents.
3. Decoding: Converts these latents into occupancy voxels and then into a textured mesh using a FlexiDualGrid shape decoder and a Sparse U-Net texture decoder.
4. Post-processing: Performs welding, remeshing (narrow-band UDF dual contouring), quadric-simplified decimation, UV unwrapping (xatlas), and PBR baking to produce a final GLB export.

Who it’s for

• Developers and artists who want a fast, local image-to-3D workflow without Python.
• Users on diverse hardware (NVIDIA, AMD, Apple Silicon, and Vulkan-compatible GPUs) who need an optimized 3D generation tool.
• App developers looking for a C++ backend for 3D generation (e.g., as used in the Lemonade SDK).

Highlights

• Native C++/GGML: No Python required at runtime.
• Cross-Platform GPU Support: Optimized for CUDA, ROCm, Vulkan, and Metal (Apple Silicon).
• Trellis Studio: A bundled desktop app (Tauri) for a drag-and-drop image-to-3D experience.
• High Fidelity: Matches the reference CUDA post-process quality with adaptive triangulation and PBR textures.
• Memory Efficient: Uses FlashAttention with padded K/V to run high-resolution cascades on 16 GB GPUs.