Converting Vertex-Colored Meshes to Textured Meshes

Hugging Face has released a guide and a supporting library, InstantTexture, to convert 3D meshes with vertex colors into UV-mapped, textured meshes. This process is critical because while many generative 3D models (such as InstantMesh) produce vertex-colored meshes, most downstream 3D applications require UV mapping and textures for rendering.

InstantTexture Library for Rapid Conversion

The fastest way to perform this conversion is via the InstantTexture library, which automates the technical pipeline of UV mapping and texture generation.

Installation and Usage

Users can install the library directly from GitHub:

pip install git+https://github.com/dylanebert/InstantTexture

To convert a vertex-colored .obj mesh to a UV-mapped .glb mesh, the following implementation is used:

from instant_texture import Converter

input_mesh_path = "path/to/mesh.obj"
converter = Converter()
converter.convert(input_mesh_path)

Technical Pipeline for Texture Conversion

For those implementing the conversion manually, the process involves several discrete geometric and image-processing steps to translate vertex data into a 2D image map.

UV Map Generation and Remapping

The pipeline uses the xatlas library to generate UV maps. This is the most computationally expensive part of the process. Once the UV map is generated, the vertices and vertex colors are remapped to align with the new UV coordinates, and the mesh faces are updated accordingly.

Texture Buffer Filling via Barycentric Interpolation

To create the texture, the system constructs a texture buffer (typically upscaled by a factor of 2 to improve quality) and fills it using the following logic:

  1. Barycentric Interpolation: Calculates the interpolated color at a specific point within a triangle using the colors of its three vertices.
  2. Point-in-Triangle Test: Determines if a specific pixel coordinate in the texture buffer lies within the triangle defined by the UV coordinates of a mesh face.
  3. Filling Loop: The system iterates through every face, determines its bounding box on the texture buffer, and applies the point-in-triangle test to assign interpolated colors to the corresponding pixels.

Texture Refinement and Post-Processing

Initial texture generation often results in "holes" or artifacts. To resolve this, the pipeline applies four sequential image-processing techniques:

  • Inpainting: Fills holes using the average color of surrounding pixels via cv2.inpaint.
  • Median Filter: Removes noise by replacing pixels with the median color of their neighborhood.
  • Gaussian Blur: Smooths the texture to eliminate remaining noise.
  • Downsampling: Resizes the image to the final desired texture size (e.g., 1024x1024) using LANCZOS resampling.

Final Mesh Construction and Limitations

Once the refined texture is generated, a PBRMaterial is created using trimesh, assigning the generated image as the baseColorTexture. This material is then applied to the mesh using TextureVisuals with the generated UV coordinates.

Current Limitations

While this method provides a quick conversion path, it has known limitations:

  • Artifacts: The resulting meshes may still contain small visual artifacts.
  • Production Quality: The quality of the generated UV maps and textures is currently below the standards required for professional production-ready meshes.

This approach is intended as a utility for users needing a rapid transition from vertex-colored generative outputs to standard textured formats.

Sources