Texture Estimation

Overview

Here, we learn hands-on how to estimate a diffuse texture entirely from images. We’ll load a mesh, render a series of training images, and optimize an instance of TexturesUV to approximate the appearance of those images. We’ll use principles from rasterization, textures, materials, lighting, and machine learning to do this.

Set up and load data

First, we need to load the 3D model that we’ll be using. We’ll render images of this textured mesh as our ground truth. We’ll then attempt to learn a texture map for the model based on the rendered images alone.

Load mesh with textures

First, we load our trusty cow.obj 3D model and inspect the texture that it comes with. The model includes a 2D image texture and a set of UV coordinates that match vertices to locations in the 2D image. PyTorch3D provides a handy utility function called texturesuv_image_matplotlib that will use Matplotlib to display the UV coordinates on top of the texture image.

Here, we load this 3D model and use the PyTorch3D utilities to inspect its textures and UVs.