Morphable models are generative and have an intuitive interface.
Computer science is increasingly embracing 3D modeling. Since the first presentation of 3D Morphable Face Models at SIGGRAPH ’99. in 2009, it has been more than years. The paper had a lasting impact both on research and applications. The morphable model has been a major advancement in deep learning over the past few years and was incorporated into many of today’s state-of-the art solutions for face analysis.
The first paper, however, is strong and can be scaled to build meaningful models of different objects. Let’s see what this paper is about. It was necessary to create a model of faces. They presented an approach to creating a generative model that has an intuitive interface.
A generative model is an algorithm that creates faces. It also has parameters that control its output. These parameters should be easy to understand by humans, according to the design of the algorithm. This technique can be used for other objects. You can. Neatsy created one for the foot.
You can see it on the diagram below. Let’s now see how it all works. We need to first understand the 3D model. A basic 3D model is composed of two parts: vertexes, which are points in the space, and faces. These triangular vertexes form the triangles. The model is connected by vertexes, which define the shapes of its faces.
The mesh is made up of faces and vertexes. A mesh contains more vertex than faces. This results in more 3D objects. Chrschn’s example of a 3-D model Each vertex coordinate can be changed, however these changes will most likely not make sense because it will cause the object structure to break.
We want the best possible model changes. We need to decompose our model from this perspective. This is an old task in machine learning. It solves two problems: noise and dimension reduction. There are many solutions to this problem. PCA is one of the most common. PCA is an excellent choice because it is very simple to use.
It works by finding vectors that have the largest data change. Let’s look at PCA. PCA uses a matrix of (n,d) shapes, where n represents the number and d the number feature. Every object in our example is a scan foot and each feature is a vertex coordinate.
The number of features is 3v. v refers to the number vertexes. One of the most important features of matrix X, is that each column must correspond to a particular point coordinate. To illustrate, every feature in a column should match the x coordinate for thumb finger edge.
All scans must contain the same number points. Also, these points should be arranged in the exact same order across all scans. Side effect: we get the dataset with identical faces overall meshes. It is difficult and fun to collect such a large dataset of feet, so we’ll skip this part.
You can also find the dataset details of the face model in original paper. Once we have our dataset, it is time to apply PCA. PCA can be applied to many data in many packages. We will use notation in sklearn.decomposition. Three fields are of interest to us. PCA.mean_ – This is a model that uses mean feet.
It will be used as the zero point of our morphable model. PCA.componets_ — This array contains vectors that indicate the direction of greatest change. PCA.explained_variance_ratio_ — This array with importances of every PCA component. These values will help us decide the number of components we need for our model.
We are now ready to build the model. This is the formula: Image by author. Using this formula, we create alpha parameters for the model. We then need to visualise the result. We don’t need to alter any faces in the dataset so we simply add them to our mesh. Although the result generative model looks linear at first glance, they are powerful enough to give meaning to each component.
However, these components have great practical value in real life. The interactive plot shows that the width parameter controls the height, pronation and width parameters. All these parameters can be used to describe foot. This is how you build such a model. Let’s now talk about how to apply it.
This model can be used to create real-world objects. A morphable model can be used to describe a 3D object. We can then find the parameters that will fit the object in the photograph. This allows us to interpolate non-visible areas or change camera settings. Morphable Model explained was first published on Medium by . People are responding and highlighting this story. Published via