METHOD OF CREATING AN ANIMATED REALISTIC 3D MODEL OF A PERSON

Abstract

The present invention provided a method for animating 3D models of people.

Claims

1. A method of producing a 3-dimensional model of a person's body capable of animation comprising the steps of: a) providing a static 3D model of a person's body defined as polygons with vertices represented as 3D points and a texture with UV mapping; b) defining a cost function between the static 3D model and a reference 3D model; c) determining a set of parameters T that minimizes the cost function; and d) calculating the UV texture mapping for the reference model given the UV mapping for the static model.

Description

SUMMARY OF THE INVENTION

[0011] The present invention solves the problem of creating an animated body model (rig) of a real person by using its static 3D model. We scan a person in A-pose or T-pose to create a static 3D model. Then we define a parameterized rig, where rig parameters change body shape. We consider a rig in the corresponding pose (A-pose or T-pose) and find the parameters that result in the best likeness of a rig model and the person 3D model. Although we will be discussing the full body rig, the same method can be applied to creating a rig of other objects, including human face.

[0012] We start with a static 3D model. A 3D model is described by 3D points, mesh that is defined as polygons with vertices coinciding with 3D points, and a texture with UV mapping [2]. However, instead of animating this model like methods [3, 4], we use a reference 3D model. A reference 3D model is a rigged model of a human body that has parameters defining its shape. These parameters impact the human body metrics such as height, waistline, hipline, arm length, knee circumference and other parameters. The approach is suitable for different parametric body models, including, but not limited to [7, 8]. We use such a parameterized model to create a personalized rig from a 3D scan. Here is the description of this method: [0013] 1. Put the reference model in A-shape or T-shape, corresponding to the shape the static 3D model was scanned in. [0014] 2. Define a cost function for the likeness between the two models. Here is one example of such a cost function. Let p.sub.i be 3 dimensional points that correspond to vertices of the reference model mesh, that is parameterized by a vector T, and q.sub.iof the static 3D model. Let {tilde over (q)}.sub.i=N(p.sub.i) be the function that for each point from the reference model returns the closest mesh point of the static input 3D model. Then the cost function is defined as: C(T).sub.i(q.sub.iN(p.sub.i)).sup.2. [0015] 3. Find the set of parameters T that minimizes the cost function. Any optimization algorithm can be used, we use Levenberg-Marquardt method [5, 6]. [0016] 4. Once the optimal parameters are set, we use the mapping N(p.sub.i) to calculate the UV texture mapping for the reference model given the UV mapping for the static model. If raw RGB images are available we utilize them for texture mapping of the reference model by using the reference model as a mesh in a 3D reconstruction pipeline. As a result, we have a rigged model with shape similar to the scanned static 3D model and texture mapped from the static 3D model or raw RGB data. Alternatively, only a head can be textured this way but a predefined texture is used for the rest of the body. This is achieved by modeling texture for a reference model and by keeping UV mapping constant when changing body model parameters. This creates plausible scaling of texture for different body shapes and produces excellent texture for the body and the realistic texture for the face.

REFERENCES

[0017] [1] https://en.wikipedia.org/wiki/Skeletal_animation [0018] [2] https://en.wikipedia.org/wiki/UV_mapping [0019] [3] Baran, I., & Popovi, J. (2007, August). Automatic rigging and animation of 3d characters. In ACM Transactions on Graphics (TOG) (Vol. 26, No. 3, p. 72). ACM. [0020] [4] Lopez, R., & Poirel, C. (2013, July). Raycast based auto-rigging method for humanoid meshes. In ACM SIGGRAPH 2013 Posters (p. 11). ACM. [0021] [5] Kenneth Levenberg (1944). A Method for the Solution of Certain Non-Linear Problems in Least Squares. Quarterly of Applied Mathematics 2: 164-168. [0022] [6] Marquardt, D. W. (1963). An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial & Applied Mathematics, 11(2), 431-441. [0023] [7] Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., & Davis, J. (2005, July). SCAPE: shape completion and animation of people. In ACM Transactions on Graphics (TOG) (Vol. 24, No. 3, pp. 408-416). ACM. [0024] [8] Zuffi, S., & Black, M. J. (2015). The Stitched Puppet: A Graphical Model of 3D Human Shape and Pose. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3537-3546).