The present disclosure provides systems and methods that perform face reconstruction based on an image of a face. In particular, one example system of the present disclosure combines a machine-learned image recognition model with a face modeler that uses a morphable model of a human's facial appearance. The image recognition model can be a deep learning model that generates an embedding in response to receipt of an image (e.g., an uncontrolled image of a face). The example system can further include a small, lightweight, translation model structurally positioned between the image recognition model and the face modeler. The translation model can be a machine-learned model that is trained to receive the embedding generated by the image recognition model and, in response, output a plurality of facial modeling parameter values usable by the face modeler to generate a model of the face.

Avatar animation systems disclosed herein provide high quality, real-time avatar animation that is based on the varying countenance of a human face. In some example embodiments, the real-time provision of high quality avatar animation is enabled at least in part, by a multi-frame regressor that is configured to map information descriptive of facial expressions depicted in two or more images to information descriptive of a single avatar blend shape. The two or more images may be temporally sequential images. This multi-frame regressor implements a machine learning component that generates the high quality avatar animation from information descriptive of a subject's face and/or information descriptive of avatar animation frames previously generated by the multi-frame regressor. The machine learning component may be trained using a set of training images that depict human facial expressions and avatar animation authored by professional animators to reflect facial expressions depicted in the set of training images.

Systems, methods, and non-transitory computer readable media can align face images, classify face images, and verify face images by employing a deep neural network (DNN). A 3D-aligned face image can be generated from a 2D face image. An identity of the 2D face image can be classified based on provision of the 3D-aligned face image to the DNN. The identity of the 2D face image can comprise a feature vector.

A computer implemented method of generating a new three-dimensional morphable model (3DMM) by combining a first 3DMM with a second 3DMM includes generating, using the first 3DMM, a plurality of first shapes, calculating a mapping from a plurality of second parameters of the second 3DMM to a plurality of first parameters of the first 3DMM, generating, for each of a plurality of second shapes generated using the second 3DMM, a corresponding first shape, forming a plurality of merged shapes by merging each second shape with the corresponding first shape, and performing principal component analysis on the plurality of merged shapes to generate the new 3DMM.

At least two images are acquired. At least two crop images are acquired by cropping the at least two images for face-containing images. Triangular patch deformation is performed on two neighbour images, generating a first triangular patch deformation frame image sequence and a second triangular patch deformation frame image sequence. Similarity transformation is performed on each image sequence of the first triangular patch deformation frame image sequence, acquiring a first transform frame image sequence. Similarity transformation is performed on each image sequence of the second triangular patch deformation frame image sequence, acquiring a second transform frame image sequence. The first and the second transform frame image sequences are fused, acquiring a video frame sequence corresponding to the two neighbour images. Of the at least two images, a video frame sequence generated by all neighbour images is coded, acquiring a destined video.

The present disclosure provides systems and methods that perform face reconstruction based on an image of a face. In particular, one example system of the present disclosure combines a machine-learned image recognition model with a face modeler that uses a morphable model of a human's facial appearance. The image recognition model can be a deep learning model that generates an embedding in response to receipt of an image (e.g., an uncontrolled image of a face). The example system can further include a small, lightweight, translation model structurally positioned between the image recognition model and the face modeler. The translation model can be a machine-learned model that is trained to receive the embedding generated by the image recognition model and, in response, output a plurality of facial modeling parameter values usable by the face modeler to generate a model of the face.

Provided are a method for generating a joint-based facial rig and a 3D graphics interface apparatus therefor according to exemplary embodiments of the present disclosure. A method for generating a joint-based facial rig performed by a control unit includes: generating a facial rig model by morphing at least one morph target in order to represent a facial expression; generating at least one joint corresponding to each of a plurality of facial areas of the generated facial rig model; connecting the at least one generated joint and each of the plurality of facial areas; and moving each of at least one morph target corresponding to the facial rig model, and recording a movement change value of a joint moving jointly according to each moving morph target.

A method of generating a 3D model may include receiving a plurality of 2D images of a physical object captured from a respective plurality of viewpoints in a 3D scan of the physical object in a first process. The method may include receiving a first process 3D mesh representation of the physical object and calculating respective second process estimated position and/or orientation information for each one of the respective plurality of viewpoints of the plurality of 2D images. The method may include generating a second process 3D mesh representation of the physical object using the plurality of 2D images, the second process estimated position and/or orientation information, and the first process 3D mesh representation of the physical object. The method may include generating a 3D model of the physical object by applying surface texture information from the plurality of 2D images to the second process 3D mesh representation of the physical object.

An information processing apparatus according to an embodiment of the present technology includes a display control unit. The display control unit controls a display apparatus to display a target object that is a correction target and controls the display apparatus to change, in accordance with a display state of the target object after the target object is displayed, the target object into an intermediate object between the target object and a reference object corresponding to the target object.

A computer implemented method of generating a new three-dimensional morphable model (3DMM) by combining a first 3DMM with a second 3DMM includes generating, using the first 3DMM, a plurality of first shapes, calculating a mapping from a plurality of second parameters of the second 3DMM to a plurality of first parameters of the first 3DMM, generating, for each of a plurality of second shapes generated using the second 3DMM, a corresponding first shape, forming a plurality of merged shapes by merging each second shape with the corresponding first shape, and performing principal component analysis on the plurality of merged shapes to generate the new 3DMM.