Techniques for simplifying a user interface by transforming a model and inversely transforming commands for controlling a remote device based on the transformed model are described herein. A computer system determines a warping transformation, applies the transformation to a model, and provides the transformed model to an operator display. The operator, referencing the transformed model, provides a movement command to a remote device reduces robot and the computer system inversely transforms the command to correspond to the space of the remote device.

Disclosed herein includes a system, a method, and a non-transitory computer readable medium for rendering a three-dimensional (3D) model of an avatar according to an audio stream including a vocal output of a person and image data capturing a face of the person. In one aspect, phonemes of the vocal output are predicted according to the audio stream, and the predicted phonemes of the vocal output are translated into visemes. In one aspect, a plurality of blendshapes and corresponding weights are determined, according to the corresponding image data of the face, to form the 3D model of the avatar of the person. The visemes may be combined with the 3D model of the avatar to form a 3D representation of the avatar, by synchronizing the visemes with the 3D model of the avatar in time.

A method for generating a real-time context-based emoticon may include receiving conversation information associated with a conversation between a set of users. The method may include identifying an attribute associated with the conversation. The method may include generating a base emoticon. The method may include generating an output shape based on a fiducial point of the base emoticon. The method may include transforming the output shape of the base emoticon with an accessory. The method may include generating the real-time context-based emoticon for the conversation, based on transforming the output shape of the base emoticon with the accessory.

A method of modifying a 2D image representing a 3D scene in order to preserve perceptual constancy of objects in the scene the method including the steps: processing an image of a 3D scene to generate an unmodified view of the 3D scene and one or more 3D objects within the scene; selecting one or more objects from within the scene; determining a modified view of the one or more objects; comparing the modified view of the one or more objects with the unmodified view of the one or more objects; interpolating one or more stages between the unmodified view and modified view of the one or more objects; selecting an interpolation for the one or more objects; generating a new 3D scene with the selected interpolated one or more objects; and, projecting and rendering the new 3D scene into a 2D image onto a display.

An example apparatus for correcting gaze in images includes an image receiver to receive an image comprising an eye and a target angle set to a center. The apparatus also includes a bidirectionally trained convolutional neural network (CNN) to receive the image and the target angle from the image receiver and generate a vector field and a brightness map based on the image and the target angle. The apparatus further includes an image corrector to generate a gaze corrected image based on the vector field and the brightness map.

A first three-dimensional (3D) model of a target in a 3D space is acquired based on a first two-dimensional (2D) image including the target. A 3D morphing parameter is acquired. The first 3D model is transformed into a second 3D model based on the 3D morphing parameter. First 2D coordinates are acquired by mapping the first 3D model to a 2D space. Second 2D coordinates are acquired by mapping the second 3D model to the 2D space. A second 2D image including a morphed target is acquired by morphing the target in the first 2D image based on the first 2D coordinates and the second 2D coordinates.

The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a virtual try-on digital image utilizing a unified neural network framework. For example, the disclosed systems can utilize a coarse-to-fine warping process to generate a warped version of a product digital image to fit a model digital image. In addition, the disclosed systems can utilize a texture transfer process to generate a corrected segmentation mask indicating portions of a model digital image to replace with a warped product digital image. The disclosed systems can further generate a virtual try-on digital image based on a warped product digital image, a model digital image, and a corrected segmentation mask. In some embodiments, the disclosed systems can train one or more neural networks to generate accurate outputs for various stages of generating a virtual try-on digital image.

A method for predicting a future dental situation for a patient. Acquiring historical data including at least: previous time point and context parameter values. Acquiring at a time point all the data related to the current dental situation, including at least: the current time point, context parameter values at the current time point, statistical analysis of the historical data and the current data, so as to predict, at a future time point, at least one future dental situation for the current patient. Depending on the future dental situation, (re)evaluation of the benefit of an orthodontic treatment. Steps may include creating a three-dimensional digital reference model, acquiring at least one two-dimensional image, analysing each updated image and creation, searching, for each updated image, and collecting data relative to the updated reference model and relative to the orthodontic appliance.

A computer implemented method for warping virtual content includes generating warped virtual content by transforming source virtual content. The method also includes determining whether a memory location corresponding to an X, Y location of the warped virtual content in an output frame of reference is occupied by pre-existing virtual content. The method further includes storing the warped virtual content in the memory location if the memory location is not occupied. Moreover, the method includes comparing respective Z locations of the warped virtual content and the pre-existing virtual content to identify virtual content with a Z location closer to a viewing location if the memory location is occupied. The method also includes storing the warped virtual content in the memory location corresponding to the X, Y location if a Z location of warped virtual content is closer to the viewing location than a pre-existing Z location of pre-existing virtual content.

Techniques related to capturing 3D faces using image and temporal tracking neural networks and modifying output video using the captured 3D faces are discussed. Such techniques include applying a first neural network to an input vector corresponding to a first video image having a representation of a human face to generate a morphable model parameter vector, applying a second neural network to an input vector corresponding to a first and second temporally subsequent to generate a morphable model parameter delta vector, generating a 3D face model of the human face using the morphable model parameter vector and the morphable model parameter delta vector, and generating output video using the 3D face model.