A method for training a depth estimation model with depth hints is disclosed. For each image pair: for a first image, a depth prediction is determined by the depth estimation model and a depth hint is obtained; the second image is projected onto the first image once to generate a synthetic frame based on the depth prediction and again to generate a hinted synthetic frame based on the depth hint; a primary loss is calculated with the synthetic frame; a hinted loss is calculated with the hinted synthetic frame; and an overall loss is calculated for the image pair based on a per-pixel determination of whether the primary loss or the hinted loss is smaller, wherein if the hinted loss is smaller than the primary loss, then the overall loss includes the primary loss and a supervised depth loss between depth prediction and depth hint. The depth estimation model is trained by minimizing the overall losses for the image pairs.

A method for training a depth estimation model with depth hints is disclosed. For each image pair: for a first image, a depth prediction is determined by the depth estimation model and a depth hint is obtained; the second image is projected onto the first image once to generate a synthetic frame based on the depth prediction and again to generate a hinted synthetic frame based on the depth hint; a primary loss is calculated with the synthetic frame; a hinted loss is calculated with the hinted synthetic frame; and an overall loss is calculated for the image pair based on a per-pixel determination of whether the primary loss or the hinted loss is smaller, wherein if the hinted loss is smaller than the primary loss, then the overall loss includes the primary loss and a supervised depth loss between depth prediction and depth hint. The depth estimation model is trained by minimizing the overall losses for the image pairs.

Techniques for rendering a 3D virtual object in an augmented-reality system are described. A system, a method, and a non-transitory memory device for augmented reality rendering of three-dimensional, virtual objects are described. In an example, a number of images of an environment are acquired; relative movement of a camera acquiring the number of images is tracked; camera pose is determined relative to the environment using the number of images and tracked relative movement of the camera; depth and normal surfaces of objects in the environment are estimated using a depth map and a normal map; a surface geometry of the environment is reconstructed using the depth map and the normal map; and the virtual object is rendered using the surface geometry of the environment.

Techniques for rendering a 3D virtual object in an augmented-reality system are described. A system, a method, and a non-transitory memory device for augmented reality rendering of three-dimensional, virtual objects are described. In an example, a number of images of an environment are acquired; relative movement of a camera acquiring the number of images is tracked; camera pose is determined relative to the environment using the number of images and tracked relative movement of the camera; depth and normal surfaces of objects in the environment are estimated using a depth map and a normal map; a surface geometry of the environment is reconstructed using the depth map and the normal map; and the virtual object is rendered using the surface geometry of the environment.

Methods and systems are provided for a reprojection engine for augmented-reality devices. The augmented-reality device projects virtual content within a real-world environment. The augmented-reality device tracks a six degrees of freedom headpose of the augmented-reality device, depth information of the virtual content, motion vectors that correspond to movement of the virtual content, and a color buffer for a reprojection engine. The reprojection engine generates a reprojection of the virtual content defined by an extrapolation of a first frame using the headpose, the depth information, motion vectors, and the color surface data structure. The reprojected virtual content continues to appear as if positioned with the real-world environment regardless of changes in the headpose of the augmented-reality device or motion of the virtual content.

One or more stereoscopic images are generated based on a single monoscopic image that may be obtained from a camera sensor. Each stereoscopic image includes a first digital image and a second digital image that, when viewed using any suitable stereoscopic viewing technique, result in a user or software program receiving a three-dimensional effect with respect to the elements included in the stereoscopic images. The monoscopic image may depict a geographic setting of a particular geographic location and the resulting stereoscopic image may provide a three-dimensional (3D) rendering of the geographic setting. Use of the stereoscopic to image helps a system obtain more accurate detection and ranging capabilities. The stereoscopic image may be any configuration of the first digital image (monoscopic) and the second digital image (monoscopic) that together may generate a 3D effect as perceived by a viewer or software program.

A method, system and computer readable storage media for visualizing to a patient a magnified dental treatment site. By obtaining raw data from a stereo camera recording a dental treatment site, an enlarged, well-lit and spatially displayed view of the dental treatment site may be visualized in real time in augmented reality and virtual reality systems for diagnoses and treatment planning.

A method, system and computer readable storage media for visualizing to a patient a magnified dental treatment site. By obtaining raw data from a stereo camera recording a dental treatment site, an enlarged, well-lit and spatially displayed view of the dental treatment site may be visualized in real time in augmented reality and virtual reality systems for diagnoses and treatment planning.

The present invention provides a generation method for a 3D asteroid dynamic map and a portable terminal. The method comprises: obtaining a panorama image; identifying the panorama image and segmenting into a sky region, a human body region, and a ground region; calculating a panoramic depth map for the sky region, the human body region, and the ground region; respectively transforming the panorama image and the panoramic depth map to generate an asteroid image and an asteroid depth map; generating an asteroid view under a virtual viewpoint; and rendering to generate a 3D asteroid dynamic map. By automatically generating the asteroid view under the virtual viewpoint, and synthesizing and rendering same, the technical solution of the present invention generates the asteroid dynamic map having a 3D effect from the panorama image.

Methods, systems, and techniques for enhancing use of two-dimensional (2D) video analytics by using depth data. Two-dimensional image data representing an image comprising a first object is obtained, as well as depth data of a portion of the image that includes the first object. The depth data indicates a depth of the first object. An initial 2D classification of the portion of the image is generated using the 2D image data without using the depth data. The initial 2D classification is stored as an approved 2D classification when the initial 2D classification is determined consistent with the depth data. Additionally or alternatively, a confidence level of the initial 2D classification may be adjusted depending on whether the initial 2D classification is determined to be consistent with the depth data, or the depth data may be used with the 2D image data for classification.