Techniques are described herein that overcome the limitations of conventional techniques by bridging a gap between user interaction with digital content using a computing device and a user's physical environment through use of augmented reality content. In one example, user interaction with augmented reality digital content as part of a live stream of digital images of a user's environment is used to specify a size of an area that is used to filter search results to find a “best fit”. In another example, a geometric shape is used to represent a size and shape of an object included in a digital image (e.g., a two-dimensional digital image). The geometric shape is displayed as augmented reality digital content as part of a live stream of digital images to “assess fit” of the object in the user's physical environment.

Virtual entities are displayed alongside real world entities in a wearable reality overlay device worn by the user. Information related to an environment proximate to the wearable device is determined. For example, a position of the wearable device may be determined, a camera may capture an image of the environment, etc. Virtual entity image information representative of an entity desired to be virtually displayed is processed based on the determined information. An image of the entity is generated based on the processed image information as a non-transparent region of a lens of the wearable device, enabling the entity to appear to be present in the environment to the user. The image of the entity may conceal a real world entity that would otherwise be visible to the user through the wearable device. Other real world entities may be visible to the user through the wearable device.

An object tracking, in particular adapted for real-time augmented reality applications, involves determining a location of an object (20) in a current frame (10) of a video stream (15), at a point in time following output of a preceding frame (11) of the video stream (15) but preceding output of the current frame (10), by starting from a location of the object (20) determined by an object-detection server (5) for a previous frame (12) of the video stream (15) and recursively track the location of the object (20) in frames (11) of the video stream (15) following the previous frame (12) up to the current frame (10) and recursively update a model of the object (20). Accurate objection detection from an object-detection server (5) can thereby be used even if the object was detected in a past frame (12) of the video stream (15) that has already been visualized.

Embodiments of the present invention are directed towards intuitive editing of three-dimensional models. In embodiments, salient geometric features associated with a three-dimensional model defining an object are identified. Thereafter, feature attributes associated with the salient geometric features are identified. A feature set including a plurality of salient geometric features related to one another is generated based on the determined feature attributes (e.g., properties, relationships, distances). An editing handle can then be generated and displayed for the feature set enabling each of the salient geometric features within the feature set to be edited in accordance with a manipulation of the editing handle. The editing handle can be displayed in association with one of the salient geometric features of the feature set.

A method and apparatus for providing a guide for combining pattern pieces receives a selection of a first point in a first pattern piece and a selection of a second point in a second pattern piece to be combined with the first pattern piece, generates a virtual pattern piece in response to the selection of the second point being received, arranges the virtual pattern piece such that a third point in the virtual pattern piece having a position corresponding to the first point in the first pattern piece is matched to the second point in the second pattern piece, and provides a guide for combining the first pattern piece and the second pattern piece by moving the virtual pattern piece such that an outer line of the second pattern piece and an outer line of the virtual pattern piece correspond to each other.

In various embodiments, a training application trains a machine learning model to modify portions of shapes when designing 3D objects. The training application converts first structural analysis data having a first resolution to first coarse structural analysis data having a second resolution that is lower than the first resolution. Subsequently, the training application generates one or more training sets based on a first shape, the first coarse structural analysis data, and a second shape that is derived from the first shape. Each training set is associated with a different portion of the first shape. The training application then performs one or more machine learning operations on the machine learning model using the training set(s) to generate a trained machine learning model. The trained machine learning model modifies at least a portion of a shape having the first resolution based on coarse structural analysis data having the second resolution.

A computing device captures a live video of a user, determines a location of a facial region of the user by a facial region analyzer, and determines a finger vector type by a finger vector detector based on a direction in which at least one finger is pointing relative to the facial region of the user. Responsive to detecting a first finger vector type within the facial region involving a single finger, a makeup effects toolbar is displayed in the user interface. Responsive to detecting a second finger vector type involving the single finger, a selection tool for selecting a makeup effect in the makeup effects toolbar is displayed. The computing device obtains a makeup effect based on manipulation by the user of the selection tool. Responsive to detecting a target user action, virtual application of the selected makeup effect is performed on the facial region of the user.

A computer system obtains at least one 3D scan of a body surface; automatically identifies, based on the at least one 3D scan, one or more features (e.g., nose, lips, eyes, eyebrows, cheekbones, or specific portions thereof, or other features) of the body surface; and generates a digital 3D model of the body surface. The digital 3D model includes the identified features of the human body surface. In an embodiment, the step of generating of the digital 3D model is based on the at least one 3D scan and the identified features of the body surface. In an embodiment, the digital 3D model comprises a 3D mesh file. The digital 3D model can be used in various ways. For example, output of a manufacturing process (e.g., a 3D printed item, a cosmetics product, a personal care product) can be based on the digital 3D model.

The present technology generates a simplified version of a complex avatar by capturing images and 3-D volume information of segments of the complex avatar while the complex avatar is rendered. When the complex avatar is requested in an environment in which it is not desirable to display the complex avatar, the captured images and 3-D volume information can be used to provide a simplified version of the avatar. The simplified version of the avatar can have a similar visual appearance but can be easier to render. However, the present technology permits the user with the complex avatar to continue to have approximately the same visual appearance while avoiding the degraded performance on systems not capable of rendering the complex avatar quickly enough.

A method and system for improving damage assessment by automatic measurement of virtual objects within an augmented reality (AR) representation of a physical environment are disclosed. A user interacts with a mobile computing device to position and resize virtual objects within a virtual space corresponding to a physical environment, which is presented to the user as an AR environment generated by the mobile computing device. The virtual objects are positioned and sized to match damaged physical objects within the physical environment. The sizes of the physical objects are automatically determined using the virtual sizes of the virtual objects, which physical sizes are further used to determine the extent of damage.