The present disclosure describes a system for fast generation of ray traced reflections of virtually augmented objects into a real-world image, specifically on reflective surfaces. The system utilizes a standard raster graphics pipeline.

This disclosure describes methods, non-transitory computer readable storage media, and systems that generate realistic shading for three-dimensional objects inserted into digital images. The disclosed system utilizes a light encoder neural network to generate a representation embedding of lighting in a digital image. Additionally, the disclosed system determines points of the three-dimensional object visible within a camera view. The disclosed system generates a self-occlusion map for the digital three-dimensional object by determining whether fixed sets of rays uniformly sampled from the points intersects with the digital three-dimensional object. The disclosed system utilizes a generator neural network to determine a shading map for the digital three-dimensional object based on the representation embedding of lighting in the digital image and the self-occlusion map. Additionally, the disclosed system generates a modified digital image with the three-dimensional object inserted into the digital image with consistent lighting of the three-dimensional object and the digital image.

In one embodiment, a method includes, by a computing system, accessing a first and second texture associated with an output position, determining a color-blending operation, determining a first color and a first transparency level based on the first texture, determining a second color and a second transparency level based on the second texture, and identifying a color-blending optimization based on the color-blending operation and a comparison of the colors and transparency levels. The method includes determining an output color and an output transparency level by performing the color-blending operation using the colors and transparency levels. The output color is determined by copying the first or second color or the output transparency level is determined by copying the first or second transparency level without additional calculation. The method includes providing the output color and the output transparency level for display at the output position.

An apparatus for rendering a visual scene includes: a content visualization stage configured: to obtain as a first input a set of images of one or more objects, and to obtain as a second input a geometry representation of the one or more objects in a 3D-space; to obtain a final image representing the visual scene from a perspective of a target position, the visual scene including the one or more objects; to consider at least one of a lighting effect and/or an object interaction effect between the one or more objects and one or more further objects contained in the visual scene; the content visualization stage is configured to obtain a target view image from the set of images irrespective of the geometry representation. The apparatus is configured to map the target view image on the geometry representation under consideration of the target position.

An apparatus for rendering a visual scene includes: a content visualization stage configured: to obtain as a first input a set of images of one or more objects, and to obtain as a second input a geometry representation of the one or more objects in a 3D-space; to obtain a final image representing the visual scene from a perspective of a target position, the visual scene including the one or more objects; to consider at least one of a lighting effect and/or an object interaction effect between the one or more objects and one or more further objects contained in the visual scene; the content visualization stage is configured to obtain a target view image from the set of images irrespective of the geometry representation. The apparatus is configured to map the target view image on the geometry representation under consideration of the target position.

Paired images of substantially the same scene are captured with the same freestanding sensor. The paired images include reflected light illuminated with controlled polarization states that are different between the paired images. Information from the images is applied to a convolutional neural network (CNN) configured to derive a spatially varying bi-directional reflectance distribution function (SVBRDF) for objects in the paired images. Alternatively, the sensor is fixed and oriented to capture images of an object of interest in the scene while a light source traverses a path that intersects the sensor's field of view. Information from the paired images of the scene and from the images captured of the object of interest when the light source traverses the field of view are applied to a CNN to derive a SVBDRF for the object of interest. The image information and the SVBRDF are used to render a representation with artificial lighting conditions.

Systems, methods, and computer-readable media are described herein to model divergent beam ray paths between locations on a roof (e.g., of a structure) and modeled locations of the sun at different times of the day and different days during a week, month, year, or another time period. A graphical user interface allows for visualization of the modeled ray paths and graphical manipulation of the resolution and parameters of the modeling process.

Systems, methods, and computer-readable media are described herein to model divergent beam ray paths between locations on a roof (e.g., of a structure) and modeled locations of the sun at different times of the day and different days during a week, month, year, or another time period. A graphical user interface allows for visualization of the modeled ray paths and graphical manipulation of the resolution and parameters of the modeling process.

A computer implemented method of training an operator of a video management system, the method comprising inputting, into the video management system, a plurality of virtual video streams generated by virtual video cameras within a dynamic 3D virtual environment; displaying on a display, from the virtual management system, at least some of the said virtual video streams, wherein at least one output video stream shows an event of interest; receiving, in the computer, at least one command from the operator indicative of a detection of the event of interest in the dynamic 3D virtual environment by the operator.

A computer implemented method of training an operator of a video management system, the method comprising inputting, into the video management system, a plurality of virtual video streams generated by virtual video cameras within a dynamic 3D virtual environment; displaying on a display, from the virtual management system, at least some of the said virtual video streams, wherein at least one output video stream shows an event of interest; receiving, in the computer, at least one command from the operator indicative of a detection of the event of interest in the dynamic 3D virtual environment by the operator.