A computer-implemented method for a three-dimensional (3D) reconstruction of a dynamic scene includes receiving a plurality of color image sequences from a plurality of color imaging sensors, and at least one depth image sequence from at least one depth imaging sensor, where a color imaging sensor quantity is larger than a depth imaging sensor quantity. A plurality of calibrated color image sequences and at least one calibrated depth image sequence are generated based on the plurality of color imaging sequences and the at least one depth image sequence. A plurality of initial 3D patches is constructed using the plurality of calibrated color image sequences and the at least one calibrated depth image sequence. A 3D patch cloud is generated by expanding the plurality of initial 3D patches.

Techniques are provided for recognition of activity in a sequence of video image frames that include depth information. A methodology embodying the techniques includes segmenting each of the received image frames into a multiple windows and generating spatio-temporal image cells from groupings of windows from a selected sub-sequence of the frames. The method also includes calculating a four dimensional (4D) optical flow vector for each of the pixels of each of the image cells and calculating a three dimensional (3D) angular representation from each of the optical flow vectors. The method further includes generating a classification feature for each of the image cells based on a histogram of the 3D angular representations of the pixels in that image cell. The classification features are then provided to a recognition classifier configured to recognize the type of activity depicted in the video sequence, based on the generated classification features.

A target view to a 3D scene depicted by a multiview image is determined. The multiview image comprising sampled views at sampled view positions distributed throughout a viewing volume. Each sampled view in the sampled views comprises a wide-field-of-view (WFOV) image and a WFOV depth map as seen from a respective sampled view position in the sampled view positions. The target view is used to select, from the sampled views, a set of sampled views. A display image is caused to be rendered on a display of a wearable device. The display image is generated based on a WFOV image and a WFOV depth map for each sampled view in the set of sampled views.

A method for determining a mark in a data record with three-dimensional surface coordinates of a scene includes ascertaining a first collection of edge points in a three-dimensional coordinate system of the data record, fitting an equalization area into at least a subset of the edge points of the first collection of edge points to permit the edge points in the three-dimensional coordinate system to be partly positioned on a first side of the equalization area and partly positioned on a second side, lying opposite the first side, of the equalization area, displacing edge points of the first collection of edge points into the equalization area to permit a corrected collection of edge points to be formed, and determining the mark in the three-dimensional coordinate system based on the corrected collection of edge points or the corrected closed circumferential edge line.

A virtual reality (VR) device stores an encoded 360 VR video that includes a sequence of video fragments. Each video fragment includes a plurality of flat layers and each flat layer is at least one equirectangular image frame associated with an image metadata. The VR device is configured to render the plurality of flat layers in each video fragment as a plurality of concentric spherical layers projected at a plurality of depth values. The VR device is further configured to receive a plurality of user inputs associated with a modification of a set of attributes in the image metadata. The VR device is further configured to generate a modified image metadata for different concentric spherical layers and control playback of each video fragment in accordance with the modified image metadata for the different concentric spherical layer.

Systems and methods for storing images synthesized from light field image data and metadata describing the images in electronic files in accordance with embodiments of the invention are disclosed. One embodiment includes a processor and memory containing an encoding application and light field image data, where the light field image data comprises a plurality of low resolution images of a scene captured from different viewpoints. In addition, the encoding application configures the processor to synthesize a higher resolution image of the scene from a reference viewpoint using the low resolution images, where synthesizing the higher resolution image involves creating a depth map that specifies depths from the reference viewpoint for pixels in the higher resolution image; encode the higher resolution image; and create a light field image file including the encoded image, the low resolution images, and metadata including the depth map.

An information processing device has at least one memory that stores instructions, and at least one processor coupled to the at least one memory, and configured to set, on a combined image in which a plurality of images captured through a plurality of imaging units are combined, an object region that is a region to be an object of a predetermined process, and to perform, when a boundary between images included in the plurality of images in the combined image is included in the set object region, a predetermined display process on a display unit.

This invention provides a system and method for selecting the correct profile from a range of peaks generated by analyzing a surface with multiple exposure levels applied at discrete intervals. The cloud of peak information is resolved by comparison to a model profile into a best candidate to represent an accurate representation of the object profile. Illustratively, a displacement sensor projects a line of illumination on the surface and receives reflected light at a sensor assembly at a set exposure level. A processor varies the exposure level setting in a plurality of discrete increments, and stores an image of the reflected light for each of the increments. A determination process combines the stored images and aligns the combined images with respect to a model image. Points from the combined images are selected based upon closeness to the model image to provide a candidate profile of the surface.

A method of sorting is described, and which includes providing a product stream formed of individual objects of interest having feature aspects which can be detected; generating multiple images of each of the respective objects of interest; classifying the feature aspects of the objects of interest; identifying complimentary images by analyzing some of the multiplicity of images; fusing the complimentary images to form an aggregated region representation of the complimentary images; and sorting the respective objects of interest based at least in part upon the aggregated region representation which is formed.

Disclosed herein are methods, systems, and computer-readable storage media for generating three-dimensional (3D) images of a scene. According to an aspect, a method includes capturing a real-time image and a first still image of a scene. Further, the method includes displaying the real-time image of the scene on a display. The method also includes determining one or more properties of the captured images. The method also includes calculating an offset in a real-time display of the scene to indicate a target camera positional offset with respect to the first still image. Further, the method includes determining that a capture device is in a position of the target camera positional offset. The method also includes capturing a second still image. Further, the method includes correcting the captured first and second still images. The method also includes generating the three-dimensional image based on the corrected first and second still images.