A system for capturing live-action three-dimensional video is disclosed. The system includes pairs of stereo cameras and a LIDAR for generating stereo images and three-dimensional LIDAR data from which three-dimensional data may be derived. A depth-from-stereo algorithm may be used to generate the three-dimensional camera data for the three-dimensional space from the stereo images and may be combined with the three-dimensional LIDAR data taking precedence over the three-dimensional camera data to thereby generate three-dimensional data corresponding to the three-dimensional space.

A system for capturing live-action three-dimensional video is disclosed. The system includes pairs of stereo cameras and a LIDAR for generating stereo images and three-dimensional LIDAR data from which three-dimensional data may be derived. A depth-from-stereo algorithm may be used to generate the three-dimensional camera data for the three-dimensional space from the stereo images and may be combined with the three-dimensional LIDAR data taking precedence over the three-dimensional camera data to thereby generate three-dimensional data corresponding to the three-dimensional space.

Systems and methods relate to segmenting texture data and depth data of first image data of a first viewpoint into layers; generating respective multiple focal planes (MFPs) for each respective layer; blanking out pixels on the respective MFPs for each respective layer that are occluded by pixels on layers that are closer to a first viewpoint; and generating second image data for a second viewpoint to enable display of the second image data by: shifting and scaling the respective MFPs for each respective layer corresponding to the second viewpoint, wherein layers closer to an origin of the second viewpoint are shifted and scaled more than layers farther from the origin of the second viewpoint; and blanking out pixels on the shifted and scaled respective MFPs for each respective layer that are occluded by pixels on layers that are closer to the second viewpoint.

In one embodiment, a method includes determining a gaze direction of a user wearing a head-mounted device, the head-mounted device having a display configured to output an image having multiple of pixels. The method may further include determining, for each of the multiple pixels, a set of sampling locations based on the gaze direction of the user, the sets of sampling locations of the multiple pixels being a portion of a sampling pattern defined based on the gaze direction of the user. In one embodiment, the method may also include computing, for each of the multiple pixels, a color value for the pixel by sampling a scene according to the set of sampling locations associated with the pixel, generating the image using the color values of the multiple pixels, and outputting the image using the display of the head-mounted device.

The present disclosure relates to methods and devices for performing depth estimation on image data. In one example, a device performs depth estimation on first and second images captured using one or more cameras having a color filter array. Each image of the first and second images comprises multiple color channels. Each color channel of the multiple color channels corresponds to a respective color channel of the color filter array. The device performs the depth estimation by estimating disparity from the color channels of the first and second images.

Methods, systems and apparatus for compensating vision loss for a patient. In some embodiments, a computer processor receives vision loss data associated with a vision loss region of an eye of a patient from a head mounted display (HMD) device worn by the patient, generates a parameterized perceptual loss model, and then generates inverse data to correct for color loss, contrast and luminance desaturation, and visual rotational and spatial distortion suffered by the eye of the patient. The computer processor then transmits the inverse data to the HMD device being worn by the patient for use in correcting the visual rotational and spatial distortion loss of the eye of the patient.

A method for processing multi-view data of a scene. The method comprises obtaining at least two images of the scene from different cameras, determining a sharpness indication for each image and determining a confidence score for each image based on the sharpness indications. The confidence score is for use in the determination of weights when blending the images to synthesize a new virtual image.

A computer-implemented method for determining whether a video frame is of a 3D TB type (Top-Bottom) or a 3D LR type (Left-Right) frame, characterized in that it comprises the steps of: receiving a video frame (100); extracting at least three portions (121-124) of the frame, each portion belonging to a distinct quarter of the frame (100, 120) and being positioned at the same fragment of the quarter; calculating color histograms for each portion (121-124); comparing the color histograms of at least two different pairs of portions; generating a frame type indicator based on the result of comparison of the color histograms.

An electrically controlled spectacle includes a spectacle frame and optoelectronic lenses housed in the frame. The lenses include a left lens and a right lens, each of the optoelectrical lenses having a plurality of states, wherein the state of the left lens is independent of the state of the right lens. The electrically controlled spectacle also includes a control unit housed in the frame, the control unit being adapted to control the state of each of the lenses independently.

A dimensioning system that analyzes a distance map for null-data pixels to provide feedback is disclosed. Null-data pixels correspond to missing range data and having too many in a distance map may lead to dimensioning errors. Providing feedback based on the number of null-data pixels helps a user understand and adapt to different dimensioning conditions, promotes accuracy, and facilitates handheld applications.