Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

The present invention provides an image processing method, including: obtaining a first image using a camera under a display screen; processing the first image using a processor; obtaining a second image using the camera under the display screen; processing the second image using the processor; and generating a superimposed image after superimposing the first sub-image and the second sub-image.

There is provided an image pickup device and an image pickup method for estimating the depth of an image having a repetitive pattern with high accuracy. The peripheral cameras are arranged according to base line lengths based on reciprocals of different prime numbers as having a position of a reference camera, to be a reference when images from different viewpoints are imaged, as a reference. The present disclosure is capable of being applied to a light field camera and the like, for example, which includes the reference camera and the plurality of peripheral cameras, generates a parallax image from the images of plural viewpoints, and generates a refocus image by using the images from the plural viewpoints and the parallax image.

Provided is an electronic device, wherein the at least one processor is configured to execute the at least one instruction to obtain a first light field image including view images with a first number of views, obtain, from the first light field image, a second light field image including view images with a second number of views, obtain first location information corresponding to each of sub-pixels in the second light field image, obtain a first layer image by inputting the second light field image and the first location information to an artificial intelligence model configured to perform factorization, obtain a third light field image including view images with a third number of views by inputting the first layer image to a simulation model, and train the artificial intelligence model, based on a result of comparing the first light field image and the third light field image with each other.

A depth-based effect may be applied to a multi-view video stream to generate a modified multi-view video stream. User input may designate a boundary between a foreground region and a background region, at a different depth from the foreground region, of a reference image of the video stream. Based on the user input, a reference mask may be generated to indicate the foreground region and the background region. The reference mask may be used to generate one or more other masks that indicate the foreground and background regions for one or more different images, from different frames and/or different views from the reference image. The reference mask and other mask(s) may be used to apply the effect to the multi-view video stream to generate the modified multi-view video stream.

Certain aspects relate to systems and techniques for efficiently recording captured plenoptic image data and for rendering images from the captured plenoptic data. The plenoptic image data can be captured by a plenoptic or other light field camera. In some implementations, four dimensional radiance data can be transformed into three dimensional data by performing a Radon transform to define the image by planes instead of rays. A resulting Radon image can represent the summed values of energy over each plane. The original three-dimensional luminous density of the scene can be recovered, for example, by performing an inverse Radon transform. Images from different views and/or having different focus can be rendered from the luminous density.

Methods of determining the depth of a scene and associated systems are disclosed herein. In some embodiments, a method can include augmenting depth data of a scene captured with a depth sensor with depth data from one or more images of the scene. For example, the method can include capturing image data of the scene with a plurality of cameras. The method can further include generating a point cloud representative of the scene based on the depth data from the depth sensor and identifying a missing region of the point cloud, such as a region occluded from the view of the depth sensor. The method can then include generating depth data for the missing region based on the image data. Finally, the depth data for the missing region can be merged with the depth data from the depth sensor to generate a merged point cloud representative of the scene.

An HMD device identifies a pose of the device and identifies a subset of a plurality of camera viewpoints of a light-field based on the pose. The HMD device interpolates image data of the light-field based on the pose and the subset of the plurality of camera viewpoints to generate an interpolated view; and displays at the HMD device an image based on the interpolated view. By interpolating based on the subset of camera viewpoints, the HMD device can reduce processing overhead and improve the user experience.

An image display apparatus includes an acquisition unit configured to acquire an image file that includes reconstructable image data that includes information corresponding to a light field, a display unit configured to display the image data included in the image file acquired by the acquisition unit as an image, and a control unit configured to control the display unit so as to display an image and perform display for informing a user that the image file corresponding to the image is a reconstructable image file.

A method for generating stereoscopic light-field data is provided. The method includes the following steps: obtaining three-dimensional image data; performing a multi-view generation process to convert the three-dimensional image data into multi-view data; and performing a light-field conversion process to convert the multi-view data into a stereoscopic light-field pair.