A multi-aperture imaging device that is, on the one hand, able to provide image information on a scene and, on the other hand, allows obtaining high lateral resolution and/or a wide total field of view, is described. The multi-aperture imaging device is provided with a first plurality of optical channels for projecting overlapping first partial fields of view of a total field of view on first image sensor areas of an image sensor of the multi-aperture imaging device, as well as with a second arrangement of optical channels for projecting at least a part of of the total field of view on a second image sensor area of the image sensor.

The present disclosure relates to a signal processing device and an imaging device capable of improving an image quality of the imaging device that does not use an imaging lens. A signal processing device includes a restoration unit that restores one restored image by using a plurality of detection signal sets obtained by an imaging element in a plurality of states in which at least one of a position or orientation with respect to a subject is different, the imaging element that includes a plurality of pixel output units that receives incident light from the subject incident without an intervention of an imaging lens or a pinhole and each outputs one detection signal indicating an output pixel value modulated by an incident angle of the incident light, and outputs a detection signal set including a plurality of detection signals output from the plurality of pixel output units. The present disclosure is applicable to, for example, an imaging system that images using a plurality of imaging devices.

The invention relates to a method of 3D reconstruction of a scene by means of 2D panoramic images of the scene, which comprises a step of processing these images. These images arise from a panoramic system moving in displacement along a determined trajectory, such that the image of at least one point of the scene is in at least 3 successive images obtained according to various directions; the step of processing these 2D successive images comprises the sub-steps: a) determining reconstruction planes in the scene to be reconstructed, b) determining, on the basis of pairs of panoramic images and for each pair, rectification planes corresponding to the reconstruction planes and projecting onto each of them a sector of each image of the pair, in a direct manner, so as to obtain two 2D rectified images, c) matching the two 2D rectified images so as to obtain an intermediate 3D reconstruction, d) transforming each intermediate 3D reconstruction into a 3D frame including the reconstruction planes so as to obtain a transformed intermediate 3D reconstruction, e) repeating steps b) to d) on the basis of a new pair of 2D panoramic images and of at least one other rectification plane, so as to obtain at least one other transformed intermediate 3D reconstruction, f) temporally fusing at least two transformed intermediate 3D reconstructions so as to obtain a 3D reconstruction of the scene.

Methods and apparatus for using selective resolution reduction on images to be transmitted and/or used by a playback device are described. Prior to transmission one or more images of an environment are captured. Based on image content, motion detection and/or user input a resolution reduction operation is selected and performed. The reduced resolution image is communicated to a playback device along with information indicating a UV map corresponding to the selected resolution allocation that should be used by the playback device for rendering the communicated image. By changing the resolution allocation used and which UV map is used by the playback device different resolution allocations can be made with respect to different portions of the environment while allowing the number of pixels in transmitted images to remain constant. The playback device renders the individual images with the UV map corresponding to the resolution allocation used to generate the individual images.

Multiple clients (viewers) are allowed to share their VR spaces for communication with one another. A server-distributed stream including a video stream obtained by encoding a background image is received from a server. A client-transmitted stream including representative image meta information for displaying a representative image of another client is received from another client apparatus. The video stream is decoded to obtain the background image. The image data of the representative image is generated on the basis of the representative image meta information. Display image data is obtained by synthesizing the representative image on the background image.

Disclosed is a cross-view image optimizing method and apparatus, and a computer equipment and a readable storage medium. The method includes: acquiring a sample image and a pre-trained cross-view image generating model; generating an multi-dimensional cross-view image of the sample image by a multi-dimensional feature extracting module of the first generator to obtain dimension features and cross-view initial images at multiple dimensions; obtaining a multi-dimensional feature map with corresponding dimension features by the second generator; inputting the multi-dimensional feature map to a multi-channel attention module of the second generator for feature extraction and calculating a feature weight of each attention channel, obtaining attention feature images, attention images and feature weights in a preset number of the attention channels; and weighting and summing the attention images and the attention feature images of all the channels according to the feature weights, and obtaining a cross-view target image.

A system and methods for a CODEC driving a real-time light field display for multi-dimensional video streaming, interactive gaming and other light field display applications is provided applying a layered scene decomposition strategy. Multi-dimensional scene data is divided into a plurality of data layers. One or more polygons representative of corresponding portions of objects in the scene are obtained for each layer and used to determine a view-independent representation. This view independent representation and data layers are sampled using a plenoptic sampling scheme and rendered using hybrid rendering to encode light fields corresponding to each data layer. The resulting compressed, (layered) core representation of the multi-dimensional scene data is produced at predictable rates, reconstructed and merged at the light field display in real-time by applying view synthesis protocols, including edge adaptive interpolation, to reconstruct pixel arrays in stages (e.g. columns then rows) from reference elemental images.

A system and methods for a CODEC driving a real-time light field display for multi-dimensional video streaming, interactive gaming and other light field display applications is provided applying a layered scene decomposition strategy. Multi-dimensional scene data is divided into a plurality of data layers of increasing depths as the distance between a given layer and the display surface increases. Data layers are sampled using a plenoptic sampling scheme and rendered using hybrid rendering, such as perspective and oblique rendering, to encode light fields corresponding to each data layer. The resulting compressed, (layered) core representation of the multi-dimensional scene data is produced at predictable rates, reconstructed and merged at the light field display in real-time by applying view synthesis protocols, including edge adaptive interpolation, to reconstruct pixel arrays in stages (e.g. columns then rows) from reference elemental images.

Multiple digital cameras view a large scene, such as a part of a city. Some of the cameras view different parts of that scene, and video feeds from the cameras are processed at a computer to generate a photo-realistic synthetic 3D model of the scene. This enables the scene to be viewed from any viewing angle, including angles that the original, real cameras do not occupy—i.e. as though viewed from a ‘virtual camera’ that can be positioned in any arbitrary position. The 3D model combines both static elements that do not alter in real-time, and also dynamic elements that do alter in real-time or near real-time.

The present disclosure relates to a method for transmitting two consecutive pairs of images. The method may include decimating each image with a ratio of 2, assembling the two decimated images of each pair in a composite image, transmitting the composite images, and reconstructing complete images from the composite images. In decimation, the information removed from the images of the first pair may be kept in the images of the second pair, from the spatial point of view, and the complete images may be reconstructed by de-interlacing processing from the composite images.