Techniques to facilitate compression of depth data and real-time reconstruction of high-quality light fields. A parameter space of values for a line, pairs of endpoints on different sides of the line, and a palette index for each pixel of a pixel tile of a depth image is sampled. Values for the line, the pairs of endpoints, and the palette index that minimize an error are determined and stored.

A distributed, cross reality system efficiently and accurately compares location information that includes image frames. Each of the frames may be represented as a numeric descriptor that enables identification of frames with similar content. The resolution of the descriptors may vary for different computing devices in the distributed system based on degree of ambiguity in image comparisons and/or computing resources for the device. A descriptor computed for a cloud-based component operating on maps of large areas that can result in ambiguous identification of multiple image frames may use high resolution descriptors. High resolution descriptors reduce computationally intensive disambiguation processing. A portable device, which is more likely to operate on smaller maps and less likely to have the computational resources to compute a high resolution descriptor, may use a lower resolution descriptor.

Indirect time-of-flight (i-ToF) image sensor pixels, i-ToF image sensors including such pixels, stereo cameras including such image sensors, and sensing methods to obtain i-ToF detection and phase detection information using such image sensors and stereo cameras. An i-ToF image sensor pixel may comprise a plurality of sub-pixels, each sub-pixel including a photodiode, a single microlens covering the plurality of sub-pixels and a read-out circuit for extracting i-ToF phase signals of each sub-pixel individually.

A distributed, cross reality system efficiently and accurately compares location information that includes image frames. Each of the frames may be represented as a numeric descriptor that enables identification of frames with similar content. The resolution of the descriptors may vary for different computing devices in the distributed system based on degree of ambiguity in image comparisons and/or computing resources for the device. A descriptor computed for a cloud-based component operating on maps of large areas that can result in ambiguous identification of multiple image frames may use high resolution descriptors. High resolution descriptors reduce computationally intensive disambiguation processing. A portable device, which is more likely to operate on smaller maps and less likely to have the computational resources to compute a high resolution descriptor, may use a lower resolution descriptor.

A method of displaying a light field to at least one viewer of a light field display device, the light field based on a 3D model, the light field display device comprising a plurality of spatially distributed display elements, the method including the steps of: (a) determining the viewpoints of the eyes of the at least one viewer relative to the display device; (b) for each eye viewpoint and each of a plurality of the display elements, rendering a partial view image representing a view of the 3D model from the eye viewpoint through the display element; and (c) displaying, via each display element, the set of partial view images rendered for that display element.

A method of displaying a light field to at least one viewer of a light field display device, the light field based on a 3D model, the light field display device comprising a plurality of spatially distributed display elements, the method including the steps of: (a) determining the viewpoints of the eyes of the at least one viewer relative to the display device; (b) for each eye viewpoint and each of a plurality of the display elements, rendering a partial view image representing a view of the 3D model from the eye viewpoint through the display element; and (c) displaying, via each display element, the set of partial view images rendered for that display element.

The present invention generally relates to an apparatus and a method for obtaining a registration error map representing a level of sharpness of an image. Many methods are known which allow determining the position of a camera with respect to an object, based on the knowledge of a 3D model of the object and the intrinsic parameters of the camera. However, regardless of the visual servoing technique used, there is no control in the image space and the object may get out of the camera field of view during servoing. It is proposed to obtain a registration error map relating to an image of the object of interest generated by computing an intersection of a re-focusing surface obtained from a 3D model of said object of interest and a focal stack based on acquired four-dimensional light-field data relating to said object of interest.

The present invention generally relates to an apparatus and a method for obtaining a registration error map representing a level of sharpness of an image. Many methods are known which allow determining the position of a camera with respect to an object, based on the knowledge of a 3D model of the object and the intrinsic parameters of the camera. However, regardless of the visual servoing technique used, there is no control in the image space and the object may get out of the camera field of view during servoing. It is proposed to obtain a registration error map relating to an image of the object of interest generated by computing an intersection of a re-focusing surface obtained from a 3D model of said object of interest and a focal stack based on acquired four-dimensional light-field data relating to said object of interest.

The present application provides a compound eye camera device comprising a plurality of ommatidia arranged in a column or a row, and each of the ommatidia comprises an optical element and corresponding photosensitive units; each of the ommatidium columns corresponds to at least one ommatidium-column visual plane, the at least one ommatidium-column visual plane passing through the optical center of each ommatidium in the ommatidium column and a position near the center of at least one photosensitive unit of each ommatidium; each photosensitive unit intersects at least one ommatidium-column visual plane, and sight line of each photosensitive unit passes through the center of the photosensitive unit and the optical center of the ommatidium where the photosensitive unit is located; and a processor is configured to generate images based on information received by the photosensitive units, and to process the images to obtain information regarding the photographed object.

A method of synthesizing intermediate views of a light field includes selecting a configuration of specific input views of a light field collected by a light field acquiring device, specifying coordinates of intermediate views to be synthesized and inputting the specified coordinates to a neural network, and synthesizing intermediate views based on a scene disparity, a selected configuration of the specific input views, and the specified coordinates of the intermediate views, using a neural network.