Using the same image sensor to capture both a two-dimensional (2D) image of a three-dimensional (3D) object and 3D depth measurements for the object. A laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation. Each row of pixels in the pixel array forms an epipolar line of the corresponding laser scan line. Timestamping provides a correspondence between the pixel location of a captured light spot and the respective scan angle of the laser to remove any ambiguity in triangulation. An Analog-to-Digital Converter (ADC) in the image sensor generates a multi-bit output in the 2D mode and a binary output in the 3D mode to generate timestamps. Strong ambient light is rejected by switching the image sensor to a 3D logarithmic mode from a 3D linear mode.

An electronic system for generating panoramic light fields is provided. The electronic system includes a camera, a depth estimation circuit, and a light field generation circuit. The camera is configured to capture a panoramic image of a scene. The depth estimation circuit is configured to estimate panoramic depth information of the scene based on the panoramic image captured by the camera. The light field generation circuit is configured to generate a panoramic light field based on the estimated panoramic depth information.

A near-eye light-field display apparatus is provided, which includes a display module, a first lens module, an aperture device, and a second lens module. The display module displays light-field data. The first lens module collimates light rays emitted from the display module. The aperture device generates a plurality of coded apertures. The collimated light rays are modulated by the coded apertures. The second lens module focuses the modulated light rays on an image plane to form a real image.

A method of encoding a video data signal (15) is provided, together with a method for decoding. The encoding comprises providing color information (51) for pixels in an image, providing a depth map with depth information (52) for the pixels, providing transition information (56, 57, 60, 70, 71) being representative of a width (63, 73) of a transition region (61, 72) in the image, the transition region (61, 72) comprising a depth transition (62) and blended pixels in which colors of a foreground object and a background object are blended, and generating (24) the video data signal (15) comprising encoded data representing the color information (51), the depth map (52) and the transition information (56, 57, 60, 70, 71). The decoding comprises using the transition information (56, 57, 60, 70, 71) for determining the width (63, 73) of the transition regions (61, 72) and for determining alpha values (53) for pixels inside the transition regions (61, 72). The determined alpha values (53) are used for determining the color of a blended pixel at the transition of a foreground object and a background object.

The usual coding order according to which the reference view is coded prior to the dependent view, and within each view, a depth map is coded subsequent to the respective picture, may be maintained and does lead to a sacrifice of efficiency in performing inter-view redundancy removal by, for example, predicting motion data of the current picture of the dependent view from motion data of the current picture of the reference view. Rather, a depth map estimate of the current picture of the dependent view is obtained by warping the depth map of the current picture of the reference view into the dependent view, thereby enabling various methods of inter-view redundancy reduction more efficiently by bridging the gap between the views. According to another aspect, the following discovery is exploited: the overhead associated with an enlarged list of motion predictor candidates for a block of a picture of a dependent view is comparatively low compared to a gain in motion vector prediction quality resulting from an adding of a motion vector candidate which is determined from an, in disparity-compensated sense, co-located block of a reference view.

This disclosure contains methods and systems that allow filmmakers to port filmmaking and editing skills to produce content to be used in other environments, such as video game environments, and augmented reality, virtual reality, mixed reality, and non-linear storytelling environments.

Using the same image sensor to capture both a two-dimensional (2D) image of a three-dimensional (3D) object and 3D depth measurements for the object. A laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation. Each row of pixels in the pixel array forms an epipolar line of the corresponding laser scan line. Timestamping provides a correspondence between the pixel location of a captured light spot and the respective scan angle of the laser to remove any ambiguity in triangulation. An Analog-to-Digital Converter (ADC) in the image sensor generates a multi-bit output in the 2D mode and a binary output in the 3D mode to generate timestamps. Strong ambient light is rejected by switching the image sensor to a 3D logarithmic mode from a 3D linear mode.

Using the same image sensor to capture a two-dimensional (2D) image and three-dimensional (3D) depth measurements for a 3D object. A laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation. Each row of pixels in the pixel array forms an epipolar line of the corresponding laser scan line. Timestamping provides a correspondence between the pixel location of a captured light spot and the respective scan angle of the laser to remove any ambiguity in triangulation. An Analog-to-Digital Converter (ADC) in the image sensor operates as a Time-to-Digital (TDC) converter to generate timestamps. A timestamp calibration circuit is provided on-board to record the propagation delay of each column of pixels in the pixel array and to provide necessary corrections to the timestamp values generated during 3D depth measurements.

A method of encoding a video data signal (15) is provided, together with a method for decoding. The encoding comprises providing color information (51) for pixels in an image, providing a depth map with depth information (52) for the pixels, providing transition information (56, 57, 60, 70, 71) being representative of a width (63, 73) of a transition region (61, 72) in the image, the transition region (61, 72) comprising a depth transition (62) and blended pixels in which colors of a foreground object and a background object are blended, and generating (24) the video data signal (15) comprising encoded data representing the color information (51), the depth map (52) and the transition information (56, 57, 60, 70, 71). The decoding comprises using the transition information (56, 57, 60, 70, 71) for determining the width (63, 73) of the transition regions (61, 72) and for determining alpha values (53) for pixels inside the transition regions (61, 72). The determined alpha values (53) are used for determining the color of a blended pixel at the transition of a foreground object and a background object.

A method and corresponding system for reconstructing the surface geometry of a three-dimensional object is disclosed. The system comprises a cluster of heterogeneous sensors, including a two-dimensional high-resolution camera and a three-dimensional depth camera, and a turntable operable to rotate incrementally. In operation, the turntable is rotated to first and second positions and two-dimensional and three-dimensional data sets are obtained using the two-dimensional high-resolution camera and the three-dimensional depth camera. Corresponding features from the two-dimensional data sets are identified and used to identify the same corresponding features in the three-dimensional data sets. The three-dimensional corresponding features are used to calculate a three-dimensional homography, which is used to align the three-dimensional data sets. Following alignment, a three-dimensional mesh is generated from the aligned data sets.