An example method for training a machine learning model is provided. The method includes receiving training data collected by a three-dimensional (3D) imager, the training data comprising a plurality of training sets. The method further includes generating, using the training data, a machine learning model from which a disparity map can be inferred from a pair of images that capture a scene where a light pattern is projected onto an object.

The present invention is generally directed to methods and systems of estimating visibility around a vehicle and automatically configuring one or more systems in response to the visibility level. The visibility level can be estimated by comparing two images of the vehicle's surroundings, each taken from a different perspective. Distance of objects in the images can be estimated based on the disparity between the two images, and the visibility level (e.g., a distance) can be estimated based on the farthest object that is visible in the images.

An image processing apparatus includes a receiving unit configured to receive at least two parallax images that are obtained from a subject image captured via a single optical system, where the at least two parallax images include an image in a first viewpoint direction and an image in a second viewpoint direction, an average calculating unit configured to calculate, for each pixel, an arithmetic average and a geometric average between the image in the first viewpoint direction and the image in the second viewpoint direction, a ratio calculating unit configured to calculate, for each pixel, a ratio of the arithmetic average to the geometric average, and a disparity calculating unit configured to calculate, on a pixel-by-pixel basis, a disparity between the image in the first viewpoint direction and the image in the second viewpoint direction based on the ratio.

A system and method are provided for autonomously navigating a vehicle. The method captures a sequence of image pairs using a stereo camera. A navigation application stores a vehicle pose (history of vehicle position). The application detects a plurality of matching feature points in a first matching image pair, and determines a plurality of corresponding object points in three-dimensional (3D) space from the first image pair. A plurality of feature points are tracked from the first image pair to a second image pair, and the plurality of corresponding object points in 3D space are determined from the second image pair. From this, a vehicle pose transformation is calculated using the object points from the first and second image pairs. The rotation angle and translation are determined from the vehicle pose transformation. If the rotation angle or translation exceed a minimum threshold, the stored vehicle pose is updated.

Method and apparatus for estimating the depth of focused plenoptic data are suggested. The method includes: estimating the inherent shift of in-focus pixels of the focused plenoptic data; calculating a level of homogeneity of the pixels of the focused plenoptic data; determining the pixels of the focused plenoptic data which either have disparities equal to the inherent shift or belong to homogeneous areas, as a function of the level of homogeneity of the pixels of the focused plenoptic data; and estimating the depth of the focused plenoptic data by a disparity estimation without considering the determined pixels. According to the disclosure, the pixels of the focused plenoptic data which either have a disparity equal to the inherent shift or belong to a homogeneous area will not be considered for the estimation of the depth, which can reduce computational costs and at the same time increase accuracy of estimations for in-focus parts of the scene.

Disclosed is a method for hole-filling in 3D models. The method includes extracting static background information from a current frame of an input image and extracting virtual static background information using the static background information, warping a color image and a depth map of the current frame to acquire a virtual image and a virtual depth map, and labeling a hole area formed in the virtual depth map to extract local background information, performing a first hole-filling onto the virtual image and the virtual depth map using a similarity between the virtual static background information and the local background information, and performing a second hole-filling with respect to remaining holes after the first hole-filling in a manner of an exemplar-based in-painting method to which a priority function including a depth term is applied.

A display apparatus and a control method thereof are provided. The display apparatus includes a communication interface configured to receive captured images and information related to the captured images; a display; and a processor configured to: obtain an object disparity of an object included in the captured images and a number of the captured images based on the information related to the captured images; identify whether a display disparity representable by the display matches the object disparity; based on the display disparity not matching the object disparity, generate interpolated images by performing image interpolation based on the display disparity, the object disparity, and the number of the captured images; and control the display to display a three-dimensional content based on the captured images and the interpolated images.

A method and system for segmenting a plurality of images. The method comprises the steps of segmenting the image through a novel clustering technique that is, generating a composite depth map including temporally stable segments of the image as well as segments in subsequent images that have changed. These changes may be determined by determining one or more differences between the temporally stable depth map and segments included in one or more subsequent frames. Thereafter, the portions of the one or more subsequent frames that include segments including changes from their corresponding segments in the temporally stable depth map are processed and are combined with the segments from the temporally stable depth map to compute their associated disparities in one or more subsequent frames. The images may include a pair of stereo images acquired through a stereo camera system at a substantially similar time.

An apparatus a receiver (201) which receives a wide angle image with a first projection where a vertical image position of a scene position depends on a horizontal distance from the scene position to an optical axis for the image. Thus, the vertical image position of the scene point may depend on the horizontal image position. A mapper (203) generates a modified image having a modified projection by applying a mapping to the first wide angle image corresponding to a mapping from the first projection to a perspective projection followed by a non-linear vertical mapping from the perspective projection to a modified vertical projection of the modified projection and a non-linear horizontal mapping from the perspective projection to a modified horizontal projection of the modified projection. A disparity estimator (205) generates disparities for the modified image relative to a second image and representing a different view point than the first wide angle image.

A stereo matching method includes obtaining a first feature map associated with a first view image and a second feature map associated with a second view image using a neural network model-based feature extractor, determining respective matching costs between a reference pixel of the first view image and candidate pixels of the second view image using the first feature map and the second feature map, and determining a pixel corresponding to the reference pixel among the candidate pixels based on the determined matching costs.