According to examples, a display system may include a head-mounted display (HMD) headset. The HMD may comprise at least one exterior-facing RBG camera mounted on a front face of the HMD and on a same visual plane of a user's eye. The at least one exterior-facing RBG camera may collects images for view synthesis. The HMD may include a processor, and a memory storing instructions, which when executed by the processor, cause the processor to provide view synthesis in accordance with a machine-learning (ML) based technique comprising at least one of the following: depth estimation, imaging to sharpening, forward splatting, disocclusion filtering, or fusion.

A method for training a depth estimation model with depth hints is disclosed. For each image pair: for a first image, a depth prediction is determined by the depth estimation model and a depth hint is obtained; the second image is projected onto the first image once to generate a synthetic frame based on the depth prediction and again to generate a hinted synthetic frame based on the depth hint; a primary loss is calculated with the synthetic frame; a hinted loss is calculated with the hinted synthetic frame; and an overall loss is calculated for the image pair based on a per-pixel determination of whether the primary loss or the hinted loss is smaller, wherein if the hinted loss is smaller than the primary loss, then the overall loss includes the primary loss and a supervised depth loss between depth prediction and depth hint. The depth estimation model is trained by minimizing the overall losses for the image pairs.

A method for deriving at least one item of information from images of a stereo camera. A resource-saving and at the same time accurate derivation of information from the images is achieved by alternately using and processing the images with methods of monocular image processing and deriving at least one item of information from the results. A motor vehicle comprising a stereo camera and comprising a control device which carries out the method, are also described.

According to an aspect, there is provided a method comprising controlling a structural light source of a modelling arrangement to produce a diffraction pattern of a known geometry on a surface to be modeled, the diffraction pattern accurately complying with a mathematical-physical model and wherein beam output angles of the diffraction pattern are accurately known based on the mathematical-physical model; recording a first image of the surface comprising the diffraction pattern with a first camera and a second image of the surface comprising the diffraction pattern with a second camera substantially simultaneously; determining a point cloud comprising primary points from the diffraction pattern visible in the first image; identifying the corresponding primary points from the second image; and using each primary point of the point cloud in the first and second images as an initial point for search spaces for secondary points in the first and second images.

An image processing device includes a rotation processor and an image processor. The rotation processor receives an input image and generates a temporary image according to the input image. The image processor is coupled to the rotation processor and outputs a processed image according to the temporary image, wherein the image processor has a predetermined image processing width, a width of the input image is larger than the predetermined image processing width, and a width of the temporary image is less than the predetermined image processing width.

A method for generating a 3D image includes driving structured light projector(s) to project a pattern of light on an intraoral 3D surface, and driving camera(s) to capture images, each image including at least a portion of the projected pattern, each one of the camera(s) comprising an array of pixels. A processor compares a series of images captured by each camera and determines which of the portions of the projected pattern can be tracked across the images. The processor constructs a three-dimensional model of the intraoral three-dimensional surface based at least in part on the comparison of the series of images. Other embodiments are also described.

Depth information can be used to assist with image processing functionality, such as image stabilization and blur reduction. In at least some embodiments, depth information obtained from stereo imaging or distance sensing, for example, can be used to determine a foreground object and background object(s) for an image or frame of video. The foreground object then can be located in later frames of video or subsequent images. Small offsets of the foreground object can be determined, and the offset accounted for by adjusting the subsequent frames or images. Such an approach provides image stabilization for at least a foreground object, while providing simplified processing and reduce power consumption. Similarly processes can be used to reduce blur for an identified foreground object in a series of images, where the blur of the identified object is analyzed.

A stereo camera apparatus includes a first imaging unit including a first imaging optical system provided with a plurality of lens groups, and a first actuator configured to change a focal length by driving at least one of the plurality of lens groups of the first imaging optical system; a second imaging unit including a second imaging optical system provided with a plurality of lens groups, and a second actuator configured to change a focal length by driving at least one of the plurality of lens groups of the second imaging optical system; a focal length controller configured to output synchronized driving signals to the first and second actuators; and an image processing unit configured to calculate a distance to a subject by using images captured by the first imaging unit and the second imaging unit.

Indirect time-of-flight (i-ToF) image sensor pixels, i-ToF image sensors including such pixels, stereo cameras including such image sensors, and sensing methods to obtain i-ToF detection and phase detection information using such image sensors and stereo cameras. An i-ToF image sensor pixel may comprise a plurality of sub-pixels, each sub-pixel including a photodiode, a single microlens covering the plurality of sub-pixels and a read-out circuit for extracting i-ToF phase signals of each sub-pixel individually.

A depth estimation system is described capable of determining depth information using two images from two cameras. A first camera captures a first image and a second camera captures a second image, both images including a plurality of light channels. A scan direction is selected from a plurality of scan directions. For the selected scan direction, along each of a plurality of scanlines, the system compares pixels from the first image to pixels from the second image. The comparison is based on calculating a census transform for each pixel in the first image and a census transform for each pixel in the second image. This comparison is used to determine a stereo correspondence between the pixels in the first image and the pixels in the second image. The system generates a depth map based on the stereo correspondence.