A method for deriving at least one item of information from images of a stereo camera. A resource-saving and at the same time accurate derivation of information from the images is achieved by alternately using and processing the images with methods of monocular image processing and deriving at least one item of information from the results. A motor vehicle comprising a stereo camera and comprising a control device which carries out the method, are also described.

A system for displaying a stereo image is disclosed. The system includes a stereoscopic head up display (HUD) that includes a combiner and a display configured to display a stereo image. The system further includes either a head tracking sensor or an eye tracking sensor configured to generate a tracking dataset. The system further includes a processor and a memory with instructions that cause the processor to receive the tracking dataset, generate a distortion map based on the tracking dataset and a distortion function, receive a media stereo image, generate a stereo signal based on the distortion map and the media stereo image, and transmit the stereo signal to the display. The system is capable of conveying an image to an operator in the absence of a relay lens.

A system for displaying a stereo image is disclosed. The system includes a stereoscopic head up display (HUD) that includes a combiner and a display configured to display a stereo image. The system further includes either a head tracking sensor or an eye tracking sensor configured to generate a tracking dataset. The system further includes a processor and a memory with instructions that cause the processor to receive the tracking dataset, generate a distortion map based on the tracking dataset and a distortion function, receive a media stereo image, generate a stereo signal based on the distortion map and the media stereo image, and transmit the stereo signal to the display. The system is capable of conveying an image to an operator in the absence of a relay lens.

A depth estimation system is described capable of determining depth information using two images from two cameras. A first camera captures a first image and a second camera captures a second image, both images including a plurality of light channels. A scan direction is selected from a plurality of scan directions. For the selected scan direction, along each of a plurality of scanlines, the system compares pixels from the first image to pixels from the second image. The comparison is based on calculating a census transform for each pixel in the first image and a census transform for each pixel in the second image. This comparison is used to determine a stereo correspondence between the pixels in the first image and the pixels in the second image. The system generates a depth map based on the stereo correspondence.

A depth estimation system is described capable of determining depth information using two images from two cameras. A first camera captures a first image and a second camera captures a second image, both images including a plurality of light channels. A scan direction is selected from a plurality of scan directions. For the selected scan direction, along each of a plurality of scanlines, the system compares pixels from the first image to pixels from the second image. The comparison is based on calculating a census transform for each pixel in the first image and a census transform for each pixel in the second image. This comparison is used to determine a stereo correspondence between the pixels in the first image and the pixels in the second image. The system generates a depth map based on the stereo correspondence.

An image processing apparatus that performs processing concerning the display of a stereoscopic image so as to improve the convenience of a user in viewing a stereoscopic image. A head mount display displays a stereoscopic image using image data including a plurality of images having different viewpoints. The image data is processed based on metadata attached to the image data. In a case where information indicating that the image data is associated with a file format which cannot cause the head mount display to perform display is included in the metadata, the image data is converted into a file format which can cause the head mount display to perform display.

Disclosed herein is a method for using a neural network across multiple devices. The method can include receiving, by a first device configured with a first one or more layers of a neural network, input data for processing via the neural network implemented across the first device and a second device. The method can include outputting, by the first one or more layers of the neural network implemented on the first device, a data set that is reduced in size relative to the input data while identifying one or more features of the input data for processing by a second one or more layers of the neural network. The method can include communicating, by the first device, the data set to the second device for processing via the second one or more layers of the neural network implemented on the second device.

Disclosed herein is a method for using a neural network across multiple devices. The method can include receiving, by a first device configured with a first one or more layers of a neural network, input data for processing via the neural network implemented across the first device and a second device. The method can include outputting, by the first one or more layers of the neural network implemented on the first device, a data set that is reduced in size relative to the input data while identifying one or more features of the input data for processing by a second one or more layers of the neural network. The method can include communicating, by the first device, the data set to the second device for processing via the second one or more layers of the neural network implemented on the second device.

An information processing apparatus determines which of a first method for generating virtual viewpoint video upon receiving from an external apparatus object data used for generation of the virtual viewpoint video or a second method for receiving from the external apparatus virtual viewpoint video generated upon a virtual viewpoint designated for generation of the virtual viewpoint video to use as a delivery method for delivery of the virtual viewpoint video; and obtains the virtual viewpoint video by using the determined delivery method.

Methods and/or apparatus provide for obtaining intrinsics and extrinsics of a plurality of cameras arranged to capture a scene and based thereon, generate a voxel representation of the scene; obtaining extrinsics and intrinsics of a virtual camera; mapping the extrinsics and intrinsics of the virtual camera to at least one voxel surface defined in a data structure and to identify at least one camera pair suitable for obtaining depth data for that voxel surface based on the corresponding camera pair identifier; and receiving an input from the mapping unit, and in response obtaining images captured by the cameras in the at least one camera pair identified as being suitable for reconstructing the scene from the viewpoint of the virtual camera, and to generate an image of the scene from the viewpoint of the virtual camera based on the obtained images.