Techniques for aligning images generated by an integrated camera physically mounted to an HMD with images generated by a detached camera physically unmounted from the HMD are disclosed. A 3D feature map is generated and shared with the detached camera. Both the integrated camera and the detached camera use the 3D feature map to relocalize themselves and to determine their respective 6 DOF poses. The HMD receives the detached camera's image of the environment and the 6 DOF pose of the detached camera. A depth map of the environment is accessed. An overlaid image is generated by reprojecting a perspective of the detached camera's image to align with a perspective of the integrated camera and by overlaying the reprojected detached camera's image onto the integrated camera's image.

In one embodiment, a method includes accessing first image data generated by a first image sensor having a first filter array that has a first filter pattern. The first filter pattern includes a number of first filter types. The method also includes accessing second image data generated by a second image sensor having a second filter array that has a second filter pattern different from the first filter pattern. The second filter pattern includes a number of second filter types, the number of second filter types and the number of first filter types have at least one filter type in common. The method also includes determining a correspondence between one or more first pixels of the first image data and one or more second pixels of the second image data based on a portion of the first image data associated with the filter type in common.

There is disclosed an apparatus and a method for spatial computing service session description for volumetric XR conversation. In accordance an embodiment the method comprises receiving volumetric video from a plurality of sources, the volumetric video data comprising color images and depth images representing at least a part of a scene; determining which of the plurality of sources belong to a same group; selecting a group identifier for the group; associating the volumetric video with the group identifier indicative of from which group the volumetric video data is received; associating the volumetric video with a source identifier indicative of from which source of the group the volumetric video data is received; associating the color image of the volumetric video with a color data identifier; and associating the depth image of the volumetric video with a depth data identifier.

A method for acquiring a distance from a moving body to an object located in any direction of the moving body includes steps of: an image processing device (a) instructing a sweep network to project pixels of images, generated by cameras covering all directions of the moving body, onto main virtual geometries and apply 3D concatenation operation thereon to generate an initial 4D cost volume, (b) generating a final main 3D cost volume therefrom through a cost volume computation network, and (c) generating sub inverse distance indices corresponding to inverse values of sub separation distances between a sub reference point and sub virtual geometries, and main inverse distance indices corresponding to inverse values of main separation distances between a main reference point and the main virtual geometries, by using a sub cost volume and the final main 3D cost volume, to thereby acquire the distance to the object.

Methods and devices for manipulating an image are described. The method comprises receiving image data, the image data including a first image obtained from a first camera and a second image obtained from a second camera, the first camera and the second camera being oriented in a common direction; identifying one or more boundaries of an object in the image data by analyzing the first image and the second image; and displaying a manipulated image based on the image data, wherein the manipulated image includes manipulation of at least a portion of the first image based on boundaries of the object.

ADAS includes a processing circuit and a memory which stores instructions executable by the processing circuit. The processing circuit executes the instructions to cause the ADAS to receive, from a vehicle that is in motion, a video sequence, generate a position image including at least one object included in the stereo image, generate a second position information associated with the at least one object based on reflected signals received from the vehicle, determine regions each including at least a portion of the at least one object as candidate bounding boxes based on the stereo image and the position image, and selectively adjusting class scores of respective ones of the candidate bounding boxes associated with the at least one object based on whether a respective first position information of the respective ones of the candidate bounding boxes matches the second position information.

An image display apparatus according to an embodiment of the present technology includes a plurality of projection units, a screen, and an image generation unit. The plurality of projection units projects image light corresponding to image data with reference to projection axes thereof and is disposed such that the projection axes face in directions different from each other along a first plane. The screen is disposed to intersect with the first plane at a first elevation angle (φ.sub.i) and diffuses and outputs the image light projected along the first plane at a second elevation angle (φ.sub.o) different from the first elevation angle (φ.sub.i). The image generation unit generates the image data for displaying a plurality of viewpoint images corresponding to viewpoints at which the screen is observed at the second elevation angle (φ.sub.o) on the basis of the directions of the projection axes on the first plane.

This disclosure relates to techniques for generating robust depth estimations for captured images using semantic segmentation. Semantic segmentation may be defined as a process of creating a mask over an image, wherein pixels are segmented into a predefined set of semantic classes. Such segmentations may be binary (e.g., a ‘person pixel’ or a ‘non-person pixel’) or multi-class (e.g., a pixel may be labelled as: ‘person,’ ‘dog,’ ‘cat,’ etc.). As semantic segmentation techniques grow in accuracy and adoption, it is becoming increasingly important to develop methods of utilizing such segmentations and developing flexible techniques for integrating segmentation information into existing computer vision applications, such as depth and/or disparity estimation, to yield improved results in a wide range of image capture scenarios. In some embodiments, an optimization framework may be employed to optimize a camera device's initial scene depth/disparity estimates that employs both semantic segmentation and color regularization in a robust fashion.

A handheld device for the image-based measurement of a remote object, comprising a housing having a front side and a rear side, a first and second camera, which are arranged having a stereo base on the rear side, for recording images of the object, an analysis unit having an algorithm for the stereophotogrammetric analysis of the images of the cameras and a display unit, which is arranged on the front side, for displaying images of the object and results of the stereophotogrammetric analysis, wherein the housing has a longitudinal axis, the stereo base is aligned diagonally relative to the longitudinal axis, and the analysis unit is designed for the purpose of taking into consideration the relative alignment of the stereo base during the stereophotogrammetric analysis.

An image processing apparatus acquires first shape information representing a three-dimensional shape about an object located within an image capturing region based on one or more images obtained by one or more imaging apparatuses for performing image capturing of the image capturing region from a plurality of directions, acquires second shape information representing a three-dimensional shape about an object located within the image capturing region based on one or more images obtained by one or more imaging apparatuses, acquires viewpoint information indicating a position and direction of a viewpoint, and generates a virtual viewpoint image corresponding to the position and direction of the viewpoint indicated by the acquired viewpoint information based on the acquired first shape information and the acquired second shape information, such that at least a part of the object corresponding to the second shape information is displayed in a translucent way within the virtual viewpoint image.