Methods of encoding and decoding depth data are disclosed. In an encoding method, depth values and occupancy data are both encoded into a depth map. The method adapts how the depth values and occupancy data are converted to map values in the depth map. For example, it may adaptively select a threshold, above or below which all values represent unoccupied pixels. By adapting how the depth and occupancy are encoded, based on analysis of the depth values, the method can enable more effective encoding and transmission of the depth data and occupancy data. The encoding method outputs metadata defining the adaptive encoding. This metadata can be used by a corresponding decoding method, to decode the map values. Also provided are an encoder and a decoder for depth data, and a corresponding bitstream, comprising a depth map and its associated metadata.

Methods of encoding and decoding depth data are disclosed. In an encoding method, depth values and occupancy data are both encoded into a depth map. The method adapts how the depth values and occupancy data are converted to map values in the depth map. For example, it may adaptively select a threshold, above or below which all values represent unoccupied pixels. By adapting how the depth and occupancy are encoded, based on analysis of the depth values, the method can enable more effective encoding and transmission of the depth data and occupancy data. The encoding method outputs metadata defining the adaptive encoding. This metadata can be used by a corresponding decoding method, to decode the map values. Also provided are an encoder and a decoder for depth data, and a corresponding bitstream, comprising a depth map and its associated metadata.

Systems and methods for implementing methods for cloud-based rendering of interactive augmented reality (AR) and/or virtual reality (VR) experiences. A client device may initiate execution of a content application on a server and provide information associated with the content application to the server. The client device may initialize, while awaiting a notification from the server, local systems associated with the content application and, upon receipt of the notification, provide, to the server, information associated with the local systems. Further, the client device may receive, from the server, data associated with the content application and render an AR/VR scene based on the received data. The data may be based, at least in part, on the information associated with the local system. The providing and receiving may be performed periodically, e.g., at a rate to sustain a comfortable viewing environment of the AR/VR scene by a user of the client device.

Systems and methods for implementing methods for cloud-based rendering of interactive augmented reality (AR) and/or virtual reality (VR) experiences. A client device may initiate execution of a content application on a server and provide information associated with the content application to the server. The client device may initialize, while awaiting a notification from the server, local systems associated with the content application and, upon receipt of the notification, provide, to the server, information associated with the local systems. Further, the client device may receive, from the server, data associated with the content application and render an AR/VR scene based on the received data. The data may be based, at least in part, on the information associated with the local system. The providing and receiving may be performed periodically, e.g., at a rate to sustain a comfortable viewing environment of the AR/VR scene by a user of the client device.

A process and an apparatus for the plenoptic capture of photographic or cinematographic images of an object or a 3D scene (10) of interest are based on a correlated light emitting source and correlation measurement, along the line of “Correlation Plenoptic Imaging” (CPI). A first image sensor (Da) and a second image sensor (Db) detect images along a path of a first light beam (a) and a second light beam (b), respectively. A processing unit (100) of the intensities detected by the synchronized image sensors (Da, Db) is configured to retrieve the propagation direction of light by measuring spatio-temporal correlations between light intensities detected in the image planes of at least two arbitrary planes (P′, P″; D′b, D″a) chosen in the vicinity of the object or within the 3D scene (10).

A process and an apparatus for the plenoptic capture of photographic or cinematographic images of an object or a 3D scene (10) of interest are based on a correlated light emitting source and correlation measurement, along the line of “Correlation Plenoptic Imaging” (CPI). A first image sensor (Da) and a second image sensor (Db) detect images along a path of a first light beam (a) and a second light beam (b), respectively. A processing unit (100) of the intensities detected by the synchronized image sensors (Da, Db) is configured to retrieve the propagation direction of light by measuring spatio-temporal correlations between light intensities detected in the image planes of at least two arbitrary planes (P′, P″; D′b, D″a) chosen in the vicinity of the object or within the 3D scene (10).

A system for depth estimation, comprises at least a first and a second depth estimation optical systems, each configured for receiving a light beam from a scene and estimating depths within the scene, wherein the first system is a monocular depth estimation optical system; and an image processor, configured for receiving depth information from the first and second systems, and generating a depth map or a three-dimensional image of the scene based on the received depth information.

A system and method for generating a depth map from image data in a surgical robotic system that employs a robotic subsystem having a camera assembly with first and second cameras for generating image data. The system and method generates based on the image data a plurality of depth maps, and then converts the plurality of depth maps into a single combined depth map having distance data associated therewith. The system and method can then control the camera assembly based on the distance data in the single combined depth map.

A system to capture a plurality of two dimensional digital source images of scene by user, a smart device having a memory device for storing an instruction, a processor in communication with the memory and configured to execute the instruction, a plurality of digital image capture devices in communication with the processor and each image capture device configured to capture a digital image of the scene, the plurality of digital image capture devices positioned linearly in series within approximately an interpupillary distance, wherein a first digital image capture devices is centered proximate a first end of the interpupillary distance, a second digital image capture devices is centered on a second end of the interpupillary distance, and any remaining the plurality of digital image capture devices are evenly spaced therebetween, a display in communication with the processor, display configured to display multidimensional digital image and add audio file thereto.

Systems are configured for generating temporally corrected pass-through images. In some instances, the systems obtain depth maps of an environment at a first timepoint, generate a 3D representation of the environment by unprojecting the depth information represented in the depth map, and obtain one or more first images of the environment captured at a second timepoint. The systems may also be configured to perform a first intermediate projection to identify first texture information from the one or more first images, identify a display pose associated with the system, generate a display projection of the 3D representation, and creating a composite image based on the display projection and the first texture information.