Methods, apparatuses and streams are disclosed for transmitting tiled volumetric video and, at the receiver, for generating an atlas image compatible with a legacy decoder. At the server side, viewport information is obtained and a first list of central tiles and a second list of border tiles are selected. A central tile is a part of an image obtained by projecting the 3D scene onto an image plane according to a central point of view. A border tile is an image comprising dis-occluding patches. Sizes and shapes of border tiles are function of size and shape of central tiles. At the client side, tiles are arranged according to a layout selected in a set of layouts according to the number, sizes and shapes of border tiles.

FIG. 8.

Methods, apparatuses and streams are disclosed for transmitting tiled volumetric video and, at the receiver, for generating an atlas image compatible with a legacy decoder. At the server side, viewport information is obtained and a first list of central tiles and a second list of border tiles are selected. A central tile is a part of an image obtained by projecting the 3D scene onto an image plane according to a central point of view. A border tile is an image comprising dis-occluding patches. Sizes and shapes of border tiles are function of size and shape of central tiles. At the client side, tiles are arranged according to a layout selected in a set of layouts according to the number, sizes and shapes of border tiles.

FIG. 8.

Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations (MIDMRs). In one aspect, a method for automatically generating a MIDMR comprises obtaining a first MIDMR and a second MIDMR. The first MIDMR includes a convex or concave motion capture using a recording device and is a general object MIDMR. The second MIDMR is a specific feature MIDMR. The first and second MIDMRs may be obtained using different capture motions. A third MIDMR is generated from the first and second MIDMRs, and is a combined embedded MIDMR. The combined embedded MIDMR may comprise the second MIDMR being embedded in the first MIDMR, forming an embedded second MIDMR. The third MIDMR may include a general view in which the first MIDMR is displayed for interactive viewing by a user on a user device. The embedded second MIDMR may not be viewable in the general view.

Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations (MIDMRs). In one aspect, a method for automatically generating a MIDMR comprises obtaining a first MIDMR and a second MIDMR. The first MIDMR includes a convex or concave motion capture using a recording device and is a general object MIDMR. The second MIDMR is a specific feature MIDMR. The first and second MIDMRs may be obtained using different capture motions. A third MIDMR is generated from the first and second MIDMRs, and is a combined embedded MIDMR. The combined embedded MIDMR may comprise the second MIDMR being embedded in the first MIDMR, forming an embedded second MIDMR. The third MIDMR may include a general view in which the first MIDMR is displayed for interactive viewing by a user on a user device. The embedded second MIDMR may not be viewable in the general view.

An image processing apparatus determines whether an image represented by image data includes two circular areas and executes predetermined processing on the image if it is determined that the image includes two circular areas. The image processing apparatus can recognize if the image data is taken with a dual-eye lens even when an image capture apparatus that has recorded the image data has not recognized that a dual-eye lens unit has been attached and processing the image appropriately.

An image processing apparatus determines whether an image represented by image data includes two circular areas and executes predetermined processing on the image if it is determined that the image includes two circular areas. The image processing apparatus can recognize if the image data is taken with a dual-eye lens even when an image capture apparatus that has recorded the image data has not recognized that a dual-eye lens unit has been attached and processing the image appropriately.

A video transmission method that includes predicting, from a texture picture or a depth picture of an anchor viewing position, a picture for a target viewing position on the basis of target viewing position information and processing a prediction error with respect to the predicted picture on the basis of a source picture of the target viewing position. An error-prone region map is generated on the basis of the predicted picture and the source picture. The video transmission method also includes patch packing the prediction error-processed picture on the basis of the error-prone region map and encoding the packed patch on the basis of the texture picture or the depth picture of the anchor viewing position.

A video transmission method that includes predicting, from a texture picture or a depth picture of an anchor viewing position, a picture for a target viewing position on the basis of target viewing position information and processing a prediction error with respect to the predicted picture on the basis of a source picture of the target viewing position. An error-prone region map is generated on the basis of the predicted picture and the source picture. The video transmission method also includes patch packing the prediction error-processed picture on the basis of the error-prone region map and encoding the packed patch on the basis of the texture picture or the depth picture of the anchor viewing position.

Methods of encoding and decoding depth data are disclosed. In an encoding method, depth values and occupancy data are both encoded into a depth map. The method adapts how the depth values and occupancy data are converted to map values in the depth map. For example, it may adaptively select a threshold, above or below which all values represent unoccupied pixels. By adapting how the depth and occupancy are encoded, based on analysis of the depth values, the method can enable more effective encoding and transmission of the depth data and occupancy data. The encoding method outputs metadata defining the adaptive encoding. This metadata can be used by a corresponding decoding method, to decode the map values. Also provided are an encoder and a decoder for depth data, and a corresponding bitstream, comprising a depth map and its associated metadata.

Methods of encoding and decoding depth data are disclosed. In an encoding method, depth values and occupancy data are both encoded into a depth map. The method adapts how the depth values and occupancy data are converted to map values in the depth map. For example, it may adaptively select a threshold, above or below which all values represent unoccupied pixels. By adapting how the depth and occupancy are encoded, based on analysis of the depth values, the method can enable more effective encoding and transmission of the depth data and occupancy data. The encoding method outputs metadata defining the adaptive encoding. This metadata can be used by a corresponding decoding method, to decode the map values. Also provided are an encoder and a decoder for depth data, and a corresponding bitstream, comprising a depth map and its associated metadata.