Methods and devices for encoding and decoding data representative of a 3D scene are disclosed. A set of first patches is generated from a first MVD content acquired from a first region of the 3D scene. A patch is a part of one of the views of the MVD content. A set of second patches is generated from a second MVD content acquired from a second region of the 3D scene. An atlas packing first and second patches is generated and associated with metadata indicating, for a patch of the atlas, whether the patch is a first or a second patch At the decoding side, first patches are used for rendering the viewport image and second patches are used for pre-processing or post-processing the viewport image.

Methods and devices for encoding and decoding data representative of a 3D scene are disclosed. A set of first patches is generated from a first MVD content acquired from a first region of the 3D scene. A patch is a part of one of the views of the MVD content. A set of second patches is generated from a second MVD content acquired from a second region of the 3D scene. An atlas packing first and second patches is generated and associated with metadata indicating, for a patch of the atlas, whether the patch is a first or a second patch At the decoding side, first patches are used for rendering the viewport image and second patches are used for pre-processing or post-processing the viewport image.

An encoding method comprises obtaining (101) an input set of volumetric image data, selecting (103) data from the image data for multiple views based on a visibility of the data from a respective viewpoint at a respective viewing direction and/or within a respective field of view such that a plurality of the views comprises only a part of the image data, encoding (105) each of the views as a separate output set (31), and generating (107) metadata which indicates the viewpoints. A decoding method comprises determining (121) a desired user viewpoint, obtaining (123) the metadata, selecting (125) one or more of the available viewpoints based on the desired user viewpoint, obtaining (127) one or more sets of image data in which one or more available views corresponding to the selected one or more available viewpoints have been encoded, and decoding (129) at least one of the one or more available views.

A system and methods are provided for large-scale generation of photorealistic 3D models, including training texture map and 3D mesh encoder and decoder neural networks, and training a sampler neural network to convert random seeds into input vectors for the texture map and 3D mesh decoder networks. Training the sampler neural network may include feeding random seeds to the sampler neural network, generating training 3D models from the texture map and 3D mesh decoders, rendering 2D images from the training 3D models, back-propagating output of realism classifier and of a uniqueness function of the 2D images to the sampler neural network; and providing the trained sampler neural network with additional random seed inputs to generate multiple respective input vectors for the texture map and 3D mesh decoders, and responsively generating by the texture map and 3D mesh decoders multiple new 3D models.

Disclosed herein is a method of transmitting point cloud data. The method may include encoding geometry data of the point cloud data, encoding attribute data of the point cloud data based on the geometry data, and transmitting the encoded geometry data, the encoded attribute data and signaling data, the geometry encoding includes splitting the geometry data into one or more prediction units, and inter-prediction encoding the geometry data by selectively applying a motion vector to each of the split prediction units, and the signaling data includes information for identifying whether the motion vector is applied for each prediction unit.

Disclosed herein is a method of transmitting point cloud data. The method may include encoding geometry data of the point cloud data, encoding attribute data of the point cloud data based on the geometry data, and transmitting the encoded geometry data, the encoded attribute data and signaling data, the geometry encoding includes splitting the geometry data into one or more prediction units, and inter-prediction encoding the geometry data by selectively applying a motion vector to each of the split prediction units, and the signaling data includes information for identifying whether the motion vector is applied for each prediction unit.

A computer-implemented method for designing a 3D modeled object via user-interaction. The method includes obtaining the 3D modeled object and a machine-learnt decoder. The machine-learnt decoder is a differentiable function taking values in a latent space and outputting values in a 3D modeled object space. The method further includes defining a deformation constraint for a part of the 3D modeled object. The method further comprises determining an optimal vector. The optimal vector minimizes an energy. The energy explores latent vectors. The energy comprises a term which penalizes, for each explored latent vector, non-respect of the deformation constraint by the result of applying the decoder to the explored latent vector. The method further includes applying the decoder to the optimal latent vector. This constitutes an improved method for designing a 3D modeled object via user-interaction.

A method, apparatus, and computer program product are provided for training a neural network or providing a pre-trained neural network with the weight-updates being compressible using at least a weight-update compression loss function and/or task loss function. The weight-update compression loss function can comprise a weight-update vector defined as a latest weight vector minus an initial weight vector before training. A pre-trained neural network can be compressed by pruning one or more small-valued weights. The training of the neural network can consider the compressibility of the neural network, for instance, using a compression loss function, such as a task loss and/or a weight-update compression loss. The compressed neural network can be applied within a decoding loop of an encoder side or in a post-processing stage, as well as at a decoder side.

System, methods, and other embodiments described herein relate to training a depth model for joint depth completion and prediction. In one arrangement, a method includes generating depth features from sparse depth data according to a sparse auxiliary network (SAN) of a depth model. The method includes generating a first depth map from a monocular image and a second depth map from the monocular image and the depth features using the depth model. The method includes generating a depth loss from the second depth map and the sparse depth data and an image loss from the first depth map and the sparse depth data. The method includes updating the depth model including the SAN using the depth loss and the image loss.

System, methods, and other embodiments described herein relate to training a depth model for joint depth completion and prediction. In one arrangement, a method includes generating depth features from sparse depth data according to a sparse auxiliary network (SAN) of a depth model. The method includes generating a first depth map from a monocular image and a second depth map from the monocular image and the depth features using the depth model. The method includes generating a depth loss from the second depth map and the sparse depth data and an image loss from the first depth map and the sparse depth data. The method includes updating the depth model including the SAN using the depth loss and the image loss.