A method of generating a 3D reconstruction of a scene, the scene comprising a plurality of cameras positioned around the scene, comprises: obtaining the extrinsics and intrinsics of a virtual camera within a scene; accessing a data structure so as to determine a camera pair that is to be used in reconstructing the scene from the viewpoint of the virtual camera; wherein the data structure defines a voxel representation of the scene, the voxel representation comprising a plurality of voxels, at least some of the voxel surfaces being associated with respective camera pair identifiers; wherein each camera pair identifier associated with a respective voxel surface corresponds to a camera pair that has been identified as being suitable for obtaining depth data for the part of the scene within that voxel and for which the averaged pose of the camera pair is oriented towards the voxel surface; identifying, based on the obtained extrinsics and intrinsics of the virtual camera, at least one voxel that is within the field of view of the virtual camera and a corresponding voxel surface that is oriented towards the virtual camera; identifying, based on the accessed data structure, at least one camera pair that is suitable for reconstructing the scene from the viewpoint of the virtual camera, and generating a reconstruction of the scene from the viewpoint of the virtual camera based on the images captured by the cameras in the identified at least one camera pair.

A method of generating a 3D reconstruction of a scene, the scene comprising a plurality of cameras positioned around the scene, comprises: obtaining the extrinsics and intrinsics of a virtual camera within a scene; accessing a data structure so as to determine a camera pair that is to be used in reconstructing the scene from the viewpoint of the virtual camera; wherein the data structure defines a voxel representation of the scene, the voxel representation comprising a plurality of voxels, at least some of the voxel surfaces being associated with respective camera pair identifiers; wherein each camera pair identifier associated with a respective voxel surface corresponds to a camera pair that has been identified as being suitable for obtaining depth data for the part of the scene within that voxel and for which the averaged pose of the camera pair is oriented towards the voxel surface; identifying, based on the obtained extrinsics and intrinsics of the virtual camera, at least one voxel that is within the field of view of the virtual camera and a corresponding voxel surface that is oriented towards the virtual camera; identifying, based on the accessed data structure, at least one camera pair that is suitable for reconstructing the scene from the viewpoint of the virtual camera, and generating a reconstruction of the scene from the viewpoint of the virtual camera based on the images captured by the cameras in the identified at least one camera pair.

Described are systems and processes for generating multi-view interactive digital media representations (MIDMR) for display on a user device. In one aspect, a mobile device is provided which comprises a display, one or more processors, memory, and one or more programs stored in memory. The one or more programs comprise instructions for locking the mobile device, and providing a lock screen on the display in a lock mode upon receiving user input for accessing the mobile device. The lock screen may display a dynamic MIDMR that dynamically changes without user input, which provides an interactive three-dimensional representation of an object that is responsive to user interaction with the mobile device. The dynamic MIDMR displayed is selected based on predetermined criteria, and may change based on a predetermined algorithm that includes weighted predetermined criteria factors as variables and recalculates the algorithm value to determine changes to the dynamic MIDMR.

A method and apparatus for performing a single view depth and texture calibration are described. In one embodiment, the apparatus comprises a calibration unit operable to perform a single view calibration process using a captured single view a target having a plurality of plane geometries having detectable features and being at a single orientation and to generate calibration parameters to calibrate one or more of the projector and multiple cameras using the single view of the target.

A user equipment, media control unit, media resource function, or another device or function capable of receiving, manipulating, and transmitting data may be configured to: receive an omnidirectional video; determine a viewport of a user equipment; determining a delivery mode; determine a region of the omnidirectional video based, at least partially, on the determined viewport and the determined delivery mode; encode the determined region; packetizing the encoded region; and transmit the packetized region to the user equipment based, at least partially, on the determined delivery mode.

A user equipment, media control unit, media resource function, or another device or function capable of receiving, manipulating, and transmitting data may be configured to: receive an omnidirectional video; determine a viewport of a user equipment; determining a delivery mode; determine a region of the omnidirectional video based, at least partially, on the determined viewport and the determined delivery mode; encode the determined region; packetizing the encoded region; and transmit the packetized region to the user equipment based, at least partially, on the determined delivery mode.

A process for coding an image of a view from among a plurality of views, including the following steps: selecting a first or a second coding method to code image data from the image; generating a data signal containing information indicating whether it is the first or the second coding method that has been selected, and, if it is the first coding method, coding the original image data so as to provide coded original data, and, if it is the second coding method, coding processed image data from the image obtained by image processing of the original image data so as to provide coded processed data; and coding information describing the image processing which has been applied.

There is provided an image processing device that processes a projection image presented to a plurality of persons at the same time. The image processing device specifies an overlapping area in which fields of view of two or more users overlap based on information on each user, classifies objects included in the overlapping area into a first object group and a second object group, generates a common image common to all users, made up of the first object group, generates individual images different for each user, made up of the second object group, and determines an output protocol for displaying the individual images.

There is provided an image processing device that processes a projection image presented to a plurality of persons at the same time. The image processing device specifies an overlapping area in which fields of view of two or more users overlap based on information on each user, classifies objects included in the overlapping area into a first object group and a second object group, generates a common image common to all users, made up of the first object group, generates individual images different for each user, made up of the second object group, and determines an output protocol for displaying the individual images.

In one embodiment, a method includes accessing first image data generated by a first image sensor having a first filter array that has a first filter pattern. The first filter pattern includes a number of first filter types. The method also includes accessing second image data generated by a second image sensor having a second filter array that has a second filter pattern different from the first filter pattern. The second filter pattern includes a number of second filter types, the number of second filter types and the number of first filter types have at least one filter type in common. The method also includes determining a correspondence between one or more first pixels of the first image data and one or more second pixels of the second image data based on a portion of the first image data associated with the filter type in common.