A head mounted display system with video-see-through (VST) is taught. The system and method process video images captured by at least two forward facing video cameras mounted to the headset to produce generated images whose viewpoints correspond to the viewpoint of the user if the user was not wearing the display system. By generating VST images which have viewpoints corresponding to the user's viewpoint, errors in sizing, distances and positions of objects in the VST images are prevented.

A head mounted display system with video-see-through (VST) is taught. The system and method process video images captured by at least two forward facing video cameras mounted to the headset to produce generated images whose viewpoints correspond to the viewpoint of the user if the user was not wearing the display system. By generating VST images which have viewpoints corresponding to the user's viewpoint, errors in sizing, distances and positions of objects in the VST images are prevented.

A method of transferring data can comprise dividing an experience space into polyhedrons comprising a network of interconnected planes. Sheets of experience space data of the experience space can be collected via a distributed sensor array. Each sensor of the distributed sensor array can comprise a corresponding sheet of experience space data. One of the polyhedrons can be selected as a client location polyhedron. Client location data can be extracted from the experience space data. The client location data can correspond to the client location polyhedron. Extraction can be by intersecting ones of the sheets of experience space data to form a polyhedral data set.

A method of transferring data can comprise dividing an experience space into polyhedrons comprising a network of interconnected planes. Sheets of experience space data of the experience space can be collected via a distributed sensor array. Each sensor of the distributed sensor array can comprise a corresponding sheet of experience space data. One of the polyhedrons can be selected as a client location polyhedron. Client location data can be extracted from the experience space data. The client location data can correspond to the client location polyhedron. Extraction can be by intersecting ones of the sheets of experience space data to form a polyhedral data set.

Techniques for aligning images generated by an integrated camera physically mounted to an HMD with images generated by a detached camera physically unmounted from the HMD are disclosed. A 3D feature map is generated and shared with the detached camera. Both the integrated camera and the detached camera use the 3D feature map to relocalize themselves and to determine their respective 6 DOF poses. The HMD receives the detached camera's image of the environment and the 6 DOF pose of the detached camera. A depth map of the environment is accessed. An overlaid image is generated by reprojecting a perspective of the detached camera's image to align with a perspective of the integrated camera and by overlaying the reprojected detached camera's image onto the integrated camera's image.

A multi-viewpoint 3D display method is provided, comprising: obtaining a distance between a user and a multi-viewpoint 3D display screen; and dynamically rendering subpixels in composite subpixels in the multi-viewpoint 3D display screen based on 3D signals in response to a change in a distance. The method can implement flexible projection from multiple viewpoints. A multi-viewpoint 3D display device, a computer readable storage medium, and a computer program product are also provided.

Provided is an information processing device that performs processing on a content. An information processing device is provided with an estimation unit that estimates sounding coordinates at which a sound image is generated on the basis of a video stream and an audio stream, a video output control unit that controls an output of the video stream, and an audio output control unit that controls an output of the audio stream so as to generate the sound image at the sounding coordinates. A discrimination unit that discriminates a gazing point of a user who views video and audio is further provided, in which the estimation unit estimates the sounding coordinates at which the sound image of the object gazed by the user is generated on the basis of a discrimination result.

Provided is an information processing device that performs processing on a content. An information processing device is provided with an estimation unit that estimates sounding coordinates at which a sound image is generated on the basis of a video stream and an audio stream, a video output control unit that controls an output of the video stream, and an audio output control unit that controls an output of the audio stream so as to generate the sound image at the sounding coordinates. A discrimination unit that discriminates a gazing point of a user who views video and audio is further provided, in which the estimation unit estimates the sounding coordinates at which the sound image of the object gazed by the user is generated on the basis of a discrimination result.

Systems and methods are provided for generating a viewport for display. A user preference for a character and/or a genre of a scene in a spherical media content item is determined, wherein the spherical media content item comprises a plurality of tiles. A tile of the plurality of tiles is identified based on the determined user preference. A viewport to be generated for display at a computing device is predicted, based on the identified tile. A first tile to be transmitted to a computing device at a first resolution is identified, based on the predicted viewport to be generated for display. The tile is transmitted, to the computing device, at the first resolution.

Systems and methods are provided for generating a viewport for display. A user preference for a character and/or a genre of a scene in a spherical media content item is determined, wherein the spherical media content item comprises a plurality of tiles. A tile of the plurality of tiles is identified based on the determined user preference. A viewport to be generated for display at a computing device is predicted, based on the identified tile. A first tile to be transmitted to a computing device at a first resolution is identified, based on the predicted viewport to be generated for display. The tile is transmitted, to the computing device, at the first resolution.