Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. A video summary can be generated including one or more of the identified best scenes. The video summary can be generated using a video summary template with slots corresponding to video clips selected from among sets of candidate video clips. Best scenes can also be identified by receiving an indication of an event of interest within video from a user during the capture of the video. Metadata patterns representing activities identified within video clips can be identified within other videos, which can subsequently be associated with the identified activities.

A display control method with which a captioned image can be displayed in the visual field of a user irrespective of a change in the line-of-sight direction of the user. The display control method includes a step of generating visual-field image data, which represents a visual-field image, based on the line-of-sight direction of a user and 360-degree space image data. The method further includes a step of receiving captioned image data, which represents a captioned image. The method further includes a step of displaying on an HMD a composite image of the visual-field image and from the captioned image, based on the visual-field image data and the captioned image data. The composite image is displayed is displayed at a given place in the visual-field image irrespective of a change in the line-of-sight direction of the user.

Image reconstruction methods, systems, devices, and computer-readable storage media are provided. The method includes: acquiring a multi-angle free-perspective image combination, parameter data of the image combination, and virtual viewpoint position information based on user interaction, where the image combination includes multiple groups of texture images and depth maps that are synchronized at multiple angles and have corresponding relationships; selecting a corresponding group of texture images and depth maps in the image combination at a user interaction moment based on a preset rule according to the virtual viewpoint position information and the parameter data of the image combination; and combining and rendering the selected corresponding group of texture images and depth maps in the image combination at the user interaction moment based on the virtual viewpoint position information and parameter data corresponding to the corresponding group of texture images and depth maps in the image combination at the user interaction moment.

Image reconstruction methods, systems, devices, and computer-readable storage media are provided. The method includes: acquiring a multi-angle free-perspective image combination, parameter data of the image combination, and virtual viewpoint position information based on user interaction, where the image combination includes multiple groups of texture images and depth maps that are synchronized at multiple angles and have corresponding relationships; selecting a corresponding group of texture images and depth maps in the image combination at a user interaction moment based on a preset rule according to the virtual viewpoint position information and the parameter data of the image combination; and combining and rendering the selected corresponding group of texture images and depth maps in the image combination at the user interaction moment based on the virtual viewpoint position information and parameter data corresponding to the corresponding group of texture images and depth maps in the image combination at the user interaction moment.

A method, a non-transitory computer readable medium, and a computer system is provided for handling video streams. The method may include: receiving a 360° video stream, the 360° video stream including a multiple overlay parameter; when the multiple overlay parameter is a first value, determining that overlaying the 360° video stream with two or more overlays is permissible by the user terminal, and displaying, by the user terminal, the 360° video stream with the two or more overlays; and when the multiple overlay parameter is a second value, determining that overlaying the 360° video stream with two or more overlays is not permissible and displaying the 360° video stream with zero overlays or one overlay.

A method, a non-transitory computer readable medium, and a computer system is provided for handling video streams. The method may include: receiving a 360° video stream, the 360° video stream including a multiple overlay parameter; when the multiple overlay parameter is a first value, determining that overlaying the 360° video stream with two or more overlays is permissible by the user terminal, and displaying, by the user terminal, the 360° video stream with the two or more overlays; and when the multiple overlay parameter is a second value, determining that overlaying the 360° video stream with two or more overlays is not permissible and displaying the 360° video stream with zero overlays or one overlay.

A video data transmission unit transmits data generated by a video data generation unit to an HMD. A sensing data reception unit receives data transmitted from the HMD, the data including size data indicating a size of the data successfully received by the HMD. A required time specification unit specifies time it took for a round-trip communication in half-duplex wireless communication between a relay apparatus and the HMD. A generation control unit controls a size of the data generated by the video data generation unit per unit time based on the specified time, the size indicated in the size data, and a given target time regarding the round-trip communication.

A video data transmission unit transmits data generated by a video data generation unit to an HMD. A sensing data reception unit receives data transmitted from the HMD, the data including size data indicating a size of the data successfully received by the HMD. A required time specification unit specifies time it took for a round-trip communication in half-duplex wireless communication between a relay apparatus and the HMD. A generation control unit controls a size of the data generated by the video data generation unit per unit time based on the specified time, the size indicated in the size data, and a given target time regarding the round-trip communication.

A file-processing apparatus 20 acquires (S104) access information that is used for requesting material data for generating a virtual viewpoint image that is generated based on a virtual viewpoint and captured images acquired from a plurality of imaging apparatuses, generates (S107) a virtual viewpoint image file that contains the acquired access information and a virtual viewpoint image data corresponding to the virtual viewpoint image, and outputs (S108) the generated virtual viewpoint image file.

A file-processing apparatus 20 acquires (S104) access information that is used for requesting material data for generating a virtual viewpoint image that is generated based on a virtual viewpoint and captured images acquired from a plurality of imaging apparatuses, generates (S107) a virtual viewpoint image file that contains the acquired access information and a virtual viewpoint image data corresponding to the virtual viewpoint image, and outputs (S108) the generated virtual viewpoint image file.