Based on viewing tracking data, a viewer's view direction to a three-dimensional (3D) scene depicted by a first video image is determined. The first video image has been streamed in a video stream to the streaming client device before the first time point and rendered with the streaming client device to the viewer at the first time point. Based on the viewer's view direction, a target view portion is identified in a second video image to be streamed in the video stream to the streaming client device to be rendered at a second time point subsequent to the first time point. The target view portion is encoded into the video stream with a higher target spatiotemporal resolution than that used to encode remaining non-target view portions in the second video image.

Based on viewing tracking data, a viewer's view direction to a three-dimensional (3D) scene depicted by a first video image is determined. The first video image has been streamed in a video stream to the streaming client device before the first time point and rendered with the streaming client device to the viewer at the first time point. Based on the viewer's view direction, a target view portion is identified in a second video image to be streamed in the video stream to the streaming client device to be rendered at a second time point subsequent to the first time point. The target view portion is encoded into the video stream with a higher target spatiotemporal resolution than that used to encode remaining non-target view portions in the second video image.

Provided is a 360-degree image data processing method performed by a 360-degree video reception apparatus. The method includes receiving 360-degree image data, obtaining information on an encoded picture and metadata from the 360-degree image data, decoding a picture based on the information on the encoded picture, rendering the decoded picture and an overlay based on the metadata, in which the metadata includes overlay related metadata, the overlay is rendered based on the overlay related metadata, the overlay related metadata includes information on an alpha plane of the overlay, and the information on the alpha plane of the overlay is included in a image item or a video track.

Provided is a 360-degree image data processing method performed by a 360-degree video reception apparatus. The method includes receiving 360-degree image data, obtaining information on an encoded picture and metadata from the 360-degree image data, decoding a picture based on the information on the encoded picture, rendering the decoded picture and an overlay based on the metadata, in which the metadata includes overlay related metadata, the overlay is rendered based on the overlay related metadata, the overlay related metadata includes information on an alpha plane of the overlay, and the information on the alpha plane of the overlay is included in a image item or a video track.

An image processing apparatus that performs processing concerning the display of a stereoscopic image so as to improve the convenience of a user in viewing a stereoscopic image. A head mount display displays a stereoscopic image using image data including a plurality of images having different viewpoints. The image data is processed based on metadata attached to the image data. In a case where information indicating that the image data is associated with a file format which cannot cause the head mount display to perform display is included in the metadata, the image data is converted into a file format which can cause the head mount display to perform display.

An image processing apparatus that performs processing concerning the display of a stereoscopic image so as to improve the convenience of a user in viewing a stereoscopic image. A head mount display displays a stereoscopic image using image data including a plurality of images having different viewpoints. The image data is processed based on metadata attached to the image data. In a case where information indicating that the image data is associated with a file format which cannot cause the head mount display to perform display is included in the metadata, the image data is converted into a file format which can cause the head mount display to perform display.

An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

An apparatus for receiving a video according to embodiments of the present invention comprises a decoder configured to decode bitstream based on viewing position and viewport information; an unpacker configured to unpack pictures in the decoded bitstream; a view regenerator configured to perform view regenerating the unpacked pictures; and a view synthesizer configured to perform view synthesis on the view regenerated pictures. A method of transmitting a video, the method comprising inter-view redundancy removing pictures for multiple viewing positions; packing the inter-view redundancy removed pictures; and encoding the packed pictures and signaling information.

An apparatus for receiving a video according to embodiments of the present invention comprises a decoder configured to decode bitstream based on viewing position and viewport information; an unpacker configured to unpack pictures in the decoded bitstream; a view regenerator configured to perform view regenerating the unpacked pictures; and a view synthesizer configured to perform view synthesis on the view regenerated pictures. A method of transmitting a video, the method comprising inter-view redundancy removing pictures for multiple viewing positions; packing the inter-view redundancy removed pictures; and encoding the packed pictures and signaling information.