Techniques and tools for reducing latency in video encoding and decoding by constraining latency due to reordering of video frames, and by indicating the constraint on frame reordering latency with one or more syntax elements that accompany encoded data for the video frames. For example, a real-time communication tool with a video encoder sets a syntax element that indicates a constraint on frame reordering latency, which is consistent with inter-frame dependencies between multiple frames of a video sequence, then outputs the syntax element. A corresponding real-time communication tool with a video decoder receives the syntax element that indicates the constraint on frame reordering latency, determines the constraint on frame reordering latency based on the syntax element, and uses the constraint on frame reordering latency to determine when a reconstructed frame is ready for output (in terms of output order).

A moving picture coding method includes: making a determination as to whether or not to code all blocks in a current picture in the skip mode; setting, based on a result of the determination, a first flag indicating whether or not a temporally neighboring block is to be referenced, a value of a parameter for determining a total number of merging candidates, and a second flag for each block included in the current picture, the second flag indicating whether or not the block is to be coded in the skip mode; calculating, as a merging candidate, a neighboring block usable for merging; and coding an index which indicates a merging candidate to be used for coding of the current block and attaching the coded index to a bitstream.

A moving picture coding method includes: making a determination as to whether or not to code all blocks in a current picture in the skip mode; setting, based on a result of the determination, a first flag indicating whether or not a temporally neighboring block is to be referenced, a value of a parameter for determining a total number of merging candidates, and a second flag for each block included in the current picture, the second flag indicating whether or not the block is to be coded in the skip mode; calculating, as a merging candidate, a neighboring block usable for merging; and coding an index which indicates a merging candidate to be used for coding of the current block and attaching the coded index to a bitstream.

A video server is configured to provide streaming video to players of computer games over a computing network. The video server can provided video of different games to different players simultaneously. This is accomplished by rendering several video streams in parallel using a single GPU (Graphics Processing Unit). The output of the GPU is provided to graphics processing pipelines that are each associated with a specific client/player and are dynamically allocated as needed. A client qualifier may be used to assure that only clients capable of presenting the streaming video to a player at a minimum level of quality receive the video stream. Video frames provided by the video server optionally include overlays added to the output of the GPU. These overlays can include voice data received from another game player.

A video server is configured to provide streaming video to players of computer games over a computing network. The video server can provided video of different games to different players simultaneously. This is accomplished by rendering several video streams in parallel using a single GPU (Graphics Processing Unit). The output of the GPU is provided to graphics processing pipelines that are each associated with a specific client/player and are dynamically allocated as needed. A client qualifier may be used to assure that only clients capable of presenting the streaming video to a player at a minimum level of quality receive the video stream. Video frames provided by the video server optionally include overlays added to the output of the GPU. These overlays can include voice data received from another game player.

An image coding method of hierarchically coding a plurality of pictures to generate a bitstream, includes: coding each of the plurality of pictures, which belongs to any one of a plurality of hierarchical layers, with reference to a picture belonging to a hierarchical layer which is same as or lower than a hierarchical layer to which the picture belongs, and without reference to a picture belonging to a hierarchical layer which is higher than the hierarchical layer to which the picture belongs; and generating the bitstream by coding the coded pictures and time information indicating decoding times of the coded pictures. The time information indicates that the decoding times are set at equal intervals for low-layer pictures which are the plurality of pictures other than highest-layer pictures belonging to a highest layer among the plurality of hierarchical layers.

An image coding method of hierarchically coding a plurality of pictures to generate a bitstream, includes: coding each of the plurality of pictures, which belongs to any one of a plurality of hierarchical layers, with reference to a picture belonging to a hierarchical layer which is same as or lower than a hierarchical layer to which the picture belongs, and without reference to a picture belonging to a hierarchical layer which is higher than the hierarchical layer to which the picture belongs; and generating the bitstream by coding the coded pictures and time information indicating decoding times of the coded pictures. The time information indicates that the decoding times are set at equal intervals for low-layer pictures which are the plurality of pictures other than highest-layer pictures belonging to a highest layer among the plurality of hierarchical layers.

Particular embodiments provide a variable, BitDepth, that may be set at a value based on a number of bits used to represent pixels in pictures of a video. The variable may be used in syntax elements in HEVC, such as the HEVC range extension, but other coding standards may be used. By using the variable, different resolutions for the video may be accommodated during the encoding and decoding process. For example, the number of pixels in the pictures may be represented by 8 bits, 10 bits, 12 bits, or another number of bits depending on the resolution. Using the BitDepth variable in the syntax provides flexibility in the motion estimation and motion compensation process. For example, syntax elements used in the weighted prediction process may take into account different numbers of bits used to represent the pictures.

Particular embodiments provide a variable, BitDepth, that may be set at a value based on a number of bits used to represent pixels in pictures of a video. The variable may be used in syntax elements in HEVC, such as the HEVC range extension, but other coding standards may be used. By using the variable, different resolutions for the video may be accommodated during the encoding and decoding process. For example, the number of pixels in the pictures may be represented by 8 bits, 10 bits, 12 bits, or another number of bits depending on the resolution. Using the BitDepth variable in the syntax provides flexibility in the motion estimation and motion compensation process. For example, syntax elements used in the weighted prediction process may take into account different numbers of bits used to represent the pictures.

In one embodiment, an apparatus comprises a memory and a processor. The memory is to store visual data associated with a visual representation captured by one or more sensors. The processor is to: obtain the visual data associated with the visual representation captured by the one or more sensors, wherein the visual data comprises uncompressed visual data or compressed visual data; process the visual data using a convolutional neural network (CNN), wherein the CNN comprises a plurality of layers, wherein the plurality of layers comprises a plurality of filters, and wherein the plurality of filters comprises one or more pixel-domain filters to perform processing associated with uncompressed data and one or more compressed-domain filters to perform processing associated with compressed data; and classify the visual data based on an output of the CNN.