Apparatus and methods for implementing a real-time Versatile Video Coding (VVC) decoder use multiple threads to address the limitation with existing parallelization techniques and fully utilizes the available CPU computation resource without compromising on the coding efficiency. The proposed Multi-threaded (MT) framework uses CTU level parallel processing techniques without compromising on the memory bandwidth. Picture level parallel processing separates the sequence into temporal levels by considering the picture's referencing hierarchy. Embodiments are provided using various optimization techniques to achieve real-time VVC decoding on heterogenous platforms with multi-core CPUs, for those bitstreams generated using a VVC reference encoder with a default configuration.

The Cloud-based Video Streaming Service (CVSS) architecture is disclosed to transcode video streams in an on-demand manner. The architecture provides a platform for streaming service providers to utilize cloud resources in a cost-efficient manner and with respect to the Quality of Service (QoS) demands of video streams. In particular, the architecture includes a QoS-aware scheduling method to efficiently map video streams to cloud resources, and a cost-aware dynamic (i.e., elastic) resource provisioning policy that adapts the resource acquisition with respect to the video streaming QoS demands. Simulation results based on realistic cloud traces and with various workload conditions, demonstrate that the CVSS architecture can satisfy video streaming QoS demands and reduces the incurred cost of stream providers up to 70%.

The Cloud-based Video Streaming Service (CVSS) architecture is disclosed to transcode video streams in an on-demand manner. The architecture provides a platform for streaming service providers to utilize cloud resources in a cost-efficient manner and with respect to the Quality of Service (QoS) demands of video streams. In particular, the architecture includes a QoS-aware scheduling method to efficiently map video streams to cloud resources, and a cost-aware dynamic (i.e., elastic) resource provisioning policy that adapts the resource acquisition with respect to the video streaming QoS demands. Simulation results based on realistic cloud traces and with various workload conditions, demonstrate that the CVSS architecture can satisfy video streaming QoS demands and reduces the incurred cost of stream providers up to 70%.

A load balancing method for video decoding. The load balancing includes first determining which hardware devices are suitable for the new decoding process, and determining the current load of each of the suitable hardware devices. From the suitable devices potential devices are selected having a current load less than a threshold and overloaded devices are selected having a load greater than or equal to the threshold. If there are no suitable devices, then the decoding process is implemented by software decoding. If the list of potential hardware devices includes only one potential hardware device, then the decoding process is implemented on the hardware device. If the list of potential hardware devices includes more than one potential hardware device, then it is determined how many decoding processes are currently running on each potential hardware device, and the new decoding process is implemented on the potential hardware device having the fewest processes.

A differential video rendering system, including a graphics processing unit (GPU); a graphical display coupled to the GPU; a video decoder configured to decode a bitstream of encoded data into a plurality of sets of decoded blocks; at least one processor configured to: generate, based on a first set of the plurality of sets of decoded blocks, a first differential video frame comprising a plurality of sets of differential regions, normalize each set of the plurality of sets of differential regions to a fixed size block to provide a normalized plurality of sets of differential regions, map a respective set of the normalized plurality of sets of differential regions to align with a respective tile size region of a plurality of tile size regions conforming with the GPU, generate a hierarchal region tree based on the normalized plurality of sets of differential regions mapped to the plurality of tile size regions, and generate a plurality of optimal regions based on the hierarchal region tree satisfying a predefined criteria corresponding to a pre-defined optimal number of regions and a predefined efficiency parameter; and a graphics rendering engine configured to render the first differential video frame on the graphical display based on the plurality of optimal regions and a group of differential regions.

A differential video rendering system, including a graphics processing unit (GPU); a graphical display coupled to the GPU; a video decoder configured to decode a bitstream of encoded data into a plurality of sets of decoded blocks; at least one processor configured to: generate, based on a first set of the plurality of sets of decoded blocks, a first differential video frame comprising a plurality of sets of differential regions, normalize each set of the plurality of sets of differential regions to a fixed size block to provide a normalized plurality of sets of differential regions, map a respective set of the normalized plurality of sets of differential regions to align with a respective tile size region of a plurality of tile size regions conforming with the GPU, generate a hierarchal region tree based on the normalized plurality of sets of differential regions mapped to the plurality of tile size regions, and generate a plurality of optimal regions based on the hierarchal region tree satisfying a predefined criteria corresponding to a pre-defined optimal number of regions and a predefined efficiency parameter; and a graphics rendering engine configured to render the first differential video frame on the graphical display based on the plurality of optimal regions and a group of differential regions.

This disclosure relates generally to method and system for live video streaming with integrated encoding and transmission semantics. The system receives a set of frames associated with a live video stream encoded to generate a set of data fragments using a reference encoder and a delta encoder. Transmitter unit of the live video streaming protocol transmits each packet of the set of full frames and the set of delta frames in sequence with a payload specific header based on a packet mode. Further, the receiver unit receives each packet of the full frames and each packet of the delta frames based on the packet mode to reconstruct an original sequence from the foreground pixels by estimating a total number of packets expected at each frame interval and loss incurred in each packet of the set of full frames and the set of delta frames.

A video processing method is provided to include performing a conversion between a video including a plurality of pictures each including at least one slice and a bitstream of the video, wherein the bitstream conforms to a format rule; wherein the format rule specifies that a first flag in the bitstream indicates whether a picture header syntax structure of a picture is included in a slice header of a slice of the picture; and wherein the format rule specifies that, responsive to any of six flags in a picture parameter set associated with the picture is equal to 1, the first flag is equal to 0.

This disclosure describes systems, methods, and devices related to bit-rate-based hybrid encoding. A device may generate a list of encodes based on pairs of resolution and quantization parameters (QP) pairs associated with one or more video segments received from a source. The device may generate an estimated bit rate associated with the one or more video segments based on an analysis of the one or more video segments. The device may compare the estimated bit rate to a threshold. The device may switch between a software encoder and a hardware encoder based on the comparison of the estimated bit rate to the threshold. The device may encode each of the one or more video segments for transmission using the hardware encoder or the software encoder.

This disclosure describes systems, methods, and devices related to bit-rate-based hybrid encoding. A device may generate a list of encodes based on pairs of resolution and quantization parameters (QP) pairs associated with one or more video segments received from a source. The device may generate an estimated bit rate associated with the one or more video segments based on an analysis of the one or more video segments. The device may compare the estimated bit rate to a threshold. The device may switch between a software encoder and a hardware encoder based on the comparison of the estimated bit rate to the threshold. The device may encode each of the one or more video segments for transmission using the hardware encoder or the software encoder.