Methods and apparatus, including computer program products, for controlling a Pan-Tilt-Zoom (PTZ) camera. A video stream is provided by the PTZ camera. The video stream includes a main image and a smaller picture-in-picture overlay. The main image and the overlay show the same view. The overlay has both a higher frame rate and a lower resolution compared to the main image. A movement control instruction is received, by the PTZ camera, in response to an evaluation of at least one of the main image and the overlay.

A method for encoding including executing game logic built on a game engine of a video game at a cloud gaming server to generate video frames. The method including executing scene change logic to predict a scene change in the video frames based on game state collected during execution of the game logic. The method including identifying a range of video frames that is predicted to include the scene change. The method including generating a scene change hint using the scene change logic, wherein the scene change hint identifies the range of video frames, wherein the range of video frames includes a first video frame. The method including delivering the first video frame to an encoder. The method including sending the scene change hint from the scene change logic to the encoder. The method including encoding the first video frame as an I-frame based on the scene change hint.

A method for encoding including executing game logic built on a game engine of a video game at a cloud gaming server to generate video frames. The method including executing scene change logic to predict a scene change in the video frames based on game state collected during execution of the game logic. The method including identifying a range of video frames that is predicted to include the scene change. The method including generating a scene change hint using the scene change logic, wherein the scene change hint identifies the range of video frames, wherein the range of video frames includes a first video frame. The method including delivering the first video frame to an encoder. The method including sending the scene change hint from the scene change logic to the encoder. The method including encoding the first video frame as an I-frame based on the scene change hint.

Methods and apparatus are described for delivering streams of media content in ways that smooth out the peaks that might otherwise occur due to the bit rate variations that result from encoding of the media content. This is accomplished by controlling the timing of the transmission of packets of the encoded media content.

Methods and apparatus are described for delivering streams of media content in ways that smooth out the peaks that might otherwise occur due to the bit rate variations that result from encoding of the media content. This is accomplished by controlling the timing of the transmission of packets of the encoded media content.

Techniques for encoding image data are discussed. Image data containing multiple image frames can be received from an image sensor associated with a vehicle. A first image frame and a second image frame may be associated with a first data chunk. A first metadata associated with the first image frame can be identified. Additionally, a second metadata associated with the second image frame can be identified. The first data chunk can be created that includes the first image frame, the second image frame, the first metadata, and the second metadata. The first data chunk can be stored in a video container, where the video container can be indexed to access the first image frame with the first metadata or the second image frame with the second metadata.

A method for developing an enhancement model for low-quality visual data, the method comprising the steps of receiving one or more sections of higher-quality visual data; and training a hierarchical algorithm. The hierarchical algorithm is operable to increase the quality of one or more sections of lower-quality visual data so as to substantially reproduce the one or more sections of higher-quality visual data. The hierarchical algorithm is then outputted.

A method for developing an enhancement model for low-quality visual data, the method comprising the steps of receiving one or more sections of higher-quality visual data; and training a hierarchical algorithm. The hierarchical algorithm is operable to increase the quality of one or more sections of lower-quality visual data so as to substantially reproduce the one or more sections of higher-quality visual data. The hierarchical algorithm is then outputted.

One or more multi-stage optimization iterations are performed with respect to a compression algorithm. A given iteration comprises a first stage in which hyper-parameters of a perceptual quality algorithm are tuned independently of the compression algorithm. A second stage of the iteration comprises tuning hyper-parameters of the compression algorithm using a set of perceptual quality scores generated by the tuned perceptual quality algorithm. The final stage of the iteration comprises performing a compression quality evaluation test on the tuned compression algorithm.

Described herein are video streaming techniques for applications and workloads executed in the cloud. In one example, the cloud server device encodes display frames using low-delay encoding techniques for transmission to a client device. The cloud server device receives an overlay bitstream from a client device, combines the overlay data with the display frames, and encodes the frames for the viewers using statistics from the display frames encoded for the client device and/or from the overlay data. The cloud server device can then transmit the bitstream to a third device for viewing (e.g., to a viewer device or a streaming server device).