A computer-implemented method of encoding video streams for low-bandwidth transmissions includes identifying a salient data and a non-salient data in a high-resolution video stream. The salient data and the non-salient data is segmented. The non-salient data is compressed to a lower resolution. The salient data and the compressed non-salient data are transmitted in a low-bandwidth transmission.

A computer-implemented method of encoding video streams for low-bandwidth transmissions includes identifying a salient data and a non-salient data in a high-resolution video stream. The salient data and the non-salient data is segmented. The non-salient data is compressed to a lower resolution. The salient data and the compressed non-salient data are transmitted in a low-bandwidth transmission.

Highly accurate shape registration processing is performed. In the image processing apparatus, data, which correspond to N frames (N is an integer not less than 2), in units of frames including a plurality of three-dimensional models per frame is acquired. Then, shape registration processing is performed for the acquired three-dimensional models corresponding to the N frames by using information indicating a correspondence relationship of the three-dimensional models between frames of the N frames.

Systems and methods for gaze-driven recording of video are described. Some implementations may include accessing gaze data captured using one or more gaze-tracking sensors; applying a temporal filter to the gaze data to obtain a smoothed gaze estimate; determining a region of interest based on the smoothed gaze estimate, wherein the region of interest identifies a subset of a field of view; accessing a frame of video; recording a portion of the frame associated with the region of interest as an enhanced frame of video, wherein the portion of the frame corresponds to a smaller field of view than the frame; and storing, transmitting, or displaying the enhanced frame of video.

Systems and methods for gaze-driven recording of video are described. Some implementations may include accessing gaze data captured using one or more gaze-tracking sensors; applying a temporal filter to the gaze data to obtain a smoothed gaze estimate; determining a region of interest based on the smoothed gaze estimate, wherein the region of interest identifies a subset of a field of view; accessing a frame of video; recording a portion of the frame associated with the region of interest as an enhanced frame of video, wherein the portion of the frame corresponds to a smaller field of view than the frame; and storing, transmitting, or displaying the enhanced frame of video.

Computer processor hardware receives a first set of adjustment values. The first set of adjustment values specify adjustments to be made to a predicted rendition of a signal generated at a first level of quality to reconstruct a rendition of the signal at the first level of quality. The computer processor hardware processes the first set of adjustment values and derives a second set of adjustment values based on the first set of adjustment values and a rendition of the signal at a second level of quality. The second level of quality is lower than the first level of quality.

Computer processor hardware receives a first set of adjustment values. The first set of adjustment values specify adjustments to be made to a predicted rendition of a signal generated at a first level of quality to reconstruct a rendition of the signal at the first level of quality. The computer processor hardware processes the first set of adjustment values and derives a second set of adjustment values based on the first set of adjustment values and a rendition of the signal at a second level of quality. The second level of quality is lower than the first level of quality.

Innovations in flexible reference picture management are described. For example, a video encoder and video decoder use a global reference picture set (“GRPS”) of reference pictures that remain in memory, and hence are available for use in video encoding/decoding, longer than conventional reference pictures. In particular, reference pictures of the GRPS remain available across random access boundaries. Or, as another example, a video encoder and video decoder clip a reference picture so that useful regions of the reference picture are retained in memory, while unhelpful or redundant regions of the reference picture are discarded. Reference picture clipping can reduce the amount of memory needed to store reference pictures or improve the utilization of available memory by providing better options for motion compensation. Or, as still another example, a video encoder and video decoder filter a reference picture to remove random noise (e.g., capture noise due to camera imperfections during capture).

Innovations in flexible reference picture management are described. For example, a video encoder and video decoder use a global reference picture set (“GRPS”) of reference pictures that remain in memory, and hence are available for use in video encoding/decoding, longer than conventional reference pictures. In particular, reference pictures of the GRPS remain available across random access boundaries. Or, as another example, a video encoder and video decoder clip a reference picture so that useful regions of the reference picture are retained in memory, while unhelpful or redundant regions of the reference picture are discarded. Reference picture clipping can reduce the amount of memory needed to store reference pictures or improve the utilization of available memory by providing better options for motion compensation. Or, as still another example, a video encoder and video decoder filter a reference picture to remove random noise (e.g., capture noise due to camera imperfections during capture).

Video coding using constructed reference frames may include generating, by a processor in response to instructions stored on a non-transitory computer readable medium, a reconstructed video. Generating the reconstructed video may include receiving an encoded bitstream. Video coding using constructed reference frames may include generating a reconstructed non-showable reference frame. Generating the reconstructed non-showable reference frame may include decoding a first encoded frame from the encoded bitstream. Video coding using constructed reference frames may include generating a reconstructed frame. Generating the reconstructed frame may include decoding a second encoded frame from the encoded bitstream using the reconstructed non-showable reference frame as a reference frame. Video coding using constructed reference frames may include including the reconstructed frame in the reconstructed video and outputting the reconstructed video.