A method and apparatus for video encoding or decoding utilizing adaptive background residual prediction is disclosed. The present invention adaptively applies background residual prediction to a current block based on a selection decision. The coding block is split into one or more coding sub-blocks. A reference sub-block in a reference picture is located for a current coding sub-block of the current coding block according to a motion vector associated with the current coding block. A background reference sub-block in a background picture is located for the reference sub-block, where the background reference sub-block is at a first co-located location as the reference sub-block. The method then selects a first predictor or a second predictor to encode or decode the current sub-block based on a selection decision. The first predictor corresponds to the reference sub-block, and the second predictor is derived according to the reference sub-block and the background picture.

A method and apparatus for video encoding or decoding utilizing adaptive background residual prediction is disclosed. The present invention adaptively applies background residual prediction to a current block based on a selection decision. The coding block is split into one or more coding sub-blocks. A reference sub-block in a reference picture is located for a current coding sub-block of the current coding block according to a motion vector associated with the current coding block. A background reference sub-block in a background picture is located for the reference sub-block, where the background reference sub-block is at a first co-located location as the reference sub-block. The method then selects a first predictor or a second predictor to encode or decode the current sub-block based on a selection decision. The first predictor corresponds to the reference sub-block, and the second predictor is derived according to the reference sub-block and the background picture.

A region determination circuit determines, for a first block encoded by referring to a first prediction block generated by applying a bidirectional prediction mode for a first component of a pixel value from among blocks in a coding-target picture included in video data, a partial region to which a unidirectional prediction mode is to be applied for a second component on the basis of a difference value for the first component between corresponding pixels belonging to the first prediction block and the first block. A prediction circuit generates a second prediction block for the second component by applying a unidirectional prediction mode to the partial region and a bidirectional prediction mode to a region that is not the partial region. An encoder calculates a prediction error for the second component between corresponding pixels belonging to the first block and the second prediction block and encodes the prediction error.

A device for decoding video data is configured to perform interpolation filtering using an N-tap filter to generate an interpolated search space for a first block of video data; obtain a first predictive block in the interpolated search space; determine that a second block of video data is encoded using a bi-directional inter prediction mode and a bi-directional optical flow (BIO) process; perform an inter prediction process for the second block of video data using the bi-directional inter prediction mode to determine a second predictive block; perform the BIO process on the second predictive block to determine a BIO-refined version of the second predictive block, wherein a number of reference samples used for calculating intermediate values for BIO offsets is limited to a region of (W+N1)(H+N1) integer samples, wherein W and H correspond to a width and height of the second block in integer samples.

An image decoding method includes: obtaining, for each of processing units obtained by splitting a current frame, motion vectors assigned to the processing unit; selecting, for each of small regions obtained by splitting a processing unit among the processing units, a motion vector to be used from among the motion vectors assigned to the processing unit, based on the motion vectors and reference frames at different times; generating, for each of the small regions, a predicted image using the motion vector selected for the small region; and decoding each of the small regions using the predicted image generated for the small region.

A motion information calculation unit acquires motion information between a plurality of target images. An occlusion information calculation unit generates occlusion information between the target images. An image interpolation processing unit determines priority of the motion information based on the motion information and the occlusion information, and performs predetermined image processing for the target images by using motion information that is weighted based on the priority.

Systems and methods may provide for occlusion detection in frame rate conversion. Detecting the occlusion allows frame rate conversion to be more accurately performed. In some embodiments, one or more stereoscopic depth cameras may be used to determine the depth of a moving object to more accurately determine the occlusion. In some embodiments, the compression ratio may be adjusted to balance the frame rate and power to help ensure compliance with a power budget. In at least some embodiments, the motion of a camera may be passed from a 3D render pipe to an encoder to avoid motion calculation and thereby saving power.

Devices and methods for generating a three-dimensional video stream starting from a sequence of video images. The sequence includes a first view (V.sub.0), at least one second view (V.sub.1) of a scene, and a depth map (D.sub.0) of said first view (V.sub.0), or a disparity map of said at least one second view (V.sub.1) with respect to the first view (V.sub.0). At least one occlusion image (O.sub.1) including the occluded pixels of said second view (V.sub.1) is obtained by starting from said depth map (D.sub.0) or from said disparity map. A compacted occlusion image (OC.sub.1)is generated by spatially repositioning said occluded pixels of said at least one occlusion image (O.sub.1), so as to move said pixels closer to one another. The three-dimensional video stream may include said first view (V.sub.0), said depth map (D.sub.0) or said disparity map, and said at least one compacted occlusion image (OC.sub.1).

Devices and methods for generating a three-dimensional video stream starting from a sequence of video images. The sequence includes a first view (V.sub.0), at least one second view (V.sub.1) of a scene, and a depth map (D.sub.0) of said first view (V.sub.0), or a disparity map of said at least one second view (V.sub.1) with respect to the first view (V.sub.0). At least one occlusion image (O.sub.1) including the occluded pixels of said second view (V.sub.1) is obtained by starting from said depth map (D.sub.0) or from said disparity map. A compacted occlusion image (OC.sub.1)is generated by spatially repositioning said occluded pixels of said at least one occlusion image (O.sub.1), so as to move said pixels closer to one another. The three-dimensional video stream may include said first view (V.sub.0), said depth map (D.sub.0) or said disparity map, and said at least one compacted occlusion image (OC.sub.1).

One embodiment discloses an image processing apparatus which generates an interpolation frame from consecutive first and second frames, including: a motion estimation section which assigns, to the interpolation frame, motion vectors from the first frame to the second frame; a first degree-of-difference calculation section which extracts calculates a first degree of difference in terms of pixel values; a second degree-of-difference calculation section which calculates a second degree of difference in terms of vectors; and an interpolation frame generation section which generates the interpolation frame by determining a motion vector that should be assigned to a pixel of attention on the basis of combined weights obtained by combining the first degrees of difference and the second degrees of difference, respectively.