A method implemented by a video coding apparatus. The method includes applying a neural network (NN) filter to an unfiltered sample of a video unit to generate a filtered sample, where the NN filter includes an NN filter model generated based on partitioning information of the video unit; and performing a conversion between a video media file and a bitstream based on the filtered sample.

A method implemented by a video coding apparatus. The method includes applying a neural network (NN) filter to an unfiltered sample of a video unit to generate a filtered sample, where the NN filter includes an NN filter model generated based on partitioning information of the video unit; and performing a conversion between a video media file and a bitstream based on the filtered sample.

A disclosed system may include a hardware distortion data pipeline that may include (1) a quantization module that generates a quantized data set, (2) an inverse quantization module that generates, from the quantized data set, an inverse quantized data set by executing an inverse quantization of the quantized data set, and (3) an inverse transformation module that generates an inversely transformed data set by executing an inverse transformation of the inverse quantized data set. The system may also include a hardware determination pipeline that determines a distortion metric based on the inversely transformed data set and the residual frame data set, and a hardware token rate pipeline that determines, based on the quantized data set, a token rate for an encoding of the residual frame data set via a video encoding pipeline. Various other methods, systems, and computer-readable media are also disclosed.

A disclosed system may include a hardware distortion data pipeline that may include (1) a quantization module that generates a quantized data set, (2) an inverse quantization module that generates, from the quantized data set, an inverse quantized data set by executing an inverse quantization of the quantized data set, and (3) an inverse transformation module that generates an inversely transformed data set by executing an inverse transformation of the inverse quantized data set. The system may also include a hardware determination pipeline that determines a distortion metric based on the inversely transformed data set and the residual frame data set, and a hardware token rate pipeline that determines, based on the quantized data set, a token rate for an encoding of the residual frame data set via a video encoding pipeline. Various other methods, systems, and computer-readable media are also disclosed.

A computer implemented method can decode a frame of video data comprising an array of pixels to obtain decoded luma values and decoded chroma values corresponding to the array of pixels, and extract a frame mask based on the decoded luma values. The frame mask can include an array of mask values respectively corresponding to the array of pixels. A mask value indicates whether a corresponding pixel is in foreground or background of the frame. The method can perform a morphological operation to the frame mask to change one or more mask values to indicate their corresponding pixels are removed from the foreground and added to the background of the frame. The method can also identify foreground pixels after performing the morphological operation to the frame mask, and render a foreground image for display based on the decoded luma values and decoded chroma values of the foreground pixels.

A computer implemented method can decode a frame of video data comprising an array of pixels to obtain decoded luma values and decoded chroma values corresponding to the array of pixels, and extract a frame mask based on the decoded luma values. The frame mask can include an array of mask values respectively corresponding to the array of pixels. A mask value indicates whether a corresponding pixel is in foreground or background of the frame. The method can perform a morphological operation to the frame mask to change one or more mask values to indicate their corresponding pixels are removed from the foreground and added to the background of the frame. The method can also identify foreground pixels after performing the morphological operation to the frame mask, and render a foreground image for display based on the decoded luma values and decoded chroma values of the foreground pixels.

An apparatus can include a prediction mode decoding module configured to derive a luma intra prediction mode and a chroma intra prediction mode; a prediction size determining module configured to determine a size of a luma transform unit and a size of a chroma transform unit using transform size information; a reference pixel generating module configured to generate referential pixels if at least one reference pixel is unavailable; a reference pixel filtering module configured to adaptively filter the reference pixels of a current luma block based on the luma intra prediction mode and the size of the luma transform unit, and not to filter the reference pixels of a current chroma block; a prediction block generating module configured to generate prediction blocks of the current luma block and the current chroma block; a residual bock generating module configured to generate a luma residual block and a chroma residual block; and an adder.

An apparatus can include a prediction mode decoding module configured to derive a luma intra prediction mode and a chroma intra prediction mode; a prediction size determining module configured to determine a size of a luma transform unit and a size of a chroma transform unit using transform size information; a reference pixel generating module configured to generate referential pixels if at least one reference pixel is unavailable; a reference pixel filtering module configured to adaptively filter the reference pixels of a current luma block based on the luma intra prediction mode and the size of the luma transform unit, and not to filter the reference pixels of a current chroma block; a prediction block generating module configured to generate prediction blocks of the current luma block and the current chroma block; a residual bock generating module configured to generate a luma residual block and a chroma residual block; and an adder.

A method for processing image or video data is performed in an image processing pipeline. Color filtered mosaiced raw image or video data is received. A one-level wavelet transform of subbands of the color filtered mosaiced raw image or video data to provide LL, HH, LH and HL subbands. The LH and HL subbands are de-correlated by summing and difference operations to provide decorrelated sum and difference subbands. Additional n-level wavelet transformation on the sum and difference subbands and the LL and HH subbands to provide sparsified subbands for encoding. LL and HH and sum subbands are recombined into standard color images e.g., red, green, and blue color components, which are subsequently processed by color correction, white balance, and gamma correction. The sparsified subbands are encoded.

A method for processing image or video data is performed in an image processing pipeline. Color filtered mosaiced raw image or video data is received. A one-level wavelet transform of subbands of the color filtered mosaiced raw image or video data to provide LL, HH, LH and HL subbands. The LH and HL subbands are de-correlated by summing and difference operations to provide decorrelated sum and difference subbands. Additional n-level wavelet transformation on the sum and difference subbands and the LL and HH subbands to provide sparsified subbands for encoding. LL and HH and sum subbands are recombined into standard color images e.g., red, green, and blue color components, which are subsequently processed by color correction, white balance, and gamma correction. The sparsified subbands are encoded.