Method of sample adaptive offset processing for video coding
10659817 ยท 2020-05-19
Assignee
Inventors
Cpc classification
H04N19/91
ELECTRICITY
H04N19/70
ELECTRICITY
International classification
H04N19/91
ELECTRICITY
H04N19/70
ELECTRICITY
Abstract
A method of modified SAO (sample-adaptive offset) processing for a reconstructed picture in a video coding system to improve the performance is disclosed. In one example, a SAO-sign threshold is introduced to determine the sign of the difference between a current reconstructed pixel and a neighboring reconstructed pixel. A range of difference values greater than the negative SAO-sign threshold and smaller than the positive SAO-sign threshold is assigned to have a sign value of 0. In another example, the SAO-offset value is derived by multiplying the SAO-offset sign with a result from applying left shift by the SAO-bit-shift value to the absolute SAO-offset value. In yet another example, the absolute SAO-offset value is coded by truncated Rice (TR) codes and a maximum TR value is indicated by a syntax element.
Claims
1. A method of an edge offset (EO) sample-adaptive offset (SAO) processing for reducing an image noise in a video coding system, the method comprising: receiving an input data associated with a reconstructed picture; determining a first difference between a current reconstructed pixel and a first neighboring reconstructed pixel and a second difference between the current reconstructed pixel and a second neighboring reconstructed pixel; determining a first SAO sign of the first difference based on the first difference and a SAO-sign threshold that is greater than zero according to:
the EO classification index=the first SAO sign+the second SAO sign+2; and compensating the current reconstructed pixel by adding a SAO-offset value associated with the EO classification index to the current reconstructed pixel, wherein the SAO-offset value associated with the EO classification index for the current reconstructed pixel is determined according to a SAO-offset sign, an absolute SAO-offset value and a SAO-bit-shift value, and the SAO-offset value is derived by multiplying the SAO-offset sign with a result from applying a left shift by the SAO-bit-shift value to the absolute SAO-offset value.
2. The method of claim 1, wherein the first neighboring reconstructed pixel, the current reconstructed pixel, and the second neighboring reconstructed pixel correspond to three consecutive pixels in a horizontal direction, a vertical direction, a 45-degree diagonal direction or a 135-degree diagonal direction.
3. The method of claim 1, wherein a syntax element corresponding to the SAO-sign threshold is signaled in a bitstream.
4. The method of claim 3, wherein the syntax element corresponding to the SAO-sign threshold is signaled in a slice header in the bitstream.
5. The method of claim 1, wherein the SAO-sign threshold is determined at an encoder side of the video coding system according to a rate-distortion optimization (RDO) procedure.
6. The method of claim 1, wherein a syntax element corresponding to the SAO-bit-shift value is signaled in a bitstream.
7. The method of claim 6, wherein the syntax element corresponding to the SAO-bit-shift value is signaled in a slice header in the bitstream.
8. The method of claim 1, wherein the absolute SAO-offset value is coded by truncated Rice (TR) codes and a maximum TR value is indicated by a syntax element.
9. The method of claim 1, wherein the SAO-bit-shift value is set to 0 when quantization parameters (QP) associated with the current reconstructed pixel is smaller than a first threshold and the first threshold is dependent on a bit depth of video data.
10. An apparatus of an edge offset (ED) sample-adaptive offset (SAO) processing for reducing an image noise in a video coding system, comprising a circuitry configured to: receive an input data associated with a reconstructed picture; determine a first difference between a current reconstructed pixel and a first neighboring reconstructed pixel and a second difference between the current reconstructed pixel and a second neighboring reconstructed pixel; determine a first SAD sign of the first difference based on the first difference and a SAO-sign threshold that is greater than zero according to;
11. The apparatus of claim 10, wherein the circuity is configured to signal a syntax element corresponding to the SAO-sign threshold in a bitstream.
12. The apparatus of claim 10, wherein the circuitry is configured to determine the SAO-sign threshold according to a rate-distortion optimization (RDO) procedure.
13. The apparatus of claim 10, wherein the circuitry is configured to signal a syntax element corresponding to the SAO-bit-shift value in a bitstream.
14. The apparatus of claim 10, wherein the absolute SAO-offset value is coded by truncated Rice (TR) codes and a maximum TR value is indicated by a syntax element.
15. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method of an edge offset (EO) sample-adaptive offset (SAO) processing for reducing an image noise in a video coding system, the method comprising: receiving an input data associated with a reconstructed picture; determining a first difference between a current reconstructed pixel and a first neighboring reconstructed pixel and a second difference between the current reconstructed pixel and a second neighboring reconstructed pixel; determining a first SAO sign of the first difference based on the first difference and a SAO-sign threshold that is greater than zero according to;
16. The non-transitory computer-readable medium of claim 15, wherein a syntax element corresponding to the SAO-sign threshold is signaled in a bitstream.
17. The non-transitory computer-readable medium of claim 15, wherein the SAO-sign threshold is determined at an encoder side of the video coding system according to a rate-distortion optimization (RDO) procedure.
18. The non-transitory computer-readable medium of claim 15, wherein a syntax element corresponding to the SAO-bit-shift value is signaled in a bitstream.
19. The non-transitory computer-readable medium of claim 15, wherein the absolute SAO-offset value is coded by truncated Rice (TR) codes and a maximum TR value is indicated by a syntax element.
Description
BRIEF DESCRIPTION OF DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
DETAILED DESCRIPTION
(14) In order to improve the performance of the SAO processing in the existing High Efficiency Video Coding (HEVC) standard, modified SAO processing is disclosed. The modified SAO processing involves modified processing along with introduction of new syntax elements.
(15) In one embodiment according to the present invention, modified SAO sign determination is introduced. The SAO sign according to the existing HEVC standard is the sign of an underlying difference between the current sample (c) and a neighboring sample (c.sub.1 or c.sub.1) directly as shown in equations (1) and (2). Therefore, only the zero-valued underlying difference has its corresponding sao_sign assigned a value of 0. Embodiments of the present invention extend the data range of the underlying difference to be assigned a value of 0 by introducing a SAO-sign threshold, sao_sign_thre. The SAO sign of the underlying difference according to embodiments of the present invention is determined according to the following equation:
(16)
where variable saoSignThre is specified by the syntax element sao_sign_thre.
(17) Accordingly, an underlying difference having a value greater than saoSignThre and smaller than saoSignThre will have the sao_sign assigned a value of 0. At the same time, the data ranges associated with sao_sign values, 1 and 1 will be reduced. The syntax element sao_sign_thre is utilized in the modified SAO processing for pixel classification for the EO type. A proper sao_sign_thre value may be selected for a sequence, picture, a slice or other picture unit to achieve improved performance. For example, the encoder may use the rate-distortion procedure to determine a sao_sign_thre value that achieves the best performance. The category index, cat_idx, for the current pixel c according to the modified SAO processing is determined in the same way as the HEVC standard, i.e.:
cat_idx=sao_sign(cc.sub.1)+sao_sign(cc.sub.1)+2(6)
(18) In another embodiment, a second modification to the SAO processing of the existing HEVC standard is related to representing the SAO offset value, SaoOffsetVal. The syntax element sao_bit_shift is utilized in the modified SAO processing for representing the offset value, SaoOffsetVal, as follows:
SaoOffsetVal=offsetSign*(sao_offset_abs<<saoBitShift),(7)
where variable saoBitShift is specified by the syntax element sao_bit_shift. On the other hand, the value for the left shift operation to obtain SaoOffsetVal according to the existing HEVC standard is (bitDepthMin(bitDepth,10). The modified SAO operation along to obtain the SAO offset value along with the new syntax element sao_bit_shift can help to improve performance especially for high sample bit depth. This second modification to the SAO processing can be applied to edge offset (EO), band offset (BO), or both.
(19) Sample syntax design to signal the sao_bit_shift according to an embodiment of the present invention is shown in
(20)
(21) In the HEVC standard, the absolute offset value is coded using the truncated Rice (TR) codes with the largest possible value (cMax) set according to Equation (4) and Rice parameter (cRiceParam) set to 0. The modified SAO processing according to an embodiment of the present invention uses a new syntax element, sao_offset_max to represent the largest offset value (cMax) directly. With the cMax and cRiceParam specified as shown below, the syntax element sao_offset_abs can be entropy coded using truncated Rice (TR) binarization process accordingly,
cMax=saoOffsetMax, and cRiceParam=0
where variable saoOffsetMax is specified by syntax element sao_offset_max.
(22) The SAO-bit-shift value can be set to 0 when quantization parameters (QP) associated with the current reconstructed pixel is smaller than a threshold. For example, the threshold can be dependent on the bit depth of video data, where a smaller threshold can be used for larger bit depth.
(23) The performance of a video coding system incorporating an embodiment of the present invention according to modified SAO processing is compared with the performance of a conventional system based on HTM-10.0 as shown in Tables 2-4 for ALL Intra, Random Access and Low Delay B picture coding configurations at various coding tiers (Main, High and Super High) respectively. The BD-rate differences are shown for individual video components (Y, U and V). A negative value in the BD-rate indicates that the present invention has a better performance. As shown in Tables 2-4, the BD-rates for individual components (Y, U and V) incorporating an embodiment of the present invention are reduced by 0.1% to 1.6%.
(24) TABLE-US-00002 TABLE 2 All Intra All Intra All Intra Super High- Main-tier High-tier tier Y U V Y U V Y U V RGB 4:4:4 0.3% 0.2% 0.3% 0.2% 0.1% 0.2% 0.1% 0.1% 0.1% YCbCr 4:4:4 0.3% 0.3% 0.4% 0.2% 0.2% 0.3% 0.2% 0.2% 0.2% YCbCr 4:2:2 0.1% 0.6% 0.9% 0.1% 0.2% 0.4% 0.1% 0.1% 0.2% Overall 0.2% 0.3% 0.5% 0.2% 0.2% 0.3% 0.1% 0.1% 0.2% Enc Time [%] 106% 105% 105% Dec Time [%] 104% 103% 103%
(25) TABLE-US-00003 TABLE 3 Random Access Random Access Main-tier High-tier Y U V Y U V RGB 4:4:4 0.7% 1.2% 0.6% 0.6% 0.9% 0.5% YCbCr 4:4:4 0.4% 2.3% 0.2% 0.5% 1.6% 0.0% YCbCr 4:2:2 0.2% 1.3% 1.4% 0.3% 1.1% 0.7% Overall 0.4% 1.6% 0.7% 0.5% 1.2% 0.4% Enc Time [%] 102% 102% Dec Time [%] 102% 103%
(26) TABLE-US-00004 TABLE 4 Low delay Low delay B Main-tier B High-tier Y U V Y U V RGB 4:4:4 0.9% 1.2% 1.1% 0.6% 0.8% 0.7% YCbCr 4:4:4 0.9% 2.0% 0.6% 0.8% 1.4% 0.4% YCbCr 4:2:2 0.3% 1.3% 1.3% 0.3% 0.8% 0.7% Overall 0.7% 1.5% 1.0% 0.6% 1.0% 0.6% Enc Time [%] 101% 101% Dec Time [%] 99% 101%
(27) The performance of a video coding system with higher bit depths at 16 bits and 12 bits incorporating an embodiment of the present invention using sao_luma_bit_shift and sao_chroma_bit_shift is compared to the performance of a conventional system based on HTM-10.0 as shown in Table 5 and Table 6 for All Intra and Low Delay coding configurations respectively. The performance comparison is based on different sets of test data listed in the first column. As shown in Table 5, the BD-rates for individual components (Y/G, Cb/B and Cr/R) incorporating an embodiment of the present invention are about the same as the conventional HEVC for the All Intra configuration. However, as shown in Table 6, the BD-rates for individual components (Y/G, Cb/B and Cr/R) incorporating an embodiment of the present invention are reduced by 0.1% to 3.3% for Low Delay B picture configuration.
(28) TABLE-US-00005 TABLE 5 Average BD-rate Change (%) - All Tiers Intra 16-bit Intra 12-bit (internal bit depth) (internal bit depth) Y/G Cb/B Cr/R Y/G Cb/B Cr/R SVT 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% RExt RGB 0.0% 0.0% 0.1% 0.1% 0.0% 0.1% HDR 0.1% 0.0% 0.0% 0.1% 0.0% 0.0% 4K from HDR 0.0% 0.1% 0.0% 0.0% 0.1% 0.0% HDR 4:0:0 0.2% 0.3% Medical 4:0:0 Medical 0.0% 0.0% Synthetic RGB 0.0% 0.1% 0.0% 0.0% 0.1% 0.0% HDR (+2 MSBs) Synthetic RGB 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% HDR (+4 MSBs) Synthetic RGB 0.0% 0.1% 0.1% 0.0% 0.0% 0.0% HDR (+6 MSBs) All Classes 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
(29) TABLE-US-00006 TABLE 6 Average BD-rate Change (%) - All Tiers Lowdelay 16-bit Lowdelay 12-bit (internal bit depth) (internal bit depth) Y/G Cb/B Cr/R Y/G Cb/B Cr/R SVT 0.5% 0.3% 0.3% 0.3% 0.3% 0.2% RExt RGB 0.2% 0.2% 0.2% 0.3% 0.2% 0.3% HDR 0.3% 0.4% 0.3% 0.2% 0.3% 0.3% 4K from HDR HDR 4:0:0 3.2% 3.3% Medical 4:0:0 Medical 0.7% 0.5% Synthetic 0.1% 0.3% 0.2% 0.1% 0.3% 0.2% RGB HDR (+2 MSBs) Synthetic 0.0% 0.1% 0.0% 0.2% 0.0% 0.2% RGB HDR (+4 MSBs) Synthetic 0.8% 0.0% 0.0% 0.7% 0.1% 0.1% RGB HDR (+6 MSBs) All Classes 0.7% 0.2% 0.2% 0.6% 0.2% 0.1%
(30)
(31)
(32)
(33) The flowcharts shown above are intended to illustrate examples of modified SAO processing according to the present invention. A person skilled in the art may modify each step, re-arranges the steps, split a step, or combine steps to practice the present invention without departing from the spirit of the present invention.
(34) The above description is presented to enable a person of ordinary skill in the art to practice the present invention as provided in the context of a particular application and its requirement. Various modifications to the described embodiments will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed. In the above detailed description, various specific details are illustrated in order to provide a thorough understanding of the present invention. Nevertheless, it will be understood by those skilled in the art that the present invention may be practiced.
(35) Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware code may be developed in different programming languages and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.
(36) The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.