Intra-prediction apparatus, encoding apparatus, decoding apparatus and methods for non-square video coding blocks
11483554 · 2022-10-25
Assignee
Inventors
Cpc classification
H04N19/159
ELECTRICITY
H04N19/105
ELECTRICITY
H04N19/134
ELECTRICITY
International classification
H04N19/159
ELECTRICITY
Abstract
A device and method for intra-predicting a rectangular and non-square video coding block, the device including an intra-prediction apparatus configured for intra-predicting a rectangular and non-square video coding block according to a set of directional intra-prediction modes distributed over a set of directional ranges, where the intra-prediction apparatus being configured for configured for intra-predicting the rectangular and non-square video coding block includes the intra-prediction apparatus being configured to select a directional range from the set of directional ranges, and determine a number of directional intra-prediction modes of the selected directional range according to a size of a block.
Claims
1. A device, comprising: a non-transitory computer-readable storage medium storing instructions; and one or more processors in communication with the medium and, upon execution of the instructions, being configured to: select a directional range from a set of directional ranges; and determine a number of directional intra-prediction modes of the selected directional range according to a size of a block, wherein the block is a rectangular and non-square video coding block, the selected directional range comprises a basis directional range associated with a long side of the block, and further comprises an extension directional range that is associated with the long side of the block and that is dependent on an aspect ratio of the block, the extension directional range is outside of, and adjacent to the basis directional range along the long side of the block, and wherein the number of directional intra-prediction modes for the selected directional range comprises a number ΔM0 of directional intra-prediction modes of the extension directional range, and wherein the number ΔM0 of directional intra-prediction modes of the extension directional range is based on a lookup table having a plurality of numbers of directional intra-prediction modes for the extension directional range.
2. The device according to claim 1, wherein the one or more processors being configured to determine the number of directional intra-prediction modes comprises the one or more processors being configured to determine the number of directional intra-prediction modes of the selected directional range according to the size of the block and the aspect ratio of the block.
3. The device according to claim 1, wherein the one or more processors being configured to select the directional range comprises the processor being configured to select, from the set of directional ranges, a short directional range associated with a short side of the block and a long directional range associated with the long side of the block; and wherein the one or more processors being configured to determine the number of directional intra-prediction modes comprises the one or more processors being configured to determine a number ΔMshort of directional intra-prediction modes of the short directional range and a number ΔMlong of directional intra-prediction modes of the long directional range according to:
ΔMlong/Llong<ΔMshort/Lshort; wherein Lshort is a length of the short side of the block, and Llong is a length of the long side of the block.
4. The device according to claim 1, wherein the one or more processors being configured to select the directional range comprises the one or more processors being configured to select, from the set of directional ranges, a short directional range associated with a short side of the block, and a long directional range associated with the long side of the block; and wherein the one or more processors being configured to determine the number of directional intra-prediction modes comprises the one or more processors being configured to determine a number ΔMshort of directional intra-prediction modes of the short directional range and a number ΔMlong of directional intra-prediction modes of the long directional range, wherein an angular resolution ARshort of the short directional range is higher than an angular resolution ARlong of the long directional range; wherein an angular resolution ARG of a given directional range is proportional to a number ΔMG of directional intra-prediction modes of the given directional range, and is inversely proportional to an angle αG of the given directional range.
5. The device according to claim 4, wherein the angular resolution ARG of the given directional range is defined by:
ARG=ΔMG/αG.
6. The device according to claim 4, wherein two neighboring directional intra-prediction modes are at least one of two neighboring directional intra-prediction modes of the short directional range that are separated by a fixed angle β, or two neighboring directional intra-prediction modes of the long directional range that are separated by a fixed angle α.
7. The device according to claim 1, wherein the basis directional range and the extension directional range are disjoint; wherein the one or more processors being configured to select the directional range comprises the one or more processors being configured to select the extension directional range; and wherein the one or more processors being configured to determine the number of directional intra-prediction modes comprises the one or more processors being configured to determine the number ΔM0 of directional intra-prediction modes of the extension directional range according to a difference between a length L0/long of the long side of the block and a length L0/short of a short side of the block.
8. The device according to claim 1, wherein the basis directional range and the extension directional range are disjoint wherein the one or more processors being configured to select the directional range comprises the one or more processors being configured to select the extension directional range, and wherein the one or more processors being configured to determine the number of directional intra-prediction modes comprises the one or more processors being configured to determine the number ΔM0 of directional intra-prediction modes of the extension directional range.
9. The device according to claim 1, wherein the one or more processors being configured to determine the number of directional intra-prediction modes comprises the one or more processors being configured to: determine the number of directional intra-prediction modes of the selected directional range from the lookup table according to a block width and block height.
10. The device according to claim 1, wherein the device is an encoding apparatus for encoding a rectangular and non-square video coding block; and wherein the one or more processors, upon execution of the instructions, are further configured to: provide a predicted video coding block; and encode the video coding block into an encoded video bitstream according to of the predicted video coding block.
11. The device according to claim 1, wherein the device is a decoding apparatus for decoding an encoded video bitstream; wherein the one or more processors, upon execution of the instructions, are further configured to: provide a predicted video coding block; and restore the rectangular and non-square video coding block from the encoded video bitstream according to the predicted video coding block.
12. A method, comprising: selecting a directional range from a set of directional ranges, and determining a number of directional intra-prediction modes of the selected directional range according to a size of a block, wherein the block is a rectangular and non-square video coding block, the selected directional range comprises a basis directional range associated with a long side of the block, and further comprises an extension directional range that is associated with the long side of the block and that is dependent on an aspect ratio of the block, wherein the extension directional range is outside of, and adjacent to the basis directional range along the long side of the block, wherein the number of directional intra-prediction modes for the selected directional range comprises a number ΔM0 of directional intra-prediction modes of the extension directional range, and wherein the number ΔM0 of directional intra-prediction modes of the extension directional range is based on a lookup table having a plurality of numbers of directional intra-prediction modes for the extension directional range.
13. The method of claim 12, wherein the determining the number of directional intra-prediction modes comprises determining the number of directional intra-prediction modes of the selected directional range according to the size of the block and the aspect ratio of the block.
14. The method of claim 12, wherein the wherein the selecting the directional range comprises selecting, from the set of directional ranges, a short directional range associated with a short side of the block and a long directional range associated with the long side of the block; and wherein the determining the number of directional intra-prediction modes determining a number ΔMshort of directional intra-prediction modes of the short directional range and a number ΔMlong of directional intra-prediction modes of the long directional range according to:
ΔMlong/Llong<ΔMshort/Lshort wherein Lshort is a length of the short side of the block, and Llong is a length of the long side of the block.
15. The method of claim 12, wherein the selecting the directional range comprises selecting, from the set of directional ranges, a short directional range associated with a short side of the block, and a long directional range associated with the long side of the block; and wherein the determining the number of directional intra-prediction modes comprises determining a number ΔMshort of directional intra-prediction modes of the short directional range and a number ΔMlong of directional intra-prediction modes of the long directional range, wherein an angular resolution ARshort of the short directional range is higher than an angular resolution ARlong of the long directional range; wherein an angular resolution ARG of a given directional range is proportional to a number ΔMG of directional intra-prediction modes of the given directional range, and is inversely proportional to an angle αG of the given directional range.
16. The method of claim 12, wherein the extension directional range is dependent on the aspect ratio of the block; and wherein the basis directional range and the extension directional range are disjoint.
17. The method of claim 12, wherein the determining the number of directional intra-prediction modes comprises determining the number of directional intra-prediction modes of the selected directional range from the lookup table according to a block width and block height.
18. The method according to claim 12, further comprising encoding the rectangular and non-square video coding block, the encoding the rectangular and non-square video coding block comprising obtaining a predicted video coding block; and encoding the video coding block into an encoded video bitstream according to the predicted video coding block.
19. The method according to claim 12, further comprising decoding an encoded video bitstream, the decoding an encoded video bitstream comprising: obtaining a predicted video coding block; and restoring the rectangular and non-square video coding block from the encoded video bitstream according to the predicted video coding block.
20. A non-transitory computer readable medium having stored thereon instructions for execution by one or more processors, the instructions including instructions for: selecting a directional range from a set of directional ranges, and determining a number of directional intra-prediction modes of the selected directional range depending on a size of a block, wherein the block is a rectangular and non-square video coding block, the selected directional range comprises a basis directional range associated with a long side of the block, and further comprises an extension directional range that is associated with the long side of the block and that is dependent on an aspect ratio of the block, wherein the extension directional range is outside of, and adjacent to the basis directional range along the long side of the block, wherein the number of directional intra-prediction modes for the selected directional range comprises a number ΔM0 of directional intra-prediction modes of the extension directional range, and wherein the number ΔM0 of directional intra-prediction modes of the extension directional range is based on a lookup table having a plurality of numbers of directional intra-prediction modes for the extension directional range.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The above aspects and implementation forms of the invention will be explained in the following description of specific embodiments in relation to the enclosed drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS
(19)
(20) It is shown an intra-prediction apparatus 1300 for intra-predicting a rectangular and non-square video coding block based on a set of directional intra-prediction modes distributed over a set of directional ranges. The intra-prediction apparatus 1300 is configured to select 1301 a directional range from the set of directional ranges, and to determine 1302 a number of directional intra-prediction modes of the selected directional range depending on the size of the block.
(21)
(22) It is shown an encoding apparatus 1400 for encoding a rectangular and non-square video coding block. The encoding apparatus 1400 comprises the intra-prediction apparatus 1300 for providing a predicted video coding block. The encoding apparatus 1400 is configured to encode 1401 the video coding block into an encoded video bitstream on the basis of the predicted video coding block.
(23)
(24) It is shown a decoding apparatus 1500 for decoding an encoded video bitstream. The decoding apparatus 1500 comprises the intra-prediction apparatus 1300 for providing a predicted video coding block. The decoding apparatus 1500 is configured to restore 1501 a rectangular and non-square video coding block from the encoded video bitstream on the basis of the predicted video coding block.
(25)
(26) It is shown an intra-prediction method 1600 for intra-predicting a rectangular and non-square video coding block based on a set of directional intra-prediction modes distributed over a set of directional ranges. The intra-prediction method comprises selecting 1601 a directional range from the set of directional ranges. The intra-prediction method comprises determining a number of directional intra-prediction modes of the selected directional range depending on the size of the block.
(27)
(28)
(29) A rectangular and non-square video coding block 900 is shown. The block 900 has a short side 901 and a long side 902. The intra-prediction of the block 900 is based on a set of directional intra-prediction modes. The set of directional intra-prediction modes comprises diagonal modes 910-920. The diagonal modes are distributed over a set of directional ranges.
(30) The set of directional ranges comprises a short directional range 930 associated with the short side 901 of the block 900 and a long directional range 931 associated with the long side 902 of the block 900. The short directional range 930 can be associated with the entire short side 901 and the long directional range 931 can be associated with the entire long side 902.
(31) The set of directional intra-prediction modes comprises two diagonal modes 910, 918 delimiting the short directional range 930. These two diagonal modes 910, 918 present an angle of ϕ, respectively γ. The set of directional intra-prediction modes also comprises a diagonal mode 920 that delimits, together with the diagonal mode 918, the long directional range 931.
(32) The short directional range 930 comprises the two diagonal modes 910, 918 and further diagonal modes 911, 912, 913, 914, 915, 916, 917 between said diagonal modes 910, 918. The number of directional intra-prediction modes 910-918 of the short directional range 930 is ΔM.sub.short.
(33) Similarly, the long directional range 931 comprises the two delimiting diagonal modes 918, 920 and further diagonal modes 919 located between the two delimiting diagonal modes 918, 920. The number of directional intra-prediction modes 918-920 of the long directional range 931 is ΔM.sub.long.
(34) The number ΔM.sub.short and/or the number ΔM.sub.long can be determined depending on the size of the block 900, e.g. depending on the length L.sub.short of the short side 901 of the block 900 and depending on the length L.sub.long of the long side 902 of the block 900.
(35) The numbers ΔM.sub.short and ΔM.sub.long of directional intra-prediction modes of respectively the short directional range 930 and the long directional range 931 can be determined according to the equation:
ΔM.sub.long/L.sub.long<ΔM.sub.short/L.sub.short
(36) It is proposed to keep different number of directional intra-prediction for each side of the rectangular block 900. The actual number of these modes can be defined by the length of each side of the block 900. More particularly, the number of these modes can be defined by the following characteristics of the block goo: the aspect ratio and length of each side.
(37) As shown in
(38) To switch between different angular resolutions β, a of directional modes, the angles ϕ, γ of the two diagonal modes 910, 918 delimiting the short directional range 930 can be determined. These angles can be calculated as follows:
(39)
(40) wherein H is the height of the block 900 that corresponds to the length L.sub.short of the short side 901 of the block 900, and W is the width of the block 900 that corresponds to the length L.sub.long of the long side 902.
(41) These two equations correspond to the horizontal orientation of the block 900, and can be adapted to the vertical block orientation.
(42)
(43) A rectangular and non-square video coding block 1000 is shown. The block 1000 has a short side 1001 and a long side 1002. The intra-prediction of the block 1000 is based on a set of directional intra-prediction modes. The set of directional intra-prediction modes comprises diagonal modes 1010-1027. The diagonal modes are distributed over a set of directional ranges.
(44) The set of directional ranges comprises a short directional range 1030 associated with the short side 1001 of the block 1000. The short directional range 1030 can be associated with the entire short side 1001. The short directional range 1030 comprises two delimiting diagonal modes 1010, 1012 and further diagonal modes 1011 between said delimiting diagonal modes 1010, 1012. The delimiting diagonal mode 1010 has an angle I.sub.θ.
(45) The set of directional ranges comprises a basis directional range 1031 and an extension directional range 1032 that are associated with the long side 1002 of the block 1000. The basis directional range 1031 and the extension directional range 1032 can be separated or disjoint. The basis directional range 1031 and the extension directional range 1032 can share a common delimiting diagonal mode 1014. The basis directional range 1031 comprises diagonal modes 1012, 1013, 1014. The extension directional range 1032 comprises delimiting diagonal modes 1014, 1027 and further diagonal modes 1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027 comprised between said delimiting diagonal modes 1014, 1027.
(46) The number ΔM.sub.0 of directional intra-prediction modes 1020-1027 of the extension directional range 1032 depends on the difference between the length L.sub.0/long of the long side 1002 of the block 1000 and the length L.sub.0/short of the short side 1001 of the block 1000.
(47) Particularly, if the difference L.sub.0/long−L.sub.0/short increases, then the number ΔM.sub.0 can be increased, and vice versa.
(48)
(49) For an identical aspect ratio R.sub.A0=R.sub.A1, it is proposed that if the length L.sub.0/long of the long side 1052 of the block 1050 is smaller than the length L.sub.0/long of the long side 1002 of the block 1000, then the number ΔM.sub.1 of directional intra-prediction modes 1060-1063 of the extension directional range 1053 of block 1050 shall be smaller than the number ΔM.sub.0 of directional intra-prediction modes of the extension directional range 1032 of block 1000, and vice versa.
(50) In the embodiment illustrated in
(51) In the embodiment illustrated in
(52) The extension directional range 1032 is preferably added if the block 1000 is a non-square block. The angle α.sub.ext of the extension directional range 1032 is defined as the angle between its two delimiting directional intra-prediction modes 1014, 1027. The angle of the extension directional range 1032 can be given by the following formula:
(53)
(54)
(55) The asymmetry of the non-square block is used to improve the directional intra-prediction mechanism by increasing its prediction accuracy. This is achieved by building the extension directional range. The angle α.sub.ext of the extension directional range is indicated as complimentary angle in
(56) The extension directional range can comprise, in addition to the mode 66, additional directional intra-prediction modes ext.sub.1, ext.sub.2, . . . ext.sub.N. The number of modes N+1 of the extension directional range depends on the size of the block.
(57) If a set of directional intra-prediction modes is extended, i.e. if the extension directional range is used, it is desirable to signal the newly extended modes, which may not be accomplished using existing conventional mechanisms. For this purpose, a 2-step signaling mechanism for the extension of directional intra-prediction modes is proposed.
(58)
(59)
(60)
(61)
(62) The selection of the directional intra-prediction modes can be based on the intra mode selection proposed in JVET-D0113 (V. Seregin, X. Zhao, and M. Karczewicz, “Variable number of intra modes”, Contribution JVET-C0024 to the 4th JVET meeting, Chengdu, China, October 2016), which is a mechanism where the number of directional intra-prediction is adjustable. In particular, it proposes to further increase number of intra modes to 131 for large blocks sizes, while reducing the number of intra modes for small block sizes. The switching of intra modes number based on block sizes is controlled by two threshold values, which are signaled in SPS as log 2 values minus 4 and minus 6, respectively. The first threshold indicates the largest block size that can have 35 intra mode directions, and the second threshold indicates the largest block size that has 67 intra mode directions, all other blocks use 131 mode directions. In the default setting, thresholds are signaled as 4 and 6, respectively, and for higher resolution pictures it is set to 5 and 8.
(63) In an implementation, an intra mode index is always represented by 131 mode range regardless of the number of intra modes actually used. For 67 modes being actually used, only every second angular mode is allowed, and for 35 modes, only every fourth angular mode is allowed. Therefore, during intra mode signaling, intra mode of the neighboring block may need to be rounded towards nearest second or fourth angular intra mode if the current block uses smaller than 131 intra mode directions as explained in
(64) With regard to
(65)
(66) The mapping process 1100 is used to deduce 1108 a mode index {circumflex over (M)}.
(67) In step 1102, the width and height of the block 900 and a selected intra-prediction mode index M are taken as input values.
(68) In step 1103, the aspect ratio is computed.
(69) In step 1104, a first look-up table (LUT) is used to fetch mode indices I.sub.γ and I.sub.φ corresponding to the angles of the delimiting directional intra-prediction modes 910, 918. These angles can be used to switch between different resolutions of directional intra-prediction modes as explained above.
(70) In step 1105, it is checked whether the mode index M belongs to the short directional range 930.
(71) Depending on the result of step 1105, a decimation factor is fetched from a given LUT. This decimation factor F is used to map the mode index M to the mode index {circumflex over (M)} and depends on the length of a block side (width or height).
(72) A decimation factor of 2 can e.g. mean that only every second angular mode is allowed, and a decimation factor of 4 can mean that only every fourth angular mode is allowed, as shown with respect to
(73)
(74) Similarly to
(75) The following table 1 provides examples of directional intra-prediction mode subsets that are available for different lengths of block sides and for different aspect rations of blocks. Diagonal mode with index I.sub.γ is not considered in the number of available directional intra-prediction mode per side for the general case.
(76) TABLE-US-00001 TABLE 1 Examples of mode ranges available for different side lengths: The number of available directional intra-prediction modes General General case case Partic- (FIG. 9) (FIG. 9) ular Total M
,
M
, case direc- Block Block m ∈ M, m ∈ M, (FIG. 10) tional width height m < I.sub.γ m > I.sub.γ ΔM.sub.l modes 4 4 8 8 0 17 4 8 10 6 0 17 4 16 27 5 0 33 4 32 59 5 0 65 4 64 124 4 0 129 8 4 10 6 3 17 8 8 16 16 0 33 8 16 19 13 0 33 8 32 53 11 0 65 8 64 119 9 0 129 16 4 5 27 6 33 16 8 13 19 3 33 16 16 32 32 0 65 16 32 83 45 0 129 16 64 74 54 0 129 32 4 5 59 14 65 32 8 11 53 11 65 32 16 45 83 14 129 32 32 32 32 0 65 32 64 13 91 0 129 64 4 4 124 30 129 64 8 9 119 0 129 64 16 54 74 0 129 64 32 91 13 0 129 64 64 64 64 0 129
(77) Particular values of indices of diagonal directions for the case of 131 intra prediction modes (129 of them are directional ones) are given in Table 2.
(78) TABLE-US-00002 TABLE 2 Exemplary values of mode indices I.sub.γ, I.sub.φ and I.sub.θ mentioned in FIGS. 11 and 12 Values of Values of mode index mode indices Block aspect ratio I.sub.γ I.sub.φ (I.sub.θ) 1:2 (vertically oriented block) 117 79 1:4 108 88 1:8 103 93 1:16 101 96 2:1 (horizontally oriented block) 53 15 4:1 44 24 8:1 39 29 16:1 36 31
(79) The main advantages of the invention are as follows, including a set of available directional intra-prediction modes is defined subject to the actual lengths of each block side and optionally to the aspect ratio a block, the proposed signaling mechanism uses not only the lengths of block sides but also the aspect ratio of a block to select what directional modes should be skipped, and the proposed invention can be applied to the extension directional range of directional intra-prediction modes.
Definitions of Acronyms and Glossary
(80) ALF—Adaptive Loop Filter
(81) AMT—Adaptive Multiple Transform
(82) ARSS—Adaptive Reference Sample Smoothing
(83) BD-Rate—Bjøntegaard Delta Rate
(84) CABAC—Context-Adaptive Binary Arithmetic Coding
(85) CAVLC—Context-Adaptive Variable Length Coding
(86) CU—Coding Unit
(87) CTU—Coding Tree Unit
(88) DART—Direction-Adaptive Residual Transform
(89) DBF—De-Blocking Filter
(90) DCT—Discrete Cosine Transform
(91) DST—Discrete Sine Transform
(92) EIP—Enhanced Intra-Prediction
(93) EMT—Enhanced Multiple Transform
(94) HEVC—High-Efficiency Video Coding
(95) ICT—Integer Cosine Transform
(96) HM—HEVC Reference Model
(97) JEM—Joint Exploration Model
(98) KLT—Karhunen-Loéve-Transform
(99) KTA—Key Technical Area
(100) LCS—Luminance Coding Speedup
(101) LSB—Least Significant Bit
(102) LUT—Look-Up Table
(103) MDDT—Mode-Dependent Directional Transform
(104) MDCS—Mode-Dependent Coefficient Scan
(105) MPI—Multi-parameter Intra prediction
(106) MSB—Most Significant Bit
(107) MTT—Multi-Type Tree
(108) PDPC—Position Dependent Intra Prediction Combination
(109) PU—Prediction Unit
(110) QT—Quad-Tree
(111) QTBT—Quad-Tree Binary Tree
(112) RD—Rate-Distortion
(113) RD—cost-Rate-Distortion Cost
(114) RDO—Rate-Distortion Optimization
(115) RDOT—Rate-Distortion-Optimized Transform
(116) RQT—Residual Quad-Tree
(117) RSAF—Reference Sample Adaptive Filter
(118) SAO—Sample Adaptive Offset
(119) SBH—Sign Bits Hiding
(120) TB—Transform Block
(121) TU—Transform Unit
(122) VTM—VVC Test Model
(123) VVC—Versatile Video Coding, the standardization project developed by JVET
(124) CTU—the root of a coding a structure of the video sequence of a pre-defined size, containing a part of a picture (e.g. 64×64 pixels). A CTU can be partitioned into CUs.
(125) CU—a basic coding structure of the video sequence of a pre-defined size, containing a part of a picture, which is belonging to a CTU. A CU can be partitioned into further CUs.
(126) PU—a coding structure which is the result of the partitioning of a CU. Can be partitioned to several TUs or coincide with a single TU.
(127) While the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. From reading the present disclosure, other modifications will be apparent to a person skilled in the art. Such modifications may involve other features, which are already known in the art and may be used instead of or in addition to features already described herein.
(128) In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.