Asymmetric coding unit size block dependent ratio
11533502 · 2022-12-20
Assignee
Inventors
- Tangi Poirier (Cesson-Sevigne, FR)
- Franck Galpin (Cesson-Sevigne, FR)
- Fabrice Leleannec (Cesson-Sevigne, FR)
Cpc classification
H04N19/119
ELECTRICITY
International classification
H04N19/119
ELECTRICITY
Abstract
A block of video data is split and coded using existing transform sizes through one of several embodiments. In one embodiment, the block is split in alternate dimensions, depending on the block size. In another embodiment, the video block can be coded after splitting the block into at least two rectangular sub-blocks using horizontal or vertical divisions. Successive divisions using asymmetric splitting are forbidden if an equivalent split can be attained using only symmetrical splitting, and only one succession of divisions is permitted when there are other successions of asymmetric splitting that result in the identical sub-blocks. In another embodiment, a video block is split using successive splits, but the second type of split is dependent on the first type of split. Methods, apparatus, and signal embodiments are provided for encoding and decoding.
Claims
1. A method for coding a block of video data, comprising: dividing said block into at least two rectangular sub-blocks, and encoding each sub-block using processing such that a transform that corresponds to each sub-block size is used, wherein said dividing comprises: splitting said block into at least two rectangular sub-blocks using horizontal or vertical divisions, wherein successive divisions using asymmetric splitting are forbidden if an equivalent split can be attained using only symmetrical splitting, and wherein only one succession of divisions is permitted when there are other successions of asymmetric splitting that result in identical sub-blocks.
2. A method for decoding a block of video data, comprising: decoding at least one sub-block of a plurality of sub-blocks that comprise said block, using processing such that a transform that corresponds to each sub-block size is used, and reassembling the plurality of sub-blocks into said block, wherein reassembling comprises an inverse operation of dividing the block, wherein said dividing comprises: splitting said block into at least two rectangular sub-blocks using horizontal or vertical divisions, wherein successive divisions using asymmetric splitting are forbidden if an equivalent split can be attained using only symmetrical splitting, and wherein only one succession of divisions is permitted when there are other successions of asymmetric splitting that result in identical sub-blocks.
3. The method of claim 2, wherein said dividing further comprises: splitting the block into a first division of multiple sub-blocks if said block has a size dimension that is not a multiple of three, and wherein the block is split into a second division of multiple sub-blocks if said block has a size dimension that is a multiple of three.
4. The method or the apparatus of claim 3, wherein said first division is a one-fourth and three-fourths split, and said second division is a one-third and two-thirds split.
5. The method or the apparatus of claim 3, wherein said size dimension is height.
6. The method or the apparatus of claim 3, wherein said size dimension is width.
7. The method of claim 2, wherein said dividing comprises: splitting said block into a first division of multiple sub-blocks; splitting at least one of said sub-blocks into a second division of multiple, smaller sub-blocks, based on said first division.
8. The method or the apparatus of claim 7, wherein a mode representative of said second division is signaled.
9. An apparatus for coding a block of video data, comprising: a memory, and a processor, configured to: divide said block into at least two rectangular sub-blocks, and encode each sub-block using processing such that a transform that corresponds to each sub-block size is used, wherein said dividing comprises: splitting said block into at least two rectangular sub-blocks using horizontal or vertical divisions, wherein successive divisions using asymmetric splitting are forbidden if an equivalent split can be attained using only symmetrical splitting, and wherein only one succession of divisions is permitted when there are other successions of asymmetric splitting that result in identical sub-blocks.
10. An apparatus for coding a block of video data, comprising: a memory, and a processor, configured to: decode at least one sub-block of a plurality of sub-blocks that comprise said block, using processing such that a transform that corresponds to each sub-block size is used, and reassemble the plurality of sub-blocks into said block, wherein reassembling comprises an inverse operation of dividing the block, wherein said dividing comprises: splitting said block into at least two rectangular sub-blocks using horizontal or vertical divisions, wherein successive divisions using asymmetric splitting are forbidden if an equivalent split can be attained using only symmetrical splitting, and wherein only one succession of divisions is permitted when there are other successions of asymmetric splitting that result in identical sub-blocks.
11. A non-transitory computer readable medium containing data content generated according to the method of claim 1, for playback using a processor.
12. A non-transitory computer program product comprising instructions which, when the program is executed by a computer, cause the computer to generate a signal comprising video data generated according to the method of claim 1, for playback using a processor.
13. A non-transitory computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of claim 2.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
DETAILED DESCRIPTION
(22) An approach is described for improved compression efficiency and reduced complexity in video compression.
(23) At least one embodiment of this description proposes a new representation of Coding Structure, so as to represent the picture in a better way in the compressed domain.
(24) In this description, a new representation of Coding Structure is introduced for an improved representation in the compressed domain. For clarity, in this description, “dividing”, “segmenting” and “splitting” all mean the same thing, which is a the act of performing a straight-line division of a block of pixels. Similarly, “splits”, and “divisions” mean the same thing, a grouping of pixels as a result of dividing, splitting or segmenting a block or a sub-block.
(25) In the HEVC video compression standard, a picture is divided into so-called Coding Tree Units (CTU), which size is typically 64×64, 128×128, or 256×256 pixels. Each CTU is represented by a Coding Tree in the compressed domain. This is a quad-tree division of the CTU, where each leaf is called a Coding Unit (CU), see
(26) Each CU is then given some Intra or Inter prediction parameters (Prediction Info). To do so, it is spatially partitioned into one or more Prediction Units (PUs), each PU being assigned some prediction information. The Intra or Inter coding mode is assigned on the CU level, see
(27) The Partitioning of a Coding Unit into Prediction Unit(s) is done according to the partition type, signaled in the bit-stream. For Intra coding unit, only the partition types 2N×2N and N×N, illustrated in
(28) On the contrary, Inter Coding Units can use all partition types shown in
(29) According to the HEVC standard, Coding Units are also divided into so-called transform units, in a recursive way, following a “transform tree”. Thus, a transform tree is a quad-tree division of a coding unit, and transform units are the leaf of the transform tree. A transform unit encapsulates the square transform blocks of each picture component corresponding to a considered square spatial area. A transform block is a square block of samples in a single component, where the same transform is applied.
(30) New emerging video compression tools include a Coding Tree Unit representation in the compressed domain is proposed, in order to represent picture data in a more flexible way in the compressed domain. The advantage of this flexible representation of the coding tree is that it provides increased compression efficiency compared to the CU/PU/TU arrangement of the HEVC standard.
(31) The Quad-Tree plus Binary-Tree (QTBT) coding tool provides this increased flexibility. It consists in a coding tree where coding units can be split both in a quad-tree and in a binary-tree fashion. Such coding tree representation of a Coding Tree Unit is illustrated in
(32) The splitting of a coding unit is decided on the encoder side through a rate distortion optimization procedure, which consists in determining the QTBT representation of the CTU with minimal rate distortion cost.
(33) In the QTBT technology, a CU has either square or rectangular shape. The size of coding unit is always a power of 2, and typically goes from 4 to 128.
(34) In additional to this variety of rectangular shapes for a coding unit, this new CTU representation has the following different characteristics compared to HEVC: The QTBT decomposition of a CTU is made of two stages: first the CTU is split in a quad-tree fashion, then each quad-tree leaf can be further divide in a binary fashion. This is illustrated on the right of
(35) In other words, each Coding Unit is systematically made of a single prediction unit (previously 2N×2N prediction unit partition type) and single transform unit (no division into a transform tree).
(36) The described embodiments concern the domain of picture coding unit representation in compression and aims at further improved compression efficiency compared to QTBT technology.
(37) In another application, (Asymmetric Coding Units Codec Architecture, EP-IPA 16306308.4), it is proposed to introduce new asymmetric partitions in QTBT. These new shapes consist in sizes equal to 3.Math.2.sup.n in width and/or height. Furthermore, a CU with a size multiple of 3 in width or height can be further split in a binary fashion, horizontally or vertically. We call this type of split, a one quarter split.
(38) As a consequence, a square coding unit with size (w,h) (width and height) that would be split through one of the proposed one quarter split modes, for example HOR_UP (horizontal-up), would lead to 2 sub-coding units with respective rectangular sizes
(39)
(40) One problem solved by these embodiments is how to support the coding/decoding with current one quarter split, when one of the sub-blocks has a size 3.Math.2.sup.n in one direction. If we split this sub-block again with a one quarter split in the same direction, we will obtain 2 blocks of size 3˜2.sup.n-2 and 3.sup.2.Math.2.sup.n-2 in the same direction. For example two successive asymmetric horizontal top split: a block 32×32 is first divided into 32×8 and 32×24, the second sub-block is then divided further in 32×6 and 32×18.
(41) A drawback is that blocks of size 18 cannot be divided by 4, so it can't be further split. Successive asymmetric split will lead to many different block sizes. Many different block sizes will require many new transform sizes associated with each block size. These transforms require a lot of memory for a fast and efficient implementation.
(42) In a prior approach, triple-trees are introduced. One of the advantage is that all sub-blocks are a power of 2. The main drawback is that it does not offer as much flexibility to optimally choose the partitioning of the block as it enforces 3 sub-blocks, even if only 2 are necessary. For example, when the boundary of an object is passing close to the border (see
(43) The basic idea of the proposed approach is to have an asymmetric split ratio dependent on the block size. For a coding unit with size (w, h) (width and height) with height 2.sup.n, asymmetric split type HOR_UP (horizontal-up) leads to 2 sub-coding units with respective rectangular sizes
(44)
But for a coding unit with size (w,h) with height 3.Math.2.sup.n, asymmetric split type HOR_DOWN (horizontal-down), will lead to 2 sub-coding units with respective rectangular sizes
(45)
We call this type of split, a one third split. An example is shown in
(46) For example, for a block 32×32, a first asymmetric split leads to 32×8 and 32×24 blocks, a second asymmetric split for the second sub-block leads to 32×16 and 32×8 sub-blocks. So, the splitting size (one-quarter/one-third split) depends on the current size of the block.
(47) The syntax as described in the aforementioned prior approach is not impacted by the proposed embodiments described herein, only the interpretation of the split is changed.
(48) The codec module that is impacted concerns the division of the picture to code/decode in blocks (see 105 in
(49) TABLE-US-00001 TABLE 1 modified coding binary-tree syntax according to this proposed embodiment Descriptor coding_binary_tree( x0, y0, width, height, cqtDepth) { if(btSplitAllowed(x0,y0,width,height){ bt_split_mode(x0,y0,width,height,cqtDepth) } if( btSplitFlag ) { if(btSplitMode==HOR) { x1 =x0 y1 = y0 + ( height / 2 ) sub_width_1 = sub_width_0 = width; sub_height_1 = sub_height_0 = (height / 2) } if(btSplitMode==VER) { x1 = x0 + ( width / 2 ) y1 = y0 sub_width_1 = sub_width_0 = ( width / 2 ) sub_height_1 = sub_height_0 = height } if(btSplitMode==HOR_UP) { x1 = x0 sub_width_1 = sub_width_0 = width if( height % 3 != 0) { y1 = y0 + ( height / 4 ) sub_height_0 = (height / 4) sub_height_1 = ( (height * 3) / 4) } else { y1 = y0 + ( height / 3 ) sub_height_0 = (height / 3) sub_height_1 = ( (height * 2) / 3) } if(btSplitMode==HOR_DOWN) { x1 = x0 sub_width_1 = sub_width_0 = width if( height % 3 != 0) { y1 = y0 + ((height * 3) / 4 ) sub_height_0 = ( (height * 3) / 4) sub_height_1 = (height / 4) } else { y1 = y0 + ( (height * 2) / 3 ) sub_height_0 = ( (height * 2) / 3) sub_height_1 = (height / 3) } } if(btSplitMode==VER_LEFT) { y1 = y0 sub_height_1 = sub_height_0 = height if( width % 3 != 0) { x1 = x0 + ( width / 4 ) sub_width_0 = width / 4 sub_width_1 = (width *3) / 4 } else { x1 = x0 + ( width / 3 ) sub_width_0 = width / 3 sub_width_1 = (width * 2) / 3 } } if(btSplitMode==VER_RIGHT) { y1 = y0 sub_height_1 = sub_height_0 = height if( width % 3 != 0) { x1 = x0 + ( width*3) / 4 sub_width_0 = (width*3) / 4 sub_width_1 = width / 4 } else { x1 = x0 + ( width * 2) / 3 sub_width_0 = (width * 2) / 3 sub_width_1 = width / 3 } } coding_binary_tree( x0, y0, sub_width, sub_height, cqtDepth ) if( x1 < pic_width_in_luma_samples && y1 < pic_height_in_luma_samples) coding_binary_tree( x1, y1, sub_width, sub_height, cqtDepth ) } } else coding_unit( x0, y0, width, height ) }
In embodiment 2, splitting of asymmetric CUs leads to more redundancy in the syntax, many successive splits lead to the same partitioning. In
(50) The codec module that is impacted concerns the division of the picture to code/decode in blocks (see 105 in
(51) A third embodiment is now described. Some ACU block sizes play a specific role. For some pre-defined block sizes, we may not want to have a dedicated transform. For example, for a large transform (typically 48×N), there is no dedicated transform in order to limit the transform memory buffer size. On the contrary, for small blocks (typically 12×N), there are advantages to having an asymmetric split compared to a symmetric split. Therefore, the CU with these sizes is used as a transition towards some sub-CUs. It is done by inferring after a one quarter split, a one third split on the other side. For example, signaling of a first split corresponding to VER_LEFT leads to inferring a second split corresponding to VER_RIGHT.
(52) According to a fourth embodiment, some CU size are used as a transition stage towards multiple possible splitting configurations. An example is illustrated in
(53) This respectively corresponds to the binary splitting modes VER_RIGHT and VER, as shown in
(54) The support for transition CUs can be performed through a truncated syntax for the BT split mode signaling, compared to the initial complete syntax of Table 2. The bt split mode syntax for a CU that is known as a transition CU in one orientation (here the exemplary vertical orientation is considered) is depicted by Table 3. As can be seen, some syntax elements have been removed. First, the btSplitFlag is omitted, since we know the CU is necessarily subdivided. Moreover, if the vertical binary splitting is used for this transition CU, then only the verticalAsymmetricFlag syntax element is signaled for that CU, and the vertical_split_type syntax element is not signal, because the VER_LEFT bt split type is forbidden to avoid emulating 2 successive symmetric vertical splittings.
(55) TABLE-US-00002 TABLE 2 bt split mode syntax according used for coding CUs, according to the initial asymmetric CU tool Descriptor Bt_split_mode(x0,y0,width,height,cqtDepth){ if(btSplitAllowed(x0,y0,width,height){ btSplitFlag ae(v) if(horizontalSplitAllowed && horizontalSplitAllowed){ btSplitOrientation ae(v) if(btSplitOrientation==HOR && horizontal_asymmetric_allowed){ horAsymmetricSplitFlag ae(v) if(horAsymmetricSplitFlag==true){ horizontal_asymmetric_type ae(v) } } if(btSplitOrientation==VER && vertical_asymmetric_allowed){ verAsymmetricSplitFlag ae(v) if(verAsymmetricSplitFlag==true){ vertical_asymmetric_type ae(v) } } } }
(56) TABLE-US-00003 TABLE 3 modified bt split mode syntax according to signal the splitting of a transition CU in the vertical orientation Descriptor Bt_split_mode(x0,y0,width,height,cqtDepth){ if(btSplitAllowed(x0,y0,width,height){ ae(v) if(horizontalSplitAllowed && horizontalSplitAllowed){ btSplitOrientation ae(v) if(btSplitOrientation==HOR && horizontal_asymmetric_allowed){ horAsymmetricSplitFlag ae(v) if(horAsymmetricSplitFlag==true){ horizontal_asymmetric_type ae(v) } } if(btSplitOrientation==VER && vertical_asymmetric_allowed) { verAsymmetricSplitFlag ae(v)
} } }
According to variants of these embodiments, some CU size/orientation case can be considered a transition stages towards further subdivided configurations: 48×N: CU width equal to 48. According to some variant, N can be one or several values in the set {4,6,8,12,16,24,32,48,64} N×48: CU height equal to 48. According to some variant, N can be one or several values in the set {4,6,8,12,16,24,32,48,64} 24×N: CU width equal to 24. According to some variant, N can be one or several values in the set {4,6,8,12,16,24,32,48,64} N×24: CU height equal to 24. According to some variant, N can be one or several values in the set {4,6,8,12,16,24,32,48,64}
(57) The aforementioned embodiments have been described with respect to an encoder or encoding operation. However, the corresponding inverse operations are applicable to a decoder or decoding operation. For example, a decoding operation can perform decoding of at least one sub-block of a plurality of sub-blocks that comprise the block, using processing such that a transform that corresponds to each sub-block size is used, and reassembling the plurality of sub-blocks into the block, wherein reassembling comprises an inverse operation of dividing the block. The reassembling operation is substantially the inverse of the encoding dividing operations.
(58)
(59)
(60)
(61)
(62)
(63) The functions of the various elements shown in the figures can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage.
(64) Other hardware, conventional and/or custom, can also be included. Similarly, any switches shown in the figures are conceptual only. Their function can be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
(65) The present description illustrates the present ideas. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the present ideas and are included within its scope.
(66) All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the present principles and the concepts contributed by the inventor(s) to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.
(67) Moreover, all statements herein reciting principles, aspects, and embodiments of the present principles, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
(68) Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the present principles. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which can be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
(69) In the claims hereof, any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The present principles as defined by such claims reside in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
(70) Reference in the specification to “one embodiment” or “an embodiment” of the present principles, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present principles. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment”, as well any other variations, appearing in various places throughout the specification are not necessarily all referring to the same embodiment.