Method and apparatus of simplified sub-mode for video coding

Abstract

A method and apparatus of Inter prediction for video coding are disclosed. According to one method, a sub-block motion vector prediction (MVP) mode is turned off for small size coding units (CUs). In another method, if the neighbouring reference block for a current coding unit (CU) is in a root CU region, the neighbouring reference block is not used to derive a Merge candidate or a modified neighbouring reference block on the shared boundary of the root CU is used to derive the Merge candidate for the current block. In yet another method, a shared sub-block Merge candidate list is derived for sub-CUs within a root CU region or an MER (Merge estimation region). If a neighbouring reference block is within the same MER as a current sub-CU, the neighbouring reference block is not used for deriving a candidate for the shared sub-CU Merge list.

Claims

1. A method of Inter prediction for video coding, the method comprising: receiving input data related to a current block in a current picture at a video encoder side or a video bitstream corresponding to compressed data including the current block in the current picture at a video decoder side, wherein the current block corresponds to one target leaf block among multiple leaf blocks under a root node resulted from block partitioning of an area including the current block; determining an MER (Merge Estimation Region) enclosing said multiple leaf blocks; if a reference block for the current block is inside the MER, excluding a target candidate associated with the reference block from a Merge candidate list or including a modified target candidate in the Merge candidate list, wherein the modified target candidate is derived based on a modified reference block outside the MER, wherein the Merge candidate list is derived for a sub-block mode, wherein excluding a target candidate comprises setting a temporal motion vector of a sub-block temporal MVP mode to zero; and encoding or decoding current motion information associated with the current block using the Merge candidate list.

2. The method of claim 1, wherein the Merge candidate list is derived for SbTMVP (Subblock-based Temporal Motion Vector Prediction).

3. The method of claim 1, wherein the modified reference block is located adjacent to a boundary of the MER.

4. An apparatus of Inter prediction for video coding, wherein said video coding allows sub-block mode motion vector prediction, the apparatus comprising one or more electronic circuits or processors arranged to: receive input data related to a current block in a current picture at a video encoder side or a video bitstream corresponding to compressed data including the current block in the current picture at a video decoder side, wherein the current block corresponds to one target leaf block among multiple leaf blocks under a root node resulted from block partitioning of an area including the current block; determine an MER (Merge Estimation Region) enclosing said multiple leaf blocks; if a reference block for the current block is inside the MER, exclude a target candidate associated with the reference block from a Merge candidate list or include a modified target candidate in the Merge candidate list, wherein the modified target candidate is derived based on a modified reference block outside the MER, wherein the Merge candidate list is derived for a sub-block mode, wherein excluding a target candidate comprises setting a temporal motion vector of a sub-block temporal MVP mode to zero; and encode or decode current motion information associated with the current block using the Merge candidate list.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 illustrates an example of block partition using quadtree structure to partition a coding tree unit (CTU) into coding units (CUs).

(2) FIG. 2 illustrates asymmetric motion partition (AMP) according to High Efficiency Video Coding (HEVC), where the AMP defines eight shapes for splitting a CU into PU.

(3) FIG. 3 illustrates an example of various binary splitting types used by a binary tree partitioning structure, where a block can be recursively split into two smaller blocks using the splitting types.

(4) FIG. 4 illustrates an example of block partitioning and its corresponding binary tree, where in each splitting node (i.e., non-leaf node) of the binary tree, one syntax is used to indicate which splitting type (horizontal or vertical) is used, where 0 may indicate horizontal splitting and 1 may indicate vertical splitting.

(5) FIG. 5 illustrates an example of block partitioning and its corresponding QTBT, where the solid lines indicate quadtree splitting and dotted lines indicate binary tree splitting.

(6) FIG. 6 illustrates an example of tree-type signalling for block partitioning according to MTT block partitioning, where RT signalling may be similar to the quad-tree signalling in QTBT block partitioning.

(7) FIG. 7 shows the neighbouring PUs used to derive the spatial and temporal MVPs for both AMVP and Merge scheme.

(8) FIG. 8 illustrates an example of the derivation process for combined bi-predictive Merge candidate.

(9) FIG. 9A illustrates an example of the derivation process of the scaled bi-predictive Merge candidate, where candidate list on the left corresponds to an original candidate list and the candidate list on the right corresponds to the expanded candidate list including two generated bi-prediction MVPs.

(10) FIG. 9B illustrates an example of adding zero vector Merge candidates, where the candidate list on the left corresponds to an original Merge candidate list and the candidate list on the right corresponds to the extended Merge candidate list by adding zero candidates.

(11) FIG. 9C illustrates an example for adding zero vector AMVP candidates, where the candidate lists on the top correspond to original AMVP candidate lists (L0 on the left and L1 on the right) and the candidate lists at the bottom correspond to the extended AMVP candidate lists (L0 on the left and L1 on the right) by adding zero candidates.

(12) FIG. 10 illustrates an example of four-parameter affine model, where a current block and a reference block are shown.

(13) FIG. 11A illustrates an example of inherited affine candidate derivation, where the current block inherits the affine model of a neighbouring block by inheriting the control-point MVs of the neighbouring block as the control-point MVs of the current block.

(14) FIG. 11B illustrates the neighbouring block set used for deriving the corner-derived affine candidate, where one MV is derived from each neighbouring group.

(15) FIG. 12A-FIG. 12C illustrates examples of shared Merge list for sub-CUs within a root CU.

(16) FIG. 13 illustrates an example of sub-tree, where the sub-tree root is a tree node inside the QTBT split tree.

(17) FIG. 14 illustrates a flowchart of an exemplary Inter prediction for video coding, wherein said video coding allows sub-block mode motion vector prediction according to an embodiment of the present invention.

(18) FIG. 15 illustrates a flowchart of an exemplary Inter prediction for video coding, wherein neighbouring blocks inside one root region are cancelled or pushed for Merge candidate according to an embodiment of the present invention.

(19) FIG. 16 illustrates a flowchart of an exemplary Inter prediction for video coding, wherein child blocks inside one root region share a common Merge candidate list according to an embodiment of the present invention.

DETAILED DESCRIPTION OF PREFERRED IMPLEMENTATIONS

(20) The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

(21) In the present invention, some techniques to simplify sub-block Merge mode are disclosed.

(22) Method—Shared Candidate List

(23) To simplify the codec operation complexity, a method of shared candidate list is proposed. The “candidate list” may correspond to Merge candidate list, AMVP candidate list or other types of prediction candidate list (e.g. DMVR (Decoder-side Motion Vector Refinement) or bi-lateral refinement candidate list). The basic idea of “shared candidate list” is to generate the candidate list on a bigger boundary (or one root of a sub-tree in QTBT Tree) so that the generated candidate list can be shared by all leaf-CUs inside the boundary or inside the sub-tree. Some examples of shared candidate lists are shown in FIG. 12A to FIG. 12C. In FIG. 12A, the root CU (1210) of sub-tree is shown by the large dashed box. A split leaf CU (1212) is shown as a smaller dashed box. The dashed box 1210 associated with the root CU also corresponds to a shared boundary for leaf CUs under the root leaf. In FIG. 12B, the shared boundary (1220) is shown by the large dashed box. A small leaf CU (1222) is shown as a smaller dashed box. FIG. 12C shows four examples of Merge sharing nodes. The shared merging candidate list is generated for the dotted virtual CU (i.e., Merge sharing node). In partition 1232, the Merge sharing node corresponding to an 8×8 block is split into 4 4×4 blocks. In partition 1234, the Merge sharing node corresponding to an 8×8 block is split into 2 4×8 blocks. In partition 1236, the Merge sharing node corresponding to a 4×16 block is split into 2 4×8 blocks. In partition 1238, the Merge sharing node corresponding to a 4×16 block is split into 2 4×4 blocks and 1 8×8 block.

(24) There are two main embodiments of “shared candidate list”: one is to share the candidate list inside a sub-tree; and the other is to share the candidate list inside a “common shared boundary”.

(25) Embodiment—Shared Candidate List Inside One Sub-Tree

(26) The term “sub-tree” is defined as a sub-tree of QTBT split tree (e.g. the QTBT split tree 120 as shown in FIG. 1). One example of “sub-tree” (1310) is shown in FIG. 13, where the sub-tree root is a tree node (1312) inside the QTBT split tree. The final split leaf CUs of the sub-tree are inside this sub-tree. The block partition 1320 corresponds to the sub-tree 1310 in FIG. 13. In the proposed method, the candidate list (Merge mode, AMVP mode candidate or other type of prediction candidate list) can be generated on a shared-block-boundary base, where examples of the shared-block-boundary is based on the root CU boundary of sub-tree as shown in FIG. 12A. The candidate list is then re-used for all leaf CU inside the sub-tree. The common shared candidate list is generated by the root of the sub-tree. In other words, the spatial neighbour position and the temporal neighbouring position are all based on the rectangular boundary (i.e., shared boundary) of the root CU boundary of the sub-tree.

(27) Embodiment—Shared Candidate List inside One “Common Shared Boundary”

(28) In this embodiment, a “common shared boundary” is defined. One “common shared boundary” is a rectangular area of minimum-block (e.g. 4×4) aligned inside picture. Every CU inside the “common shared boundary” can use a common shared candidate list, where the common shared candidate list is generated based on the “common shared boundary”. For example, the sub-blocks within the common shared boundary 1210 can share a Merge candidate list, where one or more Merge candidates are derived based on neighbouring blocks on the common shared boundary. In other words, the spatial neighbour position and the temporal neighbouring position are all based on the “common shared boundary”.

(29) Sub-CU Merge Candidate Handling in the Shared Candidate List

(30) Some methods to achieve shared list for sub-CU candidate (e.g. ATMVP, STMVP or Affine merge in Merge mode). Examples of sub-CU candidate for the shared candidate list according to the present invention are shown as follows.

(31) One method is to turn off the sub-CU candidate in the shared candidate list. Another method is to build the sub-CU candidate on the root CU (or built on shared-block-boundary), and for each “child CU for sharing”, it directly retrieves the (corresponding area of) sub-CU motion information from the sub-CU candidate. For example, for a shared 16×8 boundary using ATMVP mode, the ATMVP can be generated on the shared boundary 16×8 as the conventional ATMVP method. When sharing this candidate list for 2 child CU (e.g. 8×8) inside the common boundary (16×8), the left 8×8 motion information of the 16×8 ATMVP can be directly retrieved to form the new 8×8 ATMVP for left 8×8 child CU. Similarly, the right 8×8 motion information of the 16×8 ATMVP can be directly retrieved to form the new 8×8 ATMVP for the right 8×8 child CU. Accordingly, the ATMVP candidates suitable for 8×8 are generated.

(32) In another embodiment, the initial MV of the ATMVP (for deriving the collocated MV reference block in ATMVP) is derived by using the size/depth/shape/area/width/height of the root CU or the shared boundary. The initial MV of the ATMVP of the root CU or the shared boundary can be shared for the children CUs. The shared initial MV can be used to derive the collocated MV reference block of the child CU, and then derive the block MV or sub-block MVs of the child CU.

(33) According to another embodiment of the present invention, it removes some candidates according to CU size. If a CU size is smaller than a pre-defined threshold related to CU size (e.g. area=16), some candidates are removed from the construction of the candidate list. As is known in the field, the CU size may correspond to the block width, block height or the block area. While the area is used as an example of the pre-defined threshold related to CU size, the width, height or both can be used as the pre-defined threshold related to CU size. There are several embodiments of removing some candidates. For example, ATMVP can be removed based on one or more pre-defined threshold related to the CU size.

(34) According to one embodiment of the present invention, the sub-CU candidate in the shared candidate list is turned off. According to another embodiment of the present invention, the sub-CU candidate is built on the root CU (or built on shared-block-boundary). For each “child CU for sharing”, the corresponding area of sub-CU motion information is directly retrieved from the sub-CU candidate. In an example of a shared boundary (size 16×8) with ATMVP mode, the ATMVP can be generated based on the shared boundary (size 16×8) as the conventional ATMVP method. When sharing this candidate list for two child CUs having size 8×8 inside the common boundary (i.e., 16×8), the left 8×8 motion information of the 16×8 ATMVP can be directly retrieved to form the new 8×8 ATMVP candidate for the left 8×8 child CU. Similarly, the right 8×8 motion information of the 16×8 ATMVP candidate can be directly retrieved to form the new 8×8 ATMVP candidate for the right 8×8 child CU. Accordingly, the ATMVP candidate suitable for the 8×8 can be derived based on the 16×8 block.

(35) The proposed “shared candidate list”, “shared Merge index” and other shared-attribute proposal according to embodiments of the present invention can be applied to other type of Merge list construction method, such as “history-based Merge mode construction”, and “non-adjacent Merge candidate”. In other words, the shared-attribute proposal is generally applicable to all Merge mode algorithms and AMVP mode algorithm.

(36) Moreover, we further propose to signal a flag to switch on or off for the proposed sharing method. In one embodiment, a flag may be signalled to indicate whether “shared candidate list” is enabled. The minimum sizes of units in the signalling can also be separately coded in the sequence level, picture level, slice level, or PU level.

(37) In one embodiment, when deriving the initial vector for ATMVP, the ATMVP is not used if the referenced neighbouring MV is inside of the root CU or shared boundary.

(38) Method—Turning Off Sub-CU Merge Mode for Small CU

(39) In this embodiment, it turns off (or called exclude) sub-CU Merge mode (e.g. ATMVP, STMVP or Affine merge) for small CUs (e.g. CU size lower than a threshold or any other CU-size related feature lower than a threshold).

(40) In the sub-block Merge list, more than one ATMVP candidate can be inserted. For example, two ATMVP candidates can be inserted. In one embodiment, the two ATMVP candidates are inserted in front of the sub-block Merge list. In another embodiment, one ATMVP candidate is inserted in front of the sub-block Merge list, and the other one is inserted after one or more other types of sub-block candidate (e.g. affine candidate). In one example, the ATMVP is inserted at the third, fourth or fifth position of the sub-block merge list. In another example, the ATMVP is inserted after certain affine candidates in the sub-block merge list (e.g. after some affine inherited candidates or before the affine constructed candidates). In another embodiment, both ATMVP candidates are inserted after one or more other types of sub-block candidates (e.g. affine candidates).

(41) Method—Shared List for Affine Coded Blocks

(42) In the proposed shared list methods (e.g. Shared Candidate List inside One Sub-Tree and Common Shared Boundary), the root CU (also called the parent CU) or the shared boundary size/depth/shape/width/height is used to derive the candidate list. In the candidate list derivation, for any position-based derivation (e.g. the reference block position derivation according to the current block/CU/PU position/size/depth/shape/width/height), the root CU or the shared boundary position and shape/size/depth/width/height is used. In one embodiment, for affine inherit candidate derivation, the reference block position is first derived. When applying the shared list, the reference block position is derived by using the root CU or the shared boundary position, and shape/size/depth/width/height. In one example, the reference block positions are stored. When the child CU in the root CU or the shared boundary, the stored reference block position are used to find the reference block for affine candidate derivation.

(43) In another embodiment, the control point MVs of the root CU or the shared boundary of each affine candidate in the candidate list are derived. The control point MVs of the root CU or the shared boundary of each affine candidate are shared for the children CUs in this root CU or the shared boundary. In one example, the derived control point MVs can be stored for the children CUs. For each child CU in the root CU or the shared boundary, the control point MVs of the root CU or the shared boundary are used to derive the control point MVs of the child CU or are used to derive the sub-block MVs of the child CU. In one example, the sub-block MVs of the child CU is derived from the control point MVs of child CUs, which are derived from the control MVs of the root CU or the shared boundary. In one example, the sub-block MVs of the child CU is derived from the control point MVs of the root CU or the shared boundary. In one example, the MVs of the sub-blocks in the root CU or the shared boundary can be derived at the root CU or the shared boundary. The derived sub-block MVs can be directly used. For the CU in the neighbouring CU outside of the root CU or the shared boundary, the control point MVs derived from the control point MVs of the root CU or the shared boundary are used to derive the affine inherited candidate. In another example, the control point MVs of the root CU or the shared boundary are used to derive the affine inherited candidate. In another example, the stored sub-block MVs of a CU are used to derive the affine inherited candidate. In another example, the stored sub-block MVs of the root CU or the shared boundary are used to derive the affine inherited candidate. In one embodiment, for a neighbouring reference CU in the above CTU row, the stored sub-block MVs (e.g. the bottom-left and bottom-right sub-block MVs, the bottom-left and bottom-centre sub-block MVs, or the bottom-centre and the bottom-right sub-block MVs) of the neighbouring reference CU are used to derive the affine inherited candidate instead of the control points of the root CU.

(44) In another example, when coding the child CU, the position and shape/width/height/size of the root CU or the shared boundary can be stored or derived for the affine candidate reference block derivation. The 4-parameter affine model (in equation (3)) and 6-parameter affine model (in equation (4)) can be used to derive the affine candidate or the control point MVs of the children CUs. For example, in FIG. 12A, the CU inside the root CU can reference blocks A.sub.0, A.sub.1, B.sub.0, B.sub.1, B.sub.2 and collocated block T.sub.BR and T.sub.CTR to derive the affine candidate. In another embodiment, for affine inherited candidate derivation, the current child CU position and shape/size/depth/width/ height is used. If the reference block is inside the root CU or the shared boundary, it is not used for deriving the affine candidate.

(45) $\begin{array}{cc}\{\begin{array}{c}{v}_x=\frac{\left(v_{1x}-v_{0x}\right)}{w}x-\frac{\left(v_{1y}-v_{0y}\right)}{w}y+v_{0x}\\ v_y=\frac{\left(v_{1y}-v_{0y}\right)}{w}x+\frac{\left(v_{1x}-v_{0x}\right)}{w}y+v_{0y}\end{array}& \left(3\right)\\ \{\begin{array}{c}{v}_x=\frac{\left(v_{1x}-v_{0x}\right)}{x_1-x_0}x+\frac{\left(v_{2x}-v_{0x}\right)}{x_2-x_0}y+v_{0x}\\ v_y=-\frac{\left(v_{1y}-v_{0y}\right)}{x_1-x_0}x+\frac{\left(v_{2y}-v_{0y}\right)}{y_2-x_0}y+v_{0y}\end{array}& \left(4\right)\end{array}$

(46) For affine corner derived candidate, the corner derived candidates for the child CU are not used according to one embodiment of the present invention. In another embodiment, the current child CU position and shape/size/depth/width/height is used. If the reference block/MV is inside the root CU or the shared boundary, it is not used for derive the affine candidate. In another embodiment, the shape/size/depth/width/height of the root CU or the shared boundary is used. The corner reference block/MV is derived based on the shape/size/depth/width/height of the root CU or the shared boundary. The derived MVs can be directly used as the control point MVs. In another embodiment, the corner reference block/MV is derived based on the shape/size/depth/width/height of the root CU or the shared boundary. The reference MV and its position can be used to derive the affine candidate by using the affine model (e.g. 4-parameter affine model or 6-parameter affine model). For example, the derived corner control pint MVs can be treated as the control point MVs of the root CU or the CU of the shared boundary. The affine candidate for child CU can be derived by using the equation (3) and/or (4). In other words, the affine parameters are generated based on neighbouring reference blocks on the shared boundary. Furthermore, the affine parameters may correspond to the gradients, starting MVs, and the locations of the neighbouring reference blocks as shown in the equation (3) and/or (4).

(47) The control point MVs of the constructed affine candidate of the root CU or the root shared boundary can be stored. For the child CU in the root CU or the shared boundary, the stored reference block position are used to find the reference block for affine candidate derivation. In another embodiment, the control point MVs of the root CU or the shared boundary of each affine candidates in the candidate list are derived. The control point MVs of the root CU or the shared boundary of each affine candidates are shared for the children CUs in this root CU or the shared boundary. In one example, the derived control-point MVs can be stored for the children CUs. For each child CU in the root CU or the shared boundary, the control-point MVs of the root CU or the shared boundary are used to derive the control-point MVs of the child CU or are used to derive the sub-block MVs of the child CU. In one example, the sub-block MVs of the child CU is derived from the control point MVs of child CUs, which are derived from the control MVs of the root CU or the shared boundary. In one example, the sub-block MVs of the child CU is derived from the control point MVs of the root CU or the shared boundary. In one example, the MVs of the sub-blocks in the root CU or the shared boundary can be derived at the root CU or the shared boundary. The derived sub-block MVs can be directly used. For the CU in the neighbouring CU outside of the root CU or the shared boundary, the control-point MVs derived from the control point MVs of the root CU or the shared boundary are used to derive the affine inherited candidate. In another example, the control-point MVs of the root CU or the shared boundary are used to derive the affine inherited candidate. In another example, the stored sub-block MVs of a CU are used to derive the affine inherited candidate. In another example, the stored sub-block MVs of the root CU or the shared boundary are used to derive the affine inherited candidate. In one embodiment, for a neighbouring reference CU in the above CTU row, the stored sub-block MVs (e.g. the bottom-left and bottom-right sub-block MVs, or the bottom-left and bottom-centre sub-block MVs, or the bottom-centre and the bottom-right sub-block MVs) of the neighbouring reference CU are used to derive the affine inherited candidate instead of the control points of the root CU or the shared boundary that contains the neighbouring reference CU, or instead of the control point MVs of the neighbouring reference CU.

(48) In another embodiment, the derived control point MVs from the root CU and the shared boundary can be used directly without affine model transforming.

(49) In another embodiment, for the proposed shared list methods (e.g. shared candidate list inside one sub-tree and common shared boundary), when deriving the reference block position, the current block position/size/depth/shape/width/height is used. However, if the reference block is inside of the root CU or the shared boundary, the reference block position is pushed or moved outside of the root CU or the shared boundary. For example, in FIG. 7, the block B.sub.1 is the above block of the top-right sample of the current block. If the block B.sub.1 is inside of the root CU or the shared boundary, the position of block B1 is moved above to the first nearest block out side of the root CU or the shared boundary. In another embodiment, when deriving the reference block position, the current block position/size/depth/shape/width/height is used. However, if the reference block is inside of the root CU or the shared boundary, the reference block/MV is not used (treated as unavailable). In another embodiment, when deriving the reference block position, the current block position/size/depth/shape/width/height is used. However, if the reference block is inside of the root CU or the shared boundary, the reference block/MV is not used (treated as unavailable).

(50) In one embodiment, for the temporal collocated MV derivation, the collocated MV of the root CU or the shared boundary is shared/used for all the children CUs. In another embodiment, for the temporal collocated MV derivation, the collocated MV of each CU/block is used instead of the shared temporal collocated MV.

(51) Method—MER and Shared List Both Existing for QTMTT Structure

(52) In this method, both MER (Merge Estimation Region) and shared list concept may both be enabled in QTMTT structure. Merge estimation region as referred in HEVC corresponds to a region that all leaf CUs inside this region can be processed in parallel. In other words, the dependency among the leaf CUs inside this region can be eliminated. The QTMTT corresponds to a type of multi-type tree (MTT) block partitioning where quadtree and another partition tree (e.g. binary tree (BT) or ternary tree (TT)) are used for MTT. In one embodiment, for normal Merge and ATMVP, the leaf CUs in the root CU uses shared list. But for affine merge, it uses QTMTT-based MER. In another embodiment, for some prediction mode, the leaf CUs in the root CU uses shared list, but for other Merge mode or AMVP mode, it uses MER concept.

(53) In one embodiment, the concept of the Merge Estimation Region (MER) in HEVC can be extend to the QTBT or the QTBTTT (quadtree/binary tree/ternary tree) structure. The MER can be non-square. The MER can be in different shape or size depending on the structure partition. The size/depth/area/width/height can be predefined or signalled in the sequence/picture/slice-level. For the width/height of the MER, the log 2 value of the width/height can be signalled. For the area/size of the MER, the log 2 value of the size/area can be signalled. When a MER is defined for a region, the CU/PU in this MER cannot be used as the reference CU/PU for Merge mode candidate derivation, such that the CU/PU in this MER is excluded from Merge mode candidate derivation. For example, the MVs or the affine parameters of the CU/PU in this MER cannot be referenced by the CU/PU in the same MER for Merge candidate or affine merge candidate derivation. Those MVs and/or affine parameters are treated as unavailable for the CU/PU in the same MER. For sub-block mode (e.g. ATMVP mode) derivation, the size/depth/shape/area/width/height of the current CU is used. If the reference CU is in the same MER, the MV information of the reference CU cannot be used.

(54) When a MER area/size/depth/shape/area/width/height is defined (e.g. predefined or signalled), if the current CU is larger than or equal to the defined area/size/shape/area/width/height and one of the child partition or all of the child partitions or part of the child partition is smaller than the area/size/shape/area/width/height the current CU is one MER. In another example, if the depth of the current CU is smaller than or equal to the defined depth and the depth of one of child partition, or all of the child partition or part of the child partition is larger than the defined depth, the current CU is one MER. In another embodiment, if the current CU is smaller than or equal to the defined area/size/shape/area/width/height and the parent CU is larger than the defined area/size/shape/area/width/height, the current CU is one MER. In another example, if the depth of the current CU is larger than or equal to the defined depth and the parent is smaller than the defined depth, the current CU is one MER. For example, if the defined area is 1024 and a CU size is 64×32 (width equal to 64 and height equal to 32), and the vertical TT split is used (e.g. the 64×32 CU partitioned into a 16×32 sub-CU, a 32×32 sub-CU, and a 16×32 sub-CU), the 64×32 block is one MER in one embodiment. The child CU in this 64×32 uses the shared list. In another embodiment, the 64×32 block is not the MER. However, the 16×32 sub-CU, the 32×32 sub-CU and the 16×32 sub-CU are MERs, respectively. In another embodiment, for a defined MER area/size/depth/shape/area/width/height, when doing the TT split, the MER area/size/depth/shape/area/width/height can be different in different TT partition. For example, for the first and the third partitions, the threshold of MER area/size/shape/area/width/height can be divided by 2 (or the depth increased by 1). For the second partition, the threshold of MER area/size/depth/shape/area/width/height keeps the same.

(55) Method—Reduced Merge Candidate List Depending on CU Dimension

(56) This method is applied to Merge mode or other candidate list modes, such as affine Merge list (only affine Merge candidates), or unified affine-ATMVP Merge list (grouping affine Merge candidate and ATMVP candidate into a list), or ATMVP-normal Merge list (the translational Merge candidate and ATMVP candidate in one list), and so on.

(57) In another embodiment, it removes some candidates according to CU size or any other CU-size related feature. If a CU size is larger than a pre-defined threshold (e.g. area=16), some candidates (e.g. ATMVP) are removed (or called excluded) from the construction of the candidate list

(58) In another embodiment, the turning-off of ATMVP depends on the width or height of the current CU, which also relate to CU size. In one example, if the CU width is smaller than a pre-defined threshold, the ATMVP is turned off. In another example, if the CU height is smaller than a pre-defined threshold, the ATMVP is turned off. In another embodiment, if the CU height is smaller than a pre-defined threshold or the CU width is smaller than another pre-defined threshold, the ATMVP is turned off. In another embodiment, if the CU height is smaller than a pre-defined threshold and the CU width is smaller than another pre-defined threshold, the ATMVP is turned off.

(59) In one example, the ATMVP is turned off (i.e., disabled or excluded) when the width is smaller than 8 or the height is smaller than 8.

(60) In another embodiment, the turning-off of ATMVP depends on the width or height of the current CU. For example, if the CU width is larger than a pre-defined threshold, the ATMVP is turned off. In another example, if the CU height is larger than a pre-defined threshold, the ATMVP is turned off. In another embodiment, if the CU height is larger than a pre-defined threshold or the CU width is larger than another pre-defined threshold, the ATMVP is turned off. In another embodiment, if the CU height is larger than a pre-defined threshold and the CU width is larger than another pre-defined threshold, the ATMVP is turned off.

(61) In another embodiment, the turning-off of ATMVP depends on the shape of current CU. For example, if the CU aspect ratio (i.e., width/height or height/width) is smaller than a pre-defined threshold, the ATMVP is turned off. In another example, if the CU aspect ratio (i.e., width/height or height/width) is larger than a pre-defined threshold, the ATMVP is turned off.

(62) The size threshold, width threshold or height threshold can be fixed and pre-defined for all picture size and all bit-stream. In another embodiment, the size threshold, width threshold or height threshold can be determined adaptively according to the picture size. For different picture sizes, the thresholds may be different. In another embodiment, the size threshold, width threshold or height threshold can be signalled from the encoder to the decoder. The minimum sizes of units for signalling the size threshold/width threshold/height threshold can also be separately coded in the sequence level, picture level, slice level or PU level.

(63) Shared Merge List MV for the Merge/Inter/Affine-Merge/Affine-Inter/ATMVP/Sub-Block Candidate List Construction

(64) The candidate list generated at the root CU or the shared boundary can be used for the Merge/Inter/affine-Merge/affine-Inter/ATMVP/sub-block candidate list construction of the children CUs even when the shared list is not enabled. The candidates of the root CU or the shared boundary can be added into the candidate list of the children CUs. The shape/size/depth/width/height of the root CU or the shared boundary can be predefined, signalled (e.g. in the sequence/picture/slice/tile/CTU-row/CTU-level), or derived. For example, the root CU can be the parent N-level CU. The N can be an integer.

(65) In one embodiment, two thresholds can be defined: one is larger and one is smaller. A larger root CU or a larger shared boundary is defined/determined by the larger threshold. A candidate list is generated at the larger root CU or the larger shared boundary. For all the child CUs in the larger root CU or the larger shared boundary, the candidates of the larger root CU or the larger shared boundary can be added into the candidate list of the children CUs. A smaller root CU or a smaller shared boundary is defined/determined by the smaller threshold. A candidate list is generated at the smaller root CU or the smaller shared boundary. When generating the candidate list of the smaller root CU or the smaller shared boundary, the candidates of the larger root CU or the larger shared boundary can be added. For the children CUs in the smaller root CU or the smaller shared boundary, the candidate list generated at the smaller root CU or the smaller shared boundary is used.

(66) The foregoing proposed methods can be implemented in encoders and/or decoders. For example, any of the proposed methods can be implemented in an entropy encoding module or a block partition module in an encoder, and/or an entropy parser module or a block partition module in a decoder. Alternatively, any of the proposed methods can be implemented as a circuit coupled to the entropy encoding module or the block partition module in the encoder, and/or the entropy parser module or the block partition module in the decoder, so as to provide the information needed by the entropy parser module or the block partition module.

(67) FIG. 14 illustrates a flowchart of an exemplary Inter prediction for video coding, wherein said video coding allows sub-block mode motion vector prediction according to an embodiment of the present invention. The steps shown in the flowchart, as well as other following flowcharts in this disclosure, may be implemented as program codes executable on one or more processors (e.g., one or more CPUs) at the encoder side and/or the decoder side. The steps shown in the flowchart may also be implemented based hardware such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to this method, input data related to a current block in a current picture are received at a video encoder side or a video bitstream corresponding to compressed data including the current block in the current picture are received at a video decoder side are received in step 1410. A condition is determined based on a block size of the current block in step 1420. If the condition is satisfied, the current motion information associated with the current block is encoded or decoded using a coding tool set excluding a target sub-block mode coding tool in step 1430. If the condition is not satisfied, the current motion information associated with the current block is encoded or decoded using the coding tool set including the target sub-block mode coding tool in step 1440.

(68) FIG. 15 illustrates a flowchart of an exemplary Inter prediction for video coding, wherein neighbouring blocks inside one root region are cancelled or pushed for Merge candidate according to an embodiment of the present invention. According to this method, input data related to a current block in a current picture are received at a video encoder side or a video bitstream corresponding to compressed data including the current block in the current picture are received at a video decoder side are received in step 1510, where the current block corresponds to one target leaf block among multiple leaf blocks under a root node resulted from block partitioning of an area including the current block. An MER (Merge Estimation Region) enclosing said multiple leaf blocks is determined in step 1520. If a reference block for the current block is inside the MER, a target candidate associated with the reference block is excluded from a Merge candidate list or a modified target candidate is included in the Merge candidate list in step 1530, wherein the modified target candidate is derived based on a modified reference block outside the MER. Current motion information associated with the current block is encoded or decoded using the Merge candidate list in step 1540.

(69) FIG. 16 illustrates a flowchart of an exemplary Inter prediction for video coding, wherein child blocks inside one root region share a common Merge candidate list according to an embodiment of the present invention. According to this method, input data related to a current CU (coding unit) in a current picture are received at a video encoder side or a video bitstream corresponding to compressed data including the current CU in the current picture is received at a video decoder side in step 1610, where the current CU corresponds to one target child CU among multiple child CUs under a root node resulted from block partitioning of an area including the current CU are received. A shared boundary enclosing said multiple child CUs is determined in step 1620. A shared Merge list is derived based on one or more neighbouring reference blocks located adjacent to the shared boundary in step 1630. Current motion information associated with said multiple child CUs is encoded or decoded using the shared Merge list in step 1640.

(70) The flowcharts shown are intended to illustrate an example of video coding according to the present invention. A person skilled in the art may modify each step, re-arranges the steps, split a step, or combine steps to practice the present invention without departing from the spirit of the present invention. In the disclosure, specific syntax and semantics have been used to illustrate examples to implement embodiments of the present invention. A skilled person may practice the present invention by substituting the syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.

(71) The above description is presented to enable a person of ordinary skill in the art to practice the present invention as provided in the context of a particular application and its requirement. Various modifications to the described embodiments will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed. In the above detailed description, various specific details are illustrated in order to provide a thorough understanding of the present invention. Nevertheless, it will be understood by those skilled in the art that the present invention may be practiced.

(72) Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be one or more circuit circuits integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware code may be developed in different programming languages and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

(73) The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.