Method for encoding and decoding images, device for encoding and decoding images and corresponding computer programs
11350106 · 2022-05-31
Assignee
Inventors
Cpc classification
H04N19/88
ELECTRICITY
H04N19/126
ELECTRICITY
H04N19/45
ELECTRICITY
International classification
H04N19/88
ELECTRICITY
H04N19/12
ELECTRICITY
H04N19/126
ELECTRICITY
Abstract
A method for decoding a data signal representative of at least one image cut into blocks. The method includes, for a current block to be decoded: transforming a residual data block obtained from the data signal; determining a permutation-transformation pair including a transformation mathematical operation and a permutation mathematical operation, the transformation mathematical operation and the permutation mathematical operation belonging to respectively a plurality of permutation operations and a plurality of transformation operations; applying the determined transformation mathematical operation to the data of the residual data block to produce transformed data; applying the determined permutation mathematical operation to the transformed data to produce permuted data, the determined permutation mathematical operation being an operation of transposing the transformed data; and using the permuted data to reconstruct the current block by a predictive decoding.
Claims
1. A method for decoding a data signal representative of at least one image cut into blocks, wherein the method includes, for a current block to be decoded, the following acts performed by a decoding device: determining, in said data signal, a residual data block associated with the current block to be decoded, said residual data block having data rows and data columns, determining a permutation-transformation pair comprising a transformation mathematical operation and a permutation mathematical operation to be applied in combination to said residual data block, said transformation mathematical operation and said permutation mathematical operation belonging to respectively at least one transformation operation and a plurality of permutation operations, firstly transforming the data of said residual data block by the determined transformation mathematical operation to produce transformed data, said determined transformation mathematical operation being applied to said data rows and data columns of said residual data block, secondly permuting the transformed data by the determined permutation mathematical operation to produce data transformed then permuted, said determined permutation mathematical operation being an operation of transposing said transformed data, and using said data transformed then permuted to reconstruct said current block by a predictive decoding.
2. The method of decoding as claimed in claim 1, in which the act of determining said pair of mathematical operations implements the following acts: reading, in a data signal, information representative of said pair of permutation and transformation operations, determining the inverse permutation and transformation operations of the permutation and transformation operations of the pair associated with the information read.
3. The method of decoding as claimed in claim 1, in which the act of determining said permutation-transformation pair comprises: applying to said data of said residual data block a plurality of pairs of mathematical operations including a transformation operation followed by a permutation operation, said transformation and permutation operations being chosen from among the at least one transformation operation and the plurality of permutation operations, reconstructing a plurality of blocks from said plurality of pairs applied, selecting a reconstructed block from among the plurality of reconstructed blocks, by maximizing a criterion of coherence applied to the plurality of reconstructed blocks.
4. The method of decoding as claimed in claim 1, in which said determined transformation operation is a discrete sine transform.
5. The method of decoding as claimed in claim 1, in which said operation of transposing comprises gradually moving said data obtained as a result of applying the transformation operation of the determined pair.
6. The method as claimed in claim 1, wherein the plurality of permutation operations comprises at least one matrix containing only 1s and 0s, and the at least one transformation operation comprises at least one 1-dimension transform of size M×M operating on vectors of size M, M being an integer variable.
7. A device for decoding a data signal representative of at least one image cut into blocks, including, for a current block to be decoded: a processor; and a non-transitory computer-readable medium comprising instructions stored thereon, which when executed by the processor configure the device for decoding to perform acts comprising: determining, in said data signal, a residual data block associated with the current block to be decoded, said residual data block having data rows and data columns, determining a permutation-transformation pair comprising a transformation mathematical operation and a permutation mathematical operation to be applied in combination to said residual data block, said transformation mathematical operation and said permutation mathematical operation belonging to respectively at least one transformation operation and a plurality of permutation operations, firstly transforming the data of said residual data block by the determined transformation mathematical operation to produce transformed data, said determined transformation mathematical operation being applied to said data rows and data columns of said residual data block, secondly permuting the transformed data by the determined permutation mathematical operation to produce data transformed then permuted, said determined permutation mathematical operation being an operation of transposing said transformed data, and using said data transformed then permuted to reconstruct said current block by a predictive decoding.
8. The device as claimed in claim 6, wherein the plurality of permutation operations comprises at least one matrix containing only 1s and 0s, and the at least one transformation operation comprises at least one 1-dimension transform of size M×M operating on vectors of size M, M being an integer variable.
9. A non-transitory computer-readable medium comprising a computer program stored thereon, which comprises instructions for implementing a method of decoding a data signal representative of at least one image cut into blocks, when said method of decoding is executed on a computer of a decoding device, wherein the instructions configure the decoding device to perform the following acts for a current block to be decoded: determining, in said data signal, a residual data block associated with the current block to be decoded, said residual data block having data rows and data columns, determining a permutation-transformation pair comprising a transformation mathematical operation and a permutation mathematical operation to be applied in combination to said residual data block, said transformation mathematical operation and said permutation mathematical operation belonging to respectively at least one transformation operation and a plurality of permutation operations, firstly transforming the data of said residual data block by the determined transformation mathematical operation to produce transformed data, said determined transformation mathematical operation being applied to said data rows and data columns of said residual data block, and secondly permuting the transformed data by the determined permutation mathematical operation to produce data transformed then permuted, said determined permutation mathematical operation being an operation of transposing said transformed data, and using said data transformed then permuted to reconstruct said current block by a predictive decoding.
10. The non-transitory computer-readable medium as claimed in claim 9, wherein the plurality of permutation operations comprises at least one matrix containing only 1s and 0s, and the at least one transformation operation comprises at least one 1-dimension transform of size M×M operating on vectors of size M, M being an integer variable.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Other features and advantages will appear on reading a preferred embodiment described with reference to the figures in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS
(10) Detailed Description of the Encoding Portion
(11) One embodiment of the invention will now be described, in which the method of encoding according to the invention is used for encoding an image or a sequence of images in a binary stream similar to that obtained by an encoding compliant, for example, with the HEVC standard.
(12) In this embodiment, the method of encoding according to the invention is, for example, implemented in software or hardware by modifications to an encoder initially compliant with the HEVC standard. The method of encoding according to the invention is represented in the form of an algorithm comprising steps C1 to C15 as represented in
(13) According to the embodiment of the invention, the method of encoding according to the invention is implemented in an encoding device CO represented in
(14) As illustrated in
(15) The method of encoding represented in
(16) In the course of a step C1 represented in
(17) It should be noted that within the meaning of the invention, the term “block” means “coding unit” (from the English term). The latter terminology is used notably in the HEVC standard, e.g. in the document “ITU-T Recommendation H.265/Standard ISO/IEC 23008-2”.
(18) In particular, such a coding unit groups together sets of pixels of rectangular or square shape, also called blocks, macroblocks, or sets of pixels exhibiting other geometric shapes.
(19) Preferably in the invention, the blocks to be encoded are particularly of a size of a multiple power of 2, e.g. of type 4×4 and 8×8. However, the invention also applies more generally to blocks of size M×M (M=16, 32 or 64). P×M and M×P blocks are also usable with P and M taking their value in the set 2, 4, 8, 16, 32 and 64.
(20) Said blocks B.sub.1, B.sub.2, . . . , B.sub.u, . . . , B.sub.S are intended to be encoded according to a predetermined scanning order, which is, for example, of the raster scan type. This means that the blocks are encoded one after the other, from left to right, then downward.
(21) Other types of scanning are, of course, possible. Thus, it is possible to cut the image IC.sub.j into multiple subimages called slices and independently apply this type of cutting on each subimage. It is also possible to encode not a succession of rows, as explained above, but a succession of columns. It is also possible to scan the rows or columns in one direction or the other.
(22) Each block may itself be further divided into subblocks which are themselves subdivisible.
(23) In the course of a step C2 represented in
(24) In the course of a step C3 represented in
(25) Such a predictor block is, for example, a block of pixels which has already been encoded or encoded then decoded or not decoded. Such a predictor block is previously stored in the buffer memory MT_CO of the encoder CO as represented in
(26) At the end of the prediction step C3, an optimal predictor block BP.sub.opt is obtained after said predetermined modes of prediction have been placed in competition, e.g. by minimizing a rate-distortion criterion well known to the person skilled in the art. The block BP.sub.opt is considered as an approximation of the current block B.sub.u. The information items relating to this prediction are intended to be included in a signal or data stream to be transmitted to a decoder. Such information items notably include the type of prediction (inter or intra), and, where applicable, the mode of prediction selected, the type of partitioning of the current block if the latter has been subdivided, the reference image index and the motion vector used in the case where a mode of inter prediction has been selected. These information items are compressed by the encoder CO.
(27) In the course of a step C4 represented in
(28) A set of residual data, called a residual block Br.sub.u is then obtained at the end of step C4.
(29) Steps C3 and C4 are implemented by a predictive encoding software module PRED_CO represented in
(30) In the course of a step C5 represented in
(31) Such a step C5 is implemented by a calculation software module CAL_CO represented in
(32) The plurality of transformation operations OT.sub.0, OT.sub.1, . . . , OT.sub.n, . . . , OT.sub.L-1 includes one or more 1D transforms.
(33) A 1D transform means a discrete transform of size M×M operating on vectors of size M. For this purpose:
(34) a first transformation operation is applied vector by vector on a block of size M×M,
(35) a transposition operation is applied on the transformed block,
(36) a second transformation operation is applied on the block obtained after transposition, also vector after vector.
(37) This 1D transform rests on bases expressed in trigonometric form in the way that discrete sine (DST) or discrete cosine (DCT) transforms are. Preferably, the transforms belonging to the plurality of transforms OT.sub.0, OT.sub.1, . . . , OT.sub.n, . . . , OT.sub.L-1 are DCT II, III and IV transforms and DST IV, VI and VII transforms, which are described in the document “Relationship between DCT-II, DCT-VI, and DST-VII transforms”, Reznik ICASSP 2013.
(38) Among these, DCT II, III, and DST VII and IV transforms are particularly useful for their ability to make the residual signal compact in the frequency domain.
(39) Such transforms are, for example, of size 4×4, 8×8 etc.
(40) As a variant, the transforms used may be alternating, i.e. a transform of a first type and a transform of a second type are applied respectively on the rows and on the columns of the current residual block Br.sub.u of which the data have been previously permuted as will be described below in the description.
(41) According to this variant, the aforementioned transforms may also be rectangular. For a current residual block Br.sub.u of rectangular shape, the row transform will have a different size from the column transform.
(42) It should be noted that the combination of the row transform and the column transform may be applied directly to the current residual block Br.sub.u.
(43) In the field of video encoding/decoding, with a view to making an implementation possible on fixed point processors, such transforms are approximated, i.e. the real values are approximated by rational numbers. Therefore the transforms are not exactly retrieved in the strict sense but rather in an approximated form.
(44) Thus, for example, in the case of a DST VII transform of size 4, the latter may be expressed according to the equation:
(45)
(46) The following 4×4 matrix of coefficients is then obtained, with i evolving along the columns and k evolving along the rows:
(47) TABLE-US-00001 0.2280 0.4285 0.5774 0.6565 0.5774 0.5774 0.0000 0.5774 0.6565 0.2280 0.5774 0.4285 0.4285 0.6565 0.5774 0.2280
(48) Said coefficient values above may also be represented in the form of fractions in the following way, which makes an effective fixed point implementation possible:
(49) TABLE-US-00002 29/128 55/128 74/128 84/128 74/128 74/128 0 −74/128 84/128 −29/128 −74/128 55/128 55/128 −84/128 74/128 −29/128
(50) The plurality of permutation operations OP.sub.0, OP.sub.1, . . . , OP.sub.m, . . . , OP.sub.K-1 includes one or more permutation operations. Within the meaning of the invention, a permutation operation, an example of which is represented below, is defined in the form of a matrix containing only 1s and 0s. In the example represented below, such a permutation matrix is applied to a set E.sub.EN of successive coefficients, e.g. sixteen in number for a block of size 4×4, of respective indices I.sub.0, I.sub.1, I.sub.2, I.sub.3, I.sub.4, I.sub.5, I.sub.6, I.sub.7, I.sub.8, I.sub.9, I.sub.10, I.sub.11, I.sub.12, I.sub.13, I.sub.14, I.sub.15.
(51) TABLE-US-00003 E.sub.SO permutation matrix E.sub.EN I.sub.3 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 I.sub.0 I.sub.2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 I.sub.1 I.sub.1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I.sub.2 I.sub.0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I.sub.3 I.sub.7 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 I.sub.4 I.sub.6 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 I.sub.5 I.sub.5 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 I.sub.6 I.sub.4 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 I.sub.7 I.sub.11 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 I.sub.8 I.sub.10 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 I.sub.9 I.sub.9 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 I.sub.10 I.sub.8 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 I.sub.11 I.sub.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 I.sub.12 I.sub.14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 I.sub.13 I.sub.13 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 I.sub.14 I.sub.12 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 I.sub.15
(52) At the end of the permutation operation above, a set E.sub.SO of permuted indices I.sub.3, I.sub.2, I.sub.1, I.sub.0, I.sub.7, I.sub.6, I.sub.5, I.sub.4, I.sub.11, I.sub.10, I.sub.9, I.sub.8, I.sub.15, I.sub.14, I.sub.13, I.sub.12 is obtained. Such a permutation operation has the advantage of not generating calculating complexity in the encoder CO in
(53) More precisely in the course of the aforementioned step C5, the choice is made of the permutation-transformation pair to be used which optimizes the encoding performance of the current block B.sub.u to be encoded. Such a choice may be made in several ways.
(54) According to one embodiment, the predetermined criterion of choice used in step C5 is the rate-distortion criterion well known to the person skilled in the art which consists in measuring a weighting of the distortion after quantization with the transmission cost of the quantized samples. Typically, for a permutation OP.sub.m—transformation OT.sub.n pair considered, the criterion J.sub.m,n is expressed in the following way:
J.sub.m,n(lambda)=∥x−OP.sub.m.sup.tOT.sub.n.sup.−1└Xm,n┘∥.sup.2+lambda.Math.R(└Xm,n┘), where:
(55) Lambda is a Lagrangian weighting operator governing the transmission rate constraint,
(56) ( ).sup.t is the transposition matrix operator,
(57) OT.sup.−1.sub.n represents the inverse transformation,
(58) └X.sub.m,n┘ represents the approximation of X.sub.m,n performed according to the quantization operator, X.sub.m,n being the result of the transformation of the current block B.sub.u to be encoded by the pair of operations OT.sub.n, OP.sub.m considered.
(59) R( ) is a function calculating the rate required for transmitting the quantized values.
(60) The function R may be obtained by complete encoding of the information items, i.e. by way of counting the information items necessary for transmitting the quantized and encoded values, or by an estimate of the rate, e.g. which may based on a calculation of the sum of the absolute values of the quantized values or on a count of the values quantized to zero.
(61) Following a comparison between the various values obtained for J.sub.m,n, the lowest value calculated by the calculation module CAL_CO in
(62) In a simplified embodiment of the invention, the plurality of permutation operations includes, for example, two permutations OP.sub.0 and OP.sub.1, while the plurality of transformation operations includes a single transformation OT.sub.0, e.g. the DST VII. For this purpose, in the course of the aforementioned step C5, only two rate-distortion criteria are calculated, namely:
(63) the criterion J.sub.0,0 which depends on the permutation OP.sub.0 and the transformation OT.sub.0,
(64) and the criterion J.sub.1,0 which depends on the permutation OP.sub.1 and the transformation OT.sub.0.
(65) The calculation module CAL_CO in
(66) According to one variant of the embodiment of the aforementioned selection step C5, the latter may be performed in conjunction with the search for the optimal predictor block B.sub.opt during the aforementioned prediction step C3. More precisely, for each mode of prediction considered, one permutation-transformation pair is determined as minimizing the aforementioned rate-distortion criterion J.sub.m,n. In this way, the rate information items during the estimation of the criterion J.sub.m,n take into account the transmission of the mode of prediction. Thus, all the prediction/permutation-transformation combinations are evaluated for retaining that which minimizes the rate-distortion criterion.
(67) Such a variant may be further extended by taking into account various possible sizes of the current block. The utility of a partitioning of the current block into subblocks may be decided taking into account the rate-distortion criterion, by taking into account the available permutation-transformation pairs.
(68) Examples of transformation-permutation pairs are represented with reference to
(69) In the example of
(70) In the example of
(71) With reference to
(72) Each column P.sub.0 to P.sub.27 is associated with a particular DST or DCT transform according to the permutation considered, said particular transform being indicated in the last row of table T1.
(73) With reference to
(74) In the example of
(75) The permutations indicated in tables T1 and T2 are established by learning. For this purpose, the permutations are derived from simple geometric transformations: order inversion of columns/rows, movement of blocks, movement of rows or columns.
(76) Some permutations are determined for their simplicity of implementation, in the sense that the transformations considered are slightly affected when the inter-pixel relationship is not modified, i.e. after permutation, each pixel remains close to its neighbors before permutation. A preferred permutation is, for example, a transposition operator which gradually moves the data of the pixel block, in a predetermined direction. Other permutations are envisaged.
(77) For example, in table T1 represented in
(78) the permutation P.sub.1 is a mirror operator of the pixel block, the pixel indices being inverted with respect to each other by means of a vertical symmetry,
(79) the permutation P.sub.4 makes it possible to invert the pixel indices by transposition, i.e. diagonal symmetry, with respect to a 45° angle dividing the residual block Br.sub.u.
(80) The various permutations have been learned through a study of the intrinsic properties of the residual blocks. According to the luminous power of the average residue per pixel, the pixels are reordered so that they are compatible with a particular transformation. It is known in the prior art that the DCT is appropriate for encoding residues having a profile of constant brightness, i.e. the residual block has a relatively uniform brightness per pixel. The DST has an appropriate profile for residual blocks having less brightness in the pixels located at the top of the block (e.g. indices I.sub.0 to I.sub.3 in
(81) Thus, in particular in table T1 of
(82) The “identity” permutation P.sub.0 forms part of the permutations retained, when the residual block Br.sub.u has a shape compatible with the associated transformation. On the other hand, when the residual block is incompatible with the associated transformation, then the aforementioned rate-distortion criterion J.sub.m,n will have a much lower value with a permutation other than the permutation P.sub.0.
(83) Other configurations of permutation-transformation pairs are, of course, possible and some will be mentioned farther on in the description.
(84) In the course of a step C6 represented in
(85) Step C6 is implemented by a permutation software module MPER_CO, as represented in
(86) In the course of a step C7 represented in
(87) Step C7 is implemented by a transformation software module MT_CO, as represented in
(88) The transformation operation OT.sub.n* consists in transforming the permuted block Bp.sub.u from the spatial domain to the frequency domain in the following form:
X.sub.Bt.sub.
(89) where:
(90) x.sub.Bp.sub.
(91) X.sub.Bt.sub.
(92) B and A represent 1D transforms from the families presented above.
(93) Note that the result of the transform may also be written in the mathematically equivalent form:
X=Sx
(94) where:
(95) X and x then appear in the form of vectors,
(96) S represents a transform matrix which contains the square of the number of elements contained in x. The matrix S is written as a generalized product of the 1D matrices B and A. In other words the matrix S is formed by constructing an M×M matrix (M is the size of the 1D transforms), constructed by repetition of the matrix A by blocks and multiplication of each of the matrices A by a term of the matrix B. As illustrated below, for the case M=4, the matrix S of type 16×16 is then obtained:
(97) TABLE-US-00004 B.sub.0, 0*A B.sub.0, 1*A B.sub.0, 2*A B.sub.0, 3*A B.sub.1, 0*A B.sub.1, 1*A B.sub.1, 2*A B.sub.1, 3*A B.sub.2, 0*A B.sub.2, 1*A B.sub.2, 2*A B.sub.2, 3*A B.sub.3, 0*A B.sub.3, 1*A B.sub.3, 2*A B.sub.3, 3*A
(98) In a preferred embodiment, said matrix S is a matrix representative of a DST.
(99) The permutation operation OP.sub.m* consists in applying a permutation on the entries of the matrix X.sub.Br.sub.
(100) TABLE-US-00005 Bp.sub.u permutation matrix P.sub.m Br.sub.u I.sub.3 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 I.sub.0 I.sub.2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 I.sub.1 I.sub.1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I.sub.2 I.sub.0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I.sub.3 I.sub.7 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 I.sub.4 I.sub.6 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 I.sub.5 I.sub.5 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 I.sub.6 I.sub.4 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 I.sub.7 I.sub.11 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 I.sub.8 I.sub.10 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 I.sub.9 I.sub.9 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 I.sub.10 I.sub.8 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 I.sub.11 I.sub.15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 I.sub.12 I.sub.14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 I.sub.13 I.sub.13 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 I.sub.14 I.sub.12 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 I.sub.15
(101) The relationship X.sub.Bt.sub.
(102) where P.sub.m represents a permutation matrix of the indices of x, which has the effect of an inversion of the pixel positions.
(103) In the course of a step C8 represented in
(104) In the course of a step C9 represented in
(105) reading the symbol or symbols which are associated with said current block,
(106) associating digital information, such as bits, with the symbol(s) read.
(107) Such an entropy encoding step is implemented by an entropy encoding software module MCE represented in
(108) In the course of said step C9, the index m*,n* of the criterion J.sub.m*,n* retained in the aforementioned step C5 is optionally encoded. Such a step is necessary, when the decoder is not capable of retrieving this information.
(109) If m*,n* must be transmitted to the decoder, it is then, for example, expressed in a fixed-length binary code.
(110) If, for example, four values of permutation-transformation pairs: 0, 1, 2, 3 must be signaled,
(111) the value 0 is associated, for example, with the code 00,
(112) the value 1 is associated, for example, with the code 01,
(113) the value 2 is associated, for example, with the code 10,
(114) the value 3 is associated, for example, with the code 11.
(115) As a variant, the index m*,n* may be expressed with the aid of a variable-length code.
(116) In the case, for example, of four values of permutation-transformation pairs: 0, 1, 2, 3 to be signaled:
(117) the value 0 is associated, for example, with the code 0,
(118) the value 1 is associated, for example, with the code 10,
(119) the value 2 is associated, for example, with the code 110,
(120) the value 3 is associated, for example, with the code 111.
(121) According to a preferred embodiment, the transmission of the index m*,n* is separated into two indicators:
(122) a flag signaling the first bit of the codes above,
(123) and a complementary syntax element which indicates the less probable permutation-transformation pairs in binary form.
(124) The flag is advantageously encoded with the aid of an arithmetic encoder separate from the encoding of the pair number retained. A CABAC encoder is preferably used, with a particular context according to the size of the subdivision of the current block. It is thus possible to obtain a significant reduction in the cost of this flag when the latter is very frequently employed in a given state, i.e. its probability of being ‘0’ or ‘1’ is very high.
(125) Thus, according to this preferred mode:
(126) the value 0 is associated, for example, with the flag set to 0,
(127) the value 1 is associated, for example, with the flag set to 1, with complement 0,
(128) the value 2 is associated, for example, with the flag set to 1, and complement 10,
(129) the value 3 is associated, for example, with the flag set to 1, and complement 11.
(130) According to another preferred embodiment, the index m*,n* is transmitted only by means of a flag. This preferred mode is applicable in the case where a single transform, e.g. a DST, is applied and where two permutation operations OP.sub.0 and OP.sub.1 may be applied in combination with the DST. The permutation-transformation pair SOP.sub.0 or SOP.sub.1 selected in the aforementioned step C5 will then be signaled by one bit. According to a given convention by way of example:
(131) the selection of the pair SOP.sub.0 is represented by one bit set to ‘1’,
(132) the selection of the pair SOP.sub.1 is represented by one bit set to ‘0’.
(133) In this preferred embodiment, a CABAC encoder is preferably used, with a particular context according to the size of the subdivision of the current block. It is thus possible to obtain a significant reduction in the cost of this flag when the latter is very frequently employed in a given state, i.e. its probability of being ‘0’ or ‘1’ is very high.
(134) In the case where more than two permutation-transformation pairs may be selected, a fixed- or variable-length code will be used.
(135) According to another possible embodiment, in the course of step C9, a partial encoding of the value of the index m*,n* is performed. As will be described farther on in the description, such an embodiment is possible, when the decoder is itself capable of retrieving all or part of the index m*,n*.
(136) As a variant, this other embodiment may also be combined with the signaling implemented in the preferred embodiment described earlier.
(137) According to a particular configuration given by way of example:
(138) in the case where a single transform S, e.g. a DST, is applied and where a permutation operation OP.sub.0 is applied in combination with the DST, the permutation-transformation pair SOP.sub.0 selected in the aforementioned step C5 will then be signaled by one bit set to ‘0’,
(139) in the case where a single transform S, e.g. a DST, is applied and where multiple permutation operations may be applied in combination with the DST, the permutation-transformation pair selected in the aforementioned step C5 will then be signaled by one bit set to ‘1’.
(140) At the end of the encoding step C9, a signal or data stream F which contains the set of the encoded data of the block of quantized coefficients Bq.sub.u and optionally the entirety or one part of the index m*,n* encoded is then delivered. Such a stream is then transmitted via a communication network (not represented), to a remote terminal. This comprises the decoder DO represented in
(141) In the course of a step C10 represented in
(142) Such a dequantization step is performed by an inverse quantization software module MQ.sup.−1_CO, as represented in
(143) In the course of a step C11 represented in
(144) Step C11 is implemented by a transformation software module MT.sup.−1_CO, as represented in
(145) In the course of a step C12 represented in
(146) Step C12 is implemented by a permutation software module MPER.sup.−1_CO, as represented in
(147) In the course of a step C13 represented in
(148) Such a step is implemented by an inverse predictive encoding software module PRED.sup.−1_CO represented in
(149) In the course of a step C14 represented in
(150) If the current block is the last block of the image IC.sub.j, in the course of a following step C15 represented in
(151) If such is not the case, the step C2 of selecting the following block to be encoded is repeated in accordance with the aforementioned raster scan scanning order, then the steps C3 to C14 are repeated for this following block selected.
(152) The encoding steps that have just been described above are implemented for all the blocks B.sub.1, B.sub.2, . . . , B.sub.u, . . . , B.sub.S to be encoded of the current image IC.sub.j considered.
(153) Particular Configurations of Permutation-Transformation Pairs
(154) Other possible configurations of permutation-transformation pairs will now be mentioned below by way of additional examples.
(155) According to one embodiment, rather than using the set of permutations and transformations presented above, notably in connection with
(156) Thus, for blocks of size 4×4, for example, various compromises, according to the number of permutation-transformation pairs, are provided below.
(157) For this purpose, a 1+Y mode is determined which defines a number of different permutation-transformation pairs to be used.
(158) For Y=0, the 1+0 mode defines a pair including one transformation and one “identity” permutation P.sub.0.
(159) For Y>0, the 1+Y mode defines multiple pairs formed from a combination of a plurality of transformations and a plurality of permutations.
(160) By way of examples:
(161) for Y=1, the 1+1 mode is a mode that defines two possible transformation and permutation pairs, where two different transformations are provided: one pair formed of one “identity” permutation P.sub.0 and one DST transformation and another pair formed of one “identity” permutation P.sub.0 and one DCT transformation,
(162) for Y=2, the 1+2 mode is a mode that defines three possible transformation and permutation pairs, where two different transformations and three different permutations P.sub.0, P.sub.1, P.sub.2 are provided so as to constitute the following three permutation-transformation pairs: P.sub.0-DST, P1-DST, P.sub.2-DCT.
(163) The aforementioned 1+0, 1+1 and 1+2 modes offer various compromises of complexity for reducing the number of transformations to be used, thereby reducing the number of pairs to be tested during the encoding for selecting the best permutation-transformation pair.
(164) Thus in the 1+1 mode, in the course of the selection step C5 represented in
(165) In the 1+2 mode, in the course of the aforementioned selection step C5, only three rate-distortion criteria need to be compared since there are three permutation-transformation pairs in such a mode.
(166) The storage of the permutations may also be reduced in the case where the number of permutation-transformation pairs is limited.
(167) A set of compromises according to the number of permutation-transformation pairs made available in the encoder CO and according to the size of the blocks is represented below.
(168) The P.sub.0+dst mode, i.e. a purely dst approach, is placed in competition with a set of permutation-transformation pairs based on other transformations, such as the DCT, for example.
(169) TABLE-US-00006 Mode 1 + 1 1 + 2 1 + 3 1 + 4 1 + 8 1 + 16 pair Perm Transf. Perm Transf. Perm Transf. Perm Transf. Perm Transf. Perm Transf. 0 P.sub.0 dst P.sub.0 dst P.sub.1 dst P.sub.0 dst P.sub.0 dst P.sub.0 dst 1 P.sub.0 dct P.sub.1 dst P.sub.27 dst P.sub.2 dst P.sub.5 dst P.sub.11 dst 2 P.sub.2 dct P.sub.6 dst P.sub.1 dst P.sub.6 dst P.sub.12 dst 3 P.sub.3 dct P.sub.7 dst P.sub.13 dst 4 P.sub.4 dct P.sub.1 dst P.sub.14 dst 5 P.sub.8 dct P.sub.15 dst 6 P.sub.9 dct P.sub.16 dst 7 P.sub.3 dct P.sub.17 dst 8 P.sub.10 dct P.sub.18 dst 9 P.sub.19 dct 10 P.sub.20 dct 11 P.sub.21 dct 12 P.sub.22 dct 13 P.sub.23 dct 14 P.sub.24 dct 15 P.sub.25 dct 16 P.sub.26 dct
(170) Number of Permutation-Transformation Pairs for Blocks of Size 4×4
(171) TABLE-US-00007 Mode 1 + 2 1 + 3 1 + 4 1 + 8 1 + 16 Pair Perm Transf. Perm Transf. Perm Transf. Perm Transf. Perm Transf. 0 P.sub.0 dct P.sub.0 dct P.sub.0 dct P.sub.0 dct P.sub.0 dct 1 P.sub.1 dct P.sub.3 dct P.sub.5 dct P.sub.5 dct P.sub.9 dct 2 P.sub.2 dct P.sub.4 dct P.sub.6 dct P.sub.8 dct P.sub.10 dct 3 P.sub.5 dct P.sub.7 dct P.sub.9 dct P.sub.11 dct 4 P.sub.8 dct P.sub.16 dct P.sub.12 dct 5 P.sub.17 dct P.sub.13 dct 6 P.sub.18 dct P.sub.14 dct 7 P.sub.22 dct P.sub.15 dct 8 P.sub.23 dct P.sub.19 dct 9 P.sub.20 dct 10 P.sub.21 dct 11 P.sub.22 dct 12 P.sub.24 dct 13 P.sub.25 dct 14 P.sub.26 dct 15 P.sub.27 dct 16 P.sub.28 dct
(172) Number of Permutation-Transformation Pairs for Blocks of Size 8×8
(173) Thus, the encoding of the current block is effectively improved by providing the encoder CO with a plurality of different permutation-transformation pairs.
(174) It is also possible to combine the sizes of the permutation-transformation pairs. For example, a very good compromise in complexity and quality may be achieved by retaining the 1+2 mode for 4×4 blocks and the 1+8 mode for 8×8 blocks. The encoder CO then chooses the best 4×4 transform from among those provided, the same for size 8×8, by minimizing the rate-distortion criterion. The combined choice may be made when choosing the partition of the current block. By way of example, the following combinations are presented below with their associated performances in terms of reducing the transmission rate for a given image quality:
(175) TABLE-US-00008 Combination 4 × 4 mode 8 × 8 mode Performance 0 1 + 2 P.sub.0 + dct −0.76% 1 1 + 4 P.sub.0 + dct −0.78% 2 1 + 8 P.sub.0 + dct −0.86% 3 1 + 16 P.sub.0 + dct −0.80% 4 1 + 2 1 + 2 −0.72% 5 1 + 2 1 + 4 −0.78% 6 1 + 2 1 + 8 −1.09% 7 1 + 2 1 + 16 −0.92% 8 P.sub.0 + dst 1 + 2 −0.10% 9 P.sub.0 + dst 1 + 4 −0.24% 10 P.sub.0 + dst 1 + 8 −0.37% 11 P.sub.0 0 + dst 1 + 16 −0.41%
(176) In particular, a reduction in rate of 1.09% is obtained with the aid of the combination 6 using the 1+2 mode for 4×4 blocks and the 1+8 mode for 8×8 blocks. This reduction in rate is quite significant in video encoding, notably for a technique of low complexity.
(177) In the case where the encoder CO in
(178) TABLE-US-00009 4 × 4 mode Performance 1 + 2 −4.36% 1 + 4 −6.13% 1 + 8 −8.28% 1 + 16 −10.35%
Detailed Description of the Decoding Portion
(179) One embodiment of the method of decoding according to the invention will now be described, in which the method of decoding is implemented in software or hardware by modifications to a decoder initially compliant with the HEVC standard. The method of decoding according to the invention is represented in the form of an algorithm comprising steps D1 to D12 as represented in
(180) As illustrated in
(181) The method of decoding represented in
(182) For this purpose, information items representative of the current image IC.sub.j to be decoded are identified in a signal or data stream F received at the decoder, as delivered as the result of the method of encoding in
(183) With reference to
(184) Such an identification step is implemented by a stream analysis identification module MI_DO, as represented in
(185) Said quantized residues Bq.sub.1, Bq.sub.2, . . . , Bq.sub.u, . . . , Bq.sub.S are intended to be decoded according to a predetermined scanning order, which is, for example, sequential, i.e. said residues are intended to be decoded one after the other in accordance with the raster scan order in which they have been encoded.
(186) Types of scanning other than that which has just been described above are, of course, possible and depend on the scanning order chosen in the encoding, examples of which have been mentioned above.
(187) Preferably in the invention, the blocks to be decoded are particularly of a size that is a multiple power of 2, e.g. 4×4 and 8×8. However, the invention also applies more generally to blocks of size M×M (M=16, 32 or 64). P×M and M×P blocks are also usable with P and M taking their value in the set 2, 4, 8, 16, 32 and 64.
(188) Each block may itself be further divided into subblocks which are themselves subdivisible.
(189) In the course of a step D2 represented in
(190) In the course of a step D3 represented in
(191) In particular, the mode of prediction selected is directly read in the stream F in the case where the encoder CO in
(192) Such a predictor block is, for example, a block of pixels which has already been decoded or not decoded. Such a predictor block is previously stored in the buffer memory MT_DO of the decoder DO as represented in
(193) At the end of the prediction step D3, an optimal predictor block BP.sub.opt compliant with the mode of prediction selected is obtained. The block BP.sub.opt is an approximation of the current block B.sub.u to be decoded. The information items relating to this prediction are read in the data stream F. Such information items notably include the type of prediction (inter or intra), the type of partitioning of the current block if the latter has been subdivided, the reference image index and the motion vector used in the case where a mode of inter prediction has been selected. These information items will subsequently be decoded in a conventional way.
(194) Step D3 is implemented by a predictive decoding software module PRED_DO represented in
(195) In the course of a step D4 represented in
(196) reading the symbol or symbols which are associated with said current block to be decoded,
(197) associating digital information, such as bits, with the symbol(s) read.
(198) Such an entropy decoding step is implemented by an entropy decoding software module MDE represented in
(199) In the course of a step D5 represented in
(200) Such a dequantization step is performed by an inverse quantization software module MQ.sup.−1_DO, as represented in
(201) In the course of a step D6 represented in
(202) Such a step D6 is implemented by a determination software module MDT_DO represented in
(203) As represented in the
(204) in the course of a substep D610 represented in
(205) in the course of a substep D611 represented in
(206) Such steps are necessary when the decoder DO in
(207) The substeps D610 and D611 are implemented by a software submodule SMDT1_DO represented in
(208) In the course of said step D6, the index m*,n* is decoded.
(209) More precisely, the decoder DO reads, for example, a fixed-length binary code for retrieving the value of the permutation-transformation pair associated with the criterion J.sub.m*,n* and associating the corresponding inverse permutation-inverse transformation therewith.
(210) In the case, for example, as has been described in connection with the aforementioned step C9, where four values of permutation-transformation pairs: 0, 1, 2, 3 are likely to be signaled in the stream F,
(211) if the decoder DO reads the code 00, it assigns the value 0 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(212) if the decoder DO reads the code 01, it assigns the value 1 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(213) if the decoder DO reads the code 10, it assigns the value 2 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(214) if the decoder DO reads the code 11, it assigns the value 3 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*.
(215) In a variant which has been described in connection with the aforementioned step C9, the index m*,n* intended to be read by the decoder DO is expressed with the aid of a variable-length code.
(216) In the case, for example, as has been described in connection with the aforementioned step C9, where four values of permutation-transformation pairs: 0, 1, 2, 3 are likely to be signaled in the stream F,
(217) if the decoder DO reads the code 0, it assigns the value 0 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(218) if the decoder DO reads the code 10, it assigns the value 1 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(219) if the decoder DO reads the code 110, it assigns the value 2 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(220) if the decoder DO reads the code 111, it assigns the value 3 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*.
(221) According to a preferred embodiment described in connection with the aforementioned step C9, the index m*,n* intended to be read by the decoder DO includes two indicators:
(222) a flag signaling the first bit of the code associated with the permutation-transformation pair OP.sub.m*, OT.sub.n* selected in the encoding, and
(223) a complementary syntax element which indicates the less probable permutation-transformation pairs in binary form.
(224) In the case, for example, as has been described in connection with the aforementioned step C9, where four values of permutation-transformation pairs: 0, 1, 2, 3 are likely to be signaled in the stream F,
(225) if the decoder DO reads the flag ‘0’, it assigns the value 0 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(226) if the decoder DO reads the flag ‘1’ and the binary complement ‘0’, it assigns the value 1 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(227) if the decoder DO reads the flag ‘1’ and the binary complement ‘10’, it assigns the value 2 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(228) if the decoder DO reads the flag ‘1’ and the binary complement ‘11’, it assigns the value 3 to the permutation-transformation pair which, in the encoding, has been associated with the criterion J.sub.m*,n*.
(229) According to another preferred embodiment described in connection with the aforementioned step C9, the index m*,n* intended to be read by the decoder DO consists only of one flag. This other preferred mode is applicable in the case where a single transform, e.g. a DST, is applied in the coding and where two permutation operations OP.sub.0 and OP.sub.1 are likely to be applied in combination with the DST.
(230) For this purpose,
(231) if the decoder DO reads a bit set to ‘1’, it assigns the value of this bit to the permutation-transformation pair SOP.sub.0 which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(232) if the decoder DO reads a bit set to ‘0’, it assigns the value of this bit to the permutation-transformation pair SOP.sub.1 which, in the encoding, has been associated with the criterion J.sub.m*,n*.
(233) In the case where more than two permutation-transformation pairs are likely to be signaled in the stream F, the decoder DO is suitable for reading a fixed- or variable-length code.
(234) According to another possible embodiment described in connection with the aforementioned step C9, the index m*,n* is not signaled in the stream F or only a part of the index m*,n* is signaled in the stream F.
(235) As represented in
(236) in the course of a substep D620 represented in
(237) in the course of a substep D621 represented in
(238) in the course of a substep D622 represented in
(239) At the end of the substep D622, the pair of transformation and permutation operations associated with the reconstructed block selected is then identified.
(240) The substeps D620 to D622 are implemented when the decoder DO in
(241) Such substeps are implemented by a software submodule SMDT2_DO represented in
(242) The aforementioned criterion of coherence consists in measuring the coherence of the pixels of a reconstructed block considered with respect to the neighborhood thereof. The coherence measurement may be a measurement of continuity between the pixels of the reconstructed block and its neighbors, by calculating a sum of the absolute values of the differences.
(243) According to another particular configuration given by way of example and which has been described in connection with the aforementioned step C9:
(244) in the case where a single transform, e.g. a DST, is likely to be applied to the encoding or decoding and where a permutation operation OP.sub.0 is applied in combination with the DST, if the decoder DO reads a bit set to ‘0’, it assigns the value of this bit to the permutation-transformation pair SOP.sub.0 which, in the encoding, has been associated with the criterion J.sub.m*,n*,
(245) in the case where a single transform, e.g. a DST, is likely to be applied and where multiple permutation operations may be applied in combination with the DST, if the decoder DO reads a bit set to ‘1’, it assigns the value of this bit to the plurality of possible permutation-transformation pairs, likely to be applied to the decoding, and itself deduces which is the pair that has actually been signaled, e.g. with the aid of coherence measurements.
(246) The plurality of inverse permutation operations OP.sup.−1.sub.0, OP.sup.−1.sub.1, . . . , OP.sup.−1.sub.m, . . . , OP.sup.−1.sub.K-1 corresponds to the inverse of the plurality of permutation operations OP.sub.0, OP.sub.1, . . . , OP.sub.m, . . . , OP.sub.K-1 examples of which have been cited in the description above of the method of encoding.
(247) The plurality of transformation operations OT.sup.−1.sub.0, OT.sup.−1.sub.1, . . . , OT.sup.−1.sub.n, . . . , OT.sup.−1.sub.L-1 corresponds to the inverse of the plurality of transformation operations OT.sub.0, OT.sub.1, . . . , OT.sub.n, . . . , OT.sub.L-1 examples of which have been cited in the description above of the method of encoding. In the course of a step D7 represented in
(248) Step D7 is implemented by a transformation software module MT.sup.−1_DO, as represented in
(249) In the course of a step D8 represented in
(250) Step D8 is implemented by a permutation software module MPER.sup.−1_DO, as represented in
(251) In the course of a step D9 represented in
(252) Said step D9 is implemented by an inverse prediction software module PRED.sup.−1_DO represented in
(253) A decoded block BD.sub.u is then obtained and stored in the buffer memory MT_DO in
(254) In the course of a step D10 represented in
(255) In the course of a step D11 represented in
(256) If such is the case, in the course of a step D12 represented in
(257) If such is not the case, in the course of step D2, the selection is performed of the following residual block to be decoded in accordance with the aforementioned raster scan scanning order.
(258) The decoding steps that have just been described above are implemented for all the blocks B.sub.1, B.sub.2, . . . , B.sub.u, . . . , B.sub.S to be decoded of the current image IC.sub.j considered.
(259) It goes without saying that the embodiments that have been described above have been given as purely indicative and are in no way restrictive, and that numerous modifications may easily be made by the person skilled in the art without, however, departing from the scope of the invention.
(260) Thus, the method of encoding and/or decoding according to the invention, instead of being applied to current blocks of size 4×4 or 8×8, may also be applied to current blocks having different possible sizes, such as, for example, 16×16 and 32×32.