Method of Coding and Decoding Images, Coding and Decoding Device and Computer Programs Corresponding Thereto
20220060755 · 2022-02-24
Inventors
Cpc classification
H04N19/91
ELECTRICITY
H04N19/48
ELECTRICITY
H04N19/122
ELECTRICITY
H04N19/119
ELECTRICITY
H04N19/46
ELECTRICITY
H04N19/13
ELECTRICITY
H04N19/159
ELECTRICITY
H04N19/105
ELECTRICITY
H04N19/184
ELECTRICITY
H04N19/44
ELECTRICITY
H04N19/463
ELECTRICITY
H04N19/14
ELECTRICITY
International classification
H04N19/91
ELECTRICITY
H04N19/105
ELECTRICITY
H04N19/119
ELECTRICITY
H04N19/122
ELECTRICITY
H04N19/13
ELECTRICITY
H04N19/14
ELECTRICITY
H04N19/159
ELECTRICITY
H04N19/184
ELECTRICITY
H04N19/44
ELECTRICITY
H04N19/46
ELECTRICITY
H04N19/463
ELECTRICITY
H04N19/467
ELECTRICITY
H04N19/48
ELECTRICITY
Abstract
A method for encoding an image having been cut up into partitions. The method includes: predicting data of a current partition based on an already encoded and then decoded reference partition, generating a predicted partition; determining residual data by comparing data relating to the current partition with the predicted partition, the residual data associated with various digital data items. Prior producing a signal containing the encoded information, performing the following steps: determining, from the predetermined residual data, a subset containing residual data capable of being modified; calculating the value of a function representative of the residual data; comparing the calculated value with a value of at least one of the digital data items; based on the comparison, modification or non-modification of at least one of the residual data items of the subset; and, in the event of a modification, entropy encoding the at least one modified residual data item.
Claims
1. (canceled)
2. A non-transitory computer-readable medium for storing data representing a sign-data-hiding enabled block of an image, comprising: a bitstream stored in the non-transitory computer-readable medium, the bitstream comprising: a set of context-based adaptive binary arithmetic coding (CABAC) encoded coefficients representing a set of coefficients of a residual block of the sign-data-hiding enabled block, the set of coefficients including a particular non-zero coefficient that is without a sign designation; and an information item representing a prediction mode of the sign-data-hiding enabled block, wherein remainder data, which is based on an operation representing a division between a sum of non-zero coefficients in the set of coefficients and a specific number, is used to designate a sign for the particular non-zero coefficient, wherein the residual block of the sign-data-hiding enabled block corresponds to a difference between an original block and a prediction block generated by using the prediction mode, and wherein a count of coefficient between a first non-zero coefficient and a last non-zero coefficient in the set of CABAC encoded coefficients is greater than a threshold number.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0080] Other features and advantages will become clear upon reading about two preferred embodiments described with reference to the drawings in which:
[0081]
[0082]
[0083]
[0084]
DETAILED DESCRIPTION OF THE ENCODING PART
[0085] An embodiment of the invention will now be described, in which the encoding method according to the invention is used to encode a sequence of images according to a binary stream close to that obtained by an encoding according to the H.264/MPEG-4 AVC standard. In this embodiment, the encoding method according to the invention is for example implemented in software or hardware form by modifications of an encoder initially compliant with the H.264/MPEG-4 AVC standard. The encoding method according to the invention is represented in the form of an algorithm including steps C1 to C40, represented in
[0086] According to the embodiment of the invention, the encoding method according to the invention is implemented in an encoding device or encoder CO, an embodiment of which is represented in
[0087] In accordance with the invention, prior to the actual encoding step, an image IE of a sequence of images to be encoded in a predetermined order is split into a plurality Z of partitions B.sub.1, B.sub.2, . . . , B.sub.i, . . . , B.sub.Z, as represented in
[0088] It is appropriate to note that in the sense of the invention, the term “partition” means coding unit. This latter terminology is notably used in the HEVC/H.265 standard being drafted, for example in the document accessible at the following Internet address: http://phenix.int-evry.fr/jct/doc_end_user/current_document.php?id=3286
[0089] In particular, such a coding unit groups together sets of rectangular or square shape pixels, also called blocks, macroblocks, or sets of pixels exhibiting other geometric shapes.
[0090] In the example represented in
[0091] Each block or macroblock can moreover be itself divided into subblocks which are themselves subdividable.
[0092] Such splitting is performed by a partitioning module PCO represented in
[0093] Following said splitting step, each of the current partitions B; (where i is an integer such that 1≤i≤Z) of said image IE is encoded.
[0094] In the example represented in
[0095] The encoding according to the invention is implemented in an encoding software module MC_CO of the encoder CO, as represented in
[0096] During a step C1 represented in
[0097] During a step C2 represented in
[0098] Needless to say other intra prediction modes as proposed in the H.264 standard are possible.
[0099] The current block B.sub.1 can also be subjected to a predictive encoding in inter mode, during which the current block is predicted with respect to a block from a previously encoded and decoded image. Other types of prediction can of course be envisaged. Among the predictions possible for a current block, the optimal prediction is chosen according to a rate distortion criterion that is well known to the person skilled in the art.
[0100] Said abovementioned predictive encoding step provides for constructing a predicted block Bp.sub.1 which is an approximation of the current block B.sub.1. The information items relating to this predictive encoding are intended to be included in a signal to be transmitted to the decoder. Such information items comprise notably the type of prediction (inter or intra), and if necessary, the intra prediction mode, the type of partitioning of a block or macroblock if the latter has been subdivided, the reference image index and the motion vector which are used in the inter prediction mode. These information items are compressed by the encoder CO.
[0101] During a next step C3 represented in
[0102] During a next step C4 represented in
[0103] During a next step C5 represented in
[0104] During a next step C6 represented in
[0105] Assume that in the example represented, L=16 and that the list E.sub.1 contains the following sixteen coefficients: E.sub.1=(0, +9, −7, 0, 0, +1, 0, −1, +2, 0, 0, +1, 0, 0, 0, 0).
[0106] In this particular case: [0107] for each coefficient located before the last non-zero coefficient in the list E.sub.1, a digital information item, such as a bit, is intended to be entropically encoded to indicate whether or not the coefficient is zero: if the coefficient is zero, it is for example the bit of value 0 which will be encoded, while if the coefficient is not zero, it is the bit of value 1 which will be encoded; [0108] for each non-zero coefficient +9, −7, +1, −1, +2, +1, a digital information item, such as a bit, is intended to be entropically encoded to indicate whether or not the absolute value of the coefficient is equal to one: if it is equal to 1, it is for example the bit of value 1 which will be encoded, while if it is not equal to 1, it is the bit of value 0 which will be encoded; [0109] for each non-zero coefficient and for which the absolute value is not equal to one and which is located before the last non-zero coefficient, such as the coefficients of value +9, −7, +2, an amplitude information item (absolute value of the coefficient at which the value two is subtracted) is entropically encoded; [0110] for each non-zero coefficient, the sign assigned to it is encoded by a digital information item, such as a bit for example set to ‘0’ (for the + sign) or set to ‘1’ (for the − sign).
[0111] With reference to
[0112] In accordance with the invention, it is decided to avoid entropically encoding at least one of the abovementioned information items. For the reasons explained earlier in the description, in a preferred embodiment, it is decided to not entropically encode at least one sign of one of said coefficients in the list E.sub.1.
[0113] By way of alternative example, it could notably be decided to entropically encode the least significant bit of the binary representation of the amplitude of the first non-zero coefficient in said list E.sub.1.
[0114] To this end, during a step C7 represented in
[0115] In the preferred embodiment, the number of signs to be hidden is one or zero. Additionally, in accordance with said preferred embodiment, it is the sign of the first non-zero coefficient which is intended to be hidden. In the example represented, it is therefore the sign of the coefficient ε2=+9 that is hidden.
[0116] In an alternative embodiment, the number of signs to be hidden is either zero, one, two, three or more.
[0117] In accordance with the preferred embodiment of step C7, during a first substep C71 represented in
[0118] According to the invention, a coefficient is modifiable if the modification of its quantized value does not cause desynchronization at the decoder, once this modified coefficient is processed by the decoder. Thus, the processing module MTR_CO is configured initially to not modify: [0119] the zero coefficient or coefficients located before the first non-zero coefficient such that the decoder does not affect the value of the sign hidden at this or these zero coefficients, [0120] and for reasons of computation complexity, the zero coefficient or coefficients located after the last non-zero coefficient.
[0121] In the example represented, at the end of substep C71, the sublist SE.sub.1 obtained is such that SE.sub.1=(9, −7.0, 0, 1, 0, −1, 2, 0, 0, 1). Consequently, eleven modifiable coefficients are obtained.
[0122] During a next substep C72 represented in
[0123] If the number of modifiable coefficients is less than the threshold TSIG, then during a step C20 represented in
[0124] If the number of modifiable coefficients is greater than the threshold TSIG, then during a step C8 represented in
[0125] In the preferred embodiment in which only one sign is intended to be hidden in the signal to be transmitted to the decoder, the function f is the parity of the sum of the coefficients in the sublist SE.sub.1.
[0126] During a step C9 represented in
[0127] In the example proposed, said convention is such that a positive sign is associated with a bit of value equal to zero, while a negative sign is associated with a bit of value equal to one.
[0128] If, in accordance with the convention adopted in the encoder CO according to the invention, the sign is positive, which corresponds to an encoding bit value of zero, and if the sum of the coefficients in the sublist SE.sub.1 is even, then step C20 for the entropy encoding of the coefficients in the aforementioned list E.sub.1 is carried out, with the exception of the sign of the coefficient ε2.
[0129] It, still in accordance with the convention adopted in the encoder CO according to the invention, the sign is negative, which corresponds to an encoding bit value of one, and if the sum of the coefficients in the sublist SE.sub.1 is odd, then also step C20 for the entropy encoding of the coefficients in the aforementioned list E.sub.1 is carried out, with the exception of the sign of the coefficient ε2.
[0130] If, in accordance with the convention adopted in the encoder CO according to the invention, the sign is positive, which corresponds to an encoding bit value of zero, and if the sum of the coefficients in the sublist SE.sub.1 is odd, then during a step C10 represented in
[0131] If, still in accordance with the convention adopted in the encoder CO according to the invention, the sign is negative, which corresponds to an encoding bit value of one, and if the sum of the coefficients in the sublist SE.sub.1 is even, then also at step C10, at least one modifiable coefficient in the sublist SE.sub.1 is modified.
[0132] Such a modification operation is carried out by the processing module MTR_CO in
[0133] In the example embodiment in which SE.sub.1=(+9, −7, 0, 0, +1, 0, −1, +2, 0, 0, +1), the total sum f of the coefficients is equal to 5, and is therefore odd. In order that the decoder can reconstruct the positive sign assigned to the first non-zero coefficient ε2=+9, without the encoder CO having to transmit this coefficient to the decoder, the parity of the sum must become even. Consequently, the processing module MTR_CO tests, during said step C10, various modifications of coefficients in the sublist SE.sub.1, all aiming to change the parity of the sum of the coefficients. In the preferred embodiment, +1 or −1 is added to each modifiable coefficient and a modification is selected from among those which are carried out.
[0134] In the preferred embodiment, such a selection forms the optimal prediction according to a performance criterion which is for example the rate distortion criterion that is well known to the person skilled in the art. Such a criterion is expressed by equation (1) below:
J=D+λR (1)
where D represents the distortion between the original macroblock and the reconstructed macroblock, R represents the encoding cost in bits of the encoding information items and λ represents a Lagrange multiplier, the value of which can be fixed prior to the encoding.
[0135] In the example proposed, the modification which brings about an optimal prediction according to the abovementioned rate distortion criterion is the addition of the value 1 to the second coefficient −7 in the sublist SE.sub.1.
[0136] At the end of step C10, a modified sublist is hence obtained, SEm.sub.1=(+9, −6, 0, 0, +1, 0, −1, +2, 0, 0, +1).
[0137] It is appropriate to note that during this step, certain modifications are prohibited. Thus, in the case in which the first non-zero coefficient ε2 would have the value +1, it would not have been possible to add −1 to it, since it would have become zero, and it would then have lost its characteristic of first non-zero coefficient in the list E.sub.1. The decoder would then have later attributed the decoded sign (by calculation of the parity of the sum of the coefficients) to another coefficient, and there would then have been a decoding error.
[0138] During a step C11 represented in
[0139] Then step C20 for the entropy encoding of the coefficients in the aforementioned list Em.sub.1 is carried out, with the exception of the sign of the coefficient ε2, which is the + sign of the coefficient 9 in the proposed example, which sign is hidden in the parity of the sum of the coefficients.
[0140] It is appropriate to note that the set of amplitudes of the coefficients in the list E.sub.1 or in the modified list Em.sub.1 is encoded before the set of signs, with the exclusion of the sign of the first non-zero coefficient ε2 which is not encoded, as has been explained above.
[0141] During a next step C30 represented in
[0142] If the current block is the last block of the image IE, then during a step C40 represented in
[0143] If this is not the case, the next block B.sub.i is selected, which is then encoded in accordance with the order of the previously mentioned raster scan, by repeating steps C1 to C20, for 1≤i≤Z.
[0144] Once the entropy encoding of all the blocks B.sub.1 to B.sub.Z is carried out, a signal F is constructed, representing, in binary form, said encoded blocks.
[0145] The construction of the binary signal F is implemented in a stream construction software module CF, as represented in
[0146] The stream F is then transmitted via a communication network (not represented) to a remote terminal. The latter includes a decoder which will be described further in detail later in the description.
[0147] There will now be described, mainly with reference to
[0148] This other embodiment is distinguished from the previous one only by the number of coefficients to be hidden which is either 0, or N, where N is an integer such that N≥2.
[0149] To this end, previously mentioned comparison substep C72 is replaced by substep C72a represented in dotted-line in
[0150] If the number of modifiable coefficients is less than the first threshold TSIG_1, then during abovementioned step C20, conventional entropy encoding of the coefficients in the list E.sub.1 is carried out. To this end, the sign of each non-zero coefficient in the list E.sub.1 is entropically encoded.
[0151] If the number of modifiable coefficients is between the threshold TSIG_N and TSIG_N+1, then during a step C8 represented in
[0152] In this other embodiment, since the decision at the encoder is to hide N signs, the function f is the modulo 2.sup.N remainder of the sum of the coefficients in the sublist SE.sub.1. It is assumed in the proposed example that N=2, the two signs to be hidden being the two first signs of the two first non-zero coefficients respectively, i.e. ε2 and ε3.
[0153] During next step C9 represented in
[0154] In the example proposed where N=2, there are 2.sup.2=4 different configurations of signs.
[0155] These four configurations comply with a convention at the encoder CO, which convention is for example determined as follows: [0156] a remainder equal to zero corresponds to two consecutive positive signs: +, +; [0157] a remainder equal to one corresponds to, consecutively, a positive sign and a negative sign: +, −; [0158] a remainder equal to two corresponds to, consecutively, a negative sign and a positive sign: −, +; [0159] a remainder equal to three corresponds to two consecutive negative signs: −, −.
[0160] If the configuration of N signs corresponds to the value of the modulo 2.sup.N remainder of the sum of the coefficients in the sublist SE.sub.1, then step C20 for the entropy encoding of the coefficients in the abovementioned list E.sub.1 is carried out, with the exception of the sign of the coefficient ε2 and of the coefficient ε3, which signs are hidden in the parity of the modulo 2.sup.N sum of the coefficients.
[0161] If this is not the case, then step C10 for modifying at least one modifiable coefficient in the sublist SE.sub.1 is carried out. Such a modification is executed by the processing module MTR_CO in
[0162] During previously mentioned step C11, the processing module MTR_CO carries out a corresponding modification of the list E.sub.1. A modified list Em.sub.1 is hence obtained.
[0163] Then step C20 for the entropy encoding of the coefficients in the aforementioned list Em.sub.1 is carried out, with the exception of the sign of the coefficient ε2 and the sign of the coefficient ε3, which signs are hidden in the parity of the modulo 2.sup.N sum of the coefficients.
Detailed Description of the Decoding Part
[0164] An embodiment of the decoding method according to the invention will now be described, in which the decoding method is implemented in software or hardware form by modifications of a decoder initially compliant with the H.264/MPEG-4 AVC standard.
[0165] The decoding method according to the invention is represented in the form of an algorithm including steps D1 to D12, represented in
[0166] According to the embodiment of the invention, the decoding method according to the invention is implemented in a decoding device or decoder DO, as represented in
[0167] During a preliminary step not represented in
[0168] Each block or macroblock can moreover be itself divided into subblocks which are themselves subdividable.
[0169] Such an identification is executed by a stream analysis software module EX_DO, as represented in
[0170] During a step D1 represented in
[0171] Then the decoding of each of the selected encoded blocks is carried out.
[0172] In the example represented in
[0173] The decoding according to the invention is implemented in a decoding software module MD_DO of the decoder DO, as represented in
[0174] During a step D2 represented in
[0175] During a step D3 represented in
[0176] In the preferred embodiment, the number of hidden signs is one or zero. Additionally, in accordance with said preferred embodiment, it is the sign of the first non-zero coefficient which is hidden. In the example represented, it is therefore the positive sign of the coefficient ε2=+9.
[0177] In an alternative embodiment, the number of hidden signs is either zero, one, two, three or more.
[0178] In accordance with the preferred embodiment of step D3, during a first substep D31 represented in
[0179] Such a determination is performed the same way as in previously mentioned encoding step C7.
[0180] Like the previously mentioned processing module MTR_CO, the processing module MTR_DO is initially configured to not modify: [0181] the zero coefficient or coefficients located before the first non-zero coefficient, [0182] and for reasons of computation complexity, the zero coefficient or coefficients located after the last non-zero coefficient.
[0183] In the example represented, at the end of substep D31, there is the sublist SEm.sub.1 such that SEm.sub.1=(9, −6, 0, 0, 1, 0, −1, 2, 0, 0, 1). Consequently, eleven coefficients capable of having been modified are obtained.
[0184] During a next substep D32 represented in
[0185] If the number of coefficients capable of having been modified is less than the threshold TSIG, then during a step D4 represented in
[0186] If the number of coefficients capable of having been modified is greater than the threshold TSIG, then during said step D4, a conventional entropy decoding of all the signs of the coefficients in the list Em.sub.1 is carried out, with the exception of the sign of the first non-zero coefficient ε2.
[0187] During a step D5 represented in
[0188] In the preferred embodiment where only one sign is hidden in the signal F, the function f is the parity of the sum of the coefficients in the sublist SEm.sub.1.
[0189] In accordance with the convention used at the encoder CO, which is the same at the decoder DO, an even value of the sum of the coefficients in the sublist SEm.sub.1 means that the sign of the first non-zero coefficient in the modified list Em.sub.1 is positive, while an odd value of the sum of the coefficients in the sublist SEm.sub.1 means that the sign of the first non-zero coefficient in the modified list Em.sub.1 is negative.
[0190] In the example embodiment in which SEm.sub.1=(+9, −6, 0, 0, +1, 0, −1, +2, 0, 0, +1), the total sum of the coefficients is equal to 6, and is therefore even. Consequently, at the end of step D5, the processing module MTR_DO deduces therefrom that the hidden sign of the first non-zero coefficient ε2 is positive.
[0191] During a step D6 represented in
[0192] During a step D7 represented in
[0193] During a step D8 represented in
[0194] During a step D9 represented in
[0195] Needless to say other intra prediction modes as proposed in the H.264 standard are possible.
[0196] During this step, the predictive decoding is carried out using decoded syntax elements at the previous step and notably comprising the type of prediction (inter or intra), and if necessary, the intra prediction mode, the type of partitioning of a block or macroblock if the latter has been subdivided, the reference image index and the motion vector which are used in the inter prediction mode.
[0197] Said abovementioned predictive decoding step provides for constructing a predicted block Bp.sub.1.
[0198] During a step D10 represented in
[0199] During a step D11 represented in
[0200] If the current block is the last block in the signal F, then during a step D12 represented in
[0201] If this is not the case, the next block B.sub.i is selected, to be decoded in accordance with the order of the previously mentioned raster scan, by repeating steps D1 to D10, for 1≤i≤Z.
[0202] There will now be described, mainly with reference to
[0203] This other embodiment is distinguished from the previous one only by the number of hidden coefficients which is either 0, or N, where N is an integer such that N≥2.
[0204] To this end, previously mentioned comparison substep D32 is replaced by substep D32a represented in dotted-line in
[0205] If the number of said coefficients is less than the first threshold TSIG_1, then during previously mentioned step D4, the conventional entropy decoding of all the signs of the coefficients in the list E.sub.1 is carried out. To this end, the sign of each non-zero coefficient in the list E.sub.1 is entropically decoded.
[0206] If the number of said coefficients is between the threshold TSIG_N and TSIG_N+1, then during previously mentioned step D4, the conventional entropy decoding of all the signs of the coefficients in the list E.sub.1 is carried out, with the exception of the N respective signs of the first non-zero coefficients in said modified list Em.sub.1, said N signs being hidden.
[0207] In this other embodiment, the processing module MTR_DO calculates, during step D5, the value of the function f which is the modulo 2.sup.N remainder of the sum of the coefficients in the sublist SEm.sub.1. It is assumed in the proposed example that N=2.
[0208] The processing module MTR_DO hence deduces therefrom the configuration of the two hidden signs which are assigned to each of the two first non-zero coefficients ε2 and ε3 respectively, according to the convention used at the encoding.
[0209] Once these two signs have been reconstructed, steps D6 to D12 described above are carried out.
[0210] It goes without saying that the embodiments which have been described above have been given purely by way of indication and are not at all limiting, and that a number of modifications can easily be brought about by the person skilled in the art without thereby departing from the scope of the invention.
[0211] Thus for example, according to a simplified embodiment with respect to that represented in
[0212] Additionally, the decision criterion applied at encoding step C72 and at decoding step D32 could be replaced by another type of criterion. To this end, instead of comparing the number of modifiable coefficients or the number of coefficients capable of having been modified with a threshold, the processing module MTR_CO or MTR_DO could apply a decision criterion which is a function of the sum of the amplitudes of the coefficients that are modifiable or capable of having been modified, respectively, or of the number of zeros present among the coefficients that are modifiable or capable of having been modified, respectively.