GENERATION AND ENCODING OF RESIDUAL INTEGRAL IMAGES
20170230654 · 2017-08-10
Inventors
Cpc classification
H04N19/102
ELECTRICITY
H04N19/597
ELECTRICITY
International classification
H04N19/102
ELECTRICITY
Abstract
The invention concerns the encoding of at least one current integral image (II.sub.j) captured by an image capture device, comprising the steps consisting of: —decomposing (C1) the current integral image into at least one frame (Vu) representing a given perspective of a scene and, from at least one image capturing parameter associated with the image capture device, —encoding (C2) said at least one frame, —decoding (C4) said at least one frame, —recomposing (C5) the current integral image from said at least one decoded frame by applying an inverse decomposition of said decomposition of the integral image and from said at least one image capturing parameter associated with the image capture device, said encoding method being characterised in that it implements the steps consisting of: —determining (C6) a residual integral image by comparing said at least one current integral image with said recomposed integral image, —encoding (C7) the data associated with the residual integral image and said at least one image capturing parameter associated with the image capture device.
Claims
1. A method of coding at least one current integral image (II.sub.j) captured by an image capture device, including the following steps: decomposing (C1) the current integral image into at least one view (V.sub.u) representing a given perspective of a scene and from at least one image capture parameter associated with the image capture device, coding (C2) said at least one view, decoding (C4) said at least one view, recomposing (C5) the current integral image from said at least one decoded view by application of an inverse decomposition of said decomposition of the integral image and from said at least one image capture parameter associated with the image capture device, determining (C6) a residual integral image (IIres.sub.j) by comparison of said at least one current integral image with said recomposed integral image, coding (C7) the data associated with the residual integral image and said at least one image capture parameter associated with the image capture device.
2. The coding method as claimed in claim 1, including: a step (C3) of generating a first data signal (F.sub.1) containing data coming from the coding of said at least one view, and a step (C8) of generating a second data signal (F.sub.2) containing the data associated with the coded residual integral image, said at least one image capture parameter associated with the image capture device being contained either in the first signal or in the second signal or in another data signal to be generated.
3. The coding method as claimed in claim 1, in which the number and the position of the views to be coded of the current integral image are selected as being those that optimize a predetermined coding performance criterion.
4. The coding method as claimed in claim 3, in which the number and the position of the views of the current integral image are selected as being those that minimize the quantity of data associated with the residual integral image.
5. The coding method as claimed in claim 1, during which there is selection of a value of a first quantization parameter (QP.sub.1) to be applied during the step (C2) of coding said at least one view and selection of a value of a second quantization parameter (QP.sub.2) to be applied during the step (C7) of coding the data associated with the residual integral image, said values being selected as optimizing a predetermined coding performance criterion.
6. The coding method as claimed in claim 1, further including a step (CI1; CI2; CI3) of modification of the image that is applied: between the step (C1) of decomposition of the current integral image into said at least one view and the step (C2) of coding said at least one view, and/or between the step (C4) of decoding said at least one view and the step (C5) of recomposition of the current integral image, and/or between the step (C5) of recomposition of the current integral image and the step (C6) of determination of the residual integral image.
7. The coding method as claimed in claim 6, in which said modification step is implemented using a modification parameter (p.sub.tr1; p.sub.tr2; p.sub.tr3) that is selected as optimizing a predetermined coding performance criterion or as minimizing the quantity of data associated with the residual integral image.
8. A device for coding at least one current integral image (II.sub.j) captured by an image capture device, including: first processing means (MDCV_CO) for decomposing the current integral image into at least one view representing a given perspective of a scene and from at least one image capture parameter associated with the image capture device, first coding means (MCV_CO) for coding said at least one view, decoding means (MDV_CO) for decoding said at least one view, second processing means (MDCV.sup.−1_CO) for recomposing the current integral image from said decoded view by application of an inverse decomposition of said decomposition of the integral image and from said at least one image capture parameter associated with the image capture device, calculation means (CAL2_CO) for determining a residual integral image by comparison of said at least one current integral image with said recomposed integral image, second coding means (MDCI) for coding the data associated with the residual integral image and said at least one image capture parameter associated with the image capture device.
9. A computer program including program code instructions for execution of steps of a coding method when it is executed on a computer, the method of coding at least one current integral image (II.sub.j) captured by an image capture device, including the following steps: decomposing (C1) the current integral image into at least one view (V.sub.u) representing a given perspective of a scene and from at least one image capture parameter associated with the image capture device, coding (C2) said at least one view, decoding (C4) said at least one view, recomposing (C5) the current integral image from said at least one decoded view by application of an inverse decomposition of said decomposition of the integral image and from said at least one image capture parameter associated with the image capture device, determining (C6) a residual integral image (IIres.sub.j) by comparison of said at least one current integral image with said recomposed integral image, coding (C7) the data associated with the residual integral image and said at least one image capture parameter associated with the image capture device.
10. A method of decoding a data signal (F) representing at least one current integral image (II.sub.j) acquired by an image capture device and that has previously been coded, said method including the following steps: identifying (D1a) in the data signal at least one image capture parameter associated with the image capture device, decoding (D2) at least one view of said current integral image from said at least one image capture parameter that has been identified, said at least one view representing a given perspective of a scene, recomposing (D3) an integral image as predictor of the current integral image from said at least one decoded view and from said at least one image capture parameter associated with the image capture device, decoding (D4) coded data representing the difference between said at least one current integral image and the recomposed integral image, reconstructing (D5) the current integral image from said recomposed integral image and decoded data representing said difference.
11. The decoding method as claimed in claim 10, in which the step of recomposition of said current integral image is implemented using a position parameter of said at least one decoded view in the current integral image to be decoded, said position parameter being predetermined or read in said data signal.
12. The decoding method as claimed in claim 10, in which: the step of decoding said at least one view includes the execution of a dequantization step (D22) based on a first quantization parameter the value of which is identified in said data signal, the step of decoding the coded data representing the difference between said at least one current integral image and the recomposed integral image includes the execution of a dequantization step (D42) based on a second quantization parameter the value of which is identified in said data signal.
13. The decoding method as claimed in claim 10, further including a step (DI1; DI2; DI3) of image modification based on a modification parameter that is predetermined or read in the data signal, said modification step being applied: between the step (D2) of decoding said at least one view and the step (D3) of recomposition of the current integral image as predictor of the current integral image, and/or between the step (D3) of recomposition of the current integral image as predictor of the current integral image and the step (D4) of reconstruction of the current integral image.
14. A device (DO) for decoding a data signal representing at least one current integral image (II.sub.j) acquired by an image capture device and that has previously been coded, the decoding device including: analysis means (MI_DO) for identifying in the data signal at least one image capture parameter associated with the image capture device, first decoding means (MDV_DO) for decoding at least one view of said current integral image based on said at least one image capture parameter that has been identified, said at least one view representing a given perspective of a scene, processing means (MDCV.sup.−1_DO) for recomposing an integral image as predictor of the current integral image from said decoded view and from said at least one image capture parameter associated with the image capture device, second decoding means (MDDI) for decoding coded data representing the difference between said at least one current integral image and the recomposed integral image, reconstruction means (CAL3.sup.−1_DO) for reconstructing the current integral image from said recomposed integral image and decoded data representing said difference.
15. A computer program including program code instructions for execution of steps of a decoding method when it is executed on a computer, the method of decoding a data signal (F) representing at least one current integral image (II.sub.j) acquired by an image capture device and that has previously been coded, said method including the following steps: identifying (D1a) in the data signal at least one image capture parameter associated with the image capture device, decoding (D2) at least one view of said current integral image from said at least one image capture parameter that has been identified, said at least one view representing a given perspective of a scene, recomposing (D3) an integral image as predictor of the current integral image from said at least one decoded view and from said at least one image capture parameter associated with the image capture device, decoding (D4) coded data representing the difference between said at least one current integral image and the recomposed integral image, reconstructing (D5) the current integral image from said recomposed integral image and decoded data representing said difference.
16. The coding method as claimed in claim 2, in which the number and the position of the views to be coded of the current integral image are selected as being those that optimize a predetermined coding performance criterion.
17. The coding method as claimed in claim 2, during which there is selection of a value of a first quantization parameter (QP.sub.1) to be applied during the step (C2) of coding said at least one view and selection of a value of a second quantization parameter (QP.sub.2) to be applied during the step (C7) of coding the data associated with the residual integral image, said values being selected as optimizing a predetermined coding performance criterion.
18. The decoding method as claimed in claim 11, in which: the step of decoding said at least one view includes the execution of a dequantization step (D22) based on a first quantization parameter the value of which is identified in said data signal, the step of decoding the coded data representing the difference between said at least one current integral image and the recomposed integral image includes the execution of a dequantization step (D42) based on a second quantization parameter the value of which is identified in said data signal.
19. The decoding method as claimed in claim 11, further including a step (DI1; DI2; DI3) of image modification based on a modification parameter that is predetermined or read in the data signal, said modification step being applied: between the step (D2) of decoding said at least one view and the step (D3) of recomposition of the current integral image as predictor of the current integral image, and/or between the step (D3) of recomposition of the current integral image as predictor of the current integral image and the step (D4) of reconstruction of the current integral image.
20. The decoding method as claimed in claim 12, further including a step (DI1; DI2; DI3) of image modification based on a modification parameter that is predetermined or read in the data signal, said modification step being applied: between the step (D2) of decoding said at least one view and the step (D3) of recomposition of the current integral image as predictor of the current integral image, and/or between the step (D3) of recomposition of the current integral image as predictor of the current integral image and the step (D4) of reconstruction of the current integral image.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0080] Other features and advantages will become apparent on reading the description of a preferred embodiment with reference to the figures, in which:
[0081]
[0082]
[0083]
[0084]
[0085]
[0086]
[0087]
[0088]
DETAILED DESCRIPTION OF THE CODING PART
[0089] One embodiment of the invention will now be described in which the coding method according to the invention is used to code an integral image or a sequence of integral images according to a binary signal close to that produced by coding employed in a coder conforming to any present or future video coding standard.
[0090] In this embodiment, the coding method according to the invention is implemented in software or hardware by modification of such a coder, for example. The coding method according to the invention is represented in the form of an algorithm including steps C1 to C8 as shown in
[0091] According to the embodiment of the invention, the coding method according to the invention is implemented in a coding device or coder CO as represented in
[0092] As shown in
[0093] The coding method as represented in
[0094] As represented in
[0095] As represented in
[0096] The lens array RL comprises a plurality of microlenses, of which only three microlenses ML.sub.1, ML.sub.2, ML.sub.3 are represented in
[0097] During the acquisition of the integral image II.sub.j, light rays coming from the scene SC pass through each microlens ML.sub.1, ML.sub.2, ML.sub.3 and then impinge on the pixels of each cell of the sensor, in the focal plane of said microlenses. Given the particular configuration of the lens array RL and the matrix of pixels constituting the screen, the light rays: [0098] impinge at a first angle of incidence on the pixels P.sub.1,1, P.sub.2,1, P.sub.3,1, [0099] impinge at a second angle of incidence on the pixels P.sub.1,2, P.sub.2,2, P.sub.3,2, [0100] impinge at a third angle of incidence on the pixels P.sub.1,3, P.sub.2,3, P.sub.3,3, [0101] impinge at a fourth angle of incidence on the pixels P.sub.1,4, P.sub.2,4, P.sub.3,4, [0102] impinge at a fifth angle of incidence on the pixels P.sub.1,5, P.sub.2,5, P.sub.3,5.
[0103] In the manner known in itself, each angle of incidence corresponds to a particular visualization angle according to which an observer has the possibility of visualizing the scene SC. The values of these angles are contained within the characteristic angular range of the microlens concerned. One such angular range, corresponding for example to that of the microlens ML.sub.2, is represented in solid bold line in
[0104] Depending on the required coding context, the image capture device is configured to adapt the number of microlenses that constitute it and their resolution. Accordingly, any integral image acquired by an image capture device of this kind is characterized by a given number of micro-images and a given resolution for each of those micro-images, these two numbers constituting parameters associated with the image capture device.
[0105] During a step C1 represented in
[0106] The step C1 is implemented by a processing software module MDCV_CO as represented in
[0107] According to a first example, if each of the micro-images of the current integral image II.sub.j contains K pixels of different colors, the current integral image II.sub.j is decomposed into: [0108] a view that contains the first pixel of the micro-image MI.sub.1, the first pixel of the micro-image MI.sub.2, . . . , the first pixel of the micro-image MI.sub.N, and/or [0109] a view that contains the second pixel of the micro-image MI.sub.1, the second pixel of the micro-image MI.sub.2, . . . , the second pixel of the micro-image MI.sub.N, [0110] . . . , and/or [0111] a view that contains the Kth pixel of the micro-image MI.sub.1, the Kth pixel of the micro-image MI.sub.2, . . . , the Kth pixel of the micro-image MI.sub.N.
[0112] According to a second example, if each of the micro-images of the current integral image II.sub.j contains K pixels of different colors, the current integral image II.sub.j is decomposed into: [0113] a view that contains the first four pixels of the micro-image MI.sub.1, the first four pixels of the micro-image MI.sub.2, . . . , the first four pixels of the micro-image MI.sub.N, these first four pixels consisting, for the micro-image in question, for example, of the first and second pixels of the first line of that micro-image together with the first and second pixels of the second line of that micro-image, and/or [0114] a view that contains the second four pixels of the micro-image MI.sub.1, the second four pixels of the micro-image MI.sub.2, . . . , the second four pixels of the micro-image MI.sub.N, these second four pixels consisting, for the micro-image in question, for example, of the third and fourth pixels of the first line of that micro-image together with the third and fourth pixels of the second line of that micro-image, [0115] . . . , and/or [0116] a view that contains the last four pixels of the micro-image MI.sub.1, the last four pixels of the micro-image MI.sub.2, . . . , the last four pixels of the micro-image MI.sub.N, these last four pixels consisting, for the micro-image in question, for example, of the penultimate and last pixels of the penultimate line of that micro-image together with the penultimate and last pixels of the last line of that micro-image.
[0117] According to an embodiment represented in
[0118] According to another embodiment represented in
[0122] Said at least one view V.sub.u is coded during a step C2 represented in
[0123] A step C2 of this kind is implemented by a view coding software module MCV_CO as represented in
[0124] A view coding module MCV_CO of this kind conventionally includes: [0125] a prediction module PRED1_CO, [0126] a residual data calculation module CAL1_CO, [0127] a transformation module MT1_CO of type DCT (from the English “Discrete Cosine Transform”), DST (from the English “Discrete Sine Transform”), DWT (from the English “Discrete Wavelet Transform”) [0128] a quantization module MQ1_CO, [0129] an entropy coding module MCE1_CO for example of the type CABAC (from the English “Context Adaptive Binary Arithmetic Coder”) or a Huffman coder known in itself.
[0130] In the manner known in itself, in the step C2 there is executed a sub-step C21 of prediction of said view V.sub.u by known intra and/or inter prediction techniques. To this end, the view V.sub.u is divided into groups of pixels, for example into blocks of pixels, each block being predicted relative to at least one predictor block in accordance with a prediction mode selected from a plurality of predetermined prediction modes. For example, in the case of HEVC coding in Intra prediction mode, there are thirty five predetermined prediction modes.
[0131] A predictor block of this kind is a block of pixels that has already been coded or coded and then decoded or not. A predictor block of this kind is stored beforehand in the buffer memory TAMP_CO of the coder CO as represented in
[0132] For the block in question of the view V.sub.u, an optimum predictor block is obtained following competition between said predetermined prediction modes, for example by minimization of a distortion bit rate criterion well known to the person skilled in the art. The optimum predictor block is considered an approximation of the block in question.
[0133] A predictive view Vp.sub.u is obtained following the sub-step C21.
[0134] A sub-step C21 of this kind is implemented by the prediction module PRED1_CO represented in
[0135] The coding step C2 also includes a sub-step C22 during which the difference is calculated between each block of the view V.sub.u and each respective corresponding optimum predictor block that was selected during the sub-step C21.
[0136] A plurality of residual data blocks is then obtained following the sub-step C22, those residual blocks constituting a residual view Vr.sub.u.
[0137] A sub-step C22 of this kind is implemented by the calculation module CAL1_CO represented in
[0138] The coding step C2 also includes a sub-step C23 in which each residual data block is transformed in accordance with a standard direct transformation operation such as a DCT, DST or DWT type transformation, for example, to produce a plurality of transformed blocks that constitute the transformed view V.sub.u, designated by the reference Vt.sub.u. An operation of this kind is effected by the module MT1_CO as represented in
[0139] The coding step C2 also includes a sub-step C24 in which each transformed block is quantized in accordance with a standard quantization operation such as scalar or vector quantization, for example. A set Vq.sub.u of quantized coefficients is then obtained following this sub-step. A sub-step C24 of this kind is effected by means of the quantization module MQ1_CO as represented in
[0140] A quantization sub-step of this kind is implemented by means of a predetermined quantization step QP.sub.1. How this parameter is determined will be described later in the description.
[0141] The coding step C2 also includes a sub-step C25 in which the set Vq.sub.u of coefficients is coded in a predetermined order, such as in particular the “raster scan” order. In the preferred embodiment, the coding effected is Huffman or arithmetic type entropy coding. The sub-step C25 then consists in: [0142] reading the symbol or symbols of a predetermined set of symbols that are associated with each quantized data block of the view V.sub.u, [0143] associating digital information, such as bits, with the symbol or symbols that have been read.
[0144] An entropy coding sub-step C25 of this kind is implemented by the entropy coding module MCE1_CO represented in
[0145] The value of the quantization step QP.sub.1 is also entropically coded during the sub-step C25.
[0146] Following the step C2 of coding the view V.sub.u, during a step C3 represented in
[0147] The step C3 of producing a stream of this kind is implemented by the data stream generation module MGF1 as represented in
[0148] In the manner known in itself, the signal F.sub.1 contains the information used during the prediction sub-step C21. Such information notably includes the type of prediction (inter or intra) and where applicable the prediction mode selected, the type of partitioning of the blocks of the view V.sub.u if the latter have been subdivided, the index of the reference view and the displacement vector used if an inter prediction mode has been selected. Such information is entropically coded before being transmitted in the stream F.sub.1.
[0149] In particular, the signal F.sub.1 also contains the entropically coded value of the quantization step QP.sub.1.
[0150] According to the invention, said at least one view V.sub.u is decoded during a step C4 represented in
[0151] A step C4 of this kind is implemented by a view decoding software module MDV_CO as shown in
[0152] A view coding module MDV_CO of this kind conventionally includes: [0153] an entropy decoding module MDE1_CO, for example of CABAC (from the English “Context Adaptive Binary Arithmetic Coder”) type or a Huffman decoder known in itself, [0154] a dequantization module MQ1.sup.−1_CO, [0155] an inverse transformation module MT1.sup.−1_CO of type DCT.sup.−1 (from the English “Discrete Cosine Transform”), DST.sup.−1 (from the English “Discrete Sine Transform”), DWT.sup.−1 (from the English “Discrete Wavelet Transform”), [0156] an inverse prediction module PRED1.sup.−1_CO, [0157] a view reconstruction calculation module CAL1.sup.−1_CO.
[0158] The step C4 includes a sub-step C41 of entropy decoding of the entropically coded data associated with the view V.sub.u according to a predetermined order, such as in particular the “raster scan” order. In the preferred embodiment, the decoding effected is Huffman or arithmetic type entropy decoding. The sub-step C41 then consists in: [0159] reading the symbol or symbols of a predetermined set of symbols associated with each quantized block of the view V.sub.u, [0160] associating digital information, such as bits, with the symbol or symbols that have been read.
[0161] Following the aforementioned sub-step C41 there is obtained a plurality of digital information items associated with the quantized data items that were coded during the aforementioned step C25.
[0162] An entropy decoding sub-step C41 of this kind is implemented by the entropy decoding module MDE1_CO represented in
[0163] During the aforementioned sub-step C41 the index of each optimum predictor block that was used to predict each block of the view V.sub.u in the sub-step C21 from
[0164] The coding step C4 also includes a sub-step C42 during which the digital information obtained following the sub-step C41 is dequantized according to a standard dequantization operation that is the inverse operation of the quantization carried out in the sub-step C24. A set VDq.sub.u of dequantized coefficients is then obtained following this sub-step. A sub-step C42 of this kind is effected by means of the dequantization module MQ1.sup.−1_CO as represented in
[0165] The coding step C4 also includes a sub-step C43 during which the set VDq.sub.u of quantized coefficients that is of type DCT.sup.−1, DST.sup.−1 or DWT.sup.−1 is transformed. This transformation is the inverse operation of the transformation effected in the sub-step C23. Following this sub-step there is obtained a plurality of decoded residual data blocks that constitute the decoded residual view V.sub.u designated by the reference VDr.sub.u. An operation of this kind is effected by the module MT1.sup.−1_CO represented in
[0166] The coding step C4 also includes a sub-step C44 of prediction of said view V.sub.u to be decoded by selection of the optimum predictive view consisting of the optimum predictor blocks obtained after the aforementioned sub-step C21.
[0167] A sub-step C44 of this kind is implemented by the prediction module PRED1.sup.−1_CO represented in
[0168] The coding step C4 also includes a calculation sub-step C45 during which the decoded view VD.sub.u is constructed by adding to each of the decoded residual blocks of the decoded residual view VDr.sub.u obtained following the sub-step C43 each of the respective corresponding predictor blocks that were identified in the aforementioned sub-step C41.
[0169] A sub-step C45 of this kind is implemented by the calculation module CAL1.sup.−1_CO represented in
[0170] According to the invention, during a step C5 represented in
[0171] The step C5 is implemented by a view recomposition software module MDCV.sup.−1_CO as represented in
[0172] According to a first example, if each of the micro-images of the current integral image II.sub.j contains K pixels of different colors, the current integral image IIrec.sub.j is recomposed into: [0173] a view that contains the first pixel of the micro-image MI.sub.1, the first pixel of the micro-image MI.sub.2, . . . , the first pixel of the micro-image MI.sub.N, and/or [0174] a view that contains the second pixel of the micro-image MI.sub.1, the second pixel of the micro-image MI.sub.2, . . . , the second pixel of the micro-image MI.sub.N, [0175] . . . , and/or [0176] a view that contains the Kth pixel of the micro-image MI.sub.1, the Kth pixel of the micro-image MI.sub.2, . . . , the Kth pixel of the micro-image MI.sub.N.
[0177] According to a second example, if each of the micro-images of the current integral image II.sub.j contains K pixels of different colors, the current integral image II.sub.j is recomposed into: [0178] a view that contains the first four pixels of the micro-image MI.sub.1, the first four pixels of the micro-image MI.sub.2, . . . , the first four pixels of the micro-image MI.sub.N, these first four pixels consisting, for the micro-image in question, for example, of the first and second pixels of the first line of that micro-image and the first and second pixels of the second line of that micro-image, and/or [0179] a view that contains the second four pixels of the micro-image MI.sub.1, the second four pixels of the micro-image MI.sub.2, . . . , the second four pixels of the micro-image MI.sub.N, these second four pixels consisting, for the micro-image in question, for example, of the third and fourth pixels of the first line of that micro-image and the third and fourth pixels of the second line of that micro-image, [0180] . . . , and/or [0181] a view that contains the last four pixels of the micro-image MI.sub.1, the last four pixels of the micro-image MI.sub.2, . . . , the last four pixels of the micro-image MI.sub.N, these last four pixels consisting, for the micro-image in question, for example, of the penultimate and last pixels of the penultimate line of that micro-image and the penultimate and last pixels of the last line of that micro-image.
[0182] According to the embodiment represented in
[0183] According to the other embodiment represented in
[0187] According to the invention, during a step C6 represented in
[0188] A current residual integral image IIres.sub.j is then obtained following the step C6.
[0189] A step C6 of this kind is implemented by a calculation module CAL2_CO represented in
[0190] A current residual integral image IIres.sub.j of this kind is then considered a two-dimensional (2D) image that undergoes 2D coding during a step C7 represented in
[0191] A step C7 of this kind is implemented by an image coding software module MDCI as represented in
[0192] A two-dimensional image coding module MDCI of this kind may include, for example: [0193] a prediction module PRED2_CO, [0194] a residual data calculation module CAL3_CO, [0195] a transformation module MT2_CO of type DCT (from the English “Discrete Cosine Transform”), DST (from the English “Discrete Sine Transform”), DWT (from the English “Discrete Wavelet Transform”), [0196] a quantization module MQ2_CO, [0197] an entropy coding module MCE2_CO, for example of CABAC type (from the English “Context Adaptive Binary Arithmetic Coder”) or a Huffman coder known in itself.
[0198] The step C7 includes a sub-step C71 of prediction of said residual integral image IIres.sub.j using known intra and/or inter prediction techniques. To this end, the residual integral image IIres.sub.j is subdivided into groups of pixels, for example into blocks of pixels, each block being predicted relative to at least one predictor block in accordance with a prediction mode selected from a plurality of predetermined prediction modes. For example, in the case of HEVC coding in Intra prediction mode, there are thirty-five predetermined prediction modes.
[0199] A predictor block of this kind is a block of pixels that may have already been coded or coded and then decoded. A predictor block of this kind is stored beforehand in the buffer memory TAMP_CO of the coder CO as represented in
[0200] For the block in question of the residual integral image IIres.sub.j, an optimum predictor block is obtained following competition between said predetermined prediction modes, for example by minimization of a distortion bit rate criterion well known to the person skilled in the art. The optimum predictor block is considered an approximation of the block in question.
[0201] A predictive residual integral image IIresp.sub.j is obtained following the sub-step C71
[0202] A sub-step C71 of this kind is implemented by the prediction module PRED2_CO represented in
[0203] The coding step C7 also includes a sub-step C72 during which the difference is calculated between each block of the residual integral image IIres.sub.j and each respective corresponding optimum predictor block that was selected during the sub-step C71.
[0204] A plurality of residual data blocks is then obtained following the sub-step C72, these residual blocks constituting a residual integral image IIresr.sub.j.
[0205] A sub-step C72 of this kind is implemented by the calculation module CAL3_CO represented in
[0206] The coding step C7 also includes a sub-step C73 in which each residual data block obtained in the sub-step C72 is transformed according to a standard direct transformation operation such as for example a transformation of type DCT, DST or DWT, to produce a plurality of transformed blocks that constitute the transformed residual integral image designated by the reference IIrest.sub.j. An operation of this kind is effected by the module MT2_CO represented in
[0207] The coding step C7 also includes a sub-step C74 during which each transformed block obtained in the sub-step C73 is quantized in accordance with a standard quantization operation such as for example scalar or vector quantization. A set IIresq.sub.u of quantized coefficients is then obtained following this sub-step. A sub-step C74 of this kind is effected by means of the quantization module MQ2_CO as represented in
[0208] Also, a quantization sub-step of this kind is implemented by means of a predetermined quantization step QP.sub.2. How this parameter is determined will be described later in the description.
[0209] The coding step C7 also includes a sub-step C75 during which the set IIresq.sub.u of coefficients is coded according to a predetermined order, such as in particular the “raster scan” order. In the preferred embodiment, the coding effected is Huffman or arithmetic type entropy coding. The sub-step C75 then consists in: [0210] reading the symbol or symbols of a predetermined set of symbols that are associated with each quantized data block of the current residual integral image [0211] associating digital information, such as bits, with the symbol or symbols that have been read.
[0212] An entropy coding sub-step C75 of this kind is implemented by the entropy coding module MCE2_CO represented in
[0213] The value of the quantization step QP.sub.2 is also entropically coded during the sub-step C75.
[0214] Following the step C7 of coding the residual integral image IIres.sub.j, during a step C8 represented in
[0215] The step C8 of producing a stream of this kind is implemented by the data stream generation module MGF2 as represented in
[0216] In the manner known in itself, the signal F.sub.2 contains information used during the prediction sub-step C71. Information of this kind notably includes the type of prediction (inter or intra) and where appropriate the prediction mode selected, the type of partitioning of the blocks of the residual integral image IIres.sub.j if the latter have been subdivided, the index of the reference integral image and the displacement vector used when an inter prediction mode has been selected. Information of this kind is entropically coded before being transmitted in the stream F.sub.2.
[0217] In particular, the signal F.sub.2 also contains the entropically coded value of the quantization step QP.sub.2.
[0218] In the embodiment represented in
[0219] Alternatively, the data signals F.sub.1 and F.sub.2 could be transmitted separately to the decoder DO.
[0220] For its part the micro-images resolution parameter of the current integral image II.sub.j, denoted p.sub.rsl in
[0221] Before the step C1 of decomposition into at least one view V.sub.u, the number of views and their position are selected as being those that optimize a predetermined coding performance criterion. For example, the number and the position of the views are selected by minimization for example of a distortion bit rate criterion well known to the person skilled in the art, which is applied in the coding step C2 and in the coding step C7.
[0222] According to a second embodiment, the number and the position of the views are selected as being those that minimize the quantity of data associated with the residual integral image IIres.sub.j obtained following the step C6 from
[0223] Depending on the required coding context, the number and the position of the views that have been selected in this way may be transmitted to the decoder DO.
[0224] The value of the quantization step QP.sub.1 used in the quantization sub-step C24 and the value of the quantization step QP.sub.2 used in the quantization sub-step C74 are selected as optimizing a predetermined coding performance criterion.
[0225] According to one embodiment, the quantization steps QP.sub.1 and QP.sub.2 are each selected by minimization for example of a distortion bit rate criterion well known to the person skilled in the art that is applied in the coding step C2 and in the coding step C7.
[0226] In a variant of the embodiment that has just been described above, as shown in
[0230] The object of this transformation or these transformations is to reduce the amount of data to be coded in the residual integral image IIres.sub.j obtained following the step C6 from
[0231] In a preferred variant, only said transformation step CI2 is applied because it makes it possible to obtain the most effective recomposition of the current integral image II.sub.j.
[0232] Said transformation steps being optional, they are represented in dashed line in
[0233] The transformation steps CI1, CI2 and CI3 are implemented by respective transformation software modules TR1_CO, TR2_CO and TR3_CO as represented in dashed line in
[0234] Each of the transformation steps CI1, CI2 and CI3 uses a corresponding transformation parameter Ptr.sub.1, Ptr.sub.2, Ptr.sub.3.
[0235] According to a first embodiment, the transformation parameters Ptr.sub.1, Ptr.sub.2, Ptr.sub.3 are selected as being those that optimize a predetermined coding performance criterion. For example, the latter are selected by minimization for example of a distortion bit rate criterion well known to the person skilled in the art, which is applied in the coding step C2 and in the coding step C7.
[0236] According to a second embodiment, the transformation parameters Ptr.sub.1, Ptr.sub.2, Ptr.sub.3 are selected as being those that minimize the quantity of data associated with the residual integral image IIres.sub.j obtained following the step C6 from
[0237] Depending on the required coding context, the transformation parameters Ptr.sub.1, Ptr.sub.2, Ptr.sub.3 may be transmitted to the decoder DO.
[0238] One possible example of transformation is smoothing. The parameter associated with this transformation corresponds to the size in pixels of the smoothing filter applied to the image.
[0239] Because decomposition into views leads to a loss of information, i.e. loss of the information contained in the views that are not extracted during the decomposition step C1, during the inverse decomposition step C5 erroneous information is introduced in place of this missing information. The recomposed integral image IIrec.sub.j then contains details that do not correspond to the details of the current original integral image II.sub.j.
[0240] As a result of this: [0241] the details of the current original integral image II.sub.j are not compensated/cancelled during the subtraction step C6 and remain in the residual integral image IIres.sub.j, [0242] details of the recomposed integral image IIrec.sub.j that correspond to erroneous (and therefore useless) information come to be added into the residual integral image IIres.sub.j.
[0243] Smoothing the recomposed integral image IIrec.sub.j (and/or said at least one view V.sub.u and/or said at least one reconstructed view VD.sub.u) makes it possible to smooth these detail zones and consequently to smooth the residual integral image IIres.sub.j, which makes the latter less costly to code.
[0244] In a variant of this smoothing, a geometrical transformation (expansion, erosion, rotation, symmetry, etc.) could be applied to said at least one view V.sub.u and/or to said at least one reconstructed view VD.sub.u and/or to the recomposed integral image
Detailed Description of the Decoding Part
[0245] One embodiment of the decoding method according to the invention will now be described in which the decoding method is implemented in software or hardware by modification of a decoder initially conforming to any current or future video decoding standards.
[0246] The decoding method according to the invention is represented in the form of an algorithm including steps D1 to D5 as represented in
[0247] The decoding method represented in
[0248] To this end, information representing the current integral image II.sub.j to be decoded is identified in the data signal F received at the decoder, as delivered following the coding method from
[0249] Referring to
[0257] Referring to
[0264] The identification steps D1a) and D1b) are implemented by a signal analysis module MI_DO as represented in
[0265] According to the invention, following the step D1a), during a step D2 represented in
[0266] A step D2 of this kind is implemented by a view decoding software module MDV_DO as represented in
[0267] A view decoding module MDV_DO of this kind conventionally includes: [0268] an entropy decoding module MDE1_DO, for example of CABAC type (from the English “Context Adaptive Binary Arithmetic Coder”) or a Huffman decoder known in itself, [0269] a dequantization module MQ1.sup.−1_DO, [0270] an inverse transformation module MT1.sup.−1_DO of type DCT.sup.−1 (from the English “Discrete Cosine Transform”), DST.sup.−1 (from the English “Discrete Sine Transform”), DWT.sup.−1 (from the English “Discrete Wavelet Transform”), [0271] an inverse prediction module PRED1.sup.−1_DO, [0272] a view reconstruction calculation module CAL1.sup.−1_DO.
[0273] The decoding step D2 includes a sub-step D21 of entropy decoding of the entropically coded data associated with the view V.sub.u in a predetermined order such as in particular the “raster scan” order. In the preferred embodiment, the decoding effected is Huffman or arithmetic type entropy decoding. The sub-step D21 then consists in: [0274] reading the symbol or symbols of a predetermined set of symbols that are associated with each quantized block of the view V.sub.u, [0275] associating digital information, such as bits, with the symbol or symbols that have been read.
[0276] Following the aforementioned sub-step D21 there is obtained a plurality of digital information items associated with the quantized data items that were coded during the entropy coding step C25 from
[0277] An entropic decoding sub-step D21 of this kind is implemented by the entropy decoding module MDE1_DO represented in
[0278] During the aforementioned sub-step D21 the index is also decoded of each optimum predictor block that was used to predict each block of the view V.sub.u during the sub-step C21 from
[0279] The decoding step D2 also includes a sub-step D22 during which, using the entropically decoded value of the quantization parameter QP.sub.1, digital information obtained following the sub-step D21 is dequantized according to a standard dequantization operation that is the inverse operation of the quantization effected during the quantization sub-step C24 from
[0280] The decoding step D2 also includes a sub-step D23 during which the set VDq.sub.u of quantized coefficients that is of type DCT.sup.−1, DST.sup.−1 or DWT.sup.−1 is transformed. This transformation is the inverse operation of the transformation effected in the sub-step C23 from
[0281] The decoding step D2 also includes a sub-step D24 of prediction of said value V.sub.u to be decoded by selection of the optimum predictive view consisting of the optimum predictor blocks obtained following the aforementioned sub-step D21.
[0282] A sub-step D24 of this kind is implemented by the prediction module PRED1.sup.−1_DO represented in
[0283] The decoding step D2 also includes a calculation sub-step D25 during which the decoded view VD.sub.u is constructed by adding to each of the decoded residual blocks of the decoded residual view VDr.sub.u obtained following the sub-step D23 each of the respective corresponding predictor blocks that were identified in the aforementioned sub-step D21.
[0284] A sub-step D25 of this kind is implemented by the calculation module CAL1.sup.−1_DO represented in
[0285] According to the invention, during a step D3 represented in
[0286] The step D3 is implemented by a view recomposition software module MDCV.sup.−1_DO as represented in
[0287] During the step D3: [0288] either the number of views and their position are entropically decoded in the data signal F.sub.1 if the coder CO from
[0290] If the number of views and their position are known from the decoder DO: [0291] according to a first embodiment, these parameters were selected during coding as being those that optimize a predetermined coding performance criterion; for example, the number and the position of the views are selected by minimization of a distortion bit rate criterion well known to the person skilled in the art, for example, that was applied in the coding step C2 and in the coding step C7 from
[0293] The integral image II.sub.j to be decoded is then recomposed into one or more views, as represented in the examples from
[0294] According to the invention, following the step D1b), during a step D4 represented in
[0295] A step D4 of this kind is implemented by an image decoding software module MDDI as represented in
[0296] In a corresponding manner to the module MDCI represented in
[0302] The decoding step D4 includes a sub-step D41 of entropy decoding of the entropically coded data associated with the current residual integral image IIres.sub.j as obtained following the sub-step C75 from
[0305] Following the aforementioned sub-step D41 a plurality of digital information items is obtained associated with the quantized data items that were coded during the entropy coding step C75 from
[0306] An entropy decoding sub-step D41 of this kind is implemented by the entropy decoding module MDE2_DO represented in
[0307] During the aforementioned sub-step D41 the index is also decoded of each optimum predictor block that was used to predict each block of the current residual integral image IIres.sub.j during the sub-step C71 from
[0308] The decoding step D4 also includes a sub-step D42 during which there is dequantized, using the value of the entropically decoded quantization parameter QP.sub.2, digital information obtained following the sub-step D41, in accordance with a standard dequantization operation that is the inverse operation of the quantization carried out during the quantization sub-step C74 from
[0309] The decoding step D4 also includes a sub-step D43 during which there is effected a transformation of the set IIDresq.sub.j of quantized coefficients that is of type DCT.sup.−1, DST.sup.−1 or DWT.sup.−1. This transformation is the inverse operation of the transformation effected in the sub-step C73 from
[0310] The decoding step D4 also includes a sub-step D44 of prediction of said residual integral image IIres.sub.j to be decoded by selection of the optimum predictive residual integral image consisting of the optimum predictor blocks obtained following the aforementioned sub-step D41.
[0311] A sub-step D44 of this kind is implemented by the prediction module PRED2.sup.−1_DO represented in
[0312] The decoding step D4 also includes a calculation sub-step D45 during which the decoded residual integral image IIDres.sub.j is constructed by adding to each of the decoded residual blocks of the decoded residual integral image IIDresr.sub.j obtained following the sub-step D43 each of the respective corresponding predictor blocks that were identified in the aforementioned sub-step D41.
[0313] A sub-step D45 of this kind is implemented by the calculation module CAL2.sup.−1_DO represented in
[0314] According to the invention, during a step D5 represented in
[0315] A current decoded integral image IID.sub.j is then obtained following the step D5.
[0316] A step D5 of this kind is implemented by a calculation software module CAL3.sup.−1_DO represented in
[0317] In a variant of the embodiment that has just been described above, as shown in
[0320] In a preferred variant, only said transformation step DI2 is applied using the transformation parameter p.sub.tr2 because it makes it possible to achieve the most efficaceous recomposition of the current decoded integral image.
[0321] Said transformation steps being optional, they are represented in dashed line in
[0322] The transformation steps DI1, DI2, DI3 are implemented by respective transformation software modules TR1_DO, TR2_DO and TR3_DO, as represented in dashed line in
[0323] Each of the transformation steps DI1, DI2 and DI3 uses a corresponding transformation parameter Ptr.sub.1, Ptr.sub.2, Ptr.sub.3.
[0324] During each of these steps DI1, DI2, DI3: [0325] either the parameter or parameters Ptr.sub.1, Ptr.sub.2, Ptr.sub.3 are entropically decoded either during the step D21 or during the step D41 if the coder CO from
[0327] When the parameter or parameters Ptr.sub.1, Ptr.sub.2, Ptr.sub.3 are known to the decoder DO: [0328] according to a first embodiment, these parameters were selected during coding as being those that optimize a predetermined coding performance criterion; for example, the parameter or parameters Ptr.sub.1, Ptr.sub.2, Ptr.sub.3 are selected by minimization of a distortion bit rate criterion well known to the person skilled in the art, for example, that was applied in the coding step C2 and in the coding step C7 from
[0330] It goes without saying that the embodiments that have been described above have been described by way of nonlimiting and illustrative example only and that numerous modifications may easily be made by the person skilled in the art without this departing from the scope of the invention.