Method and device for selecting an image dynamic range conversion operator

Abstract

The invention relates to a method for selecting an image dynamic range conversion operator from among a set of image dynamic range conversion operators by optimizing a quality criterion, each image dynamic range conversion operator being applied to an original image to obtain an image, called the modified image, whose dynamic range is lower than that of the original image, wherein said quality criterion is calculated, for each image dynamic range conversion operator, as a function of a distortion calculated from a reconstructed version after coding and decoding the original image modified by said image dynamic range conversion operator.

Claims

1. A method for selecting an image dynamic range conversion operator from among a set of image dynamic range conversion operators, said method comprising: applying each image dynamic range conversion operator to an original image to obtain a modified image, whose dynamic range is lower than that of the original image; calculating a quality criterion for each image dynamic range conversion operator, as a function of a distortion calculated between said modified image and a reconstructed version after coding; selecting an image dynamic with a minimum distortion among the distortion calculated; and decoding the image modified by said image dynamic range conversion operator based on the selected image dynamic.

2. The method according to claim 1, wherein the distortion is calculated between said modified image and said reconstructed version once their dynamic ranges have been extended by application of an operator which is the inverse of that conversion operator used to obtain said modified image.

3. The method according to claim 1, wherein said quality criterion is also a function of a bitrate of the modified and coded image.

4. A method for coding an original image of given dynamic range using an image whose dynamic range lower than that of the original image has been obtained by application of an image dynamic range conversion operator to the original image, wherein the image dynamic range conversion operator is selected from among a set of image dynamic range conversion operators according to the method of claim 1.

5. A device for selecting an image dynamic range conversion operator from among a set of image dynamic range operators, said device comprising for a processor configured to: apply an image dynamic range conversion operator to an original image to obtain an image, called the modified image, whose dynamic range is lower that of the original image, calculate a quality criterion for each image dynamic range conversion operator as a function of a distortion calculated between said modified image and a reconstructed version after coding; select an image dynamic with a minimum distortion among the distortion calculated; and decode the image modified by said image dynamic range conversion operator based on the selected image dynamic.

6. An apparatus for converting an original image sequence whose luminance values belong to a given dynamic value range, said apparatus comprising: several dynamic range conversion operators intended to be applied to luminance component of each image of the original image sequence to obtain a modified luminance component whose values belong to a lower dynamic value range than that of the original image sequence, and a device for selecting a dynamic range conversion operator from among said image dynamic range conversion operators which is in accordance with claim 5.

7. A device for coding an original image of given dynamic range using an image whose dynamic range lower than that of the original image has been obtained by application of an image dynamic range conversion operator to the original image, wherein the image dynamic range conversion operator is selected from among a set of image dynamic range conversion operators according to the device of claim 5.

Description

4. LIST OF FIGURES

(1) The invention will be better understood and illustrated by means of non-restrictive embodiments and advantageous implementations, with reference to the accompanying drawings, wherein:

(2) FIG. 1 shows a diagram of an embodiment of the method for selecting an image dynamic range conversion operator;

(3) FIG. 2 shows a diagram of a variant of the method for selecting an image dynamic range conversion operator;

(4) FIG. 3 shows a diagram of a variant of the method for selecting an image dynamic range conversion operator;

(5) FIG. 4 shows a block diagram of a device for selecting an image dynamic range conversion operator which implements a method according to the invention;

(6) FIG. 5 shows a block diagram of a system for converting image dynamic range according to the invention.

5. DETAILED DESCRIPTION OF THE INVENTION

(7) Hereafter are described several embodiments and variants of a method for selecting an image dynamic range conversion operator from among a set of M image dynamic range conversion operators TMOi (i=1 to M). The references common to these embodiments and variants designate the same entities.

(8) This set can, according to an example, comprise one or more of the image dynamic range conversion operators described in the introductory section. Moreover, this term of image dynamic range conversion operator can also include processing which makes it possible to ensure temporal coherence of the brightness between the images of the original image sequence such as that described in the introductory section. This processing, which is applied following an operator, can be applied to one or more of the operators of the set.

(9) In the case where the dynamic range of all the images of an image sequence must be modified (converted or adapted), the image dynamic range conversion operator is applied to each image of the image sequence.

(10) FIG. 1 diagrammatically shows an embodiment of the method for selecting an image dynamic range conversion operator TMOo from among a set of M image dynamic range conversion operators TMOi (i=1 to M).

(11) The method comprises a step 10 of applying image dynamic range conversion operators during which each operator TMOi of the set is applied to each image I.sub.1 presenting itself at the input of the method. A number M of images I.sub.2.sup.i are then obtained at the output of step 10. Each of these images I.sub.2.sup.i is an image called a modified image that is to say an image whose dynamic range is lower than that of the original image I.sub.1. Note that there are M images I.sub.2.sup.i for each image I.sub.1.

(12) The method also comprises a step 20 of coding during which each image I.sub.2.sup.i is coded according to a known method. This coding method can be, for example, of JPEG type in the case of still images or an image sequence coder for example of ITU-T H.265/MPEG-H Part 2 ‘High Efficiency Video Codec’ (HEVC) type.

(13) The method also comprises a step 30 of reconstructing an image during which an image I.sub.3.sup.i is reconstructed after decoding the code obtained by coding each image I.sub.2.sup.i. For this purpose, the code at the output of step 20 is decoded by a dual method of the coding method used during step 20. As many images I.sub.3.sup.i are therefore reconstructed as images I.sub.2.sup.i.

(14) In a general way, the method also comprises a step 40 of optimising a quality criterion during which a quality criterion Qi is calculated for each image dynamic range conversion operator TMOi as a function of a distortion Di calculated from a reconstructed version I.sub.3.sup.i after coding and decoding the image I.sub.2.sup.i modified by the image dynamic range conversion operator TMOi.

(15) According to the embodiment of FIG. 1, during step 40, the distortion Di is calculated between an image I.sub.2.sup.i and an image I.sub.3.sup.i.

(16) FIG. 2 diagrammatically shows a variant of the embodiment described in relation to FIG. 1.

(17) According to this variant, the method comprises a step 50 of expanding dynamic range during which the dynamic ranges of each image I.sub.3.sup.i and of each image I.sub.2.sup.i are extended by application of an operator EOi which is the inverse of that TMOi used to obtain said image I.sub.2.sup.i.

(18) The distortion Di is thus calculated between two images whose dynamic range is that of the original image. More specifically, the distortion Di is calculated between the original image modified I.sub.2.sup.i by said image dynamic range conversion operator TMOi, and a reconstructed version after coding and decoding the original image modified I.sub.2.sup.i by said image dynamic range conversion operator TMOi, the dynamic ranges of the image I.sub.2.sup.i and of this reconstructed version having been extended during step 50.

(19) This variant makes it possible to know which is the image dynamic range conversion operator which makes it possible to obtain a better compression rate without needing to refer to the original image.

(20) FIG. 3 diagrammatically shows a variant of the embodiment described in relation to FIG. 1.

(21) According to this variant, the distortion Di is calculated between the original image I.sub.1 and an image I.sub.3.sup.i, the dynamic range of this reconstructed version I.sub.3.sup.i being extended by application of an operator EOi which is the inverse of that TMOi used to obtain the modified image I.sub.2.sup.i (step 50).

(22) The invention is in no way limited to one type of image or image sequence coding or image dynamic range conversion operator.

(23) However, in the case where a method for image sequence coding uses a temporal predictor, the inventor has observed that the image dynamic range conversion operator selected was that (or one of those) whose resultant images were processed in order to preserve the temporal coherence between the images of the original image sequence. In fact, there are two major reasons which cause the degradation of the performances of a motion estimator used in a temporal predictor.

(24) The first comes from a flickering in the luminance values from one image to another which appears when no processing is applied to the images modified by an image dynamic range conversion operator. These flickering artefacts are due to the change in the key values relative to temporally successive images. In fact, if the key k changes from one image to another, the modification curve (of sigmoid form) also changes, which results in different modifications for luminance values nevertheless similar in the original images.

(25) The second comes from the non-preservation of the temporal coherence of the brightness of the original images. In fact, as the quality of a motion estimator lies in the low dynamic range of the residue image obtained from the difference between the original image and the predicted image, the more incoherent the prediction image with the original image in terms of brightness, the more this residue image will have a high dynamic range.

(26) According to a variant of the method, the quality criterion Qi is also a function of the bitrate Ri of the modified I.sub.2.sup.i and coded original image.

(27) The quality criterion Qi is then given by:
Qi=γ*Ri+Di

(28) where Ri is the size of an image or the bitrate of an image sequence calculated from the images I.sub.2.sup.i.

(29) The distortion Di is for example given by a distance of mean square error (MSE) type. The invention is not limited to such a metric and any other metric can be used.

(30) The image dynamic range conversion operator which is selected at the end of the method is therefore that which optimises (minimises in the case of a square error) the quality criterion Qi.

(31) In the case where the image dynamic range conversion operator is given by equations (2) and (4), the corresponding operator EOi is given by:

(32) $L_s=\frac{L_d}{1-L_d}$$\mathrm{where}$$L_w=L_s.Math.\frac{k}{a}$

(33) and where a is the chosen exposure value and k is the key given by equation (3).

(34) If the operator calculates a key as a function of the pixels of this image and the pixels of a predetermined number of images which temporally precede the image to be converted in the image sequence then the new key must be used.

(35) In the case where the image dynamic range conversion operator compares an original image using a visual response of the human visual system (HVS) with an image whose luminance values belong to a dynamic range adapted to a screen, a minimisation of a residual error between the results of the responses of the human visual system is calculated during these methods. This minimisation results in a conversion curve which is piece-wise linear and calculated image by image. The corresponding operator EOi is then calculated by inverting the conversion curve which is piece-wise linear and calculated image by image, and this for each image.

(36) According to a variant relating to the case where an image dynamic range conversion operator is followed by processing which preserves the temporal coherence of the images of the original image sequence, the inverse processing of this processing is applied to the image I.sub.3.sup.i before the inverse operator is applied to this image.

(37) For example, in the case where the processing is that given by equation (9), the inverse processing will be given by:

(38) $L_d=\frac{1}{R}{L}_d^{\prime }$

(39) where L′.sub.d is the luminance component of the image I.sub.3.sup.i.

(40) According to one of its hardware aspects, the invention relates to a device 400 for selecting an image dynamic range conversion operator described with reference to FIG. 4.

(41) Device 400 comprises the following elements, interconnected by a digital address and data bus 40: A calculation unit 43 (also called a central processing unit); A memory 45; A network interface 44, for interconnections between device 400 and other remote devices connected via a connection 41;

(42) The calculation unit 43 can be implemented by a (possibly dedicated) microprocessor, a (possibly also dedicated) microcontroller, etc. The memory 45 can be implemented in a volatile and/or non-volatile form such as a RAM (random access memory), a hard disc, an EPROM (erasable programmable ROM), etc. Device 400 is configured to implement a method according to the invention described in relation to FIGS. 1 to 3.

(43) For this purpose, means 43, 44 and possibly 45 cooperate with each other to apply an image dynamic range conversion operator TMOi to an original image I.sub.1 to obtain a modified image I.sub.2.sup.i, whose dynamic range is lower than that of the original image, and to optimise a quality criterion to select a dynamic range conversion operator. The means for optimising the quality criterion are configured so that said quality criterion Qi is calculated for each image dynamic range conversion operator TMOi as a function of a distortion Di calculated from a reconstructed version I.sub.3.sup.i after coding and decoding the original image modified I.sub.2.sup.i by said image dynamic range conversion operator TMOi.

(44) According to another hardware aspect, the invention relates to a system CONV for converting the dynamic range of an image whose luminance values belong to a given dynamic value range (HDR) to an image whose luminance values belong to a lower dynamic value range (LDR) than that of the original image. This is usually referred to as HDR to LDR dynamic range reduction.

(45) FIG. 5 shows a block diagram of such a system for converting an image which implements a method according to the invention.

(46) The system CONV comprises M (>1) dynamic range conversion operators TMOi which are intended to be applied to luminance component Lw of an original image SIO to obtain a modified image whose luminance values belong to a lower dynamic value range LDR than that of image SIO (HDR).

(47) In the case of a colour image, the system CONV comprises means GLW for obtaining the luminance component Lw of this colour image SIO. For example if an image SIO is expressed in an (R, G, B) colour space, the image is transformed in order to be expressed in the (X, Y, Z) colour space so as to recover the Y channel of the (X, Y, Z) space which forms the luminance component Lw. It is widely known to use such colour space transformation means. Other examples of means GLW can be used without leaving the scope of the invention. The system CONV also comprises means DIV and MULT for the purpose of conserving a constant saturation and the hue of the colours. These means DIV are configured to divide the R, G and B colour components corresponding to a colour image SIO by component Lw and means MULT are configured to multiply the R, G and B colour components thus modified by the modified luminance component. The three components originating from this multiplication are then expressed in floating values. To obtain a modified image LDR, these three components originating from this multiplication are submitted at the input of means Ftol of the system CONV and undergo a conversion of their values to whole values which belong to a dynamic value range for the screen on which the modified colour image must be displayed.

(48) According to the invention, the conversion system also comprises a device for selecting the dynamic range conversion operator TMOo from among the M conversion operators TMOi. This device is in accordance with device 400 described in relation to FIG. 4.

(49) According to another of its aspects, the invention relates to a method for coding an original image of given dynamic range and an image whose dynamic range lower than that of the original image has been obtained by application of an image dynamic range conversion operator to the original image.

(50) An example of such a coding method is to predict each original image (an image often called the HDR image) using an image whose dynamic range is lower (an image often called the LDR image). This type of coding method also includes those which encode both the HDR image and the LDR image.

(51) This type of coding method is characterised, according to the invention, in that the image dynamic range conversion operator used to obtain the LDR image is selected from among a set of image dynamic range conversion operators according to one of the methods described in relation to FIGS. 1 to 4. Moreover, in the case where an operator EOi is used by the selection method, the operator EOi carries out an operation which is the inverse of that produced by an operator TMOi.

(52) In FIGS. 1 to 5, the modules shown are functional units that may or may not correspond to physically distinguishable units. For example, these modules or some of them can be grouped together in a single component or circuit, or constitute functions of the same software. On the contrary, some modules may be composed of separate physical entities. The devices and systems compatible with the invention are implemented according to a purely hardware embodiment, for example in the form of a dedicated component (for example in an ASIC (application specific integrated circuit) or FPGA (field-programmable gate array) or VLSI (very large scale integration) or of several electronic components integrated into a device or even in the form of a mixture of hardware elements and software elements.