METHOD OF MAPPING SOURCE COLORS OF AN IMAGE USING A LUT HAVING COLORS OUTSIDE A SOURCE COLOR GAMUT
20170359491 · 2017-12-14
Inventors
Cpc classification
H04N1/6058
ELECTRICITY
H04N1/6005
ELECTRICITY
H04N1/646
ELECTRICITY
International classification
Abstract
According to this method, the mapping color LUT has input colors that sample not only a source color gamut (included in an input encoding color space in which inputs colors of this LUT are encoded) but also a portion of the input encoding color space which is not included in the source color gamut. Preferably, this color LUT further includes output colors located outside the target color gamut. Accuracy of the mapping is improved, notably for source colors located near the boundary of the source color gamut.
Claims
1. A method of mapping source colors of an image coded in an input encoding color space into target colors coded in an output encoding color space, from a source color gamut included in said input encoding color space into a target color gamut included in said output encoding color space, said method comprising: applying a mapping color LUT to source colors of said image, resulting into mapped colors, wherein said mapping color LUT is formed by pairs of input colors included in said input encoding color space and of corresponding output colors included in said output encoding color space, wherein said mapping color LUT has input colors that sample not only said source color gamut but also a portion of said input encoding color space not included in said source color gamut, and applying a color extension operator to said mapped colors to get said target colors, wherein said color extension operator is such that the application of said color extension operator to output colors of said mapping color LUT results in extended output colors, some of which are located outside said output encoding color space.
2. The method of mapping according to claim 1, wherein said portion of said input encoding color space is connex to said source color gamut.
3. The method of mapping according to claim 1, wherein said color extension operator is such that the application of said color extension operator to the surface obtained by the application of said mapping color LUT to said source color gamut results approximately in said target color gamut.
4. An image processing device configured for mapping source colors of an image coded in an input encoding color space into target colors coded in an output encoding color space, from a source color gamut included in said input encoding color space into a target color gamut included in said output encoding color space, and configured for: applying a mapping color LUT to source colors of said image, resulting into mapped colors, wherein said mapping color LUT is formed by pairs of input colors included in said input encoding color space and of corresponding output colors included in said output encoding color space, wherein said mapping color LUT has input colors that sample not only said source color gamut but also a portion of said input encoding color space not included in said source color gamut, and applying a color extension operator to said mapped colors to get said target colors, wherein said color extension operator is such that the application of said color extension operator to output colors of said mapping color LUT results in extended output colors, some of which are located outside said output encoding color space.
5. An electronic device incorporating an image processing device according to claim 4.
6. The electronic device of claim 5 comprising also a module configured for receiving said image, said color mapping LUT and said color extension operator.
7. The electronic device of claim 5 selected in the group consisting of a camera, a TV set, a monitor, a head mounted display, a set top box, a gateway, a smartphone and a tablet.
8. A computer program product comprising program code instructions to execute the steps of the method according to claim 1, when this program is executed by a processor.
9. A method of mapping source colors of an image coded in an input encoding color space into target colors coded in an output encoding color space, from a source color gamut included in said input encoding color space in a target color gamut included in said output encoding color space, comprising: building an original color LUT such that each of its input colors corresponds to an output color defined by said mapping of said input color, wherein input colors of said original color LUT sample not only said source color gamut but also a portion of said input encoding color space not included in said source color gamut, and wherein at least one output color of said original color LUT is located outside said output encoding color space, applying a color compression operator to compress said original color LUT into a mapping color LUT such that each of its output colors belongs to said output encoding color space, applying said mapping color LUT to source colors of said image to get mapped colors, applying a color extension operator to said mapped colors to get said target colors, wherein said color extension operator is built as the inverse of said color compression operator.
10. The method of mapping of claim 9 comprising transmitting said image, said mapping color LUT and said color extension operator before applying said mapping color LUT to source colors of said image.
Description
BRIEF DESCRIPTION OF DRAWINGS
[0043] The invention will be more clearly understood on reading the description which follows, given by way of non-limiting example and with reference to the appended figures in which:
[0044]
[0045]
[0046]
[0047]
[0048]
[0049]
[0050]
[0051]
DESCRIPTION OF EMBODIMENTS
[0052] It will be appreciated by those skilled in the art that flow charts presented herein represent conceptual views of illustrative circuitry embodying the invention. They may be substantially represented in computer readable media and so executed by a data processing device, whether or not such device is explicitly shown. The functions of the various elements shown in the figures may be provided through the use of hardware capable of executing software in association with appropriate software. Such hardware capable of executing such software generally uses processor, controller, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and non-volatile storage.
[0053] The invention, and notably the steps 3 and 4 of the method described below, may notably be implemented by any device capable of processing images, notably capable of implementing color gamut mapping of images. Therefore, the invention can be notably implemented in an image capture device such as a camera, an image display device such as a TV set, a monitor, a head mounted display, or a set top box or a gateway. The invention can also be implemented in a device comprising both an image capture display device and an image display device, such as a smartphone or a tablet. All such devices comprise hardware capable of executing software that can be adapted in a manner known per se to implement the invention.
First Embodiment
[0054] A first embodiment of the method of color mapping will now be described through four different steps in reference to
[0055] An input encoding color space is chosen such that the source color gamut is included in this input encoding color space. An output encoding color space is chosen such that the target color gamut is included in this output encoding color space as shown on
[0056] The first and second steps below may be implemented for instance on the side of production of a video content on platforms that resize, encode, or generally reformat video content for specific distribution channels such as optical discs, broadcasting or cinema.
1st Step: Building an Original Color LUT Using Input(s) Color(s) Outside the Source Color Gamut:
[0057] In this first step, from the source color gamut and the target color gamut, an original color LUT is built such as to be adapted to map colors from this source color gamut into the target color gamut. This original color LUT is precalculated as in the prior art, but instead of taking input colors sampling only the source color gamut for this precalculation, input colors are taken that sample at least a portion of the input encoding color space which is not included in the source color gamut. Preferably, this portion surrounds the source color gamut.
[0058] Generally, through such a process, the input colors of the color LUT that are outside the source color gamut are generally mapped into corresponding output colors that are located outside the target color gamut, as shown on
[0059] Having obtained this original color LUT, two situations are possible: [0060] a first situation (N on
2.SUP.nd .Step Only in the Second Situation: Color Compression:
[0062] As shown on
[0063] Besides this color compression operation, a color extension operator is built in a manner known per se as the inverse of the color compression operator. This color extension operator will be used in the fourth step below.
[0064] The third and fourth steps below may be implemented for instance on the side of consumption of a video content, more precisely on a device capable of processing images, i.e. for instance on a consumer device such as TV set, a set-top-box, a tablet or any other consumer device mentioned above, or for instance on a professional device such as a video production platform that resize, encode, or generally reformat video content for specific distribution channels such as optical discs, broadcasting or cinema.
[0065] The mapping color LUT, the color extension operator and of course the image to map are provided to this device capable of processing images in a manner known per se, notably by transmission through any usual transmission network as a broadcast, a LAN or a WiFi network. Source colors defining the image is notably provided to such a device through color coordinates representing these source colors in the input encoding color space.
3rd Step: Applying the Mapping Color LUT:
[0066] In this third step, the mapping color LUT—resulting from the 1.sup.st or 2.sup.nd step above depending on the situation N or Y—is applied to source colors of the image to get mapped colors, including, when appropriate, any interpolation from different output colors of this LUT as in the prior art. Such an interpolation is considered as inherent to this third step.
[0067] In the first situation in which all output colors of the original LUT are located inside the output encoding color space after the first step (see
4th Step Only in the Second Situation: Color Extension:
[0068] In the second situation in which a color compression operator has been applied in order to get all output colors of the LUT inside the output encoding color space as shown on
[0069] Note that the application of this color extension operator to these mapped colors changes indeed the mapped colors; such an application is then different from a conversion of colors, i.e. from a change of representation of colors—as for instance a change of color space. For instance, the application of a color conversion operator—as obtained for instance by a combination of a source forward transform representative of a source display device and of a target inverse transform representative of a target display device—would not change colors but only their representation.
[0070] Globally, the combination of applications of the compressed mapping color LUT and of the color extension operator is approximately equivalent to the application of the original color LUT. Due to the interpolation inherent to the third step, it is possible that the result of this combination of applications—including inherent interpolations from different pairs of inputs-outputs extracted from the compressed mapping LUT—differs slightly to the direct application of the original color LUT including the same inherent interpolations, notably because these inherent interpolations are not applied at the same level: as a matter of fact, the color extension step is applied here after these interpolations.
[0071] At the end of the above process, a color mapped image is obtained, in which even source colors located near the boundary of the source color gamut are precisely mapped into target colors.
Second Embodiment
[0072] A second embodiment of the invention will now be described. In this second embodiment, the source color gamut is defined by the D65 white point and the P3 primary colors specified in SMPTE RP 431-2:2011 “D-Cinema Quality—Reference Projector and Environment”, in the following called P3 color gamut. The target color gamut is the color gamut according to ITU-R BT.709, in the following called BT.709 color gamut. The input encoding color space is chosen to be according to ITU-R BT.2020, in the following called BT.2020 color space.
1st Step: Building an Original Color LUT Using Input Colors Outside the P3 Source Color Gamut:
[0073] In this first step, from the source color gamut and the target color gamut, an original color LUT is built such as to be adapted to map colors from the P3 color gamut into the BT.709 color gamut. This original color LUT is precalculated such as known in the prior art according to the following steps:
[0074] More specifically, this original Look Up Table is calculated using the perception-based gamut mapping operating in CIELAB color space, according to the following steps.
[0075] Input colors lying on a regular grid sampling the input color space—for example 9×9×9—are selected, such that they have input color coordinates R.sub.iG.sub.iB.sub.i—for example i=0 . . . 9.sup.3−1—in this input color space. These input color coordinates R.sub.iG.sub.iB.sub.i form the entries of the original Look Up Table. For a number of these input colors, the following steps are applied: [0076] 1. The linear input color coordinates R.sub.iG.sub.iB.sub.i are transformed into linear XYZ color coordinates X.sub.iY.sub.iZ.sub.i according to
where the matrix M.sub.RGB-to-XYZ is defined according to: [0077] 0.636958 0.144617 0.168881 [0078] 0.2627 0.677998 0.059302 [0079] 0 0.028073 1.060985 [0080] 2. The linear XYZ color coordinates X.sub.iY.sub.iZ.sub.i are transformed in a manner known per se into coordinates L.sub.ia.sub.ib.sub.i of CIELAB color space using D65 as white point.
[0081] Then, in a first mapping step, hue mapping modifies the hue of the colors of the input colors by mapping these coordinates L.sub.ia.sub.ib.sub.i into hue mapped coordinates L′.sub.ia′.sub.ib′.sub.i. Hue is an aspect of colour that should generally be preserved during color gamut mapping. However, misalignment of hues of primary and secondary colors may lead to suboptimal behavior of succeeding steps of colour gamut mapping. In fact, non-uniform saturation modifications may occur when the primary colors defining the source colour gamut and the primary colors defining the target colour gamut are significantly mismatched. Hue mapping aims to improve the uniformity of the saturation modification induced by colour gamut mapping, notably to minimize the degradation of colour neighborhood while minimizing the average change of hue. Hue mapping adaptively changes the hue of a P3 primary color towards the hue of the corresponding BT.709 primary color based on the difference in hue and chroma of the P3 and BT.709 primary colors. Preferably, the hue change resulting from the color gamut mapping should be stronger when the difference of saturation between the P3 primary color and the corresponding BT.709 primary color is smaller. The following steps are carried out: [0082] For each primary color, a triangle is formed in a-b plane by the origin and the P3 and BT.709 primary colors. One angle of the triangle is the hue difference Δh. [0083] For each primary color, the difference between the two angles of the triangle opposed to Δh is calculated and called Δα. This difference expresses geometrically the difference of saturation of the P3 and BT.709 primary colors. [0084] For each primary color, the hue is changed by the offset
This offset is the stronger the smaller is the difference of saturation between the primary colors reflected by Δα. [0085] For any color of the input signal to be mapped, an offset is applied to its hue that is interpolated from the offsets of the primary colors. The interpolation is bilinear in hue angles. After application of the offset, first mapped colors are obtained described by coordinates L″.sub.ia″.sub.ib″.sub.i of CIELAB color space.
[0086] Then a chroma mapping acts mainly on the chroma of the first mapped colors resulting into second mapped colors such that all second mapped colors are within the BT.709 color gamut. The following steps are carried out: [0087] For each first mapped color, a straight mapping trajectory is determined including the first mapped color and an anchor point lying on the lightness L axis. [0088] Each first mapped colour is second mapped onto a second mapped colour defined by coordinates L″.sub.ia″.sub.ib″.sub.i in direction to the anchor point along the mapping trajectory. The mapping of a first mapped colour can be described as a modification of the distance D of the first mapped colour from the anchor point into a distance D′ of the second mapped colour from the same anchor point. In order to map D to D′, a knee function is used having a first, linear segment having a slope of one covering 50% of the BT.709 chroma and a second, linear segment compressing all other colors within the BT.709 gamut:
where [0089] D.sub.709 is the distance between the anchor point and the intersection of the mapping trajectory with the BT.709 color gamut, [0090] D.sub.50%=0.5 D.sub.709, and [0091] D.sub.max is the distance between the anchor point and the intersection of the mapping trajectory with the input color gamut (either BT.2020 or P3). [0092] It should be noted that first mapped colors with D>D.sub.max are mapped to second mapped colors with D′>D.sub.709. This means that first mapped colors outside the P3 source color gamut are mapped to colors outside the BT.709 target color gamut.
[0093] To conclude the calculation of the original Look Up Table, the second mapped coordinates L″.sub.ia″.sub.ib″.sub.i are transformed back into linear XYZ color coordinates X′.sub.iY′.sub.iZ′.sub.i using D65 as white point, and the linear XYZ color coordinates X′.sub.iY′.sub.iZ′.sub.i of a mapped color are transformed into linear BT.709 color coordinates R′.sub.iG′.sub.iB′.sub.i according to
where the matrix M.sub.XYZ-to-RGB is defined according to [0094] 15 3.240970 −1.537383 −0.498611 [0095] −0.969244 1.875968 0.041555 [0096] 0.055630 −0.203977 1.056972,
these coordinates R′.sub.iG′.sub.iB′.sub.i being the output colors of the original Look Up Table.
[0097] It should be noted that for this look up table precalculation, instead of taking input colors sampling only the P3 color gamut for this precalculation, input colors are taken that sample the BT.2020 input encoding color space which is significantly larger than the P3 color gamut. In other words, there is a large portion of the BT.2020 input encoding color space which is not included in the P3 source color gamut. By this process, the input colors outside the P3 source color gamut are mapped to output colors outside the BT.709 target color gamut, as shown on
[0098] Having obtained this original color LUT, whether there are output colors outside the output encoding color space, such as shown on
2.SUP.nd .Step: Color Compression:
[0099] As shown on
[0100] This compression operator is defined here to pre-scale the linear BT.709 color coordinates R′.sub.iG′.sub.iB′.sub.i obtained in the described 1.sup.st step resulting in pre-scaled, or compressed linear BT.709 color coordinates R″.sub.iG″.sub.iB″.sub.i. Pre-scaling, or compression, handles color coordinates being out-of-range for those colors having been chroma mapped with a distance D>D.sub.max before chroma mapping. Prescaling of R′.sub.i color coordinates is done according to:
and equivalently for G″.sub.i, B″.sub.i. The prescaled, linear BT.709 color coordinates R″.sub.iG″.sub.iB″.sub.i build an entry of the mapping color LUT.
[0101] Besides this color compression operation, a color extension operator is built in a manner known per se as the inverse of the color compression operator. This color extension operator will be used in the fourth step below.
3rd Step: Applying the Mapping Color LUT:
[0102] In this third step, the mapping color LUT—resulting from the 2.sup.nd step above—is applied to source colors of the image to get mapped colors, including, when appropriate, any interpolation from different output colors of this LUT as in the prior art.
[0103] Thanks to input colors of this LUT located outside the source color gamut, the problem of precision of color mapping interpolation for source color located near the boundary of the source color gamut is advantageously avoided.
4.SUP.th .Step Only in the Second Situation: Color Extension:
[0104] The color extension operator defined at the 2.sup.nd step above as the inverse of the color compression operator is applied to the mapped colors obtained at the 3.sup.rd step above in order to get the final target colors.
[0105] When R.sub.LUTG.sub.LUTB.sub.LUT are the color coordinates of the mapped colors of the 3.sup.rd step, the color extension operator is a linear post-scaling according to:
R.sub.709=m.sub.RR.sub.LUT+b.sub.R
G.sub.709=m.sub.GG.sub.LUT+b.sub.G
B.sub.709=m.sub.BB.sub.LUT+b.sub.B
resulting in color coordinates R.sub.709G.sub.709B.sub.709 of the final BT.709 target colors.
[0106] Globally, the successive applications of the color compression operator, of the compressed mapping color LUT and of the color extension operator is equivalent in terms of color to the application of the original color LUT. However, due to any interpolation from different output colors of the LUT, slight differences may occur. To avoid R.sub.709G.sub.709B.sub.709 coordinates corresponding to colors outside the BT.709 color gamut due to any interpolation, these color coordinates are clipped such that the clipped color coordinates correspond to colors within the B T.709 target color gamut.
[0107] At the end of the above process, a color mapped image is obtained, in which even source colors located near the boundary of the source color gamut are precisely mapped into target colors.
Variant of Second Embodiment:
[0108] The second embodiment described above may include the following variation. When applying a mapping color LUT in an BT.2020 input encoding color space to colors included within PR source color gamut that is smaller than BT.2020, some of the entries of the mapping color LUT may never be used. In this case it can be advantageous to apply an input compression operator and an input extension operator according to the following way:
Modification of the 1.sup.st Step of the 2.sup.nd Embodiment:
[0109] The modification is to apply additionally an input compression operator to the R.sub.iG.sub.iB.sub.i color coordinates before transformation into XYZ color space. The compression operator can be for example:
R′″=R/0.96875
G′″=G/0.96875
and R′″ and G′″, respectively, is used instead of R,G for transformation into XYZ color space. This results in a modified original color LUT. The 2.sup.nd step is not modified but leads no a modified color mapping LUT since input to the 2.sup.nd step is the modified original color LUT.
Modification of the 3.sup.rd Step of the 2.sup.nd Embodiment:
[0110] The modification is to apply additionally an input extension operator being the inverse of the described input compression operator to the color coordinates of the source colors of the image before application of the modified color mapping LUT according to
R″″=0.96875 R.sub.source color of image
G″″=0.96875 G.sub.source color of image
[0111] And then using R″″ and G″″ instead of the color coordinates of the source colors of the image.
[0112] The application of the input extension operator and the modified mapping color LUT gives approximately the same colors as not applying the input extension operator and applying the mapping color LUT. The advantage of the described variation is that the modified mapping color LUT has less unused entries than the mapping color LUT and finally the whole color mapping has a higher precision. For example, when LUTs of size 33×33×33 are used for mapping P3 source colors of images represented in BT.2020 input encoding color space, and if color coordinates of these images are normalized within the range [0,1], the red and green color coordinates never go beyond the value of 0.96875 . Therefore, it is advantageous to use this value in the input extension and input compression operators.
[0113] Other types of input extension and input compression operators are possible that use non-linear terms such as for example
R′″=R.sup.2/0.96875
[0114] or cross color channel terms such as for example
R′″=R G/0.96875.
[0115] Although the illustrative embodiment of the invention has been described herein with reference to the accompanying drawings, it is to be understood that the present invention is not limited to this precise embodiment, and that various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the invention, notably when implementing separately steps 3 and 4 of the above embodiment. All such changes and modifications are intended to be included within the scope of the present invention as set forth in the appended claims. The present invention as claimed therefore includes variations from the preferred embodiment described herein, as will be apparent to one of skill in the art.
[0116] While some of the specific embodiments may be described and claimed separately, it is understood that the various features of embodiments described and claimed herein may be used in combination.