Gamut mapping using luminance parameters
11727895 · 2023-08-15
Assignee
Inventors
Cpc classification
H04N1/6061
ELECTRICITY
G09G2320/0666
PHYSICS
G09G2360/08
PHYSICS
G09G2360/02
PHYSICS
International classification
Abstract
A colour processor for mapping an image from source to destination colour gamuts has an input for receiving a source image including a plurality of source colour points expressed according to the source gamut; a colour characterizer configured to, for each source colour point in the source image, determine a position of intersection of a curve with the boundary of the destination gamut; and a gamut mapper configured to, for each source colour point in the source image: if the source colour point lies inside the destination gamut, apply a first translation factor to translate the source colour point to a destination colour point within a first range of values; or if the source colour point lies outside the destination gamut, apply a second translation factor, different to the first translation factor, to translate the source colour point to a destination colour point within a second range of values.
Claims
1. A computer-implemented method of mapping a source colour point expressed in accordance with a source colour gamut to a destination colour point expressed in accordance with a destination colour gamut in YUV colour space, wherein the luminance of the source colour point is modified in dependence on: the square of the distance of the source colour point from the axis U=V=0; or a maximum magnitude of the U and V coordinates of the source colour point in the YUV colour space.
2. The computer-implemented method as claimed in claim 1, wherein the source colour point is outside the destination colour gamut and the destination colour point is within the destination colour gamut.
3. The computer-implemented method as claimed in claim 1, wherein the source colour point is comprised by a source image and the destination colour point is comprised by a destination image.
4. The computer-implemented method as claimed in claim 1, wherein the luminance of the source colour point is modified in accordance with Y′=Y=(1−τ)*k(Y)*(U.sup.2+V.sup.2), where Y′ is a transformed luminance value for the destination colour point, Y is a luminance value of the source colour point, τ is a translation factor, k(Y) is a polynomial function of the Y coordinate of the source colour point, and (U.sup.2+V.sup.2) is the square of the distance of the source colour point from the axis U=V=0.
5. The computer-implemented method as claimed in claim 4, wherein the polynomial function has form k(Y)=C*(1−2Y), where C is a free parameter.
6. The computer-implemented method as claimed in claim 5, wherein the size of the free parameter, C, can be tuned so as to optimise a colour mapping of a source image comprising the source colour point.
7. The computer-implemented method as claimed in claim 1, wherein the luminance of the source colour point is modified in accordance with Y′=Y+k.sub.1(Y)*max(|U|,|V|), where Y′ is a transformed luminance value for the destination colour point, Y is a luminance value of the source colour point, k.sub.1(Y) is a polynomial function of the Y coordinate of the source colour point, and max (|U|,|V|) is the maximum magnitude of the U and V coordinates in the YUV colour space of the source colour point.
8. The computer-implemented method as claimed in claim 7, wherein the polynomial function has the form k.sub.1(Y)=C*(1−2Y).sup.3, where C is a free parameter.
9. The computer-implemented method as claimed in claim 8, wherein the size of the free parameter, C, can be tuned so as to optimise a colour mapping of a source image comprising the source colour point.
10. The computer-implemented method as claimed in claim 1, wherein the luminance of the source colour point is modified in dependence on a polynomial function of the Y coordinate of the source colour point, the degree of the polynomial function being odd.
11. The computer-implemented method as claimed in claim 10, wherein the polynomial function has the form k.sub.2(Y)=C.sub.2*(2Y−1).sup.5+C.sub.3*(1−2Y).sup.3, where C.sub.2 and C.sub.3 are free parameters.
12. The computer-implemented method as claimed in claim 11, wherein the size of one or both of the free parameters, C.sub.2 and C.sub.3 can be tuned so as to optimise a colour mapping of a source image comprising the source colour point.
13. The computer-implemented method as claimed in claim 10, wherein the polynomial function is selected such that its maxima in magnitude lie substantially at the Y values of first and second vertices of a surface defining the boundary of the destination colour gamut in YUV colour space.
14. The computer-implemented method as claimed in claim 13, wherein the first vertex is the vertex of the surface having the greatest Y value and the second vertex is the vertex of the surface having the smallest Y value.
15. The computer-implemented method as claimed in claim 1, wherein the luminance of the source colour point is shifted along a curve of constant hue and varying luminance, wherein luminance along that curve varies with distance from the centre of the destination colour gamut.
16. The computer-implemented method as claimed in claim 15, wherein the centre of the destination colour gamut is: at the origin of the YUV colour space at U=0, V=0, Y=0.5; a centroid of the destination colour gamut; or a weighted average of the position of some or all of the colour points in the destination gamut.
17. The computer-implemented method as claimed in claim 1, wherein the luminance of the source colour point is shifted along a curve of constant hue and varying luminance towards the luminance value at the centre of the destination colour gamut.
18. The computer-implemented method as claimed in claim 17, wherein the centre of the destination colour gamut is: at the origin of the YUV colour space at U=0, V=0, Y=0.5; a centroid of the destination colour gamut; or a weighted average of the position of some or all of the colour points in the destination gamut.
19. A colour processor configured to map a source colour point expressed in accordance with a source colour gamut to a destination colour point expressed in accordance with a destination colour gamut in YUV colour space, wherein the luminance of the source colour point is modified in dependence on: the square of the distance of the source colour point from the axis U=V=0; or a maximum magnitude of the U and V coordinates of the source colour point in the YUV colour space.
20. A non-transitory machine readable storage medium having stored thereon machine readable instructions that, when processed at a computer system, cause the computer system to implement a method of mapping a source colour point expressed in accordance with a source colour gamut to a destination colour point expressed in accordance with a destination colour gamut in YUV colour space, wherein the luminance of the source colour point is modified in dependence on: the square of the distance of the source colour point from the axis U=V=0; or a maximum magnitude of the U and V coordinates of the source colour point in the YUV colour space.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The present invention will now be described by way of example with reference to the accompanying drawings. In the drawings:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
DETAILED DESCRIPTION
(14) The following description is presented by way of example to enable any person skilled in the art to make and use the invention. The present invention is not limited to the embodiments described herein and various modifications to the disclosed embodiments will be readily apparent to those skilled in the art.
(15) There is provided a processor, which could be embodied in software or hardware, for mapping a source image between colour gamuts. Such a processor is referred to herein as a “colour processor”.
(16)
(17) Colour space converter 503 is used to convert the source image 501 from the colour space in which it is encoded into the colour space in which the colour processor is configured to operate. Colour space converter 506 is used to convert the output of the colour processor from the colour space in which the colour processor is configured to operate into the desired colour space for the output image 502 (e.g. an sRGB colour space for a display screen). If the image processor operates in the same colour space according to which one or both of the source and output images are encoded, then one or both of the colour space converters 503 and 506 will not be required.
(18) In the examples which will now be described, the source image 501 is a YUV encoded image from an image processor and the output image 502 is an RGB encoded image. The colour processor 500 operates in the same YUV colour space as the input image and colour space converter 503 is not therefore required. Colour space converter 506 is configured to convert the output of the colour processor from YUV into the desired RGB colour space. Such conversion could, for example, be performed according to the transform given in equation (2) above. The colour processor 500 is configured to ensure that the colours of the source image are mapped into the available gamut of colours of the output image 503.
(19) From the relationships between Y, U, V and luminance, hue and saturation shown above in equation set (1), transforming from a point (Y,U,V) to a point (Y′,U′,V′) in YUV colour space whilst maintaining constant luminance and hue but allowing saturation to vary can be expressed as:
Y′=Y
U′=τU
V′=τV (3)
where τ is a translation factor. Thus, for a given colour point (Y,U,V), the corresponding line 300 (e.g. as shown in
(20) A point (Y′,U′, V′) can in fact be expressed directly in RGB space as the point (R′,G′,B′) using the coordinates (R,G,B) of the source point (Y,U,V) in RGB space:
R′=τR+(1−τ)Y
G′=τG+(1−τ)Y
B′=τB+(1−τ)Y
(21) It follows that gamut mapper 505 can be configured to directly output the colour points of an output image expressed in RGB format and colour space converter 506 is not required.
(22) Parametric Gamut Mapping
(23) In the following examples, the colour processor is configured to perform the mapping of a source colour point to a colour point within the destination gamut of the output image along a line of constant luminance and hue. The saturation of the colour point is allowed to vary in accordance with the mechanisms described herein.
(24) The operation of colour processor 500 according to a first example will now be described with reference to
(25) Point 302 on line 300 indicates the destination gamut boundary of the colour gamut of the output image along that line. Colour points lying between 302 and 303 on line 300 (i.e. range 305) therefore lie outside the destination colour gamut and must be mapped to colour points lying within the destination colour gamut between 301 and 302 on the line (i.e. range 304 extending over distance 309). As indicated in
(26) A given colour point in a source image 501 defines the line of constant hue and luminance 300 for that colour point, such that the line passes through the colour point. In order to identify the value of τ required to translate a given colour point from its source value to its destination value in YUV colour space, the translation factor τ can be expressed with respect to a suitable scaling parameter γ. The scaling parameter γ for a given colour point represents the value of τ in equation (3) above required to move that colour point along its respective line of constant luminance and hue to the destination gamut boundary. γ represents a relationship between the position of the source colour point and the destination gamut boundary, and can be considered to be a measure of distance of the source colour point to the destination gamut boundary. In the present example, the value of τ in equation (3) above required to move the colour point along its respective line of constant luminance and hue to the source gamut boundary is given by γs. This has been found to minimise the colour distortion introduced into the converted image. γs represents a relationship between the position of the source colour point and the source gamut boundary can be considered to be a measure of distance of the source colour point to the source gamut boundary.
(27) More generally, γs can take any value in the range (1, ∞) and could be a constant over a source image 501. γs could be a value determined from a measure of average size of the source and/or destination colour gamuts. The average size of a gamut could be the average distance from the centre of the gamut (typically the origin of the colour space) to the surface of the gamut, as calculated from one or more points on the surface of the gamut (e.g. the vertices, or a sparse sampling of surface points). For example, γs could be calculated as the ratio of the average size of the source gamut to the average size of the destination gamut. In other examples, γs could be calculated as the ratio of the maximum size of the source gamut to the maximum size of the destination gamut. Fixing γs for a frame can reduce the computational demands of the colour space transformation but can lead to colour distortion. In cases where the translated colour points lie outside the destination gamut, those colour points can be clipped to the surface of the destination gamut along the line of constant luma and hue.
(28) Line 601 in
(29) Given the values of the scaling parameters γ and γs, the translation factor τ for translating that colour point to a point within the destination gamut can be calculated for the two ranges 304 and 305 in
(30)
where A is a free parameter in the range
(31)
to 1 which determines the steepness of the gamut mapping as indicated by 604 in
(32) The free parameter A could be selected from a stored set of parameters for a given source image type, or it could be based on the characteristics of a source image such as the colour content and/or distribution of colours in the image. Alternatively, the free parameter A could be selectable by a user and provided to the gamut mapper 505 as parameter selection 507 in order to optimise the appearance of a colour-compressed image. This can be particularly advantageous because the optimal expression of a given source image in a limited colour gamut can be very subjective, depending on the nature of the image and the perceptions of the particular user. In order to provide the user with a means for optimising the appearance of an image mapped between gamuts by the colour processor, software could be configured to provide an interface through which a user of a data processing system (e.g. a personal computer) comprising the colour processor 500 (embodied in either hardware or software) can adjust parameter A until an optimal converted image is achieved.
(33) For each colour in a received source image 501, the colour characterizer 504 of the colour processor 500 is configured to identify the destination and source gamut boundaries for a respective line of constant hue and luminance 300. This can be achieved by calculating the intersection of the line of constant hue and luminance for the colour point with the respective boundaries of the source and destination colour gamuts specified for the source and output images. The scaling parameters γ and ys required to move the source point along the line of constant hue and luminance to the destination and source gamut boundaries, respectively, can then be identified. The scaling parameter γ is indicative of a relationship (e.g. the ratio) between the position of the source colour point on the line 300 and the position of the destination gamut boundary 302 on the line 300. For example, if x is the position of the source colour point on the line 300 and D is the position of the destination gamut boundary 302 on the line 300, then
(34)
Similarly, in the present example, the scaling parameter γ.sub.s is indicative of a relationship (e.g. the ratio) between the position of the source colour point on the line 300 and the position of the source gamut boundary 303 on the line 300. For example, if x is the position of the source colour point on the line 300 and S is the position of the source gamut boundary 303 on the line 300, then
(35)
Therefore, by finding the positions on the line 300 of the source colour point (x), the destination gamut boundary (D) and the source gamut boundary (S), the values of the scaling parameters γ and γ.sub.s can be determined. From γ and ys the translation factor τ for that colour point can be calculated and applied by gamut mapper 505 so as to effect the colour mapping.
(36) In less preferred examples, the value of γs is determined as a fixed value for source image 501 and need not be calculated in respect of each source colour point in the image.
(37) The colour points at γ values between 0 and 1 are the set of points on the line 300 which lie outside the destination gamut and the colour points at γ values between 1 and γ.sub.s lie within the destination gamut. As shown in
(38) By way of comparison,
(39) The principles set out in the example shown in
(40)
(41) As for the example described in relation to
(42)
where A is a free parameter in the range
(43)
to 1, and B is a free parameter in the range
(44)
to γ.sub.s which determines the compression performed by the respective scaling operations performed on colour point sets 405 and 406. The free parameters determine the position of transition points 410 and 411 shown in
(45)
(46) Line 701 in
(47) The choice of A and B in equations (4) above determine the properties of the gamut mapping. Parameter A determines the position of transition point 704 in
(48) For each colour in a received source image 501, the colour characterizer 504 of the colour processor 500 is configured to identify the destination and source gamut boundaries for a respective line of constant hue and luminance 300. This can be achieved by calculating the intersection of the line of constant hue and luminance for the colour point with the respective boundaries of the source and destination colour gamuts specified for the source and output images. The scaling parameters γ and γ.sub.s required to move the source point along the line of constant hue and luminance to the destination and source gamut boundaries, respectively, can then be identified. From γ and γ.sub.s the translation factor τ for that colour point can be calculated and applied by gamut mapper 505 so as to effect the colour mapping.
(49) A flowchart illustrating an exemplary method of performing a parametric gamut mapping is shown in
(50) At 853 a determination is made as to whether the source colour point lies inside or outside the destination gamut. If the colour point lies inside the boundary of the destination gamut, a translation factor τ.sub.1 is applied 854 to the colour point to translate it into a first range of values in the destination gamut (e.g. indicated by 306 in
(51)
Otherwise, a translation factor τ.sub.2 is applied 855 to the colour point to translate it into a second range of values in the destination gamut (e.g. indicated by 307 in
(52) The method shown in
(53) More generally, the principles of the above parametric approach to performing gamut mapping can be extended so as to translate colours along curves which maintain two colour characteristics constant other than luminance and hue. For example, any two of luminance, hue, chrominance, brightness, lightness, colourfulness, and saturation can be maintained constant. This can be appropriate in colour spaces other than YUV and for particular applications of gamut mapping—e.g. when processing false colour images, or with images processed so as to yield artistic effects or to provide enhancement of particular features in the images.
(54) Pseudo-Perceptual Mapping
(55) In combination with the above parametric approach to performing gamut mapping or as an alternative to the above parametric approach, a pseudo-perceptual gamut mapping technique can be employed to improve a mapping between colour gamuts. This approach is particularly useful in improving the appearance of images that contain colour gradients (e.g. computer generated or synthetic images), especially where those colours (when represented geometrically in colour space) occur close to the vertices of the destination colour gamut that present a large angle with respect to a colour space axis whose value is being held constant during the mapping. For example, the yellow 106 and blue 107 vertices of the RGB cuboid shown within YUV space in
(56) The pseudo-perceptual gamut mapping technique will now be explained by way of example with reference to
(57) Colour processor 500 can be configured to perform the pseudo-perceptual gamut mapping technique described herein either in combination with the above parametric gamut mapping, or as a modification to a conventional gamut scaling operation (in which case colour characterizer 504 may not be present). The variables C, C.sub.1 and C.sub.2 described below can be introduced to the gamut mapper 505 as a parameter selection 507, for example from an automated algorithm configured to select appropriate values for the variables (e.g. in dependence on characteristics of the source image 501), or from a user looking to optimise output image 502.
(58) A colour translation in YUV space along a line of constant luminance and hue is set out in equation set (3) above. Rather than maintaining luminance at a constant value during a mapping, the pseudo-perceptual gamut mapping technique modifies luminance during a mapping. In a first example, a colour translation is performed with luma being scaled during a mapping operation such that colour points at large values of U and V are mapped along a curve so as to enter the destination gamut at a point closer to the centre of the destination gamut than would be the case if the mapping were performed at constant luma (representing luminance). As will now be demonstrated with reference to
(59) The origin of the YUV colour space at Y=0.5, U=V=0 typically defines the centre of the destination gamut, but it will be appreciated that the centre of the destination gamut could be anywhere in the YUV colour space within the source gamut. The following example can be readily extended to the case in which the centre of the destination gamut does not lie at the origin of the YUV colour space. The centre of the destination gamut can be calculated in any suitable manner: it could be implicit from the colour space in which the gamut is defined, it could be the centroid of the gamut, a weighted average of the position of some or all of the colour points in the destination gamut, or any other measure of a centre point of the gamut. Whichever the technique used to calculate its position, the centre of the destination gamut need not be determined precisely. The centre of the destination gamut could be a rough approximation determined by any suitable technique—e.g. by determining the centre point using a sparse sampling of colour point positions in the gamut, or by using only the positions of the vertex points of the surface of the colour gamut in an estimate, to give just two examples. A determined centre of the destination gamut could be within, say, 20, 30 or 40% of a “true” measure of the chosen destination gamut centre.
(60)
(61) In a first example, a gamut mapping transformation can be performed in accordance with the following set of transformations:
Y′=Y+(1−τ)*k(Y)*(U.sup.2+V.sup.2)
U′=τU
V′=τV
where τ is the translation factor introduced in the parametric gamut mapping described above and k(Y)=C*(1−2Y) with C being a free parameter which is greater than or equal to zero whose size can be tuned so as to modify the strength of the modification in luma performed by a mapping operation. This set of transformations represents a mapping operator in this example. For example, it can be advantageous for C to be determined by a user so as to provide the user with a means for optimising the appearance of an image mapped between gamuts by the colour processor. This could be achieved through the provision of appropriate software to provide an interface through which a user of a data processing system (e.g. a personal computer) comprising the colour processor 500 (embodied in either hardware or software) can adjust parameter C until an optimal converted image is achieved.
(62) In a second example, the luma transform in a mapping operator is instead given by:
Y′=Y+k.sub.1(Y)*max(|U|,|V|) (5)
where k.sub.1(Y)=C*(1−2Y).sup.3, with C being a free parameter whose size can be tuned so as to optimise a colour mapping of a given source image. This luma transform similarly applies the greatest luma modification to colour points having U and V values which are large in magnitude (such as the blue and yellow vertices of a cuboid describing an RGB gamut). From a computational point of view, finding the maximum of the magnitudes of two numbers is generally more straightforward than calculating the sum of two squares.
(63) The variation of k.sub.1(Y) with Y is illustrated in
(64) It can be seen from the above luma values that colour points close to the blue and yellow vertices will experience the largest modification in luma during a mapping operation as the factor (1−2Y).sup.3, contained in k(Y) achieves its smallest (Y=0.11, Blue) and largest (Y=0.89, Yellow) values among all the ones corresponding to the other vertices of the operating gamut (Red, Purple, Green and Cyan). The corners corresponding to black and white would correspond to an even larger value of such factor, but they lie in the U=V=0 axis so the contribution of the factor is mitigated by the term max (|U|, |V|) in equation (5) above.
(65) The performance of the k(Y) function can be further improved through the use of higher order polynomials so as to move the peak values of the function away from the extremes of luma towards the values of Y at which the blue and yellow vertices occur. This is because the greatest colour distortion when operating in YUV colour gamuts occurs close to the blue and yellow vertices, not at the extremes of luma (the black and white vertices). A third example of the luma transform in a mapping operator is illustrated in
k.sub.2(Y)=C.sub.2*(2Y−1).sup.5+C.sub.3*(1−2Y).sup.3
(66) In
(67) A flowchart illustrating an exemplary method of performing a pseudo-perceptual mapping is shown in
(68) A colour point can be any kind of representation of a colour in an image according to the respective colour space of that image. For example, a colour point can be the colour value of a pixel in a digital image expressed in an RGB colour space within some defined gamut.
(69) The value of a substantially constant value can be maintained within a few percent (e.g. 2, 3, 4, or 5 percent) of its initial or average value, or exactly at a constant value.
(70) The colour processor of
(71) Colour processors configured in accordance with the present invention could be embodied in hardware, software or any suitable combination of hardware and software. A colour processor of the present invention could comprise, for example, software for execution at one or more processors (such as at a CPU and/or GPU), and/or one or more dedicated processors (such as ASICs), and/or one or more programmable processors (such as FPGAs) suitably programmed so as to provide functionalities of the data processing device, and/or heterogeneous processors comprising one or more dedicated, programmable and general purpose processing functionalities. In preferred embodiments of the present invention, the colour processor comprises one or more processors and one or more memories having program code stored thereon, the data processors and the memories being such as to, in combination, provide the claimed colour processor and/or perform the claimed methods.
(72) The term software as used herein includes executable code for processors (e.g. CPUs and/or GPUs), firmware, bytecode, programming language code such as C or OpenCL, and modules for reconfigurable logic devices such as FPGAs. Machine-readable code includes software and code for defining hardware, such as register transfer level (RTL) code as might be generated in Verilog or VHDL.
(73) Any one or more of the algorithms and methods described herein could be performed by one or more physical processing units executing program code that causes the unit(s) to perform the algorithms/methods. The or each physical processing unit could be any suitable processor, such as a CPU or GPU (or a core thereof), or fixed function or programmable hardware. The program code could be stored in non-transitory form at a machine readable medium such as an integrated circuit memory, or optical or magnetic storage. A machine readable medium might comprise several memories, such as on-chip memories, computer working memories, and non-volatile storage devices.
(74) The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The applicant indicates that aspects of the present invention may consist of any such individual feature or combination of features. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.