Color gamut size metric estimation
09961238 ยท 2018-05-01
Assignee
Inventors
Cpc classification
H04N1/6061
ELECTRICITY
International classification
Abstract
A gamut size metric is used in all phases of color image processing (e.g., capture, transmission, and display). In general, the gamut size metric is a single-valued metric that changes as image content changes. More particularly, a gamut boundary histogram is determined and used to estimate a gamut size metric. A gamut size metric identifies a minimum size gamut needed to encompass each pixel in an image, where the gamut size is limited at one end by a first device independent gamut (S.sub.1), and at another end by a second device independent color space (S.sub.2), where S.sub.1 is wholly enclosed within S.sub.2. The gamut size metric may be based on strict pixel color value differences. In other embodiments the gamut size metric may take into effect perceptual color differences and significance.
Claims
1. A method for gamut size estimation in image processing, comprising: obtaining a color image; processing, by a computer processor, each of a plurality of pixels of the color image by determining a gamut boundary value for a pixel, wherein the gamut boundary value identifies a smallest size gamut that encapsulates the pixel, and updating a bin of a gamut boundary histogram that corresponds to the gamut boundary value; and determining, by the computer processor, a gamut size metric based on the gamut boundary histogram, wherein the gamut size metric is indicative of a minimum gamut needed to represent the color image.
2. The method of claim 1, wherein determining the gamut size metric comprises: selecting a relevant portion of the gamut boundary histogram; determining an equilibrium point of the selected relevant portion of the gamut boundary histogram; and determining the gamut size metric based on the selected portion of the gamut boundary histogram and the identified equilibrium point.
3. The method of claim 2, wherein selecting a relevant portion of the gamut boundary histogram comprises: identifying a percentile value; identifying a bin in the gamut boundary histogram corresponding to the percentile value, the bin corresponding to the percentile value forming a first end of the relevant portion of the gamut boundary histogram; and setting a second end of the relevant portion of the gamut boundary histogram equal to a highest bin of the gamut boundary histogram.
4. The method of claim 2, wherein determining an equilibrium point of the selected relevant portion of the gamut boundary histogram comprises: assigning a weight corresponding to a count for each of the bins of the relevant portion of the gamut boundary histogram; and determining a balance point of the relevant portion of the gamut boundary histogram based on moments corresponding to the weight for each of the bins of the relevant portion of the gamut boundary histogram.
5. The method of claim 4, wherein determining the gamut size metric is further based on a function of the balance point.
6. The method of claim 1, wherein acquiring a color image comprises: acquiring an original color image; and obtaining a lower resolution version of the original color image.
7. The method of claim 1, wherein updating a bin of a gamut boundary histogram comprises updating the bin of the gamut boundary histogram corresponding to the gamut boundary value by an amount equal to a visibility weight value for the pixel.
8. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to: obtain a color image; process each of a plurality of pixels of the color image by determining a gamut boundary value for a pixel, wherein the gamut boundary value identifies a smallest size gamut that encapsulates the pixel, and updating a bin of a gamut boundary histogram that corresponds to the gamut boundary value; and determine a gamut size metric based on the gamut boundary histogram, wherein the gamut size metric is indicative of a minimum gamut needed to represent the color image.
9. The non-transitory program storage device of claim 8, wherein the instructions to cause the one or more processors to determine a gamut size metric further comprise instructions to cause the one or more processors to: select a relevant portion of the gamut boundary histogram; determine an equilibrium point of the selected relevant portion of the gamut boundary histogram; and determine the gamut size metric based on the selected portion of the gamut boundary histogram and the identified equilibrium point.
10. The non-transitory program storage device of claim 9, wherein the instructions to cause the one or more processors to select a relevant portion of the gamut boundary histogram comprise instructions to cause the one or more processors to: identify a percentile value; identify a bin in the gamut boundary histogram corresponding to the percentile value, the bin corresponding to the percentile value forming a first end of the relevant portion of the gamut boundary histogram; and set a second end of the relevant portion of the gamut boundary histogram equal to a highest bin of the gamut boundary histogram.
11. The non-transitory program storage device of claim 9, wherein the instructions to cause the one or more processors to determine an equilibrium point of the selected relevant portion of the gamut boundary histogram comprise instructions to cause the one or more processors to: assign a weight corresponding to a count for each of the bins of the relevant portion of the gamut boundary histogram; and determine a balance point of the relevant portion of the gamut boundary histogram based on moments corresponding to the weight for each of the bins of the relevant portion of the gamut boundary histogram.
12. The non-transitory program storage device of claim 11, wherein the instructions to cause the one or more processors to determine the gamut size metric further comprise instructions to cause the one or more processors to determine the gamut size metric based on a function of the balance point.
13. The non-transitory program storage device of claim 8, wherein the instructions to cause the one or more processors to acquire a color image comprise instructions to cause the one or more processors to: acquire an original color image; and obtain a lower resolution version of the original color image.
14. The non-transitory program storage device of claim 8, wherein the instructions to cause the one or more processors to update a bin of a gamut boundary histogram comprise instructions to cause the one or more processors to increment the bin of the gamut boundary histogram corresponding to the gamut boundary value by an amount equal to a visibility weight value of the pixel.
15. An electronic device, comprising: a memory; and one or more processors operatively coupled to the memory and configured to execute instructions stored in the memory to cause the electronic device to obtain a color image from the memory; process each of a plurality of pixels of the color image by determining a gamut boundary value for a pixel, wherein the gamut boundary value identifies a smallest size gamut that encapsulates the pixel, and updating a bin of a gamut boundary histogram that corresponds to the gamut boundary value; and determine a gamut size metric based on the gamut boundary histogram, wherein the gamut size metric is indicative of a minimum gamut needed to represent the color image.
16. The electronic device of claim 15, wherein the instructions to cause the electronic device to determine a gamut size metric further comprise instructions to cause the electronic device to: select a relevant portion of the gamut boundary histogram; determine an equilibrium point of the selected relevant portion of the gamut boundary histogram; and determine the gamut size metric based on the selected portion of the gamut boundary histogram and the identified equilibrium point.
17. The electronic device of claim 16, wherein the instructions to cause the electronic device to select a relevant portion of the gamut boundary histogram comprise instructions to cause the electronic device to: identify a percentile value; identify a bin in the gamut boundary histogram corresponding to the percentile value, the bin corresponding to the percentile value forming a first end of the relevant portion of the gamut boundary histogram; and set a second end of the relevant portion of the gamut boundary histogram equal to a highest bin of the gamut boundary histogram.
18. The electronic device of claim 16, wherein the instructions to cause the electronic device to determine an equilibrium point of the selected relevant portion of the gamut boundary histogram comprise instructions to cause the electronic device to: assign a weight corresponding to a count for each of the bins of the relevant portion of the gamut boundary histogram; and determine a balance point of the relevant portion of the gamut boundary histogram based on moments corresponding to the weight for each of the bins of the relevant portion of the gamut boundary histogram.
19. The electronic device of claim 18, wherein the instructions to cause the electronic device to determine the gamut size metric further comprise instructions to cause the electronic device to determine the gamut size metric based on a function of the balance point.
20. The electronic device of claim 15, wherein the instructions to cause the electronic device to acquire a color image comprise instructions to cause the electronic device to: acquire an original color image; and obtain a lower resolution version of the original color image.
21. The electronic device of claim 15, wherein the instructions to cause the electronic device to update a bin of a gamut boundary histogram comprise instructions to cause the electronic device to update the bin of the gamut boundary histogram corresponding to the gamut boundary value by an amount equal to a visibility weight value of the pixel.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
DETAILED DESCRIPTION
(6) This disclosure pertains to systems, methods, and computer readable media to identify a gamut size metric for use in all phases of color image processing (e.g., capture, transmission, and display). In general, techniques are disclosed for determining a single-valued metric that changes as image content changes. More particularly, techniques disclosed herein may be used to efficiently determine a gamut boundary histogram which may be used to estimate a gamut size metric. A gamut size metric as disclosed herein identifies a minimum size gamut needed to encompass each pixel in an image, where the gamut size is limited at one end by a first device independent gamut (S.sub.1), and at another end by a second device independent color space (S.sub.2), where S.sub.1 is wholly enclosed within S.sub.2. The described gamut size metric may be based on strict pixel color value differences. In other embodiments the gamut size metric may take into effect perceptual color differences and significance.
(7) In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the disclosed concepts. As part of this description, some of this disclosure's drawings represent structures and devices in block diagram form in order to avoid obscuring the novel aspects of the disclosed concepts. In the interest of clarity, not all features of an actual implementation are described. Moreover, the language used in this disclosure has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such inventive subject matter. Reference in this disclosure to one embodiment or to an embodiment means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosed subject matter, and multiple references to one embodiment or an embodiment should not be understood as necessarily all referring to the same embodiment.
(8) It will be appreciated that in the development of any actual implementation (as in any software and/or hardware development project), numerous decisions must be made to achieve the developers' specific goals (e.g., compliance with system- and business-related constraints), and that these goals may vary from one implementation to another. It will also be appreciated that such development efforts might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the design and implementation of color processing systems having the benefit of this disclosure.
(9) By far the most common color representations or models used in digital imaging involve three (3) primaries. For simplicity but not limitation, further discussion herein will be limited to linear 3-primary additive color systems where colors are characterized by 3-tuples representing a linear combination of 3 primary colors. These color spaces are typically defined by the color values of their primaries. For example, one such color model is the sRGB color model (see
(10) Even with these constraints, color gamut size measurement is an ill-defined problem, as 2 sets of primaries may have the same size but cover different regions of a color space. Compare, for example, sRGB gamut 105 and CMYK gamut 110 which cover approximately the same area, but slightly different regions within wide gamut RGB gamut 100. Even in a truly perceptually uniform color space, gamut size comparison between two gamuts where one doesn't enclose another is still not very meaningful. As a consequence, the gamut size metric in accordance with this disclosure is further constrained to 3-color systems S.sub.1 and S.sub.2 with primaries [R.sub.1, G.sub.1, B.sub.1] and [R.sub.2, G.sub.2, B.sub.2] respectively, where S.sub.1's color gamut is completely enclosed by S.sub.2 and that the corresponding primaries of the 2 systems are reasonably similar in hue (e.g., both S.sub.1 and S.sub.2 have RGB primaries). One such illustrative relationship is illustrated in
(11) For convenience and not limitation, a gamut size metric in accordance with this disclosure will be on an interval scale with 0 corresponding to the smaller gamut of S.sub.1, and 1 corresponding to the larger gamut of S.sub.2. In such a system, the gamut size metric described herein may be thought of as indicating whether the gamut needed for an image is closer to S.sub.1 or S.sub.2's gamut. Color systems S.sub.1 and S.sub.2 may be defined per application and include any color gamuts as long as they satisfy the constraints identified above. With this formulation, any collection of hypothetical primaries that are mixtures of S.sub.1 and S.sub.2 may be described as:
[R.sub.x,G.sub.x,B.sub.x]=(1x)[R.sub.1,G.sub.1,B.sub.1]+x[R.sub.2,G.sub.2,B.sub.2];0<x<1.EQ. 1.
Primaries of the type that satisfy EQ. 1 satisfy the constraint that any set of primaries with a smaller x is enclosed by those primaries with larger x, and therefore a gamut comparison among them is meaningful.
(12) For each pixel in an image, the minimum value x (also referred to as the gamut boundary value) for which the gamut defined by [R.sub.x, G.sub.x, B.sub.x] can enclose the pixel's color may now be determined. Let M.sub.1 be the color matrix that transforms a device-independent color space (e.g. xyz) to the display color space of S.sub.1, and M.sub.2 be the color matrix that transforms the same device-independent color space to the display color space S.sub.2. From EQ. 1 it is easy to see that the color matrix that transforms a given device-independent color space to the display color space of S.sub.x is a linear mixture of M.sub.1 and M.sub.2:
M.sub.x=(1x)M.sub.1+xM.sub.2.EQ. 2
(13) If [r.sub.x, g.sub.x, b.sub.x] represents the color values (as rendered for color system S.sub.x having primaries [R.sub.x, G.sub.x, B.sub.x]) of a pixel with device-independent values [r.sub.0, g.sub.0, b.sub.0], then:
[r.sub.x,g.sub.x,b.sub.x].sup.t=M.sub.x[r.sub.0,g.sub.0,b.sub.0].sup.t=[(1x)M.sub.1+xM.sub.2][r.sub.0,g.sub.0,b.sub.0].sup.t.EQ. 3
For all 3 color values to be within the gamut range of [R.sub.x, G.sub.x, B.sub.x] the following must be true:
0<[(1xr)M.sub.1+xrM.sub.2]r.sub.0<1,EQ. 4A
0<[(1xg)M.sub.1+xgM.sub.2]g.sub.0<1,EQ. 4B
0<[(1xb)M.sub.1+xbM.sub.2]b.sub.0<1.EQ. 4C
(14) It is relatively easy to solve for the minimum x.sub.r, x.sub.g, and x.sub.b gamut boundary values needed to satisfy the constraints identified in EQ. 4 (and the computational load to do so is light). By way of example, let S.sub.1 be the sRGB color space having RGB values [r, g, b] and let S.sub.2 be a wide gamut color space P.sub.3 (having the same, or nearly the same, white point as S.sub.1 so that white point adaptation need not be considered) having RGB values [r.sub.2, g.sub.2, b.sub.2] (note, this is inverted from the exposition above, but models what an implementer of a system in accordance with this disclosure is likely to see). From above,
(15)
In accordance with EQ. 4, the following must hold:
(16)
(representing sRGB's r value), then
(17)
If r.sub.a<r.sub.2 (the case when sRGB P.sub.3), the EQ. 5E leads to:
(18)
Similarly for G and B primaries:
(19)
Using EQS. 5F-5H, estimates for gamut boundary values of x.sub.r, x.sub.g and x.sub.b needed to satisfy the constraints in EQ. 4 are simply the boundary values in the above equations, and thus the combined gamut boundary value x for this pixel is:
x=max(x.sub.r,x.sub.g,x.sub.b).EQ. 6
(20) Referring to
(21) It may be worth noting that the gamut boundary histogram may be better generated on a low-resolution version of the acquired image. Saturated colors at scattered pixels are common when an image is noisy. Saturated colors at scattered pixels (as compared to a large block of saturated colors) are often neither reflective of an object's actual color, nor visually significant in the image. Using a low resolution version of the acquired image (e.g., one that is blurred and subsampled) reduces the impact of such pixels and has been found to give a gamut boundary histogram more suitable for estimating the true gamut range for the image. It may be also worth noting that gamut range estimation results from different low resolution images of an original full resolution image may be combined to identify an optimal gamut boundary for the image. It will be understood that lower resolution images are inherently less sensitive to noise, so a lower resolution version of an original image may be used for gamut boundary estimation of a high noise image.
(22) Using EQ. 6 to calculate the gamut boundary value x at block 220 treats the contribution of r, g, and b equally. Since most display RGB spaces are not perceptually uniform, this treatment may be considered suboptimal, especially if the goal is to avoid widening a color gamut for a perceptually insignificant gain in color accuracy. This perceptual visibility issue may be taken into account by finding a visibility weight for each out-of-gamut pixel in addition to its gamut boundary value x. For a pixel with device-independent values [r.sub.0, g.sub.0, b.sub.0], its rendered color values in space S.sub.2 may be expressed as:
[r.sub.2,g.sub.2,b.sub.2]=clip(M.sub.2[r.sub.0,g.sub.0,b.sub.0].sup.t,0,1),EQ. 7
and in S.sub.1 as:
[r.sub.1,g.sub.1,b.sub.1]=clip(M1[r.sub.0,g.sub.0,b.sub.0].sup.t,0,1).EQ. 8
Converting the S.sub.1-clipped pixels into the S.sub.2 color space, gives:
(r.sub.t,g.sub.t,b.sub.t)=M.sub.2M.sub.1.sup.1(r.sub.1,g.sub.1,b.sub.1).sup.t,EQ. 9
which does not require clipping since S.sub.2 has a larger gamut that S.sub.1. Thus, (r.sub.t, g.sub.t, b.sub.t) is guaranteed to be in the [0, 1] range.
(23) If computational constraints are not limiting, a perceptual color difference such as the CIE DeltaE2000 value dE between [r.sub.t, g.sub.t, b.sub.t] and [r.sub.2, g.sub.2, b.sub.2] may be determined and used as a pixel's visibility weight. If computational constraints are limiting, as may be the case on a mobile platform (e.g., a mobile phone or tablet computer system), dE may be approximated by using a weighted sum of pixel RGB differences between [r.sub.t, g.sub.t, b.sub.t] and [r.sub.2, g.sub.2, b.sub.2]:
w=[c.sub.r,c.sub.g,c.sub.b][d.sub.r,d.sub.g,d.sub.d].sup.t,EQ. 10
where [d.sub.r, d.sub.b, d.sub.b] represents the pixel RGB error introduced by rendering S.sub.2 colors into S.sub.1 and clipping to the range used for [r, g, b] values:
d.sub.r=|r.sub.tr.sub.2|,EQ. 11A
d.sub.g=|g.sub.tg.sub.2|, andEQ. 11B
d.sub.b|b.sub.tb.sub.2|.EQ. 11C
In this approach values for [c.sub.r, c.sub.g, c.sub.b] need to be determined such that the difference between w and dE are minimized for typical colors that are out of the S.sub.1 gamut but within the S.sub.2 gamut. In one embodiment this may be done by simulating the RGB responses in S.sub.1 and in S.sub.2 of highly colorful object surfaces under various illuminants, and finding the best fitting [c.sub.r, c.sub.g, c.sub.b] that minimized the mean squared error between w and dE. The gamut boundary histogram may then be determined by using the sum of visibility weights for each pixel calculated in this way, instead of using pixel count per bin. By way of example, during gamut boundary histogram generation, for each pixel instead of adding a count of 1 to the bin it belongs to, a visibility weight value such as dE may be added to the bin. More generally, an accumulation may be made by taking the sum of w.sup.n (also represented as power(w, n)), so that the summation of perceptual weight is more in line with experimental evidence on supra-threshold summation. In one embodiment, for example, n=2 may be used.
(24) Once a gamut boundary histogram has been determined, the minimum gamut size needed to enclose all pixels in an image may be established by finding the gamut size corresponding to the last non-zero bin of the histogram (i.e., the 100% cutoff point). In general however, a 100% cutoff estimate has not been found very useful in practice for two reasons. First, sometimes a single pixel can require a large gamut while the rest of the image may fit in a much smaller or narrow gamut. In some applications it may be desirable to go to a narrower gamut for such images. Second, using a 100% cutoff can cause the gamut size to swing wildly when a small number of highly saturated colors enter or exit an image frame (e.g., during real-time processing of a video stream), making a gamut size metric unstable. In one embodiment this stability issue may be addressed by using a balance point or equilibrium point or fulcrum instead of a cutoff point to determine a gamut size.
(25) A gamut size metric in accordance with this disclosure may be determined in accordance with the following steps as illustrated in
(26) Referring to
(27) Referring to
(28) Processor 505, display 510, user interface 515, graphics hardware 520, device sensors 525, communications circuitry 545, memory 560 and storage 565 may be of the same or similar type and serve the same function as the similarly named component described above with respect to
(29) As described herein, a single valued gamut size metric may be determined that varies as the images to which it is applied vary (e.g., from frame-to-frame in a video sequence of frames). Where the identified gamut size is limited at one end by a first device independent gamut (S.sub.1) and at another end by a second device independent color space (S.sub.2), where S.sub.1 is wholly enclosed within S.sub.2. The disclosed gamut size metric may be based on a gamut boundary histogram that measures, represents or characterizes the minimal difference needed to render a given pixel from an image into a gamut defined by [R.sub.x, R.sub.g, R.sub.b]. An efficient means to determine a gamut boundary histogram is described. A gamut boundary histogram, and therefore the disclosed gamut size metric, may (but need not) take into account the perceptual color differences and significance between the S.sub.1 and S.sub.2 color spaces.
(30) It is to be understood that the above description is intended to be illustrative, and not restrictive. The material has been presented to enable any person skilled in the art to make and use the disclosed subject matter as claimed and is provided in the context of particular embodiments, variations of which will be readily apparent to those skilled in the art. In one or more embodiments, one or more of the disclosed steps illustrated in