Image processing method based on peripheral reduction of contrast

Abstract

An object of the invention is to avoid or at least limit the aforementioned drawback by a specific processing of video images to display, notably on large size screens. For this purpose, a method for processing at least one image of a video sequence is described. The method includes defining at least two peripheral areas P.sub.L, P.sub.R in the at least one image, and reducing contrast of pixels in the defined peripheral area(s) having a local contrast R.sub.L and/or R.sub.D above a local contrast threshold R.sub.th or reducing contrast of pixels in subregions 1, 2, . . . , i, . . . , N of these defined peripheral areas having a local contrast R.sub.i, R′.sub.i above a local contrast threshold R.sub.i-th.

Claims

1. A method for processing at least one image of a video sequence comprising: defining at least two peripheral areas in the at least one image, reducing contrast of pixels in subregions of said defined peripheral areas, wherein the subregions have a local contrast above a local contrast threshold of each subregion, wherein said local contrast threshold is proportional to a width of a screen of a display device used to view said at least one image, and wherein reducing contrast of pixels is performed such that largest pixel luminance values of said pixels are decreased and smallest pixel luminance values of said pixels are increased.

2. The method of claim 1, wherein local contrast threshold of each subregion decreases as a function of a distance of a center of this subregion from a medium vertical straight line centered on the at least one image or from a center of the at least one image.

3. The method of claim 1, wherein said peripheral areas are defined such as to be symmetrically distributed between a left part and a right part of the image.

4. The method of claim 1, wherein the local contrast of a peripheral area or of a subregion is based on a ratio defined as: the difference between the lowest luminance of the last decile of the distribution of luminance values of pixels of this OF subregion and the highest luminance of the first decile of this distribution, divided by the difference between the highest luminance and the lowest luminance of this distribution, wherein the last decile corresponds to the highest luminance values of the distribution and the first decile corresponds to the lowest luminance values of the distribution.

5. The method of claim 4, wherein the local contrast of a subregion of the at least one image is defined as said ratio multiplied by the standard deviation σ.sub.i of luminance values of pixels of the subregion.

6. An image processing device comprising at least one processor configured for implementing a method for processing at least one image of a video sequence, comprising: defining at least two peripheral areas in at least one image, reducing contrast of pixels in subregions of said defined peripheral areas, wherein the subregions have a local contrast above a local contrast threshold of each subregion, wherein said local contrast threshold is proportional to a width of a screen of a display device used to view said at least on image, and wherein reducing contrast of pixels is performed such that lamest pixel luminance values of said pixels are decreased and smallest pixel luminance values of said pixels are increased.

7. The image processing device of claim 6, wherein local contrast threshold of each subregion decreases as a function of a distance of a center of this subregion from a medium vertical straight line centered on the at least one image or from a center of the at least one image.

8. The image processing device of claim 6, wherein the local contrast of a subregion is based on a ratio defined as: the difference between the lowest luminance of the last decile of the distribution of luminance values of pixels of this subregion and the highest luminance values of the first decile of this distribution, divided by the difference between the highest luminance and the lowest luminance of this distribution, wherein the last decile corresponds to the highest luminance values of the distribution and the first decile corresponds to the lowest luminance values of the distribution.

9. The image processing device of claim 8, wherein the local contrast of a subregion of the at least one image is defined as said ratio multiplied by the standard deviation σ.sub.i of luminance values of pixels of the subregion.

10. A non-transitory computer readable storage medium comprising stored instructions that when executed by a processor performs a method for processing at least one image of a video sequence, comprising: defining at least two peripheral areas in at least one image, reducing contrast of pixels in subregions of said defined peripheral areas, wherein the subregions have a local contrast above a local contrast threshold of each subregion, wherein said local contrast threshold is proportional to a width of a screen of a display device used to view said at least one image, and wherein reducing contrast of pixels is performed such that lamest pixel luminance values of said pixels are decreased and smallest pixel luminance values of said pixels are increased.

11. The non-transitory computer readable storage medium of claim 10, wherein local contrast threshold of each subregion decreases monotonically in function of a distance of a center of this subregion from a medium vertical straight line centered on the at least one image or from a center of the at least one image.

12. The non-transitory computer readable storage medium of claim 10, wherein the local contrast of a subregion is based on a ratio defined as: the difference between the lowest luminance of the last decile of the distribution of luminance values of pixels of this subregion and the highest luminance values of the first decile of this distribution, divided by the difference between the highest luminance and the lowest luminance of this distribution, wherein the last decile corresponds to the highest luminance values of the distribution and the first decile corresponds to the lowest luminance values of the distribution.

13. The non-transitory computer readable storage medium of claim 12, wherein the local contrast of a subregion of the at least one image is defined as said ratio multiplied by the standard deviation σ.sub.i of luminance values of pixels of the subregion.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) The invention will be more clearly understood on reading the description which follows, given by way of non-limiting example and with reference to the appended figures in which:

(2) FIG. 1 shows threshold flicker isograms of the distribution of directional motion responsiveness of the human eye;

(3) FIG. 2 shows a peripheral area of a screen that will likely create flicker in the corresponding peripheral vision area of the retina, distinct from a central area of this screen that will create no flicker on corresponding central vision area of the retina;

(4) FIG. 3 illustrates a first and a second variant of defining peripheral areas in an image in the first step of the first and second embodiment of the image processing method;

(5) FIG. 4 illustrates a third variant of defining peripheral areas in an image in the first step of the first and second embodiment of the image processing method;

(6) FIG. 5 illustrates a gradient function that may be used for the graduation of the reduction of contrast in the fourth step of the first embodiment of the image processing method;

(7) FIG. 6 illustrates the application of the gradient function of FIG. 5 for the graduation of the reduction of contrast in the left and the right peripheral zones performed in the fourth step of the first embodiment of the image processing method;

(8) FIG. 7 illustrates the distribution of luminance values in a subregion of an image having a low local contrast, in the context of the second embodiment of the image processing method;

(9) FIG. 8 illustrates the distribution of luminance values in a subregion of an image having a high local contrast, in the context of the second embodiment of the image processing method;

(10) FIG. 9 illustrates a flowchart of the first and second embodiment of the image processing method.

DESCRIPTION OF EMBODIMENTS

(11) A first embodiment of the method for processing images of a video sequence will now be described in reference to FIG. 9, in the context of displaying these images on a screen of a display device, having a width L and a height H.

(12) Images of the video sequence are all defined in a same geometrical format, here a rectangular format delimited by a top and a bottom horizontal side of width L and a left and a right vertical side of height H. The horizontal sides are larger than the vertical sides.

(13) In a first step of this first embodiment, peripheral areas of images are defined, preferably at least one peripheral area P.sub.L on the left side and at least one peripheral area P.sub.R on the right side.

(14) In a first variant of this first step illustrated on the left part of FIG. 3, the inner border of this peripheral area is anchored on four circles of radius R.sub.1 that are disposed symmetrically in reference to the center of the image, each tangent to a horizontal side of the image and centered at a given distance I.sub.1>R.sub.1 from a lateral side of the image. As shown on FIG. 3, typically R.sub.1≤H/2 and I.sub.1−R.sub.1=7% L.

(15) In this variant, the inner border of the peripheral area is defined as an outer envelope of the four circles with straight lines between the circles, and the outer border of the peripheral area corresponds to the outer border of the image.

(16) As defined in this first variant, the left peripheral area P.sub.L and the right peripheral area P.sub.R are disconnected. These peripheral areas of images of width L and height H are then defined such that, for any pixel x,y of this area, we then have:
∀y ∈[0, R.sub.1], x≤√{square root over (R.sub.1.sup.2−(y−I.sub.1).sup.2)}+I.sub.1
∀y ∈[R.sub.1, H−R.sub.1], x≤0.07*L ou x≥0.93*L
∀y ∈[H−R.sub.1, H], x≤√{square root over (R.sub.1.sup.2−(y−(H−R.sub.1)).sup.2)}+I.sub.1

(17) In a second variant illustrated on the right part of FIG. 3, the inner border of this peripheral area is anchored on six circles of radius R.sub.2 that are disposed symmetrically in reference to the center of the image, four of which are tangent to a horizontal side of the image and centered at a given distance I.sub.2>R.sub.2 from a lateral side of the image, and two of which are equidistant from the two horizontal sides and tangent to two of the four other circles. As shown on FIG. 3, typically R.sub.2=H/2 and I.sub.2−R.sub.2=7% L.

(18) In this variant, the inner borders of the peripheral areas are defined as an outer envelope of the four circles and an inner envelope of the two circles, and the outer border of the peripheral areas correspond to the outer border of the image.

(19) As defined in this second variant, the left and right peripheral areas are also disconnected. The peripheral areas of images of width L and height H are then defined such that, for any pixel x,y of this area, we then have:
∀y ∈[0, R.sub.2], x≤√{square root over (R.sub.2.sup.2−(y−L.sub.1).sup.2)}+L.sub.1
∀y ∈[R.sub.2, H−R.sub.2], x≤0.07*L or x≥0.93*L
∀y ∈[H−R.sub.2, H], x≤√{square root over (R.sub.2.sup.2−(y−(H−R.sub.2)).sup.2)}+L.sub.1

(20) In a third variant illustrated on FIG. 4, the inner border of the peripheral areas is defined as a circle of radius H/2<R.sub.3<L/2 centered on the center of the image, and its outer border corresponds to the outer border of the image.

(21) Therefore, for any pixel x,y belonging to peripheral areas of the images of width L and height H, we have:
∀(x,y)x≤L, y≤H, √{square root over ((L/2−x).sup.2+(H/2−y).sup.2)}≤R.sub.3

(22) In this variant, the left and right peripheral areas are also disconnected. These two parts are disconnected because R.sub.3>H/2. This third variant is advantageous because it requires less computational resources.

(23) In a fourth variant, the inner border of the peripheral area is defined as a rectangle centered on the center of images, of width L.(1−2k), of height H corresponding the height of the image, where, for instance k=20%. In this variant, the left and right peripheral areas are also disconnected.

(24) Therefore, for any pixel x,y belonging to peripheral areas of images of width L and height H, we have:
∀y ∈[0, H], x≤0.2*L ou x≥0.8*L,

(25) In a second step of the first embodiment, local contrast values are computed at least in the peripheral areas defined in the first step.

(26) For such a computing, any known method of evaluation of local contrast can be used, notably a method based on luminance of pixels in the sub-regions. See notably the article entitled “Contrast in complex images”, by Eli Peli, published in October 1990, in J. Opt. Soc. Am. A, Vol. 7, No 10, p.2032-2040.

(27) Luminance value of a pixel is calculated in a manner known per se from its color, notably from its RGB values.

(28) These luminance values are for instance within the whole interval [0, 2.sup.p−1], where p is the number of digits under which color values of pixels are coded.

(29) Luminance value of a pixel p.sub.j is for instance computed according to the well-known following formula:
L.sub.j=0.3*R.sub.j+0.59*G.sub.j+0.11*B.sub.j
with Max(R.sub.j)=Max(G.sub.j)=Max(B.sub.j)=Max(L.sub.j)≤2.sup.P
where R.sub.j, G.sub.j, B.sub.j are the RGB components of the color of this pixel p.sub.j.

(30) The local contrast value R.sub.L, R.sub.D of a peripheral area P.sub.L, P.sub.R is for instance based on a ratio defined as:

(31) the difference L.sub.1090 between the lowest luminance of the last decile of the distribution of luminance values of all pixels of this peripheral area P.sub.L, P.sub.R and the highest luminance of the first decile of this distribution,

(32) divided by the difference L.sub.100% between the highest luminance and the lowest luminance of this distribution,

(33) wherein the last decile corresponds to the highest luminance values of the distribution and the first decile corresponds to the lowest luminance values of the distribution.

(34) In a first variant, this local contrast value is equal to this ratio.

(35) In a second variant, this local contrast value is equal to this ratio multiplied by the standard deviation σ.sub.L, σ.sub.R of luminance values of all pixels of the peripheral area.

(36) This standard deviation σ.sub.L of the left peripheral area P.sub.L can be typically computed as:

(37) $σ_{L} = \sqrt[2]{({.Math.}_{j = 1}^{j = N_{L}} {(L_{j} - μ_{j})}^{2})} / N_{j}$
where N.sub.L is the total number of pixels in the left peripheral area P.sub.L, L.sub.j is the luminance of a pixel p.sub.j of this peripheral area, μ.sub.j is the average luminance over all pixels of this peripheral area. The standard deviation σ.sub.R of the right peripheral area P.sub.R is defined accordingly.

(38) In a third step of the first embodiment, local contrast values R.sub.L, R.sub.D computed from the second step for each peripheral area P.sub.L, P.sub.R are compared to a local contrast threshold R.sub.th, for instance equal to 0.5. If R.sub.L and/or R.sub.D>R.sub.th, then the result of the comparison is positive, and if R.sub.L and/or R.sub.D≤R.sub.th, the result of the comparison is negative. The value of the local contrast threshold R.sub.th is set such that a positive result of the comparison means a significant risk of flickering in the eyes of a viewer viewing the image on the screen of the display device, and that a negative result of the comparison implies practically no risk of such a flickering in the eyes of a viewer. This value of the local contrast threshold R.sub.th can be set using any adapted testing of such a flickering. As the risk of such a flickering is proportional to the width of the screen, the set value of the local contrast threshold R.sub.th is preferably proportional to the width of the screen.

(39) In a fourth step of the first embodiment, the local contrast of the peripheral area P.sub.L and/or P.sub.R that get a positive result of comparison at the third step is reduced.

(40) This reduction of local contrast can be performed by applying any well-known contrast reduction filter to the peripheral area(s) P.sub.L and/or P.sub.R having a positive result of comparison. As a first variant, a digital neutral density filter may be used to reduce the contrast. As a second preferred variant, the reduction of local contrast in a peripheral area is performed such that the largest pixel luminance values of pixels of this peripheral area are decreased and the smallest pixel luminance values are increased.

(41) Preferably, in order to smooth the reduction of contrast over the image, this reduction of local contrast is graduated from a lower reduction of local contrast for pixels of the peripheral area closer to a center of the image or to a central vertical line in the image, up to a higher reduction of local contrast for pixels of the peripheral area further to this center or to this central vertical line, namely for pixels located near the border of the image. This graduation in the reduction of local contrast may advantageously follow the curve illustrated on FIG. 5, corresponding to the following gradient function
f.sub.α(x)=x.sup.α/(x.sup.α+(1−x).sup.α),
where x is a distance separating a pixel to be filtered from the center of the image or from the central vertical line in the image, and
where, for instance: α=2.

(42) As illustrated on FIG. 6 showing rectangular peripheral areas on the left side and on the right side of the image, the highest reduction of local contrast is for instance set to 60% (i.e. 40% remaining contrast shown on the figure) for pixels of the peripheral areas located on the border of the image and the lowest reduction of local contrast is 0% (i.e. 100% remaining contrast shown on the figure) for pixels of the peripheral area(s) near the central part of the image. For all pixels of the central part of the image (i.e. defined as the area between the two opposite peripheral areas), the contrast is not changed (i.e. 100% remaining contrast). Such a gradual reduction of local contrast prevents advantageously from the creation of color artifacts in the image.

(43) A black filtering overlayer can be built in a manner known per se such as, when applied on the image, a reduction of local contrast is obtained. Such a black filtering overlayer can be represented by a LUT.

(44) As a variant, a histogram equalization filter can be used to get a local reduction of contrast. The advantage of such a filter is that the mean luminance of pixels of the filtered peripheral area is not changed.

(45) As another variant, a convolutional filter can be used.

(46) At the end of the above process, a processed image is obtained. Of course, luminance values of pixels that do not belong to the peripheral areas (i.e. that belong to a central area of the image) are generally not processed to get the processed image.

(47) Advantageously, when displaying this image of a screen having a large size, notably a high width, no flickering will occur in the eyes of a viewer. Other images of the sequence are preferably processed using the same method.

(48) In a second embodiment of the method for processing images of a video sequence illustrated on FIG. 9 too, the peripheral areas are divided in N subregions 1, 2, . . . , i, . . . , N.

(49) In an example, these subregions are identical squared areas, each of then having a side for instance equal to 3.%*L, where L is the width of the image.

(50) Then, a contrast value R.sub.i is computed for each subregion i, providing then a local contrast value for this subregion.

(51) The local contrast R.sub.i of a subregion i is for instance based on a ratio defined as:

(52) the difference L.sub.1090 between the lowest luminance of the last decile of the distribution of luminance values of all pixels of this subregion and the highest luminance of the first decile of this distribution,

(53) divided by the difference L.sub.100% between the highest luminance and the lowest luminance of this distribution,

(54) wherein the last decile corresponds to the highest luminance values of the distribution and the first decile corresponds to the lowest luminance values of the distribution.

(55) In a first variant of computing local contrast of a subregion i, its local contrast R.sub.i is equal to this ratio.

(56) In a second variant of computing local contrast in a subregion i, the local contrast is defined as R′.sub.i=R.sub.i*σ.sub.i, where σ.sub.i is the standard deviation σ.sub.i of luminance values of all pixels of the subregion.

(57) This standard deviation σ.sub.i can be typically computed as:

(58) $σ_{i} = \sqrt[2]{({.Math.}_{j = 1}^{j = N_{i}} {(L_{j} - μ_{i})}^{2})} / N_{i}$
where N.sub.i is the total number of pixels in the subregion i, L.sub.j is the luminance of a pixel p.sub.j of this subregion, μ.sub.i is the average luminance over all pixels of this subregion.

(59) FIG. 7 illustrates a subregion of an image having a low contrast as R.sub.i=L.sub.1090/L.sub.100%=(233−173)/255=23% is low and as 2σ.sub.i=2×“StandardDev”/255=2×25/255=20% is low too.

(60) FIG. 8 illustrates another subregion of an image having a high contrast as R.sub.i=L.sub.1090/L.sub.100%=(202−21)/255=71% is high and as 2σ.sub.i=2×“StandardDev”/255=2×60/255=47% is high too.

(61) In a third step of this second embodiment, local contrast values R.sub.i of each subregion i is compared to a local contrast threshold R.sub.i-th.

(62) In a first variant, local contrast thresholds R.sub.i-th are the same for all subregions i, for instance equal to 0.5.

(63) In a second advantageous variant, the local contrast threshold R.sub.i-th of a subregion i is a monotonous decreasing function of a distance of a center of this subregion i from a medium vertical straight line centered on the image or from a center of this image. Such a medium vertical straight line would generally be parallel to the left and right sides of the image and would divide the image into equal parts. Thank to this variant, more a subregion is distant from the center of the image more its flickering effect is critical, and more the filtering step (see below) is triggered for a low contrast threshold.

(64) In a fourth step of the second embodiment, the local contrast of each subregion i having a local contrast above the local contrast threshold R.sub.i-th of this subregion is reduced as in the fourth step of the first embodiment above. Preferably, the reduction of local contrast in a subregion is performed such that, if a pixel in this subregion has a larger luminance value than other pixels of this subregion, then the large pixel luminance value is decreased, and, if a pixel has a smaller luminance value than other pixels in the subregion, then the small pixel luminance value is increased. Preferably, such a reduction of local contrast in pixels of a subregion is proportional to distance of a center of a filtered pixel of this subregion from a medium vertical straight line centered on the image or from a center of image. Having a contrast reduction increasing with eccentricity of the subregion within the image will advantageously compensate for the known increase of Critical Flicker Frequency with retinal eccentricity.

(65) Preferably, the filtered image which is obtained is smoothed in a manner known per se to lower any color artefacts which may have been created by the filtering of the selected subregions.

(66) At the end of this processing, a processed image is obtained, in which colors of pixels belonging to the peripheral area are processed such as to reduce temporal and/or spatial flickering feeling by reducing light stimulation of the corresponding peripheral region of the eye, and notably to keep temporal and/or spatial flickering frequency of the content below the critical flicker frequency of the corresponding peripheral region of the eye. Of course, luminance values of pixels that do not belong to the sub-regions (i.e. that belong to a central area of the image) are generally not processed to get the processed image.

(67) Other images of the sequence are preferably processed using the same method.

(68) The above methods for processing images of a video sequence may be implemented in any image processing device comprising at least one processor. This image processing device may be part of a Set-Top-Box, a Gateway, a TV set, a tablet, a smartphone, a laptop, a Head-Mounted-Display, or any other electronic device. The processor is configured in a manner known per se to implement one of these image processing methods.

(69) Although the illustrative embodiments of the image processing method have been described herein with reference to the accompanying drawings, it is to be understood that the present invention is not limited to those precise embodiments, and that various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the invention. All such changes and modifications are intended to be included within the scope of the appended claims. The present invention as claimed therefore includes variations from the particular examples and preferred embodiments described herein, as will be apparent to one of skill in the art.

Image processing method based on peripheral reduction of contrast

Assignee

Inventors

Cpc classification

Classification Explorer

G09G3/2003

PHYSICS

Classification Explorer

G09G5/10

PHYSICS

Classification Explorer

G09G3/3611

PHYSICS

Classification Explorer

G06T2207/10016

PHYSICS

Classification Explorer

G09G2320/0247

PHYSICS

Classification Explorer

G09G2310/0232

PHYSICS

Classification Explorer

G09G2320/066

PHYSICS

Classification Explorer

G09G3/20

PHYSICS

Classification Explorer

G09G2320/0266

PHYSICS

Classification Explorer

G09G2320/0686

PHYSICS

Classification Explorer

G06T5/40

PHYSICS

Classification Explorer

G09G2320/0276

PHYSICS

Classification Explorer

G06T2207/10024

PHYSICS

Classification Explorer

G06T5/008

PHYSICS

Classification Explorer

G09G2360/16

PHYSICS

Classification Explorer

G06F3/14

PHYSICS

International classification

Classification Explorer

G09G3/20

PHYSICS

Classification Explorer

G09G3/36

PHYSICS

Classification Explorer

G09G5/10

PHYSICS

Abstract

Claims

Description