Method for evaluating quality of tone-mapping image based on exposure analysis
20170372175 · 2017-12-28
Assignee
Inventors
Cpc classification
H04N1/00002
ELECTRICITY
H04N23/741
ELECTRICITY
H04N1/6027
ELECTRICITY
International classification
Abstract
A method for evaluating quality of tone-mapping image based on exposure analysis is provided, which explores the exposure properties on each area of the high dynamic range image utilizing the pre-exposure method and divides the high dynamic range image into three parts of an easy overexposed area, an easy underexposed area and an easy natural-exposed area, wherein different quality characteristics are extracted in different areas, which is capable of ensuring that the follow-up quality characteristic extraction is more targeted. The present invention takes the difference of distortion between the tone-mapping image and the conventional image into account, and extracts image characteristics such as the abnormal exposure rate, the underexposed residual energy, the overexposed residual energy and the exposure color index, so as to accurately reflect the quality degradation of the tone-mapping image.
Claims
1. A method for evaluating quality of a tone-mapping image based on exposure analysis, comprising steps of: (1) denoting S.sub.HDR as a high dynamic range image which is unprocessed with a width W and a height H, S.sub.HDR is an input signal of a conventional low dynamic range display devices; denoting S.sub.TM as a tone-mapping image generated from S.sub.HDR after processing with a tone-mapping operator, wherein S.sub.TM serves as a tone-mapping image to be evaluated; (2) performing pre-exposure processing on a luminance component of S.sub.HDR under different exposure degrees, so as to generate an overexposed image and an underexposed image of the luminance component of S.sub.HDR, which is respectively denoted as EI.sub.over and EI.sub.under; (3) dividing EI.sub.over into
2. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 2, wherein an obtaining process of EI.sub.over and EI.sub.over in the step (2) is: a pixel value of a pixel at a coordinate position of (x,y) in EI.sub.over is denoted as EI.sub.over (x,y), a pixel value of a pixel at a coordinate position of (x,y) in EI.sub.under is denoted as EI.sub.under (x,y),
3. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 1, wherein in the step (3), a specific process of finding out overexposed image blocks from all the image blocks of EI.sub.over is: for an ith image block in EI.sub.over, calculating an average value of pixel values of all pixels in the ith image block, if the average value of pixel values of all pixels in the ith image block is greater than an overexposed threshold TH.sub.over the ith image block is determined as an overexposed image block; wherein
4. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 2, wherein in the step (3), a specific process of finding out overexposed image blocks from all the image blocks of EI.sub.over is: for an ith image block in EI.sub.over, calculating an average value of pixel values of all pixels in the ith image block, if the average value of pixel values of all pixels in the ith image block is greater than an overexposed threshold TH.sub.over, the ith image block is determined as an overexposed image block; wherein
5. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 3, wherein a value of the overexposed threshold is TH.sub.over=α.sub.over×
6. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 4, wherein a value of the overexposed threshold is TH.sub.over=α.sub.over×
7. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 5, wherein a determining process of R.sub.over R.sub.under and R.sub.normal in the step (4) comprising steps of: (4)-1a: defining current image block to be processed in S.sub.HDR as a current image block; (4)-1b: defining the current image block as an ith image block in S.sub.HDR which is denoted as B.sub.HDR.sup.i, wherein
8. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 6, wherein a determining process of R.sub.over R.sub.under and R.sub.normal in the step (4) comprising steps of: (4)-1a: defining current image block to be processed in S.sub.HDR as a current image block; (4)-1b: defining the current image block as an ith image block in S.sub.HDR which is denoted as B.sub.HDR.sup.i, wherein
9. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 7, wherein in the step (6), a judging process of the tone-mapping overexposed block in R.sub.1 comprising steps of: dividing a luminance component in S.sub.TM into
10. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 8, wherein in the step (6), a judging process of the tone-mapping overexposed block R.sub.1 comprising steps of: dividing a luminance component in S.sub.TM into
11. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 9, wherein in the step (11), a process of obtaining a support vector regression training model comprises steps of: (11)-1a: selecting n.sub.test high dynamic range images; then generating N.sub.test tone-mapping images by different tone-mapping operators; taking a set of the N.sub.test tone-mapping images as a training image set, which is denoted as D.sub.test; then utilizing a subjective quality evaluation method to evaluate the tone-mapping images in D.sub.test to obtain a subjective quality evaluation score of each tone-mapping image in D.sub.test; denoting a subjective quality evaluation score of an mth tone-mapping image in D.sub.test as DMOS.sub.m; then obtaining a characteristic vector of each tone-mapping image in D.sub.test in an identical way as a process of the step (1) to (10) , denoting a characteristic vector of an mth tone-mapping image in D.sub.test as
12. The method for evaluating quality of a tone-mapping image based on exposure analysis, as recited in claim 10, wherein in the step (11), a process of obtaining a support vector regression training model comprises steps of: (11)-1a: selecting n.sub.test high dynamic range images; then generating N.sub.test tone-mapping images by different tone-mapping operators; taking a set of the N.sub.test tone-mapping images as a training image set, which is denoted as D.sub.test, then utilizing a subjective quality evaluation method to evaluate the tone-mapping images in D.sub.test to obtain a subjective quality evaluation score of each tone-mapping image in D.sub.test; denoting a subjective quality evaluation score of an mth tone-mapping image in D.sub.test as DMOS.sub.m; then obtaining a characteristic vector of each tone-mapping image in D.sub.test in an identical way as a process of the step (1) to (10) , denoting a characteristic vector of an mth tone-mapping image in D.sub.test as
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0042]
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
[0043] Further description of the present invention is illustrated combining with the accompanying drawings and the preferred embodiments.
[0044] An overall flow chart is as shown in the Figure, the present invention provides a method for evaluating quality of a tone-mapping image based on exposure analysis, comprising steps of:
[0045] (1) denoting S.sub.HDR as a high dynamic range image which is unprocessed with a width W and a height H, S.sub.HDR is an input signal of a conventional low dynamic range display devices; denoting S.sub.TM as a tone-mapping image generated from S.sub.HDR after processing with a tone-mapping operator, wherein S.sub.TM serves as a tone-mapping image to be evaluated;
[0046] (2) performing pre-exposure processing on a luminance component of S.sub.HDR under different exposure degrees, so as to generate an overexposed image and an underexposed image of the luminance component of S.sub.HDR which is respectively denoted as EI.sub.over and EI.sub.under, here, both EI.sub.over and EI.sub.under are extreme exposure condition images; [0047] wherein in the preferred embodiment, an obtaining process of EI.sub.over and EI.sub.over in the step (2) is: a pixel value of a pixel at a coordinate position of (x,y) in EI.sub.over is denoted as EI.sub.over (x,y), a pixel value of a pixel at a coordinate position of (x,y) in EI.sub.under is denoted as EI.sub.under (x,y),
wherein 1≦x≦W, 1≦y≦H, symbol └ ┘ is a floor operation symbol, F.sub.over represents a sunlight parameter corresponding to EI.sub.over, F.sub.over=8, F.sub.under represents a sunlight parameter corresponding to EI.sub.under, F.sub.under=0, I.sub.HDR (x,y) represents a pixel value of a pixel at a coordinate position of (x,y) in a luminance component I.sub.HDR of S.sub.HDR, γ is a gamma correction parameter; wherein γ=2.2 in the preferred embodiment;
[0048] (3) dividing EI.sub.over into
non-overlapped image blocks with a size of 2.sup.u×2.sup.v; then finding out overexposed image blocks from all image blocks of EI.sub.over; forming all of the overexposed image blocks in EI.sub.over into an overexposed area which is denoted as R.sub.over.sup.Exposure; wherein └ ┘ is a floor operation symbol, u and v are identical integer selected from an interval of.sup.[2, 5]. [0049] dividing EI.sub.under into
non-overlapped image blocks with a size of 2.sup.u×2.sup.v; then finding out underexposed image blocks from all image blocks of EI.sub.under; forming all of the underexposed image blocks in EI.sub.under under into an underexposed image area which is denoted as R.sub.under.sup.Exposure; [0050] in the preferred embodiment, in the step (3), a specific process of finding out overexposed image blocks from all the image blocks of EI.sub.over is: for an ith image block in EI.sub.over, calculating an average value of pixel values of all pixels in the ith image block, if the average value of pixel values of all pixels in the ith image block is greater than an overexposed threshold TH.sub.over, the ith image block is determined as an overexposed image block; wherein
[0053] (4) dividing S.sub.HDR into
non-overlapped image blocks with a size of 2.sup.u×2.sup.v; then, according to R.sub.over.sup.Exposure and R.sub.under.sup.Exposure, dividing S.sup.HDR into an easy overexposed area, an easy underexposed area and an easy normal-exposed area, which are respectively denoted as R.sub.over, R.sub.under and R.sub.normal;
[0054] In the preferred embodiment, a determining process of R.sub.over, R.sub.under and R.sub.normal in the step (4) comprising steps of:
[0055] (4)-1a: defining current image block to be processed in S.sub.HDR as a current image block;
[0056] (4)-1b: defining the current image block as an ith image block in S.sub.HDR which is denoted as B.sub.HDR.sup.i, wherein
[0057] (4)-1c: if B.sub.over.sup.i∈R.sub.over.sup.Exposure and B.sub.under.sup.i.Math.R.sub.under.sup.Exposure, determining B.sub.HDR.sup.i as an easy overexposed block; if B.sub.under.sup.i∈R.sub.under.sup.Exposure and B.sub.over.sup.i.Math.R.sub.over.sup.Exposure, determining B.sub.HDR.sup.i as an easy underexposed block; if B.sub.over.sup.i∈R.sub.over.sup.Exposure and B.sub.under.sup.i∈R.sub.under.sup.Exposure, determining B.sub.HDR.sup.i as an easy normal-exposed block; wherein B.sub.over.sup.i represents an ith image blocks in EI.sub.over, B.sub.under.sup.i represents an ith image blocks in EI.sub.under;
[0058] (4)-1d: taking a next image block to be processed in S.sub.HDR as a current image block, then returning to (4)-1b and performing continuously until all image blocks in S.sub.HDR are processed, then taking an area formed by all easy overexposed blocks in S.sub.HDR as an easy overexposed area R.sub.over, taking an area formed by all easy underexposed blocks in S.sub.HDR as an easy underexposed area R.sub.under; and taking an area formed by all easy normal-exposed blocks in S.sub.HDR as an easy normal-exposed area R.sub.normal;
[0059] (5) dividing S.sub.TM into
non-overlapped image blocks with a size of 2.sup.u×2.sup.v; then denoting an area corresponding to R.sub.over in S.sub.TM as a tone-mapping easy overexposed area R.sub.1; then denoting an area corresponding to R.sub.under in S.sub.TM as a tone-mapping easy underexposed area R.sub.2; denoting an area corresponding to R.sub.normal in S.sub.TM as a tone-mapping easy normal-exposed area R.sub.3;
[0060] (6) judging if there is overexposure in each of the image blocks in R.sub.1, if yes, determining image blocks which are overexposed as tone-mapping overexposed blocks, then counting a total number of the tone-mapping overexposed blocks which is denoted as N.sub.over.sup.TM; [0061] judging if there is underexposure in each of the image blocks in R.sub.2, if yes, determining image blocks which are underexposed as tone-mapping underexposed blocks, then counting a total number of the tone-mapping underexposed blocks which is denoted as N.sub.under.sup.TM; [0062] wherein in the step (6), a judging process of the tone-mapping overexposed block in R.sub.1 comprising steps of: dividing a luminance component in S.sub.TM into
non-overlapped image blocks with a size of 2.sup.u×2.sup.v, wherein an nth image blocks in R.sub.1 is denoted as R.sub.1,n calculating an average value and a standard deviation of pixel values of all pixels in an image block corresponding to R.sub.1,n in a luminance component of S.sub.TM which are respectively denoted as μ.sub.1,n and σ.sub.1,n; if μ.sub.1,n>200 and σ.sub.1,n<σ.sub.TM, R.sub.1,n is judged as a tone-mapping overexposed block; wherein 1≦n≦N.sub.R.sub.
non-overlapped image blocks with a size of 2.sup.u×2.sup.v, wherein an n'th image blocks in R.sub.2 is denoted as R.sub.2,n′, calculating an average value and a standard deviation of pixel values of all pixels in an image block corresponding to R.sub.2,n′ in a luminance component of S.sub.TM, which are respectively denoted as μ.sub.2,n′ and σ.sub.2,n′, if μ.sub.2,n′<50 and σ.sub.2,n′<σ.sub.TM, R.sub.2,n′ is judged as a tone-mapping underexposed block; wherein 1≦n′≦N.sub.R.sub.
[0064] (7) according to N.sub.over.sup.TM and N.sub.under.sup.TM, calculating an abnormal exposure rate of S.sub.TM which is denoted as η.sub.abnormal,
wherein N.sub.R.sub.
[0065] (8) calculating an overexposed residual energy of R.sub.1, which is denoted as E.sub.1, wherein
calculating an underexposed residual energy of R.sub.2, which is denoted as E.sub.2, wherein
wherein μ.sub.1,n represents an average value of pixel values of all pixels in a corresponding area of an nth image blocks in R.sub.1 in an luminance component of S.sub.TM; μ.sub.2,n′ represents an average value of pixel values of all pixels in a corresponding area of an n'th image blocks in R.sub.2 in the luminance component of S.sub.TM; L.sub.over.sup.E is an extreme overexposure brightness value, L.sub.over.sup.E=255; L.sub.under.sup.E is an extreme underexposure brightness value, L.sub.under.sup.E=0;
[0066] (9) converting R.sub.3 from an RGB (red green blue) color space into an opponent color space, which is denoted as R.sub.3′; then calculating an average value and a variance of a component value of a red-green channel of all pixels in R.sub.3′; which are respectively denoted as μ.sub.rg and σ.sub.rg; calculating an average value and a variance of a component value of yellow-blue channel of all pixels in R.sub.3′, which are respectively denoted as μ.sub.y,b and σ.sub.yb; then calculating an exposure color index of R.sub.3′, which is denoted as C.sub.3, C.sub.3=√{square root over (σ.sub.rg.sup.2+σ.sub.yb.sup.2)}+ω.sub.c×√{square root over (μ.sub.rg.sup.2+μ.sub.yb.sup.2)}; wherein ω.sub.c represents weighing of an average color value, wherein in the preferred embodiment ω.sub.c=0.3
[0067] (10) obtaining a characteristic vector of S.sub.TM , which is denoted as
[0068] (11) testing
[0070] (11)-1a: selecting n.sub.test high dynamic range images; then generating N.sub.test tone-mapping images by different tone-mapping operators; taking a set of the N.sub.test tone-mapping images as a training images set, which is denoted as D.sub.test, then utilizing a subjective quality evaluation method to obtain a subjective quality evaluation score of each tone-mapping image in D.sub.test; denoting a subjective quality evaluation score of an mth tone-mapping image in D.sub.test as DMOS.sub.m; then obtaining a characteristic vector of each tone-mapping image in D.sub.test in an identical way as a process of the step (1) to (10), denoting a characteristic vector of an mth tone-mapping image in D.sub.test as
[0071] (11)-1b: training each subjective quality evaluation score and characteristic vector of all tone-mapping images in D.sub.test utilizing support vector regression, so as to make a regression function has a minimum error to the subjective quality evaluation scores through training, fitting to obtain an optimal weight vector V.sub.best and an optimal bias b.sub.best; then obtaining a support vector regression training model utilizing V.sub.best and b.sub.best.
[0072] In the preferred embodiment, the TMID tone-mapping image database, TMID database for short, provided by the LIVE laboratory of the University of Texas at Austin is selected for testing. The TMID database comprises 15 original high dynamic range images of different scene types, and 8 different tone-mapping operators are adopted for performing tone-mapping process on each of the original high dynamic range images, and 120 tone-mapping images are generated. During the test process, the 120 tone-mapping images are randomly classified into two parts including a training image set and a test image set. According to the process of the step (1) to the step (11), identical method is adopted to calculate for obtaining the objective quality evaluation predictive value of each tone-mapping image in the test image set, then the objective quality evaluation predictive values and the corresponding subjective quality evaluation scores (in the preferred embodiment, Differential Mean Opinion Score (DMOS) is adopted as the subjective quality evaluation score) are performed with four parameter logistic function non-linear fitting, so as to finally obtain the index value between the objective evaluation result and the subjective perception. Here, three commonly used objective parameters of the evaluation method for image quality are adopted for serving as the evaluation index: Correlation coefficient (CC), Spearman Rank Order Correlation coefficient (SROCC) and Rooted Mean Squared Error (RMSE). Values of CC and ROCC are at a range of [0,1], wherein the more closer are the values of CC and ROCC to 1, the more accurate is the objective evaluation results; vice versa, the less accurate. The CC, SROCC and RMSE indexes representing the evaluation performances of the method of the present invention are as shown in Table. 1. It can be seen from the data listed in the Table. 1 that there is a good correlation between the objective quality evaluation predicted value of the tone-mapping image obtained by the method of the present invention and the subjective quality evaluation score; wherein the value of CC reaches 0.8802, the value of SROCC reaches 0.8512 and the value of RMSE is as low as 0.8342. The results indicate that the objective evaluation result of the method of the present invention is consistent with the result of subjective perception of human eyes, and the validity of the method of the present invention is fully explained.
TABLE-US-00001 TABLE 1 Performance indexes of correlation between the objective quality evaluation predicted value and the subjective quality evaluation score of the tone-mapping image in the test image set calculated by the method of the present invention. Index CC SROCC RMSE Final result 0.8802 0.8512 0.8342
[0073] One skilled in the art will understand that the embodiment of the present invention as shown in the drawings and described above is exemplary only and not intended to be limiting.
[0074] It will thus be seen that the objects of the present invention have been fully and effectively accomplished. Its embodiments have been shown and described for the purposes of illustrating the functional and structural principles of the present invention and is subject to change without departure from such principles. Therefore, this invention includes all modifications encompassed within the spirit and scope of the following claims.