Priori constraint and outlier suppression based image deblurring method

Abstract

Provided is an a priori constraint and outlier suppression based image deblurring method. A convolution model is used for fitting a blurring process of a clear image and then the blurred image I is restored, so that the purpose of image deblurring is achieved. The method comprises an evaluation process of the significant structure of a blurred image, a process of blurring kernel estimation and outlier suppression, and a process of restoring the blurred image by non-blind deconvolution. A structure in the blurred image is obtained by use of L0 norm constraint and heavy-tailed a priori information. The L0 norm constraint is used to evaluate the blurring kernel. The evaluated blurring kernel is subjected to outlier suppression. The final restored image is obtained by using a non-blind deconvolution algorithm. The present invention can prominently improve the restoration level of the blurred image.

Claims

1. A priori constraint and outlier suppression based image deblurring method, comprising: obtaining a significant structure S of a blurred image L by solving the following formula using an iterative process: wherein k is a blur kernel; λ.sub.1, λ.sub.2, λ.sub.3, β and γ are constants; M is a binary mask indicating a texture region in the significant structure S and (1−M) a binary mask indicating a smooth region in the significant structure S; and u and w are two auxiliary variables; and wherein at each iteration the values of u and w are updated in sequence, an updated S is subsequently obtained using the updated u and v and a process involving Fourier transform; estimating the blurring kernel k by an iterative process, wherein at each iteration k is updated using gradient information and the obtained significant structure S; and restoring the blurred image by using the estimated blurring kernel with a non-blind deconvolution technology.

2. The image deblurring method of claim 1, wherein the binary mask M is defined with the following formulas: $r\left(x\right)=\frac{.Math..Math.\underset{y\in N_h\left(x\right)}{.Math.}\nabla S\left(y\right).Math..Math.}{\underset{y\in N_h\left(x\right)}{.Math.}.Math.\nabla S\left(y\right).Math.+0.5}$
M(x)=H(r(x)−τ.sub.r) wherein, x indicates a location of a pixel point, y represents pixel points within a window N.sub.h(x) of a given size that is centered at pixel point x, and r(x) indicates a degree that the pixel point at the location x belongs to the texture part; the value of r(x) is between 0 and 1, and r(x) is in proportion to the possibility that x belongs to the texture part; M is obtained by Heaviside step function, wherein τ.sub.r indicates a threshold of r for distinguishing a texture region and the smooth region in the significant structure S.

3. The image deblurring method of claim 2, wherein τ.sub.r is obtained with a histogram equalization method.

4. The image deblurring method of claim 1, wherein a blurring process to produce the blurred image L from a clear image I is:
L=I.Math.k+η wherein η indicates noise.

5. The image deblurring method of claim 4, wherein the distribution of the noise η is Gaussian distribution.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a flow diagram of specific implementation of the present invention;

(2) FIG. 2 is a flow block diagram of the method of the present invention;

(3) Wherein, k.sup.n indicates a blurring kernel obtained by evaluating an image with minimum size, and k.sup.0 indicates evaluated blurring kernel finally obtained.

(4) FIG. 3 is a texture diagram of a significant structure of a blurred image in an embodiment of the present invention;

(5) Wherein, (a) indicates an original blurred image; (b) indicates graded distribution of the original blurred image; (c) indicates distribution of a gradient histogram; and (d) indicates an energy diagram of r value expressed by color information, i.e., a value distribution diagram of an original image r(x).

(6) FIG. 4 is an exemplary diagram for the significant structure of the blurred image in the embodiment of the present invention.

(7) FIG. 5 is significant structures of the blurred images with different sizes in the embodiment of the present invention; Wherein, (a)-(d) indicate the blurred images with different sizes.

(8) FIG. 6 is a blurring kernel obtained through iterative update of the embodiment of the present invention.

(9) FIG. 7 is an image after restoring the blurred image of the embodiment of the present invention.

(10) FIG. 8 is a comparison diagram by magnifying the blurred image and restored image thereof in the embodiment of the present invention.

(11) FIG. 9 is comparison of the blurred image, the significant structure and the restored diagram in the embodiment of the present invention;

(12) Wherein, Figure (a) indicates the original blurred image; Figure (b) indicates the significant structure; and Figure (c) indicates the blurred restored image.

DETAILED DESCRIPTION OF THE INVENTION

(13) Further description is made as follows to the present invention through an embodiment in combination with the drawings, but the range of the present invention is not limited in any way.

(14) A priori constraint and outlier suppression based deblurring method proposed by the present invention is shown in FIG. 2. In the Figure, k.sup.n indicates a blurring kernel obtained by evaluating an image with minimum size, and k.sup.0 indicates evaluated blurring kernel finally obtained. In the method of the present invention, the blurring kernel is continuously updated with several sampling ways, and the final restored image is obtained by using a non-blind deconvolution algorithm.

(15) The method of the present invention includes the specific steps as follows:

(16) Table 1 is a description for names of parameters adopted in the following steps and corresponding parameter meanings thereof

(17) TABLE-US-00001 TABLE 1 Parameter List Parameter name Description λ1 Coefficient of a substitution variable w in an evaluation process of a significant structure S λ2 Coefficient of a texture region in the evaluation process of the significant structure S λ3 Coefficient of a smooth region in the evaluation process of the significant structure S ψ1 Weight of a blurring kernel k in an evaluation process of the blurring kernel ψ2 Weight of a substitution variable v in the evaluation process of the blurring kernel γ Weight of iterative update in the evaluation process of the significant structure S β Weight of iterative update in the evaluation process of the significant structure S φ Weight of iterative update in the evaluation process of the blurring kernel

(18) Step 1. Selection of a blurring model: the present invention adopts a model of Formula 1, and assumes that the noise follows Gaussian distribution, and an optimized equation shown in Formula 2 is obtained;

(19) In the present invention, a convolution model is used for fitting a blurring process of a clear image, as shown in the Formula 1:
L=I.Math.k+η  (Formula 1)

(20) wherein L indicates a blurred image, k indicates a blurring kernel, and i indicates noise (assume the distribution thereof is Gaussian noise);

(21) A priori constraint with a heavy-tailed effect is taken as distribution of a significant structure gradient of the blurred image, as shown in Formula 2:

(22) $\begin{array}{cc}\underset{s}{\min }\left({.Math.S.Math.k-L.Math.}^2+{\lambda }_1{.Math.\nabla S.Math.}^{0.5}\right)& \left(\mathrm{Formula}2\right)\end{array}$

(23) Wherein, S indicates the significant structure of the blurred image;

(24) Step 2: evaluation of the significant structure of the blurred image:

(25) Firstly, obtaining texture calibration of the significant structure with Formula 4 and Formula 5, as shown in FIG. 3(b). FIG. 3 is a texture diagram of a significant structure of a blurred image in an embodiment, wherein, (a) indicates an original blurred image; (b) indicates graded distribution of the original blurred image; (c) indicates distribution of a gradient histogram; and (d) indicates a value distribution of the original image r(x). The texture calibration of the significant structure with the Formula 4 and the Formula 5 is specifically as follows:

(26) We introduce L0 norm to constrain texture of the significant structure S of the blurred image, and meanwhile, limit noise of a smooth region in S with L2 norm. The updated formula is shown in the Formula 3:

(27) $\begin{array}{cc}\underset{s}{\min (}{.Math.S.Math.k-L.Math.}^2+{\lambda }_1{.Math.\nabla S.Math.}^{0.5}+{\lambda }_2{.Math.\nabla S\circ M.Math.}_0+{\lambda }_3{.Math.\nabla S\circ \left(1-M\right).Math.}_2^2)& \left(\mathrm{Formula}3\right)\end{array}$

(28) Wherein, M is a binary mask indicating texture region in the significant structure S and (1−M) a binary mask indicating smooth region in S, (1−M) a binary mask indicating smooth region in S. We define M with the Formula 4 and the Formula 5:

(29) $\begin{array}{cc}r\left(x\right)=\frac{.Math.\underset{y\in N_h\left(x\right)}{.Math.}\nabla S\left(y\right).Math.}{\underset{y\in N_h\left(x\right)}{.Math.}.Math.\nabla S\left(y\right).Math.+0.5}& \left(\mathrm{Formula}4\right)\\ M=H\left(r-{\tau }_r\right)& \left(\mathrm{Formula}5\right)\end{array}$

(30) In the Formula 4, x indicates a location of a pixel point, y represents pixel points within a window N.sub.h (x) of a given size that is centered at pixel point x and r(x) indicates a degree that the pixel point at the location x belongs to the texture part. The texture in S can be preliminarily divided with the Formula 4, the value of r(x) is between 0 and 1, and r(x) is in proportion to the possibility that x belongs to the texture part. Meanwhile, the Formula 4 also limits the appearance of a mutational texture (when the size of the blurring kernel is greater than that of blurred image detail, the image restoration is failed, therefore, the mutational texture should be limited). M in the Formula 5 is obtained by Heaviside step function, wherein τ.sub.r indicates a threshold of r, which is used for distinguishing a texture region and the smooth region in the significant structure S. In the present invention, we obtain τ.sub.r with a histogram equalization method.

(31) Secondly, obtaining the significant structure of the blurred image with Formulas 8, 9 and 11, as shown in FIG. 4. FIG. 5 is different structures obtained by estimation at different sizes, specifically as follows:

(32) We introduce two substitution variables u and w to selectively substitute ∇S to obtain the Formula 3, and update S with an iterative method. A variant of the Formula 3 is as follows:

(33) $\begin{array}{cc}\underset{S,u,w}{\min }\left({.Math.S.Math.k-L.Math.}^2+{\lambda }_1{.Math.w.Math.}^{0.5}+{\lambda }_2{.Math.u\circ M.Math.}_0+{\lambda }_3{.Math.u\circ \left(1-M\right).Math.}_2^2+\beta {.Math.u-\nabla S.Math.}_2^2+\gamma {.Math.w-\nabla S.Math.}_2^2\right)& \left(\mathrm{Formula}6\right)\end{array}$

(34) We obtain the solution of each iteration S, u and w with alternately updated method; Solution of the variable u:

(35) $\begin{array}{cc}\underset{u}{\min }\left({\lambda }_2{.Math.u\circ M.Math.}_0+{\lambda }_3{.Math.u\circ \left(1-M\right).Math.}_2^2+\beta {.Math.u-\nabla S.Math.}_2^2\right)& \left(\mathrm{Formula}7\right)\\ u=\{\begin{array}{cc}\frac{\beta }{{\lambda }_3+\beta }\nabla S,& M=0\\ \nabla S,& M\ne 0,\nabla S^2\ge \frac{{\lambda }_2}{\beta }\\ 0,& \mathrm{otherwise}\end{array}& \left(\mathrm{Formula}8\right)\end{array}$

(36) Solution of the variable w:

(37) $\begin{array}{cc}\underset{w}{\min }\left({\lambda }_1{.Math.w.Math.}^{0.5}+\gamma {.Math.w-\nabla S.Math.}_2^2\right)& \left(\mathrm{Formula}9\right)\end{array}$

(38) We solve the Formula 9 with relatively total variation (RTV);

(39) Solution of the variable S:

(40) $\begin{array}{cc}\underset{s}{\min }\left({.Math.S.Math.k-L.Math.}^2+{\lambda }_1{.Math.w.Math.}^{0.5}+\beta {.Math.u-\nabla S.Math.}_2^2+\gamma {.Math.w-\nabla S.Math.}_2^2\right)& \left(\mathrm{Formula}10\right)\end{array}$

(41) Based on Parseval's theorem, we carry out Fourier transform on Formula 10 to obtain S:

(42) $\begin{array}{cc}S={ℱ}^{-1}\left(\frac{\begin{array}{c}F\left(L\right)\circ ℱ\left(k\right)+\\ \mathrm{\beta ℱ}\left(u\right)\circ \stackrel{\_}{ℱ}\left(\nabla L\right)+\mathrm{\gamma ℱ}\left(w\right)\circ \stackrel{\_}{ℱ}\left(\nabla L\right)\end{array}}{\begin{array}{c}ℱ\left(k\right)\circ \stackrel{\_}{ℱ}\left(k\right)+\\ \mathrm{\beta ℱ}\left(\nabla L\right)\circ \stackrel{\_}{ℱ}\left(\nabla L\right)+\mathrm{\gamma ℱ}\left(\nabla L\right)\circ \stackrel{\_}{ℱ}\left(\nabla L\right)\end{array}}\right)& \left(\mathrm{Formula}11\right)\end{array}$

(43) Wherein F indicates the Fourier transform, and F.sup.−1 indicates Fourier inversion.

(44) Step 3. blurring kernel estimation, specifically as follows:

(45) In the present invention, the blurring kernel is estimated with gradient information and the significant structure, and the blurring kernel trajectory is obtained through iterative update of Formula 14 and Formula 15, as shown in FIG. 6, diagram on the right of FIG. 8 and FIG. 9(c).

(46) Specifically, the blurring kernel is estimated with the significant structure S of the evaluated blurring image in the present invention. We suppress the outlier in the blurring kernel with L0 norm, and the optimization process is as follows:

(47) 0$\begin{array}{cc}\underset{k}{\min }\left({.Math.\nabla S.Math.k-\nabla L.Math.}_2^2+{\psi }_1{.Math.k.Math.}_2^2+{\psi }_2{.Math.\nabla k.Math.}_0\right)s.t.k\ge 0,\text{.Math.}{.Math.k.Math.}_1=1& \left(\mathrm{Formula}12\right)\end{array}$

(48) Similarly, we introduce a substitution variable v for iterative update, and the variant of the Formula 12 is as follows:

(49) $\begin{array}{cc}\underset{v,k}{\min }\left({.Math.\nabla S.Math.k-\nabla L.Math.}_2^2+{\psi }_1{.Math.k.Math.}_2^2+{\psi }_2{.Math.v.Math.}_0+\phi {.Math.v-\nabla k.Math.}_2^2\right)& \left(\mathrm{Formula}13\right)\end{array}$

(50) The solutions of the two variables (v and k) are as follows:

(51) $\begin{array}{cc}v=\{\begin{array}{cc}\nabla k,& \nabla k^2\ge \frac{{\psi }_2}{\phi }\\ 0,& \mathrm{otherwise}\end{array}& \left(\mathrm{Formula}14\right)\\ k={ℱ}^{-1}\left(\frac{ℱ\left(\nabla L\right)\circ \stackrel{\_}{ℱ}\left(\nabla S\right)+\mathrm{\phi ℱ}\left(v\right)\circ \stackrel{\_}{ℱ}\left(\nabla L\right)}{ℱ\left(\nabla S\right)\circ \stackrel{\_}{ℱ}\left(\nabla S\right)+\mathrm{\phi ℱ}\left(\nabla L\right)\circ \stackrel{\_}{ℱ}\left(\nabla L\right)}\right)& \left(\mathrm{Formula}15\right)\end{array}$

(52) Step 4: non-blind deconvolution, specifically as follows:

(53) Any existing non-blind deconvolution algorithm can be adopted here.

(54) The steps above can be expressed as the following algorithm flow:

(55) TABLE-US-00002 Algorithm: a priori constraint and outlier suppression based image deblurring method Input: blurred image I Output: blurred kernel k, and blurred restored image L Start: Conducting down-sampling I.sup.(0).fwdarw.I.sup.(1), . . . , I.sup.(n) to the blurred image (wherein I.sup.(n) has minimum size); Initializing the blurring kernel corresponding to the blurred image with the minimum size; Iteration: Updating the blurred image I.sup.(n) .fwdarw.S; Iteration: Estimation of a significant structure: Using Formula 8 to solve u; Using Formula 9 to solve w Using Formula 11 to solve the significant structure S; Estimation of the blurring kernel: Using Formula 14 to solve v; Using Formula 15 to solve k; Stopping after the iterations reach 6; n−1.fwdarw.n; Stopping until n = 0; Using non-blind deconvolution algorithm to obtain a restored image L.

(56) In the implementation of the present invention, non-blind deconvolution is carried out by using algorithm proposed in Literature 1 (Perrone, Daniele, and Paolo Favaro. “Total variation blind deconvolution: The devil is in the details.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014). FIG. 7 is the restored image of the blurred image. Taking FIG. 8 for example additionally, the left figure is the blurred image and a magnifying region thereof, and the right figure is a restoration effect and corresponding magnifying region. Wherein the blurring kernel is expressed at the upper left corner of the right figure in a form of the energy diagram.

(57) It should be noted that, the publicity of the embodiment aims at helping further understand the present invention, but those skilled in the art can understand that: all kinds of replacements and modifications may be possible without departing from the spirit and range of the present invention and claims attached. Therefore, the present invention should not be limited to the content disclosed by the embodiment, and the range protected as required by the present invention is subject to the range defined by the claims.