IMAGE FUSION METHOD BASED ON FOURIER SPECTRUM EXTRACTION

Abstract

The present invention discloses an image fusion method based on Fourier spectrum extraction, which comprises: step 1, inputting a plurality of to-be-fused images to a processor of a computer by an input unit of the computer, and performing the following steps by the processor of the computer: performing Fourier transform on images at different focus positions, extracting a frequency component corresponding to an image with the maximum frequency amplitude in the images at different focus positions in a transformed frequency domain space, taking the frequency component as a frequency component of a fused image at the corresponding spatial frequency, traversing each frequency to generate a frequency domain component of the fused image, finally performing inverse Fourier transform on the frequency domain component of the fused image to obtain the fused image; and step 2, outputting the fused image obtained by the processor by an output unit of the computer.

Claims

1. An image fusion method based on a Fourier spectrum extraction, comprising: step 1, inputting a plurality of to-be-fused images to a processor of a computer by an input unit of the computer, and performing the following steps by the processor of the computer: performing a Fourier transform on images at different focus positions, extracting a frequency component corresponding to an image with a maximum frequency amplitude in the images at the different focus positions in a transformed frequency domain space, taking the frequency component as a frequency component of a fused image at a corresponding spatial frequency, traversing each of a frequency to generate a frequency domain component of the fused image, finally performing an inverse Fourier transform on the frequency domain component of the fused image to obtain the fused image; and step 2, outputting the fused image, which is obtained by the processor by an output unit of the computer.

2. The image fusion method based on the Fourier spectrum extraction according to claim 1, wherein performing the steps by the processor specifically comprises: (1) obtaining a gray image information of each of the image from the plurality of to-be-fused images:
f.sub.n(x,y), (x<K,y<L), n=1, 2, . . . , N wherein, (x,y) is a pixel coordinate of a gray image, K and L are boundary values of the image in X and Y directions, respectively, and N is a total number of the images; (2) transforming N gray images in a spatial domain, which is obtained in the step (1) into the N gray images in a frequency domain by using a two-dimensional discrete Fourier transform to obtain the frequency component of each of the image: $F_n\left(f_x,f_y\right)=\underset{x=0}{\overset{K-1}{.Math.}}\underset{y=0}{\overset{L-1}{.Math.}}{f}_n\left(x,y\right)\exp \left[-j2\pi \left(x{f}_x/K+y{f}_y/L\right)\right],$ wherein, (f.sub.x,f.sub.y) is the coordinate of a spatial frequency, and represents the spatial frequency in the X and Y directions, and |F.sub.n(f.sub.x,f.sub.y) is a frequency domain amplitude; (3) extracting the frequency component corresponding to the image with the maximum frequency amplitude |F.sub.n(f.sub.x,f.sub.y)| in N images at the spatial frequency (f.sub.x,f.sub.y) as the frequency component of the fused image at the spatial frequency according to the frequency domain amplitude |F.sub.n(f.sub.x,f.sub.y)| obtained in the step (2), traversing each of points in the frequency domain, and finally generating the frequency domain component of N fused images by adopting the method:
F.sub.n(f.sub.x,f.sub.y).fwdarw.F(f.sub.x,f.sub.y); and (4) performing an inverse transform on a fused frequency domain component obtained in the step (3) by using a two-dimensional discrete inverse Fourier transform to obtain the gray image, which is reconstructed in the spatial domain, i.e., an image of the N images after fusion: $f\left(x,y\right)=\underset{x=0}{\overset{K-1}{.Math.}}\underset{y=0}{\overset{L-1}{.Math.}}F\left(f_x,f_y\right)\exp \left[-j2\pi \left(x{f}_x/K+y{f}_y/L\right)\right],$ f(x,y) is the gray image, which is obtained after reconstruction.

3. The image fusion method based on the Fourier spectrum extraction according to claim 2, wherein: in the step (1), a number of a plurality of the images N is greater than or equal to 2.

4. The image fusion method based on the Fourier spectrum extraction according to claim 2, wherein: in the step (1), a plurality of fused images have a same field of view and resolution.

5. The image fusion method based on the Fourier spectrum extraction according to claim 2, wherein: in the step (1), the plurality of images have different focus depths for objects at different depth positions or a same object.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG. 1 is a diagram of a system structure corresponding to the image fusion method based on Fourier spectrum extraction according to the present invention;

[0023] FIG. 2 is a flowchart of the image fusion method based on Fourier spectrum extraction according to the present invention;

[0024] FIG. 3 are to-be-fused images in the same field of view with the same camera shooting different focal planes in the present invention;

[0025] FIG. 4 are images of spatial frequency domain distribution after the two-dimensional discrete Fourier transform corresponding to the three images in FIG. 3;

[0026] FIG. 5 is a frequency domain image obtained by fusing the three images of spatial frequency domain distribution in FIG. 4; and

[0027] FIG. 6 is an image in the spatial domain reconstructed by two-dimensional discrete inverse Fourier transform on the frequency domain image in FIG. 5.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The present invention will be better understood from the following embodiments. However, it is easily understood by those skilled in the art that the descriptions of the embodiments are only for illustrating the present invention and should not and will not limit the present invention as detailed in the claims.

[0029] As shown in FIG. 1, the image fusion method based on Fourier spectrum extraction of the present invention comprises:

[0030] step 1, inputting a plurality of to-be-fused images to a processor of a computer by an input unit of the computer, and performing the following steps by the processor of the computer: performing Fourier transform on images at different focus positions, extracting a frequency component corresponding to an image with the maximum frequency amplitude in the images at different focus positions in a transformed frequency domain space, taking the frequency component as a frequency component of a fused image at the corresponding spatial frequency, traversing each frequency to generate a frequency domain component of the fused image, finally performing inverse Fourier transform on the frequency domain component of the fused image to obtain the fused image; and

[0031] step 2, outputting the fused image obtained by the processor by an output unit of the computer.

[0032] The input unit and the output unit are an input interface and an output interface of the processor, respectively, and the input interface and the output interface can be a network communication interface, a USB serial port communication interface, a hard disk interface and the like.

[0033] The image fusion method of the present invention extracts a clear image area based on Fourier transform, thereby generating a picture containing detailed information of objects at different depths in the shooting direction by fusing a plurality of images under the shooting condition of small depth of field. The algorithm of the present invention requires to shoot N images in different depth directions (Z direction) within the same field of view by changing focus positions of lens. Because of the limitation of the depth of field of the lens, each image can be clearly focused on the image plane (X, Y direction) only at a small depth in front and back near the focus plane. In order to display three-dimensional (X, Y, Z direction) information of a photographed object (or space) on one picture, N images are fused to generate one image. Detailed information (X, Y direction) of objects at different depth positions can be obtained from the image.

[0034] As shown in FIGS. 2-6, in the image fusion method based on Fourier spectrum extraction of the present invention, performing the steps by the processor specifically comprises:

[0035] (1) obtaining gray image information of each image from the plurality of to-be-fused images:

f.sub.n(x,y),(x<K,y<L), n=1, 2, . . . , N

[0036] wherein, (x,y) is the pixel coordinate of the gray image, and K and L are the boundary values of the image in X and Y directions, respectively; N is the total number of the images, and N is greater than or equal to 2; the plurality of images have the same field of view and resolution; the plurality of images have different focus depths for objects at different depth positions or the same object;

[0037] (2) transforming the N gray images in a spatial domain obtained in the step (1) into the N gray images in a frequency domain by using two-dimensional discrete Fourier transform to obtain the frequency component of each image:

[00003]$F_n\left(f_x,f_y\right)=\underset{x=0}{\overset{K-1}{.Math.}}\underset{y=0}{\overset{L-1}{.Math.}}{f}_n\left(x,y\right)\exp \left[-j2\pi \left(x{f}_x/K+y{f}_y/L\right)\right]$

[0038] wherein, (f.sub.x,f.sub.y) is the coordinate of the spatial frequency, and represents the spatial frequency in X and Y directions, |F.sub.n(f.sub.x,f.sub.y)| is the frequency domain amplitude, and the larger the amplitude is, the richer the frequency component is, and the more the detailed information of the image is;

[0039] (3) extracting the frequency component corresponding to the image with the maximum frequency amplitude |F.sub.n(f.sub.x,f.sub.y)| in the N images at the spatial frequency (f.sub.x,f.sub.y) as a frequency component of the fused image at the spatial frequency according to the frequency domain amplitude |F.sub.n(f.sub.x,f.sub.y)| obtained in the step (2), traversing each point (i.e., each spatial frequency (f.sub.x,f.sub.y)) in the frequency domain, and finally generating a frequency domain component of the N fused images by adopting the method:

F.sub.n(f.sub.x,f.sub.y).fwdarw.F(f.sub.x,f.sub.y)

[0040] (4) performing the spatial domain image reconstruction step, wherein F(f.sub.x,f.sub.y) comprises the detailed information of the images at different positions in the N images because of the directionality of f.sub.x,f.sub.y, in order to obtain the fused image effect from the frequency domain to the spatial domain, performing inverse transform on the fused frequency domain component obtained in the step (3) by using two-dimensional discrete inverse Fourier transform to obtain a gray image reconstructed in the spatial domain, i.e., an image of the N fused images:

[00004]$f\left(x,y\right)=\underset{x=0}{\overset{K-1}{.Math.}}\underset{y=0}{\overset{L-1}{.Math.}}F\left(f_x,f_y\right)\exp \left[-j2\pi \left(x{f}_x/K+y{f}_y/L\right)\right]$

[0041] f(x,y) is the gray image obtained after reconstruction.

[0042] In FIG. 3, the images in the pictures are clearly displayed only at focus positions, that is, the edge and detailed texture information is richer; in the fused image (FIG. 6), the detailed information of the three focus positions is well fused into one image, that is, the detailed information of the objects at different shooting depth positions can be seen from one picture, so that the image fusion effect is effectively realized.