Shearogram generation algorithm for moving platform based shearography systems
09818181 · 2017-11-14
Assignee
Inventors
- Andrew N. Acker (Honolulu, HI)
- Michael J. DeWeert (Kailua, HI)
- Michael L. Dichner (Kailua, HI)
- Brian P. Farm (South Portland, ME, US)
- Tamara O'Toole (Pensacola Beach, FL)
- Clint Yoshimura (Honolulu, HI)
Cpc classification
International classification
Abstract
A system and method are presented for generating shearograms from raw specklegram images which may, for example, be collected from airborne or other mobile shearography equipment. The system and method is used to detect and characterize buried mines, improvised explosive devices (IEDs), and underground tunnels, bunkers, and other structures. Amongst other purposes, the system and method may also be used for rapid scanning of ship hulls and aircraft for hidden structural defects, rapid pipeline inspection, and non-contact acoustic sensing for in-water and underground sources.
Claims
1. A method comprising: providing a specklegram set comprising a plurality of specklegram images of a target surface; high pass filtering the specklegram images; co-registering the specklegram images to produce a co-registered specklegram set such that speckle patterns in the plurality of co-registered specklegram images are aligned with one another; generating a shearogram image from the co-registered specklegram set; and using the shearogram image to detect surface changes of the target surface.
2. The method of claim 1 wherein the plurality of specklegram images comprises at least four specklegram images.
3. The method of claim 1 wherein using the shearogram image comprises using the shearogram image to detect subsurface structures or movements.
4. The method of claim 1 wherein the high pass filtering comprises low pass filtering the input image to produce a low pass filtered image; and dividing the input image by the low pass filtered image.
5. The method claim of 4 wherein the low pass filtering is achieved with one of a rectangular filter kernel and a Gaussian filter kernel.
6. The method of claim 2 wherein the co-registering comprises shifting three of the at least four specklegram images to a frame of reference of the other of the at least four specklegram images.
7. The method claim of 2 wherein the at least four specklegram images comprise first, second, third and fourth specklegram images; and the co-registering comprises shifting each of the second, third and fourth specklegram images to a frame of reference of the first specklegram image.
8. The method claim of 7 wherein the co-registering comprises determining a first shift value between the first specklegram image and the second specklegram image, a second shift value between the first specklegram image and the third specklegram image and a third shift value between the first specklegram image and the fourth specklegram image.
9. The method of claim 6 wherein the calculation further comprises finding a pair of X and Y translation values which maximize the cross correlation between the first and fourth images.
10. The method of claim 9 further comprising computing a second shift value for the second specklegram image by linear interpolation based on the pair of X and Y translation values.
11. The method of claim 10 further comprising computing a third shift value for the third specklegram image by linear interpolation based on the pair of X and Y translation values.
12. The method of claim 6 wherein the co-registering comprises selecting a primary sub-region of a given one of the specklegram images; and producing a registration solution based on the primary sub-region.
13. The method claim of 12 wherein the registration solution is a valid registration solution defined as a shift of at least 2 pixels in a given direction.
14. The method claim of 12 wherein if the registration solution is not a valid registration solution, the co-registering further comprises selecting a secondary sub-region of the given one of the specklegram images; and producing a registration solution based on the secondary sub-region.
15. The method of claim 1 wherein the generation of the shearogram image from the specklegram set utilizes a baseline method for a phased-resolved shearogram, which is
16. The method of claim 1 wherein the generation of the shearogram image from the specklegram set utilizes a peak straddling method for a phased-resolved shearogram, which is
17. The method of claim 1 wherein the generation of the shearogram image from the specklegram set utilizes a generic method for a phased-resolved shearogram, which is
18. The method of claim 1 wherein the generation of the shearogram image from the specklegram set utilizes a non-phased resolved shearogram, which is
Shr.sub.NPR=|Speck.sub.4−Speck.sub.1|, wherein Speck.sub.1 and Speck.sub.4 are the first specklegram image and fourth specklegram image respectively.
19. A system comprising: shearography equipment comprising an interferometer which has a processing unit and is configured to collect a plurality of specklegram images of a target surface; wherein the processing unit is configured to high pass filter the specklegram images, co-register the specklegram set to produce a co-registered specklegram set such that speckle patterns in the plurality of specklegram images are aligned with one another, and generate a shearogram image from the co-registered specklegram set such that the shearogram image is usable to detect surface changes of the target surface.
20. The system of claim 19 wherein the shearogram image is usable to detect subsurface structures or movements.
Description
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
(1) A sample embodiment of the invention is set forth in the following description, is shown in the drawings and is particularly and distinctly pointed out and set forth in the appended claims.
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16) Similar numbers refer to similar parts throughout the drawings.
DETAILED DESCRIPTION
(17)
(18) System 1 includes an algorithm and method to generate shearograms from raw specklegram images collected from the shearography equipment thereof. System 1 may be an airborne shearography system known as Synoptic Airborne Multi-Sensor Systems (SAMSS). SAMSS is an airborne system configured to extend capabilities such as detecting and characterizing mines, IEDs, and other underground or underwater objects of military interest. System 1 in one example provides motion-compensated shearography as described in the below-noted Kokobun reference, which is a practical and robust method for finding hidden structures and objects. The algorithm and method disclosed herein can be used for the following and other purposes: 1) mine and IED detection; 2) airborne detection and characterization of underground tunnels, bunkers, and structures; 3) rapid scanning of ship hulls and aircraft for hidden structural defects; 4) rapid pipeline inspection; and 5) non-contact acoustic sensing for in-water and underground sources. The algorithm and method includes several steps, including co-registration of specklegram images used to align and emphasize the high frequency speckle noise structure of the images. It performs illumination normalization on input images and allows generation of both phase resolved (PR) and non-phase resolved (NPR) specklegram images using multiple techniques. U.S. Pat. No. 8,717,577 granted to Kokobun et al. and entitled “Compensating aperture for utilizing shearography from a moving platform” describes a motion compensated shearography apparatus or system and methods which may be used with or be part of the systems and methods described herein. Said Publication is incorporated fully herein by reference.
(19) In operation, as platform 2 and equipment 1 are moving relative to target 4, the target excitation device is used to excite target surface 4, such as with acoustic energy or seismic energy, which can cause different surface deformations of target surface 4, such as over buried objects relative to the background. These deformations are measured with laser shearography techniques, by illuminating the ground or other target surface 4 with coherent laser light emitted from the laser transmitters of equipment 1, collecting specklegrams or specklegram images over the illuminated region with a shearing interferometer of equipment 1, and generating shearograms from the collected specklegrams.
(20) More specifically, one of the one or more laser transmitters of shearography equipment 1 transmits, emits or shoots a laser beam which impacts target surface or target area 4 and is reflected from surface 4 as a reflected laser beam image back to apparatus 1 into the beam splitter, onto the mirrors and into the image shearing camera, which captures the reflected image in two copies which are laterally displaced (sheared image copies) and combined to form a specklegram. The reflected laser beam images and specklegrams are collected and stored or saved in a memory of the equipment 1 computer or CPU, which includes a computer program of the shearography system which is configured to process the specklegrams to produce a shearogram from which can be discerned surface changes of target surface 4 and corresponding subsurface structures or movements. The computer program run by the CPU is configured to perform and calculate the various relevant steps and equations discussed herein in order to effect the methods discussed herein.
(21)
(22) Method or algorithm 20 typically takes inputs of a series of at least four specklegrams collected from a moving platform. These specklegrams are co-registered such that the speckle patterns in each specklegram are aligned together or with one another. Co-registration may be accomplished by applying normalization and filtration by a high pass filter to the images. Then, a Fast Fourier Transform (FFT) process can be applied to locate the cross-correlation maximum. The high pass filtering process suppresses illumination variation across images and maximizes the high frequency speckle noise content of the imagery. The cross-correlation process determines the translational offset between pairs of filtered specklegrams. The translational offsets are used to co-reregister the specklegram set. Shearograms are then generated from the set of four co-registered filtered specklegrams.
(23) Input specklegrams are high-pass filtered (spatially filtered) and normalized to suppress low spatial frequency variation in the images so that filtered output high frequency speckle noise dominates in the resultant. The high pass normalization filter first applies a low pass filter by convolving the input specklegram image with a normalized filter kernel to produce a low pass filtered specklegram image. Either a rectangular (boxcar) filter kernel or a Gaussian filter kernel can be used for this operation. A filter kernel herein is an array of numbers or elements, and normalized filter kernel herein means that the sum of elements in the filter kernel is one. For instance, a 5×5 normalized boxcar filter kernel is a 5×5 array of elements, each with the value of 1/(5×5)=1/25. In practice, application of a normalized filter kernel to an image (convolving the filter kernel with the image) leaves the mean value of the image unchanged.
(24) The low pass filtered image LP may be obtained by convolving the input image I with the low pass filter kernel K:
LP=I{circle around (x)}K
(25) For a square boxcar filter kernel of dimension (2m+1), K can be written as
(26)
(27) where M is a (2m+1)×(2m+1) matrix with M.sub.i,j=1 for all matrix elements. For this case, the elements of the filtered image LP are given by
(28)
(29) The effect of the low pass boxcar filtering operation is to replace I(i, j) by a local average computed in a (2m+1)×(2m+1) square region center at pixel I(i, j).
(30) The original input specklegram image is then divided by the low pass filtered specklegram image, producing a high pass filtered normalized specklegram image as an output. Particularly, the division process is applied on a pixel-by-pixel basis, that is, if Output (i,j), Input (i,j), and LP (i,j) represents a pixel in the output image, input image, and low pass filtered image respectively, then Output (i,j)=Input (i,j)/LP (i,j). More particularly, the low pass filter kernel should be selected such that it is sufficiently large compared to a spot size (in pixel) produced by sensors or image shearing cameras, but smaller than the typical intensity variations across the image. For example, a sensor or image shearing camera with a spot size of order 1-pixel, a 21×21 pixel rectangular filter kennel (boxcar) is effective.
(31) For shearographic systems the nominal spot size generated by an image shearing camera with a diffraction limited lens is approximately 1 pixel as measured on the camera's focal plane. In a system using 532 nm laser light, the physical spot size on the focal plane is approximately
spot_size=(f/#)×1.3μ
(32) In constructing a shearography system, a lens is typically chosen with an f-number (f/#) that produces a spot size about equal to one pixel on the focal plane (the pixel pitch), i.e.,
(33)
(34) For typical 1-pixel spot size systems, the filter kernel may be at least 11×11 pixels, i.e., m=5 in a (2m+1)×(2m+1) filter kernel. Intensity variations across the image may either be caused by illumination non-uniformity of the laser source or reflectivity variation across the scene. The actual scale of the intensity variation can change from scene to scene.
(35) Table 1 below shows a MatLab implementation of a high pass filter kernel. Correspondingly,
(36) TABLE-US-00001 TABLE 1 normfilt = ones(21,21) / (21{circumflex over ( )}2); % Smooth the specklegrams prior to registration div1 = filter2(normfilt, speck1_img); div2 = filter2(normfilt, speck2_img); div3 = filter2(normfilt, speck3_img); div4 = filter2(normfilt, speck4_img); speck1_img = speck1_img ./ div1; speck2_img = speck2 img ./ div2; speck3_img = speck3_img ./ div3; speck4_img = speck4_img ./ div4;
(37) At least four specklegrams are typically used to generate phase resolved (PR) shearograms. A given set of specklegrams is co-registered prior to generating a shearogram from that given set. A top level image co-registration procedure is illustrated in Table 2 below and its corresponding MatLab code snippet is provided in Table 3 below.
(38) TABLE-US-00002 TABLE 2 Compute registration .fwdarw. Compute Registration .fwdarw. Shift offset Offsets Speck4, Speck3, Speck4 to Speck1 Speck2-Speck1 Speck1 To Using dftRegM Speck 3-Speck1 Speck1 Basis Function By interpolation
(39) TABLE-US-00003 TABLE 3 %calculate registration offset speck4_img relative to speck1_img [dX4 dY4] = dftRegM(speck1_img, speck4_img, 1); % interpolation method dX2 = dX4/3; dY2 = dY4/3; dX3 = 2*dX4/3; dY3 = 2*dY4/3; %shift images speck2_img = imgShift(speck2_img, dX2, (−1.0*dY2)); speck3_img = imgShift(speck3_img, dX3, (−1.0*dY3)); speck4_img = imgShift(speck4_img, dX4, (−1.0*dY4));
(40) In the co-registration procedure, as shown in Table 2, a computation of registration offset between pairs of specklegrams is performed to compute a given shift value. For instance, a shift value may be computed between the first and second specklegrams, another shift value may be computed between the first and third specklegrams, and another shift value may be computed between the first and fourth specklegrams, so that the second, third and fourth specklegrams may be shifted to the basis or frame of reference of the first specklegram. For example, a shift value (translation offset) between the first and fourth specklegrams (specklegrams #1 and #4) collected in the specklegram set is computed by finding the pair of X and Y translation values that maximize the cross-correlation between the two images. Second and third shift values for intermediate specklegrams (specklegrams #2 and #3) may then be computed by linear interpolation. The velocity of the moving platform 4 and equipment 1 can be set constant over the short period (less than milliseconds) when the specklegram set is acquired.
(41)
(42) In one example, the image registration is more robust if it is calculated using only the central portion of the image. The process may make two attempts to find a desirable or best co-registration solution for the specklegram pair using two image partitions. First, for example, a primary sub-region 34 such as shown in
(43) Table 4 below depicts top level MatLab implemented functions (“dftRegM” and “dftReg”) which may handle image registration, calculating translational offsets dX,dY. The function “dftRegM” is used to define sub-setting options. The function “dftReg” is used to 1) apply the sub-setting options; 2) take the Fourier transform of the filtered image segment; and 3) call a cross-correlation function “dftregistration”.
(44) TABLE-US-00004 TABLE 4 function [dx dy] = dftRegM(img1,img2) %first call display(‘mode M’); subset=4; [dx1 dy1] = dftReg(img1,img2,subset); %test result MShift=max(abs(dx1),abs(dy1)); if (MShift<2) subset=3; display(‘trying alternate registration’) [dx dy] = dftReg(img1,img2,subset); else dx=dx1; dy=dy1; end function [dx dy] = dftReg(img1,img2,subset) % subset image (crop out lo signal regions) % (NN=4); use NN=3 for central third if (nargin > 2 && subset ~= 0) NN=subset; % determine lines/samples corresponding to central half of image % (NN=4); use NN=3 for central third [lines1,samples1]=size(img1); LL1=round(lines1./NN); LS1=round(samples1./NN); UL1=lines1-LL1; US1=samples1-LS1; [lines2,samples2]=size(img2); LL2=round(lines2./NN); LS2=round(samples2./NN); UL2=lines2-LL2; US2=samp1es2-LS2; img1 = img1(LL1:UL1,LS1:US1); img2 = img2(LL2:UL2,LS2:US2); end % take FFT of each image I1 = fft2(img1); I2 = fft2(img2); % call the script from Mathworks [output] = dftregistration(I1,I2,10); % extract shift values dx = output(4); dy = output(3); end
(45) Once a pair of translational offsets (dx,dy) is determined for a given specklegram, here the fourth specklegram, the given or fourth specklegram image (specklegram #4) is then shifted to the frame reference of the first specklegram image (specklegram #1). The second specklegram image (specklegram #2) and the third specklegram image (specklegram #3) are shifted to the frame reference of the first specklegram (specklegram #1) by 2-dimensional linear interpolation of the input image to new pixel center coordinate shifted by (dx,dy). This image shifting may be implemented in the MatLab function “imgShift” shown in Table 5 below.
(46) TABLE-US-00005 TABLE 5 % function ims = imgShift(img,dx,dy) % function ims = imgShift(img,dx,dy) if ( isnan(dx) ) error(‘Illegal value for input argument “dx”; must be a positive integer.’); end if ( isnan(dy) ) error(‘Illegal value for input argument “dy”; must be a positive integer.’); end [r c b] = size(img); if (b>1) error(‘Function “imgShift” only defined for single band images.’); end [X,Y] = meshgrid(1:c,1:r); [XI,YI] = meshgrid((1-dx):1:(c-dx),(1+dy):1:(r+dy)); ims = interp2(X,Y,double(img),XI,YI); end
(47) The preprocessed (i.e., high pass filtered and normalized) co-registered specklegrams can now be combined to produce both phase-resolved (PR) and non-phase resolved (NPR) shearograms. The process may, for instance, support the following three PR shearogram methods and NPR shearogram method: PR method 1 (Baseline method)
(48)
(49)
(50)
Shr.sub.NPR=|Speck.sub.4−Speck.sub.1| (4)
(51) where Speck.sub.1, Speck.sub.2, Speck.sub.3, and Speck.sub.4 are the input set of co-registered specklegram images. Each image set is acquired over a time span on the order of ¼ to ½ the period of the ground vibration (or other target vibration), thereby ensuring that the image set covers a significant fraction of the available ground (target) amplitude change. For example, if specklegrams are collected at a rate of 1 KHz, then the time period over which a four-specklegram set is collected is three (3) milliseconds. This sampling rate is nominal for detecting ground (target) vibration with periods in the range if 6-12 milliseconds (83-167 Hz).
(52) Each of the three PR methods is optimized for a different phase relationship between the specklegram sampling time and the ground vibration. The raw shearograms are spatially filtered prior to display. The NPR shearogram may be filtered with a simple Gaussian filter. The PR shearograms are subject to “salt and pepper” type noise, and may be filtered using a median filter. The shearogram generation process including filtering may be implemented in the MatLab function shown in Table 6 below.
(53) TABLE-US-00006 TABLE 6 function [shear_img] = make15Sharorgrams(speck1_img,speck2_img,speck3_img, speck4_img,method, mf) % function to compute gen 1.5 PR and NPR shearograms from a set of 4 input % specklegrams %INPUT: % specklegrams (all specklegrams are assumed co-registered) %speck1_img input specklegram #1 %speck2_img input specklegram #2 %speck3_img input specklegram #3 %speck4_img input specklegram #4 % method: % 0 =NPR % 1 =PR method 1 original method (use as default) % 2 =PR method 2 peak straddling % 3 =PR method 3 generic % % mf= median filter kernal size (should be 5,9 or 13 only) For method=0 no median filter is applied switch method case 0 %NPR Shrearogram h = normpdf(1:7,4,1.5); h = h′ * h; shear_img = fi1ter2(h,abs(speck4_img - speck1_img)); case 1 % Standard PR this uses “original” normalization factor T_41=speck4_img-speck1_img; T_23=speck2_img-speck3_img; shear_img = ((1*T_41)./(sqrt(3)*T_23)); shear_img=medfilt2(shear_img,[mf,mf]); case 2 % Peak Straddling case (no normalization included) TM2_1=((speck2_img-speck1_img) + (speck2_img- speck3_img) + (speck2_img-speck4_img)); TM2_2=((speck2_img-speck1_img) + (speck3_img- speck4_img)); shear_img=TM2_1./TM2_2; shear_img=medfilt2(shear_img,[mf,mf]); case 3 % Generaic case (no normalization included) TM3_1=(2*(speck2_img-speck1_img) + (speck2_img- speck3_img)); TM3_2=((speck2_img-speck1_img) + (speck3_img- speck1_img)); shear_img=TM3_1./TM3_2; shear_img=medfilt2(shear_img,[mf,mf]); otherwise disp(‘invalid method’); end
(54) Sample shearograms generated from this MatLab script are shown in
(55) The use of shearagraphy equipment 1 and the shearograms or shearogram images produced thereby allow for the discernment, detection or identification of underground anomalies such as underground ordnance or landmines (including improvised explosive devices or IED) or other subsurface or underground objects or structures such as rooms, tunnels, pipes and so forth. Shearogram images may be used to detect subsurface structures or movements in a target which defines the target surface, including underground and underwater objects or structures or movements. Subsurface movements may include leaking underground pipes (e.g., movement of water or other liquid) or the digging or building of a tunnel or underground room.
(56) The present shearography system and process may also be used in the medical field, for instance, including non-contact monitoring of respiration, non-contact cardiograms and non-contact ultrasound imaging. There may be many other uses—any phenomenon (such as sound) which can convey information from inside a body to make a physical change in the surface might be exploited. In such cases, target surface 4 may, for instance, represent a skin surface of a person (or animal) undergoing the respective shearography procedure, which would thus allow via shearograms for the determining or ascertaining subsurface structures or movement within a human body (or animal).
(57) In the foregoing description, certain terms have been used for brevity, clearness, and understanding. No unnecessary limitations are to be implied therefrom beyond the requirement of the prior art because such terms are used for descriptive purposes and are intended to be broadly construed. Moreover, the description and illustration set out herein are an example and not limited to the exact details shown or described.