Three-dimensional shape, displacement, and strain measurement device and method using periodic pattern, and program therefor

Abstract

In a conventional moir method, achieving both measurement accuracy and dynamic measurement and balancing field of view and measurement accuracy have been difficult. The present invention makes it possible to handle conventional moir fringes as a grating for generating phase-shifted second-order moir fringes, use a spatial phase shift method algorithm to accurately analyze the phases of the second-order moir fringes before and after deformation, and determine shape from the phase differences between gratings projected onto the surface of an object of measurement and a reference surface and determine deformation and strain from the phase differences between the second-order moir fringes, before and after deformation, of a repeating pattern on the object surface or a produced grating. As a result, it is possible to measure the three-dimensional shape and deformation distribution of an object accurately and with a wide field of view or dynamically and with a high degree of accuracy.

Claims

1. A method for measuring a displacement distribution: comprising a first step for obtaining a first moir fringe by first down-sampling and intensity interpolation to a grating image in a predetermined direction, the grating image displaying a lattice on a specimen; a second step for generating a second moir fringe by second down-sampling and intensity interpolation to the first moir fringe in the predetermined direction; and a third step for calculating the displacement distribution in the predetermined direction or a depth direction based on a phase difference distribution of the second moir fringe before and after deformation.

2. The method according to claim 1, wherein the first step comprises a step of generating T frames of the first moir fringes with phases shifted by t times of a reciprocal of T, t denoting an integer greater than or equal to 0 and less than or equal to T1, T denoting a first down-sampling pitch for the first down-sampling; and the second step comprises a step of generating T.sup.(2) frames of the second moir fringe with phases shifted by k times of a reciprocal of T.sup.(2), k denoting an integer greater than or equal to 0 and less than or equal to T.sup.(2)1, T.sup.(2) denoting a second down-sampling pitch for the second down-sampling; and a step of generating the phase distribution of the second moir fringe by performing Fourier transform of the T.Math.T.sup.(2) frames of the second moir fringe.

3. The method according to claim 1, further comprising a fourth step of adjusting the pitch of the first moir fringe by multiplying the phase of the first moir fringe by an adjustment factor N, N denoting an integer greater than or equal to 2; wherein the first moir fringe with the adjusted phase is applied to the second step.

4. The method according to claim 1, wherein the first step is characterized by scanning the grating image with a predetermined first down-sampling pitch to record the first moir fringe.

5. The method according to claim 1, further comprising a fourth step of calculating a strain distribution by differentiating the displacement distribution in the predetermined direction; wherein the grating is imparted on the surface of the specimen.

6. The method according to claim 1, wherein the phase difference distribution is a difference between the phase distribution of the second moir fringe derived from the grating on a reference surface and the phase distribution of the second moir fringe derived from the grating on the specimen, the grating projected by a projecting device; and the third step comprises a step of calculating the displacement distribution in the depth direction based on the phase difference distribution and a conversing factor regarding a relation between a distance in the depth direction and the phase difference, the displacement distribution being a shape of the specimen.

7. An apparatus for measuring a displacement distribution: comprising storage circuitry configured to obtain a first moir fringe by first down-sampling and intensity interpolation to a grating image in a predetermined direction, the grating image displaying a lattice on a specimen; generation circuitry configured to generate a second moir fringe by second down-sampling and intensity interpolation to the first moir fringe in the predetermined direction; and arithmetic operation circuitry configured to calculate the displacement distribution in the predetermined direction or a depth direction based on a phase difference distribution of the second moir fringe before and after deformation.

8. A computer program product stored in a non-transitory computer-readable medium that is programmed for measuring a displacement distribution to perform: a first step of obtaining a first moir fringe by first down-sampling and intensity interpolation to a grating image in a predetermined direction, the grating image displaying a lattice on a specimen; a second step of generating a second moir fringe by second down-sampling and intensity interpolation to the first moir fringe in the predetermined direction; and a third step of calculating the displacement distribution in the predetermined direction or a depth direction based on a phase difference distribution of the second moir fringe before and after deformation.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) FIG. 1 is a diagram showing principles of different phase analysis techniques.

(2) FIG. 2 shows fields of view in different moir methods.

(3) FIG. 3(a) is a diagram showing traditional moir fringes, and FIG. 3(b) is a diagram showing the principle of formation of the 2nd-order moir fringes.

(4) FIG. 4 is a diagram showing the principle of calculation of the phase distribution of the obtained 2nd-order moir fringes by performing down-sampling processing on a moir fringe pattern.

(5) FIG. 5 is a diagram showing the principle of calculation of the phase distribution of the obtained 2nd-order moir fringes by performing down-sampling processing on a grating image in two stages.

(6) FIG. 6 is a flow diagram showing the principle of generation of multiplication type moir fringes using a phase distribution and imaging processing.

(7) FIG. 7 is a configuration diagram of the three-dimensional shape measurement device according to the present invention.

(8) FIG. 8 is a configuration diagram of a displacement and strain measurement device according to the present invention.

(9) FIG. 9 is a flowchart of the deformation measurement step in the proposed technique.

(10) FIG. 10 is a flowchart of the deformation measurement program in the proposed technique.

(11) FIG. 11 is a table showing comparison between distributions of strain measurement accuracies of different moir methods.

(12) FIG. 12 shows an example in which the 2nd-order moir method is applied to any two-dimensional repetitive patterns.

(13) FIG. 13 shows an example in which multiplication type moir fringes are generated by the third method according to the present invention.

(14) FIG. 14 shows an example of phase analysis performed on a projection grating by the 2nd-order moir method.

(15) FIG. 15 is a diagram showing comparison between phase analysis results of projection gratings according to the traditional methods and the present invention.

(16) FIG. 16 shows an example in which the 2nd-order moir method is applied to the measurement of the three-dimensional non-contact shape through grating projection.

(17) FIG. 17 is a diagram showing comparison between height measurement results of objects according to the traditional methods and the present invention.

(18) FIG. 18 is a diagram showing simulation verification results of uniform deformation measurement in the proposed technique.

(19) FIG. 19 is a diagram showing simulation verification results of non-uniform deformation measurement in the proposed technique.

(20) FIG. 20 is a diagram showing analysis results of displacement and strain distributions obtained using inclined gratings having random noise in the case in which strain values obtained by simulation are 0 and 1000.

(21) FIG. 21(a) is diagram showing the error of the mean strain measured by the 2nd-order moir method on the basis of the strain value (preset) obtained by simulation, and FIG. 21(b) is diagram showing the standard deviation of the measured strain.

(22) FIG. 22 is photographs showing an aluminum test piece and setup of a tension experiment using a laser microscope.

(23) FIG. 23 is a diagram showing results of displacement and strain distribution measurement performed by the 2nd-order moir method when the strain gauge value is 3000 during the tension test for the aluminum test piece.

(24) FIG. 24 is a diagram showing comparison between the stress curve and the strain curve of the aluminum specimen which are obtained using the proposed technique and a strain gauge.

DESCRIPTION OF EMBODIMENTS

Example 1

(25) Confirmation of Principle of Generation of 2nd-Order Moir Fringes of any Repetitive Pattern Obtained by Simulation

(26) Here, effects in the case in which a 2nd-order moir method is applied to any two-dimensional repetitive pattern are described.

(27) FIG. 12(a) shows a repetitive pattern (an alphabetical character A) which exists or is fabricated on the surface of an object.

(28) FIG. 12(b) shows the moir fringe image obtained by performing first down-sampling on the pattern shown in FIG. 12(a) at a fixed pitch in the x-direction and the y-direction.

(29) This moir fringe image is regarded as a grating, and FIG. 12(c) shows the moir fringe image obtained by further performing second down-sampling on the moir fringe image at a fixed pitch in the x-direction and the y-direction.

(30) It can be understood that the original repetitive pattern A is further enlarged.

(31) This is called 2nd-order moir.

Example 2

(32) Confirmation of Principle of Generation of Multiplication Type Moir Fringes Obtained by Simulation

(33) In the present invention, the interval between moir fringes obtained by performing down-sampling processing on a grating image may be large.

(34) In this case, it is possible to reconfigure multiplication type moir fringes in which the interval between moir fringes becomes small so that an appropriate interval between moir fringes is obtained by the third method according to the present invention.

(35) FIG. 13 shows results of generation of (multiplication type) moir fringes by simulation.

(36) FIG. 13(a) shows a grating pattern having a pitch interval of 3.02 pixels generated by the simulation.

(37) FIG. 13(b) shows two phase-shifted moir fringe images obtained by performing thinning processing on a grating image of FIG. 13(a) with a down-sampling pitch of 2 pixels.

(38) Since the grating pitch P (3.02 pixels) is greatly different from the sampling pitch T (2 pixels), it can be confirmed that moir fringes cannot be generated.

(39) FIG. 13(c) shows three phase-shifted moir fringe images obtained by performing thinning processing on the grating image of FIG. 13(a) with a down-sampling pitch of 3 pixels.

(40) Since the grating pitch P (3.02 pixels) and the sampling pitch T (3 pixels) are close to each other, distinct moir fringes can be generated, but it can be understood that the interval between the moir fringes is large.

(41) The application of the 2nd-order moir method in this situation results in a problem in that the spatial resolution is significantly decreased.

(42) FIG. 13(d) shows four moir fringe images generated when the number of thinning out for shifting four phases obtained by thinning out the grating image of FIG. 13(a) with a down-sampling pitch of 4 pixels.

(43) Similarly to FIG. 13(b), since the grating pitch P (3.02 pixels) and a sampling pitch T (4 pixels) are greatly different from each other, it can be understood that moir fringes cannot be generated.

(44) FIG. 13(e) shows a phase distribution image of moir fringes calculated using the moir fringe image (FIG. 13(c)) generated when the sampling pitch is 3 pixels.

(45) FIG. 13(f) shows results of the moir fringe image reconfigured (multiplication type) in the case in which the adjustment factor N is changed from 1 to 20 by using the phase distribution image.

(46) In this manner, it is possible to adjust the interval between moir fringes from a sparse state to a dense state by changing the adjustment factor N.

Example 3

(47) Verification of Non-Contact Measurement Experiment of Three-Dimensional Shape of Object which is Performed by Grating Projection Method

(48) Effects of the improvement in the accuracy of non-contact measurement of the three-dimensional shape of an object which is performed by the 2nd-order moir method proposed in the present invention are confirmed through an experiment.

(49) FIG. 14 shows an example of analysis results obtained in the case in which the image of a grating, which is projected by a grating projection device, is captured by a camera, and is applied to a 2nd-order moir method using the three-dimensional shape measurement device as shown in FIG. 7.

(50) Here, the grating image captured by projecting sine waves having fixed cycles (cycles of 8 pixels in a projector) on the reference surface is shown in FIG. 14(a).

(51) FIG. 14(b) shows a phase distribution of moir fringes calculated using 25 phase-shifted moir fringes by applying the traditional sampling moir method to FIG. 14(a) to perform down-sampling and intensity interpolation on 25 pixels in the horizontal direction.

(52) FIG. 14(c) shows the phase distribution of 2nd-order moir fringes calculated by two-dimensional discrete Fourier transform by regarding these moir fringe images as phase-shifted grating images, further performing down-sampling and intensity interpolation on 100 pixels in the horizontal direction, and using 2500 of 10025 phase-shifted 2nd-order moir fringe images to be generated.

(53) FIG. 14(d) shows the phase distribution of the original projection grating obtained by adding the phase of the sampling point to the phase distribution of 2nd-order moir fringes (Equation (7)).

(54) In the case of this example, it is possible to generate 2500 phase-shifted moir fringes from one grating image and to perform phase analysis with a higher level of accuracy.

(55) FIG. 15 shows comparison results of analysis performed on the grating image on the reference surface as shown in the FIG. 14(a) according to the phase shifting method, the sampling moir method, and the spatiotemporal phase shifting method which are traditional techniques and the present invention.

(56) FIG. 15(a) shows results (phase gradient) obtained by horizontally spatially differentiating a phase distribution obtained by calculating eight captured phase-shifted grating images by the traditional phase shifting method.

(57) The left diagram shows a phase gradient distribution, and the right diagram is the cross-sectional view of one horizontal center line.

(58) Similarly, FIG. 15(b) shows results obtained by calculating one captured grating image by the sampling moir method (the number of thinning-out is 25 pixels).

(59) FIG. 15(c) shows results obtained by calculating eight captured phase-shifted grating images by the spatiotemporal phase shifting method (the number of thinning-out is 25 pixels).

(60) FIG. 15(d) shows results obtained by calculating one captured grating image by the present invention (the number of thinning-out is 25 pixels at the first stage and is 100 pixels at the second stage).

(61) A correct result is that the phase gradient of the reference surface is a smooth straight line with a certain inclination.

(62) As can be seen from the results of FIG. 15, the traditional phase shifting methods have great measurement errors.

(63) The sampling moir method can perform phase calculation using one grating image but has a measurement error.

(64) Since the spatiotemporal phase shifting method uses both temporal and spatial intensity information, the spatiotemporal phase shifting method is more accurate than the phase shifting method and the sampling moir method but is not suitable for dynamic measurement because a plurality of images are required.

(65) On the other hand, according to the present invention, the most accurate results are obtained using only one grating image, and thus the effectiveness of the present invention is shown.

(66) Next, an example in which the dynamic three-dimensional shape is measured with high accuracy by the second method according to the present invention will be described.

(67) FIG. 16 shows analysis results obtained by applying a reference surface and a grating image of an object (hand) obtained by capturing a grating, which is projected by a grating projection device, by a camera to the 2nd-order moir method using the three-dimensional shape measurement device as shown in FIG. 7.

(68) FIG. 16(a) shows images captured by cameras installed at different angles by projecting two types of sinusoidal grating patterns with grating pitch ratios of 8 and 9 on the reference surface and the object in order from the top.

(69) Phase distributions calculated by performing the 2nd-order moir method on these images are shown in FIG. 16(b).

(70) Here, two types of grating patterns with grating pitch ratios of 8 and 9 are used so that a height can be measured even with an object having a depth.

(71) The phase difference between the reference surface and the object in the case in which the grating pitch ratio is 8 is shown in FIG. 16(c), and similarly, the phase difference between the reference surface and the object in the case in which the grating pitch ratio is 9 is shown in FIG. 16(d).

(72) Analysis conditions in the case in which the grating pitch ratio is 8 are the same as those in FIG. 14.

(73) In analysis conditions in the case in which the grating pitch ratio is 9, the first down-sampling pitch is 29 pixels and the second down-sampling pitch is 110 pixels.

(74) FIG. 16(e) shows the phase distribution after phase connection of the object (hand) which is obtained by applying the two types of phase difference results shown in FIGS. 16(c) and 16(d) to an algorithm of phase connection using the existing grating with a plurality of pitches.

(75) The height distribution of the object can be finally obtained by multiplying FIG. 16(e) by the height factor obtained by calibration.

(76) FIG. 17 shows results of measurement of the height of the hand which are obtained by the traditional phase shifting method and the sampling moir method and the 2nd-order moir method according to the present invention.

(77) FIG. 17(a) shows results of measurement performed by the traditional phase shifting method, and shows data regarding the shape of the hand which is obtained using a total of 17 images of eight images obtained by shifting the phase of the grating image having the grating pitch ratio of 8 by 2 /8 and nine images obtained by shifting the phase of the grating image having the grating pitch ratio of 9 by 2/9.

(78) The left side shows shape distributions, and the right side shows data regarding one horizontal line.

(79) FIG. 17(b) shows results of measurement performed by the sampling moir method. A dynamic shape can be measured using only a total of two images of an image having the grating pitch ratio of 8 and an image having the grating pitch ratio of 9, but there is a problem in that the measurement results have much noise.

(80) FIG. 17(c) shows results obtained by the 2nd-order moir method according to the present invention.

(81) This example shows analysis results in a condition in which the signal-noise (SN) ratio of the projected grating pattern is extremely low.

(82) Therefore, the phase shifting method (FIG. 17(a)) and the spatiotemporal phase shifting method require a large number of phase-shifted grating images, and thus it is difficult to apply the methods to dynamic measurement.

(83) On the other hand, the sampling moir method (FIG. 17(b)) is suitable for dynamic measurement. However, measurement results have much noise, and there are many locations where the shape cannot be measured well.

(84) According to the present invention (FIG. 17(c)), it can be confirmed that it is possible to calculate the shape of the hand with less measurement errors using only a total of two images of an image having the grating pitch ratio of 8 and an image having the grating pitch ratio of 9.

(85) As can be seen from the experiment results of FIG. 17, the present invention is effective for high-accuracy and dynamic three-dimensional shape measurement.

(86) However, the present technique using the 2nd-order moir method uses intensity information in a wide field of view, and thus it is necessary to note that the present technique is not suitable for measurement of a rapidly changing object.

Example 4

(87) Simulation Verification of Proposed Technique in Uniform Deformation Measurement

(88) Example 4 is an example in which accuracy is verified in the case in which the proposed 2nd-order moir method is performed through simulation and uniform deformation measurement is performed.

(89) A regular grating having a 10-pixel pitch can be linearly extended and the current strain is 100 (FIG. 18(a)). A grating image of the regular grating was 10,000 pixels long.

(90) The micron strain () mentioned here is a minus sixth power of 10 in the unit of the amount of strain.

(91) When the pitch of the reference grating was 9 pixels, well-known digital sampling moir fringes before and after deformation were obtained (FIG. 18(b)).

(92) Both intervals between the well-known moir fringes before and after deformation were approximately 90 pixels.

(93) Known moir fringes were treated as regular gratings and down-sampled by 88 pixels, and 2nd-order moir fringes before and after deformation were obtained in combination with the spatial phase shifting method (FIG. 18(c)).

(94) The phase of the 2nd-order moir fringes before and after deformation was calculated using the present invention (FIG. 18(d)).

(95) Next, changes in phase and displacements having the same distribution characteristics were measured (FIG. 18(e)).

(96) Finally, the strain distribution was determined, so that the mean strain of 101 was obtained.

(97) As compared with the preset strain 100, the measurement error was only 1, and the accuracy of this measurement method was confirmed.

Example 5

(98) Simulation Verification of Proposed Technique in Non-Uniform Deformation Measurement

(99) The present example is an example indicating that distribution characteristics (FIG. 19) of measurement of the non-uniform displacement and strain are the same as theoretical characteristics.

(100) A numerical value obtained from MATLAB function peaks (x, y) was added to the phase of a regular grating of 8.1 pixels per pitch (the left side in FIG. 19(a), before deformation) to deform the regular grating (the right side in FIG. 19(a), after deformation).

(101) A grating image was 256180 pixels.

(102) When the reference grating pitch was set to 7 pixels, known moir fringes before and after deformation appeared (FIG. 19(b)).

(103) An interval between the known moir fringes before and after deformation was 46 to 57 pixels.

(104) Phase-shifted 2nd-order moir fringes before and after deformation were obtained in combination with the spatial phase shifting method by down-sampling the known moir fringes by 41 pixels (FIG. 19(c)).

(105) The phase of the moir fringes before and after deformation was obtained using the present invention (FIG. 19(d)).

(106) Changes in the phases (FIG. 19(e)) and displacement and strain distributions were measured.

(107) Distribution characteristics of the measured displacement and strain (FIG. 19(f)) matched theoretically obtained results.

Example 6

(108) Simulation Verification of Proposed Technique of Inclined Grating with Random Noise

(109) In the present example, measurement of the displacement of an inclined grating with random noise will be described.

(110) The angle of inclination of the grating line was 0.0086 rad, the grating pitch in the vertical direction was 2.0572 pixels, and the grating image was 1300900 pixels.

(111) A random noise with amplitude of 10% of the grating amplitude was added to the inclined grating, and images in the cases in which tensile strains equivalent to 50, 100, 500, 1000 and 2000 were loaded were generated through simulation.

(112) The process of generating moir fringes and 2nd-order moir fringes in the cases in which strains are 0 and 1000 is shown in FIG. 20. It is possible to measure displacement and strain from the phase difference between 2nd-order moir fringes.

(113) Errors of the mean strain measured by setting strains given through simulation to the horizontal axis are shown in FIG. 21(a).

(114) FIG. 21(b) shows the standard deviation of the measured strain.

(115) Since all of relative errors are less than 1% and standard deviations are less than 410.sup.5, it can be confirmed that high-accuracy strain measurement can be performed.

Example 7

(116) Experimental Verification of Deformation of Loaded Aluminum

(117) In the present example, validity and accuracy of the proposed deformation measurement are shown through experiment.

(118) An aluminum specimen was stretched under a laser microscope (FIG. 22).

(119) Dimensions of the aluminum specimen were 27, 6.3, and 0.5 (mm) in length, width, and thickness, respectively.

(120) A grating having a pitch of 3.0 m was manufactured on the surface of the aluminum specimen by UV nanoimprint lithography.

(121) After a strain gauge is attached to the side opposite to the specimen, a tensile load was applied under the microscope.

(122) Laser scanning moir fringes generated due to interference between the grating on the specimen and the scanning line was observed by making the magnification of the objective lens as 5 times.

(123) A series of laser scanning moir fringes were recorded during the tension process.

(124) FIG. 23 shows the deformation measurement process for the aluminum specimen under the tensile load by taking the case in which the value of the strain gauge is 3000 as an example.

(125) FIG. 24 shows stress-strain curves of the aluminum specimen obtained using the mean strain evaluated from the proposed technique and the strain evaluated from the strain gauge.

(126) The value of the mean strain evaluated from the proposed technique is well matched with that of the strain gauge method, and thus it is possible to verify the validity and measurement accuracy of the proposed technique.

INDUSTRIAL APPLICABILITY

(127) The proposed 2nd-order moir method and its program can be used to measure full-field displacements and strains of various materials and structure from nanoscale to meter scale.

(128) Analyzable objects include metals, polymers, ceramics, semiconductors, composite materials, hybrid structures, and thin films in industrial fields such as aerospace, automobiles, electronic packaging, biomedical fields, military, and material manufacture.

(129) Typical applications in the industrial field mainly include the following four aspects.

(130) 1) Evaluation of Separation, Crack Propagation, and Instability Mode

(131) The proposed technique can predict the region where a crack occurs by finding the strain concentration region from full-field deformation.

(132) The form of crack propagation can be evaluated by measuring the deformation distribution near the tip of the crack.

(133) It is possible to quantitatively analyze modes of damages including an instability mode, layer separation, buckling, and a crack due to various mechanical loads, electric loads, thermal loads, magnetic loads, and combined loads thereof.

(134) 2) Evaluation of Residual Strain and Stress Distribution, and Provision of Optimized Design Guidelines

(135) It is possible to detect the residual strain inside a material and the influence on its structural stability in combination with a stress release method (heating, a hole drilling method, a ring core method, or the like).

(136) Measurement of the deformation distribution in the vicinity of an interface helps to find areas that are easily destroyed and provides guidance for strengthening and toughening materials and optimal design of materials and interfaces.

(137) 3) Evaluation of Mechanical Properties and Monitoring of Structural Health

(138) Deformation measurement information can be used to determine material constants such as a stress-strain curve, a Young's modulus, a Poisson's ratio, an elastic limit, a yield strength, and an ultimate strength.

(139) It is possible to evaluate displacement and strain distributions caused by a mechanical, electrical, or thermal load for structural health monitoring.

(140) 4) Evaluation of Height, Depth, and Flatness and Control of Manufacturing Quality

(141) It is possible to quantitatively evaluate the height, depth, surface flatness, and the out-of-plane displacement of an object from the measured three-dimensional shape. The proposed technique is useful for industrial fields such as control of manufacturing quality, machine vision, and automatic processing.

REFERENCE SIGNS LIST

(142) 1 Computer 2 Monitor 3 Imaging sensor (various cameras and microscopes) 4 Grating projection device 5 Three-dimensional object as measurement target 6 Reference surface (planar object) 7 Structural material as measurement target 8 Minute grating (one-dimensional or two-dimensional) 11 Grating projection control unit 12 Fringe grating image storage unit 13 2nd-order moir fringe generation unit and phase analysis arithmetic operation unit 14 Three-dimensional shape arithmetic operation unit 15 Displacement and strain arithmetic operation unit