Three-dimensional-shape measurement device, three-dimensional-shape measurement method, and program

Abstract

A control device acquires a plurality of observation signals observed using a plurality of projection patterns having different spatial frequencies, as observation signals for a measurement point on a measurement object. The control device repeatedly executes processing for estimating two component signals included in each observation signal, so as to separate the observation signal into two component signals, and calculate a three-dimensional position of the measurement point based on the phases of the separated component signals.

Claims

1. A three-dimensional shape measurement device, comprising: a projection device; an imaging device; and a control device configured to measure a three-dimensional shape of a measurement object based on an observation signal observed by the imaging device, in a state of projecting a projection pattern, which has cyclicity in a time direction and a space direction, from the projection device to the measurement object, wherein the observation signal comprises a composite signal of a first component signal and a second component signal, the first component signal and the second component signal having phases that are different from each other, wherein the control device is configured to: acquire a plurality of observation signals observed using a plurality of projection patterns having mutually different spatial frequencies as the observation signals with respect to a measurement point on the measurement object; set an initial value of an estimation value of the first component signal for each observation signal; estimate the first component signal and the second component signal included in each observation signal by repeatedly executing operations comprising: determining a corrected estimation value of the first component signal of each observation signal by applying first correction processing, which reduces errors of amplitude and a phase, to the estimation value of the first component signal of each observation signal; determining an estimation value of the second component signal of each observation signal by subtracting the corrected estimation value of the first component signal from each observation signal; determining a corrected estimation value of the second component signal of each observation signal by applying second correction processing, which reduces errors of amplitude and a phase, to the estimation value of the second component signal of each observation signal; and determining an estimation value of the first component signal of each observation signal by subtracting the corrected estimation value of the second component signal from each observation signal; and calculate a three-dimensional position of the measurement point based on a phase of the estimated first component signal or second component signal.

2. The three-dimensional shape measurement device according to claim 1, wherein one of the first component signal and the second component signal comprises a direct component signal, which corresponds to light which is projected from the projection device, and is reflected at the measurement point and enters the imaging device, and the other signal of the first component signal and the second component signal comprises an indirect component signal, which corresponds to light which is projected from the projection device, is reflected by another reflection surface, and is reflected at the measurement point and enters the imaging device.

3. The three-dimensional shape measurement device according to claim 1, wherein the first correction processing comprises: acquiring a plurality of amplitude values and a plurality of phase values for each spatial frequency of the plurality of projection patterns by calculating the amplitude and the phase from the estimation value of the first component signal of each observation signal, determining a corrected amplitude value such that the corrected amplitude value is greater than a minimum value and smaller than a maximum value of the plurality of amplitude values, normalizing the plurality of phase values and determining a corrected phase value such that the corrected phase value is greater than a minimum value and smaller than a maximum value of the plurality of normalized phase values, and reproducing a brightness signal for each spatial frequency of the plurality of projection patterns based on the corrected amplitude value and the corrected phase value, wherein the reproduced brightness signal for each spatial frequency comprises the corrected estimation value of the first component signal of each observation signal.

4. The three-dimensional shape measurement device according to claim 1, wherein the second correction processing comprises: acquiring a plurality of amplitude values and a plurality of phase values for each spatial frequency of the plurality of projection patterns by calculating the amplitude and the phase from the estimation value of the second component signal of each observation signal, determining a corrected amplitude value such that the corrected amplitude value is greater than a minimum value and smaller than a maximum value of the plurality of amplitude values, normalizing the plurality of phase values and determining a corrected phase value such that the corrected phase value is greater than a minimum value and smaller than a maximum value of the plurality of normalized phase values, and reproducing a brightness signal for each spatial frequency of the plurality of projection patterns based on the corrected amplitude value and the corrected phase value, wherein the reproduced brightness signal for each spatial frequency comprises the corrected estimation value of the second component signal of each observation signal.

5. The three-dimensional shape measurement device according to claim 3, wherein the corrected amplitude value comprises a mean value or a median value of the plurality of amplitude values.

6. The three-dimensional shape measurement device according to claim 3, wherein the corrected phase value comprises a mean value or a median value of the plurality of normalized phase values.

7. The three-dimensional shape measurement device according to claim 3, wherein the processing for normalizing the plurality of phase values comprises processing for converting the plurality of phase values into a phase value with respect to a common spatial frequency.

8. The three-dimensional shape measurement device according to claim 1, wherein the control device is configured to calculate the three-dimensional position of the measurement point by the phase shift method or micro phase shifting based on the phase of the estimated first component signal or second component signal.

9. A three-dimensional shape measurement method for measuring a three-dimensional shape of a measurement object based on an observation signal observed by an imaging device, in a state of projecting a projection pattern, which has cyclicity in a time direction and a space direction, from a projection device to the measurement object, wherein the observation signal comprises a composite signal of a first component signal and a second component signal, the first component signal and the second component signal having phases that are different from each other, the three-dimensional shape measurement method comprising: acquiring a plurality of observation signals observed using a plurality of projection patterns having mutually different spatial frequencies as the observation signals with respect to a measurement point on the measurement object; setting an initial value of an estimation value of the first component signal for each observation signal; estimating the first component signal and the second component signal included in each observation signal by repeatedly executing operations comprising: determining a corrected estimation value of the first component signal of each observation signal by applying first correction processing, which reduces errors of amplitude and a phase, to the estimation value of the first component signal of each observation signal; determining an estimation value of the second component signal of each observation signal by subtracting the corrected estimation value of the first component signal from each observation signal; determining a corrected estimation value of the second component signal of each observation signal by applying second correction processing, which reduces errors of amplitude and a phase, to the estimation value of the second component signal of each observation signal; and determining an estimation value of the first component signal of each observation signal by subtracting the corrected estimation value of the second component signal from each observation signal; and calculating a three-dimensional position of the measurement point based on a phase of the estimated first component signal or second component signal.

10. A non-transitory computer-readable storage medium storing a program that causes a computer to execute each step of the three-dimensional shape measurement method according to claim 9.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) FIG. 1 is a schematic diagram depicting a hardware configuration of a three-dimensional shape measurement device.

(2) FIG. 2 is a block diagram depicting functions related to the three-dimensional shape measurement.

(3) FIG. 3A and FIG. 3B are schematic diagrams depicting models of inter-reflection and an observation signal.

(4) FIG. 4 is a flow chart depicting signal separation processing.

(5) FIG. 5 is a flow chart depicting the three-dimensional shape measurement processing.

(6) FIG. 6 is a diagram depicting the inter-reflection.

DESCRIPTION OF EMBODIMENTS

(7) The present invention relates to a three-dimensional shape measurement technique to measure a three-dimensional shape of a measurement object based on the change of the phase of a pattern projected to the measurement object, and more particularly to a method of reducing the influence of inter-reflection caused by a reflection surface that exists around the measurement object, and improving the measurement accuracy. The present invention can be suitably applied to a three-dimensional shape measurement using a phase shift method or micro phase shifting (MPS) which is an improved phase shift method, and to a three-dimensional shape measurement using a time of flight (TOF) type phase shift method. The three-dimensional shape measurement according to the present invention can be applied to various apparatuses, such as a three-dimensional scanner which measures the surface shape of an object, an inspection apparatus which inspects an object based on a measured three-dimensional shape, and an apparatus which performs object recognition or individual identification based on a measured three-dimensional shape. For example, in the case of a visual inspection apparatus for a surface mount substrate, inter-reflection caused by the metal components on the substrate may drop inspection accuracy, therefore the three-dimensional shape measurement according to the present invention can be especially useful if applied.

(8) An example of a preferred embodiment of the present invention will be described with reference to the drawings. However the configurations and operations of the apparatuses described in the following embodiment are examples, and are not intended to limit the scope of the invention only to these configurations and operations.

(9) (Configuration of Three-Dimensional Shape Measurement Device)

(10) A general configuration of a three-dimensional shape measurement device according to this embodiment will be described with reference to FIG. 1. FIG. 1 is a schematic diagram depicting a hardware configuration of the three-dimensional shape measurement device. The three-dimensional shape measurement device 1 is an apparatus to measure a three-dimensional shape of a measurement object O, and is constituted of a projection device (projector) 10, an imaging device (camera) 11, and a control device (computer) 12.

(11) The projection device 10 is a projection unit that projects a pattern which has a cyclicity in the time direction and the space direction to the measurement object O. A number of projection devices 10 may be one, but a plurality of projection devices 10 may be disposed so that the pattern can be projected to the measurement object O in a plurality of directions. For the projection device 10, a digital light processing (DLP) projector using a digital mirror device can be preferably used. This is because a DLP projector can easily change the projection patterns.

(12) The imaging device 11 is a unit that captures an image of a measurement object O to which a pattern is projected, and outputs a digital image. The imaging device 11 is constituted of an optical system and an image sensor, for example. When the three-dimensional shape measurement is performed, the imaging device 11 captures a plurality of images while changing the phase of the pattern projected from the projection device 10. In this description, a brightness of the measurement object O that is observed by the imaging device 11 is called an “observation signal”, and an image captured by the imaging device 11 is called an “observation image”. In other words, a value of each pixel of an observation image indicates an observation signal (brightness value) at a corresponding measurement point on the measurement object. In this embodiment, a plurality of observation images are acquired while changing the spatial frequency of the projection pattern.

(13) The control device 12 is a unit that has such functions as controlling the projection device 10 and the imaging device 11, processing an image captured by the imaging device 11, and measuring a three-dimensional shape. The control device 12 may be a computer which includes a CPU (processor), a memory, a non-volatile storage device (e.g. hard disk, flash memory), an input device (e.g. keyboard, mouse, touch panel), and a display device (e.g. liquid crystal display). Each function of the control device 12, which will be described later, can be implemented by loading a program stored in the non-volatile storage device and the CPU executing this program. However all or a part of the functions of the control device 12 may be performed by an ASIC, an FPGA or dedicated hardware. Further, the functions of the control device 12 may be implemented by the collaboration of a plurality of computers, utilizing techniques of distributed computing and cloud computing.

(14) FIG. 2 is a block diagram depicting the functions of the control device 12 related to the three-dimensional shape measurement. For the functions related to the three-dimensional shape measurement, the control device 12 includes an image acquisition unit 20, a signal separation unit 21, a phase calculation unit 22, and a three-dimensional restoration unit 23.

(15) The image acquisition unit 20 has a function to acquire a plurality of observation images used for the three-dimensional shape measurement from the imaging device 11. The signal separation unit 21 has a function to separate the observation signal into two component signals for each pixel of the observation image. The phase calculation unit 22 has a function to calculate the phase values of the separated component signals. The three-dimensional restoration unit 23 has a function to calculate a three-dimensional position of each pixel (that is, each measurement point on the surface of the measurement object O) on the observation image based on the phase values calculated by the phase calculation unit 22. These functions will be described later.

(16) (Model of Inter-Reflection and Observation Signal)

(17) FIG. 3A and FIG. 3B illustrate the models of inter-reflection and an observation signal.

(18) FIG. 3A is an example when a reflection surface 30 having a high specularity (e.g. surface of a metal object) exists near a measurement object O having a diffusion reflection surface. FIG. 3B is an example when a diffusion reflection surface 31 exists near a measurement object O reflection surface of which has high specularity.

(19) In the case where a projection image 100 having a sinusoidal fringe pattern is projected from the projection device 10, it is assumed that light L.sub.D corresponding to the position X.sub.D on the projection image 100 (the light L.sub.D is called direct light), and the reflection light L.sub.R of the light corresponding to the position x.sub.R on the projection image 100, reflected by the reflection surface 30 or 31 (this light L.sub.R is called indirect light) reach the measurement point P on the measurement object O.

(20) In this case, the observation signal I observed by the imaging device 11 becomes a composite signal of the light I.sub.D (direct component signal), which is generated when the direct light L.sub.D was reflected at the measurement point P and entered the imaging device 11, and the light I.sub.R (indirect component signal), which is generated when the indirect light L.sub.R was reflected at the measurement point P, and entered the imaging device 11. The desired signal which is necessary to the three-dimensional shape measurement is the direct component signal I.sub.D. In the state illustrated in FIG. 3A, the indirect component signal I.sub.R attenuates during the reflection on the reflection surface 30, hence the intensity of the direct component signal I.sub.D becomes higher than the intensity of the indirect component signal I.sub.R. In the state illustrated in FIG. 3B, on the other hand, the light L.sub.D is mostly reflected on the specular reflection surface in the positive reflection direction, hence the intensity of the indirect component signal I.sub.R may become higher than the intensity of the direct component signal I.sub.D.

(21) It is assumed that the spatial frequency of the projection pattern is w, the frequency in the time direction (time frequency) is v, the amplitude of the direct component signal I.sub.D is A.sub.D, and the amplitude of the indirect component signal I.sub.R is A.sub.R. In this case, an observation signal I (ω, t) at time t can be expressed as an addition of a direct component signal I.sub.D (ω, t) and an indirect component signal I.sub.R (ω, t) as follows.
I(ω,t)=I.sub.D(ω,t)+I.sub.R(ω,t)
I.sub.D(ω,t)=A.sub.D cos(ωX.sub.Dvt)+C.sub.D
I.sub.R(ω,t)=A.sub.R cos(ωx.sub.R+vt)+C.sub.R  [Math. 1]
If the phase φ′ (ω, Δx) and the amplitude A′ (ω,Δx) are calculated regarding the observation signal as a single component signal, then the following is estimated (where Δx=x.sub.D−x.sub.R).

(22) $\begin{array}{cc}{A}^{\mathrm{\prime 2}}\left(\omega ,\Delta x\right)=A_D^2+A_R^2+2A_D{A}_R\cos 2\omega \Delta x{\varphi }^{\prime }\left(\omega ,\Delta x\right)=\omega x_D+{\tan }^{-1}\frac{A_R\sin 2\omega \Delta x}{A_D+A_R\cos 2\omega \Delta x}& \left[\mathrm{Math}.2\right]\end{array}$

(23) The desired signal is the direct component signal I.sub.D, hence the true value of the amplitude is A.sub.D and the true value of the phase is ωx.sub.D. However an error is generated in both the amplitude and the phase because of the influence of inter-reflection (indirect component signal I.sub.R). As the above expression indicates, this error depends on the spatial frequency ω of the projection pattern and the difference Δx.

(24) (Signal Separation Method)

(25) A method of separating the direct component signal (desired signal) from the observation signal will be described next.

(26) In this embodiment, it is assumed that the observation signal is a composite signal of two types of component signals phases of which are mutually different. Out of the two types of component signals, a signal amplitude of which is larger and the signal intensity is higher is defined as a primary component signal (subscript “P” is attached), and the other signal is defined as a secondary component signal (subscript “s” is attached). In most cases, the primary component signal corresponds to the direct component signal, and the secondary component signal corresponds to the indirect component signal, therefore in this embodiment, the primary component signal, separated from the observation signal, is used for the three-dimensional shape measurement. There may be a case where the signal intensity of the direct component signal becomes lower, as illustrated in FIG. 3B, but in this case, the secondary component signal may be used for the three-dimensional shape measurement.

(27) The signal separation method of this embodiment utilizes a property in which errors of the amplitude and phase generated by the influence of inter-reflection depend on the spatial frequency of the projection pattern. Therefore for input, a plurality of observation signals, which were observed using a plurality of projection patterns having mutually different spatial frequencies, are used. When the spatial frequency of the projection pattern is ω.sub.k (k=1, . . . , K) and the time frequency is v.sub.k, then the observation signal can be expressed as follows.
I.sub.o(ω.sub.kt)=A.sub.D(ω.sub.k)cos(ω.sub.kx.sub.D+v.sub.kt)+A.sub.R(ω.sub.k)cos(ω.sub.kx.sub.R+v.sub.kt)+C(ω.sub.k)  [Math. 3]

(28) For preprocessing, an offset component (DC component) C (ω.sub.k) is removed from the observation signal I.sub.o (ω.sub.k, t). For the repeated operation in the subsequent steps, the observation signal I (ω.sub.k, t) determined by removing the offset component as follows is used.
I(ω.sub.k,t)≙I.sub.o(ω.sub.k,t)−C(ω.sub.k)  [Math. 4]

(29) The value of the offset component C (ω.sub.k) in the spatial frequency ω.sub.k is calculated as follows, using the sum of the observation signals in one cycle in the time direction, for example.

(30) $\begin{array}{cc}C\left({\omega }_k\right)=\frac{1}{T}\underset{t=0}{\overset{T-1}{.Math.}}{I}_O\left({\omega }_k,t\right)& \left[\mathrm{Math}.5\right]\end{array}$

(31) Instead of estimating the value of the offset component for each spatial frequency, a value of the offset component estimated using another observation signal having a similar spatial frequency may be used as a substitute of the value of the offset component for each spatial frequency. If this substitute of the offset is not used, at least three observation images having mutually different phases are necessary for each spatial frequency ω.sub.k, to implement desired processing. If this substitute of the offset is used, on the other hand, desired processing can be implemented if at least two observation images having mutually different phases are acquired for each spatial frequency. Therefore by using this substitute of the offset, a necessary number of times of projections and number of times of image capturing can be reduced, whereby the three-dimensional shape measurement can be performed faster.

(32) FIG. 4 is a flow chart depicting the signal separation processing that is executed by the signal separation unit 21. The processing in FIG. 4 is executed for each pixel of the observation image. Hereafter a processing target pixel is called a “pixel of interest”.

(33) In step S40, the signal separation unit 21 acquires a set I of the observation signals I (ω.sub.k, t) of the pixels of interest (in FIG. 4, a set is denoted in bold). An observation signal set I is constituted of a plurality of observation signals I (ω.sub.1, t), . . . , I (ω.sub.K, t) corresponding to a plurality of spatial frequencies ω.sub.1, . . . , ω.sub.K. The observation signal of each spatial frequency includes a plurality of observation signals I (ω.sub.k, 0), I (ω.sub.k, 1), . . . , I (ω.sub.k, T.sub.k−1), which are observed at a plurality of times (T=0, 1, . . . , T.sub.k−1), having mutually different phases. The observation signal I (ω.sub.k, t) is a signal after the offset component is removed, and T.sub.k indicates a time cycle of the projection pattern.

(34) The observation signal I (ω.sub.k, t) is a composite signal of a primary component signal I.sub.P (ω.sub.k, t) and a secondary component signal I.sub.S ((ω.sub.k, t), but the ratio of these component signals (intensity ratio) is unknown. Therefore in this embodiment, the signal separation unit 21 sequentially updates the estimated value (approximate value) of each component signal by repeatedly applying the single component estimation method having low separation performance to each component signal, so as to estimate a true value of each component signal.

(35) First in step S41, the signal separation unit 21 sets the initial values of the primary component signal I.sub.P (ω.sub.k, t) and the secondary component signal I.sub.S (ω.sub.k, t) for the observation signal I (ω.sub.k, t) of each spatial frequency ω.sub.k. In this embodiment, the observation signal I (ω.sub.k, t) is set as the initial value of the primary component signal I.sub.P (ω.sub.k, t), and the initial value of the secondary component signal I.sub.S (ω.sub.k, t) is set to 0. In FIG. 4, the bold symbol I.sub.P indicates the primary component signal set, and the bold symbol I.sub.S indicates the secondary component signal set.

(36) In step S42, the signal separation unit 21 calculates the amplitude value A.sub.P (ω.sub.k) and the phase value φ.sub.P (ω.sub.k) from the primary component signal I.sub.P (ω.sub.k, t) for each spatial frequency ω.sub.k. Generally when signals having the same spatial frequency ω.sub.k are observed with equal time intervals for one cycle, the signal at each time t is given by the following expression using the time frequency v.sub.k.
I.sub.P(ω.sub.k,t)=A(ω.sub.k)cos(ω.sub.kx+v.sub.kt),t=0,1, . . . T.sub.k−1  [Math. 6]

(37) These signals I.sub.P (ω.sub.k, 0), I.sub.P (ω.sub.k, 1), . . . , I.sub.P (ω.sub.k, T.sub.k−1) are frequency-converted, then the component J (ω.sub.k, v.sub.k) of time frequency v.sub.k, which is a first harmonic, becomes as follows, (where j is an imaginary unit).
J(ω.sub.k,v.sub.k)=A(ω.sub.k)e.sup.jω.sup.k.sup.x  [Math. 7]
Therefore the amplitude value A.sub.P (ω.sub.k) and the phase value φ.sub.P (ω.sub.k), with respect to the spatial frequency ω.sub.k, can be calculated as follows.

(38) $\begin{array}{cc}{A}_P\left({\omega }_k\right)=\sqrt{{\left(\mathrm{Re}\left(J\left({\omega }_k,v_k\right)\right)\right)}^2+{\left(\mathrm{Im}\left(J\left({\omega }_k,v_k\right)\right)\right)}^2}{\varphi }_P\left({\omega }_k\right)={\omega }_{k}x={\tan }^{-1}\frac{\mathrm{Im}\left(J\left({\omega }_k,v_k\right)\right)}{\mathrm{Re}\left(J\left({\omega }_k,v_k\right)\right)}& \left[\mathrm{Math}.8\right]\end{array}$
Here Re ( ) and Im ( ) are operator symbols which indicate a real part and imaginary part of a complex number respectively.

(39) The calculated phase value φ.sub.P (ω.sub.k) is normalized for the processing in subsequent stages. For example, using the phase unwrapping, the signal separation unit 21 converts a plurality of phase values φ.sub.P (ω.sub.1), φ.sub.P (ω.sub.2), . . . , φ.sub.P (ω.sub.K) at different spatial frequencies ω.sub.k into phase values θ.sub.P (ω.sub.1), θ.sub.P (ω.sub.2), . . . , θ.sub.P (ω.sub.K) at a common spatial frequency ω.sub.L. For example, if φ.sub.s is a phase value in a projection pattern at the spatial frequency ω.sub.k, if φ.sub.L is a phase value in a projection pattern at the spatial frequency ω.sub.L for unwrapping, and if α is an unwrapping scale factor, then the phase value θ after the conversion can be calculated as follows.

(40) $\begin{array}{cc}\theta =\frac{{\varphi }_S}{\alpha }+\left({\varphi }_L-\frac{{\varphi }_S}{\alpha }\right)\mathrm{mod}\frac{2\pi }{\alpha }& \left[\mathrm{Math}.9\right]\end{array}$

(41) The outputs of the processing in step S42 are the amplitude value set A.sub.P of each spatial frequency ω.sub.k, and the normalized phase value set θ.sub.P.
A.sub.P={A.sub.P(ω.sub.1),A.sub.P(ω.sub.2), . . . ,A.sub.P(ω.sub.k)}
θ.sub.P={θ.sub.P(ω.sub.1),θ.sub.P(ω.sub.2), . . . ,θ.sub.P(ω.sub.k)}  [Math. 10]

(42) In the processing in step S42, the amplitude value and the phase value are calculated regarding the primary component signal I.sub.P (ω.sub.k, t) as a single component signal. However, in a stage where the primary component signal I.sub.P (ω.sub.k, t) is not yet sufficiently separated, errors of the amplitude value and the phase value calculated in step S42 with respect to the true values are large. Therefore in step S43, the signal separation unit 21 estimates an amplitude value and a phase value which are closer to the true values (that is, smaller errors) based on the amplitude value set A.sub.P and the phase value set θ.sub.P determined in step S42. The estimation method here need not be a method that completely eliminates errors, but may be a method that can reduce errors, even slightly (this is called a “weak estimation”). By repeatedly applying this weak estimation, the amplitude value and the phase value gradually become closer to the true values, thereby a practical separation accuracy can be acquired.

(43) The specific method of weak estimation does not matter. The magnitude of the error of each amplitude value and the positive/negative of the error differ depending on the spatial frequency ω.sub.k, hence a plurality of amplitude values A.sub.P (ω.sub.k) calculated in step S42 are more likely to be distributed on the positive side and negative side of the true value respectively. In other words, the true value of the amplitude value is more likely to exist between the minimum value and the maximum value of the plurality of amplitude values A.sub.P (ω.sub.k). Therefore an amplitude value close to the true value is expected to be acquired by determining the amplitude value, so as to be a value that is greater than the minimum value and smaller than the maximum value of the plurality of amplitude values A.sub.P (ω.sub.k). For example, a mean value or a median value of the plurality of amplitude values A.sub.P (ω.sub.k) is determined. This is also applicable to the phase value θp (ω.sub.k).

(44) In this embodiment, “averaging” is used as a method for the weak estimation. In concrete terms, in step S43, the signal separation unit 21 calculates the corrected amplitude value (A.sub.P hat) and the corrected phase value (θ.sub.P hat) which are defined as follows.

(45) $\begin{array}{cc}{\hat{A}}_P\stackrel{\Delta }{=}\frac{1}{K}\underset{k=1}{\overset{K}{.Math.}}{A}_P\left({\omega }_k\right){\hat{\theta }}_P\stackrel{\Delta }{=}\frac{1}{K}\underset{k=1}{\overset{K}{.Math.}}{\theta }_P\left({\omega }_k\right)& \left[\mathrm{Math}.11\right]\end{array}$

(46) If the damping ratio of the amplitude is different depending on the spatial frequency ω.sub.k, the corrected amplitude value may be calculated as follows.

(47) $\begin{array}{cc}{\hat{A}}_P\stackrel{\Delta }{=}\frac{1}{K}\underset{k=1}{\overset{K}{.Math.}}\frac{1}{r_k}{A}_P\left({\omega }_k\right)& \left[\mathrm{Math}.12\right]\end{array}$
Here r.sub.k is a damping ratio of the amplitude of the projection pattern at the spatial frequency ω.sub.k.

(48) In step S44, the signal separation unit 21 reproduces the brightness signal with respect to each spatial frequency ω.sub.k based on the corrected amplitude value and the corrected phase value determined in step S43. If the damping ratio of the amplitude can be regarded as constant, regardless what the spatial frequency ω.sub.k is, then the brightness signal can be calculated as follows.

(49) $\begin{array}{cc}{I}_P^{\prime }\left({\omega }_k,t\right)={\hat{A}}_P\cos \left(\frac{{\omega }_k}{{\omega }_L}{\hat{\theta }}_P+v_{k}t\right)& \left[\mathrm{Math}.13\right]\end{array}$
If the damping ratio of the amplitude is different depending on the spatial frequency ω.sub.k, on the other hand, the brightness signal can be calculated as follows.

(50) $\begin{array}{cc}{I}_P^{\prime }\left({\omega }_k,t\right)=r_k{\hat{A}}_P\cos \left(\frac{{\omega }_k}{{\omega }_L}{\hat{\theta }}_P+v_{k}t\right)& \left[\mathrm{Math}.14\right]\end{array}$

(51) The brightness signal I.sub.P′ (ω.sub.k, t) reproduced like this is a signal errors of which of the amplitude and the phase are reduced compared with the original primary component signal I.sub.P (ω.sub.k, t) used in step S42. The above processing in steps S42 to S44 is an example of the “first step” according to the present invention, the original brightness signal I.sub.P (ω.sub.k, t) is an example of the “estimation value of the first component signal” according to the present invention, and the reproduced brightness signal I.sub.P′ (ω.sub.k, t) is an example of the “corrected estimation value of the first component signal” according to the present invention.

(52) In step S45, the signal separation unit 21 subtracts the brightness signal I.sub.P′ (ω.sub.k, t) determined in step S44 from the observation signal I (ω.sub.k, t) at each spatial frequency ω.sub.k. Thereby the estimation value of the secondary component signal I.sub.S (ω.sub.k, t) can be updated. Compared with the original primary component signal I.sub.P (ω.sub.k, t), the error of the brightness signal I.sub.P′ (ω.sub.k, t) is smaller, hence the error of the secondary component signal I.sub.S (ω.sub.k, t) can also be reduced by this update processing. This processing in step S45 is an example of the “second step” according to the present invention, and the brightness signal I.sub.S (ω.sub.k, t) determined in step S45 is an example of the “estimation value of the second component signal” according to the present invention.

(53) Then in step S46, the signal separation unit 21 calculates the amplitude value A.sub.S (ω.sub.k) and the phase value θ.sub.S (ω.sub.k) from the secondary component signal I.sub.S (ω.sub.k, t) at each spatial frequency ω.sub.k. A specific processing content thereof is the same as step S42 (the subject here, however, is the secondary component signal instead of the primary component signal), hence a detailed description is omitted. As the output of the processing in step S46, the amplitude value set A.sub.S of the secondary component signal I.sub.S (ω.sub.k, t) of each spatial frequency ω.sub.k and normalized phase value set θ.sub.s are acquired.

(54) In step S47, the signal separation unit 21 calculates the corrected amplitude value (As hat) and the corrected phase value (θ.sub.S hat), which are closer to the true values, based on the amplitude value set A.sub.S and the phase value set θ.sub.S determined in step S46. A specific processing content thereof is the same as step S43 (the subject here, however, is the secondary component signal, instead of the primary component signal), hence a detailed description is omitted.

(55) In step S48, the signal separation unit 21 reproduces the brightness signal with respect to each spatial frequency ω.sub.k based on the corrected amplitude value and the corrected phase value determined in step S47. If the damping ratio of the amplitude can be regarded as constant, regardless what the spatial frequency ω.sub.k is, then the brightness signal can be calculated as follows.

(56) $\begin{array}{cc}{I}_S^{\prime }\left({\omega }_k,t\right)={\hat{A}}_S\cos \left(\frac{{\omega }_k}{{\omega }_L}{\hat{\theta }}_S+v_{k}t\right)& \left[\mathrm{Math}.15\right]\end{array}$
If the damping ratio of the amplitude is different depending on the spatial frequency ω.sub.k, on the other hand, the brightness signal can be calculated as follows.

(57) 0$\begin{array}{cc}{I}_S^{\prime }\left({\omega }_k,t\right)=r_k{\hat{A}}_S\cos \left(\frac{{\omega }_k}{{\omega }_L}{\hat{\theta }}_S+v_{k}t\right)& \left[\mathrm{Math}.16\right]\end{array}$

(58) The brightness signal I.sub.S′ (ω.sub.k, t) reproduced like this is a signal errors of which of the amplitude and the phase are reduced compared with the original secondary component signal I.sub.S (ω.sub.k, t) used in step S46. The above processing in step S46 to S48 is an example of the “third step” according to the present invention, the original brightness signal I.sub.S (ω.sub.k, t) is an example of the “estimation value of the second component signal” according to the present invention, and the reproduced brightness signal I.sub.S′ (ω.sub.k, t) is an example of the “corrected estimation value of the second component signal” according to the present invention.

(59) In step S49, the signal separation unit 21 subtracts the brightness signal I.sub.S′ (ω.sub.k, t) determined in step S48 from the observation signal I (ω.sub.k, t) at each spatial frequency ω.sub.k. Thereby the estimation value of the primary component signal I.sub.P (ω.sub.k, t) can be updated. Compared with the original secondary component signal I.sub.S (ω.sub.k, t), the error of the brightness signal I.sub.S′ (ω.sub.k, t) is smaller, hence the error of the primary component signal I.sub.P (ω.sub.k, t) can also be reduced by this update processing. This processing in step S49 is an example of the “fourth step” according to the present invention, and the brightness signal I.sub.P (ω.sub.k, t) determined in step S49 is an example of the “estimation value of the primary component signal” according to the present invention.

(60) By repeating the above mentioned operations in step S42 to S49 a plurality of times, the estimation value of each component signal is sequentially updated so that the errors of the amplitude and the phase of each component signal (differences from the true values) decrease. Therefore the observation signal I (ω.sub.k, t) can be accurately separated into the two component signals, I.sub.P (ω.sub.k, t) and I.sub.S (ω.sub.k, t). The number of times of repeat may be determined in advance, or may be dynamically changed by evaluating the degree of convergence of the estimation value of each component signal using a predetermined evaluation formula.

Three-Dimensional Shape Measurement

(61) An example of the processing flow of the three-dimensional shape measurement according to this embodiment will be described with reference to the flow chart in FIG. 5.

(62) In step S50, the control device 12 controls the projection device 10 and the imaging device 11, and captures a plurality of observation images. For example, the control device 12 projects a fringe pattern, wherein brightness of the fringe pattern changes sinusoidally, four times while changing the phase by π/2 at a time, and captures four observation images.

(63) In order to achieve high measurement accuracy (resolution), it is preferable that the spatial frequency ω of the fringe pattern is high frequency. However the measurement range depends on the cycle of the fringe pattern. Therefore, the measurement range decreases, which is disadvantageous, as the frequency of the fringe pattern becomes higher. Therefore the measurement range is expanded by converting the high frequency phase into a low frequency phase. This method is called “phase unwrapping”. By phase unwrapping, high measurement accuracy and a wide measurement range are both implemented.

(64) To perform the phase unwrapping, a plurality of observation images having mutually different spatial frequencies are acquired using a plurality of projection patterns having mutually different spatial frequencies. For example, in the case of phase unwrapping that is normally used, images of a projection pattern at high frequency for measurement and a projection pattern at low frequency for unwrapping are captured, and in the case of MPS, images of a plurality of types of projection patterns at high frequency are captured. The data of the observation image is loaded into the control device 12 by the image acquisition unit 20, and is stored by the memory or the non-volatile storage device.

(65) In step S51, the signal separation unit 21 performs the processing for separating the observation signal at each pixel of the observation image into two component signals. Specific processing of the signal separation is as described in FIG. 4. In this embodiment, the primary component signal is regarded as the direct component signal.

(66) In step S52, the phase calculation unit 22 calculates the phase value of the pixel of interest from the direct component signal separated in step S51. The phase value acquired in the calculation step of the signal separation in step S51 (e.g. corrected phase value (θ.sub.P hat) determined in step S43 in FIG. 4) may be used.

(67) Then in step S53, the three-dimensional restoration unit 23 calculates the three-dimensional position of each pixel of the observation image (that is, each measurement point on the surface of the measurement object) based on the phase value of each pixel calculated by the phase calculation unit 22. Thereby the three-dimensional shape of the measurement object surface can be restored. A method of converting the phase value into a three-dimensional position (height information) is, for example, determining a conversion formula between the phase and the height in advance by calibration, and determining the height information by inserting the phase value into the conversion formula.

(68) According to the above mentioned signal separation method of this embodiment, the direct component signal can be accurately separated from the observation signal. Then by using the phase of the separated direct component signal for calculating the three-dimensional position of the measurement point, highly accurate three-dimensional shape measurement, in which the influence of inter-reflection is reduced, can be implemented. The method of this embodiment can be used, even if the ratio of the direct component signal and the indirect component signal included in the observation signal is totally unknown, that is, even if special prior knowledge and preconditions are not available. For example, in an actual situation where this method is applied, conditions, such as the degree of influence of inter-reflection, the shape of the causative object of inter-reflection (shape and orientation of the reflection surface), reflection characteristics (e.g. reflectance), and the positional relationship between the causative object of inter-reflection and the measurement object, vary, but the method of this embodiment can be universally used under all conditions. This means that the method of this embodiment can be easily implemented in actual apparatuses.

(69) The above description of the embodiment merely exemplifies the present invention. The present invention is not limited to this embodiment, but may be modified in various ways within the scope of the technical idea thereof.

REFERENCE SIGNS LIST

(70) 1 Three-dimensional shape measurement device 10 Projection device 11 Imaging device 12 Control device 20 Image acquisition unit 21 Signal separation unit 22 Phase calculation unit 23 Three-dimensional restoration unit 30 Reflection surface 31 Reflection surface