Fringe projection for determining topography of a body

Abstract

A fringe projection method for determining the topography of a body (12) comprising the steps: projecting a series of sets of patterns (Ti) onto a surface (20) of the body (12), wherein each set has at least two patterns (Ti) and wherein each pattern (Ti) has S fringes; for each pattern (Ti), recording an image (24.i) of the surface (20) having the projected pattern, so that a sequence of recordings is formed; and calculating the topography from the images (24.i), wherein such patterns are projected in which each fringe has an intensity distribution perpendicular to the fringe longitudinal direction (L) and each intensity distribution can be expressed by a function (Q) which has a spatial phase position (). According to the invention, the phase position () changes as a function of a code (g.sub.(s)) of the ordinal number (s) of the fringe.

Claims

1. A fringe projection method for determining the topography of a body comprising: projecting a sequence of sets of patterns onto a surface of the body, wherein each of the sets of patterns has at least two projected patterns and wherein each of the at least two projected patterns has the same total number of fringes; for each pattern, recording an image of the surface having the projected pattern to form a sequence of recordings; and determining the topography of the body from the sequence of recordings, wherein each fringe has an intensity distribution perpendicular to the fringe longitudinal direction and each intensity distribution is expressed by a spatial phase position, wherein the spatial phase position is a function of an ordinal number corresponding to the fringe, and wherein the spatial phase position depends on a single position of a code word corresponding to the ordinal number.

2. The fringe projection method of claim 1, wherein: the expression of the spatial phase position is periodic; and the intensity distributions of patterns of a set have the same spatial frequency.

3. The fringe projection method of claim 1, wherein the phase positions for all patterns within each of the sets are different from each other.

4. The fringe projection method of claim 1, wherein a phase position in a s-th fringe of a k-th pattern within a m-th set contains a sum of a first term, which contains s, however does not contain k, and a second term, which contains k, however does not contain s or m, wherein: k is an index of a pattern within a set; s is an index of a fringe; and m is an index of a set.

5. The fringe projection method of claim 4, wherein the phase position for the s-th fringe of the k-th pattern within the m-th set contains term $\frac{360\mathrm{Sx}}{X}+k+g_{s,m},$ wherein: S is a total number of fringes (s=0,1, . . . , S1); x is a coordinate perpendicular to a longitudinal direction of a fringe; X is an upper interval limit of x values, (x{0, . . . ,X}); is a difference angle where mod 3600; g.sub.s,m is a m-th position of a Gray code of s; and is a phase shift parameter, where mod 3600.

6. The fringe projection method of claim 5, wherein the intensity distribution of the k-th pattern within the m-th set is expressed by: $Q_{k,s,m}\left(x\right))=a.Math.p\left[\frac{360S}{X}x+k+g_{s,m}\right]$ wherein: Q is an intensity; x is an x-coordinate of a pattern; p is a 360-periodic trigonometric function; a is a constant with a real value; b is a constant with a real value; and c is a constant with a real value.

7. The fringe projection method of claim 6, wherein a shift parameter is expressed by: $.Math..Math.\frac{360}{N-1},$ wherein N is a number of symbols of a code-alphabet.

8. A fringe projection device for determining the topography of a body, comprising: a projector for projecting a sequence of sets of patterns onto the surface of the body; a camera for recording an image of the surface having the projected pattern; and a computer, which is connected to the projector and the camera configured to: project a sequence of sets of patterns onto a surface of the body, wherein each of the sets of patterns has at least two projected patterns and wherein each of the at least two projected patterns has the same total number of fringes; for each of the sequence of sets of patterns, record an image of the surface having the projected patterns to form a sequence of recordings; and determine the topography of the body from the sequence of recordings, wherein each fringe has an intensity distribution perpendicular to the fringe longitudinal direction and each intensity distribution is expressed by a spatial phase position, wherein the spatial phase position is a function an ordinal number corresponding to the fringe, and wherein the spatial phase position depends on a single position of a code word corresponding to the ordinal number.

9. A non-transitory computer readable medium comprising instructions that if executed cause a controller to: project a sequence of sets of patterns onto a surface of a body, wherein each of the sets of patterns has at least two projected patterns and wherein each of the at least two projected patterns has the same total number of fringes; for each of the sequence of sets of patterns, record an image of the surface having the projected patterns to form a sequence of recordings; and determine the topography of the body from the sequence of recordings, wherein each fringe has an intensity distribution perpendicular to the fringe longitudinal direction and each intensity distribution is expressed by a spatial phase position, wherein the spatial phase position is a function an ordinal number corresponding to the fringe, and wherein the spatial phase position depends on a single position of a code word corresponding to the ordinal number.

Description

(1) The invention is explained in the following by means of attached drawings. In particular

(2) FIG. 1 shows a schematic view of an inventive fringe projection device for performing a method according to the invention,

(3) FIG. 2 shows a diagram explaining an embodiment of a method according to the invention,

(4) FIG. 3 shows a sequence of patterns, which are projected according to an embodiment, and

(5) FIG. 4 shows an example of a sequence of camera images, which are generated by projecting patterns according to FIG. 3.

(6) FIG. 1 shows an inventive fringe projection device 10 for determining the topography of a body 12, which device has a projector 14, a camera 16 and a controller 18. The projector 14 is configured for projecting a sequence of sets of patterns T.sub.1, T.sub.2, . . . onto a surface 20 of body 12. The sequential patterns are drawn together, wherein a time ray 22 indicates that single patterns are projected according to a temporal sequence. It is to be noted that it is however also possible to project two or more patterns simultaneously, if the colors of patterns are different.

(7) Due to the pattern T.sub.i, which is projected onto the surface 20, a bright-dark-pattern is created on the surface 20. Camera 16 records, for each pattern, an image 24.

(8) Camera records image 24.1 for the first patter T.sub.1, image 24.2 for the second pattern T.sub.2, and in general image 24.i for the i-th pattern. FIG. 3 shows images 24.1 to 24.8 for a possible embodiment of the inventive method.

(9) In order to calculate the topography of body 12, in particular then the three-dimensional coordinates of surface 20 in a predetermined coordinate system, for each detection point of camera 16 a theoretical observation ray 28 is construed and intersected by a light plane 30. Based on the coordinates of projector 14 and camera 16 and the angles, at which the observation ray 28 and the light plane 30 extend, the coordinates of point P on the surface 20 is determined, which has been hit by the light plane 30 and which has been detected by the observation ray 28. This calculation is already known. The problem consists in determining the light plane 30 incident at point P based on the sequence of light intensities Q.sub.k,m for a predetermined camera pixel.

(10) The present description refers to a fringe projection method, in which a sequence of sets of patterns is projected onto a surface 20, which allows a particularly precise determination of the coordinates of points on surface 20. Moreover, the sequence of sets of patterns used in the inventive method is particularly suitable for measuring to bodies 12 with glossy surfaces.

(11) FIGS. 1, 2 and 3 show that the patterns T.sub.i respectively have a constant number of S-fringes.

(12) FIG. 2 shows a diagram for the sequence of sets of patterns for an inventive fringe projection method. In the uppermost part, the independent coordinate x is drawn. The y-direction, which is perpendicular to the x-direction, is indicated as the fringe longitudinal direction L. The brightness along the fringe longitudinal direction L for each fringe in a pattern T.sub.i is technically constant. This means that the projected light intensity Q.sub.proj may oscillate, although these oscillations are so small as to be negligible in the evaluation.

(13) FIG. 2 schematically shows a sequence of sets of patterns T.sub.i for S=16 fringes. The number index s of fringes starts with 0, i.e. the fringes are numbered consecutively with s=0, s=1, . . . s=15. In the coordinate system, the fringes extend within the interval [0, X], i.e. the smallest x-value is by definition 0, while the highest x-value is X.

(14) In the lower section, the intensity distributions for the temporal sequence of patterns T.sub.1, T.sub.2, . . . are shown. In the present case, Q.sub.proj=Q.sub.m,k,s=sin(360 Sx/X+k+g.sub.s,m), wherein =360/N. N is the number of symbols in the alphabet of the Gray code in use. Thus, the intensity distribution for pattern T.sub.1 is Q.sub.proj=sin(360 Sx/X). The first pattern T.sub.1 is part of the first set, thus m=0, and of the first pattern, thus k=0.

(15) Since each set comprises K patterns, which are phase-shifted to each other by , a K-dimensional intensity vector Q.sub.m may be defined, the components of which are the intensities of the patter within a set. From the intensity vector Q.sub.m a two-dimensional phase vector {right arrow over ()}.sub.m may be calculated.

(16) It can be seen that both patterns in the first set, in the present case, are selected in a way that they are continuously merged at the boundaries, where two fringes are adjacent to each other.

(17) In the second set in FIG. 2, four separated segments with respective four fringes, are present, wherein the intensity distributions are discontinuous at the boundaries between the four segments. In the third set the intensity distributions are discontinuous on all boundaries between adjacent fringes, except on those boundaries, at which in the previous pattern sets discontinuities were already present.

(18) Based on the described sequence of sets of patterns, the topography of the body is computed based on the image points recorded by camera 16 (see FIG. 1) in the following way:

(19) At the 0-th pattern set, i.e. m=0, the phase position is independent from the ordinal number s of fringe. This can be achieved in that for the 0-th position in the Gray code word g(s) of S the value 0 is always selected, i.e. g.sub.s,0=0 for all s(0, 1, . . . , s1).

(20) This corresponds to an expansion of the Gray code of an additional 0-th position, i.e. for S=16 fringes, 3-digits code words are used, although for coding of 16 fringes with a quaternary Gray code, 2 digits would be sufficient.

(21) Calculation of the Topography from Measurement Data

(22) The intensity distribution of k-th pattern in m-th set is, as already described, Q.sub.k,s,m(x)=a p [360 Sx/X+k+g.sub.s,m]+c, with phase position
=k.Math.+g.sub.s,m.Math.

(23) The patterns are projected onto a body, are reflected by it and are recorded by a camera. The intensities Q.sub.k,m incident on a respective camera pixel correspond to
Q.sub.k,m=sin(360 Sx/X+k+g.sub.s,m).

(24) Herein, x still indicates the x-coordinate of pattern, which is responsible for the recorded intensity of the camera pixel. The value of x is thus constant but unknown (correspondence problem). Herein, s still indicates the ordinal number of fringe and is also constant although unknown.

(25) The projector fringe s and the x-coordinate have to be computed, in order to solve the correspondence problem and to perform a triangulation. Moreover, the amplitude arriving at the camera pixel is also unknown, since it is influenced by the reflective properties of body.

(26) The offset c, or offset c in the intensities Q.sub.k,m of camera pixels is eliminated by a usual preprocessing (by subtraction with other projected patterns). The knowns are thus S, X, k, , m, as well as the encoding used for g.sub.s,m. Initially s is sought, then x.

(27) It is also assumed, that the 0-th pattern set (m=0 or reference signal) does not contain any phase shifting, thus g.sub.s,0=0 for all s{0, 1, . . . , S1}. In case of projection of the 0-th pattern set, the intensities of the camera pixels are thus
Q.sub.k,0= sin(360 Sx/X+k).

(28) For the calculation of s and x, we initially group all intensities recorded in a camera pixel with k{0, 1, . . . , K1} of a respective pattern set to form an intensity vector {right arrow over (Q)}.sub.m

(29) $\begin{array}{cc}{\stackrel{.fwdarw.}{Q}}_m=\left(\begin{array}{c}{\stackrel{\_}{Q}}_{0,m}\\ {\stackrel{\_}{Q}}_{0,m}\\ \mathrm{.Math.}\\ {\stackrel{\_}{Q}}_{K-1,m}\end{array}\right)=\left(\begin{array}{ccc}\tilde{a}.Math.\sin (360.Math.S.Math.x/X+& 0& .Math.+g_{s,m}.Math.)\\ \tilde{a}.Math.\sin (360.Math.S.Math.x/X+& 1& .Math.+g_{s,m}.Math.)\\ & \mathrm{.Math.}& \\ \tilde{a}.Math.\sin (360.Math.S.Math.x/X+& \left(K-1\right)& .Math.+g_{s,m}.Math.)\end{array}\right)& \left(1\right)\end{array}$

(30) According to the Addition theorem of trigonometry:

(31) $\sin \left(+\right)=\cos \left(\right)\sin \left(\right)+\sin \left(\right)\cos \left(\right)=\left(\cos \left(\right)\sin \left(\right)\right)\left(\begin{array}{c}\sin \left(\right)\\ \cos \left(\right)\end{array}\right)$

(32) By equivalent transformation of (1) it derives

(33) ${\stackrel{.fwdarw.}{Q}}_m=\underset{\underset{H}{}}{\left(\begin{array}{cc}\cos \left(0.Math.\right)& \sin \left(0.Math.\right)\\ \cos \left(1.Math.\right)& \sin \left(1.Math.\right)\\ \mathrm{.Math.}& \mathrm{.Math.}\\ \cos \left(\left(K-1\right).Math.\right)& \sin \left(\left(K-1\right).Math.\right)\end{array}\right)}\underset{\underset{{\stackrel{.fwdarw.}{}}_m}{}}{\left(\begin{array}{c}\tilde{a}.Math.\sin \left(360.Math.S.Math.x/X+g_{s,m}.Math.\right)\\ \tilde{a}.Math.\cos \left(360.Math.S.Math.x/X+g_{s,m}.Math.\right)\end{array}\right)}$

(34) With known constant matrix H and unknown two-dimensional phase-vector {right arrow over ()}.sub.m.

(35) In case of K=2 and =90 H becomes the unit matrix and the simple relationship holds
{right arrow over (Q)}.sub.m={right arrow over ()}.sub.m

(36) In the general case, the phase vector {right arrow over ()}.sub.m may be computed by means of the pseudo-inverse (Moore-Penrose-Inverse) H.sup.+:
{right arrow over ()}.sub.m=H.sup.+.Math.{right arrow over (Q)}.sub.m
The precondition for an univocal solution for {right arrow over ()}.sub.m is that the rank of H is higher than 1. This is exactly the case when: K2 and 1 mod 1800.

(37) For the calculation of s there are now two possibilities.

(38) Method A

(39) The oriented angle between the 0-th phase vector {right arrow over ()}.sub.0 and the m-th phase vector {right arrow over ()}.sub.m corresponds to the phase jump g.sub.s,m:
g.sub.s,m({right arrow over ()}.sub.0,{right arrow over ()}.sub.m)mod 360(2)

(40) The angular function ({right arrow over (a)}{right arrow over (b)}) herein describes the positive directed angle in two dimensions of vector {right arrow over (a)} after vector {right arrow over (b)} in the mathematically positive sense of rotation. This angle may be computed, for example, by means of the known a tan 2-function:

(41) $\left(\left(\begin{array}{c}{a}_x\\ a_y\end{array}\right),\left(\begin{array}{c}{b}_x\\ b_y\end{array}\right)\right)=\left[a\tan 2\left(a_x{b}_y-a_y{b}_x,ab+a_y{b}_y\right)+360\right]\mathrm{mod}360$

(42) For the calculation of the sought projection fringe s, the numerical value s{0, 1, . . . , S1} is now selected, for which the congruence equation (2) is solved for all m{0, 1, . . . , M1}.

(43) Method B

(44) More robust is determination of s by means of maximization. To this end, the intensity vector {right arrow over ()}.sub.0 of the reference signal is rotated in such a way that, if s is appropriately selected, it is directed in the same direction as {right arrow over ()}.sub.m and in this case the scalar product <.,.> is maximized.

(45) $s=\underset{\tilde{s}\left(0,\mathrm{.Math.},S-1\right)}{\arg \max }\underset{m=1}{\overset{M-1}{.Math.}}.Math.\left(\begin{array}{cc}\cos \left(g_{\tilde{s},m}.Math.\right)& -\sin \left(g_{\tilde{s},m}.Math.\right)\\ \sin \left(g_{\tilde{s},m}.Math.\right)& \cos \left(g_{\tilde{s},m}.Math.\right)\end{array}\right){\stackrel{.fwdarw.}{}}_0,{\stackrel{.fwdarw.}{}}_m.Math.$

(46) After calculating the projector fringe s, the x-coordinate of pattern may be determined by a calculation method known from the phase-shift approach. To this end, it is sufficient to observe the 0-th intensity vector. The formula

(47) $x=\left[s+\left(\left(\begin{array}{c}1\\ 0\end{array}\right).Math.{\stackrel{.fwdarw.}{}}_0\right)/360\right].Math.X/S$

(48) Then provides the x-coordinate of the projected pattern, which is detected by the camera pixel.

(49) From the so computed x-coordinate of projectors and the coordinate of the camera pixel, the spatial position of camera 16 and projector 10, which is determined by a calibration measurement, as well as the imaging properties of camera 16 and projector 10, the coordinates of P are then determined as known, by triangulation. This method is automatically performed by means of the controller 18 for all pixels of camera 18 and thus coordinates of points P of surface 20 are calculated. The point cloud thus formed describes the topography of body 12.

REFERENCE LIST

(50) 10 fringe projection device 12 body 14 projector 16 camera 18 controller 20 surface 22 time ray 24 camera image 28 observation ray 30 light plane phase position phase shift parameter difference angle a real number b real number c real number g function assigning to a number its Gray code g.sub.s,m m-th position of the Gray code of s k index of pattern within set K number of patterns for each set L longitudinal direction of fringe m index of set M number of sets N number of symbols of code-alphabet p function P point on surface Q.sub.k,m detected intensity Q.sub.proj projected light intensity t time s index of fringes S number of fringes T pattern x pattern coordinate X width of pattern, or largest coordinate possible y pattern coordinate, parallel to fringe