Stereoscopic vision three dimensional measurement method and system for calculating laser speckle as texture

Abstract

The disclosure provides a stereoscopic vision three dimensional measurement method and system calculating a laser speckle as a texture. The measurement method includes that: a forming process of a laser speckle is simulated first, and calculation is performed to obtain a digitized laser speckle diagram, then the laser speckle diagram is outputted onto a film or a photographic dry plate, the laser speckle diagram on the film or the photographic dry plate is projected onto a surface of a measured object subsequently; a left view and a right view of the measured object are acquired finally, and all matching points corresponding to each other are found in the left view and the right view, and then a three dimensional point cloud of the surface of the measured object is reconstructed.

Claims

1. A stereoscopic vision three dimensional measurement method for calculating a laser speckle as a texture, comprising: simulating a forming process of a laser speckle first, and performing calculation to obtain a digitized laser speckle diagram; then outputting the laser speckle diagram onto a film or a photographic dry plate; projecting the laser speckle diagram on the film or the photographic dry plate to a surface of a measured object subsequently; acquiring a left view and a right view of the measured object finally, then finding all matching points corresponding to each other of the surface of the measured object in the left view and the right view, and reconstructing a three dimensional point cloud of the surface of the measured object according to parallax data of the matching points; wherein the performing the calculation to obtain the digitized laser speckle diagram comprises: simulating scattering of coherent lights by a rough surface to form a plurality of beams of scattered lights; and transmitting the plurality of beams of scattered lights onto an observation plane to perform coherent superposition so as to form the laser speckle diagram; wherein a light intensity value of each point on the observation plane is obtained through numerical calculation.

2. The stereoscopic vision three dimensional measurement method for calculating the laser speckle as the texture according to claim 1, wherein the numerical calculation comprises: assuming that the amplitudes of lights scattered from any point of the rough surface to a point P of the observation plane are all 1 or a random number, then a formula of the light intensity of the point P of the observation plane is: $I\left(u,v\right)={.Math.\underset{1}{\overset{n}{.Math.}}\underset{1}{\overset{n}{.Math.}}{e}^{-i{}_{i,j}}.Math.}^2=I_R\left(u,v\right)+I_I\left(u,v\right),\mathrm{where}$ $I_R\left(u,v\right)={\left(\underset{1}{\overset{n}{.Math.}}\underset{1}{\overset{n}{.Math.}}\cos {}_{i,j}\right)}^2,I_I\left(u,v\right)={\left(\underset{1}{\overset{n}{.Math.}}\underset{1}{\overset{n}{.Math.}}\sin {}_{i,j}\right)}^2,$ the .sub.i,j is the phases of lights scattered from a scattering point (i,j) to the point P of the observation plane, and its formula is ${}_{i,j}=\frac{2}{}\left(\sqrt{{\left(u-x_i\right)}^2+{\left(v-y_i\right)}^2+Z_0^2}-2h_{i,j}\right),$ the {square root over ((ux.sub.i).sup.2+(vy.sub.i).sup.2+Z.sub.0.sup.2)}2h.sub.i,j is the distance from a scattering point S(x.sub.i, y.sub.j) to the point P (u, v) of the observation plane, the h.sub.i,j is the relative height of a scattering point, i.e. the offset of the height of a certain scattering point from the average height of all scattering points, the x.sub.i and y.sub.i are respectively the coordinates of scattering points numbered (i, j), $x_i=\frac{L_x}{m}\left(i-0.5\right),y_i=\frac{L_y}{n}\left(j-0.5\right),\frac{L_x}{m}\mathrm{and}\frac{L_y}{n}$ and therein are the intervals between adjacent transverse scattering points and between adjacent longitudinal scattering points among mn scattering points at equal intervals on a scattering plane, and the values of m and n are: $\{\begin{array}{c}m>\frac{L_{\mathrm{px}}.Math.L_x}{Z_0.Math.}\\ n>\frac{L_{\mathrm{py}}.Math.L_y}{Z_0.Math.}\end{array},$ where Z.sub.0 is the distance from the scattering plane to the observation plane, is the wavelength of a laser, L.sub.x is the horizontal size of the scattering plane, L.sub.y is the vertical size of the scattering plane, L.sub.px is the horizontal size of the observation plane, L.sub.py is the vertical size of the observation plane; the heights of the scattering points are random numbers independent of each other, and recorded as h(i, j) and the values thereof are generated by a random number generator of a computer, positive and negative values are used, the average value is 0, and the root-mean-square value is larger than or equal to a half of the wavelength; the autocorrelation lengths L.sub.cx and L.sub.cy, i.e. the size of the speckle, in the x direction and the y direction may be changed by controlling L.sub.x, L.sub.y, the wavelength and the distance Z.sub.0 between the scattering plane and the observation plane, and the relations among them are: $L_{\mathrm{cx}}=\frac{}{L_x}{Z}_0,L_{\mathrm{cy}}=\frac{}{L_y}{Z}_o.$

3. The stereoscopic vision three dimensional measurement method for calculating the laser speckle as the texture according to claim 2, wherein the autocorrelation lengths L.sub.cx and L.sub.cy of the speckle is adjusted according to a shape characteristic of the measured object, a measurement system parameter and an optimal matching precision principle.

4. The stereoscopic vision three dimensional measurement method for calculating the laser speckle as the texture according to claim 1, wherein the acquiring the left view and the right view of the measured object, and finding all matching points corresponding to each other of the surface of the measured object in the left view and the right view, and reconstructing the three dimensional point cloud of the surface of the measured object according to the parallax data of the matching points comprises: firstly aligning the left view and the right view according to the relative positions of a left image acquisition device and a right image acquisition device, wherein the relative positions are determined by a rotation matrix R and a translation vector T; then searching matching points corresponding to each other in the left view and the right view by using a matching algorithm until all matching points corresponding to each other in the left view and the right view are found, so as to obtain a parallax map between the left view and the right view; finally calculating the parallax map according to a stereoscopic vision three dimensional reconstructing formula so as to obtain three dimensional point cloud data and complete digital three dimensional imaging, wherein the three dimensional reconstructing formula is: $w\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]=Q*\left[\begin{array}{c}{u}_l\\ v_l\\ d\\ 1\end{array}\right]=\left[\begin{array}{cccc}1& 0& 0& -c_x^{\mathrm{left}}\\ 0& 1& 0& -c_y\\ 0& 0& 0& f\\ 0& 0& -1/T& \left(c_x^{\mathrm{left}}-c_x^{\mathrm{right}}\right)/T\end{array}\right]*\left[\begin{array}{c}{u}_l\\ v_l\\ d\\ 1\end{array}\right],$ where (u.sub.l, v.sub.l) are the coordinates of a certain point on the left view after the left view and the right view are aligned; d is the parallax between the point and a corresponding matching point on the right view, Q is a matrix marked by binocular stereoscopic vision.

5. The stereoscopic vision three dimensional measurement method for calculating the laser speckle as the texture according to claim 4, wherein the searching the matching points corresponding to each other in the left view and the right view by using the matching algorithm comprises: after line alignment, obtaining, through the following calculation formula, a point at a location with the smallest Sum of Absolute Difference (SAD) value in the same line by using an SAD matching algorithm when the size of an SAD window is 2m+1, so as to obtain the position of a matching point: $\mathrm{SAD}\left(i,j,d\right)=\underset{h=-m}{\overset{m}{.Math.}}\underset{k=-m}{\overset{m}{.Math.}}.Math.P_L\left(i+h,j+k\right)-P_R\left(i+h,j+k+d\right).Math.,$ where (i, j) is a point in the left view, d is the parallax between the point and a corresponding matching point on the right view, and P.sub.L and P.sub.R are the grey values on current pixels respectively.

6. The stereoscopic vision three dimensional measurement method for calculating the laser speckle as the texture according to claim 4, wherein the matching algorithm in the searching the matching points corresponding to each other in the left view and the right view by using the matching algorithm is a Sum of Squared Differences (SSD) matching algorithm, a Normalized Cross Correlation (NCC) matching algorithm, or a Zero-mean Normalized Cross-Correlation Algorithm (ZNCC) matching algorithm.

7. The stereoscopic vision three dimensional measurement method for calculating the laser speckle as the texture according to claim 1, wherein the projection lights in the projecting the laser speckle diagram on the film or the photographic dry plate to the surface of the measured object are incoherent monochromatic lights, and in the meanwhile, ambient lights besides the incoherent monochromatic lights are stopped by a filter from being acquired when the left view and the right view of the measured object are acquired.

8. A stereoscopic vision three dimensional measurement system for calculating a laser speckle as a texture, wherein comprises: an optical projecting device, configured to project a speckle diagram printed on a film or a photographic dry plate onto a surface of a measured object; a left camera and a right camera, configured to acquire a left view and a right view of the measured object, respectively; a computing device, configured to receive and store the left camera and right camera-acquired left view and right view of the measured object, and find all matching points corresponding to each other in the left view and the right view through a certain matching algorithm so as to obtain a parallax map, and then reconstruct a three dimensional point cloud of the measured object through the parallax map; wherein the speckle diagram is obtained by: simulating scattering of coherent lights by a rough surface to form a plurality of beams of scattered lights; and transmitting the plurality of beams of scattered lights onto an observation plane to perform coherent superposition so as to form the speckle diagram; wherein a light intensity value of each point on the observation plane is obtained through numerical calculation.

9. The stereoscopic vision three dimensional measurement system for calculating the laser speckle as the texture according to claim 8, wherein a light source of the optical projecting device is incoherent monochromatic lights and light filters are added in front of imaging lens of the left camera and the right camera.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a flowchart of a stereoscopic vision three dimensional measurement method calculating a laser speckle as a texture according to an embodiment of the disclosure;

(2) FIG. 2 is a diagram of optical principles of formation of a laser speckle in an embodiment of a stereoscopic vision three dimensional measurement method calculating a laser speckle as a texture according to the disclosure;

(3) FIG. 3 is a partial diagram of a laser speckle calculation diagram in an embodiment of a stereoscopic vision three dimensional measurement method calculating a laser speckle as a texture according to the disclosure;

(4) FIG. 4 is a schematic diagram of an SAD curve of a process of searching a corresponding matching point in an embodiment of a stereoscopic vision three dimensional measurement method calculating a laser speckle as a texture according to the disclosure;

(5) FIG. 5 is a structural diagram of a stereoscopic vision three dimensional measurement system calculating a laser speckle as a texture according to the disclosure; and

(6) FIG. 6 is a schematic diagram of a projection light path of a stereoscopic vision three dimensional measurement system calculating a laser speckle as a texture according to the disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

(7) The disclosure will be further expounded hereinafter with reference to the accompanying drawings and embodiments so that the purpose, technical solution and advantages of the disclosure are clearer. It should be understood that the specific embodiments described herein are only used for explaining the disclosure, instead of limiting the disclosure.

(8) As shown in FIG. 1 to FIG. 4:

(9) a stereoscopic vision three dimensional measurement method calculating a laser speckle as a texture according to an embodiment of the disclosure includes that a forming process of a laser speckle is simulated first (by a computer), and calculation is performed to obtain a digitized laser speckle diagram, then the laser speckle diagram is outputted onto a film or a photographic dry plate, the laser speckle diagram on the film or the photographic dry plate is projected to a surface of a measured object subsequently, a left view and a right view of the measured object are acquired finally, and all matching points corresponding to each other of the surface of the measured object are found in the left view and the right view, and a three dimensional point cloud of the surface of the measured object is reconstructed according to parallax data of the matching points,

(10) wherein that the calculation is performed to obtain the digitized laser speckle diagram includes that: scattering of coherent lights by a rough surface is simulated (by the computer) to form a plurality of beams of scattered lights, and the plurality of beams of scattered lights is transmitted onto an observation plane to perform coherent superposition so as to form the laser speckle diagram, wherein a light intensity value of each point on the observation plane is obtained through numerical calculation (as shown in FIG. 3). The numerical calculation specifically includes that: it is assumed that the amplitudes of lights scattered from any point of the rough surface to a point P of the observation plane are all 1 (which may be also a random number), then a formula of the light intensity of the point P of the observation plane is:

(11) $I\left(u,v\right)={.Math.\underset{1}{\overset{n}{.Math.}}\underset{1}{\overset{n}{.Math.}}{e}^{-i{}_{i,j}}.Math.}^2=I_R\left(u,v\right)+I_I\left(u,v\right);\mathrm{where}$$I_R\left(u,v\right)={\left(\underset{1}{\overset{n}{.Math.}}\underset{1}{\overset{n}{.Math.}}\cos {}_{i,j}\right)}^2,I_I\left(u,v\right)={\left(\underset{1}{\stackrel{n}{.Math.}}\underset{1}{\overset{n}{.Math.}}\sin {}_{i,j}\right)}^2;$

(12) the .sub.i,j is the phases of lights scattered from a scattering point (i, j) to the point P of the observation plane, and its formula is

(13) ${}_{i,j}=\frac{2}{}\left(\sqrt{{\left(u-x_i\right)}^2+{\left(v-y_i\right)}^2+Z_0^2}-2h_{i,j}\right),$

(14) the {square root over ((ux.sub.i).sup.2+(vy.sub.i).sup.2+Z.sub.0.sup.2)}h.sub.i,j is the distance from a scattering point S(x.sub.i, y.sub.j) to the point P(u, v) of the observation plane, the h.sub.i, j is the relative height of a scattering point, i.e. the offset of the height of a certain scattering point from the average height of all scattering points, the x.sub.i and y.sub.j are respectively the coordinates of scattering points numbered (i, j),

(15) 0$x_i=\frac{L_x}{m}\left(i-0.5\right),y_i=\frac{L_y}{n}\left(j-0.5\right),\mathrm{and}\frac{L_x}{m}\mathrm{and}\frac{L_y}{n}$
therein are the intervals between adjacent transverse scattering points and between adjacent longitudinal scattering points among mn scattering points at equal intervals on a scattering plane, and the values of m and n are:

(16) $\{\begin{array}{c}m>\frac{L_{\mathrm{px}}.Math.L_x}{Z_0.Math.}\\ n>\frac{L_{\mathrm{py}}.Math.L_y}{Z_0.Math.}\end{array};$

(17) where Z.sub.0 is the distance from the scattering plane to the observation plane, is the wavelength of a laser, L.sub.x is the length of the scattering plane, L.sub.y is the width of the scattering plane, L.sub.px is the length of the receiving plane, L.sub.py is the width of the receiving plane; the heights of the scattering points are random numbers independent of each other, and recorded as h(i, j) and the values thereof are generated by a random number generator of a computer, positive and negative values are used, the average value is 0, and the root-mean-square value is equal to 2/3.

(18) The autocorrelation lengths L.sub.cx and L.sub.cy, i.e. the size of the speckle, in the x direction and the y direction may be changed by controlling L.sub.x, L.sub.y, the wavelength and the distance Z.sub.0 between the scattering plane and the observation plane, and the relations among them are:

(19) $L_{\mathrm{cx}}=\frac{}{L_x}{Z}_0$$L_{\mathrm{cy}}=\frac{}{L_y}{Z}_o.$

(20) That the left view and the right view of the measured object are acquired, and all matching points corresponding to each other of the surface of the measured object are found in the left view and the right view, and the three dimensional point cloud of the surface of the measured object is reconstructed according to the parallax data of the matching points specifically includes that: the left view and the right view are aligned first according to the relative positions of a left image acquisition device and a right image acquisition device, wherein the relative positions are determined by a rotation matrix R and a translation vector T; then matching points corresponding to each other in the left view and the right view are searched by using a matching algorithm until all matching points corresponding to each other in the left view and the right view are found, so as to obtain a parallax map between the left view and the right view, finally the parallax map is calculated according to a stereoscopic vision three dimensional reconstructing formula so as to obtain three dimensional point cloud data and complete digital three dimensional imaging, wherein the three dimensional reconstructing formula is:

(21) $w\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]=Q*\left[\begin{array}{c}{u}_l\\ v_l\\ d\\ 1\end{array}\right]=\left[\begin{array}{cccc}1& 0& 0& -c_x^{\mathrm{left}}\\ 0& 1& 0& -c_y\\ 0& 0& 0& f\\ 0& 0& -1/T& \left(c_x^{\mathrm{left}}-c_x^{\mathrm{right}}\right)/T\end{array}\right]*\left[\begin{array}{c}{u}_l\\ v_l\\ d\\ 1\end{array}\right];$

(22) where (u.sub.l, v.sub.l) are the coordinates of a certain point on the left view after the left view and the right view are aligned; d is the parallax between the point and a corresponding matching point on the right view, Q is a matrix marked by binocular stereoscopic vision (literature: Bradski G, Kaehler A. Learning OpenCV: Computer vision with the OpenCV library[M]. O'Reilly Media, Inc., 2008).

(23) During the process of obtaining the parallax map through the matching, a point at a location with the smallest SAD value in the same line is obtained through the following calculation formula by using an SAD matching algorithm when the size of an SAD window is 2m+1, so as to obtain the position of a matching point:

(24) $\mathrm{SAD}\left(i,j,d\right)=\underset{h=-m}{\overset{m}{.Math.}}\underset{k=-m}{\overset{m}{.Math.}}.Math.P_L\left(i+h,j+k\right)-P_R\left(i+h,j+k+d\right).Math.;$

(25) in the formula, (i, j) is a point in the left view, d is the parallax between the point and a corresponding matching point on the right view, and P.sub.L and P.sub.R are the grey values on the current pixels respectively. In addition, the horizontal and vertical autocorrelation lengths L.sub.cx and L.sub.cy of the speckle may be adjusted according to a shape characteristic of the measured object, a measurement system parameter and an optimal matching precision principle.

(26) Besides, the projection lights in that the laser speckle diagram on the film or the photographic dry plate is projected to the surface of the measured object are incoherent monochromatic lights, and in the meanwhile, ambient lights besides the incoherent monochromatic lights are stopped by a narrow band filter from being acquired when the left view and the right view of the measured object are acquired.

(27) For example, as shown in FIG. 2, the length of the rough scattering plane on an XOY plane is Lx, and the width is Ly, the height (x, y) of the surface fluctuates around the XOY plane, and the root-mean-square value of a fluctuation amount is larger than a half of a laser. In other words:

(28) $\sqrt{\frac{1}{\mathrm{Lx}.Math.\mathrm{Ly}}{h}^2\left(x,y\right)\mathrm{dxdy}}\frac{}{2};$

(29) UOV is the observation plane, and the distance therefrom to the scattering plane is Z0; illumination lights are uniform plane waves vertically entering the scattering plane from right to left, and the phase on the XOY plane is set as 0.

(30) During measurement, parameters in a left camera are:

(31) $\left[\begin{array}{ccc}2809.6& 0& 784.0\\ 0& 2807.8& 615.7\\ 0& 0& 1\end{array}\right],$

(32) and parameters in a right camera are:

(33) $\left[\begin{array}{ccc}2779.5& 0& 806.1\\ 0& 2780.9& 697.8\\ 0& 0& 1\end{array}\right],$

(34) a transformation matrix between the left camera and the right camera is:

(35) $R=\left[\begin{array}{ccc}0.9419& -0.0124& 0.3356\\ 0.0155& 0.9999& -0.0066\\ -0.3355& 0.0114& 0.9420\end{array}\right],T=\left[\begin{array}{c}-212.4\\ -2.3\\ 45.5\end{array}\right];$

(36) After the computer receives the left view and the right view acquired by the left camera and the right camera, lines may be aligned and corrected. The SAD algorithm is used on the left view and the right view with lines aligned, and the corresponding matching points (as shown in FIG. 4) are searched. FIG. 4 is shows a correct corresponding matching point. The point is a point with the smallest SAD value among candidate matching points. The abscissa of the point is the abscissa of a point in the right view, wherein the point corresponds to a point observed in the left view. The parallax diagram between the left view and the right view may be obtained after finding all matching points corresponding to each other in the left view and the right view. According to the parallax map, three dimensional reconstruction is performed to obtain the three dimensional point cloud of the surface of the measured object. The acquisition time is less than 0.1 second and the number of point clouds is smaller than 240 thousands.

(37) In this way, a stereoscopic vision three dimensional measurement method calculating a laser speckle as a texture according to the disclosure may obtain an ideal laser speckle diagram through a numerical calculation method. The speckle diagram is produced into a film, and the laser speckle diagram on the film is further projected onto a surface of a measured object to form a laser speckle texture, thereby completely avoiding the problem in an existing traditional measurement method that matching points corresponding to each other are found incorrectly or cannot be found. In addition, a brand new parameter setting method is applied, which can adjust the autocorrelation length of a speckle conveniently so as to obtain a good matching curve. A matching value of the lowest point (or the highest point) of the curve is far smaller (or larger) then the average matching value, thus resulting in extremely low mismatching probability. In the meanwhile, illumination of incoherent monochromatic lights and a narrow band light filter are applied, so as to prevent ambient lights beyond the wavelengths of the monochromatic lights from being acquired, thus enhancing the resistance to interference of the ambient lights; besides, the disclosure has quick sampling (a left acquisition device and a right acquisition device only need to take a frame of an image respectively at the same time), low cost is and is easy to popularize.

(38) Of course, there may be many matching algorithms in that the matching points corresponding to each other in the left view and the right view are searched by using the matching algorithm, such as an SSD matching algorithm, an NCC matching algorithm, or a ZNCC matching algorithm, and matching algorithms covered by the patent are not limited to the SAD algorithm.

(39) As shown in FIG. 5 and FIG. 6:

(40) the disclosure further provides a dedicated measurement system for implementing a stereoscopic vision three dimensional measurement method calculating a laser speckle as a texture, including an optical projecting device 1, a left camera 2, a right camera 3 and a computing device 4, wherein the optical projecting device 1 is mainly configured to project an image on a speckle diagram film onto a surface of a measured object 5; the left camera 2 and the right camera 3 are mainly configured to acquire a left view and a right view of the measured object 5 having the surface projected with the speckle diagram; the computing device 4 is mainly configured to find, through calculation, all points corresponding to each other in the left camera 2 and right camera 3-acquired left view and the right view of the measured object 5, so as to perform three dimensional reconstruction to obtain a three dimensional point cloud of the surface of the measured object 5. The projection direction of the speckle projecting device 1 directly faces the measured object 5, and a light source 11 thereof may be an incoherent monochromatic light source (e.g. a Light Emitting Diode (LED) light source). Light filters may be added in front of imaging lens of the left camera 2 and the right camera 3; and the optical projecting device 1, the left camera 2 and the right camera 3 are interconnected with the computing device 4.

(41) When working, the measurement system of the disclosure first uses the optical projecting device 1 to project the speckle diagram on the film onto the surface of the measured object (specifically, the film 14 is illuminated by the light source 11 through a lens 12 and a lens 13 as shown in FIG. 6), the speckle diagram on the film is projected onto the surface of the measured object through the imaging lenses 15, then the images of the measured object 5 are acquired by the left camera 2 and the right camera 3, and transmitted to the computing device 4, and the computer device 4 finally finds all points corresponding to each other in the left view and the right view by using a matching algorithm (e.g. the SAD algorithm) so as to reconstruct the three dimensional point cloud of the measured object.

(42) In this way, the stereoscopic vision three dimensional measurement system calculating the laser speckle as the texture may ensure relatively low mismatching probability of the matching points, and quick imaging. In addition, incoherent monochromatic lights are used as the light source 11, while the narrow band light filters are added in front of the imaging lenses of the left camera and the right camera of the measurement system, so as to stop ambient lights beyond the wavelengths of the monochromatic lights from being received by the cameras, thus enhancing the resistance of the system to interference of the ambient lights; besides, the system, which has fast sampling, and low cost, is easy to popularize.

(43) The content above is to further describe the disclosure in details with reference to specific preferred technical solutions and it should not be believed that specific implementation of the disclosure is limited to the description. Several simple deductions or replacements may be further made for those skilled in the art without departing from the concept of the disclosure, and should be all considered as belonging to the protection scope of the disclosure.