Method for high-precision true color three-dimensional reconstruction of a mechanical component

Abstract

A method for high-precision true color three dimensional reconstruction of mechanical component. Firstly performs image acquisition: the left and right high-resolution grayscale cameras are fixed at same height and spaced at certain distance, an optical transmitter fixed between the two grayscale cameras, and low-resolution color camera fixed above optical transmitter, thus images of measured high-precision mechanical component are shot. Then performs image processing: all images are transmitted to a computer, which uses image processing to record surface information of measured high-precision mechanical component in the point cloud by high-precision true color three-dimensional reconstruction, which reflects color texture information of the surface, so as to realize the non-contact high-precision true color three dimensional reconstruction of high-precision mechanical component. The method uses binocular high-precision grayscale cameras instead of binocular color cameras, which broadens the range of capture wavelengths, retains richer texture details of high-precision mechanical component and improves accuracy of measurement.

Claims

1. A method for high-precision true color three dimensional reconstruction of a mechanical component, comprising: (1). image acquisition 1.1). fixing a left high-resolution grayscale camera and a right high-resolution grayscale camera on a support device, where the right high-resolution grayscale camera and the left high-resolution grayscale camera are at the same height and spaced at a certain distance; fixing an optical transmitter between the left high-resolution grayscale camera and right high-resolution grayscale camera and a low-resolution color camera above the optical transmitter; 1.2). adjusting the left high-resolution grayscale camera, the right high-resolution grayscale camera, the optical transmitter and the low-resolution color camera, so that a measured high-precision mechanical component can be completely and clearly captured by the cameras and the optical transmitter can completely and clearly project non-coded light and coded light on the surface of the measured high-precision mechanical component; adjusting a calibration plate and using Zhang's calibration method to complete a camera calibration; 1.3). placing the measured high-precision mechanical component at a suitable position in the view of the cameras, then rotating the measured high-precision mechanical component in a fixed direction and acquiring images of J poses: for the measured high-precision mechanical component of the j.sup.th pose, firstly the optical transmitter projects a non-coded light on the surface of the measured high-precision mechanical component, the low-resolution color camera takes image which is denoted by low-resolution color image CI.sub.j, the left high-resolution grayscale camera takes an image which is denoted by left high-resolution grayscale image LI.sub.j, then the optical transmitter projects N frames of encoded light on the surface of the measured high-precision mechanical component, the left high-resolution grayscale camera and right high-resolution grayscale camera respectively take a grayscale image which are respectively denoted by LI.sub.jn and RI.sub.jn, n=1, 2, . . . , N; (2). image processing 2.1). reconstructing a three-dimensional point cloud of the measured high-precision mechanical component of each pose: based on the principle of binocular camera, reconstructing a three-dimensional point cloud P.sub.j of the j.sup.th pose by using the grayscale images LI.sub.j1, LI.sub.j2, . . . , LI.sub.jN and RI.sub.j1, RI.sub.j2, . . . , RI.sub.jN of the j.sup.th pose, where j=1, 2, . . . , J; 2.2). performing registrations on 3D point clouds to calculate a rigid body transformation matrix W.sub.j by using the three-dimensional point cloud P.sub.j as a target point cloud and the three-dimensional point cloud P.sub.j+1 of the (j+1).sup.th pose as a source point cloud, where j=1, 2, . . . , J−1: 2.2.1). respectively performing voxel down sampling on the target point cloud P.sub.j and the source point cloud P.sub.j+1, where the voxel down sampled target point cloud is denoted by the target point cloud P.sub.j_down and the voxel down sampled source point cloud is denoted by the source point cloud P.sub.j+1_down, and respectively performing 3D-SIFT key point extraction on the target point cloud P.sub.j_down and the target point cloud P.sub.j+1_down, where the extracted target point cloud is denoted by a target point cloud P.sub.j_key, the extracted source point cloud is denoted by a source point cloud P.sub.j+1_key; 2.2.2). initializing iteration number k=1, setting a maximum iteration number K.sub.c_reg, setting an iteration number K.sub.pp which is used for shifting a point-to-plane error function to a point-to-point error function, initializing transformed source point cloud P.sub.j+1_key_1 to the source point cloud P.sub.j+1_key; 2.2.3). for the k.sup.th iteration, to each point of the source point cloud P.sub.j+1_key_k, finding the nearest point in the target point cloud P.sub.j_key as its corresponding point by using KD tree to obtain a corresponding point-to-point set C.sup.j+1_key_k, then removing the point and its nearest point whose distance is great than threshold Thres to obtain a optimized point-to-point set C.sub.j_key_k′; 2.2.4). homogenizing the i.sup.th point p.sub.j+1_key_ki of the source point cloud P.sub.j+1_key_k in point-to-point set C.sub.j_key_k′ to obtain its corresponding vector p.sub.j+1_key_ki=(x.sub.j+1_key_ki,y.sub.j+1_key_ki,z.sub.j+1_key_ki,1).sup.T, finding corresponding point p.sub.j_key_i of the target point cloud P.sub.j_key in point-to-point set C.sub.j_key_k′ and homogenizing it to obtain its corresponding vector p.sub.j_key_i=(x.sub.j_key_i,y.sub.j_key_i,z.sub.j_key_i,1).sup.T, where i=1, 2, . . . , N.sub.cj_key_k, N.sub.cj_key_k is the number of point-to-point set C.sub.j_key_k′; if iteration number k is less than K.sub.pp, then constructing a point-to-plane error function F.sub.jk by using the distance from the point p.sub.j+1_key_ki to the tangent plane of point p.sub.j_key_i of target point cloud P.sub.j_key, where the error function F.sub.jk is: $F_{jk}=\underset{i=1}{\overset{N_{\mathrm{cj}\_\mathrm{key}\_k}}{.Math.}}{\left(\left(W_{jk}.Math.{\stackrel{\_}{p}}_{\left(j+1\right)\_\mathrm{key}\_\mathrm{ki}}-{\stackrel{\_}{p}}_{j\_\mathrm{key}\_i}\right).Math.n_{j\_\mathrm{key}\_i}\right)}^2$ where n.sub.j_key_i is the normal vector corresponding to point p.sub.j_key_i, W.sub.jk is the transformation matrix of the k.sup.th iteration; if iteration number k is nor less than K.sub.pp, then constructing a point-to-point error function F.sub.jk according to the sum of the squares of the distances between the corresponding points, where the error function F.sub.jk is: $F_{jk}=\underset{i=1}{\overset{N_{\mathrm{cj}\_\mathrm{key}\_k}}{.Math.}}{.Math.W_{jk}.Math.{\stackrel{\_}{p}}_{\left(j+1\right)\_\mathrm{key}\_\mathrm{ki}}-{\stackrel{\_}{p}}_{j\_\mathrm{key}\_i}.Math.}^2;$ 2.2.5). calculating the transformation matrix W.sub.jk that makes the value of error function F.sub.jk minimum, and then applying a transformation to the source point cloud P.sub.j+1_key_k by using error function F.sub.jk to obtain a source point cloud P.sub.j+1_key_(k+1) and recording the value of error function F.sub.jk at the k.sup.th iteration; 2.2.6). judging whether the variation |F.sub.jk−F.sub.j(k−1)| of the value of error function of two consecutive iterations is less than a threshold e or iteration number k reaches the maximum iteration number K.sub.c_reg; if yes, then recording the point-to-point set C.sub.j_key_k′′ at the k′.sup.th iteration, where k′ is the iteration number when the judgment condition is met, and calculating the optimal rigid body transformation matrix W.sub.j=W.sub.jk, W.sub.j(k′−1) . . . W.sub.j1, where the optimal rigid body transformation matrix W.sub.j is a matrix of 4×4, its formation is: $W_j=\left[\begin{array}{cc}{R}_{\mathrm{cloud}\_j}& T_{\mathrm{cloud}\_j}\\ \stackrel{.fwdarw.}{0}& 1\end{array}\right]$ where R.sub.cloud_j is a rotation matrix of 3×3 which registers the source point cloud P.sub.j+1_key to the target point cloud P.sub.j_key, T.sub.cloud_j is a translation vector of 1×3, {right arrow over (0)} is a row vector of 3×1; if no, k=k+1, and returning step 2.2.3); 2.3). fusing the left high-resolution image LI.sub.j with the low-resolution color image CI.sub.j of the j.sup.th pose, j=1, 2, . . . , m 2.3.1). calculating a projection matrix H.sub.j for registering the low-resolution color image CI.sub.j to the left high-resolution image LI.sub.j: 2.3.1.1). respectively extracting SURF feature points from the low-resolution color image CI.sub.j and the left high-resolution image LI.sub.j, and taking the extracted SURF feature points as key points; 2.3.1.2). to each key point of the low-resolution color image CI.sub.j, finding the closest key point and the next closest key point in the left high-resolution image LI.sub.j according to Euclidean distance, if the ratio of the distance of the closest key point to the distance of the next closet key point is less than a set threshold θ, then determining the closest key point in the left high-resolution image LI.sub.j as the matching point, thus a matching point pair set C.sub.cl_j is obtained; 2.3.1.3). setting a maximum iteration number K.sub.p_reg and an error threshold φ, initializing iteration number k=1, then optimizing the matching point pair set C.sub.cl_j through random sampling consistency (RANSAC) algorithm to obtain the projection matrix H.sub.j: 2.3.1.3.1). for the k.sup.th iteration in the process of optimizing the matching point pair set C.sub.cl_j, randomly selecting four point pairs, which are denoted respectively by {q.sub.cj_1, q.sub.lj_1}, {q.sub.cj_2, q.sub.lj_2}, {q.sub.cj_3, q.sub.lj_3} and {q.sub.cj_4, q.sub.lj_4}, and homogenizing them to obtain four point pairs {q.sub.cj_1, q.sub.lj_1}, {q.sub.cj_2, q.sub.lj_2}, {q.sub.cj_3, q.sub.lj_3} and {q.sub.cj_4, q.sub.lj_4}, then calculating a projection matrix H.sub.jk between the four point pairs, which satisfies the following relationship: q.sub.lj_i=H.sub.jk.Math.q.sub.cj_i, 1, 2, . . . , 4 2.3.1.3.2). homogenizing a point pair {q.sub.cj_i, q.sub.lj_i} other than the selected four point pairs from the matching point pair set C.sub.cl_j to obtain point pair {q.sub.cj_i, q.sub.lj_i} then calculating a mapping error ε.sub.jk_i=∥q.sub.lj_i,H.sub.jkq.sub.cj_i∥ according to the projection matrix H.sub.jk, where i=5, 6, . . . , N.sub.cj, N.sub.cj is the number of point pairs of the matching point pair set C.sub.cl_j, and selecting the point pair which mapping error ε.sub.jk_i<φ to form a interior point set S.sub.jk; 2.3.1.3.3). if the last iteration has been performed, namely k=K.sub.p_reg, then comparing S.sub.j1, S.sub.j2, . . . , S.sub.jK.sub.p_reg to select the projection matrix H.sub.jk as the projection matrix H.sub.j when the interior point set S.sub.jk is the largest, or k=k+1 and returning step 2.3.1.3.1; 2.3.2). registering the low-resolution color image CI.sub.j to the left high-resolution image LI.sub.j according to the projection matrix H.sub.j to obtain a high-resolution registered color image CI.sub.j_r: 2.3.2.1). creating a empty high-resolution registered color image CI.sub.j_r which has the same size with the left high-resolution image LI.sub.j, then performing coordinate transformation on the pixels of the low-resolution color image CI.sub.j to obtain a color image CI.sub.j; 2.3.2.2). determining the interpolation range: the pixel points of the high-resolution registered color image CI.sub.j_r which are located on color image CI.sub.j and other than the matching points in matching point pair set C.sub.cl_j are the points which are needed to be interpolated, namely interpolation points; 2.3.2.3). using the bilinear interpolation method to interpolate the pixel values of all interpolation points: to the i.sup.th interpolation point q.sub.cj_r_i(x.sub.cj_r_i, y.sub.cj_r_i), performing a inverse transformation according to the projection matrix H.sub.j to obtain a pixel point ρ.sub.cj_i(x.sub.cj_ρ_i, y.sub.cj_ρ_i) located on the low-resolution color image CI.sub.j, and finding four nearest pixel points which are denoted respectively by p.sub.cj_ρ_i11(x.sub.cj_ρ_i1, y.sub.cj_ρ_i1), p.sub.cj_ρ_i12(x.sub.cj_ρ_i1, y.sub.cj_ρ_i2), p.sub.cj_ρ_i21(x.sub.cj_ρ_i2, y.sub.cj_ρ_i1) and p.sub.cj_ρ_i22(x.sub.cj_ρ_i2, y.sub.cj_ρ_i2); firstly interpolating in the x direction to obtain the following: $C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i1}\right)=\frac{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i}}{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i1},y_{\mathrm{cj}\mathrm{\_\rho }\_i1}\right)+\frac{x_{\mathrm{cj}\mathrm{\_\rho }\_i}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i2},y_{\mathrm{cj}\mathrm{\_\rho }\_i1}\right)$ $C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i2}\right)=\frac{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i}}{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i1},y_{\mathrm{cj}\mathrm{\_\rho }\_i2}\right)+\frac{x_{\mathrm{cj}\mathrm{\_\rho }\_i}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i2},y_{\mathrm{cj}\mathrm{\_\rho }\_i2}\right)$ then interpolating in the y direction to obtain the following pixel value: $C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i}\right)=\mathrm{Round}\left(\frac{y_{\mathrm{cj}\mathrm{\_\rho }\_i2}-y_{\mathrm{cj}\mathrm{\_\rho }\_i}}{y_{\mathrm{cj}\mathrm{\_\rho }\_i2}-y_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i1}\right)+\frac{y_{\mathrm{cj}\mathrm{\_\rho }\_i}-y_{\mathrm{cj}\mathrm{\_\rho }\_i1}}{y_{\mathrm{cj}\mathrm{\_\rho }\_i2}-y_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i2}\right)\right)$ letting the pixel value CI.sub.j_r(x.sub.cj_r_i, y.sub.cj_r_i) of the interpolation point q.sub.cj_r_i(x.sub.cj_r_i, y.sub.cj_r_i) be CI.sub.j(x.sub.cj_ρ_i, y.sub.cj_ρ_i); 2.3.2.4). to the pixel points of the high-resolution registered color image CI.sub.j_r which are located on color image CI.sub.j and are the matching points in matching point pair set C.sub.cl_j, directly using the pixel values of their corresponding matching point in the low-resolution color image CI.sub.j to assign; and to the pixel points of the high-resolution registered color image CI.sub.j_r which are not located on color image CI.sub.j, directly using preset RGB value to assign, thus the high-resolution registered color image CI.sub.j_r is obtained; 2.3.3). fusing the left high-resolution image LI.sub.j with the high-resolution registered color image CI.sub.j_r: 2.3.3.1). performing N.sub.l-layer Laplacian pyramid decomposition on the left high-resolution image LI.sub.j to obtain N.sub.l layers of image LI.sub.j_w, performing N.sub.l-layer Laplacian pyramid decomposition on each channel of the high-resolution registered color image CI.sub.j_r to obtain N.sub.l layers of image CI.sub.j_rw on R channel, N.sub.l layers of image CI.sub.j_gw on G channel and N.sub.l layers of image CI.sub.j_bw on B channel, where w=1, 2, . . . , N.sub.l and represents the serial number of image; 2.3.3.2). respectively performing bilateral filter on the top-layer images CI.sub.j_rN.sub.l, CI.sub.j_gN.sub.l, CI.sub.j_bN.sub.l and LI.sub.j_N.sub.l to obtain top-layer images CI.sub.j_rN.sub.l.sub._b, CI.sub.j_gN.sub.l.sub._b; CI.sub.j_bN.sub.l.sub._b and LI.sub.j_N.sub.l.sub._b; 2.3.3.3). to the top-layer images of three channels of the high-resolution registered color image CI.sub.j_r, calculating the difference before and after bilateral filtering to obtain detail layer images CI.sub.j_rN.sub.l′=CI.sub.j_rN.sub.l−CI.sub.j_rN.sub.l_b, CI.sub.j_gN.sub.l′=CI.sub.j_gN.sub.l−CI.sub.j_gN.sub.l.sub._b and CI.sub.j_bN.sub.l′=CI.sub.j_bN.sub.l−CI.sub.j_bN.sub.l.sub._b, which have rich details of image; 2.3.3.4). performing Gaussian filter on the top-layer image of the left high-resolution image LI.sub.j to obtain a overall smooth image LI.sub.j_N.sub.l.sub._g, then calculating the difference between the top-layer image LI.sub.j_N.sub.l.sub._b and the overall smooth image LI.sub.j_N.sub.l.sub._g to obtain a layer image LI.sub.j_N.sub.l′=LI.sub.j_N.sub.l.sub._b−.sub.j_N.sub.l.sub._g which has rich edge information; 2.3.3.5). calculating top-layer fusion images, where the top-layer fusion image of R channel is: Fu.sub.j_N.sub.l.sub._r=CI.sub.j_rN.sub.l+αCI.sub.j_rN.sub.l′+βLI.sub.j_N.sub.l′, the top-layer fusion image of G channel is: Fu.sub.j_N.sub.l.sub._g=CI.sub.j_gN.sub.l+αCI.sub.j_gN.sub.l′+βLI.sub.j_N.sub.l′, the top-layer fusion image of B channel is: Fu.sub.j_N.sub.l.sub._r=CI.sub.j_rN.sub.l+αCI.sub.j_rN.sub.l′+βLI.sub.j_N.sub.l′; 2.3.3.6). to the layer fusion images other than the top-layer fusion images, for the reason of retaining the rich color information, assigning the layer fusion images of R channel according to equation Fu=.sub.j_w_r=CI.sub.j_rw, the layer fusion images of G channel according to equation Fu.sub.j_w_g=CI.sub.j_gw and the layer fusion images of B channel according to equation Fu.sub.j_w_b=CI.sub.j_bw, where w=1, 2, . . . , N.sub.l−1; 2.3.3.7). based on all layer fusion images of each color channel, performing N.sub.l-layer Laplacian pyramid composition to obtain fusion images of each color channel, thus the color fusion image of the j.sup.th pose is obtained and denoted by Fu.sub.j; 2.4). reconstructing a color point cloud: 2.4.1). performing color reconstruction on point cloud of each pose to obtain color point clouds {tilde over (P)}.sub.1, {tilde over (P)}.sub.2, . . . , {tilde over (P)}.sub.m: 2.4.1.1). to each point p.sub.ji(x.sub.ji, y.sub.ji, z.sub.ji) of the three-dimensional point cloud P.sub.j of the measured high-precision mechanical component of the j.sup.th pose, performing a coordinate system conversion from three-dimensional point to pixel point: $Z_c\left[\begin{array}{c}{a}_{ji}\\ b_{ji}\\ 1\end{array}\right]=\left[\begin{array}{cccc}{f}_x& 0& u_0& 0\\ 0& f_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}{R}_c& T_c\\ \stackrel{.fwdarw.}{0}& 1\end{array}\right]\left[\begin{array}{c}{x}_{ji}\\ y_{ji}\\ z_{ji}\\ 1\end{array}\right]$ where z.sub.c is a scale factor, f.sub.x and f.sub.y are respectively the focal lengths of camera in the x and y direction, u.sub.0 and v.sub.0 are respectively the coordinates of camera's main point on pixel plane, R.sub.c and T.sub.c are respectively the rotation matrix and the translation vector in camera calibration parameters, (a.sub.ji,b.sub.ji) are the coordinates of the pixel point q.sub.ij corresponding to the point p.sub.ji(x.sub.ji, y.sub.ji, z.sub.ji) of the three-dimensional point cloud P.sub.j, i is the serial number of the point; 2.4.1.2). obtaining pixel value R(a.sub.ji,b.sub.ji) of R channel, pixel value G(a.sub.ji,b.sub.ji) of G channel and pixel value B(a.sub.ji,b.sub.ji) of B channel of pixel point at coordinates (a.sub.ji,b.sub.ji) on the color fusion image Fu.sub.j, then assigning pixel values to the point p.sub.ji(x.sub.ji, y.sub.ji, z.sub.ji) of the three-dimensional point cloud P.sub.j according to the following equations: R(x.sub.ji,y.sub.ji,z.sub.ji)=R(a.sub.ji,b.sub.ji), G(x.sub.ji,y.sub.ji,z.sub.ji)=G(a.sub.ji,b.sub.ji) and B(x.sub.ji,y.sub.ji,z.sub.ji)=B(a.sub.ji,b.sub.ji) to obtain color point cloud {tilde over (P)}.sub.j, j=1, 2, . . . , m, where R(x.sub.ji, y.sub.ji, z.sub.ji), G(x.sub.ji, y.sub.ji, z.sub.ji) and B(x.sub.ji, y.sub.ji, z.sub.ji) are respectively the point values of R, G and B channel of point p.sub.ji(x.sub.ji, y.sub.ji, z.sub.ji) of color point cloud {tilde over (P)}.sub.j; 2.4.2). registering the color point clouds of all poses: based on the series of rigid body transformation matrices W.sub.1, W.sub.2, . . . , W.sub.m−1 which are obtained according to step 2.2, performing registration by respectively converting the color point clouds {tilde over (P)}.sub.2, {tilde over (P)}.sub.3, . . . , {tilde over (P)}.sub.m to the coordinate system of the color point cloud {tilde over (P)}.sub.1, where the color point clouds after conversion are respectively denoted by {tilde over (P)}.sub.2*, {tilde over (P)}.sub.3*, . . . , {tilde over (P)}.sub.m*, there have:
{tilde over (P)}.sub.2*=W.sub.1{tilde over (P)}.sub.2
{tilde over (P)}.sub.3*=W.sub.1W.sub.2{tilde over (P)}.sub.3
. . .
{tilde over (P)}.sub.m*=W.sub.1W.sub.2 . . . W.sub.m−1{tilde over (P)}.sub.m 2.4.3). fusing the overlapping part of the color point clouds: setting {tilde over (G)}.sub.1*={tilde over (P)}.sub.1*={tilde over (P)}.sub.1, then fusing the color point cloud {tilde over (P)}.sub.2* with the color point cloud {tilde over (G)}.sub.1* to obtain a color point cloud {tilde over (G)}.sub.2* to j=2, 3, . . . m−1, successfully fusing the color point cloud {tilde over (P)}.sub.j+1* with the color point cloud {tilde over (G)}.sub.j* to obtain a color point cloud {tilde over (G)}.sub.j+1*, and finally obtaining a fused overall color point cloud {tilde over (G)}.sub.m* that the high-precision true color three dimensional reconstruction is complete; where the fusion of the color point cloud {tilde over (P)}.sub.j+1* and the color point cloud {tilde over (G)}.sub.j* is: to the matching point pair in the point-to-point set C.sub.j_key_k′′, directly taking it as a correspondence point pair and putting it into a correspondence point pair set Co.sub.j; to a non-matching point {tilde over (p)}.sub.j+1_i*({tilde over (x)}.sub.j+1_i*, {tilde over (y)}.sub.j+1_i*, {tilde over (z)}.sub.j+1_i*) of the color point cloud {tilde over (P)}.sub.j+1*, finding the nearest point in the color point cloud {tilde over (P)}.sub.j+1* by using KD tree, and then calculating the average value σ of all distances between non-matching points and their respective nearest points; simultaneously, to each non-matching point of the color point cloud {tilde over (G)}.sub.j*, finding the nearest point in the color point cloud {tilde over (P)}.sub.j+1* by using KD tree, if the distance of the nearest point is less than the average value σ, then the non-matching point of the color point cloud {tilde over (G)}.sub.j* and its nearest point in the color point cloud {tilde over (P)}.sub.j+1* are taken as a correspondence point pair and put into the correspondence point pair set Co.sub.j; weighting geometric coordinates of each correspondence point pair {{tilde over (p)}.sub.j+1_i*({tilde over (x)}.sub.j+1_i*, {tilde over (y)}.sub.j+1_i*, {tilde over (z)}.sub.j+1_i*), {tilde over (g)}.sub.j_i*({tilde over (x)}.sub.j_i*, {tilde over (y)}.sub.j_i*, {tilde over (z)}.sub.j_i*)} in the correspondence point pair set Co.sub.j with the weights α.sub.i and β.sub.i to obtain the coordinates of a fused point {tilde over (g)}.sub.j+1_i*: (α.sub.i{tilde over (x)}.sub.j+1_i*+β.sub.i{tilde over (x)}.sub.j_i*,α.sub.i{tilde over (y)}.sub.j+1_i*+β.sub.i{tilde over (y)}.sub.j+1_i*,α.sub.i{tilde over (z)}.sub.j+1_i*+β.sub.iz.sub.j_i*) respectively weighting the point values of R, G and B channel of each correspondence point pair {{tilde over (p)}.sub.j+1_i*({tilde over (x)}.sub.j+1_i*, {tilde over (y)}.sub.j+1_i*, {tilde over (z)}.sub.j+1_i*), {tilde over (g)}.sub.j_i*({tilde over (x)}.sub.j_i*, {tilde over (y)}.sub.j_i*, {tilde over (z)}.sub.j_i*)} in the correspondence point pair set Co.sub.j with the weights (α.sub.ri,β.sub.ri), (α.sub.gi,β.sub.gi) and (α.sub.bi,β.sub.bi) to obtain the point values R({tilde over (g)}.sub.j+1_i*), G({tilde over (g)}.sub.j+1_i*) and B({tilde over (g)}.sub.j+1_i*) of R, G and B channel of the point {tilde over (g)}.sub.j+1_i* of the color point cloud {tilde over (G)}.sub.j+1*:
R({tilde over (g)}.sub.j+1_i*)=α.sub.riR({tilde over (p)}.sub.j+1_i*)+β.sub.riR({tilde over (g)}.sub.j_i*)
G({tilde over (g)}.sub.j+1_i*)=α.sub.giG({tilde over (p)}.sub.j+1_i*)+β.sub.giG({tilde over (g)}.sub.j_i*)
B({tilde over (g)}.sub.j+1_i*)=α.sub.biB({tilde over (p)}.sub.j+1_i*)+β.sub.biB({tilde over (g)}.sub.j_i*) where, i=1, 2, . . . , N.sub.coj, N.sub.coj is the number of the correspondence point pairs, R({tilde over (p)}.sub.j+1_i*), G({tilde over (p)}.sub.j+1_i*) and ({tilde over (p)}.sub.j+1_i*) are respectively the point values of R, G and B channel of the point {tilde over (p)}.sub.j+1_i*, R({tilde over (g)}.sub.j_i*), G({tilde over (g)}.sub.j_i*) and B({tilde over (g)}.sub.j_i*) are respectively the point values of R, G and B channel of the point {tilde over (g)}.sub.j_i* of the color point cloud {tilde over (G)}.sub.j+1*.

Description

BRIEF DESCRIPTION OF THE DRAWING

(1) The above and other objectives, features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

(2) FIG. 1 is a flow diagram of a method for high-precision true color three dimensional reconstruction of a mechanical component in accordance with the present invention;

(3) FIG. 2 is a diagram of fixation of three cameras and an optical transmitter;

(4) FIG. 3 is a diagram of fusing the left high-resolution image LI.sub.j with the low-resolution color image;

(5) FIG. 4 is a diagram of interpolating the pixel value of interpolation point by using the bilinear interpolation method;

(6) FIG. 5 is an illustration of a left high-resolution grayscale image of a dumbbell shaped standard component;

(7) FIG. 6 is an illustration of the fused overall color point cloud of a dumbbell shaped standard component under three poses;

(8) FIG. 7 is an illustration of a detail part of a direct fused overall color point cloud of a dumbbell shaped standard component under three poses;

(9) FIG. 8 is an illustration of a detail part of a fused overall color point cloud of a dumbbell shaped standard component under three poses in accordance with the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

(10) Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that the similar modules are designated by similar reference numerals although they are illustrated in different drawings. Also, in the following description, a detailed description of known functions and configurations incorporated herein will be omitted when it may obscure the subject matter of the present invention.

(11) In one embodiment of the present invention, As shown in FIG. 1, a method for high-precision true color three dimensional reconstruction of a mechanical component in accordance with the present invention comprises:

(12) Step S1: image acquisition

(13) Step S1.1: fixing cameras and optical transmitter

(14) As shown in FIG. 2, fixing a left high-resolution grayscale camera 1 and a right high-resolution grayscale camera 2 on a support device 5, where the right high-resolution grayscale camera 2 and the left high-resolution grayscale camera 1 are at the same height and spaced at a certain distance;

(15) fixing an optical transmitter 3 between the left high-resolution grayscale camera 1 and right high-resolution grayscale camera 2 and a low-resolution color camera 4 above the optical transmitter.

(16) Step S1.2: adjusting cameras and optical transmitter and completing a camera calibration

(17) Adjusting the left high-resolution grayscale camera 1, the right high-resolution grayscale camera 2, the optical transmitter 3 and the low-resolution color camera 4, so that a measured high-precision mechanical component can be completely and clearly captured by the cameras 1, 2, 4 and the optical transmitter 3 can completely and clearly project non-coded light and coded light on the surface of the measured high-precision mechanical component.

(18) The measurement user adjusts a calibration plate and uses Zhang's calibration method to complete a camera calibration.

(19) Step S1.3: taking images of measured high-precision mechanical component

(20) Placing the measured high-precision mechanical component at a suitable position in the view of the cameras 1, 2, 4, then rotating the measured high-precision mechanical component in a fixed direction and acquiring images of J poses:

(21) for the measured high-precision mechanical component of the j.sup.th pose, firstly the optical transmitter projects a non-coded light on the surface of the measured high-precision mechanical component, the low-resolution color camera takes an image which is denoted by low-resolution color image CI.sub.j, the left high-resolution grayscale camera takes an image which is denoted by left high-resolution grayscale image LI.sub.j, then the optical transmitter projects N frames of encoded light on the surface of the measured high-precision mechanical component, the left high-resolution grayscale camera and right high-resolution grayscale camera respectively take a grayscale image which are respectively denoted by LI.sub.jn and RI.sub.jn, n=1, 2, . . . , N.

(22) Step S2: image processing

(23) Step S2.1: performing a three-dimensional point cloud reconstruction for the measured high-precision mechanical component of each pose

(24) Reconstructing a three-dimensional point cloud of the measured high-precision mechanical component of each pose: based on the principle of binocular camera, reconstructing a three-dimensional point cloud P.sub.j of the j.sup.th pose by using the grayscale images LI.sub.j1, LI.sub.j2, . . . , LI.sub.jN and RI.sub.j1, RI.sub.j2, . . . , RI.sub.jN of the j.sup.th pose, where j=1, 2, . . . , J. this step belongs to the prior art, and no more detail are described here.

(25) Step S2.2: performing registrations on the 3D point clouds of poses to calculate a rigid body transformation matrix W.sub.j

(26) Performing registrations on 3D point clouds to calculate a rigid body transformation matrix W.sub.j by using the three-dimensional point cloud P.sub.j as a target point cloud and the three-dimensional point cloud P.sub.j+1 of the (j+1).sup.th pose as a source point cloud, where j=1, 2, . . . , J−1.

(27) Step S2.2.1: respectively performing voxel down sampling on the target point cloud P.sub.j and the source point cloud P.sub.j+1, where the voxel down sampled target point cloud is denoted by the target point cloud P.sub.j_down and the voxel down sampled source point cloud is denoted by the source point cloud P.sub.j+1_down, and respectively performing 3D-SIFT key point extraction on the target point cloud P.sub.j_down and the target point cloud P.sub.j+1_down, where the extracted target point cloud is denoted by a target point cloud P.sub.j_key, the extracted source point cloud is denoted by a source point cloud P.sub.j+1_key. Voxel down sampling is that voxels of N.sub.down % are retained at given number N.sub.down which is greater than 0 and less than 100. 3D-SIFT key point extraction can reduce the sample space, retain the points of rich features and reduce the amount of calculation.

(28) Step S2.2.2: initializing iteration number k=1, setting a maximum iteration number K.sub.c_reg, setting an iteration number K.sub.pp which is used for shifting a point-to-plane error function to a point-to-point error function, initializing transformed source point cloud P.sub.j+1_key_1 to the source point cloud P.sub.j+1_key.

(29) Step S2.2.3: for the k.sup.th iteration, to each point of the source point cloud P.sub.j+1_key_k, finding the nearest point in the target point cloud P.sub.j_key as its corresponding point by using KD tree to obtain a corresponding point-to-point set C.sub.j+_key_k, then removing the point and its nearest point whose distance is great than threshold Thres to obtain a optimized point-to-point set C.sub.j_key_k′.

(30) Step S2.2.4: homogenizing the i.sup.th point p.sub.j+1_key_ki of the source point cloud P.sub.j+1_key_k in point-to-point set C.sub.j_key_k′ to obtain its corresponding vector p.sub.j+1_key_ki=(x.sub.j+1_key_ki,y.sub.j+1_key_ki,z.sub.j+1_key_ki,1).sup.T, finding corresponding point p.sub.j_key_i of the target point cloud P.sub.j_key in point-to-point set C.sub.j_key_k′ and homogenizing it to obtain its corresponding vector p.sub.j_key_i=(x.sub.j_key_i,y.sub.j_key_i,z.sub.j_key_i,1).sup.T, where i=1, 2, . . . , N.sub.cj_key_k, N.sub.cj_key_k is the number of point-to-point set C.sub.j_key_k′;

(31) if iteration number k is less than K.sub.pp, then constructing a point-to-plane error function F.sub.jk by using the distance from the point p.sub.j+1_key_ki to the tangent plane of point p.sub.j_key_i of target point cloud P.sub.j_key, where the error function F.sub.jk is:

(32) $F_{jk}=\underset{i=1}{\overset{N_{\mathrm{cj}\_\mathrm{key}\_k}}{.Math.}}{\left(\left(W_{jk}.Math.{\stackrel{\_}{p}}_{\left(j+1\right)\_\mathrm{key}\_\mathrm{ki}}-{\stackrel{\_}{p}}_{j\_\mathrm{key}\_i}\right).Math.n_{j\_\mathrm{key}\_i}\right)}^2$

(33) where n.sub.j_key_i is the normal vector corresponding to point p.sub.j_key_i, W.sub.jk is the transformation matrix of the k.sup.th iteration;

(34) if iteration number k is nor less than K.sub.pp, then constructing a point-to-point error function F.sub.jk according to the sum of the squares of the distances between the corresponding points, where the error function F.sub.jk is:

(35) $F_{jk}=\underset{i=1}{\overset{N_{\mathrm{cj}\_\mathrm{key}\_k}}{.Math.}}{.Math.W_{jk}.Math.{\stackrel{\_}{p}}_{\left(j+1\right)\_\mathrm{key}\_\mathrm{ki}}-{\stackrel{\_}{p}}_{j\_\mathrm{key}\_i}.Math.}^2.$

(36) Step S2.2.5: calculating the transformation matrix W.sub.jk that makes the value of error function F.sub.jk minimum, and then applying a transformation to the source point cloud P.sub.j+1_key_k by using error function F.sub.jk to obtain a source point cloud P.sub.j+1_key_(k+1) and recording the value of error function F.sub.jk at the k.sup.th iteration.

(37) Step S2.2.6: judging whether the variation |F.sub.jk−F.sub.j(k−1)| of the value of error function of two consecutive iterations is less than a threshold e or iteration number k reaches the maximum iteration number K.sub.c_reg;

(38) if yes, then recording the point-to-point set C.sub.j_key_k′′ at the k′.sup.th iteration, where k′ is the iteration number when the judgment condition is met, and calculating the optimal rigid body transformation matrix W.sub.j=W.sub.jk,W.sub.j(k−1) . . . W.sub.j1, where the optimal rigid body transformation matrix W.sub.j is a matrix of 4×4, its formation is:

(39) $W_j=\left[\begin{array}{cc}{R}_{\mathrm{cloud}\_j}& T_{\mathrm{cloud}\_j}\\ \stackrel{.fwdarw.}{0}& 1\end{array}\right]$

(40) where R.sub.cloud_j is a rotation matrix of 3×3 which registers the source point cloud P.sub.j+1_key to the target point cloud P.sub.j_key, T.sub.cloud_j is a translation vector of 1×3, {right arrow over (0)} is a row vector of 3×1;

(41) if no, k=k+1, and returning step S2.2.3.

(42) Step S2.3: fusing the left high-resolution image LI.sub.j with the low-resolution color image CI.sub.j of the j.sup.th pose, j=1, 2, . . . , m

(43) Step S2.3.1: calculating a projection matrix H.sub.j for registering the low-resolution color image CI.sub.j to the left high-resolution image LI.sub.j:

(44) Step S2.3.1.1: respectively extracting SURF feature points from the low-resolution color image CI.sub.j and the left high-resolution image LI.sub.j, and taking the extracted SURF feature points as key points. In one embodiment, the extraction of SURF feature points is:

(45) construct different scale spaces based on box filter to form O-group S-layer image pyramid. Among them, the size of the box filter template used between groups gradually increases, the size of the filter template in different layers of the same group is the same, but the scale space factor increases layer by layer. Thus, an image composed of the determinant approximation of the Hessian matrix of each pixel in the original image is obtained in all scale spaces;

(46) the Hessian matrix discriminant value of each pixel processed by the Hessian matrix, 8 points in its neighborhood 3×3, and 3×3×2 points in the upper and lower adjacent layers of the same group are composed of 26 pixels for size comparison. If the value is less than or greater than the value of all the other 26 points, it is preliminarily recognized as a characteristic point. The key points with weak energy and the key points with wrong location are filtered out, and the final stable feature points are filtered out;

(47) for the extracted feature points, count the Harr wavelet features in the circular neighborhood. Take 0.2 rad step, count the sum of horizontal and vertical Harr wavelet features of all points in the π/3 sector, and take the direction of the maximum value as the main direction of the feature point;

(48) create a square box along the main direction on the feature points and decompose it into 4×4 squares. There are 5×5 pixels in each square. The horizontal direction relative to the main direction is marked as x and the vertical direction is marked as y. For these 25 pixels, the sum Σdx of the horizontal Harr wavelet eigenvalues relative to the main direction, the sum Σdy of the vertical Harr wavelet eigenvalues, the sum Σ|dx| of the absolute values of the horizontal Harr wavelet eigenvalues, and the sum Σ|dy| of the absolute values of the vertical Harr wavelet eigenvalues are calculated respectively, a total of four eigenvalues. For a feature point, record a total of 4×4×4=64 dimensional feature vectors.

(49) Step S2.3.1.2: to each key point of the low-resolution color image CI.sub.j, finding the closest key point and the next closest key point in the left high-resolution image LI.sub.j according to Euclidean distance, if the ratio of the distance of the closest key point to the distance of the next closet key point is less than a set threshold θ, then determining the closest key point in the left high-resolution image LI.sub.j as the matching point, thus a matching point pair set C.sub.cl_j is obtained.

(50) Step S2.3.1.3: setting a maximum iteration number K.sub.p_reg and an error threshold φ, initializing iteration number k=1, then optimizing the matching point pair set C.sub.cl_j through random sampling consistency (RANSAC) algorithm to obtain the projection matrix H.sub.j.

(51) Step S2.3.1.3.1: for the k.sup.th iteration in the process of optimizing the matching point pair set C.sub.cl_j, randomly selecting four point pairs, which are denoted respectively by {q.sub.cj_1, q.sub.lj_1}, {q.sub.cj_2, q.sub.lj_2}, {q.sub.cj_3, q.sub.lj_3} and {q.sub.cj_4, q.sub.lj_4}, and homogenizing them to obtain four point pairs {q.sub.cj_1, q.sub.lj_1}, {q.sub.cj_2, q.sub.lj_2}, {q.sub.cj_3, q.sub.lj_3} and {q.sub.cj_4, q.sub.lj_4}, then calculating a projection matrix H.sub.jk between the four point pairs, which satisfies the following relationship:
q.sub.lj_i=H.sub.jk.Math.q.sub.cj_i, i=1,2, . . . ,4

(52) Step S2.3.1.3.2: homogenizing a point pair {q.sub.cj_i, q.sub.lj_i} other than the selected four point pairs from the matching point pair set C.sub.cl_j to obtain point pair {q.sub.cj_i, q.sub.lj_i}, then calculating a mapping error ε.sub.jk_i=∥q.sub.lj_i,H.sub.jkq.sub.cj_i∥ according to the projection matrix H.sub.jk, where i=5, 5, . . . , N.sub.cj, N.sub.cj is the number of point pairs of the matching point pair set C.sub.cl_j, and selecting the point pair which mapping error ε.sub.jk_i<φ to form a interior point set S.sub.jk.

(53) Step S2.3.1.3.3: if the last iteration has been performed, namely k=K.sub.p_reg then comparing S.sub.j1, S.sub.j2, . . . , S.sub.jK.sub.p_reg to select the projection matrix H.sub.jk as the projection matrix H.sub.j when the interior point set S.sub.jk is the largest, or k=k+1 and returning step S2.3.1.3.1.

(54) Step S2.3.2: registering the low-resolution color image CI.sub.j to the left high-resolution image LI.sub.j according to the projection matrix H.sub.j to obtain a high-resolution registered color image CI.sub.j_r.

(55) Step S2.3.2.1: creating a empty high-resolution registered color image CI.sub.j_r which has the same size with the left high-resolution image LI.sub.j, then performing coordinate transformation on the pixels of the low-resolution color image CI.sub.j to obtain a color image CI.sub.j. In one embodiment, as shown in FIG. 3, the view range of the low-resolution color image CI.sub.j is Ω.sub.C, after coordinate transformation, its view range is not changed and still is Ω.sub.C, namely the view range of the color image CI.sub.j is Ω.sub.C. The pixel points after coordinate transformation have decimal coordinates, namely the decimal coordinates of the pixel points are not aligned. The view range of the left high-resolution image LI.sub.j is Ω.sub.L.

(56) Step S2.3.2.2: determining the interpolation range: the pixel points of the high-resolution registered color image CI.sub.j_r which are located on color image CI.sub.j and other than the matching points in matching point pair set C.sub.cl_j are the points which are needed to be interpolated, namely interpolation points. In one embodiment, as shown in FIG. 3, the overlapped view range is Ω.sub.CL. The pixel points within Ω.sub.CL and not in matching point pair set C.sub.cl_j are the points which are needed to be interpolated.

(57) Step S2.3.2.3: using the bilinear interpolation method to interpolate the pixel values of all interpolation points: to the i.sup.th interpolation point q.sub.cj_r_i(x.sub.cj_r_i, y.sub.cj_r_i), performing a inverse transformation according to the projection matrix H.sub.j to obtain a pixel point ρ.sub.cj_i(x.sub.cj_ρ_i, y.sub.cj_ρ_i) located on the low-resolution color image CI.sub.j, and finding four nearest pixel points which are denoted respectively by p.sub.cj_ρ_i11(x.sub.cj_ρ_i1, y.sub.cj_ρ_i1), p.sub.cj_ρ_i12(x.sub.cj_ρ_i1, y.sub.cj_ρ_i2), p.sub.cj_ρ_i21(x.sub.cj_ρ_i2, y.sub.cj_ρ_i1) and p.sub.cj_ρ_i22(x.sub.cj_ρ_i2, y.sub.cj_ρ_i2):

(58) firstly interpolating in the x direction to obtain the following pixel values:

(59) 0$C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i1}\right)=\frac{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i}}{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i1},y_{\mathrm{cj}\mathrm{\_\rho }\_i1}\right)+\frac{x_{\mathrm{cj}\mathrm{\_\rho }\_i}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i2},y_{\mathrm{cj}\mathrm{\_\rho }\_i1}\right)$$C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i2}\right)=\frac{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i}}{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i1},y_{\mathrm{cj}\mathrm{\_\rho }\_i2}\right)+\frac{x_{\mathrm{cj}\mathrm{\_\rho }\_i}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}{x_{\mathrm{cj}\mathrm{\_\rho }\_i2}-x_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i2},y_{\mathrm{cj}\mathrm{\_\rho }\_i2}\right)$

(60) then interpolating in the y direction to obtain the following pixel value:

(61) $C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i}\right)=\mathrm{Round}\left(\frac{y_{\mathrm{cj}\mathrm{\_\rho }\_i2}-y_{\mathrm{cj}\mathrm{\_\rho }\_i}}{y_{\mathrm{cj}\mathrm{\_\rho }\_i2}-y_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i1}\right)+\frac{y_{\mathrm{cj}\mathrm{\_\rho }\_i}-y_{\mathrm{cj}\mathrm{\_\rho }\_i1}}{y_{\mathrm{cj}\mathrm{\_\rho }\_i2}-y_{\mathrm{cj}\mathrm{\_\rho }\_i1}}C{I}_j\left(x_{\mathrm{cj}\mathrm{\_\rho }\_i},y_{\mathrm{cj}\mathrm{\_\rho }\_i2}\right)\right)$

(62) letting the pixel value CI.sub.j_r(x.sub.cj_r_i, y.sub.cj_r_i) of the interpolation point q.sub.cj_r_i(x.sub.cj_r_i, y.sub.cj_r_i) be CI.sub.j(x.sub.cj_ρ_i, y.sub.cj_ρ_i).

(63) In one embodiment, as shown in FIG. 4, the circles are the interpolation points of the high-resolution registered color image CI.sub.j_r. Taking the interpolation points in the lower right corner as an example, performing a inverse transformation according to the projection matrix H.sub.j to obtain a location on the low-resolution color image CI.sub.j. On the low-resolution color image CI.sub.j, the dots are the pixel points of the low-resolution color image CI.sub.j. In the embodiment, taking a interpolation point ρ.sub.cj_i(x.sub.cj_ρ_i, y.sub.cj_ρ_i) (the circle pointed to by an arrow on the low-resolution color image CI.sub.j) in the lower right corner as an example, the nearest four pixel points are p.sub.cj_ρ_i11(x.sub.cj_ρ_i1, y.sub.cj_ρ_i1), p.sub.cj_ρ_i12(x.sub.cj_ρ_i1, y.sub.cj_ρ_i2), p.sub.cj_ρ_i21(x.sub.cj_ρ_i2, y.sub.cj_ρ_i1) and p.sub.cj_ρ_i22(x.sub.cj_ρ_i2, y.sub.cj_ρ_i2), then interpolates in the x direction, namely, respectively interpolates the pixel value CI.sub.j(x.sub.cj_ρ_i, y.sub.cj_ρ_i1) of point ρ.sub.cj_i1(x.sub.cj_ρ_i1, y.sub.cj_ρ_i1) and the pixel value CI.sub.j(x.sub.cj_ρ_i, y.sub.cj_ρ_i2) of point ρ.sub.cj_i2(x.sub.cj_ρ_i, y.sub.cj_ρ_i2) by respectively using corresponding nearest pixel points in the x direction, finally interpolates in the x direction to obtain the pixel value of interpolation point ρ.sub.cj_i(x.sub.cj_ρ_i, y.sub.cj_ρ_i), i.e. the pixel value of interpolation point q.sub.cj_r_i(x.sub.cj_r_i, y.sub.cj_r_i).

(64) Step S2.3.2.4: to the pixel points of the high-resolution registered color image CI.sub.j_r which are located on color image CI.sub.j and are the matching points in matching point pair set C.sub.cl_j, directly using the pixel values of their corresponding matching point in the low-resolution color image CI.sub.j to assign; and to the pixel points of the high-resolution registered color image CI.sub.j_r which are not located on color image CI.sub.j, directly using preset RGB value to assign, thus the high-resolution registered color image CI.sub.j_r is obtained. In the embodiment, as shown in FIG. 4, the triangle points are the pixel points of the high-resolution registered color image CI.sub.j_r, which are located on color image CI.sub.j and are the matching points in matching point pair set C.sub.cl_j, and can be directly assigned by the pixel values of the corresponding points on the low-resolution color image CI.sub.j. After the interpolating and assigning, to the pixel points of the high-resolution registered color image CI.sub.j_r which are not located on color image CI.sub.j namely not the pixel points within the overlapped view range Ω.sub.CL, setting their RGB value to 0. As shown in FIG. 4, the black area on the high-resolution registered color image CI.sub.j_r are set to 0.

(65) Step S2.3.3: fusing the left high-resolution image LI.sub.j with the high-resolution registered color image CI.sub.j_r.

(66) Step S2.3.3.1: in one embodiment, as shown in FIG. 3, performing N.sub.l-layer Laplacian pyramid decomposition on the left high-resolution image LI.sub.j to obtain N.sub.l layers of image LI.sub.j_w, performing N.sub.l-layer Laplacian pyramid decomposition on each channel of the high-resolution registered color image CI.sub.j_r to obtain N.sub.l layers of image CI.sub.j_rw on R channel, N.sub.l layers of image CI.sub.j_gw on G channel and N.sub.l layers of image CI.sub.j_bw on B channel, where w=1, 2, . . . , N.sub.l and represents the serial number of image.

(67) Step S2.3.3.2: respectively performing bilateral filter on the top-layer images CI.sub.j_rN.sub.l, CI.sub.j_gN.sub.l, CI.sub.j_bN.sub.l and LI.sub.j_N.sub.l to obtain top-layer images CI.sub.j_rN.sub.l.sub._b, CI.sub.j_gN.sub.l.sub._b; CI.sub.j_bN.sub.l.sub._b and LI.sub.j_N.sub.l.sub._b;

(68) Step S2.3.3.3: to the top-layer images of three channels of the high-resolution registered color image CI.sub.j_r, calculating the difference before and after bilateral filtering to obtain detail layer images CI.sub.j_rN.sub.l′=CI.sub.j_rN.sub.l−CI.sub.j_rN.sub.l_b, CI.sub.j_gN.sub.l′=CI.sub.j_gN.sub.l−CI.sub.j_gN.sub.l.sub._b and CI.sub.j_bN.sub.l′=CI.sub.j_bN.sub.l−CI.sub.j_bN.sub.l.sub._b, which have rich details of image.

(69) Step S2.3.3.4: performing Gaussian filter on the top-layer image of the left high-resolution image LI.sub.j to obtain a overall smooth image LI.sub.j_N.sub.l.sub._g, then calculating the difference between the top-layer image LI.sub.j_N.sub.l.sub._b and the overall smooth image LI.sub.j_N.sub.l.sub._g to obtain a layer image LI.sub.j_N.sub.l′=LI.sub.j_N.sub.l.sub._b−LI.sub.j_N.sub.l.sub._g which has rich edge information.

(70) Step S2.3.3.5: calculating top-layer fusion images, where the top-layer fusion image of R channel is: Fu.sub.j_N.sub.l.sub._r=CI.sub.j_rN.sub.l+αCI.sub.j_rN.sub.l′+βLI.sub.j_N.sub.l′, the top-layer fusion image of G channel is: Fu.sub.j_N.sub.l.sub._g=CI.sub.j_gN.sub.l+αCI.sub.j_gN.sub.l′+βLI.sub.j_N.sub.l′, the top-layer fusion image of B channel is: Fu.sub.j_N.sub.l.sub._r=CI.sub.j_rN.sub.l+αCI.sub.j_rN.sub.l′+βLI.sub.j_N.sub.l′;

(71) Step S2.3.3.6: in one embodiment, as shown in FIG. 3, to the layer fusion images other than the top-layer fusion images, for the reason of retaining the rich color information, assigning the layer fusion images of R channel according to equation Fu.sub.j_w_r=CI.sub.j_rw, the layer fusion images of G channel according to equation Fu.sub.j_w_g=CI.sub.j_gw and the layer fusion images of B channel according to equation Fu.sub.j_w_b=CI.sub.j_bw, where w=1, 2, . . . , N.sub.l−1;

(72) Step S2.3.3.7: based on all layer fusion images of each color channel, performing N.sub.l-layer Laplacian pyramid composition to obtain fusion images of each color channel, thus the color fusion image of the j.sup.th pose is obtained and denoted by Fu.sub.j.

(73) Step S2.4: reconstructing a color point cloud:

(74) Step S2.4.1: performing color reconstruction on point cloud of each pose to obtain color point clouds {tilde over (P)}.sub.1, {tilde over (P)}.sub.2, . . . , {tilde over (P)}.sub.m:

(75) Step S2.4.1.1: to each point p.sub.ji(x.sub.ji, y.sub.ji, z.sub.ji) of the three-dimensional point cloud P.sub.j of the measured high-precision mechanical component of the j.sup.th pose, performing a coordinate system conversion from three-dimensional point to pixel point:

(76) $Z_c\left[\begin{array}{c}{a}_{ji}\\ b_{ji}\\ 1\end{array}\right]=\left[\begin{array}{cccc}{f}_x& 0& u_0& 0\\ 0& f_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}{R}_c& T_c\\ \stackrel{.fwdarw.}{0}& 1\end{array}\right]\left[\begin{array}{c}{x}_{ji}\\ y_{ji}\\ z_{ji}\\ 1\end{array}\right]$

(77) where z.sub.c is a scale factor, f.sub.x and f.sub.y are respectively the focal lengths of camera in the x and y direction, u.sub.0 and v.sub.0 are respectively the coordinates of camera's main point on pixel plane, R.sub.c and T.sub.c are respectively the rotation matrix and the translation vector in camera calibration parameters, (a.sub.ji,b.sub.ji) are the coordinates of the pixel point q.sub.ij corresponding to the point p.sub.ji (x.sub.ji, y.sub.ji, z.sub.ji) of the three-dimensional point cloud P.sub.j, i is the serial number of the point.

(78) Step S2.4.1.2: obtaining pixel value R(a.sub.ji,b.sub.ji) of R channel, pixel value G(a.sub.ji,b.sub.ji) of G channel and pixel value B(a.sub.ji,b.sub.ji) of B channel of pixel point at coordinates (a.sub.ji,b.sub.ji) on the color fusion image Fu.sub.j, then assigning pixel values to the point p.sub.ij(x.sub.ij, y.sub.ji, z.sub.ji) of the three-dimensional point cloud P.sub.j according to the following equations: R(x.sub.ji,y.sub.ji,z.sub.ji)=R(a.sub.ji,b.sub.ij), G(x.sub.ji,y.sub.ji,z.sub.ji)=G(a.sub.ji,b.sub.ji) and B(x.sub.ji,y.sub.ji,z.sub.ij)=B(a.sub.ji,b.sub.ji) to obtain color point cloud {tilde over (P)}.sub.j, j=1, 2, . . . , m, where R(x.sub.ji, y.sub.ji, z.sub.ji), G(x.sub.ji, y.sub.ji, z.sub.ji) and B(x.sub.ji, y.sub.ji, z.sub.ji) are respectively the point values of R, G and B channel of point p.sub.ji(x.sub.ji, y.sub.ji, z.sub.ji) of color point cloud {tilde over (P)}.sub.j.

(79) Step S2.4.2: registering the color point clouds of all poses:

(80) based on the series of rigid body transformation matrices W.sub.1, W.sub.2, . . . , W.sub.m−1 which are obtained according to step 2.2, performing registration by respectively converting the color point clouds {tilde over (P)}.sub.2, {tilde over (P)}.sub.3, {tilde over (P)}.sub.m to the coordinate system of the color point cloud {tilde over (P)}.sub.1, where the color point clouds after conversion are respectively denoted by {tilde over (P)}.sub.2, {tilde over (P)}.sub.3*, . . . , {tilde over (P)}.sub.m*, there have:
{tilde over (P)}.sub.2*=W.sub.1{tilde over (P)}.sub.2
{tilde over (P)}.sub.3*=W.sub.1W.sub.2{tilde over (P)}.sub.3
. . .
{tilde over (P)}.sub.m*=W.sub.1W.sub.2 . . . W.sub.m−1{tilde over (P)}.sub.m

(81) Step S2.4.3: fusing the overlapping part of the color point clouds:

(82) setting {tilde over (G)}.sub.1*={tilde over (P)}.sub.1*={tilde over (P)}.sub.1, then fusing the color point cloud {tilde over (P)}.sub.2* with the color point cloud {tilde over (G)}.sub.1* to obtain a color point cloud {tilde over (G)}.sub.2* to j=2, 3, . . . m−1, successfully fusing the color point cloud {tilde over (P)}.sub.j+1* with the color point cloud {tilde over (G)}.sub.j* to obtain a color point cloud {tilde over (G)}.sub.j+1* and finally obtaining a fused overall color point cloud {tilde over (G)}.sub.m* that the high-precision true color three dimensional reconstruction is complete;

(83) where the fusion of the color point cloud {tilde over (P)}.sub.j+1* and the color point cloud {tilde over (G)}.sub.j* is:

(84) to the matching point pair in the point-to-point set C.sub.j_key_k′′, directly taking it as a correspondence point pair and putting it into a correspondence point pair set Co.sub.j;

(85) to a non-matching point {tilde over (p)}.sub.j+1_i*({tilde over (x)}.sub.j+1_i*, {tilde over (y)}.sub.j+1_i*, {tilde over (z)}.sub.j+1_i*) of the color point cloud {tilde over (P)}.sub.j+1*, finding the nearest point in the color point cloud {tilde over (P)}.sub.j+1* by using KD tree, and then calculating the average value σ of all distances between non-matching points and their respective nearest points; simultaneously, to each non-matching point of the color point cloud {tilde over (G)}* finding the nearest point in the color point cloud {tilde over (P)}.sub.j+1* by using KD tree, if the distance of the nearest point is less than the average value σ, then the non-matching point of the color point cloud {tilde over (G)}.sub.j* and its nearest point in the color point cloud {tilde over (P)}.sub.j+1* are taken as a correspondence point pair and put into the correspondence point pair set Co.sub.j;

(86) weighting geometric coordinates of each correspondence point pair {{tilde over (p)}.sub.j+1_i*({tilde over (x)}.sub.j+1_i*, {tilde over (y)}.sub.j+1_i*, {tilde over (z)}.sub.j+1_i*), {tilde over (g)}.sub.j_i*({tilde over (x)}.sub.j_i*, {tilde over (y)}.sub.j_i*, {tilde over (z)}.sub.j_i*)} in the correspondence point pair set Co.sub.j with the weights α.sub.i and β.sub.i to obtain the coordinates of a fused point {tilde over (g)}.sub.j+1_i*:

(87) (α.sub.i{tilde over (x)}.sub.j+1_i*+β.sub.i{tilde over (x)}.sub.j_i*,α.sub.i{tilde over (y)}.sub.j+1_i*+β.sub.i{tilde over (y)}.sub.j_i*,α.sub.i{tilde over (z)}.sub.j+1_i*+β.sub.i{tilde over (z)}.sub.j_i*)

(88) respectively weighting the point values of R, G and B channel of each correspondence point pair {{tilde over (p)}.sub.j+1_i*({tilde over (x)}.sub.j+1_i*, {tilde over (y)}.sub.j+1_i*, {tilde over (z)}.sub.j+1_i*), {tilde over (g)}.sub.j_i*({tilde over (x)}.sub.j_i*, {tilde over (y)}.sub.j_i*, {tilde over (z)}.sub.j_i*)} in the correspondence point pair set Co.sub.j with the weights (α.sub.ri,β.sub.ri), (α.sub.gi,β.sub.gi) and (α.sub.bi,β.sub.bi) to obtain the point values R({tilde over (g)}.sub.j+1_i*), G({tilde over (g)}.sub.j+1_i*) and B({tilde over (g)}.sub.j+1_i*) of R, G and B channel of the point {tilde over (g)}.sub.j+1_i* of the color point cloud {tilde over (G)}.sub.j+1*:
R({tilde over (g)}.sub.j+1_i*)=α.sub.riR({tilde over (p)}.sub.j+1_i*)+β.sub.riR({tilde over (g)}.sub.j_i*)
G({tilde over (g)}.sub.j+1_i*)=α.sub.giG({tilde over (p)}.sub.j+1_i*)+β.sub.giG({tilde over (g)}.sub.j_i*)
B({tilde over (g)}.sub.j+1_i*)=α.sub.biB({tilde over (p)}.sub.j+1_i*)+β.sub.biB({tilde over (g)}.sub.j_i*)

(89) where, i=1, 2, . . . , N.sub.coj, N.sub.coj is the number of the correspondence point pairs, R({tilde over (p)}.sub.j+1_i*), G({tilde over (p)}.sub.j+1_i*) and ({tilde over (p)}.sub.j+1_i*) are respectively the point values of R, G and B channel of the point {tilde over (p)}.sub.j+1_i*, R({tilde over (g)}.sub.j_i*), G({tilde over (g)}.sub.j_i*) and B({tilde over (g)}.sub.j_i*) are respectively the point values of R, G and B channel of the point {tilde over (g)}.sub.j_i* of the color point cloud {tilde over (G)}.sub.j+1*.

Example

(90) In this example, a dumbbell shaped standard component is reconstructed with high-precision color 3D point cloud. Turn on the left and right high-resolution gray cameras, the optical transmitter and the low-resolution color camera, and adjust the angles and parameters of the cameras so that the dumbbell shaped standard component can be completely and clearly captured by the cameras, then adjust the angles and parameters of the optical transmitter so that the optical transmitter can completely and clearly project non-coded light and coded light on the surface of the dumbbell shaped standard component.

(91) The measurement user places a checkerboard calibration plate at a suitable position in the view of the cameras, rotates the checkerboard calibration plate in the same plane at different angles, takes a group of high-resolution gray images of the checkerboard calibration plate and transmits them to a computer. The compute calculates the parameters of the left and right high-resolution gray cameras through Zhang's calibration method. The measurement user places the dumbbell shaped standard component at a suitable position in the view of the cameras and records it as pose 1. When the optical transmitter projects the non-coded light, the measurement user takes a group of images, including the low-resolution color image and the left high-resolution grayscale image which shown in FIG. 5. When the optical transmitter projects the coded light, a group of images of the left and right high-resolution gray cameras are acquired and transmitted to the computer. The measurement user rotates the dumbbell shaped standard component at a certain angle and records it as pose 2. a group of images are acquired and transmitted to the computer. The tester rotates the dumbbell shaped standard part by a certain angle along the same direction and records it as pose 3, another of images are acquired and transmitted to the computer. Thus the images of J poses have been acquired, the image acquisition is complete, where J=3. Then the computer uses an image processing program to complete the reconstruction of color point clouds, and fuses the color point clouds under three poses together, and the fused overall color point cloud is shown in FIG. 6.

(92) FIG. 7 shows the details of the stitching of three pose point clouds. It can be seen that the direct stitching of each pose leads to uneven resolution of the overall point cloud, the density of the overlapping area is higher than that of other parts, and the points with the same name are given different color channel values. The spliced point clouds are fused to obtain the overall point cloud shown in FIG. 8. Compared with FIG. 7, the details (shown in white oval circle) in FIG. 8 do not appear layered, and the values of points with the same name are processed uniformly, and the redundant point cloud information is deleted. The point cloud reconstruction obtained by the invention realizes the restoration of high precision and real color.

(93) While illustrative embodiments of the invention have been described above, it is, of course, understand that various modifications will be apparent to those of ordinary skill in the art. Such modifications are within the spirit and scope of the invention, which is limited and defined only by the appended claims.