Point cloud denoising method based on deep learning for aircraft part

Abstract

The present disclosure provides a point cloud denoising method based on deep learning for an aircraft part, in which different degrees of Gaussian noise are added based on a theoretical data model of the aircraft part, a heightmap for each point in the theoretical data model is generated, and a deep learning training set is constructed. A deep learning network is trained based on the constructed deep learning training set, to obtain a deep learning network model. A real aircraft part is scanned via a laser scanner to obtain measured point cloud data. The normal information of the measured point cloud is predicted based on the trained deep learning network model. Based on the predicted normal information, a position of each point in the measured point cloud data is further updated, thereby completing denoising of the measured point cloud data.

Claims

1. A point cloud denoising method based on deep learning for an aircraft part, comprising: S1) based on a theoretical data model of the aircraft part, adding different degrees of Gaussian noise; generating a heightmap for each of points in the theoretical data model; and constructing a deep learning training set; S2) based on the deep learning training set constructed in S1, training a deep learning network to obtain a deep learning network model; S3) scanning a real aircraft part via a laser scanner to obtain measured point cloud data; and predicting normal information of the measured point cloud based on the deep learning network model trained in S2; and S4) based on the normal information predicted in S3, further updating a position of each of points in the measured point cloud data, thereby completing denoising of the measured point cloud data; wherein a process of generating the heightmap for each of points in the theoretical data model in S1 comprises: S11) presetting a point p in point cloud data from the theoretical data model; and constructing a local neighborhood of the point by using a recurrent neural network; S12) calculating a covariance matrix C.sub.pi, according to a formula (1) as: $\begin{array}{cc}{C}_{pi}=\frac{1}{.Math.\Psi .Math.}\underset{p_j\in \Psi }{.Math.}\left(p_i-p_j\right){\left(p_i-p_j\right)}^T;& \left(1\right)\end{array}$ wherein p.sub.i is the point in the point cloud data; p.sub.i=(x, y, z); (x.sub.i, y.sub.i, z.sub.i) is a coordinate of the point p.sub.i; p.sub.j is a point in the local neighborhood; T is the local neighborhood of the point p.sub.i; and |Ψ| is the number of points in the local neighborhood; S13) defining three eigenvalues and three eigenvectors of the covariance matrix C.sub.pi as λ.sub.1, λ.sub.2, λ.sub.3 and μ.sub.1, μ.sub.2, μ.sub.3, respectively, wherein λ.sub.1≥λ.sub.2≥λ.sub.3, and μ=(R, S, T); S14) taking the vector u.sub.3 as a normal vector of a projection plane, and constructing the projection plane P according to a formula (2) as:
R(x−x.sub.i)+S(y−y.sub.i)+T(z−z.sub.i)=0  (2); wherein (x, y, z) is a coordinate of a point on the projection plane P; and (x.sub.i, y.sub.i, z.sub.i) is the coordinate of the point p.sub.i; and S15) based on the projection plane P constructed in S14, all points in the local neighborhood are projected to the projection plane P along the vector μ.sub.3 to obtain a heightmap of the point p.sub.i; wherein S3 further comprises: S31) presetting a point p.sub.m in the measured point cloud data, and constructing a local neighborhood of the point by using a recurrent neural network; S32) calculating a covariance matrix C.sub.pm, according to a formula (4) as: wherein p.sub.m is the point in the measured point cloud data: p.sub.n is a point in the local neighborhood: Γ is a local neighborhood of p.sub.m; and |Γ| is the number of points in the local neighborhood; S33) respectively defining three eigenvalues and three eigenvectors of the covariance matrix C.sub.pm as λ′.sub.1, λ′.sub.2, λ′.sub.3 and μ′.sub.1, μ′.sub.2, μ′.sub.3, wherein λ′.sub.1≥λ′.sub.2≥λ′.sub.3, and μ′.sub.3=(R′, S′, T′); S34) taking the vector μ′, as a normal vector of a projection plane, and constructing the projection plane P′ according to a formula (2) as:
R′(x′−x′.sub.i)+S′(y′−y′.sub.i)+T′(z′−z′.sub.i)=0  (5) wherein (x′, y′, z′) is a coordinate of a point on the projection plane P′; and (x.sub.i, y.sub.i, z.sub.i) is a coordinate of p.sub.i; S35) based on the projection plane P′ constructed in S34, projecting all points in the local neighborhood to the projection plane P′ along the vector μ′, to obtain a heightmap of p.sub.m; and S36) inputting the heightmap generated in S35 into the deep learning network model obtained in S2, to obtain a normal vector of the point p.sub.m; in S4, the position of each of the points in the measured point cloud data is updated according to a formula (6) as: wherein p.sub.m is a point in the measured point cloud data; p.sub.n is a point in the local neighborhood: α.sub.m is a weight parameter: Γ is a local neighborhood of p.sub.m; n.sub.n is a normal vector of p.sub.n; and n.sub.m is a normal vector of p.sub.m.

2. The point cloud denoising method of claim 1, wherein a process of training the deep learning network to obtain the deep learning network model comprises: S21) selecting ResNet18 as a deep learning network, and constructing a loss function for the deep learning network as: $\begin{array}{cc}\mathrm{Loss}=\underset{p_i\in Q}{.Math.}{.Math.n_{pi}-{\stackrel{⌣}{n}}_{pi}.Math.}^2;& \left(3\right)\end{array}$ wherein p.sub.i is a point in the pint cloud data; Q is a point set comprising all points in the deep learning training set; n.sub.pi is a normal of the point p.sub.i predicted by the deep learning network; and n.sub.pi is a theoretical normal of the point p.sub.i; and S22) inputting the heightmap generated in S1 into the ResNet18 network; updating a weight of the ResNet18 network using a stochastic gradient descent method and a backpropagation algorithm; stopping training until a network loss value area converges, thereby obtaining the deep learning network model.

3. The point cloud denoising method of claim 2, wherein in S22, an initial learning rate is 0.001, and a preset maximum number of iterative training is 200.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a flowchart of a point cloud denoising method based on deep learning for an aircraft part according to an embodiment of the present disclosure.

(2) FIG. 2 is a diagram showing original measured data of the aircraft part according to an embodiment of the present disclosure.

(3) FIG. 3 shows a heightmap of a point cloud of the aircraft part according to an embodiment of the present disclosure.

(4) FIG. 4 shows a result of denoising the point cloud of the aircraft part according to an embodiment of the present disclosure.

(5) FIG. 5 shows a normal estimation result for the point cloud according to an embodiment of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

(6) The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

(7) Referring to FIG. 1, the point cloud denoising method based on deep learning for an aircraft part provided in the embodiment can be directly applied to scanning data processing devices in various aircraft parts, and the method can be realized by writing a corresponding program in a controller of the scanning data processing device in the specific implementation. FIG. 2 shows a scanning device of the aircraft part and the acquired 3D point cloud data. The acquired 3D point cloud data has problems such as noise interference, which has a great impact on the detection and analysis of the aircraft part. Therefore, the embodiment provides a point cloud denoising method based on deep learning for an aircraft part, including the following steps.

(8) S1) Based on a theoretical data model of the aircraft part, different degrees of Gaussian noise are added. A heightmap for each point in the theoretical data model is generated. And a deep learning training set is constructed. In the embodiment, the step that the heightmap for each point in the theoretical data model is generated includes the following steps.

(9) S11) A point p.sub.i in the point cloud data is given from the theoretical data model. And a local neighborhood of the point is constructed by using a recurrent neural network. Preferably, the neighborhood radius is specifically 5 times the length of an average distance of the point cloud model.

(10) S12) A covariance matrix C.sub.P, is calculated according to a formula (13) as follows:

(11) $\begin{array}{cc}{C}_{pi}=\frac{1}{.Math.\Psi .Math.}\underset{p_j\in \Psi }{.Math.}\left(p_i-p_j\right){\left(p_i-p_j\right)}^T;& \left(13\right)\end{array}$

(12) where p.sub.i is the point in the point cloud data; p.sub.i=(x.sub.i, y.sub.i, z.sub.i), (x.sub.i, y.sub.i, z.sub.1) is a coordinate of p.sub.i; p.sub.i is a point in the local neighborhood; Ψ is a local neighborhood of p.sub.i; and |Ψ| is the number of points in the local neighborhood;

(13) S13) Three eigenvalues and three eigenvectors of the covariance matrix C.sub.pi are respectively defined as λ.sub.1, λ.sub.2, λ.sub.3 and u.sub.1, u.sub.2, u.sub.3, where λ.sub.1≥λ.sub.2≥λ.sub.3, and u.sub.3=(R, S, T).

(14) S14) The vector u.sub.3 is taken as a normal vector of a projection plane, and the projection plane P is constructed according to a formula (14) as:
R(x−x.sub.i)+S(y−y.sub.i)+T(z−z.sub.i)=0  (14);

(15) where (x, y, z) is a coordinate of a point on the projection plane P; and (x.sub.i, y.sub.i, z.sub.i) is the coordinate of p.sub.i.

(16) S15) Based on the constructed projection plane P, all points in the local neighborhood are projected to the projection plane P along the vector u.sub.3 to obtain a heightmap of p.sub.i, in which a resolution thereof is set to 48×48, as shown in FIG. 3.

(17) S2) Based on the deep learning training set constructed in S1, a deep learning network is trained to obtain a deep learning network model. In the embodiment, ResNet18 is selected as the deep learning network, and a loss function for the deep learning network is constructed as follows:

(18) $\begin{array}{cc}\mathrm{Loss}=\underset{p_i\in Q}{.Math.}{.Math.n_{pi}-{\stackrel{⌣}{n}}_{pi}.Math.}^2;& \left(15\right)\end{array}$

(19) where p.sub.i is a point in the point cloud data; Q is a point set including all points in the deep learning training set; n.sub.pi is a normal of the point p.sub.i predicted by the deep learning network; and ň.sub.pi, is a theoretical normal of the point p.sub.i.

(20) S22) The heightmap generated in S1 is inputted into the ResNet18 network. A weight of the ResNet18 network is updated using a stochastic gradient descent method and a backpropagation algorithm, in which an initial learning rate is 0.001, and a preset maximum number of iterative training is 200. The training is stopped until a network loss value area converges, thereby obtaining the deep learning network model.

(21) S3) A real aircraft part is scanned via a laser scanner to obtain measured point cloud data. The normal information of the measured point cloud is predicted based on the deep learning network model trained in S2.

(22) S31) A point p.sub.m in the measured point cloud data is given, and a local neighborhood of the point is constructed by using a recurrent neural network. Specifically, the neighborhood radius is 3 times the length of an average distance of the point cloud model.

(23) S32) A covariance matrix C.sub.pm is calculated according to a formula (16) as follows:

(24) $\begin{array}{cc}{C}_{pm}=\frac{1}{.Math.\Gamma .Math.}\underset{p_n\in \Gamma }{.Math.}\left(p_m-p_n\right){\left(p_m-p_n\right)}^T;& \left(16\right)\end{array}$

(25) where p.sub.m is the point in the measured point cloud data; p.sub.n is a point in the local neighborhood; Γ is a local neighborhood of p.sub.m; and |Γ| is the number of points in the local neighborhood.

(26) S33) Three eigenvalues and three eigenvectors of the covariance matrix C.sub.pm are defined as λ′.sub.1, λ′.sub.2, λ′.sub.3 and u′.sub.1, u′.sub.2, u′.sub.3, respectively, where λ′.sub.1≥λ′.sub.2≥λ′.sub.3 and u′.sub.3=(R′, S′, T′).

(27) S34) The vector u′.sub.3 is taken as a normal vector of a projection plane, and the projection plane P′ is constructed according to a formula (17) as follows:
R′(x′−x′.sub.i)+S′(y′-y′.sub.i)+T′(z′−z′.sub.i)=0  (17);

(28) where (x′, y′, z′) is a coordinate of a point on the projection plane P′; and (x.sub.i, y.sub.i, z.sub.i) is a coordinate of p.sub.i.

(29) S35) Based on the constructed projection plane P′, all points in the local neighborhood are projected to the projection plane P′ along the vector u′.sub.3 to obtain a heightmap of p.sub.m.

(30) S36) The heightmap generated in S35 is inputted into the deep learning network model obtained in S2 to obtain a normal vector of the point p.sub.m.

(31) S4) Based on the normal information predicted in S3, a position of each point in the measured point cloud data is further updated. The point update function is designed to obtain the updated point cloud, i.e., the point cloud model after denoising, thereby completing denoising of the measured point cloud data. Specifically, the position of each point in the measured point cloud data is updated according to a formula (18) as follows:

(32) $\begin{array}{cc}{\tilde{P}}_m=p_m+{\alpha }_m\underset{p_n\in \Gamma }{.Math.}\left(p_m-p_n\right)\left({n_n}^T{n}_n+{n_m}^T{n}_m\right);& \left(18\right)\end{array}$

(33) where p.sub.m is a point in the measured point cloud data; p.sub.n is a point in the local neighborhood; α.sub.m is a weight parameter, which controls a step length of the point update, and its specific value is

(34) $\frac{1}{3.Math.\Gamma .Math.};$
Γ is a local neighborhood of the point p.sub.m; n.sub.n is a normal vector of the point p.sub.n; and n.sub.m is a normal vector of the point p.sub.m.

(35) FIGS. 4-5 respectively show a point cloud model after the point update (i.e., a model after denoising) and a normal estimated by deep learning. Compared to FIG. 2, it can be seen that the noise problem of the point cloud data has been solved, while the features such as the component structure have not changed. Therefore, the method provided in the present disclosure can accurately complete a denoising processing of the measured point cloud data of the aircraft part, while retaining structural characteristics of the aircraft part, so that the problems of low accuracy and low efficiency in traditional detection methods are effectively solved, thereby ensuring efficiency and accuracy of subsequent detection and analysis for the structure of the aircraft part.

(36) Described above is only a preferred embodiment of the present disclosure. It should be pointed out that, any modification and improvement made by those skilled in the art without departing from the spirit of the present disclosure shall fall within the scope of the appended claims.