POINTEFF METHOD FOR URBAN OBJECT CLASSIFICATION WITH LIDAR POINT CLOUD DATA

Abstract

The present disclosure relates to a PointEFF method for urban object classification with LiDAR point cloud data, and belongs to the field of LiDAR point cloud classification. The method comprises: point cloud data segmentation; End-to-end feature extraction layer construction; External feature fusion layer construction; and precision evaluation. The PointEFF method for urban object classification with LiDAR point cloud data fuses point cloud hand-crafted descriptors with End-to-end features obtained from a network at an up-sampling stage of a model by constructing an External Feature Fusion module, which improves a problem of local point cloud information loss caused by interpolation operation in the up-sampling process of domain feature pooling methods represented by PointNet and PointNet++, greatly improves classification precision of the model in complex ground features, especially in rough surface ground features, and is capable of being better applied to the classification of urban ground features with complex ground feature types.

Claims

1. A PointEFF method for urban object classification with LiDAR point cloud data, comprising: (1) obtaining more abundant point cloud geometric structure information by extracting point cloud hand-crafted descriptors, which solves a defect that a traditional domain feature pooling method based on deep learning does not consider structural information between points when extracting point cloud local features; and (2) constructing an External Feature Fusion module in an up-sampling stage of a model, which improves a problem of local point cloud information loss caused by interpolation operation in the up-sampling process of the traditional domain feature pooling method based on deep learning, and greatly improves classification precision of the model in complex ground feature classification, especially in rough surface classification; and the method further comprising: point cloud data segmentation, End-to-end feature extraction layer construction and External feature fusion layer construction.

2. The PointEFF method for urban object classification with LiDAR point cloud data according to claim 1, wherein the point cloud data is segmented according to the following steps: setting a set of points as R, wherein a first point is p.sub.1=(x.sub.1,y.sub.1,z.sub.1)R, and the rest points are p.sub.R-1=(x.sub.R-1,y.sub.R-1,z.sub.R-1)R, and a Euclidean distance from p.sub.R-1 to p.sub.1 is:
d.sub.R-1={square root over ((x.sub.R-1x.sub.1).sup.2+(y.sub.R-1y.sub.1).sup.2+(z.sub.R-1z.sub.1).sup.2)} according to the Euclidean distance from each sample point to p.sub.1 and a number of points m in a segmentation region, dividing a field of p.sub.1 as {p.sub.1, p.sub.1 . . . , p.sub.m} and then calculating distances from the rest points in the set of points to a point p.sub.m+1 iteratively until all the points in the set of points are divided.

3. The PointEFF method for urban object classification with LiDAR point cloud data according to claim 1, wherein the End-to-end feature extraction layer is constructed according to the following steps: the End-to-end feature extraction layer comprising a network encoder and a network decoder; processing and abstracting, by the encoder, a group of points through an abstract set operation to recursively extract multi-scale features of a point cloud local region; gradually recovering, by the decoder, a spatial dimension through a feature propagation operation, fusing the features extracted in the coding process, and completing input and output of the same scale on the premise of reducing information loss as much as possible; and transferring, by the encoder and the decoder, features of the same scale through two groups of jump link modules; the network encoder comprising thrice abstract set operations, wherein the abstract set consists of a sampling layer, a grouping layer and a feature extraction layer; firstly, inputting N LiDAR points with three-dimensional attributes (x, y, z) into the proposed PointEFF network, and selecting a point N from the sampling layer selects by an iterative Farthest Point Sampling algorithm to define N centroids of the local region; then, in the grouping layer, setting a radius r by a query ball algorithm, and searching adjacent k points in the centroid range r to construct a local region; after implementing the sampling layer and the grouping layer, sampling the LiDAR points into N central clusters, each central cluster contains k points and 36-dimensional attributes thereof, and outputting a group of set of points with a size of Nk36; finally, encoding the local regions into feature vectors through the feature extraction layer; inputting the set of points into an MLP network, and outputting NkC, wherein C is a feature extracted by MLP, max-pooling each central cluster to select the largest feature in each central cluster as a regional feature, and outputting NC; carrying out thrice abstract set operations until a global feature of 11024 is output; the network decoder consisting of thrice feature propagation operations and two groups of jump link modules, gradually recovering the spatial dimension by using an up-sampling operation, fusing the features extracted during the encoding process, and completing input and output of the same scale on the premise of reducing information loss as much as possible; in a feature propagation layer, in order to propagate learned features from a sampling point to an original point, interpolating an NC dimensional feature map obtained by the encoder firstly by using an Inverse Distance Weighted algorithm, calculating distances from each point to be interpolated to all the points, calculating weights, and interpolating the number of points from N to N to obtain an interpolated NC dimensional feature map; then linking, by the jump link modules, the C dimensional feature map obtained by the encoder at the same scale to obtain an N(C+C) dimensional feature map; and finally, obtaining an NC dimensional feature map through the multi-layer perceptron (MLP); and obtaining an N128 dimensional feature map after three feature propagation operations.

4. The PointEFF method for urban object classification with LiDAR point cloud data according to claim 1, wherein the External feature fusion layer is constructed according to the following steps: the External feature fusion layer comprising extraction of hand-crafted descriptors and the External Feature Fusion module; and the method selecting a fast point feature histogram as an input of the External Feature Fusion module; obtaining a normal of point cloud by plane fitting with least square method, and establishing a local coordinate system between two points according to obtained normal vectors: $={}_s=\begin{array}{c}\left(p_t-p_s\right)\\ {.Math.p_t-p_s.Math.}^2\end{array}=$ a difference between point normal pairs being capable of being expressed by the following angles: $-.Math.{}_t=.Math.\begin{array}{c}\left(p_t-p_s\right)\\ d\end{array}=\mathrm{arc}\tan \left(.Math.{}_t,.Math.{}_t\right)d={.Math.p_t-p_s.Math.}_2$ quantizing the angles to form a point feature histogram PFH; representing features of the fast point feature histogram as: $\mathrm{FPFH}\left(p_q\right)=\mathrm{PFH}\left(p_q\right)+\frac{1}{k}{.Math.}_{i=1}^k\frac{1}{{}_k}.Math.\mathrm{PFH}\left(p_k\right)$ after obtaining a fast point cloud point feature histogram, transferring the N128 dimensional feature map obtained from the End-to-end feature extraction layer and the N33 dimensional feature histogram into the External Feature Fusion module as inputs; in the External Feature Fusion module, the technical feature obtained by the End-to-end feature extraction layer being:
xR.sup.128 the feature of the fast point feature histogram being:
yR.sup.33 a new feature histogram obtained by a concatenate operation being:
z[x,y]R.sup.17R+33 after the concatenate operation is completed, obtaining the N128 dimensional feature map through the multi-layer perceptron, that is, each point having 128 dimensional features; and finally, inputting the feature map, and obtaining a point cloud category label through one-dimensional convolution, thus completing the PointEFF method for urban object classification with LiDAR point cloud data.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0038] FIG. 1 is a technical flow chart of the present disclosure patent.

[0039] FIG. 2 is a situation map of training data segmentation.

[0040] FIG. 3 is a situation map of test data segmentation.

[0041] FIG. 4 is a structural drawing of a PointEFF End-to-end feature extraction layer.

[0042] FIG. 5 is a structural drawing of a PointEFF External feature fusion layer.

[0043] FIG. 6 is a classification result diagram of a PointEFF network.

DETAILED DESCRIPTION

[0044] The present invention disclosure will be further illustrated below with reference to the accompany drawings and specific embodiments.

[0045] Under a Windows operating system, PyTorch is selected as a platform to build a PointEFF network, and effectiveness of the network is verified on an NMP3D Benchmark Suite data set.

[0046] The following steps are specifically comprised.

[0047] At step 1, with reference to FIG. 1 and FIG. 2, point cloud segmentation and distributions of training data and test data are illustrated. The NMP3D data set is segmented into 83 training regions according to 10000 points, of which 60 regions in MiniLille1, MiniLille2 and MiniParis 1_1 are training data and 23 regions in MiniParis1_2 are test data.

[0048] At step 2, construction of an End-to-end feature extraction layer of the PointEFF network is illustrated with reference to FIG. 3. The End-to-end feature extraction layer consists of an encoder part and a decoder part, wherein the encoder part comprises thrice abstract set operations, while the decoder part comprises thrice feature propagation operations and two groups of jump link modules.

[0049] Setting N=2048, 2048 LiDAR points with three dimensional position attributes are input into the network, and 512 central points are selected by an iterative Farthest Point Sampling algorithm in a sampling layer. In a grouping layer, it is set that a query radius r=0.2 m, and a number of query points k=32, and 32 adjacent points within 0.2 m of the centroid are searched to construct a central cluster, and a 512256 dimensional feature map is output through a feature extraction layer. The first abstraction set operation is completed.

[0050] For the 512256 dimensional feature map obtained by the first abstraction set operation, 128 central points are selected by an iterative Farthest Point Sampling algorithm in the sampling layer. In the grouping layer, it is set that a query radius r=0.4 m, and a number of query points k=64, and 64 adjacent points within 0.4 m of the centroid are searched to construct a central cluster, and a 128256 dimensional feature map is output through the feature extraction layer. The second abstraction set operation is completed.

[0051] The 128256 dimensional feature map obtained by the second abstraction set operation is subjected to the third abstract set operation to finally obtain 11024 global region features. The network encoder part is designed.

[0052] In first feature propagation, firstly, the 11024 dimensional feature map obtained by the encoder is copied to obtain a 1281024 dimensional feature map, then the 256 dimensional feature map obtained by the encoder at the same scale is linked by the jump link module to obtain a 128(1024+256) dimensional feature map, and finally a 128256 dimensional feature map is obtained by a multi-layer perceptron (MLP). The first feature propagation is completed.

[0053] In second feature propagation, the 128256 dimensional feature map obtained in the first feature propagation layer is interpolated by using an Inverse Distance Weighted algorithm (IDW) to obtain a 512256 dimensional feature map, and then the 128 dimensional feature map obtained by the encoder at the same scale is linked by the jump link module to obtain a 512(256+128) dimensional feature map, and finally a 512128 dimensional feature map is obtained by the multi-layer perceptron (MLP). The second feature propagation is completed.

[0054] In third feature propagation, the 512128 dimensional feature map obtained in the second feature propagation layer is interpolated by using the Inverse Distance Weighted algorithm (IDW) to obtain a 2048128 dimensional feature map, and finally a 2048128 dimensional feature map is obtained by the multi-layer perceptron (MLP). The third feature propagation is completed. The network decoder part is designed.

[0055] At step 3, construction of an External feature fusion layer of the network is illustrated with reference to FIG. 3.

[0056] Under the Windows operating system, a Point Cloud Library (PCL) is selected as a platform to extract a fast point feature histogram. A radius is set to be 0.03 m and a normal of point cloud is calculated. On the basis of the normal of point cloud extracted, the radius is set to be 0.04 m, and the fast point feature histogram is calculated. The fast point feature histogram obtained by calculation is stored in a pcd document. Hand-crafted descriptors extraction is completed.

[0057] After the hand-crafted descriptors extraction is, the N128 dimensional feature map obtained from the End-to-end feature extraction layer and the N33 dimensional feature histogram are transferred into the External Feature Fusion module as inputs. In the External Feature Fusion module, an N(128+33) dimensional feature map is obtained by concatenating and fusing the features obtained from the End-to-end feature extraction layer and the extracted manual design descriptors. After that, the fused feature map is used as an input of the multi-layer perceptron, and a 2048128 dimensional feature map is obtained.

[0058] Finally, the feature map is input, a category label is obtained through one-dimensional convolution, and the classification is completed.

[0059] At step 5, the PointEFF classification effects are illustrated with reference to FIG. 6.

[0060] The overall precision of the PointEFF classification is shown in Table 1, and the classification precision results of each category are shown in Table 2. It can be seen that most categories are correctly classified, especially in the classification of buildings and other scenes with rough surfaces.

TABLE-US-00001 TABLE 1 Overall precision results of PointEFF classification Evaluation index OA (%) MIoU (%) F1-score (%) Kappa (%) PointEFF 0.9792 0.7664 0.8455 0.9692

TABLE-US-00002 TABLE 2 Classification precision result of each category of PointEFF classification Telegraph Pedes- Vege- Category Ground Building pole trian Vehicle tation PointEFF 0.9894 0.9821 0.5233 0.4985 0.8518 0.9895