QUATERNION MULTI-DEGREE-OF-FREEDOM NEURON-BASED MULTISPECTRAL WELDING IMAGE RECOGNITION METHOD

Abstract

Disclosed is a quaternion multi-degree-of-freedom neuron-based multispectral welding image recognition method, comprising: using three cameras having different wavebands to obtain multispectral weld pool images, and respectively performing pre-processing and edge extraction on the weld pool images having the different wavebands obtained at a same moment by the three cameras; establishing a quaternion-based multispectral weld pool image edge model; extracting low-frequency features after a quaternion discrete cosine transform; using a quaternion-based multi-degree-of-freedom neuron network to perform classification, training and recognition on edge features of the multispectral weld pool images. Compared to traditional means, the present invention has multiple recognition information sources, strong anti-interference capabilities and high recognition accuracy.

Claims

1. A quaternion multi-degree-of-freedom neuron-based multispectral welding image recognition method, which is characterized in that: a steel plate to be welded is laid flat below; laser welding equipment is located above and faces the steel plate; the three cameras equipped with glass filters and protective glasses are arranged side by side and vertically aligned with a portion to be subject to laser welding; an upper computer uses the LabVIEW program to acquire and control the states of the three cameras through the image acquisition card; a corresponding power supply device provides power; the multispectral weld pool images obtained by the three cameras are processed by the MATLAB program, to obtain and display the weld pool image recognition result; The processing is divided into two parts: the first part is to extract the low-frequency features of the weld pool images, comprising: A, preprocessing the weld pool images captured by the three cameras at the same moment, wherein an original image is binarized first, then is expanded and corroded by means of morphological operation, and then subject to area threshold filtering to complete the image preprocessing, and finally, weld pool edge information is extracted from the preprocessed images by scanning from outside to inside line by line; B, establishing a quaternion-based multispectral weld pool image edge model, wherein for sampling points at any moment, the distance data from an edge to a centroid of each of the weld pool images with different spectra obtained by the three cameras can be expressed as a pure imaginary quaternion, so the distance sequences from the edge to the centroid of each of the weld pool images with three spectral ranges corresponding to different categories can be combined and mapped on a quaternion space, and the change of a quaternion vector sequence can be characterized as the change of the weld pool image information with different states during welding; and model establishment specifically comprises: (1) calculating the centroid of each of the three preprocessed images, and selecting edge points meeting sampling angles, wherein the centroid is calculated as follows: $\left(x_0,y_0\right)=\left(\frac{.Math.m_{10}.Math.}{.Math.m_{00}.Math.},\frac{.Math.m_{01}.Math.}{.Math.m_{00}.Math.}\right)$ a two-dimensional geometric moment m.sub.pq of the image is calculated as follows: $m_{\mathrm{pq}}=\underset{M}{.Math.}\underset{N}{.Math.}{x}^p{y}^{q}f\left(x,y\right)$ where p and q represent the order of the geometric moment m.sub.pq, x and y are the coordinates of a pixel, f(x,y) represents a gray value of the pixel here, M and N represent a size of the image, and the size of the image in the embodiment of the present invention is 1390×1040; after working out the centroid, a ray with the centroid as a starting point and an X positive direction is established in an edge graph, and the ray is rotated at a certain angle until it is rotated by 360° in total, wherein during each time of rotation, an intersection of the ray and the edge is the edge point meeting the sampling angles; in the embodiment of the invention, a total of 100 edge points need to be taken, so one edge point is taken each time the ray is rotated by 3.6° until 100 edge points are obtained; (2) for the selected edge points, calculating their distances to the centroid respectively, and storing them in the distance data sequences corresponding to the three spectra in turn in angular order, wherein the distance is calculated as follows:
d=√{square root over ((x−x.sub.0).sup.2−(y−y.sub.0).sup.2)} where (x, y) is the coordinates of the edge point, and (x.sub.0, y.sub.0) is the coordinates of the centroid; the distance data sequences corresponding to the three spectra are expressed as the following formula:
S.sub.x={x.sub.i}.sub.t=1.sup.n
S.sub.y={y.sub.i}.sub.i=1.sup.n
S.sub.z={z.sub.i}.sub.i=1.sup.n where x.sub.i, y.sub.i and z.sub.i are distance values corresponding to i.sup.th points in the distance data sequences corresponding to the three spectra, and N is a length of a signal, that is, the number of the selected edge points; on this basis, the distance data sequences from the edge of to the centroid of a multispectral weld pool can be uniformly expressed as:
s={q.sub.η=x.sub.η.Math.i+y.sub.η.Math.j+z.sub.η.Math.k|η∈1,2, . . . ,n} besides, i.sup.2=j.sup.2=k.sup.2=−1, j=k=−ji, jk=i=−kj, ki=j=−ik, and this expression is the quaternion-based multispectral weld pool image edge model; and C, extracting low-frequency features after a quaternion discrete cosine transformation, wherein a left transformation F.sup.L(ω1, ω2) and a right transformation F.sup.R (ω1, ω2) of the quaternion discrete cosine transformation (QDCT) are as follows: $F^L\left({\omega }_1,{\omega }_2\right)=\delta \left({\omega }_1\right)\delta \left({\omega }_2\right)\underset{x=0}{\overset{m-1}{.Math.}}\underset{y=0}{\overset{n-1}{.Math.}}\mathrm{uf}\left(x,y\right)\cos \left[\frac{\pi \left(2x+1\right){\omega }_1}{2m}\right]\cos \left[\frac{\pi \left(2x+1\right){\omega }_2}{2n}\right]F^R\left({\omega }_1,{\omega }_2\right)=\delta \left({\omega }_1\right)\delta \left({\omega }_2\right)\underset{x=0}{\overset{m-1}{.Math.}}\underset{y=0}{\overset{n-1}{.Math.}}f\left(x,y\right)\cos \left[\frac{\pi \left(2x+1\right){\omega }_1}{2m}\right]\cos \left[\frac{\pi \left(2x+1\right){\omega }_2}{2n}\right]u$ where u is a unit pure quaternion, u.sup.2=−1, f(x, y) is a corresponding value of (x, y) in a quaternion matrix of m×n to be transformed, F.sup.L(ω.sub.1, ω.sub.2) and F.sup.R (ω1, ω.sub.2) are quaternion coefficients in a transformation domain (ω1, ω.sub.2), and δ(ω.sub.1) and δ(ω.sub.2) are defined as: $\delta \left({\omega }_1\right)=\{\begin{array}{cc}\sqrt{1/m}& {\omega }_1=0\\ \sqrt{2/n}& {\omega }_1\ne 0\end{array}\delta \left({\omega }_2\right)=\{\begin{array}{cc}\sqrt{1/m}& {\omega }_2=0\\ \sqrt{2/n}& {\omega }_2\ne 0\end{array};$ and after the quaternion discrete cosine transformation is carried out on the quaternion-based multispectral weld pool image edge model, low-frequency components are taken as feature quaternion vectors of weld pool edge data; after the first part, that is, extracting the low-frequency features of the weld pool images, the second part can be carried out, that is, using a quaternion-based multi-degree-of-freedom neuron network to perform classified recognition on edge features of the multispectral weld pool images; first, a quaternion-based multi-degree-of-freedom neuron network is constructed by using training samples to obtain two neuron coverage areas with different characterizations, “complete penetration” and “incomplete penetration”; then the extracted low-frequency features of edge data of test samples are input into the network to calculate the Euclidean distance between a sample to be recognized and the two neuron coverage areas, wherein the characterization corresponding to the coverage area with the smallest distance is the characterization to which the test sample belongs; and finally, a recognition result is taken as the output of the MDOFNN.

2. The quaternion multi-degree-of-freedom neuron-based multispectral welding image recognition method according to claim 1, which is characterized in that: the three cameras are triggered synchronously to capture images, and the weld pool images captured at the same moment with wavelengths of 808 nm, 830 nm and 850 nm respectively are obtained.

Description

DESCRIPTION OF SEVERAL VIEWS OF THE ATTACHED DRAWINGS

[0025] FIG. 1 is a block diagram of an overall design of the present invention;

[0026] FIG. 2 is a diagram of a hardware structure of the present invention;

[0027] FIG. 3 is an algorithm flow chart in the present invention;

[0028] FIG. 4 is an original image of an embodiment of the present invention;

[0029] FIG. 5 is an image result after a series of preprocessing according to an embodiment of the present invention;

[0030] FIG. 6 is a data image of a distance from an edge of a weld pool to a welding center under typical incomplete penetration and complete penetration states of the weld pool according to an embodiment of the present invention; and

[0031] FIG. 7 is a sample image after low-frequency features are extracted under typical incomplete penetration and complete penetration states of a weld pool according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0032] According to a quaternion multi-degree-of-freedom neuron-based multispectral welding image recognition method, firstly three cameras corresponding to different wave bands are controlled by a LabVIEW program to obtain the image information of a weld pool respectively, then an image acquisition card is used for putting the information together and uploading the information, and finally images of different wave bands captured at a same moment are processed by a MATLAB program to obtain and display a weld pool image recognition result.

[0033] A steel plate to be welded is laid flat below; laser welding equipment is located above and faces the steel plate; the three cameras equipped with glass filters and protective glasses are arranged side by side and vertically aligned with a portion to be subject to laser welding; an upper computer uses the LabVIEW program to acquire and control the states of the three cameras through the image acquisition card; a corresponding power supply device provides power;

[0034] The three cameras are triggered synchronously to capture images, and the weld pool images captured at the same moment with wavelengths of 808 nm, 830 nm and 850 nm respectively are obtained.

[0035] The multispectral weld pool images obtained by the three cameras are processed by the MATLAB program, to obtain and display the weld pool image recognition result. The processing is divided into two parts:

[0036] The first part is to extract the low-frequency features of the weld pool images, comprising:

[0037] A, preprocessing the weld pool images captured by the three cameras at the same moment, wherein an original image is binarized first, then is expanded and corroded by means of morphological operation, and then subject to area threshold filtering to complete the image preprocessing, and finally, weld pool edge information is extracted from the preprocessed images by scanning from outside to inside line by line;

[0038] B, establishing a quaternion-based multispectral weld pool image edge model, wherein for sampling points at any moment, the distance data from an edge to a centroid of each of the weld pool images with different spectra obtained by the three cameras can be expressed as a pure imaginary quaternion, so the distance sequences from the edge to the centroid of each of the weld pool images with three spectral ranges corresponding to different categories can be combined and mapped on a quaternion space, and the change of a quaternion vector sequence can be characterized as the change of the weld pool image information with different states during welding; and model establishment specifically comprises:

[0039] (1) calculating the centroid of each of the three preprocessed images, and selecting edge points meeting sampling angles, [0040] wherein the centroid is calculated as follows:

[00005]$\left(x_0,y_0\right)=\left(\frac{.Math.m_{10}.Math.}{.Math.m_{00}.Math.},\frac{.Math.m_{01}.Math.}{.Math.m_{00}.Math.}\right)$

[0041] a two-dimensional geometric moment m.sub.pq of the image is calculated as follows:

[00006]$m_{\mathrm{pq}}=\underset{M}{.Math.}\underset{N}{.Math.}{x}^p{y}^{q}f\left(x,y\right)$

[0042] where p and q represent the order of the geometric moment m.sub.pg, x and y are the coordinates of a pixel, f(x,y) represents a gray value of the pixel here, M and N represent a size of the image, and the size of the image in the embodiment of the present invention is 1390×1040;

[0043] after working out the centroid, a ray with the centroid as a starting point and an X positive direction is established in an edge graph, and the ray is rotated at a certain angle until it is rotated by 360° in total, wherein during each time of rotation, an intersection of the ray and the edge is the edge point meeting the sampling angles; in the embodiment of the invention, a total of 100 edge points need to be taken, so one edge point is taken each time the ray is rotated by 3.6°

until 100 edge points are obtained.

[0044] (2) for the selected edge points, calculating their distances to the centroid respectively, and storing them in the distance data sequences corresponding to the three spectra in turn in angular order, wherein the distance is calculated as follows:

d=√{square root over ((x−x.sub.0).sup.2−(y−y.sub.0).sup.2)}

[0045] where (x, y) is the coordinates of the edge point, and (x.sub.0, y.sub.0) is the coordinates of the centroid;

[0046] the distance data sequences corresponding to the three spectra are expressed as the following formula:

S.sub.x={x.sub.i}.sub.t=1.sup.n

S.sub.y={y.sub.i}.sub.i=1.sup.n

S.sub.z={z.sub.i}.sub.i=1.sup.n

[0047] where x.sub.i, y.sub.i and z.sub.i are distance values corresponding to i.sup.th points in the distance data sequences corresponding to the three spectra, and N is a length of a signal, that is, the number of the selected edge points; on this basis, the distance data sequences from the edge of to the centroid of a multispectral weld pool can be uniformly expressed as:

s={q.sub.η=x.sub.η.Math.i+y.sub.η.Math.j+z.sub.η.Math.k|η∈1,2, . . . ,n} [0048] besides, i.sup.2=j.sup.2=k.sup.2=−1, j=k=−ji, jk=i=−kj, ki=j=−ik, and this expression is the quaternion-based multispectral weld pool image edge model; and

[0049] C, extracting low-frequency features after a quaternion discrete cosine transformation, wherein a left transformation F.sup.L(ω1, ω2) and a right transformation F.sup.R (ω1, ω2) of the quaternion discrete cosine transformation (QDCT) are as follows:

[00007]$F^L\left({\omega }_1,{\omega }_2\right)=\delta \left({\omega }_1\right)\delta \left({\omega }_2\right)\underset{x=0}{\overset{m-1}{.Math.}}\underset{y=0}{\overset{n-1}{.Math.}}\mathrm{uf}\left(x,y\right)\cos \left[\frac{\pi \left(2x+1\right){\omega }_1}{2m}\right]\cos \left[\frac{\pi \left(2x+1\right){\omega }_2}{2n}\right]F^R\left({\omega }_1,{\omega }_2\right)=\delta \left({\omega }_1\right)\delta \left({\omega }_2\right)\underset{x=0}{\overset{m-1}{.Math.}}\underset{y=0}{\overset{n-1}{.Math.}}f\left(x,y\right)\cos \left[\frac{\pi \left(2x+1\right){\omega }_1}{2m}\right]\cos \left[\frac{\pi \left(2x+1\right){\omega }_2}{2n}\right]u$

[0050] where u is a unit pure quaternion, u.sup.2=−1, f(x, y) is a corresponding value of (x, y) in a quaternion matrix of m×n to be transformed, F.sup.L(ω.sub.1, ω.sub.2) and F.sup.R(ω.sub.1, ω.sub.2) are quaternion coefficients in a transformation domain (ω.sub.1, ω.sub.2), and δ (ω.sub.1) and δ (ω.sub.2) are defined as:

[00008]$\delta \left({\omega }_1\right)=\{\begin{array}{cc}\sqrt{1/m}& {\omega }_1=0\\ \sqrt{2/n}& {\omega }_1=0\end{array}\delta \left({\omega }_2\right)=\{\begin{array}{cc}\sqrt{1/m}& {\omega }_2=0\\ \sqrt{2/n}& {\omega }_2\ne 0\end{array};$

[0051] and after the quaternion discrete cosine transformation is carried out on the quaternion-based multispectral weld pool image edge model, low-frequency components are taken as feature quaternion vectors of weld pool edge data.

[0052] FIG. 6 shows the edge distance data under typical incomplete penetration and complete penetration states of a weld pool according to an embodiment of the present invention, in which curves of three colors correspond to three spectra, the abscissa is the serial number of the edge point, and the ordinate is the distance between the point and the centroid.

[0053] FIG. 7 shows the data after the quaternion discrete cosine transformation of the edge distance data under typical incomplete penetration and complete penetration states of a weld pool according to an embodiment of the present invention, in which curves of three color correspond to three spectra, the abscissa is the serial number of the edge point, and the ordinate is the data value after change.

[0054] After the first part, that is, extracting the low-frequency features of the weld pool images, the second part can be carried out, that is, using a quaternion-based multi-degree-of-freedom neuron network to perform classified recognition on edge features of the multispectral weld pool images.

[0055] As shown in FIG. 7, from the analysis of the low-frequency feature data under typical incomplete penetration and complete penetration states of the weld pool, the sample data characterization difference between incomplete penetration and complete penetration states is quite large, so a multi-degree-of-freedom neuron network (MDOFNN) can be adopted for classified recognition.

First, a quaternion-based multi-degree-of-freedom neuron network is constructed by using training samples to obtain two neuron coverage areas with different characterizations, “complete penetration” and “incomplete penetration”; then the extracted low-frequency features of edge data of test samples are input into the network to calculate the Euclidean distance between a sample to be recognized and the two neuron coverage areas, wherein the characterization corresponding to the coverage area with the smallest distance is the characterization to which the test sample belongs; and finally, a recognition result is taken as the output of the MDOFNN.