METHOD FOR GRASPING TEXTURE-LESS METAL PARTS BASED ON BOLD IMAGE MATCHING

Abstract

A method for grasping texture-less metal parts based on BOLD image matching comprises: obtaining a real image and CAD template images by photographing, extracting a foreground part of the input part image, calculating a covariance matrix of a foreground image, establishing the direction of a temporary coordinate system, and setting directions of line segments to point to a first or second quadrant of the temporary coordinate system; constructing a descriptor of each line segment according to an angle relation between the line segment and k nearest line segments, and matching the descriptors of different line segments in the real image and the CAD template images to obtain line segment pairs; and recognizing a processed pose through a PNL algorithm to obtain a pose of a real texture-less metal part, and then inputting the pose of the real texture-less metal part to a mechanical arm to grasp the part. The present invention can correctly match line segments, obtain an accurate pose of the part by calculation, successfully grasp the part, and satisfy actual application requirements.

Claims

1. A method for grasping texture-less metal parts based on bundle of lines descriptor (BOLD) image matching, comprising the steps of: step 1: photographing a real texture-less metal part placed in a real environment by a real physical camera to obtain a real image; photographing a texture-less metal part computer aided design (CAD) model imported in a computer virtual scene by a virtual camera to obtain CAD template images; extracting a foreground part of the input real image and the input CAD template images, calculating a covariance matrix of the foreground part, and establishing a direction of a temporary coordinate system; step 2: processing the real image and all the CAD template images by means of a line segment detector, extracting edges in the real image and all the CAD template images and using the edges as line segments, traversing all the line segments in each said image, and setting directions of the line segments in the temporary coordinate system; step 3: for each said image, traversing all the line segments, and constructing a descriptor of each said line segment according to an angle relation between the line segment and k nearest line segments; step 4: in case of different k values of the descriptors k of the line segments in the real image and the CAD template images, matching the descriptors of different line segments in the real image and the CAD template images to obtain line segment pairs; and step 5: recognizing a processed pose by means of perspective n lines according to the matched line segment pairs to obtain a pose of the real texture-less metal part, and then inputting the pose of the real texture-less metal part into a mechanical arm to grasp the part.

2. The method for grasping texture-less metal parts based on BOLD image matching according to claim 1, wherein the texture-less metal part is a polyhedral metal part with a flat and smooth surface and free of pits, protrusions and textures.

3. The method for grasping texture-less metal parts based on BOLD image matching according to claim 1, wherein specifically, in Step 1, the foreground part of the images is extracted and used as a foreground image, a covariance matrix of the foreground image is calculated to obtain two feature values of the covariance matrix and feature vectors corresponding to the two feature values, the feature vector corresponding to a larger feature value is taken as an x-axis positive direction of the temporary coordinate system, and the other feature vector is taken as a y-axis positive direction of the temporary coordinate system.

4. The method for grasping texture-less metal parts based on BOLD image matching according to claim 1, wherein traversing all the line segments to set directions of the line segments in the step 2 is performed specifically as follows: a temporary coordinate system is established with any point on each said line segment as an origin of the temporary coordinate system; if the line segment passes through a first quadrant, the line segment points to the first quadrant of the temporary coordinate system; then: if the line segment passes through a second quadrant, the line segment points to the second quadrant of the temporary coordinate system; or, if the line segment does not pass through the first quadrant or the second quadrant, the line segment points to the first quadrant and the second quadrant of the temporary coordinate system.

5. The method for grasping texture-less metal parts based on BOLD image matching according to claim 1, wherein in the step 3, the k nearest line segments of each said line segment are selected in order according to distances between midpoints of the line segments.

6. The method for grasping texture-less metal parts based on BOLD image matching according to claim 1, wherein the step 3 is performed specifically as follows: 3.1: with two line segments s.sub.i and s.sub.j as one line segment and one nearest line segment thereof, a first angle α and a second angle β are calculated according to the following formula, and relative positions of the two line segments s.sub.i and s.sub.j are described by α and β; $\alpha =\{\begin{array}{c}\arccos \left(\frac{s_i.Math.t_{\mathrm{ij}}}{.Math.s_i.Math..Math..Math.t_{\mathrm{ij}}.Math.}\right),\frac{s_i\times t_{\mathrm{ij}}}{.Math.s_i\times t_{\mathrm{ij}}.Math.}.Math.n=1\\ 2-\arccos \left(\frac{s_i.Math.t_{\mathrm{ij}}}{.Math.s_i.Math..Math..Math.t_{\mathrm{ij}}.Math.}\right),\mathrm{else}\end{array}\beta =\{\begin{array}{c}\arccos \left(\frac{s_j.Math.t_{ji}}{.Math.s_j.Math..Math..Math.t_{\mathrm{ji}}.Math.}\right),\frac{s_j\times t_{ji}}{.Math.s_j\times t_{\mathrm{ji}}.Math.}.Math.n=1\\ 2\pi -\arccos \left(\frac{s_j.Math.t_{\mathrm{ji}}}{.Math.s_j.Math..Math..Math.t_{ji}.Math.}\right),\mathrm{else}\end{array}\{\begin{array}{c}{t}_{ij}=m_j-m_i\\ t_{ji}=m_i-m_j\end{array}$ wherein, s.sub.i and s.sub.j are vector representations of two line segments in the same image, respectively, and vector directions are determined by the directions of the line segments in the temporary coordinate system obtain in Step 2; n is a unit vector perpendicular to an image plane, ∥a∥ is a module length of a vector a, m.sub.i and m.sub.j are midpoints of the line segments s.sub.i and s.sub.j, respectively, and t.sub.ij is a vector that points from m.sub.i to m.sub.j; 3.2: for each said line segment in the images, the first angles α and the second angles β of the line segment and k nearest line segments are obtained according to the step 3.1, a constant contrast based BOLD of each said line segment is constructed by k pairs of first angles α and second angles β, which form a matrix to represent the descriptor.

7. The method for grasping texture-less metal parts based on BOLD image matching according to claim 1, wherein the step 4 is performed specifically as follows: 4.1: different k values for generating the descriptors of line segments in the real image and the CAD template images are k.sub.1 and k.sub.2, respectively; if k.sub.1=k.sub.2, the Euclidean distance between the descriptor of one said line segment in the real image and the descriptor of each said line segment in the CAD template images is calculated according to the following formula, two line segments corresponding to the nearest descriptors are selected and are regarded as matched to constitute a line segment pair: $d=\sqrt{\underset{i=1}{\overset{n}{.Math.}}{\left(d_i^1-d_i^2\right)}^2}$ wherein, d is the Euclidean distance between two descriptors, d.sub.i.sup.1 is an i-th element in the descriptor of the line segment in a first image, d.sub.i.sup.2 is an i-th element in the descriptor of the line segment in a second image, i is the ordinal of elements in the descriptor, and n is the total number of elements in the descriptor; if k.sub.1<k.sub.2, processing is performed as follows: first, the descriptor of each said image is calculated and corrected according to the following formula: $d_i^2=\{\begin{array}{c}0,d_i^1=0\\ d_i^2,\mathrm{else}\end{array},i=1,2...n$ wherein, d.sub.i.sup.m is an element at the i-th position of the line segment in an m-th image; then, the descriptors of all the images are normalized: $d_i^m=\frac{d_i^m}{\sqrt{\underset{i=1}{\overset{n}{.Math.}}{\left(d_i^m\right)}^2}},m=1,2$ finally, the Euclidean distance between the descriptor of one said line segment in the real image and the descriptor of each said line segment in the CAD template images is calculated according to the following formula, and two line segments corresponding to the nearest descriptors are selected and are regarded as matched: $d=\sqrt{\underset{i=1}{\overset{n}{.Math.}}{\left(d_i^1-d_i^2\right)}^2}$ wherein, d is the Euclidean distance between two descriptors, d.sub.i.sup.1 is an i-th element in the descriptor of the line segment in a first image, d.sub.i.sup.2 is an i-th element in the descriptor of the line segment in a second image, i is the ordinal of elements in the descriptor, and n is the total number of elements in the descriptor; finally, in the step 4, after line segment pairs of all the line segments in the real image are found in the CAD template images, mismatches of all the line segments are removed through an RANSAC algorithm, and a finally obtained line segment pair is used as a line segment matching result.

Description

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0040] FIG. 1 is a flow diagram of a method of the present invention;

[0041] FIG. 2 is a schematic diagram for direction definition of line segments;

[0042] FIG. 3 is a schematic diagram of descriptor calculation;

[0043] FIG. 4 is a diagram of a matching result obtained by applying BOLD to a texture-less metal part;

[0044] FIG. 5 is a diagram of a matching result obtained by applying the present invention to a texture-less metal part.

DETAILED DESCRIPTION OF THE INVENTION

[0045] The present invention will be further explained below in conjunction with the accompanying drawings and embodiments. The flow diagram of the present invention is illustrated by FIG. 1.

[0046] A specific embodiment and an implementation process thereof of the present invention are as follows:

[0047] This embodiment is implemented with a U-shaped bolt as a texture-less metal part.

[0048] Step 1: a real texture-less metal part placed in a real environment is photographed by a real physical camera to obtain a real image; a texture-less metal part CAD model imported in a computer virtual scene is photographed by a virtual camera to obtain CAD template images; a foreground part of the input real image and the input CAD template images is extracted through a grabcut algorithm, a covariance matrix of the foreground part is calculated, and the direction of a temporary coordinate system is established.

[0049] The real image and the CAD template images are specifically processed as follows: a covariance matrix of a foreground image of an input image, feature values thereof, and corresponding vector features are calculated, the feature vector corresponding to a larger feature value is taken as an x-axis positive direction of the temporary coordinate system, and the other feature vector is taken as a y-axis positive direction of the temporary coordinate system, as shown in FIG. 2.

[0050] Step 2: the real image and all the CAD template images are processed by means of a line segment detector (LSD), edges in the real image and all the CAD template images are extracted and used as line segments, all the line segments in each image are traversed, and directions of the line segments in the temporary coordinate system are set.

[0051] As shown in FIG. 2, the direction of the line segment s is set to point to a first quadrant of the temporary coordinate system.

[0052] Step 3: for each image, all the line segments are traversed, and a descriptor of each line segment is constructed according to an angle relation between the line segment and k nearest line segments;

[0053] As shown in FIG. 3, specifically:

[0054] 3.1: with two line segments s.sub.i and s.sub.j as one line segment and a nearest line segment thereof, a first angle α and a second angle β are calculated according to the following formula, as shown in FIG. 3, and relative positions of the two line segments s.sub.i and s.sub.i are described by α and β;

[00006]$\alpha =\{\begin{array}{c}\arccos \left(\frac{s_i.Math.t_{ij}}{.Math.s_i.Math..Math..Math.t_{ij}.Math.}\right),\frac{s_i\times t_{ij}}{.Math.s_i\times t_{ij}.Math.}.Math.n=1\\ 2\pi -\arccos \left(\frac{s_i.Math.t_{ij}}{.Math.s_i.Math..Math..Math.t_{\mathrm{ij}}.Math.}\right),\mathrm{else}\end{array}\beta =\{\begin{array}{c}\arccos \left(\frac{s_j.Math.t_{ji}}{.Math.s_j.Math..Math..Math.t_{\mathrm{ji}}.Math.}\right),\frac{s_j\times t_{ji}}{.Math.s_j\times t_{\mathrm{ji}}.Math.}.Math.n=1\\ 2\pi -\arccos \left(\frac{s_j.Math.t_{\mathrm{ji}}}{.Math.s_j.Math..Math..Math.t_{ji}.Math.}\right),\mathrm{else}\end{array}\{\begin{array}{c}{t}_{ij}=m_j-m_i\\ t_{ji}=m_i-m_j\end{array}$

[0055] 3.2: for each line segment in the images, the first angles α and the second angles β of the line segment and k nearest line segments are obtained according to Step 3.1, that is, a constant contrast-based BOLD of each line segment is constructed by k pairs of first angles α and second angles β, which form a matrix to represent the descriptor.

[0056] In actual implementation, each pair of first angle α and second angle β can be discretely accumulated into a 2D joint histogram, and in this specification, the discrete step length is set to π/12, and the 2D joint histogram is the descriptor of the line segment.

[0057] Step 4: in case of different k values for generating the descriptors of line segments in the real image and the CAD template images, the descriptors of different line segments in the real image and the CAD template images are matched to obtain line segment pairs;

[0058] Finally, mismatches are removed through an RANSAC algorithm, an output matching result is shown in FIG. 5, and correctly matched line segments are indicated by boxes. A matching result obtained by BOLD is shown in FIG. 4. As can be seen, the method of the present invention greatly improves the accuracy.

[0059] Step 5: a processed pose is recognized by means of perspective n lines (PNL) according to the matched line segment pairs to obtain a pose of the real texture-less metal part, and then the pose of the real texture-less metal part is input to a mechanical arm to grasp the part.

[0060] The preferred embodiments mentioned above are used to disclose the present invention, and are not intended to limit the present invention. Those ordinarily skilled in the art can make different modifications and embellishments without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention is defined by the claims.