Positive azimuth towing guidance method for road rescue equipment based on license plate corner features
11367215 · 2022-06-21
Assignee
Inventors
Cpc classification
B60R11/04
PERFORMING OPERATIONS; TRANSPORTING
B60R2011/008
PERFORMING OPERATIONS; TRANSPORTING
G06V10/25
PHYSICS
B60P3/125
PERFORMING OPERATIONS; TRANSPORTING
B62D15/029
PERFORMING OPERATIONS; TRANSPORTING
International classification
B60R11/04
PERFORMING OPERATIONS; TRANSPORTING
G06V10/25
PHYSICS
Abstract
The present invention discloses a positive azimuth towing guidance method for road rescue equipment based on license plate corner features. The method combines a structure of the road rescue equipment and characteristics of a positive azimuth towing operation. First, an image of an operation area is collected by installing a camera, and grayscale processing and Gaussian smooth filtering are performed on the image; corner detection is performed on the smoothed grayscale image, and preference is implemented according to corner strengths; hierarchical clustering is performed on the preferred corners; an effective corner set of license plate characters is sorted out to implement license plate locating; and then towing guidance is implemented according to a license plate locating result, to improve the rescue efficiency of the road rescue equipment. The guidance method provided in the present invention has good real-time performance, environmental adaptability and anti-interference ability, thereby effectively improving the rescue efficiency of the road rescue equipment.
Claims
1. A positive azimuth towing guidance method for road rescue equipment based on license plate corner features, comprising the following steps: (1) collecting an image of an operation area behind a wrecker and preprocessing the image; (2) performing corner detection and preference on the preprocessed image; (3) clustering preferred corners; (4) locating a license plate according to the preferred corners and getting a license plate center; and (5) performing towing guidance according to the license plate center.
2. The positive azimuth towing guidance method for road rescue equipment based on license plate corner features according to claim 1, wherein in step (1), an image collection and preprocessing method is as follows: installing a vehicle-mounted camera on a folding arm at the rear of the road rescue equipment to collect the image of the operation area behind the wrecker, first converting the color collected image into a grayscale image, and then performing smooth filtering on the grayscale image.
3. The positive azimuth towing guidance method for road rescue equipment based on license plate corner features according to claim 2, wherein in step (2), the smoothed grayscale image is detected by using a Harris corner detection algorithm to obtain the corners in the image and all the obtained corners are stored into a corner set O; preference is performed on the corners obtained by using the Harris corner detection algorithm to obtain a preferred corner set A, and the method is as follows: (2.1) traversing all the corners in the corner set O in ascending order of horizontal and vertical coordinates of the corners, and saving, in a circular area that uses each corner as a center and that has a radius of 5 pixels, corners that have the largest corner strength in this area into the preferred corner set A; and (2.2) retaining, after the traversal ends, only one of the corners that are repeatedly saved and that have the same coordinates and corner strength in the preferred corner set A, and deleting redundant same corners from the preferred corner set A to finally obtain N different preferred corners.
4. The positive azimuth towing guidance method for road rescue equipment based on license plate corner features according to claim 1, wherein in step (3), a corner clustering method is as follows: performing agglomerative hierarchical clustering on the preferred corner set A according to distances between the corners to obtain I preselected corner sets B of license plate characters, i=1, 2, 3, . . . , I, i representing a sequence number of a preselected corner set, and the method being as follows: (3.1) considering each corner a.sub.n, n=1, 2, 3, . . . , N in the preferred corner set A as a class, there being only one corner in each class, calculating a class distance D.sub.rg between every two of all current classes, D.sub.rg representing the class distance between an r.sup.th class and a g.sup.th class, and defining the class distance D.sub.rg as an average value of a Euclidean distance between each corner in the r.sup.th class and each corner in the g.sup.th class, wherein since there is only one corner in each current class, that is, there is only a corner a.sub.r in the r.sup.th class, and there is only a corner a.sub.g in the class, a calculation formula of the class distance D.sub.rg is:
D.sub.rg=d.sub.rg in this formula, d.sub.rg representing the Euclidean distance between corners a.sub.r and a.sub.g, that is, d.sub.rg=√{square root over ((x.sub.a.sub.
5. The positive azimuth towing guidance method for road rescue equipment based on license plate corner features according to claim 4, wherein in step (4), a license plate locating method is as follows: for the preselected corner set B.sub.i of the license plate characters obtained by clustering in step (3), i=1, 2, 3, . . . , I, sorting out an effective corner set C of the license plate characters from the preselected corner set B.sub.i of the license plate characters, and then determining a license plate position, the method being as follows: (4.1) initializing i=1; (4.2) if the preselected corner set B.sub.i of the license plate characters satisfies that num.sub.B.sub.
6. The positive azimuth towing guidance method for road rescue equipment based on license plate corner features according to claim 5, wherein in step (5), the towing guidance method is implemented as follows: giving, according to a position of the license plate center (x.sub.0, y.sub.0) in the image determined during the license plate locating in step (4), a direction prompt in real time to guide the driver to perform a reversing operation: if the license plate center of the to-be-towed vehicle in the image is on the left side of the center of the image, the driver is prompted to reverse to the right; and if the license plate center of the to-be-towed vehicle in the image is on the right side of the center of the image, the driver is prompted to reverse to the left, to implement alignment between bracket arms on two sides and two front wheels of the to-be-towed vehicle, and then fix the to-be-towed vehicle by locking tires and tow the to-be-towed vehicle away.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
DETAILED DESCRIPTION
(12) The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.
(13) With continuous growth of social economy, China's road transport infrastructure construction and automobile industry develop rapidly and the population of cars increases significantly. During traveling of a vehicle, special circumstances such as that the vehicle cannot travel normally due to improper driving or a failure of the vehicle grow increasingly. As an auto aftermarket, a road towing and rescuing industry is gradually valued by society and the public, and the importance of the road towing and rescuing industry in ensuring the safety and efficiency of road transport is increasingly prominent. However, since occurrence of traffic accidents is an objective necessity that cannot be completely prevented, after occurrence of many traffic accidents, urban traffic jams or secondary chain accidents often occur due to untimely towing and rescuing or low rescue efficiency. For example, a traffic jam is caused because the road rescue equipment cannot quickly and accurately tow a vehicle in an accident away from the scene from a positive azimuth. The reason is that, on the one hand, positive azimuth rescue environments are relatively complex. Most of the environments are narrow and jammed, and the operation space is severely restricted. As shown in
(14) Using a rescue pickup as an example, a towing device of the road rescue equipment is mainly composed of a folding arm, a telescopic arm, a swing arm and bracket arms on two sides. A specific form is shown in
(15) In view of characters of the positive azimuth towing operation of the road rescue equipment, the present invention provides a positive azimuth towing guidance method for road rescue equipment that not only has good real-time performance, but also has a relatively strong environmental adaptability and anti-interference ability. For a general process, refer to
(16) (1) Image Collection and Preprocessing
(17) A vehicle-mounted camera is installed at a middle position, that is 40 to 60 centimeters away from the ground, on the folding arm at the rear of the road rescue equipment. The camera horizontally faces the rear of the rescue equipment, and a collection range of the camera includes the swing arm and the bracket arms of the rescue equipment and rear operation areas of the front of the to-be-towed vehicle and some key parts. A specific form is shown in
(18) (2) Corner Detection and Preference
(19) It should be noted that, currently, most social vehicles in China are sedan cars with blue license plates. The guidance method provided in the present invention is mainly oriented to towing and rescuing of sedan cars with blue license plates. The license plate is a common feature of most social vehicles and is universal. License plate characters are printed characters, and character strokes contain rich corner information, so that corner features of a license plate area may be extracted through corner detection.
(20) Typical corner detection methods include the Moravec corner detection algorithm, the Susan corner detection algorithm, the Fast corner detection algorithm, the Harris corner detection algorithm, and the like. The Moravec corner detection algorithm is simple, but the calculation amount is large and the false detection rate is high. The Susan corner detection algorithm is not sensitive to noise, but the locating accuracy is poor. The Harris corner detection algorithm is simple, the calculation amount is not large, the corner extraction is relatively uniform, the adaptability is strong, and the stability is good. In the present invention, the smoothed grayscale image is detected by using the Harris corner detection algorithm to obtain the corners in the image, referring to
(21) The position of the Harris corner detection is relatively accurate, and the corner extraction is relatively uniform, but most of the obtained corner positions appear in batches in a form of neighborhood, referring to
(22) (2.1) traversing all the corners in the corner set O in ascending order of horizontal and vertical coordinates of the corners, and saving, in a circular area that uses each corner as a center and that has a radius of 5 pixels, corners that have the largest corner strength in this area into the preferred corner set A; and
(23) (2.2) retaining, after the traversal ends, only one of the corners that are repeatedly saved and that have the same coordinates and corner strength in the preferred corner set A, and deleting redundant same corners from the preferred corner set A to finally obtain N different preferred corners.
(24) (3) Corner Clustering
(25) The corners of the characters in the license plate area are distributed densely and regularly. Agglomerative hierarchical clustering may be performed on the preferred corner set A according to distances between the corners, referring to
(26) (3.1) considering each corner an, n=1, 2, 3, . . . , N in the preferred corner set A as a class, there being only one corner in each class, calculating a class distance D.sub.rg between every two of all current classes, D.sub.rg representing the class distance between an r.sup.th class and a g.sup.th class, and defining the class distance D.sub.rg as an average value of a Euclidean distance between each corner in the r.sup.th class and each corner in the g.sup.th class in the present invention, where since there is only one corner in each current class, that is, there is only a corner a.sub.r in the r.sup.th class, and there is only a corner a.sub.g in the g.sup.th class, a calculation formula of the class distance D.sub.rg is:
D.sub.rg=d.sub.rg
(27) in this formula, d.sub.rg representing the Euclidean distance between corners a.sub.r and a.sub.g, that is, d.sub.rg=√{square root over ((x.sub.a.sub.
(28) according to the physical meaning of the class distance D.sub.rg, D.sub.rg=D.sub.gr, and when r=g, D.sub.rg=0, all the class distances calculated above are expressed in a form of matrix, to obtain an initial N×N class distance matrix
(29)
the element of row r and column g in the matrix H being D.sub.rg;
(30) (3.2) traversing the current Euclidean distance matrix H to find the minimum non-diagonal element in the matrix H, that is, the current smallest class distance being set as D.sub.st, and s≠t, indicating that an s.sup.th class and a t.sup.th class are the current two closest classes, where if D.sub.st<D.sub.th, the corner in the t.sup.th class is incorporated into the s.sup.th class, then the s.sup.th class and the t.sup.th class are merged into a new class, and a total quantity of classes of corners after merging is recorded as V, to perform sub-step (3.3); otherwise, the clustering calculation is ended to obtain the I preselected corner sets B.sub.i of license plate characters, i=1, 2, 3 . . . , I, where i represents a sequence number of a preselected corner set; I represents the total quantity of preselected corner sets of the license plate characters, and I=V, where in the determination condition of this sub-step, D.sub.st is the minimum non-diagonal element in the current matrix H; and D.sub.th is a minimum class distance threshold, the threshold being within 10 to 15;
(31) (3.3) recalculating a class distance D.sub.rg between every two of current remaining classes and new classes obtained through merging, in this case, a calculation formula of the class distance D.sub.rg being:
(32)
(33) in the formula, M.sub.r representing the quantity of corners in the r.sup.th class, the corners in the class being represented as p.sub.j j=1, 2, 3, . . . , M.sub.r; M.sub.g representing the quantity of corners in the g.sup.th class, the corners in the class being represented as q.sub.k, k=1, 2, 3, . . . , M.sub.g; and d.sup.jk representing a Euclidean distance between corners p.sub.j and q.sub.k, that is, d.sub.jk=√{square root over ((x.sub.p.sub.
(34)
and returning to sub-step (3.2).
(35) (4) License Plate Locating
(36) For the preselected corner set B.sub.i of the license plate characters obtained by clustering in step (3), i=1, 2, 3 . . . , I, according to a priori knowledge of the size of the license plate, an effective corner set C of the license plate characters is sorted out from the preselected corner set of the license plate characters, referring to
(37) (4.1) initializing i=1;
(38) (4.2) if the preselected corner set IL of the license plate characters satisfies that num.sub.B.sub.
(39) (4.3) traversing the preselected corner set B.sub.i of the license plate characters to determine a maximum horizontal coordinate
(40) (4.4) if the preselected corner set B.sub.i of the license plate characters satisfies that γ.sub.B.sub.
(41) (4.5) if i<I, increasing the value of i by 1, and returning to sub-step (4.2); otherwise ending the license plate locating process, and returning to image collection and preprocessing in step (1).
(42) Two supplementary explanations for the foregoing algorithm of sorting out the effective corner set of the license plate characters are: {circle around (1)} Blue license plates in China have national standards, and each blue license plate has seven printed characters, where each character has at least one corner, and the aspect ratio of the seven printed characters is about 4. {circle around (2)} The camera is installed at a fixed middle position on the folding arm of the road rescue equipment and horizontally faces the rear of the road rescue equipment. During the positive azimuth towing operation, the to-be-towed vehicle is located in the rear area of the road rescue equipment. The operation distance is generally 1 to 5 meters, and the selected focal length of the camera may be 4 to 8 millimeters. A size of the image collected by the camera in the present invention is 960×640, and within a range of the effective operation distance of 1 to 5 meters, the pixel size of the license plate character area in the collected image varies within a range of 300 to 5000.
(43) (5) Towing Guidance
(44) As shown in
(45) is consistent with the central axis of the road rescue equipment and the towing device of the road rescue equipment, and a front license plate of the to-be-towed vehicle is generally located in the middle of the front of the to-be-towed vehicle, so that a relative position relationship between the road rescue equipment and the to-be-towed vehicle may be judged according to a left-right position relationship between a center of a picture outputted by the camera and a license plate center of the to-be-towed vehicle in the picture.
(46) During the implementation of the positive azimuth towing operation, the driver faces the front of the road rescue equipment. Therefore, according to a position of the license plate center (x.sub.0, y.sub.0) in the image determined during the license plate locating in step (4), referring to
(47) Through the foregoing steps, the method provided in the present invention can implement real-time guidance of the rescue equipment during the positive azimuth towing operation, thereby effectively improving the rescue efficiency of the road rescue equipment.