Target recognition and localization methods using a laser sensor for wheeled mobile robots
09739616 · 2017-08-22
Assignee
Inventors
- Junmin Wang (Dublin, OH, US)
- Madhu Soodhanan Govindarajan (Columbus, OH, US)
- James W. Post, II (Dublin, OH, US)
- Andrew John Fox (Powell, OH, US)
Cpc classification
G06T11/005
PHYSICS
G01B11/028
PHYSICS
International classification
G01C21/00
PHYSICS
Abstract
A localization scheme and method using a laser sensor for indoor wheeled mobile robots (IWMR), which need to localize themselves for working autonomously, is provided. In this method, a laser sensor moves inside an onboard guide way and its distance measurements are used to robustly detect and recognize a unique target based on edge detection and pattern recognition techniques. From the distance measurements with respect to the recognized target, a kinematic model is developed to determine the IWMR orientation and location in the global co-ordinates (in 2-D). Such target recognition and localization methods are validated with experimental results.
Claims
1. A computer implemented method for localizing a wheeled robot/vehicle using a laser sensor, comprising: scanning an angular section of a surrounding environment of the wheeled robot/vehicle by the laser sensor, wherein the laser sensor is mounted on a rotary actuator of the wheeled robot/vehicle; identifying a target, wherein identifying the target includes detecting edges on the target and recognizing a pattern of the target based on the scanning by the laser sensor, wherein detecting the edges on the target is based on laser sensor measurements and rotary actuator angular positions of the rotary actuator, and wherein detecting the edges on the target includes identifying two edges that have a minimum length equal to a minimum depth of the target; transforming the laser sensor measurements and the rotary actuator angular positions into vehicular coordinates, wherein the vehicular coordinates are associated to the target as identified from the laser sensor and derived from time instant points corresponding to the two edges, wherein the time instant points are joined in a straight line; determining global coordinates for the vehicular coordinates based on a slope of the straight line; and controlling the wheeled robot/vehicle to travel autonomously based on the global coordinates.
2. The computer implemented method of claim 1 wherein the determined global coordinates are compared with templates of the target to confirm that the target has been recognized.
3. The computer implemented method of claim 2 wherein comparing determined global coordinates with templates includes translating the templates that are defined in global coordinates into laser coordinates and minimizing Hausdorff distances between the determined global coordinates and the templates.
4. The computer implemented method of claim 1 further including: generating templates for the target, wherein templates for the target include three different templates that are each utilized based on the location of the wheeled robot/vehicle.
5. The computer implemented method of claim 1 wherein controlling the wheeled robot/vehicle includes: determining an orientation and location of the wheeled robot/vehicle based on the global coordinates.
6. The computer implemented method of claim 4 wherein each different template of the three different templates represent three different zones in front of the target, wherein the wheeled robot/vehicle is located within one of the three different zones.
7. The computer implemented method of claim 1 further including reducing the influence of noise in the laser sensor measurements on the determined global coordinates.
8. The computer implemented method of claim 7 further including approximating the noise of the laser sensor measurements by a white Gaussian noise.
9. The computer implemented method of claim 1 wherein the laser sensor has rotary and linear motion capabilities.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
(26)
(27)
(28)
(29)
(30)
(31)
(32)
(33)
(34)
(35)
(36)
(37)
(38)
(39)
(40)
(41)
DETAILED DESCRIPTION
(42) The subject disclosure proposes a new technique for localizing IWMRs (in an unknown static 2-D environment that is, no knowledge about the environment or landmark features in the environment is needed) based on laser sensor distance measurements alone (with respect to a target). The proposed technique is different from the existing ones in that it does not need a priori knowledge about the environment and thus gives the flexibility of employing this technique in uncertain environments. The only requirement for this technique is the capability to place the target at a known global location in an uncertain environment (no knowledge about the environment is required). Further, the complexity of SLAM algorithms can be reduced by developing a localization technique that uses a laser sensor alone, with the assumption that the unknown environment is static (i.e., there is no moving object in the entire environment).
(43) The system hardware configuration for the proposed localization scheme according to one embodiment is shown in
(44) According to one aspect, the target used in this localization method has a surface with uniquely combined depth variations. As a result, the target can be identified properly by the distance measurements when the laser sensor beam scans through the target surface. The cross-section of an example target that can be used is given in
(45)
(46) Target Design
(47) As will be appreciated by those skilled in the art, the target design, which includes the target shape, material, and color, will affect the laser sensor distance measurement accuracy, visible range and consequently influence the target recognition capability and localization accuracy of the system. Several target design guidelines generated from experimental results are described below.
(48) Generalized Target Design Methodology and Guidelines
(49)
Thus, worst angle of a laser sensor is the sector in which a laser sensor cannot operate accurately. This is depicted in
(50) An exemplary design is one that takes into consideration the worst scenario. The design (edge depth-to-width ratio) should address the event when the laser sensor is placed at the worst angle. It should be such that even if the laser sensor is viewing the target from the worst angle, all d.sub.i's in the d.sub.i and w.sub.i pairs are seen. Thus the angle formed by all of these d.sub.i and w.sub.i pairs should always be greater than the worst angle. This will reduce the chance of detecting two edges which is necessary for the target recognition technique. Thus, the edge-to-depth ratio should be greater than or equal to the tangent of the worst angle.
(51)
If the condition given in Eq. (1) is not met by any d.sub.i and w.sub.i pair, then the chance of detecting two edges is reduced. Thus, the edge-to-depth ratio should be greater than or equal to the tangent of the worst angle. An example scenario is illustrated in
(52) Laser Sensor Specific Target Design
(53) It is evident from the definition of worst angle that it depends on the specific laser sensor being used. The value of worst angle has to be determined for the specific laser sensor. Experiments were conducted for an example laser sensor used in one embodiment of this disclosure (ODSL 96B M/V6-2000-S12 manufactured by Leuze Electronics), to determine the worst angle. The laser sensor measurements can be collected in a dSPACE Micro Auto Box (MABX) as shown in
(54) The edge depth-to-width ratio according to the formula given in Eq. (2) should be greater than 1/4.0108. Taking this into consideration, a target can be designed with an edge depth-to-width ratio of 1/4 . An exemplary designed target is shown in
(55) Overall Length of the Target
(56) The overall length of the target can be chosen based on scanning time being proportional to the overall length of the target. Thus, the target should not be too long. However, the target should be long enough to accommodate at least two edges that obey the condition shown in Eq. (2). The length portions other than w.sub.i's do not carry any significance, they are merely present to separate the two edges.
(57) Material Choice for the Target
(58) Another aspect of target design is material selection. Different materials and colors can be tested for reflectivity property to a laser source. In one embodiment, the materials chosen were polished aluminum, unpolished stainless steel, and the colors chosen were white, red and orange. Polished aluminum and unpolished stainless steel were chosen for their high reflectivity and ease of merchantability. The colors chosen are the most popular ones used in roadside warnings due to their high reflectivity. In order to select the appropriate material for the target, experiments were conducted to see the influence of reflectivity on the accuracy of laser sensor measurements. Table 1 shows the laser sensor measurements when the sensor was placed at a distance of 930 mm perpendicular to target coated with all of these materials.
(59) TABLE-US-00001 TABLE 1 Laser sensor measurement errors (when laser sensor is perpendicular to the target) Laser sensor Material/color measurements Error used (in mm) (in %) White color 932.2393 0.24 Red color 937.7802 0.84 Orange color 936.9589 0.75 Aluminum 931.2392 0.13 (polished) Stainless 933.9255 0.42 steel (Unpolished)
(60) TABLE-US-00002 TABLE 2 Laser sensor measurement errors (when laser sensor is at the worst angle) Laser sensor Material/color measurements Error (in used (in mm) %) White color 933.3101 0.36 Red color 936.874 0.74 Orange color 936.276 0.67 Aluminum 0 100 (polished) Stainless 943.2503 1.42 steel (Unpolished)
(61) Table 2 above shows the laser sensor measurements when the sensor was placed at a distance of 930 mm and scanning the farthest point on the target at the worst angle (14°). In Table 2 it is evident that the polished aluminum reflector behaves like a mirror and hence will not be a good choice for the target material. Using the above experimental data, white color coating can be chosen for the target.
(62) Target Recognition Method
(63) This section will explain the target recognition method used according to one embodiment. Pattern recognition in wheeled mobile robots is used in vision-based localization techniques. The pattern recognition techniques include different measures like Image Euclidean distance and Hausdorff distance. Hausdorff distance is a measure that is widely used in pattern recognition, because of its robustness to noise. In this measure is used to compare pattern set ‘A’ with the image set ‘B’ obtained from the camera feed on the wheeled mobile robot. An exact match is shown when the Hausdorff distance between the two sets is zero. The definition for Hausdorff distance is described in Eq. (3). Given two point sets A and B, the Hausdorff distance between A and B is defined as
(64)
and ∥.∥ represents Euclidean norm. The function h (A, B) is called the directed Hausdorff distance from A to B.
(65) Target Recognition Algorithm
(66) As mentioned above, a linear actuator and rotary actuator can provide the laser sensor with linear and rotary motion capabilities. Of these, the rotary actuation is adapted to scan the frontal area of an IWMR and the linear actuation is adapted to increase the robustness (e.g., in case of an obstacle in front of the target). Here, the assumption is that 1) the linear actuator position is on the left end of the guide way or 2) at least the position at which the linear actuator is located when the laser sensor measurements are taken is known. An exemplary pseudo code for the target recognition method/algorithm is shown in
(67) In
(68)
(69) Target Recognition Algorithm
(70) Eq. (5) can be used to transform the entire set of data collected into IWMR co-ordinates (seen from the laser sensor). When two such edges are detected in the measurements, corresponding time instant points in the IWMR body-fixed co-ordinate information (X.sub.v and Y.sub.v) can be joined by a straight line. The algorithm makes sure that the corresponding time instant points are from the front surface of the target.
(71) The IWMR orientation determination section of the pseudo code is explained with reference to
(72)
(73) Eq. (6) can be used to transform the edge points into the laser sensor co-ordinates X.sub.l-Y.sub.l (seen from the laser sensor). This X.sub.l-Y.sub.l co-ordinate set is the additive inverse of the global co-ordinates x-y. This has been introduced so that the templates of the target can be translated to these co-ordinates and compared; to see if the target is recognized (will be explained in detail below).
(74) Target Template Generation
(75) Three different templates need to be created for every target based on the location of the IWMR. The three zones in front of the target where the IWMR could be located for which three different templates are needed are shown in
(76) Hence while choosing the template; this information can be included as shown in
(77) Target Recognition Experimental Validation
(78) This section shows the experimental results of target recognition for both the targets designed according to one embodiment. As explained above, the rotary actuator motion is a priority in target recognition. The rotary actuator rotates at a constant speed which can be chosen based on a few criteria: At this sampling rate the laser sensor measurements are Gaussian with a standard deviation of about 3 mm. The laser sensor used should have a small amount of latency (0.01 sec, value supplied by the manufacturer) for its measurements to be accurate. Thus, the measurement sampling rate should not be greater than 100 Hz. When the rotary sensor is scanning the entire frontal area (180°), if the scanning speed is too high, the laser sensor accuracy will be affected due to the fact mentioned in the previous point.
(79) Test runs at different speeds were done to choose the ideal speed of the rotary actuator. These results are tabulated in Table 3 below. In these test runs, the rotary actuator was oriented at 0° with respect to X.sub.v axis of the IWMR and at a distance of 675 mm to the target.
(80) TABLE-US-00003 TABLE 3 Localization error for different rotary actuator speeds Rotary Orientation Location actuator error (in error scanning deg) (in mm) speed (Orientation - (Center - (rad/s) 0°) 675 mm) 3.1416 8.4982 20 1.5708 5.2154 2.9 0.7854 1.8826 2.0 0.3927 0.2651 1.8
(81) From Table 3 it is clear that the localization accuracy is affected significantly at high speeds, and when it gets lesser than 0.7854 rad/s, the accuracy does not change much. Thus the 0.7854 rad/s speed can be used to perform target recognition, because it takes less time (compared to 0.3927 rad/s) to scan the entire front area and has good accuracy. The asymmetric target designed (shown in
(82) Determination of IWMR orientation in the global co-ordinates using the exemplar method explained above is shown in
(83) To perform target recognition, the templates have to be translated from the global co-ordinates to the laser sensor co-ordinates. For this translation, the fact that “points on the front surface of the target are selected in the edge detection algorithm” can be used. The entire template can be translated such that the points on the template (those which correspond to the points on the front surface of the target) match the points from the edge detection algorithm. In other words, the templates have to be translated in the global x direction by X.sub.l(t.sub.max(i)) and in the global y direction by the average of Y.sub.l values of points identified in the edge detection part.
(84) Here, X.sub.l(t.sub.max(i)=−252.2514 mm and average of the Y.sub.l values of the points identified in edge detection=617.2743 mm.
(85) The IWMR was located in Zone I, oriented at 90 degrees and at a distance of X=−300 mm and Y=−550 mm for the next set of experiments.
(86)
(87)
(88)
(89) In Zone III, the vehicle was oriented at 0 degrees and at a location X=300 mm and Y=−600 mm. The laser sensor measurements and rotary actuator position is plotted against time in
(90) The symmetric target designed (shown in
(91) TABLE-US-00004 TABLE 4 Target recognition. Target recognition, Location of IWMR (in Hausdorff distance (in mm) and orientation mm) (in deg) Asymmetric Symmetric X Y φ Zone target target 0 −615 45 1 36.7957 31.4817 −300 −550 90 2 40.9363 32.7109 300 −600 0 3 37.7446 15.1800
(92)
(93) IWMR Localization Method and Results
(94) The experimental results of the target recognition algorithm show that the designed targets can be recognized. Although that in itself is a validation for the target design, this section presents the experimental results that show the localization results for the designed asymmetric target.
(95)
(96) Analytical Determination of Location and Orientation
(97) Here, the fact that a quadrilateral is formed between the two distance measurements (m.sub.1L and m.sub.2R), the laser sensor guide way length (L) and the target object length (l) can be used to find the orientation of the vehicle in the global co-ordinates.
(98) The area of quadrilateral using the sides and diagonals given in Eq. (7) (which are known) can be equated to the area of quadrilateral using sides and the angles included given in Eq. (8) (one of them is unknown). The unknown angle β can be calculated from Eq. (9).
(99)
The angle made by the line joining the right end of the laser sensor guide way to the right end of the target with the global positive x axis (x) can be found from the previous step to be equal to the unknown quantity β. The vehicle orientation with respect to the global positive x-axis (x) ‘φ’ (shown in
(100)
The location of the vehicle (the left end of the scanner guide-way in this case) with respect to the left end of the target on the vehicle co-ordinate axes can be found out using simple kinematic analysis. Eq. (12) shows the formula for the same.
(101)
Here, a unit step function u.sub.s(□) is included, whose value is 1 when its argument is non-negative and zero otherwise. This unit step function is included so that based on the laser sensor measurements, the IWMR location in quadrant III or quadrant IV can be distinguished. Then the appropriate measurements can be used to determine the IWMR's position. The location on the vehicle co-ordinate axis can then be transformed to the global co-ordinate axis by using the transformation matrix shown in Eq. (13).
(102)
(103)
(104) In
(105) TABLE-US-00005 TABLE 5 Localization results. Actual location (in mm) and Localization orientation (in Localization results (in deg) results (in mm) deg) x y φ x y φ −600 −400 0 −598.72 −400 0 400 −400 0 399.28 −398.78 0 −200 −400 90 −200 −398.54 90 −200 −800 45 −199.54 −798.73 45
(106) The experimental location and the localization results for the experiments carried out in
(107) From these experimental results, it will be appreciated by those skilled in the art that the analytical determination for location and orientation works well, even with noise filled laser sensor measurements.
(108) Analysis of Design Parameters
(109) Next, optimizing design parameters, e.g., a) length of the target and b) length of the laser sensor guide way, will be described according to one embodiment. Eq. (10) shows that noise in the laser distance measurements m.sub.1L and m.sub.2R are related to the design parameters. Though there will be noise in measurements m.sub.1R and m.sub.2L, their influence on the localization results generally cannot be minimized by optimizing the design parameters. Hence, efforts are taken to reduce the influence of noise in measurement m.sub.1L on the orientation results (δφ′).
(110) Since the location results can depend directly on the orientation results, design optimization of the target is done for the orientation results only. It can be presumed that the preferred orientation results are—localization error has a magnitude equal to or less than that of noise in measurements. The orientation formula in Eq. (10) can be re-written as follows:
(111)
(112) Subtracting Eq. (15) form Eq. (14)
(113) Left hand side:
(114)
(115) If we can assume δφ′ to be small, then under small angle approximation RHS of Eq. (16) can be rewritten as
(116)
(117) Subtracting Eq. (15) from Eq. (14)
(118) Right hand side:
(119)
(120) In Eq. (20) we can see that the term
(121)
is going to go to zero, because we are dividing a small value δm by a denominator which is the fourth power of measurements. When we equate this term to zero, the terms inside the square root are exactly same and get cancelled. Thus the RHS becomes
(122)
(123) Equating Eq. (17) and Eq. (22) we get
(124)
(125) In Eq. (23), left hand side is equal to or less than one if we want the error in orientation results to be less in order than that of noise in measurement. This gives a limit on the design parameters which is given in Eq. (24).
(126)
(127) The right hand side of the above expression can vary between [0, m.sub.2Rmax]. Here, m.sub.2Rmax is the maximum value of m.sub.2R. Though mathematically, Eq. (24) has a lower bound of 0, realistically we can say that
(128)
(129) The reasoning behind that is—If we have L=0, then the technique might lose its robustness. When the laser sensor is made to stand at a single point there is a chance that the target might not be detected. Hence Eq. (25) is the choice for an upper bound. Continuing, there is also a lower limit on the design parameters. In one embodiment, the design parameters have to be greater than twice the value of noise in the laser sensor distance measurements. This gives a lower bound on the design parameters given in Eq. (26). In Eq. (26) l.sub.max denotes the maximum length of the target.
(130)
(131) Simulation Results—Localization
(132) In this section the simulations that were carried on according to one embodiment to evaluate the vehicle orientation and localization formulae are shown. In particular, a MATLAB based code was developed for the same and the results are shown below. In this simulation, the noise of measurements is approximated by a white Gaussian noise.
(133) In
(134)
(135) In one example, sensor noise was considered to be 2% with m.sub.2Rmax=2. For such a sensor, the optimal design parameters will be L=2*l.
(136)
(137) Simulation Results—Influence of Design Parameters
(138) In this section the simulations that were carried on to evaluate the design parameter analysis are shown.
(139) In this plot the target length ‘l’ is increased along with an increase in the laser sensor guide way length ‘L’ such that increase in L is greater. Intuitively one might suggest that increase in target length has to give better localization results. But
(140) The assumption thus far in the entire work is that θ.sub.1L, θ.sub.1R, θ.sub.2L, θ.sub.2R are known accurately. But the assumption cannot be always true due to limitations on the finite resolutions of angular measurements. The angular measurements have their own accuracy which will influence the orientation results as seen from the equations above. In order to have a rough estimate on how the results are going to be affected by the angular measurement resolution error, the plot shown in
(141) This disclosure presents an IWMR global localization technique using only an onboard laser sensor. Laser sensor is used to recognize the pattern of a unique target, which in turn is used as the reference to localize globally. A kinematic model for the global localization of IWMR using laser sensor alone was developed and presented. Also, the target design and the target recognition technique are presented. Hausdorff distance based template comparison technique was used as the target recognition approach.
(142) The experimental results show clearly that this technique can be implemented merely placing the designed target in a known global location in an unknown environment. The experimental results show that error in localization results is lesser than the error in laser sensor measurements which shows that this technique is robust to noise.
(143) It will further be appreciated that several of the above-disclosed and other features and functions, or alternatives or varieties thereof, may be desirably combined into many other different systems or applications. Also that various presently unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the present disclosure and/or the following claims.
(144) Nomenclature l Length of the target d.sub.i Depth of edge i in the target w.sub.i Width of depth i in the target L Length of the laser sensor guide way θ.sub.1L Angle made by the laser sensor from the left end on the guide way to the left end of the target with positive x axis. θ.sub.1R Angle made by the laser sensor from the left end on the guide way to the right end of the target with positive x axis. θ.sub.2L Angle made by the laser sensor from the right end on the guide way to the left end of the target with positive x axis. θ.sub.2R Angle made by the laser sensor from the right end on the guide way to the right end of the target with positive x axis. Φ IWMR orientation with respect to the positive Xv axis used for target recognition alone. m.sub.1L Measurements denoted using the same convention as the angles above. u.sub.s Unit step function X.sub.v, Y.sub.v IWMR body-fixed co-ordinates. φ IWMR orientation by localization. x.sub.v,y.sub.v IWMR location in the body-fixed coordinates. x,y IWMR location in global co-ordinates. δm Noise in measurement of m.sub.1n. δφ Error in orientation results due to noise in measurements. δφ′ Error in orientation results due to δm alone. l.sub.max Maximum length of the target.