Autonomously guided machine
11086331 · 2021-08-10
Assignee
Inventors
- Alejandro Dizan Vasquez-Govea (Palo Alto, CA, US)
- Jan Michalczyk (Grenoble, FR)
- Stanislas Borkowski (Brignoud, FR)
Cpc classification
International classification
Abstract
The invention relates to an autonomously guided machine (10) comprising at least a steering body and a steering mechanism (13), said machine also comprising: a guide system (11) including an optical filter (17) which has a receiving surface exposed to the exterior and which can transmit light rays directed substantially in one direction and eliminate other light rays; a screen (18) that receives the filtered light; a digital camera (19) for capturing pixel resolution images of the screen (18); a pre-processing element for selecting the pixels corresponding to an incident planar laser beam received by the guide system (11), and filtered and projected onto the screen (18), and for generating a response detection image; a line generator for generating an estimate J of the coordinates of the detected lines based on the response detection image, involving the processing of the pixel columns pixels; and a locator which receives the estimate of the coordinates of the detected lines and calculates values representative of pixel columns based on said estimate and on parameters of the guide system (11), and subsequently calculates a distance y.sub.w from the guide system (11) to the planar laser beam in order to control the steering mechanism (13) so as to minimise this distance y.sub.w.
Claims
1. An autonomously guided machine which comprises at least a steering member and a steering mechanism, wherein the machine further comprises a guidance system including an optical filter which has a receiving face exposed to the exterior and which transmits light rays directed substantially in one direction and eliminates other light rays; a screen receiving the filtered light; a digital camera to capture images resolved into pixels of said screen; a pre-processing member that selects the pixels corresponding to an incident plane laser beam received by the guidance system, filtered and projected onto said screen, and can generates a detection response image; a line detector generating, on the basis of the detection response image, by processing on the columns of pixels, an estimate J of the coordinates of the detected lines; and a locator that receives the estimate of the coordinates of the detected lines and calculates values representative of columns of pixels on the basis of said estimate of the coordinates of the detected lines and parameters of the guidance system, and then calculates a distance y.sub.w of the guidance system from the plane laser beam in order to control the steering mechanism to minimize said distance y.sub.w.
2. The machine according to claim 1, wherein the optical filter comprises a honeycomb structure with through cells, to transmit the light rays along the axis of the through cells.
3. The machine according to claim 1, wherein the detection response image is single-channel, and each pixel of said detection response image has a value which is a function of a probability that said pixel belongs to the line of the laser beam.
4. The machine according to claim 1, wherein the pre-processing member comprises a red-green-blue primary colors to hue-saturation-value (HSV) converter, which converts each pixel of the output image of the camera into a pixel of an HSV image, an equalizer for each of the saturation and value channels of the HSV image, and a calculator of an exponential function of the weighted Euclidian distance between an HSV pixel and a reference pixel:
O.sub.i,j=e−(w.sub.H(I.sup.H.sub.i,j−L.sup.H).sup.2+w.sub.S(I.sup.S.sub.i,j−L.sup.S).sup.2+w.sub.V(I.sup.V.sub.i,j−L.sup.V).sup.2) where O.sub.i,j is the value of the pixel having coordinates i and j of the single-channel image forming the pre-processing output; w.sub.H, w.sub.s, w.sub.v are the weightings for the channels H, S, V; I.sup.H.sub.i,j, I.sup.S.sub.i,j, I.sup.V.sub.i,j is the value of the input pixel in each channel; and L.sup.H, L.sup.S, L.sup.V is the value of the reference pixel in each canal.
5. The machine according to claim 1, wherein the locator comprises a calculator of the distance y.sub.w of the guidance system from the plane laser beam as a function of the estimate J of the coordinates of the detected lines, of a parameter m of distance between the screen and the optical filter, and of a parameter b of centering of the screen, where y.sub.w=m J+b.
6. The machine according to claim 1, wherein the screen is adjacent to the optical filter.
7. The machine according to claim 1, wherein the screen comprises two parallel parts, spaced apart in the direction of propagation of the incident laser beam, and the locator calculates an angular deviation α of the guidance system from the plane laser beam in order to control the steering mechanism.
8. The machine according to claim 7, wherein the locator, after the calculation of the values representative of columns of pixels, calculates the angular deviation α of the guidance system from the plane laser beam in order to control the steering mechanism to minimize said distance y.sub.w.
9. The machine according to claim 8, wherein the locator comprises a calculator of the angle between the plane of the laser beam and the axis of the guidance system, where α=Atan2(y.sub.2−y.sub.1; x.sub.1+x.sub.2), said axis being normal to each part of the screen.
10. A machine guidance assembly comprising a movable guidance system a stationary plane laser beam transmitter, the guidance system including an optical filter which has a receiving face exposed to the exterior and which transmits light rays directed substantially in one direction and eliminates other light rays; a screen receiving the filtered light; a digital camera to capture images resolved into pixels of said screen; a pre-processing member that selects the pixels corresponding to an incident plane laser beam received by the guidance system, filtered and projected onto said screen, and generates a detection response image; a line detector generating, on the basis of the detection response image, by processing on the columns of pixels, an estimate J of the coordinates of the detected lines; and a locator that receives the estimate of the coordinates of the detected lines and calculates values representative of columns of pixels on the basis of said estimate of the coordinates of the detected lines and parameters of the guidance system, and then calculates a distance y.sub.w of the guidance system from the plane laser beam in order to generate a control signal to minimize said distance y.sub.w.
Description
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
(1) Other characteristics and advantages of the invention will be apparent on examination of the following detailed description, and of the attached drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14) The attached drawings may not only clarify the understanding of the present invention but also contribute to the definition of the invention if necessary.
DETAILED DESCRIPTION OF THE INVENTION
(15) In the course of his research, the Applicant became aware of the unsatisfactory nature of the known devices. GPS has progressively become a major consumer product in its low accuracy versions. Differential GPS requires ground stations and is too costly in the field concerned. Guidance based on a probabilistic model, using a Bayes filter for example, is highly costly in terms of computer resources and requires expensive sensors such as laser distance scanners. It uses sensors specific to the machine concerned, for example wheel encoders, an inertial controller and the like, environmental sensors and a map. Some systems require beacons located in the maneuvering area of the machine. This is hardly suitable for agriculture. Accurate mapping must vary with ice, rain, wind, etc., as well as with the type of crop.
(16) There are guidance systems based on a stationary infrastructure of radar systems or video cameras communicating with the movable machine. Neither the adaptation nor the costs are feasible for agriculture or for other fields with high constraints.
(17) The Applicant has identified certain particularities of agricultural applications, and more particularly truck farming or horticultural applications. The distance to be traveled between two turns of the machine is relatively well known, and it is easy to set an upper limit. The crops are planted in straight lines. Guidance may be considered as aiming to approach a straight line as far as possible. Additionally, since the same guidance system is used for successive cultivation of the same crop, any faults inherent in the guidance system will be repeated, and therefore will have no significant effect.
(18) A guidance system based on following a plane laser beam is proposed. A laser transmitter 100 (see
(19) As shown in
(20) The machine 10 is equipped with the guidance system 11. The guidance system 11 is designed to receive the incident laser beam 15. The guidance system 11 is supported at a rear end of the machine 10 to facilitate the initial alignment when the user starts a new line. The guidance system 11 has a free end wall for directly receiving the incident laser beam 15 which is shown as a bold line. The guidance system 11 comprises an opaque casing 16 open at one end, an optical filter 17 mounted at said open end, a screen 18 mounted in the casing 16, and a digital camera 19 mounted in the wall of the casing 16 opposite the optical filter 17. The direction from the open end to the opposite end defines the X axis of the casing 16.
(21) The axis of the casing 16 is parallel to the propagation axis of the incident laser beam. The casing 16 is parallelepipedal. The casing 16 may act as a frame for other elements of the guidance system 11. The casing 16 is opaque at least at the wavelength of the laser beam 15, and is preferably opaque over a wide range of wavelengths. The screen 18 is mounted parallel to the optical filter 17. The screen 18 is mounted at a distance from the optical filter 17 and at a distance from the camera 19. In one embodiment, the screen 18 is plane and made in one piece.
(22) In the embodiment shown, the screen 18 is biplanar: see
(23) The optical filter 17 (see
(24) The cells 20 are square in the embodiment shown, and are arranged in rows and columns. In
(25) The cells 20 have a width, depth and height that define the chosen angular aperture. The aperture may be between 5° and 100, or preferably between 5° and 7°.
(26) In a preferred embodiment, the distance between the optical filter 17 and the screen 18 is zero: see
(27) The honeycomb structure is made of synthetic material which is molded or 3D printed. The color of the honeycomb structure is black. The cells 20 have a length of between 5 and 60 mm. The cells 20 have a width of between 2 and 15 mm. The honeycomb structure comprises inter-cell walls with a thickness of between 0.2 and 2 mm. A square arrangement facilitates manufacturing with a 3D printer. As shown in
(28) The camera 19 is a digital camera resolved into pixels. The resolution, which has been experimented, is 480×640. The results are satisfactory. The resolution is between 100,000 and 600,000 pixels, in order to remain in optimal operation. The camera 19 captures the image displayed on the screen 18, originating from the light penetrating into the casing 16 through the optical filter 17. Said filtered light comprises components of ambient light and of the incident laser beam 15, as long as the lateral deviation and the angular deviation of the casing 16 from the incident laser beam 15 remains limited. An output image of the camera 19 is reproduced in
(29) The lateral deviation limit is substantially equal to the half-width of the optical filter 17. The half-width of the optical filter 17 may correspond to the half-width of the casing 16. The angular deviation limit is substantially determined by the minimum of the width to depth ratio of the cells 20 and of the ratio of the half-width of the optical filter 17 to the distance between the optical filter 17 and the screen 18. In practice, the angular deviation limit is substantially determined by the shape ratio of the cells 20. In
(30) The distance between the camera 19 and the screen 18, notably the part of the screen 18 nearest to the camera 19, is chosen on the basis of the field of the camera 19, notably. The filtered light spatially illuminating the screen 18 is centered on the axis of the casing 16. The camera 19 supplies at its output a digital image, usually in three color channels, red, green and blue (abbreviated to RGB). The image capture frequency is between 10 Hz and 30 Hz.
(31) The guidance system 11 (see
(32) Advantageously, the locator 50 calculates an angular deviation α of the guidance system 11 from the plane laser beam 15. The angular deviation α is sent as the output for controlling the steering mechanism 13 so as to minimize said angular deviation α. An angular deviation α of zero signifies that the guidance system 11 is aligned on the laser beam 15. The calculation of the angular deviation α is based on differential data.
(33) The pre-processing member 30 eliminates components of the image originating from anywhere other than the incident laser beam 15. The pre-processing member 30 comprises a red-green-blue primary colors to hue-saturation-value (HSV) converter 31, converting each pixel of the output image of the camera 19 into a pixel of an HSV image. The converter 31 may be a commercially available component. The hue is encoded on a color circle. The saturation is the intensity of the color. The value is the brilliance of the color, 0 being black. The converter 31 receives an image composed of RGB pixels and outputs an image composed of HSV pixels.
(34) The pre-processing member 30 comprises an equalizer 32 using histogram equalization for each of the saturation and value channels of the HSV image. The histogram is applied independently for the saturation channel and the value channel:
(35)
(36) Where T(x.sub.k) is the output value of the pixels of level x.sub.k, L is the number of levels of the channel concerned, n is the number of pixels in the image, and n.sub.j is the number of occurrences of pixels of level x.sub.j.
(37) The aim of applying histogram equalization is to maximize the contrast between the different values, thus facilitating classification in the subsequent steps. Since it is applied individually for each channel, the histogram is not the same from one channel to another. The equalization of the three channels may be carried out simultaneously. Hue equalization may introduce distortions in the color. In practice, the Applicant has found that the improvement in contrast is more important than this distortion. The equalizer has the same output for all the values of x.sub.k when the number of occurrences is the same for all the levels of the channel.
(38) Said equalizer 32 receives the HSV image and supplies at its output an equalized HSV image providing improved contrast.
(39) The pre-processing member 30 (see
O.sub.i,j=e−(w.sub.H(I.sup.H.sub.i,j−L.sup.H).sup.2+w.sub.S(I.sup.S.sub.i,j−L.sup.S).sup.2+w.sub.V(I.sup.V.sub.i,j−L.sup.V).sup.2)
(40) where O.sub.i,j is the value of the pixel having coordinates i and j of the single-channel image forming the pre-processing output; w.sub.H, w.sub.S, w.sub.V are the weightings for the channels H, S, V; I.sup.H.sub.i,j, I.sup.S.sub.i,j, I.sup.V.sub.i,j is the value of the input pixel in each channel; and L.sup.H, L.sup.S, L.sup.V is the value of the reference pixel in each canal. The reference pixel is common to the whole image. The weightings w.sub.H, w.sub.S, w.sub.V of the H, S, V channels are chosen on the basis of samples in which the laser has been projected into the device in realistic and diverse conditions, with the aim of covering a large number of operating situations. The values of the weights are greater than 0, and, since ambient light is much more saturated than laser light, the saturation is relevant information for the detection of the laser.
(41) The output image of the pre-processing member 30 is a detection response image O, shown in
(42) The detection response image O is a synthesis of the three HSV channels. The detection response image O is single-channel. Each pixel of said detection response image has a value which is a function of the probability that said pixel does or does not belong to the line of the laser beam 15.
(43) In other words, the converter 31 can provide an image format more suitable for subsequent processing. The equalizer 32 improves the contrast. The calculator 33 makes it possible to assign to each pixel O.sub.i,j a value of 1 if the equalized HSV pixel I.sub.i,j is identical to the reference pixel and therefore has the same hue, the same saturation and the same value as an ideal pixel illuminated by the incident laser beam 15, and a very low value, O.sub.i,j=e−(w.sub.H(L.sup.H).sup.2+w.sub.S(LS).sup.2+w.sub.V(L.sup.V).sup.2), if the equalized HSV pixel I.sub.i,j is black. Thus, the output of the converter 31, forming the output of the pre-processing member 30, provides a detection response image representative of the areas illuminated by the incident laser beam 15. The pixels corresponding to the incident plane laser beam 15 received by the guidance system 11, filtered and projected on said screen 18, have been selected by the pre-processing.
(44) The line detector 40 receives at its input the detection response image O comprising the pixels O.sub.i,j. The value of the pixel O.sub.i,j is proportional to the probability that said pixel O.sub.i,j belongs to the light line projected by the incident laser beam 15 on the screen 18. The line detector 40 generates an estimate J of the coordinates (in column numbers) of the detected lines, on the basis of the detection response image O, by processing on the columns of pixels.
(45) The line detector 40 (see
(46) The line detector 40 comprises a high-pass filter 42 receiving the column parameters C.sub.j. The high-pass filter 42 eliminates the columns for which the parameter C.sub.j is too low. The high-pass filter 42 calculates the maximum C.sub.max of said column parameters C.sub.j: C.sub.max=max(C.sub.j). The high-pass filter 42 applies a chosen filter coefficient τ. The filter coefficient z is constant. The high-pass filter 42 calculates the product τC.sub.max of the maximum C.sub.max and the filter coefficient z. The high-pass filter 42 eliminates the columns for which the parameter C.sub.j is less than the product τC.sub.max. Thus the low-value columns that have a low probability of corresponding to an area of the screen 18 illuminated by the incident laser beam 15 are eliminated. The columns relevant to finding the position of the incident laser beam 15 on the screen 18 are selected.
(47) The filtering is applied to the result of the classification. In a variant, the filtering is carried out at the same time as the classification. The filtering parameter is constant.
(48) The above description of the line detector 40 makes a simplification by assuming that the detected lines originating from the incident laser beam 15 are vertical. If it is desired to allow for a relative inclination of the plane of the laser beam 15 and the plane of the pixel columns, it is possible to add an inclination estimator, for example one based on the least squares method.
(49) The line detector 40 comprises a column weighting member 43. The weighting member 43 receives the columns selected by the high-pass filter 42 with their original index j attached. The column index j is retained. This is equivalent to accepting a discontinuity in all the indices of the selected columns. The knowledge of the position of each selected column in the HSV image is preserved. The knowledge of the column is translated into a coordinate of intersection between the laser and the screen.
(50) The weighting member 43 sums the relevant column parameters C.sub.j, weighted by their index j in the output image of the camera 19, i.e. Σ.sub.j jC.sub.j. The weighting member 43 sums the relevant column parameters C.sub.j, i.e. Σ.sub.j C.sub.j. The weighting member 43 finds the ratio of the sum of the parameters weighted by their index j in the output image of the camera 19 to the sum of the parameters. The weighting member 43 supplies at its output the column index value J forming an estimate of the coordinates of the incident plane laser beam 15 in the output image of the camera 19: J=Σ.sub.j jC.sub.j/Σ.sub.j C.sub.j. The line detector 40 supplies on the basis on the detection response image an estimate of the coordinates of the incident plane laser beam 15 in the output image of the camera 19. J is between 1 and the number of columns of the output image of the camera 19.
(51) The locator 50 receives said estimate J to calculate the distance y.sub.w. The estimate J is proportional to the distance y.sub.w. In the embodiment with a plane screen 18, the locator 50 calculates the distance y.sub.w by linear interpolation y.sub.w=mJ+b. The parameter m is the distance between the screen 18 and the optical filter 17. The parameter b is representative of the centering of the screen 18. The parameters m and b are constants for a given machine. The parameters m and b may be calibration parameters which differ from one guidance system to another, or from one series of guidance systems to another.
(52) In the embodiment with a screen 18 in two parallel parts 18a, 18b, the two parts 18a, 18b are high and low respectively. The locator 50 calculates the distance y.sub.w by linear interpolation on the basis of each part 18a, 18b of the screen 18. In the above procedure, a separation is performed to provide an estimate J.sub.1 relating to the part 18a and an estimate J.sub.2 relating to the part 18b. The parameters m.sub.1 of distance between part 18a of the screen 18 and the optical filter 17, b.sub.1 of centering of part 18a of the screen 18, m.sub.2 of distance between part 18b of the screen 18 and the optical filter 17, and b.sub.2 of centering of part 18b of the screen 18 are known from the structure of the guidance system 11. Generally, b.sub.1=b.sub.2.
(53) The locator 50 performs the interpolations y.sub.1=m.sub.1J.sub.1+b.sub.1 and y.sub.2=m.sub.2J.sub.2+b.sub.2. y.sub.1 and y.sub.2 are the distances of the guidance system 11 from the plane laser beam 15 in part 18a and part 18b of the screen 18. The locator 50 finds the mean y.sub.w=(y.sub.1+y.sub.2)/2. The distance y.sub.w is a deviation in a horizontal plane between the center of the guidance system 11 and the plane laser beam 15.
(54)
(55) In the embodiment with a screen 18 in two parallel parts 18a, 18b, the locator 50 comprises a calculator 51 of the angle α between the plane of the laser beam 15 and the axis of the guidance system 11, where α=Atan 2(y.sub.2−y.sub.1; x.sub.1+x.sub.2), said axis being normal to each part of the screen 18. The sum x.sub.1+x.sub.2 is the distance between the two parts 18a, 18b of the screen 18: see
(56) By way of example, a large angle α and a zero distance y.sub.w signify that the machine is centered on the guiding laser beam but is misaligned. In the plane screen embodiment, no correction is made because the zero distance y.sub.w is satisfactory, except for appropriate adjustments of the proportional and derivative modes of the correction. The derivative y′.sub.w may provide relevant information, but this is less precise than the angle α. In the embodiment with offset partial screens, the high angle α enables a correction to be made immediately, for example from the start of a cultivation row when the machine has just crossed the guiding laser beam and the guidance system sends output information indicative of centering and misalignment.
(57) The invention also proposes a guidance assembly comprising the guidance system 11 described above and at least one appropriate laser transmitter.