Residue Spread Monitoring

Abstract

Systems and methods for monitoring the distribution of residue material from a spreader tool of an agricultural machine including an imaging sensor coupled to an unloading auger of the agricultural machine used to image an area to the rear of the agricultural machine where the Image data is analysed to determine a distribution of residue material associated with the spreader tool and one or more operational parameters of the agricultural machine or components are controlled based on the determined distribution.

Claims

1. A system for monitoring the distribution of residue material from a spreader tool of an agricultural machine, the system comprising: an imaging sensor mounted or otherwise coupled to an unloading auger of the agricultural machine, and having a sensing region rearwards of the agricultural machine; and at least one controller, configured to: receive image data from the sensor indicative of residue material spread by the spreader tool within the sensing region; analyse the image data to determine a distribution of residue material associated with the spreader tool; and output at least one control signal for controlling at least one operational parameter of the agricultural machine or at least one component of the machine based on the determined distribution.

2. The system of claim 1, comprising or otherwise communicably coupled to an inertial measurement unit (IMU) either mounted or otherwise coupled to the agricultural machine.

3. The system of claim 2, wherein the at least one controller is configured to determine the residue distribution based on data received from the IMU.

4. The system of claim 3, wherein the at least one controller is configured to determine an orientation of the imaging sensor relative to ground surface and determine the distribution based on the determined orientation.

5. The system of claim 1, wherein the at least one controller is operable to control a position of the auger.

6. The system of claim 5, comprising or otherwise communicably coupled to an inertial measurement unit (IMU) either mounted or otherwise coupled to the agricultural machine, wherein the at least one controller is configured to control a position of the auger based on orientation information obtained from the IMU.

7. The system of claim 1, wherein the at least one controller is configured to analyse the image data from the imaging sensor through a feature extraction process and identify at least one feature comprising boundaries or edges in the image data, certain shapes in the image data corresponding to an expected shape, and size of residue material in the image data.

8. The system of claim 1, wherein the at least one controller is configured to determine, from the analysis of the image data, at least one characteristic of the residue distribution.

9. The system of claim 8, wherein the at least one controller is configured to control the at least one operational parameter or component of the machine based on the at least one determined characteristic.

10. The system of claim 8, wherein the at least one characteristic of the distribution of residue material comprises: a shape of the distribution; a skew of the distribution; a distance from the spreader tool at which the residue material is spread; a maximum lateral extent of residue material spread by the spreader tool; a density of material across the distribution of residue material; and a measure of uniformity of the distribution of residue material across a width, or part of the width, of the distribution.

11. The system of claim 1, wherein the at least one operational parameter or component of the agricultural machine controllable by the system comprises a user interface which provides information corresponding to the determined distribution.

12. The system of claim 1, operable to control at least one operating parameter of the agricultural machine based on the determined distribution, wherein the at least one operational parameter comprises at least one operating parameter of the spreader tool.

13. An agricultural machine comprising the system of claim 1.

14. A method of monitoring the distribution of residue material from a spreader tool of an agricultural machine, comprising: receiving, image data from an imaging sensor mounted or otherwise coupled to an unloading auger of the agricultural machine and having a sensing region rearwards of the agricultural machine, wherein the image data is indicative of residue material spread by the spreader tool within the sensing region; analysing the image data to determine a distribution of residue material associated with the spreader tool; and controlling at least one operational parameter or component of the agricultural machine based on the determined distribution.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0036] One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0037] FIG. 1 is a schematic side cross-sectional view of an agricultural harvester embodying aspects of the invention;

[0038] FIG. 2 is a schematic view of an embodiment of a control system of the invention;

[0039] FIGS. 3A and 3B are schematic views of the rear of the combine illustrating an operational use of aspects of the invention; and

[0040] FIG. 4 is a schematic view illustrating image data obtained using aspects of the invention.

DETAILED DESCRIPTION

[0041] FIG. 1 illustrates an agricultural machine, and specifically a combine 10, embodying aspects of the present invention.

[0042] The combine 10 is coupled to a header 12 which is operable, in use, to cut and gather a strip of crop material as the combine 10 is driven across a field/area to be harvested during a harvesting operation. A conveyor section 14 conveys the cut crop material from the header 12 into a crop processing apparatus 16 operable to separate grain and non-grain (i.e. material other than grain (MOG) or residue material (used interchangeably herein)) as will be appreciated. It is noted here that apparatus for separating grain and non-grain material are well-known in the art and the present invention is not limited in this sense. The skilled person will appreciate that numerous different configurations for the crop processing apparatus may be used as appropriate. Clean grain separated from the cut crop material is collected in a grain bin 18, which may be periodically emptied, e.g. into a collection vehicle, storage container, etc. utilising unloading auger 20. The remaining non-grain material (MOG)/residue material is separately moved to a spreader tool 22 which is operable in use to eject the non-grain material or MOG from the rear of the combine 10 and onto the ground. In FIG. 1, this is represented by arrow 24 which illustrates the MOG being ejected rearwards from the combine 10. It will be appreciated that in some embodiments the combine 10 may also include a chopper tool positioned, for example, between the crop processing apparatus 16 and the spreader tool 22 and operable, in use, to cut the residue material before it is spread by the spreader tool 22.

[0043] The combine 10 also typically includes, amongst other features, an operator cab 26, wheels 28, engine (not shown) and a user interface 32.

[0044] As will be discussed in detail herein, the combine 10 additionally includes an imaging sensor in the form of a camera 30. The camera 30 is mounted on the underside of the unloading auger 20 and is used, by a control system 100 of the combine, to determine one or more characteristics of a distribution of residue material associated with the spreader tool 22 in the manner discussed herein.

[0045] FIG. 2 illustrates control system 100 further. As shown, control system 100 comprises a controller 102 having an electronic processor 104, an electronic input 106 and electronic outputs 108, 110. The processor 104 is operable to access a memory 112 of the controller 102 and execute instructions stored therein to perform the steps and functionality of the present invention discussed herein, e.g. by controlling the user interface 32, for example to provide an image or an indicator to an operator of the combine 10 illustrative of the observed residue material distribution.

[0046] The processor 104 is operable to receive sensor data via input 106 which, in the illustrated embodiment, takes the form of input signals 105 received from the camera 30. As described in detail herein, the camera 30 has a sensing region rearward of the combine 10, with the sensor data received from the camera 30 being indicative of a residue material within the sensing region.

[0047] Using this information, the processor 104 is operable to determine residue material distribution in the manner described herein. Specifically, the processor 104 is operable to analyse the image data received from the camera 30 to identify one or more residue material pieces and/or one or more characteristic(s) of the residue distribution within the image data. Based on this, control over one or more systems of the combine 10 is performed, as described herein.

[0048] The processor 104 is also operable to receive data via input 106 from an inertial measurement unit (IMU) 33 in the form of data signals 113. The data from the IMU 33 may be used as discussed herein to determine an orientation of the combine 10 relative to a horizontal surface and use this to determine the distribution in dependence thereon.

[0049] In the illustrated embodiment of FIG. 2, output 110 is operably coupled to the user interface 32 of the combine 10. Here, the control system 100 is operable to control operation of the user interface 32, e.g. through output of control signals 111 in order to display operational data to an operator of the combine 10 relating to the operation of the control system 100. Specifically, the control system 100 may be operable to control the user interface 32 to display to the operator a graphical representation of the residue material distribution from the spreader tool 22 as determined by processor 104, image data obtained from the camera 30, or other useful information, including a representation of one or more determined characteristic(s) of the distribution (see below). The user interface 32 may also be operable to receive a user input from the operator, and in such instances the output 110 may act as an input for receiving that user input at the processor 104. The user input may relate to a requested or desired distribution of residue material, for example, made by the operator of the combine 10.

[0050] In a variant, as illustrated by FIGS. 1 and 2, the processor 104 is operable to generate and output control signals 109 via the output 108 for controlling operation of the spreader tool 22, and more specifically first and second steering units of the spreader tool 22, here in the form of a first rotor 23a and a second rotor 23b, for controlling the distribution of residue material ejected from the spreader tool 22 in dependence on the determined distribution.

[0051] FIGS. 3A, 3B and 4 illustrate the operational use of aspects of the invention.

[0052] FIGS. 3A and 3B are schematic views of the rear of combine 10. In FIG. 3A, combine 10 is shown on a relatively flat ground surface G, with camera 30 positioned substantially directly above spreader tool 22, as illustrated by vertical axis Y. This is considered to be an optimum position for the camera 30 in this scenario to give a field of view of the camera 30 which encompasses either side of the combine 10 equally, or at least either side of the spreader tool 22. In FIG. 3B, the combine 10 is positioned on a ground surface which is angled with respect to a horizontal axis, here by angle θ. As shown, with auger 20 (and hence camera 30) positioned in the same place as in FIG. 3A, the camera 30 would no longer be positioned vertically above an outlet of the spreader tool 22. Rather, the present invention advantageously utilises information from the IMU 33 (not shown in FIGS. 3A and 3B) for determining the angle θ and thereby make a corresponding adjustment to the position of the auger 20 (and hence camera 30) to position the camera substantially vertically above the outlet of the spreader tool 22. This is indicated in FIG. 3B by moving the auger 20 in the direction illustrated by arrow “B”.

[0053] In an extension of the present invention, the auger 20 may be moved to move the position of the camera 30 relative to the spreader tool 22 for other reasons, including to overcome obstructions in the image data obtained of the sensing region. This can include physical obstructions, e.g. overhanging crop or other vegetation, components of the combine 10, etc. intruding on the images obtained by the camera 30, or visual obstructions including glare on the camera 30, or shadows covering at least part of the sensing region. The position of the auger 20 may be advantageously moved to adjust the camera 30 position to account for such conditions.

[0054] FIG. 4 illustrates a sample image 200 obtained through use of the camera 30, and is used here to illustrate an operational use of aspects of the invention.

[0055] The processor 104 comprises an image processing module configured to analyse, using a feature extraction process, the image 200 obtained by the camera 30 to identify different regions within the image 200. Here, this includes classifying each pixel of the image 200 based on a classifier corresponding to a value of the pixel. The invention is not limited in this sense, but preferably includes either an RGB value or greyscale value, depending on the camera type used. In the illustrated embodiment, the processor 104 is configured to classify each pixel into one of four different categories including a “Residue” category” where the pixel value is deemed to be indicative of the presence of material at that point in the image data, a “Ground” category for pixel values indicative of a ground surface over which the machine is travelling, a “Vehicle” or “Machine” category for pixel values indicative of the presence of at least part of the agricultural machine (or components thereof) in the image data, and an “Other” category for pixel values which do not fall within expected pixel values for each of the other categories making up the classification. FIG. 4 illustrates image 200 which has been analysed with pixels classified to form image data with an area of residue material 25 (split here into 25a and 25b— discussed herein), a ground surface 38, vehicle 10 and adjacent standing crop 40 classified as “Other”.

[0056] With the individual pixels classified by the processor 104, the classified image data is further analysed to identify one or more features including boundaries or edges in the image data (e.g. a boundary between different areas of classifications), or certain shapes in the image data corresponding to an expected shape and size of residue material in images obtained by the camera 30.

[0057] With the processed image data, the processor 104 is configured to determine one or more characteristics of the observed residue material distribution. FIG. 4 illustrates multiple different characteristics which may be determined from the image 200.

[0058] For instance, the processor 104 may be configured to determine an indication of the shape of the distribution.

[0059] The processor 104 may be operable to determine a skew of the distribution. Here, the skew comprises a relative measure of an amount of residue material in each of two sub-regions 25a, 25b of the sensing region of the camera 30, and specifically sub regions 25a, 25b of the area within the image 200 classified as residue material. This includes a relative measure of an amount of residue material left or right of a centreline “A” located substantially centrally along a longitudinal axis of the combine 10. In image 200 the residue distribution is shown to be skewed towards the left-hand side (in the orientation shown in FIG. 4).

[0060] The processor 104 may be operable to determine a distance at which the residue material is spread. Illustrated in FIG. 4, a maximum lateral extent of residue material spread by the spreader tool 22 is determined, shown by distance “x”, and specifically in the direction of an adjacent standing crop boundary 40. As discussed herein, preferable for residue material not to be spread into adjacent standing crop, but to be ejected to a maximum lateral distance which coincides with the standing crop boundary 40. Accordingly, processor 104 is configured to determine this distance such that appropriate actions can be taken to control the spreader tool 22 accordingly.

[0061] Using the one or more determined characteristics, such as those discussed above, the processor 104 may compare these with reference data, for instance to determine whether any corrective actions might be required. The reference data can include information indicative of an expected characteristic or characteristics for a given set of one or more operational parameters of the combine (or components thereof), and is preferably a learned model trained on a set of training images of known residue distributions for varying operating conditions, generated and/or retrieved for any given set of operating conditions through a deep learning network, with the one or more operating conditions used as inputs to the network, and the output being the reference data, which may include the expected distribution characteristics as discussed herein.

[0062] In one embodiment, the reference data comprises a reference image or reference representation of an expected skew for the residue distribution for a given set of operating parameters (including one or more of a forward speed of the agricultural machine, an operating speed of the spreader tool, an operating speed of one or more sub-systems of the machine, including one or more of a cleaning system or grain sorting system of the machine, an operational speed of an implement attached or otherwise coupled to the machine, including a header, and a conveyor speed of one or more grain transport systems on the machine, etc., or one or more external conditions, including a weather condition, temperature, wind speed and direction, and the like, for example). This may preferably be a “zero” skew image, with an equal or substantially equal split of residue between left and right sides of the combine 10. In another embodiment, the reference data comprises a reference image or reference representation of an expected maximum lateral extent for the residue distribution for a given set of operating parameters.

[0063] Based on the above analysis, the processor 104 is configured to control operation of one or more systems of the combine 10 to result in an adjustment of the residue distribution (if necessary). As discussed above, this includes control over the user interface 32 to provide information to an operator of the agricultural machine corresponding to the determined characteristic(s). This can visually illustrating the observed distribution, for example providing a representation similar to image 200 of FIG. 4. This could include providing an audible or visual indicator to the operator of the observed residue distribution or characteristic(s). For example, the user interface 32 may be used to display or otherwise indicate an error state when the observed residue distribution differs from the reference data and hence requires the attention of the operator, e.g. to prompt the operator to take one or more corrective actions (change in operational parameters of one or more systems of the combine 10) to adjust the distribution.

[0064] The processor 104 is additionally operable to control one or more operating parameters of the combine 10 in dependence on the determined characteristic(s), and specifically operating parameters of the spreader tool 22, to automate correction/adjustment of the parameters to achieve the desired/expected characteristic(s) of the residue distribution. As an example, the processor 104 may be configured to determine if the residue material distribution is skewed left or right, or is substantially uniform, e.g. in the manner discussed above. It may be that the residue material is determined to be skewed to the left (as shown in FIG. 4) and appropriate action may be taken based thereon. For example, control signals 109 may be output via electronic output 108 to spreader tool 22, and specifically to first and second rotors 23a, 23b of the spreader tool to adjust operation thereof, e.g. to reduce the speed of the left rotor 23a and/or increase the speed of the right rotor 23b, or in some instances adjust the orientation of one or more steering vanes (not shown) to reduce the skewedness of the residue distribution profile—essentially by increasing the volume of material ejected generally to the left (into the region 25a of residue shown in FIG. 4) and/or reduce the volume of material ejected generally to the right—region 25b.

[0065] Any process descriptions or blocks in flow diagrams should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the embodiments in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present disclosure.

[0066] It will be appreciated that embodiments of the present invention can be realised in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape. It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs that, when executed, implement embodiments of the present invention. Accordingly, embodiments provide a program comprising code for implementing a system or method as set out herein and a machine readable storage storing such a program. Still further, embodiments of the present invention may be conveyed electronically via any medium such as a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same.

[0067] It will be appreciated that the above embodiments are discussed by way of example only. Various changes and modifications can be made without departing from the scope of the present application.