RESIDUE SPREAD MONITORING

Abstract

Systems and methods for monitoring the distribution of residue material from a spreader tool of an agricultural machine including receiving image data from an imaging sensor indicative of a residue material spread by the spreader tool within a sensing region rearwards of the agricultural machine and where one or more image transformations are applied to image data to generate an enhanced image of the distribution of the residue material including colour, distortion and/or correction transformations for generating an enhanced image for view by an operator of the machine and controlling a user interface associated with the agricultural machine to provide an indicator indicative of the enhanced image which is easily interpreted.

Claims

1. A system for monitoring the distribution of residue material from a spreader tool of an agricultural machine, the system comprising: an imaging sensor having a sensing region rearwards of the agricultural machine; and at least one controller, configured to: receive image data from the sensor indicative of residue material spread by the spreader tool within the sensing region; apply one or more image transformations to the image data to generate an enhanced image of the residue material distribution; and output one or more control signals for a user interface associated with the agricultural machine comprising an indicator indicative of the enhanced image.

2. The system of claim 1, wherein the imaging sensor is configured to apply a Bayer filter to the image data.

3. The system of claim 1, wherein the at least one controller is configured to apply a color transformation to the received image data.

4. The system of claim 3, wherein the at least one controller is configured to convert the received image data to RGB data using a Malvar Cutler method.

5. The system of claim 1, wherein the at least one controller is configured to apply a corrective transformation to the image data; and optionally the corrective transformation applied is a Vignette Correction transformation.

6. The system of claim 1, wherein the at least one controller is configured to apply a distortion correction transformation to the image data; and optionally the distortion correction transformation applied to the image data comprises: a delta angle correction; a barrel distortion correction; or a lens distortion correction.

7. The system of claim 4, wherein the at least one controller is operable to convert the RGB data to a different colour space; and optionally the different color space is a LAB colour space; or a CIELAB colour space.

8. The system of claim 1, wherein the at least one controller is configured to remap a tonality of the image data; and optionally wherein the at least one controller is configured to apply an S-curve to the image data.

9. The system of claim 8, wherein the at least one controller is configured to apply a curve to an “L-channel” or “luminance-channel” of the image data after conversion to a LAB or CIELAB colour space.

10. The system of claim 1, wherein the at least one controller is configured to apply a haze removal transformation to the image data.

11. The system of claim 1, wherein the at least one controller is operable to employ a data buffer of a predetermined time period to the data received from the sensor.

12. The system of claim 1, wherein the at least one controller is configured to extract a value for each corresponding pixel in each stored image forming a data buffer, wherein the value comprises: a statistical value for each pixel; or a quartile value for each pixel based on the stored data in the data buffer.

13. The system of claim 1, wherein the at least one controller is configured to apply a colour transformation to processed image data and generate an enhanced image for display; and optionally wherein the colour transformation includes converting the processed image data to a jet or turbo colour space.

14. The system of claim 1, wherein the at least one controller is configured to apply a frequency filter to the image data, the frequency filter being dependent on a forward speed of the agricultural machine.

15. The system of claim 1, wherein the user interface comprises a display wherein the display is part of the agricultural machine, or an interface on a portable device which is remotely operable from the machine.

16. The system of claim 1, configured such that an indicator on the enhanced image comprising a line or overlaid on the image at a position corresponding to the maximum lateral extent at which the material is ejected from the spreader tool.

17. An agricultural machine comprising the system of claim 1.

18. A method for monitoring the distribution of residue material from a spreader tool of an agricultural machine, the method comprising: receiving image data from an imaging sensor indicative of residue material spread by the spreader tool within a sensing region rearwards of the agricultural machine; applying one or more image transformations to the image data to generate an enhanced image of the residue material distribution; and controlling a user interface associated with the agricultural machine for providing an indicator indicative of the enhanced image.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0049] One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0050] FIG. 1 is a schematic side cross-sectional view of an agricultural harvester embodying aspects of the invention;

[0051] FIG. 2 is a schematic view of an embodiment of a control system of the invention; and

[0052] FIGS. 3-6 are representative images illustrating aspects of the invention.

DETAILED DESCRIPTION

[0053] FIG. 1 illustrates an agricultural machine, and specifically a combine 10, embodying aspects of the present invention.

[0054] The combine 10 is coupled to a header 12 which is operable, in use, to cut and gather a strip of crop material as the combine 10 is driven across a field/area to be harvested during a harvesting operation. A conveyor section 14 conveys the cut crop material from the header 12 into a crop processing apparatus 16 operable to separate grain and non-grain (i.e. material other than grain (MOG) or residue material (used interchangeably herein)) as will be appreciated. It is noted here that apparatus for separating grain and non-grain material are well-known in the art and the present invention is not limited in this sense. The skilled person will appreciate that numerous different configurations for the crop processing apparatus may be used as appropriate. Clean grain separated from the cut crop material is collected in a grain bin 18, which may be periodically emptied, e.g. into a collection vehicle, storage container, etc. utilising unloading auger 20. The remaining non-grain material (MOG)/residue material is separately moved to a spreader tool 22 which is operable in use to eject the non-grain material or MOG from the rear of the combine 10 and onto the ground. In FIG. 1, this is represented by arrow 24 which illustrates the MOG being ejected rearwards from the combine 10. It will be appreciated that in some embodiments the combine 10 may also include a chopper tool positioned, for example, between the crop processing apparatus 16 and the spreader tool 22 and operable, in use, to cut the residue material before it is spread by the spreader tool 22.

[0055] The combine 10 also typically includes, amongst other features, an operator cab 26, wheels 28, engine (not shown) and a user interface 32.

[0056] As will be discussed in detail herein, the combine 10 additionally includes a sensor in the form of a camera 30. The camera 30 is used, by a control system 100 of the combine, to obtain image data of a sensing or measurement region rear of the combine 10, indicative of a distribution of residue material associated with the spreader tool 22. In the illustrated embodiment, the camera 30 is shown mounted to a rear surface of the combine 10, and is angled downwards providing a field of view which encompasses a ground surface rear of the combine 10.

[0057] FIG. 2 illustrates control system 100 further. As shown, control system 100 comprises a controller 102 having an electronic processor 104, an electronic input 106 and electronic outputs 108, 110. The processor 104 is operable to access a memory 112 of the controller 102 and execute instructions stored therein to perform the steps and functionality of the present invention discussed herein, e.g. by controlling the user interface 32, for example to provide an image to an operator of the combine 10 illustrative of the observed residue material distribution.

[0058] The processor 104 is operable to receive image data via input 106 which, in the illustrated embodiment, takes the form of input signals 105 received from the camera 30. As described in detail herein, the camera 30 has a sensing region rearward of the combine 10, with the image data received from the camera 30 being illustrative of residue material within the sensing region. Using the received image data, the processor 104 is operable to perform one or more image processing transformations on the image data to generate an enhanced image for display by the user interface 32 in the manner described herein. Advantageously, the process steps discussed herein may result in the generation of an enhanced image which more clearly illustrates the residue spread associated with the spreader tool 22 for understanding by an operator of the combine 10.

[0059] In the illustrated embodiment of FIG. 2, output 110 is operably coupled to the user interface 32 of the combine 10. Here, the control system 100 is operable to control operation of the user interface 32, e.g. through output of control signals 111 in order to display the enhanced image generated as described herein. In addition, the control system 100 may be operable to control the user interface 32 to display to the operator a graphical representation of the residue material distribution from the spreader tool 22 as determined by processor 104, image data obtained from the camera 10 (e.g. a direct image feed from the camera 10) on the combine 10, or other useful information. The user interface 32 may also be operable to receive a user input from the operator, and in such instances the output 110 may act as an input for receiving that user input at the processor 104. The user input may relate to a requested or desired distribution of residue material, for example, made by the operator of the combine 10, or may relate to a preferred selection of image type illustrating the residue spread. For instance, certain operators of the combine may prefer different level of processing on the raw image data from the camera 10 to enable them to visualise the observed spread pattern.

[0060] It will be appreciated that in alternative arrangements a user interface may instead be provided remote from the combine 10, e.g. as part of a smartphone, tablet computer or computer, for example, for a remote operator to visualise the spread pattern.

[0061] In a variant, as illustrated by FIGS. 1 and 2, the processor 104 is further operable to generate and output control signals 109 via the output 108 for controlling operation of the spreader tool 22, and more specifically first and second steering units of the spreader tool 22, here in the form of a first rotor 23a and a second rotor 23b, for controlling the distribution of residue material ejected from the spreader tool 22. This may be performed upon receipt of instructions by an operator of the combine 10, e.g. upon reviewing the enhanced image presented on user interface 32.

[0062] FIGS. 3 to 6 illustrate the operational use of aspects of the invention.

[0063] As discussed herein, aspects of the invention relate to the performance of one or more image transformations to the image data received from the camera 30 in order to generate an enhanced image for an operator of the combine 10. FIG. 3 illustrates an image 200 formed from raw image data from the camera 30, following a number of pre-processing transformations (discussed below). FIGS. 4 to 6 are representative images obtained following one or more further transformations applied to the image data forming image 200 of FIG. 3.

[0064] The following description covers an embodiment of a series of processing steps performed on the image data received from camera 30 to obtain an enhanced image 500 (FIG. 6) for display to an operator of the combine 10. However, it will be appreciated that each of the processes discussed hereinbelow may be performed as part of the combined process, or be performed independently where appropriate/possible, with an enhanced image output possible at each stage identified below.

[0065] As discussed above, FIG. 3 illustrates an image associated with the raw image data received from the camera 30, but following a number of pre-processing transformations applied thereto. These include, a Bayer filter is applied to the image data from the camera 10, and following this a first colour transformation is applied. Here, this comprises a conversion of the received image data to RGB data using a Malvar Cutler method. One or more corrective transformations are then applied, here including a vignette correction transformation and a distortion correction transformation, including applying a delta angle correction and/or a barrel correction and/or other lens distortion correction to the image data.

[0066] As a next step, the corrected RGB data is converted to a different colour space. Specifically, in the illustrated embodiment, this includes converting the RGB data to a LAB colour space before remapping a tonality of the converted LAB image data. Here, this comprises applying an S-curve to the converted image data, and specifically to the “L-channel” or “luminance-channel” of the LAB image data. Advantageously, this results in the residue material being more clearly defined with respect to the background in the image data by effectively enhancing any “luminous” pixels corresponding to the pixel values associated with residue material whilst subduing pixels corresponding to background. At this stage, the remapped image data may be used to generate an enhanced image for display. An example enhanced image 300 is shown in FIG. 4. As shown, image 300 in FIG. 4 more clearly illustrates the residue material spread when compared with image 200 of FIG. 3, where the residue is relatively difficult to distinguish from the background.

[0067] In a variant, a haze removal transformation may be applied to the image data. Advantageously, this may filter out dust and other small particles present within the image data which may otherwise obscure the residue material pieces, and suppress stray background light in the image.

[0068] In a next step, a data buffer is employed storing data from a number of sequential images obtained by the camera 10. This is typically of the order of 10-100 separate images. For equivalent pixels of each image, a value is extracted which corresponds to a statistical “value” for that pixel across the multiple images. The statistical values are used to generate an enhanced image comprising the processed pixels—see FIG. 5. Whilst having multiple data points for each residue piece reduces resolution in terms of identifying individual pieces, this instead provides a clearer representation of the overall shape and uniformity of the spread pattern provided by the spreader tool. In this embodiment, the pixel values correspond to the L-channel values of the corrected LAB image data determined above—i.e. the image data used to generate image 300 of FIG. 4. However, it will be appreciated that the data buffer may be utilised for the raw image data from camera 30, or indeed following one or more of the processing steps described herein. Again, image 400 of FIG. 5 may be output to user interface 32 for display, and more clearly illustrates the shape (in particular) of the residue material spread when compared with image 200 of FIG. 3, and indeed image 300 of FIG. 4.

[0069] In a final step of the process, a further colour transformation in applied to the processed image data. The further colour transformation here includes converting the processing image data to a false/pseudo colour space, and in particular to a jet colour space. An example image 500 obtained following the further colour transformation is shown in FIG. 6. Here, the residue material is more clearly defined with respect to the background in the displayed image, and the operator of the combine 10 can immediately assess the shape and extent of the material spread when compared with the background given the stark colour contrast in the image 500.

[0070] In an extension of the method, a frequency filter may be applied to the image data. This may be applied at any point in the above described method. The frequency filter may advantageously be dependent on a forward speed of the agricultural machine to filter from the image data non-residue “stationary” objects within the vehicle's environment.

[0071] Any process descriptions or blocks in flow diagrams should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the embodiments in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present disclosure.

[0072] It will be appreciated that embodiments of the present invention can be realised in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape. It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs that, when executed, implement embodiments of the present invention. Accordingly, embodiments provide a program comprising code for implementing a system or method as set out herein and a machine readable storage storing such a program. Still further, embodiments of the present invention may be conveyed electronically via any medium such as a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same.

[0073] It will be appreciated that the above embodiments are discussed by way of example only. Various changes and modifications can be made without departing from the scope of the present application.