METHOD AND MEASUREMENT SYSTEM FOR DETERMINING CHARACTERISTICS OF PARTICLES OF A BULK MATERIAL

Abstract

The present disclosure refers to a method for determining characteristics of particles of a bulk material such as fertilizer, seed or the like, comprising: providing a heap of particles of a bulk material to be distributed by a distribution machine; providing a measurement tool having an optical landmark on a front side in a measurement position in which the heaped particles of the bulk material are provided in proximity to the measurement tool; providing a camera device configured to detect images; detecting image data by the camera device, the image data indicative of an image of the front side of the measurement tool and the heaped particles provided in proximity to the measurement tool; and determining characteristics of the particles from image data analysis of the image data. Further, a measurement system for determining characteristics of particles of a bulk material is provided.

Claims

1. A method for determining characteristics of particles of a bulk material such as fertilizer, seed or the like, comprising: providing a heap of particles of a bulk material to be distributed such as fertilizer, seed or the like by a distribution machine; providing a measurement tool having an optical landmark on a front side of the measurement tool; providing the measurement tool in a measurement position in which the heaped particles of the bulk material are provided in proximity to the measurement tool; providing a camera device configured to detect images; detecting image data by the camera device, the image data indicative of an image of the front side of the measurement tool provided in the measurement position and the heaped particles of the bulk material provided in proximity to the measurement tool; and determining characteristics of the particles of the bulk material from image data analysis of the image data, comprising, in one or more processors: determining the optical landmark from the image data; in response to determining the optical landmark, providing dimensional data assigned to and indicative of dimensional characteristics of the optical landmark; and determining characteristics of the particles of the bulk material taking into account the dimensional data.

2. The method of claim 1, wherein the determining comprises determining dimensional characteristics of the particles of the bulk material.

3. The method of claim 1, wherein the determining comprises applying a neural network in the image data analysis for processing the image data.

4. The method of claim 3, wherein the applying comprises applying a classification regressor neural network in the image data analysis for processing the image data.

5. The method of claim 1, further comprising providing the heap of particles of the bulk material to be distributed in a container of a distribution machine.

6. The method of claim 1, further comprising providing a measurement tool having an arrangement of optical landmarks on an edge part of the measurement tool.

7. The method of claim 1, further comprising: providing a measurement tool having an opening, wherein the arrangement of optical landmark is provided on the edge part encompassing the opening at least in part; and the bulk material provided on the back side of the measurement tool occupying, in the image of the front side of the measurement tool, an image subarea assigned to the opening.

8. The method of claim 6, wherein the providing of the dimensional data comprises providing dimensional data indicative of landmark distance characteristics of the optical landmarks from the arrangement of optical landmarks.

9. The method of claim 7, wherein the providing of the dimensional data comprises providing dimensional data indicative of at least one of a size of the opening and a diameter of the opening.

10. The method of claim 1, wherein the providing the camera device comprises providing a mobile device having the one or more processors and the camera device.

11. The method of claim 1, further comprising: providing a time of flight sensor device; detecting distance data while the image data are detected, the distance data being indicative of a distance between the camera device and the measurement tool; in the one or more processors, determining the distance between the camera device and the measurement tool from the distance data; and determining the characteristics of the particles of the bulk material taking into account the distance between the camera device and the measurement tool from the distance data.

12. A measurement system for determining characteristics of particles of a bulk material such as fertilizer, seed or the like, comprising: a measurement tool; a camera device; and one or more processors; the measurement system being configured to: provide the measurement tool in a measurement position in which a heap of particles of a bulk material is provided in proximity to the measurement tool; detect image data by the camera device, the image data being indicative of an image of the measurement tool in the measurement position and the heaped particles of bulk material provided in proximity to the measurement tool; and in the one or more processors, determine characteristics of the particles of the bulk material from image data analysis of the image data, wherein the one or more processors are configured to: determine the optical landmark from the image data; in response to determining the optical landmark, provide dimensional data assigned to and indicative of dimensional characteristics of the optical landmark; and determine characteristics of the particles of the bulk material taking into account the dimensional data.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Following, further embodiments are described with reference to figures. In the figures, show:

[0037] FIG. 1 a schematic representation of an arrangement with a measurement system for determining characteristics of particles of a bulk material such as fertilizer, seed or the like;

[0038] FIG. 2 a schematic representation of a measurement tool provided with an edge part and encompassing an opening and optical landmarks placed on the edge part; and

[0039] FIG. 3 a schematic representation of another measurement tool provided with a single optical landmark.

DETAILED DESCRIPTION OF FURTHER EMBODIMENTS

[0040] FIG. 1 shows a schematic representation of an arrangement provided with a measurement tool 1, a bulk material 2 such as fertilizer, seed or the like, a camera device 3, and a data processing system 4 having one or more processors for data processing. Particles 2a of the bulk material 2 are provided in a heap (see inset in FIG. 1).

[0041] FIG. 2 shows a schematic representation of a front side of the measurement tool 1 provided with an opening 5 and an edge part 6 encompassing the opening 5. Optical landmarks 7 are provided in corner sections 8 of the edge part 6. In the embodiment shown, the optical landmarks 7 are provided as so-called ArUco-landmarks. ArUco-landmarks provide for maturity of the libraries and low cost of processing. Processing of such landmarks can be conducted very fast and reliable. They also provide perspective data because of their simple to detect square shape. Other optical landmarks known as such may be provided instead.

[0042] The camera device 3 and the data processing system 4 may be provided in a common device housing or in separate housings. For example, the camera device 3 and the data processing system 4 may be implemented by a mobile phone or some other mobile device such as laptop or tablet computer. Alternatively, image data detected by the camera device may be transmitted to the data processing system located remotely from the location of the camera device 3.

[0043] For determining the characteristics of the particles 2a of the bulk material 2, image data are detected by the camera device 3 when the measurement tool 1 is provided in a measurement position. The measurement position is characterized by having the measurement tool 1 placed in front of the bulk material 2 and the camera device 3 facing the front side of the measurement tool 1, the bulk material 2 on the backside of the measurement tool 1 occupying the opening 3 in the scene presented to the camera device 2.

[0044] Image data indicative of one or more images detected by the camera device 3 are processed by an image data analysis conducted in the data processing system 4. In the process of image data analysis, the optical landmarks 7 are determined. In response, characteristics of the bulk material 2 are determined taking into account the information about the optical landmarks.

[0045] With respect to characteristics of the particles 2a of the bulk material 2, one or more of the following characteristics may be determined: size of the particles 2a of the bulk material 2, and diameter of the particles 2a.

[0046] FIG. 3 shows a schematic representation of alternative examples for the measurement tool 1. The exemplary measurement tools 1 depicted are provided with one or two optical landmarks 7. In the images taken by the camera device only one or both measurement tools 1 may be present. The method for determining the characteristics of the bulk material 2 described above applies mutatis mutandis.

[0047] The optical landmark(s) 7 itself can contain enough information (dimensional characteristics). For a square landmark like the ArUco landmarks it is easy to get the position of the four corners on the image. The relative location of the corners the optical landmark 7 from each other may be known. The four corners of a single optical landmark 7 contain enough information for determining a homography transformation matrix. Homography refers to the relation between two images. One image is the image taken with the camera device 3 for the optical landmarks 7 and surroundings, and the other image is the known image of the optical landmark. The homographic matrix may be a 3×3 matrix containing factors required to map any pixel from the image made by the user to the known plane on which the optical landmarks 7 exist. Applying the homography to transform the image onto the flat image plane is called image rectification. It corrects the perspective and at the same time corrects for most problems arising from the use of different cameras under different conditions.

[0048] Perspective information is important. For example, mobile phones and camera devices can have different field of views, resolutions and quality. Also the user cannot be instructed to hold the camera device 3 perfectly parallel above the bulk material 2. Particles 2a of the bulk material close to the camera device 3 would appear bigger then particles further away. For determining the actual size and/or diameter of the particles 2a it is important to get such perspective transformation reversed.

[0049] Using the measurement tool 1 provided with a plurality of optical landmarks 7 (see FIG. 2) may provide for a more robust process for determining the bulk material 2. Every optical landmark 7 gives four known reference points, so optical landmarks 7 results in 16 reference points. The more reference points are available, the more reliable is it to find the homography. A RANSAC method (RANSAC—RANdom SAmple Consensus) may be applied to get an even more accurate homography matrix.

[0050] Coloured optical landmarks may be applied. This may allow for correction of colour distortion in the images taken by the camera device 3.

[0051] In an example, the following steps may be applied: [0052] 1. Taking images of the bulk material 2 with reference the measurement tool 1 on top with the camera device 3 (e.g. a mobile device or tablet); [0053] 2. (Either step 2. or 4.) Applying correction for perspective and scale based on the optical landmarks 7 by the one or more processors provided e.g. in the mobile device or the tablet; [0054] 3. Send the fixed image to a server or local device (spreader, terminal or other controller) for analyzing the data; [0055] 4. (Either step 2. or 4.) Applying correction for perspective and scale based on the measurement tool 1 on a server device or the local device (e.g. the mobile device or the tablet); [0056] 5. Analyzing the images to determine bulk material characteristics like surface roughness, shape and particle diameter by either using conventional computer vision techniques or by using deep neural networks; [0057] 6. Return data indicative of the bulk material characteristics back to the mobile device or local device; and [0058] 7. Returning spreading chart advice in a software application based on the bulk material characteristics or setup the spreader machine automatically (e.g. without user input).

[0059] In the data processing system 4, a deep neural network (DNN) may be applied to process the digital image data. One or more of the characteristics of the bulk material determined may be assigned to useable physical properties (material parameters). One of the material parameters, namely the particle size or particle diameter, may be determined. For example, this can either be conducted by classifying the particles 2a of the bulk material 2 in certain categories, for example: fine (1 mm), small (1.8 mm), normal (2.5 mm), large (3.5 mm), very large (5 mm+). Alternatively, the particle size can be determined by regressing the digital image data into just one average diameter (x mm).

[0060] The DNN may also be used to recognize at least one of the shape and the category of the bulk material such as fertilizer or seed. This is done with a classification regressor neural net-work. For each category of bulk material the DNN will return a number between 0 to 1 indicating how likely it is that the photographed bulk material is having a certain shape.

[0061] The DNN can be located on either a local information controller on an distribution machine such as a spreader, a local software application for a mobile phone or hosted in the cloud. The cloud may provide for the opportunity to collect digital image data from customers or users, these can be then be used to retrain the DNN so it keeps evolving.

[0062] The DNN may be hosted on the control device 4 on the agricultural spreader 1. If it is combined with a local advice service and database, the agricultural spreader 1 can setup itself completely automatic and offline. Continuously updating the settings depending on the current changed would become a possibility.

[0063] In the method disclosed, an image of a bulk material 2 is transformed onto a two-dimensional image plane using references (optical landmarks 7) such as ArUco landmarks with the purpose of determining properties of the particles 2a of the bulk material 2, such as the diameter of the particles 2a. Such information can be applied for assisting correctly setting up a spreader machine. The transformation is to provide an image with a fixed scale on a flat two-dimensional plane where a computer vision algorithm or neural network can be applied for further processing.

[0064] The features disclosed in this specification, the figures and/or the claims may be material for the realization of various embodiments, taken in isolation or in various combinations thereof.