OPTIMIZATION METHOD FOR IMPROVING THE RELIABILITY OF GOODS COMMISSIONING USING A ROBOT

Abstract

The invention relates to an optimization method for improving the reliability of a collection and discharge of articles in a picking process using a robot. An article is collected from or out of a first article carrier and is placed down or thrown into or onto a second article carrier by means of a gripping unit on the robot head. In an image processing step, a gripping position for the gripping unit is calculated for collecting the article(s) by determining at least one dimension based on a captured image and by a range allocation being determined by means of a comparison with dimension ranges. Depending on a confidence value, a dimension value is determined, from stored article reference data or from a normalization value of the dimension range and allocated to the determined dimension. In a preparation step, a mathematical scattering measure function is applied for the determined dimension and for the dimension ranges, and normalization values and confidence values of the dimension ranges are determined therefrom.

Claims

1. An optimization method for improving the reliability of a collection and discharge of articles according to an order in a picking process with a robot, which robot comprises a robot head which has a gripping unit and is movable relative to a robot base, in which an article is collected from or out of a first goods carrier and is placed down or thrown in/into or on/onto a second goods carrier by means of the gripping unit, and wherein, in an image processing step, a gripping surface size and a gripping surface position are determined by means of an opto-sensory preparation and analysis system, and a gripping position for the gripping unit is calculated therefrom, the gripping unit is moved to the calculated gripping position and the article is collected by the gripping unit, wherein in the image processing step, at least one dimension is determined when the gripping surface size is determined, and a range allocation is determined by comparing this dimension with dimension ranges, wherein the dimension ranges are stored in an article parameter field, and for this range allocation, a dimension value is determined and allocated to the determined dimension, wherein depending on a confidence value of the dimension range, the dimension value is selected from stored article reference data or from a normalization value of the dimension range, wherein the normalization value and the confidence value of the dimension range are stored in the article parameter field, and wherein in a preparation step, a mathematical scattering measure function is applied for the determined dimension and for the dimension ranges, and normalization values and confidence values of the dimension ranges are determined therefrom.

2. The optimization method according to claim 1, wherein, in a counting step following the collection of the article by the gripping unit, the number of collected articles is determined by determining an article weight by means of a weighing device, and comparing the article weight having weight ranges, which are stored in the parameter field, and determining a range allocation, and determining a number value for this range allocation, wherein depending on a confidence value of the weight range, it is determined from stored article reference data or determined from a normalization value of the weight range, wherein the normalization value and the confidence value of the weight range are stored in the article parameter field, and wherein in the preparation step, a mathematical scattering measure function is applied for the article weight and for the weight ranges, and normalization values and confidence values of the weight ranges are determined therefrom.

3. The optimization method according to claim 2, wherein alternatively to comparing the determined article weight to weight ranges, a quotient is formed from the article weight determined by the weighing device and a normalization value of the weight, and that the quotient is scaled and compared with weight ranges, and a range allocation is determined.

4. The optimization method according to claim 2, wherein the weighing device is arranged separately from the robot, in particular from the gripping unit, and acquires the weight of the first article carrier, and the counting step is carried out before discharge of the articles, in particular after collection of the articles from the first article carrier and before the gripping unit reaches the second article carrier.

5. The optimization method according to claim 2, wherein, in case of a determined number value which is greater than one, an examination is carried out as to whether the determined number value exceeds the target amount of articles to be picked, wherein the target amount is stored in the order, and in case of exceedance, a special handling step is performed.

6. The optimization method according to claim 2, wherein, in case of exceeding a limit value of the number value, or in case of exceeding a range limit value during the determination of the range allocation, a special handling step is performed.

7. The optimization method according to claim 2, wherein the normalization value of the weight is formed by means of a mathematical weighting function of all normalization values of the weight ranges.

8. The optimization method according to claim 3, wherein a range classification is performed for the decimal value of the quotient, and for a first range, the number value is determined by the integer part of the quotient, and for a second range, a special handling is performed.

9. The optimization method according to claim 8, wherein for a third range, the confidence values of the two adjoining weight ranges are analyzed, and the article weight is determined from the normalization value of the weight range with the higher confidence value.

10. The optimization method according to claim 1, wherein during the determination of the range allocation, a check is carried out for the determined dimension value, as to whether the determined dimension is in an overlap region of two dimension ranges, and in the case of a positive check, a special handling step is performed.

11. The optimization method according to claim 10, wherein in case of a determined position of the dimension in the overlap region, the image processing step is carried out again in the special handling step.

12. The optimization method according to claim 10, wherein in the special handling step, the robot is controlled to move the robot head to the first article carrier and to collect an article and place it down again and that subsequently, image processing is carried out again.

13. The optimization method according to claim 5, wherein in the special handling step, the robot is controlled to move the robot head to the first article carrier and to place or throw the collected article on/onto or in/into the first article carrier.

14. The optimization method according to claim 1, wherein, in the preparation step, for the allocated range, the normalization value is defined as the mean value of a normal distribution.

15. The optimization method according to claim 1, wherein the scattering measure function comprises an expectation-maximization algorithm and iteratively groups determined values, dimensions and/or weight values into local clusters.

16. The optimization method according to claim 15, wherein a probability distribution, in particular a normal distribution, is applied to the grouped values, and wherein for local clusters, a mean value and a mean value scatter are determined and the mean value is defined as the normalization value.

17. The optimization method according to claim 1, wherein the scattering measure function comprises a sum of weighted normal distributions (mixture of Gaussians) and iteratively groups values, dimensions and/or weight values into local clusters.

18. The optimization method according to claim 1, wherein, in an adaptation step, the dimension ranges are adapted to the normalization values determined in the preparation step, or wherein, in the adaptation step, the weight ranges are adapted to the normalization values determined in the preparation step.

19. The optimization method according to claim 1, wherein, for each dimension range and/or each weight range, the confidence value is determined from a scattering of the normalization value, and/or wherein the confidence value is determined from the number of preparation steps performed.

20. The optimization method according to claim 1, wherein, when forming the normalization and confidence value, a window function is applied, in each case, to the determined dimensions and article weights.

21. The optimization method according to claim 1, wherein the article reference data is transmitted, by means of a superordinate warehouse management system, to the preparation and analysis system, where it is stored in a storage means.

22. The optimization method according to claim 1, wherein, when article reference data is missing, a series of article transfers between the first and the second article carrier is carried out, and the article parameter field with normalization values and confidence values for dimensions and/or article weights is constructed from the respectively determined dimensions and/or article weights.

23. The optimization method according to claim 22, wherein a fluctuation range is determined from a series of normalization or confidence values of the dimensions and/or of the article weights, and the article transfers are stopped when falling below a threshold of the fluctuation range.

24. The optimization method according to claim 1, wherein, in case of a high confidence value for the dimension normalization value or for a weight normalization value, an automated or partly automated update of the article reference data is carried out.

25. A workstation, in particular picking station, comprising a provisioning device for one or multiple article carrier(s), a robot with a robot head that is movable with respect to a robot base, which robot head has a gripping unit for transferring articles between article carriers, an image capturing device and a data processing unit configured for controlling the robot (and possibly the conveying system) and also for evaluating data from the image capturing device wherein the data processing unit is configured for performing the method according to claim 1.

Description

[0052] For the purpose of better understanding of the invention, it will be elucidated in more detail by means of the figures below.

[0053] FIG. 1 shows an embodiment of the system and of the present method for being able to detect and handle variations of the dimensions of the outer packaging;

[0054] FIG. 2 shows an embodiment of the system and of the present method for being able to detect and handle variations of the dimensions of the outer packaging;

[0055] FIG. 3 shows a representation of value ranges as they are formed by the present method and used for determining distance values and/or number values.

[0056] FIG. 1 shows an embodiment of the present method for improving the reliability of a collection and discharge of articles, in order to be able to detect fluctuations in the dimensions of the outer packaging and to consider them when determining the gripping position.

[0057] At an automated picking location 1, articles are provided, by means of a conveying system 2, in or on article carriers 3—after picking, the article carrier(s) 3 are transported away again by the conveying system 2 and preferably, the next article carrier(s) 3 is/are provided.

[0058] The picking location 1 constitutes a possible embodiment of a workstation, which may comprise a provisioning device for one or multiple article carrier(s) 3, a robot 4 with a robot head that is movable with respect to a robot base, which robot head has a gripping unit 5 for transferring articles between article carriers 3, an image capturing device and a data processing unit configured for controlling the robot (and possibly the conveying system) and also for evaluating data from the image capturing device. According to this embodiment, the provisioning device is formed on the conveying system 2 for supplying article carriers 3 and/or transporting them away.

[0059] The provision and the transport away are not essential to the present method and are hence not further elaborated on. The article carriers 3, without limiting the specific embodiment, can be formed by containers or trays, for example. The conveying system 2, also without limiting the specific embodiment, can be formed by roller or belt conveyors, for example.

[0060] A robot 4 has a gripping unit 5 on a robot head 6, which robot head 6 is movable with respect to a robot base 7 and can be moved by a control unit 8 into any desired gripping position within the movement leeway of the robot 4. The movement leeway of the robot comprises in any case the region in which the article carriers 3 (source and target) are arranged. Thus, the gripping unit 5 can reach every area of the article carriers 3 in order to collect or discharge articles there.

[0061] For determining the gripping position, at least the source load carrier 3 is captured in a first step by an image capturing system 9 of an opto-sensory preparation and analysis system 10 of the control unit 8. If he articles to be picked are discharged at the target load carrier 3 at a specific position, a further image capturing system 9 also captures the target article carrier. The image capturing system 9 is preferably configured as a stereo camera system and, apart from an optical image, also captures a point cloud with distance data between the camera system 9 and the surface of the article on the article carrier 3 (and the boundary of the article carrier 3). The captured image 11 as well as the determined point cloud are transmitted to the control unit 8 where they are prepared and evaluated by the opto-sensory preparation and analysis system 10.

[0062] FIG. 1 shows a simplified representation of the captured image 11, in particular without the captured boundary of the article carrier 3 and in a vertical view onto the article carrier 3. However, this is insignificant for the descriptive method steps. Image 11 of the article carrier 3 shows a chaotically arranged article 12. Aside from separately arranged articles, it will always happen, in particular with article carriers having a high filling level, that articles are also arranged on top of one another. For the sake of simplification, the perspective representation of articles arranged on top of one another is incomplete.

[0063] In order to collect the article 12 and securely hold it during the transfer into the target article carrier, the gripping unit 5 must collect the article carrier 12, simply put, as much in the center of the gripping surface 13 as possible. The gripping unit 5 may, for example, be equipped with gripper arms or preferably with at least one suction element, wherein it is further of great significance for a reliable collection that the robot 4 positions the gripping unit 5 as normal to the gripping surface 13 as possible. An advantage of a collection of the article 12, that is as central as possible, is that the article is then also collected near its center of gravity, which is advantageous for securely holding it during the transfer movement. If the article 12 is for example formed by flexible polybags, a collection that is as central as possible has the further advantage that the article then hangs approximately equally far down on both sides of the gripper, in the direction of the article carrier. During the transfer, the article must be lifted high enough by the robot 4 that it bumps neither into a possibly present boundary of the article carrier 3 not into elements of the picking location 1 during the pivoting movements from the source article carrier to the target article carrier.

[0064] By means of an edge recognition method of the opto-sensory preparation and analysis system 10, article boundaries are searched for in the captured image 11 of the article carrier 3, which is possible, for example, by means of a contrast detection method. It may, for example, occur that articles 12 are by chance arranged such 14 that a perceived continuous edge is formed 15. The contrast detection method would then, for example, recognized the entire length 15 as a dimension and, based on that, would calculate an incorrect gripping position. In consequence, it would be highly probable that the article cannot be collected or would come loose from the gripping unit during the transfer. FIG. 1 shows a simplified situation, in which only two articles are located closely together and/or on top of one another. Furthermore, the possibility for delimitation, based on the distance data determined by the image capturing system 9, was not taken into consideration for this description. In a real, chaotically filled article carrier, a plurality of articles will be arranged closely together and on top of one another and will also not be reliably distinguishable via distance data (especially in the case of very flat polybags), so that the probability for recognizing a reliable edge length and thus determining an inadmissible gripping position will increase. Ultimately, such erroneous recognitions will cause the picking performance to drop significantly and/or the error rate will increase.

[0065] The safety of recognizing an individual article piece is increased according to the present method by the determined dimensions being compared with the article reference data which had been acquired during the goods-in process. As already described, these dimensions may fluctuate. For detecting a valid gripping position—for being able to reliably grip an individual article—it is thus necessary to check a determined dimension against stored dimension ranges, meaning an allocation of the stored dimension values to the determined dimensions must take place. As long as the individual dimensions of the outer dimension are clearly different, a comparison with the article reference data will deliver quite good results. Temporary changes of the outer packaging by the producer, using the same article number, cannot be captured with a method based purely on article reference data and will lead to picking errors.

[0066] With the present method, the dimensions determined in the image 11 are compared with an adaptively adapted value distribution 16 to be able to thus compensate fluctuations of the dimensions. Measurement values subjected to random fluctuations will in most cases show a normal distribution of the values. A generally cuboid article piece has three dimension coordinates, wherein the predominant number of the captured dimension values will each be distributed around one of the three dimension normalization values 17. Details on the characteristics of normal distributions are not further elaborated on here, as they are known to a person skilled in the art. As a normal distribution describes a possible distribution function, the general terms normalization value and distribution curve are used in the following, wherein in the special case of a normal distribution, the normalization value corresponds to the mean value.

[0067] The edge recognition method of the opto-sensory preparation and analysis system 10 recognized, for example the silhouette of an article and determined two dimension values X1, X2 based thereon. In order to determine which of the three possible dimensions of the outer packaging the determined dimension value X1 18 is, the determined value X1 is projected onto the value distribution 16. It is evident that the determined value X1 is near the normalization value μ3 17 and inside the distribution curve 19 belonging to said normalization value 17. According to the method according to the invention, there is a confidence value C for each distribution curve 19, which confidence value C represents a statistic for how well the normalization value 17 is supported by the distribution curve 19. A more detailed description follows below. As the determined dimension value X1 is close to the normalization value μ3 and the confidence value C3 for this distribution curve N3 is high, the dimension value is equated with the normalization value μ3.

[0068] According to the present method, as soon as the confidence value C is above a determinable threshold value, the determined normalization value is selected as a dimension value while below said threshold, the dimension value is selected from the article reference data. As the present method adapts in each case the normalization value and the confidence value in each determination of dimension values, the certainty of having determined the correct dimension values will increase with an increased performance frequency, assuming minimally scattering measurement values. As the article reference data are not updated, a deviation will occur between the article reference data and the normalization values if the outer packaging is changed as described above by way of example. Due to a possibly high confidence value, however, the values determined by the present method are used in this case and as a consequence, the gripping surface size and the gripping position are correctly determined despite differing article packaging.

[0069] Apart from the three dimension ranges μ1-μ3, a fourth range having a distribution curve 28 is additionally represented. Values falling within this range cannot be allocated to a real dimension of an article as they can only originate from capturing errors or an inadmissible article mistakenly located in/on the article carrier. In this case, the image processing step is preferably carried out again in a special handling. If such erroneous detections occur frequently, it could indicate information that articles of this type should possibly be arranged differently in the article carrier in order to avoid capturing errors.

[0070] For reliably processing an order, it is further required, in addition to a reliable collection and discharge of articles, that the correct number of articles is transferred. According to a further embodiment of the present method shown in FIG. 2, the weight of the article collected by the gripping unit 5 is evaluated for determining the number of collected articles.

[0071] The essential difference between the picking location 1 and the picking location described in FIG. 1 is that in the region of the source article carrier, a weighing device 20 is arranged. The weighing device 20 can also be arranged in the region of the target article carrier or also on the robot head 6. The arrangement in the region of the source article carrier has the advantage that a determination of the number of collected articles can be carried out immediately after article collection, in particular before the robot arm 21 moves in the direction of the target article carrier. This enables an early correction of an erroneous collection, for example when more article pieces are collected than are required for fulfilling the order. A collection of multiple articles is admissible, however, as long as this does not cause the total number of articles to be picked to be exceeded.

[0072] Due to a surface that is difficult for image analysis (for example glossy and/or high contrasts), it is possible that the gripping position was not optimally determined and the gripping unit is positioned close to the edge of the article, for example. When collecting articles, it can thus happen that multiple articles are collected. The weight change of the source article carrier is detected by the weighing device 20 and evaluated with the aid of the present method, in order to determine the reliable number of collected articles.

[0073] Just like the outer dimensions, the article weight may also be subject to slight fluctuations, or, as already described regarding the outer packaging, be temporarily changed by the producer. The fundamental problem is thus equivalent to the situation when determining the outer dimension for determining the gripping surface size and/or gripping surface position, so that this description is not repeated here.

[0074] Analogously to the dimensions, the weight change W1 22 detected by the weighing device 20 is projected onto the value distribution 23. It is evident that the determined value W1 is near the normalization value μ1 24, and the weight of the collected article piece is slightly lower than the normalization value μ1 24. In this case it can be reliably determined, assuming that a confidence value is above the threshold value, that exactly one article piece was collected, and the number value can thus be defined as one.

[0075] For example, the weighing device 20 may detect a different weight, which, after projection onto the value distribution 23, falls in the range between the normalization values μ2 and μ3 and is above the normalization value μ2. Again, depending on the confidence value, the number value is defined as 2 in this case. If at least two articles are still to be transferred according to the order, the control unit 8 will move the robot head 6 to the target article carrier and discharge the articles there. However, if only one article piece is still to be picked, the control 8 will control the gripping unit 5 to put the collected article back into the source article carrier. It can possibly be provided that the article is not discharged at the location of the collection, but rather slightly offset thereto, in order to create a changed and possibly more favorable initial position for the next determination of the gripping position.

[0076] FIG. 3 shows a representation of how a determined value (a dimension or a weight) is projected onto the value distribution and the dimension value or the number value is determined thereby. As this classification and allocation method for dimensions and article weight is generally the same procedure, the following description is explicitly to be understood for both parameters.

[0077] FIG. 3 shows a value distribution 26 with three distribution curves 19, each of which describing the distribution of captured measurement values around a normalization value 17. This value distribution 26 develops adaptively after multiple cycles of the present method as, in the preparation step, a scattering measure function is applied for each detected measurement value, and normalization values and confidence values are determined based thereon. In the first cycle of the present method, the value distribution does not exist—only after multiple cycles, and therefore multiple applications of the scattering measure function to the detected measurement values, (stable) normalization values 17 and distribution curves 19 will develop. In particular, with an increasing number of adaptation cycles, the confidence values of the individual normalization values will also increase. However, this is not to say that all confidence values must inevitably increase. For example, a measurement value may scatter strongly despite a great number of preparation steps, causing a wide distribution curve and therefore a low confidence value.

[0078] After multiple cycles of the present method, a value distribution 26 represented by way of example will form.

[0079] After the determination of a dimension and/or after the determination of an article weight (generally of a measurement value), the present method compares the dimension and/or the article weight with ranges of the value distribution. In particular, the measurement value is projected onto the value ranges 27, and a range allocation is determined. The value distribution 26 in FIG. 3 shows multiple value ranges, which are determined by the adaptive embodiment of the present method.

[0080] Around each normalization value 17, there is a region (I, III, V) in which an unambiguous allocation of the measurement value to a normalization value is possible. If the confidence value then also falls above a definable threshold, the normalization value can be allocated to the dimension value and/or the number value.

[0081] Between the individual distribution curves 19, there are transition and/or overlap regions (II, IV) in which an unambiguous allocation of a distance value and/or a number value is not possible. If the measurement value is inside one of these regions, the present method provides that special handling is performed. For example, as special handling it may be provided that the two neighboring distribution functions are analyzed and a decision is made, in particular, based on the two confidence values, which one of the two normalization values is selected. In simplified terms, the scatter and the number of the measurement values flow into the determination of the confidence value, so that in most cases the normalization value with the higher confidence value is selected. This also results in the asymmetric position of the transition regions. Without considering the confidence value, the allocation border would be exactly at the intersection of the two distribution curves 19 and/or a possible transition region would be significantly wider. By means of the advantageous present method, it is now possible to displace both the position and the width of the transition region in favor of the distribution curve with the higher confidence value. If a past allocation of a normalization value to a measurement value caused, in further consequence, a picking error, the special handling can also take the history of past allocations into account in order to thereby increase the decision quality.

[0082] The figure further shows two edge regions (A, B), which also require an error handling to be carried out in case a measurement value is allocated to this region. These regions essentially represent erroneously captured measurement values. The special handling for these cases usually consists in that the image capturing is carried out again and/or the article is put back onto the article carrier by the gripping unit. A measurement value in one of these regions may also mean that an unintended article is on the article carrier. In a special handling, a superordinate warehouse management system may be alerted and the article carrier may be transported, by means of the conveying system, to a manual working area.

[0083] The illustrated arrangement of the article carriers 3 in relation to the robot 4 is to be understood merely as an example and was selected such particularly for illustrative reasons. In any case, other configurations in particular are also possible.

[0084] Ultimately, it is to be noted that in the different embodiments described, equal parts are provided with equal reference numbers and/or equal component designations, where the disclosures contained in the entire description may be analogously transferred to equal parts with equal reference numbers and/or equal component designations. Moreover, the specifications of location, such as at the top, at the bottom, at the side, chosen in the description refer to the directly described and depicted figure and in case of a change of position, are to be analogously transferred to the new position.

[0085] The exemplary embodiments show possible embodiment variants of the invention, and it should be noted in this respect that the invention is not restricted to these particular illustrated embodiment variants of it, but that rather also various combinations of the individual embodiment variants are possible and that this possibility of variation owing to the teaching for technical action provided by the present invention lies within the ability of the person skilled in the art in this technical field. Thus, any and all conceivable embodiment variants, which are possible by combining individual details of the embodiment variant shown and described, are also covered by the scope of protection.

[0086] Finally, as a matter of form, it should be noted that for ease of understanding of the method steps, drawing elements are partially not depicted to scale and/or are enlarged and/or are reduced in size.

LIST OF REFERENCE NUMBERS

[0087] 1 picking location [0088] 2 conveying system [0089] 3 article carrier [0090] 4 robot [0091] 5 gripping unit [0092] 6 robot head [0093] 7 robot base [0094] 8 control unit [0095] 9 image capturing system [0096] 10 opto-sensory preparation and analysis system [0097] 11 image [0098] 12 article(s) [0099] 13 gripping surface [0100] 14 arrangement [0101] 15 dimension [0102] 16 value distribution for dimensions [0103] 17 normalization value [0104] 18 dimension [0105] 19 distribution curve [0106] 20 weighing device [0107] 21 robot arm [0108] 22 weight [0109] 23 value distribution for article weight [0110] 24 normalization value [0111] 25 weight [0112] 26 value distribution [0113] 27 value ranges [0114] 28 distribution curve