SYSTEM AND METHOD FOR DETECTING A NUMBER OF GOODS IN A LOAD CARRIER

Abstract

In a system and a method for detecting a number of articles in a placement region of a load carrier in an article storage, the load carrier is provisioned in an analysis region, and an image of the placement region of the load carrier is captured by an image capturing system and transmitted to a data processing unit, whereupon the image is evaluated by an algorithm for object recognition, in order to recognize the articles arranged in the placement region of the load carrier as objects and to ascertain the number of articles in the placement region of the load carrier.

Claims

1. A system (1) for detecting a number of articles in a placement region (8) of a load carrier (4) in an article storage, comprising an analysis region (3), in which the load carrier (4) can be provisioned, an image capturing system (2) for monitoring the analysis region (3), and a data processing unit (9), wherein the image capturing system (2) is configured to capture an image of the placement region (8) of the load carrier (4) and to transmit the same to the data processing unit (9), wherein the data processing unit (9) is configured to evaluate an image transmitted by the image capturing system (2) by means of an algorithm for object recognition, in order to recognize the articles (7) arranged in the placement region (8) of the load carrier (4) as objects and to ascertain the number of articles in the placement region (8) of the load carrier (4).

2. The system (1) according to claim 1, wherein the system (1) has a control unit (10), and the data processing unit (9) generates a transport stipulation for the load carrier (4) based on the number of articles and transmits it to the control unit (10).

3. The system (1) according to claim 2, wherein the control unit (10) is configured to control a conveying device (5) of the article storage, so that the conveying device (5) conveys the load carrier (4) according to the transport stipulation.

4. The system (1) according to claim 1, wherein the data processing unit (9) is configured to detect article data by means of object recognition and/or to receive article data.

5. The system (1) according to claim 1, wherein the image capturing system (2) has a first camera (6a), which is arranged for capturing the image from a first angle, in particular 90° to a placement surface of the load carrier (4).

6. The system (1) according to claim 5, wherein the image capturing system (2) has at least one further camera (6b, 6c), which is arranged for capturing at least one further image of the placement region (8) from a further angle, wherein the further angle is different from the first angle.

7. The system (1) according to claim 1, wherein the system (1) has at least one detection unit for detecting at least one load carrier parameter, preferably a total weight of the load carrier (4), wherein the detection unit transmits the at least one load carrier parameter to the data processing unit (9), and the data processing unit (9) is preferably configured to take into account the load carrier parameter when evaluating the image.

8. The system (1) according to claim 1, wherein the data processing unit (9) is connected to a data center (12) via a communication module.

9. The system (1) according to claim 8, wherein the data processing unit (9) is configured to transmit a data set, preferably comprising an image of the placement region (8) and/or the ascertained number of articles, to the data center (12) and/or to receive a data set from it, wherein the data processing unit (9) optimizes the object recognition on the basis of received data sets.

10. A data center (12) for managing data sets, wherein the data sets each comprise at least one image of a placement region (8) of a load carrier (4) and an ascertained number of articles, wherein the ascertained number of articles indicates a number of articles in the placement region, and the data center (12) is connected, via a communication module, to a plurality of systems (1) according to claim 8 for transferring data sets between, in each case, one system (1) and the data center (12).

11. The data center (12) according to claim 10, wherein an algorithm for machine learning is implemented in the data center, which algorithm can be fed data sets in order to improve an algorithm for object recognition.

12. A warehouse system comprising a plurality of load carriers (4), each with a placement region (8), a conveying device (5) for transporting the load carriers (4), and a system (1) for detecting a number of articles in the placement region (8) of one of the load carriers (4), wherein the system (1) is formed according to claim 1.

13. A method for detecting a number of articles in a placement region (8) of a load carrier (4) in an article storage, wherein the load carrier (4) is provisioned in an analysis region (3) of the system (1) according to claim 1, wherein an image of the placement region (8) of the load carrier (4) is captured by means of an image capturing system (2) and is transmitted to a data processing unit (9), wherein the image is evaluated by the data processing unit (9) by means of an algorithm for object recognition, wherein articles (7), which are arranged in the placement region (8), are recognized as objects, wherein the number of articles is ascertained, whereupon the load carrier (4) is conveyed out of the analysis region (3).

14. The method according to claim 13, wherein a transport stipulation for the load carrier (4) is generated based on the ascertained number of articles, according to which transport stipulation the load carrier (4) is conveyed.

15. The method according to claim 13, wherein the load carrier (4) is moved through the analysis region (3) at a constant speed, during which the image is captured.

16. The method according to claim 14, wherein the image is evaluated in real time, and the transport stipulation is generated in real time.

17. The method according to claim 1, wherein the number of articles is compared to a target value, wherein the load carrier (4) is transported to a working area when the target value is undershot or exceeded or is transported to a dispatch area or dispatch area when the number of articles matches the target value.

18. The method according to claim 13, wherein a total weight of the load carrier (4) is measured by a weighing device and transmitted to the data processing unit (9), whereupon the number of articles in the placement region (8) of the load carrier (4) is calculated from a known individual weight of the articles (7) and a known tare weight of the load carrier (4) and taken into account in the detection of the number of articles.

19. The method according to claim 13, wherein the algorithm for object recognition is optimized in a teaching step by means of machine learning, on the basis of existing article data and/or stored images of placement regions (8) of load carriers (4), wherein a plurality of images of placement regions (8) or a plurality of images of placement regions (8) and numbers of articles corresponding to the images are fed into a computing unit, on which computing unit the algorithm for object recognition is implemented, whereupon a test run is performed, during which a plurality of images of placement regions (8) are fed into the computing unit, and the corresponding number of articles is ascertained by the computing unit using the algorithm for object recognition.

20. The method according to claim 13, wherein the object recognition of the data processing unit (9) is optimized during ongoing operation by means of machine learning using centrally stored images and numbers of articles corresponding to said images.

21. The method according to claim 17, wherein, in case of a deviation of the ascertained number of articles from the target value, an input prompt is directed at a user, and a number of articles manually counted by the user is recorded and fed into the data processing unit (9), whereupon the ascertained number of articles is compared to the manually counted number of articles by the data processing unit, and the algorithm for object recognition is adapted based on the comparison.

Description

[0101] For the purpose of better understanding of the invention, it will be elucidated in more detail by means of the figures below.

[0102] These show in a respectively very simplified schematic representation:

[0103] FIG. 1 a system for detecting a number of articles in a placement region of a load carrier in an article storage;

[0104] FIG. 2 a top view onto a detected load carrier;

[0105] FIG. 3 an alternative embodiment of the system for detecting the number of articles;

[0106] FIG. 4 a block diagram of the system for detecting the number of articles;

[0107] FIG. 5 a network of a plurality of systems for detecting the number of articles and a data center;

[0108] FIG. 6 a flowchart for a method for detecting the number of articles at the goods-in point;

[0109] FIG. 7 a flowchart for a method for detecting the number of articles at the dispatch area;

[0110] FIG. 8 a flowchart for a method for detecting the number of articles in an automatic inventory.

[0111] First of all, it is to be noted that in the different embodiments described, equal parts are provided with equal reference numbers and/or equal component designations, where the disclosures contained in the entire description may be analogously transferred to equal parts with equal reference numbers and/or equal component designations. Moreover, the specifications of location, such as at the top, at the bottom, at the side, chosen in the description refer to the directly described and depicted figure, and in case of a change of position, are to be analogously transferred to the new position.

[0112] FIG. 1 shows a system 1 for detecting a number of articles. The system 1 comprises an image capturing system 2 as well as an analysis region 3, in which a load carrier 4 can be provided, as it is shown in FIG. 1. The analysis region 3 is essentially that region which is visible for the image capturing system 2 and/or can be represented by it.

[0113] A provisioning of the load carrier 4 can take place manually, by means of a handling robot or, as shown in FIG. 1, by means of an automatedly operated conveying device 5. In the example shown, the load carrier 4 is transported by the conveying device 5 in a conveying direction R and in this process, is moved through the analysis region 3.

[0114] According to the embodiment shown, the load carrier 4 is formed by a container, a carton, or a tray. In this regard, a placement region 8 not visible in FIG. 1 may correspond to an internal volume of the container, carton, or tray. The placement region 8 is visible from above and/or from a bird's-eye perspective. Such a load carrier 4 comprises a bottom shelf, side walls rising up from it and at least one loading opening bounded by the side walls. If the side walls are designed to be rather low, they can also be referred to as a side edge, as this would be the case for a tray.

[0115] The load carrier 4 may be transported by an automated conveying device 5, for which purpose it has a transport surface. The transport surface is formed on a bottom side of the bottom shelf facing away from the placement region 8. The placement surface is formed on an upper side of the bottom shelf facing the placement region.

[0116] Preferably, the conveying device 5 is designed such that it is able to ensure a constant, vertical distance between the image capturing system 2 and the load carrier 4 in the analysis region 3, while the load carrier 4 is being transported through the analysis region 3.

[0117] In FIG. 1, the conveying device 5 is formed as a stationary conveying device 5, for example by a belt conveying device or a roller conveying device. Alternatively, the conveying device 5 may also be formed as a mobile conveying device 5, for example comprising one or multiple, preferably driverless, transport vehicles.

[0118] The image capturing system 2 shown in FIG. 1 has a first camera 6a, which is arranged for capturing an image of the placement region 8 of the load carrier 4 from the bird's-eye perspective. For this purpose, an optical axis of the first camera 6a is oriented essentially orthogonal to a placement surface of the bottom shelf and/or essentially orthogonal to a conveying plane of the conveying device 5, on which the load carrier 4 is transported. Preferably, a placement surface of the load carrier 4 is positioned in or parallel to the conveying plane.

[0119] FIG. 2 shows the load carrier 4 from the bird's-eye perspective. In this representation, the articles 7 arranged in the placement region 8 of the load carrier 4 are visible. The articles 7 are placed in the receiving region in an essentially disorganized manner, so that an overlap of the articles 7 may possibly occur. In the example shown, the articles 7 are schematically represented as rectangles. Of course, in reality, the articles 7 may have any shape. Furthermore, it is possible that the load carrier 4 is loaded with articles 7 of just one type or with a mix of article types.

[0120] In FIG. 3, an alternative embodiment of the system 1 of FIG. 1 is shown. The system 1 is formed essentially as in the previously shown exemplary embodiment. Here, as well, essentially any conveying device 5 may be provided. Furthermore, the image capturing system 2 has a second camera 6b and a third camera 6c in addition to the first camera 6a. The second camera 6b and the third camera 6c are arranged for capturing an image of the placement region 8 of the load carrier 4 from a second perspective and a third perspective. For this purpose, the optical axis of the first camera 6a and/or the optical axis of the second camera 6b each enclose an angle with the placement surface of the placement region 8, which angle is different from 90°.

[0121] FIG. 4 shows a schematic representation of the system 1 for detecting the number of articles.

[0122] The image capturing system 2 is connected to a data processing unit 9, so that images captured by the image capturing system 2 can be transmitted to said data processing unit 9. For this purpose, the image capturing system 2 as well as the data processing unit 9 may each have a communication interface, for example a USB port or the like, so that a communication connection between the image capturing system 2 and the data processing unit 9 can be realized by means of a data cable. As an alternative thereto, the communication interface may comprise a transmission and receiving device, so that the communication interface is realized using a wireless network, in particular a local radio network (wireless local area network; WLAN) or based on a Bluetooth standard.

[0123] Moreover, the data processing unit 9 is configured to receive images from the image capturing system 2 and to evaluate them by means of an algorithm for object recognition in order to ascertain a number of articles in the placement region 8. Furthermore, the data processing unit 9 is configured to save the ascertained number of articles and/or to generate a transport stipulation for the load carrier 4, which transport stipulation is based on the ascertained number of articles.

[0124] Moreover, the data processing unit 9 is connected to an electronic control unit 10, so that the transport stipulation can be transmitted to the control unit 10. For this purpose, the control unit 10 as well as the data processing unit 9 may each have a communication interface, which comprises, for example, a USB port or the like and/or a transmission and receiving device, so that a communication connection between the data processing unit 9 and the control unit 10 may also be realized by means of a data cable or a wireless network, in particular a local radio network (wireless local area network; WLAN) or based on a Bluetooth standard. Alternatively, the data processing unit 9 and the control unit 10 may be two sub-units of a checking unit 11 of the system 1. Such a checking unit 11 is adumbrated in FIG. 4 as a dot-dashed box.

[0125] Advantageously, the control unit 10 has a communication connection, for example a wireless one or one using a cable, to the conveying device 5, so that the conveying device 5 can be controlled, and a transport of the load carrier 4 according to the transport stipulation can be prompted.

[0126] The transport stipulation preferably comprises a destination, for example a particular working area or dispatch area, a transport route, for example along a main path or along a secondary path of the conveying device 5, and/or a changing of the switch of the conveying device 5 or the like.

[0127] In order to gather additional article data, the system 1 may have further detection devices not shown in FIG. 1 to FIG. 4, which detection devices are connected to the data processing unit 9 so that gathered data can be transmitted to said data processing unit 9. For example, a weighing device may be provided, which is preferably integrated into the conveying device 5. The weighing device may be configured to gather a total weight of the load carrier 4 and to transmit said weight to the data processing unit 9.

[0128] FIG. 5 shows a schematic representation of a network with a plurality of previously described systems 1, which are interlinked and/or connected via a data center 12. The network may essentially comprise any number of systems 1.

[0129] In the example shown, at least a first system 1 and a further system 1′ are connected to the data center 12. In FIG. 5, a box with three dots adumbrates optional further systems 1′, which may also be connected to the data center 12.

[0130] Here, the systems 1, 1′ are each connected to the data center 12 by means of a bidirectional communication connection (adumbrated by a double arrow). Thereby, on the one hand, a data transfer from the system 1, 1′ to the data center 12 is possible, and on the other hand, a data transfer from the data center 12 to the system 1, 1′ is possible. For this purpose, the systems 1, 1′ may each have a first communication module, and the data center 12 may have a second communication module. The communication modules may each have a transmission and receiving device, a connection for the cable connection, for example a USB port, an ethernet port, or the like.

[0131] The communication connection between the data center 12 and the systems 1, 1′ may preferably be established by a data network, in particular an internet connection, or wirelessly, in particular using the Bluetooth standard, or via a local radio network (wireless local area network; WLAN). The data center 12 may furthermore be formed as a platform set up on a server. The server may, for example, be connected to the individual systems 1, 1′ via an internet connection.

[0132] In a method for detecting the number of articles in the placement region 8 of the load carrier 4, the load carrier 4 is provisioned in the analysis region 3, and at least one image of the placement region 8 of the load carrier 4 is captured with the image capturing system 2. The provisioning of the load carrier 4 in the analysis region 3 may comprise a continuous transport movement of the load carrier 4 through the analysis region 3 or the stopping of the load carrier 4 in the analysis region 3.

[0133] The captured image is transmitted to the data processing unit 9, whereupon the image is evaluated by means of an algorithm for object recognition, in order to determine the number of articles. Furthermore, a transport stipulation for the load carrier 4 is generated. Moreover, the number of articles can be stored.

[0134] In a next step, the transport stipulation is transmitted to the control unit 10, which controls the conveying device 5 such that the load carrier 4 is transported to a particular destination according to the transport stipulation. The destination may be a storage zone, a dispatch area in the article storage (in particular in the storage zone of the article storage), a working area, such as a picking station, a packaging and/or dispatch station, a correction working area, or the like.

[0135] FIG. 6 shows a schematic representation of a method for detecting the number of articles at the goods-in point. For this purpose, the previously described system 1 may be arranged between a delivery zone at the goods-in point and the storage zone of the article storage. The delivery zone comprises one or multiple working areas. The working area may be formed by a reloading station, where articles are reloaded into a source load carrier as described below.

[0136] In a first step S1, articles 7 delivered to the goods-in point may be reloaded into one or multiple load carriers 4, preferably source load carrier, such as source containers. Usually, only articles of a single type are provisioned on delivery pallets. Here, the articles 7 of the delivery are distributed onto one or multiple load carriers 4 such that they are arranged in a single layer or a maximum of two layers in the load carrier 4. With this optional step, the ascertainment of the number of articles can be simplified, as possibly, one image from a single perspective, preferably the bird's-eye perspective suffices for representing all articles 7 in the placement region 8. Furthermore, an evaluation result of the object recognition can be improved.

[0137] In a second step S2, the load carrier 4 or one of the load carriers 4 is provisioned in the analysis region 3 of the system 1. Preferably, this takes place manually, by means of a handling robot or the conveying device 5.

[0138] Moreover, in a third step S3, at least one image of the placement region 8 of the load carrier 4 is captured. Here, an image is captured from a first perspective, in particular from above and/or from a bird's-eye perspective, using the first camera 6a of the image capturing system 2. If a second camera 6b and/or a third camera 6c are present, it is also possible for one or multiple further images to be captured by each of the second camera 6b and/or the third camera 6c, in particular from one or multiple further perspectives.

[0139] The captured image is transmitted from the image capturing system 2 to the data processing unit 9. In order to ascertain the number of articles corresponding to the respective load carrier 4, the captured image is evaluated in a fourth step S4 by means of an algorithm for object recognition. Here, the individual articles 7 arranged in the placement region 8 of the load carrier 4 are recognized as objects by the data processing unit 9, and the ascertained number of articles is stored. This step may be performed for all images corresponding to a load carrier 4 if multiple images are captured, in particular by multiple cameras.

[0140] In a fifth step S5, a transport stipulation is generated by the data processing unit 9 on the basis of the number of articles, according to which transport stipulation the load carrier 4 is transported to a working area or to a dispatch area in the storage zone.

[0141] Steps two to five may be repeated for all load carriers 4 corresponding to one delivery, so that for each of the load carriers 4 a corresponding number of articles is ascertained. Furthermore, the relevant numbers of articles may be added up to a total number of articles and be compared to a delivery quantity. This takes place for example by means of the data processing unit 9. Thus, it is possible to check exactly upon delivery whether a correct number of the articles 7 has been delivered.

[0142] In a sixth step S6, the load carrier 4 can be stored. If the articles 7 were distributed onto multiple load carriers 4 in the first step S1, they may first be united in one load carrier 4 in order to reduce the storage space required for the articles 7, whereupon said load carrier 4 is stored.

[0143] FIG. 7 schematically shows a method for detecting the number of articles at the dispatch area. For this purpose, the previously described system 1 may be arranged between a picking zone and an article distribution zone at the dispatch area. The picking zone comprises one or multiple working areas. The working area may be formed by a picking station where articles are removed from a source load carrier and picked into a target load carrier according to an order.

[0144] The load carrier 4, in particular a target load carrier, is loaded, in a first step S1, according to an order at a picking station and is subsequently transported from the picking station to the analysis region 3. In steps two to four S2, S3, S4, the number of articles corresponding to the load carrier 4 is ascertained as described above.

[0145] In the fifth step S5, the ascertained number of articles is compared to a corresponding customer order, an according transport stipulation is generated, and the load carrier 4 is transported to the respective destination according to the transport stipulation.

[0146] If the ascertained number of articles matches the order, the load carrier 4 may be transported to a dispatch working area, for example. However, if the ascertained number of articles deviates from the order, this points to a picking error. The load carrier 4 may then, for example, be transported to a correction working area.

[0147] However, the working area may also be formed by a packing place where articles are packed into a target load carrier according to an order, wherein the target load carrier in this case corresponds to a shipping loading aid, for example a shipping carton. At the packing place, a final check may be performed. If the ascertained number of articles matches the order, the shipping loading aid may be closed and dispatched. However, if the ascertained number of articles deviates from the order, this points to a picking error. The shipping loading aids may then, for example, be transported to a correction working area.

[0148] FIG. 8 schematically shows a method for detecting the number of articles, in which an automatic inventory can be realized.

[0149] In the case of the automatic inventory, in the first step S1, a load carrier 4 is retrieved from the article storage and provisioned in the analysis region 3 of the system 1 for detecting a number of articles in the load carrier 4. This is preferably carried out by means of an automated conveying device 5.

[0150] In steps two to four S2, S3, S4, the number of articles corresponding to the load carrier 4 is ascertained and stored, as described above.

[0151] In the fifth step S5, a transport stipulation is generated by the data processing unit 9, which is based on the ascertained number of articles. Thus, a filled load carrier 4 can be transported back into the storage zone by the conveying device 5. An empty load carrier 4 may, however, be transported to a working area and, for example, be re-filled at the goods-in point.

[0152] In the case of the automatic inventory, steps one to five may, of course, be carried out sequentially for a plurality of the load carriers 4, in particular all load carriers 4, of the article storage.

[0153] Furthermore, a learning process, not shown in FIG. 6 to FIG. 8, comprising a teaching step and a test run, may be provided in each case in such methods for detecting the number of articles, which learning process is, in particular, performed before an initial operation of the system 1, 1′.

[0154] In the teaching step, an algorithm implemented in the data processing unit 9 and/or an artificial neural network present in the data processing unit 9 or the like is trained on the basis of a plurality of existing images of placement regions 8, each with a known corresponding number of articles. Here, the images as well as the corresponding numbers of articles are fed into the data processing unit 9. This may be carried out particularly on the basis of at least 100 images, preferably about 200 images.

[0155] After the teaching step, a test run is performed. In this process, a plurality of images with a known number of articles is fed into the data processing unit 9. The data processing unit 9 evaluates the images by means of object recognition in order to ascertain a corresponding number of articles for each of the images. Moreover, the data processing unit 9 issues which images were poorly evaluable and/or with low confidence and which images could be evaluated well and/or with high confidence. The ascertained number of articles of images evaluated with low confidence can be compared to the corresponding known number of articles in order to determine whether a correct number of articles was ascertained. Furthermore, the number of articles ascertained for each of the images can be compared to the corresponding known number of articles in order to determine the hit rate. Here, the hit rate indicates the portion of the images for which the ascertained number of articles matches the known number of articles. If the hit rate falls below a predefined value, for example below 80%, the test run as well as the teaching step can be repeated using further images and/or load carriers 4, until the hit rate reaches at least the predefined value.

[0156] With the described system 1, 1′ and method, it is thus possible to realize an automated detection of the number of articles at the goods-in point and/or at the dispatch area of an article storage as well as an automated inventory in an article storage, whereby an essential component of a fully automated article storage can be actualized.

[0157] Finally, it should also be noted that the scope of protection is determined by the claims. Nevertheless, the description and drawings are to be used for construing the claims.

[0158] In particular, it should also be noted that the system shown may in reality comprise more or fewer components than those shown. In some cases, the shown systems and/or their components may not be depicted to scale and/or be enlarged and/or reduced in size.

LIST OF REFERENCE NUMBERS

[0159] 1, 1′ System

[0160] 2 Image capturing system

[0161] 3 Analysis region

[0162] 4 Load carrier

[0163] 5 Conveying device

[0164] 6a, 6b, 6c Camera

[0165] 7 Article

[0166] 8 Placement region

[0167] 9 Data processing unit

[0168] 10 Control unit

[0169] 11 Control unit

[0170] 12 Data center

[0171] R Conveying direction

[0172] S1 First step

[0173] S2 Second step

[0174] S3 Third step

[0175] S4 Fourth step

[0176] S5 Fifth step

[0177] S6 Sixth step