SYSTEM FOR OPTICAL WASH ITEM RECOGNITION IN DISHWASHERS, METHOD FOR OPTICAL WASH ITEM RECOGNITION, DISHWASHER WITH AN OPTICAL WASH ITEM RECOGNITION SYSTEM AND METHOD FOR OPERATING SUCH A DISHWASHER
20220273158 · 2022-09-01
Inventors
- Harald DISCH (Elzach, DE)
- Björn S. HIMMELSBACH (Offenburg, DE)
- Sebastian HUBER (Appenweier, DE)
- Patrick DUFNER (Offenburg, DE)
Cpc classification
A47L15/4295
HUMAN NECESSITIES
A47L2401/04
HUMAN NECESSITIES
A47L2501/30
HUMAN NECESSITIES
A47L15/0076
HUMAN NECESSITIES
A47L15/0021
HUMAN NECESSITIES
A47L15/24
HUMAN NECESSITIES
G06V10/145
PHYSICS
A47L15/006
HUMAN NECESSITIES
A47L15/0047
HUMAN NECESSITIES
A47L15/0081
HUMAN NECESSITIES
A47L15/46
HUMAN NECESSITIES
International classification
A47L15/00
HUMAN NECESSITIES
A47L15/24
HUMAN NECESSITIES
A47L15/46
HUMAN NECESSITIES
G06V10/145
PHYSICS
Abstract
A system for optical wash item recognition for commercial dishwashers embodied as conveyor dishwashers or as automatic programme dishwashers, in particular hood dishwashers, wherein the system includes an optical recognition system, especially a camera, for recording at least one two-dimensional image of at least some of the wash items to be treated in the dishwasher. The system also includes an analysis device for analysing at least one recorded image in such a way that individual wash items or groups of individual wash items in the recorded image are located and classified. At least one neural network is used to analyse the at least one recorded image.
Claims
1. System for optical wash item recognition, in particular for commercial dish washers, wherein the system comprises: an optical recognition system for recording at least one two-dimensional image of at least some of the wash items to be treated in the dishwasher; and an analysis device for analysing the at least one recorded image in such a way that in the recorded image at least individual wash items or groups of individual wash items are located and classified, wherein at least one neural network is used to analyse the at least one recorded image.
2. System according to claim 1, wherein the neural network is a convolutional neural network, Yolo, Sweaty or single-shot multibox detector network architecture.
3. System according to claim 1, wherein the neural network is a single-shot multibox detector network architecture of least five layers.
4. System according to claim 1, wherein a method of supervised learning is used when analysing the at least one recorded image.
5. Method for optical wash item recognition in commercial dishwashers embodied as conveyor dishwashers or as program automation dishwashers, wherein the method includes the following procedural steps: with the help of an optical recognition system, at least one two-dimensional image of at least some of the wash items to be treated in the dishwasher is recorded; and at least one recorded image is analysed with the help of an analysis device in such a way that in the recorded image at least one individual wash item or groups of individual wash items is/are located and classified, wherein for analysing the at least one recorded image at least one neural network is used, wherein the at least one neural network is a single-shot multibox detector network architecture with at least five layers, and wherein a method of super vised learning is used when analysing the at least one recorded image.
6. Dishwasher which is embodied as a conveyor dishwasher or as a program automation dishwasher and which comprises a treatment zone or a treatment chamber that can be closed with a door or a hood in which wash items can be treated according to a pre-selected or defined treatment program, wherein the dish washer comprises an optical wash item recognition system according to claim 1.
7. Dishwasher according to claim 6, wherein the optical recognition system is accommodated within the treatment zone or treatment chamber of the dishwasher or wherein the optical recognition system is disposed in relation to the treatment zone or treatment chamber of the dishwasher such that the optical recognition system is designed so that at least one two-dimensional image of at least some of the wash items to be treated in the dishwasher is recorded when the wash items to be treated are accommodated in the treatment zone or treatment chamber of the dishwasher.
8. Dishwasher according to claim 7, wherein a lighting device is assigned to the optical recognition system to illuminate at least some areas of the wash items to be treated in the dishwasher, during image recording using the optical recognition system, whereby the lighting device is embodied to illuminate the wash items to be treated homogeneously and diffusely, with light of a colour temperature between 5,000 and 5,700 K.
9. Dishwasher according to claim 6, wherein the dishwasher comprises a control device in which different treatment programs for the treatment of wash items in the treatment zone or treatment chamber of the dishwasher are stored, wherein the treatment programs differ with respect to at least one of the treatment parameters listed below: a time duration of a washing phase, a time duration of a final rinse phase, a time duration of a drying phase, a temperature of a washing liquid to be sprayed in the treatment zone or chamber during the washing phase, a temperature of a final rinse liquid to be sprayed in the treatment zone or chamber during the final rinse phase, a nozzle pressure of the final rinse liquid to be sprayed during the final rinse phase, a quantity of washing liquid per unit time to be sprayed during the washing phase and/or a quantity of final rinse liquid to be sprayed per time unit during a final rinse phase, wherein the control device is embodied to select and adjust accordingly a treatment program or the individual treatment parameters for the treatment of the wash items, automatically, depending on the determined classification of the wash items.
10. Method for operating a dishwasher, which is embodied as a conveyor dishwasher or as a program automation dishwasher, with a treatment zone or treatment chamber that can be closed by a door or a hood and in which wash items, accommodated in at least one wash basket, can be treated according to a pre-selected or defined treatment program, wherein the method comprises: classifying the wash items to be treated with the help of an optical wash item recognition system; and treating the wash items in the treatment zone or treatment chamber of the dishwasher with treatment parameters that have been automatically selected depending on the classification of the wash items, wherein the treatment parameters relate to the following: a time duration of a washing phase, a time duration of a final rinse phase, a time duration of a drying phase, a temperature of a washing liquid to be sprayed in the treatment zone or treatment chamber during the washing phase, a temperature of a final rinse liquid to be sprayed in the treatment zone or treatment chamber during the final rinse phase, a nozzle pressure of the washing liquid to be sprayed in the treatment zone or treatment chamber during the washing phase, a nozzle pressure of the final rinse liquid to be sprayed in the treatment zone or treatment chamber during the final rinse phase, a quantity of washing liquid to be sprayed in the treatment zone or treatment chamber per unit time during the washing phase and/or a quantity of final rinse liquid to be sprayed in the treatment zone or treatment chamber per unit of time during a final rinse phase.
11. (canceled)
12. Method according to claim 10, wherein before classifying the wash items to be treated, the wash items are placed in the treatment zone or treatment chamber of the dishwasher and the treatment zone or treatment chamber is closed.
13. Method according to claim 10, wherein before the process of classification, the wash items are placed in the treatment zone or treatment chamber of the dishwasher and the treatment zone or treatment chamber is closed, and the condensation of water vapor is prevented or reduced in the closed treatment zone or treatment chamber, on a lens of the optical wash item recognition system.
14. Method according to claim 13, wherein in order to prevent or reduce water vapor condensation on the lens of the optical wash item recognition system, a pre-wash of the wash items is carried out, wherein the pre-wash lasts no longer than 10 seconds.
15. Method according to claim 14, wherein during the pre-wash liquid is sprayed in the treatment chamber, which pre-wash liquid is not heated.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0048] The invention is described in more detail below on the basis of exemplary embodiments with reference to the drawings.
[0049] In the figures:
[0050]
[0051]
DETAILED DESCRIPTION
[0052] The dishwashers 1 shown schematically in the drawings are designed in particular as hood dishwashers. Hood dishwashers can be found in sculleries, for example, of hotels, restaurants and cafés. In the case of hood dishwashers, the wash items are sorted into wash baskets, which are then pushed into the dishwasher. The washing and rinsing cycle starts by closing the hood.
[0053] Often a hood dishwasher has two tables on the left and right sides. These are used for the pre-treatment of soiled wash items as well as for sorting and buffering clean wash items. Pre-treatment is one of the most time-consuming processes in sculleries. Very soiled wash items are pre-treated in a basin next to the dishwasher. Soiling can be, for example, dried food residues or burnt-in deposits in pots and pans.
[0054] After soaking, the wash items are typically additionally washed by hand before being placed in wash baskets and washed in the dishwasher.
[0055] Another time-consuming step is post-treatment. If the washing, final rinse and drying results do not meet customer expectations (e.g. due to soiling or stains on the wash items), a second wash or manual polishing is required.
[0056] The dishwasher offers the operator the possibility to switch between several rinsing programs or treatment programs. These are specially optimized for specific types of wash items (temperature, amount of rinsing water, cycle time, etc.) to minimize pre-treatment and post-treatment.
[0057] Based on the fact that many operators never change the treatment program, the object is to replace manual program selection with automatic selection of the wash program. To do this, the dishwasher must be able to detect different types of wash items (e.g. glass, cutlery, GN containers, etc.). The operator of the dishwasher no longer has to select the treatment program manually. Instead, the dishwasher can automatically select the appropriate treatment program with the corresponding treatment parameter to improve washing, final rinse and drying results. As a result, the dishwasher operator has no or fewer secondary runs to handle, which would have been caused by choosing the wrong treatment program.
[0058] For this purpose, optical wash item recognition is specified according to the invention, wherein the wash item recognition is carried out using neural networks. According to preferred implementations, an adapted SSD network architecture evaluated in the course of an investigation is used, which is characterized in terms of its accuracy on the one hand, the necessary calculation effort on the other hand and also by the required storage capacity.
[0059] Before an exemplary embodiment of the wash item recognition system according to the invention is described in more detail, dishwashers for which the wash item recognition system is optimally suited should first be described with reference to the representations in
[0060] The dishwasher 1 according to the invention has a treatment chamber 2 for cleaning wash items that are not shown in the drawings. A washing tank 4 is arranged under the treatment chamber 2, into which liquid from the cleaning chamber 2 can flow back by gravity. The washing tank 4 can be covered at the transition to the treatment chamber 2 using a sieve that is not shown in the drawings. Washing liquid 6, which is usually water, is located in the washing tank 4, wherein if necessary a cleanser can be automatically supplied thereto in a controlled manner by a cleanser dosing device that not shown in the drawings. The washing liquid 6 can be conveyed by a washing pump 8 via a washing pipe system 10 to washing nozzles 33 and 34 and sprayed onto the wash items by the washing nozzles 33, 34 in the treatment chamber 2. The sprayed washing liquid 6 then flows back into the washing tank 4. A drain 3 with a drain pump 11 can be connected at the lower end 5 of the washing tank 4 to empty the washing tank 4 as required.
[0061] With the embodiment of the dishwasher 1 according to the invention shown in
[0062] With the embodiment of the dishwasher 1 according to the invention shown in
[0063] The fresh water tank 14 has an inlet 37, which is connected to a fresh water supply pipe 31. The fresh water tank 14 is fed either pure fresh water or fresh water with added rinse aid via this fresh water supply pipe 31.
[0064] Of course, it is also conceivable to arrange the already mentioned rinse aid dosing device that is not explicitly shown in the drawings downstream of the outlet 36 of the fresh water tank 14.
[0065] The washing nozzles 33, 34 and the final rinse nozzles 7, 9 are each preferably arranged above and below the wash item area and directed towards the wash item area of the treatment chamber 2. With the embodiment of the dishwasher 1 according to the invention represented in
[0066] With the embodiment of the dishwasher 1 according to the invention illustrated in
[0067] The backflow preventer 16 has an outlet 40, which is connected via fresh water supply pipes 24 and 41 to a water softener device 39. The water softener device 39 has on the one hand a salt tank 42 connected to the fresh water supply pipe 41, and on the other hand mutually parallel first and second water softeners 39a, 39b. The two mutually parallel water softeners 39a, 39b are connected via a corresponding fresh water pipe system and the fresh water supply pipe 24 to the outlet 40 of the backflow preventer 16. The water softeners 39a, 39b of the water softener device 39 can be operated alternately by appropriate control of valves 43 in order to soften the fresh water supplied to the boiler 21 via the fresh water supply pipes 26 and 28 and the fresh water supplied to the fresh water tank 14 via the fresh water supply pipes 26, 29 and 31.
[0068] A suitable salt or a suitable chemical may be contained in the salt tank 42 belonging to the water softener device 39. The salt or chemical is used to regenerate a water softener material added to the fresh water or a decay product resulting after the addition as required. The salt container 42 can be refilled with the salt or the chemical from the treatment chamber 2 of the dishwasher 1 via an opening that can be closed by a cover 44.
[0069] With the preferred embodiment of the dishwasher 1 according to the invention presented, the boiler 21 has an integrated steam generator 20. A corresponding steam outlet 23 of the steam generator 20 is formed at the upper part of the boiler 21. The steam outlet 23 of the steam generator 20 is connected to the treatment chamber 2 via a steam pipe 32 at a point above the washing tank 4 in order to introduce therein as required the steam produced in the steam generator 20. The outlet opening of the steam pipe 32 is preferably located between the upper nozzles 7, 33 and the lower nozzles 9,34 of the washing pipe system 10 or of the final rinse pipe system 15. Of course, other positions are also possible.
[0070] In the boiler 21, which serves not only for heating the final rinse liquid, but also for generating steam, there is a heater 45. Furthermore, a level sensor 46 may be arranged in or on the boiler, which controls a valve 19 of the fresh water pipe 17, for example.
[0071] In the preferred embodiment of the dishwasher 1 according to the invention shown in
[0072] A program control device 50 shown only schematically in the drawings is used to control at least one cleaning program and is designed and connected to the controllable components of the dishwasher 1, such as to the valves 19, 27 and 30, to the washing pump 8, to the first final rinse pump 12, to the second final rinse pump 13, and/or to a (not explicitly shown) heating control circuit for the control of the heating 45 so as to implement a cleaning process (cleaning cycle) with the following successive steps (phases): [0073] 1. a washing phase, in which washing liquid 6 from the washing tank 4 is sprayed by means of the washing pump 8 through the washing pipe system 10 into the treatment chamber 2 and can then flow back from the treatment chamber 2 into the washing tank 4 by gravity; [0074] 2. a fresh water final rinse phase, in which heated or unheated fresh water or fresh water with added rinse aid as a final rinse liquid is sprayed into the treatment chamber 2 by means of the first final rinse pump 12 or the second final rinse pump 13 and then can flow from the treatment chamber 2 into the washing tank 4 by gravity; and [0075] 3. (optional) a drying phase in which drying air circulates within the treatment chamber 2.
[0076] In a preferred implementation of the program control device 50, this is designed in such a way that, depending on the detected type of wash items, it automatically controls the controllable components of the dishwasher 1 in such a way that a steam final rinse phase is carried out after the fresh water final rinse phase, in which steam is generated by means of the steam generator 20 connected to the fresh water supply pipe 28 and is fed into the treatment chamber 2.
[0077] The dishwasher 1 according to the invention, as shown in
[0078] As already mentioned, with the dishwasher 1 according to the present invention, the program control device 50 only schematically shown in the drawings is also provided. According to the invention, the program control device 50 is designed to control different controllable components of the dishwasher 1, such as the respective pumps and valves, according to a predetermined or predeterminable program sequence, in order to be able to set suitable process parameters during the individual treatment phases (washing phase, final rinse phase and drying phase). The program control device 50 is connected in particular via a suitable communication connection to an analysis device 52 of the recognition system 51 in order to be able to call up, continuously or at predetermined times or events, preferably before the beginning of the washing cycle, the type (classification) of the wash items accommodated in the treatment chamber 2 of the dishwasher 1 determined by the analysis device 52 on the basis of the image taken by the recognition system 51.
[0079] The analysis device 52 is designed to automatically recognize the wash item type/classification of the wash items to be treated on the basis of the image taken by the recognition system 51. Preferably, the analysis device 52 can automatically recognize at least the following wash items: [0080] plates made of porcelain or a porcelain-like material; [0081] cups made of porcelain or a porcelain-like material, glass or a glass-like material; [0082] bowls made of porcelain or a porcelain-like material, glass or a glass-like material; [0083] trays or tray-like objects made of a plastic material; [0084] containers made of a metal, in particular stainless steel containers, in particular GN containers; [0085] pots made of a metal, in particular stainless steel pots; [0086] pans made of a metal, in particular stainless steel pans; [0087] cutlery and cutlery parts made of a metal, in particular stainless steel cutlery and cutlery parts; and [0088] drinking glasses made of glass or a glass-like material.
[0089] In the event that the analysis device 52 cannot determine the wash item type/classification on the basis of the image taken by the recognition system 51, the wash items in question will be identified as “other wash items”.
[0090] The solution according to the invention is characterized not only on the one hand by the automatic detection of the type/classification of the wash items to be treated and on the other hand by the automatic recognition of the wash items to be treated, but also by the additional functionality of the program control device 50, according to which this is designed to automatically select a predetermined or predeterminable treatment program depending on the recorded wash item type, according to which the wash items accommodated in the treatment chamber 2 are to be treated during at least one treatment phase, and to adjust the process parameters of the selected treatment program by suitable control of the controllable components of the dishwasher 1. For this purpose, it is generally conceivable that the program control device 50, for example, has a memory device not explicitly shown in
[0091] The invention is based inter alia on the knowledge that the optimal process parameters for the individual treatment phases depend on the type of wash items to be treated. The different types of wash items require suitable treatment programs according to which the wash items are to be treated, for example during the washing phase, the fresh water final rinse phase and/or the drying phase. For example, during the washing phase in which the washing liquid is sprayed, only slightly soiled dishes, such as drinking glasses, require only a relatively small amount of washing liquid injected per unit of time and only a relatively low nozzle pressure, in contrast to heavily soiled dishes with burnt-in and difficult to remove food residues, such as cooking utensils, warming containers, baking trays, etc. Furthermore, compared to heavily soiled dishes, only a shorter cycle duration of the washing phase is required for lightly soiled dishes.
[0092] On the other hand, the different types of wash items also require matching treatment programs for the fresh water final rinse phase following the washing phase. Also an adjustment of the process parameters that is carried out with regard to the type of wash items is also conceivable for the steam rinse phase and drying phase to be carried out after the fresh water rinse phase.
[0093] In this way, “overtreatment” can be effectively prevented in a simple but effective way, especially of only slightly soiled dishes, such as drinking glasses, so that no more resources of energy, water, chemicals etc. are used than necessary during the cleaning process (and possibly during the drying process) of such only slightly soiled dishes.
[0094] Also, the consumption of fresh water can be reduced which is sprayed onto the wash items, for example during the fresh water final rinse phase, in pure form or with further additives added. Due to the reduced fresh water consumption of the dishwasher 1, the consumption of chemicals, in particular rinse aids and/or cleansers can be reduced, without causing a change in the chemical concentration in the respective liquid (washing liquid, final rinse liquid). Due to a reduced water supply, the amount of water to be heated is reduced, which saves energy.
[0095] The invention is based inter alia on the knowledge that in conventional dishwashers, with washing and final rinse cycles running according to a factory preset program, overtreatment of the dishwasher often takes place. It is often sufficient for an adequate washing and rinsing result, for example, to have a shorter cycle duration of the washing phase, a smaller amount of washing liquid, which is injected per unit of time during the washing phase, a lower nozzle pressure, with which the washing liquid is injected during the washing phase, a smaller amount of final rinse liquid, which is sprayed overall during the fresh water final rinse phase, and/or a lower temperature of the liquid (washing liquid, final rinse liquid) to be sprayed during the washing phase and the fresh water final rinse phase.
[0096] According to the invention, the program control device 50 is therefore designed in such a way that, depending on the determined wash item type/classification, it automatically controls the washing pump 8 in such a way that at least one of the parameters specified below is adapted to the detected wash item type for the washing phase: [0097] the cycle duration of the washing phase; [0098] the amount of washing liquid injected per unit of time during the washing phase; and [0099] the pressure with which the washing liquid is injected during the washing phase.
[0100] In
[0101] As already indicated, the schematically presented dishwashers 1 shown in
[0102] Moreover, the wash item recognition system has the already briefly mentioned analysis device 52 for analyzing the at least one recorded image in such a way that individual wash item parts or groups of individual wash item parts in the recorded image are located and classified. In particular, a single-shot MultiBox detector network architecture is used as a neural network during the analysis of the at least one recorded image.
[0103] During wash item recognition, it is provided in particular that the at least one two-dimensional image is recorded of the wash items to be treated in the dishwasher 1 in the treatment chamber 2 of the dishwasher 1 with the optical recognition system 51, preferably in a condition in which the treatment chamber 2 is suitably closed.
[0104] For this purpose, the optical recognition system 51 (camera) is assigned a lighting device also preferably arranged within the treatment chamber 2, in particular to illuminate at least some areas of the wash items accommodated in the treatment chamber 2 of the dishwasher 1 during image recording using the optical recognition system 51. The illumination is carried out, for example, by means of LED strips mounted on the sides or on the ceiling of the treatment chamber 2. In this way, particularly homogeneous and diffuse illumination of the wash items can be achieved. As a color temperature for illumination/lighting of the wash items, preferably a color temperature between 5,000 to 5,700 K and in particular a color temperature of about 5,100 K is used, since with a conventional camera as an optical recognition system, the best images can be recorded at this color temperature for subsequent processing with regard to the accuracy of the subsequent classification.
[0105] Preferably, in the dishwashers 1 shown in the drawings it is further provided that the image recording is carried out only after a (short) pre-wash of the wash items with liquid, which is preferably not heated, in order in this way to reduce or avoid water vapor condensation on the optical recognition system 51 or a lens of the camera.
[0106] Studies have shown that water vapor condensation can be sufficiently minimized even with a pre-wash of 0.5 to 2 seconds.
[0107] In short, the present invention is based on the knowledge that a neural network is surprisingly excellently suitable for the localization and classification of wash items sorted into wash baskets within a treatment chamber 2 of a dishwasher 1, in particular a hood dishwasher. The neural network is preferably a simplified single-shot MultiBox detector network architecture, which consists of 7 layers and is therefore sometimes referred to as “SSD7”.
[0108] For training the SSD7 network, between 500 and 2,500 images were recorded during investigations. These show wash baskets laden with crockery (different combinations of pots, pans, GN containers, cutlery, plates, bowls, cups, glasses and plastic containers). The analysis of the SSD7 network was carried out with a separate image data set consisting of 70 images, which was not used for the training. The wash baskets in the pictures were populated with dishes, which were not used for the training of the SSD7 net (for example, bowls and glasses with different shapes). This measure was intended to determine how well the network generalizes the training images.
[0109] It was able to be shown that with the SSD7 network optical crockery recognition in the form of localization and classification of the wash items is possible. The analysis shows a sufficient recognition rate for practical use with an accuracy and hit rate above 90%.
[0110] Furthermore, the invention is based on the knowledge that it is advantageous in terms of accuracy and reproducibility when image acquisition is carried out within the treatment chamber 2 with the treatment chamber 2 closed.
[0111] In order to reduce or even avoid water vapor condensation on the optical detection systems 51 (cameras), a pre-wash of, for example, 1 second duration before the image is taken is advantageous. With this pre-wash, the water vapor condensation on the camera can be reduced or minimized to such an extent that it does not interfere with the image acquisition.
[0112] The invention is not limited to the exemplary embodiments of the dishwasher or the optical wash item recognition system shown in the drawings, but results from an overview of all the features disclosed herein.