SYSTEM COMPRISING A DISHWASHER, METHOD FOR OPERATING A DISHWASHER, AND COMPUTER PROGRAM PRODUCT

Abstract

A system includes a dishwasher including a receptacle for items to be washed, a control apparatus configured to execute a wash program in dependence upon a loading of the receptacle with items to be washed and a camera configured to capture an image of at least one partial region of the receptacle in dependence upon a pull-out position of the receptacle. The captured image is stored by the control apparatus in dependence upon the pull-out position in a memory location of a number of memory locations, which memory location is assigned to the pull-out position, with each of the number of memory locations being assigned a set of pull-out positions. An image analysis unit determines the loading of the receptacle in dependence upon images that are stored in the memory locations.

Claims

1-15. (canceled)

16. A system, comprising: a dishwasher including a receptacle for items to be washed; a control apparatus configured to execute a wash program in dependence upon a loading of the receptacle with items to be washed; a camera configured to capture an image of at least one partial region of the receptacle in dependence upon a pull-out position of the receptacle, with the captured image being stored by the control apparatus in dependence upon the pull-out position in a memory location of a number of memory locations, which memory location is assigned to the pull-out position, with each of the number of memory locations being assigned a set of pull-out positions; and an image analysis unit configured to determine the loading of the receptacle in dependence upon images that are stored in the memory locations.

17. The system of claim 16, wherein the dishwasher is embodied as a household dishwasher.

18. The system of claim 16, wherein the control apparatus is configured to identify an image position of a feature of the receptacle in the captured image and to determine the pull-out position of the receptacle in dependence upon the identified image position of the feature of the receptacle.

19. The system of claim 18, wherein the control apparatus is configured to identify the image position of the feature of the receptacle in dependence upon a region of interest of the image.

20. The system of claim 18, wherein the feature of the receptacle comprises a handle of the receptacle for manually pulling out the receptacle and for manually pushing the receptacle into the dishwasher, and/or the feature of the receptacle comprises a marking that is arranged on the handle.

21. The system of claim 16, wherein the control apparatus is configured to store a most current image of the receptacle at a specific pull-out position in a memory location that is assigned to the specific pull-out position.

22. The system of claim 16, wherein each of the memory locations is assigned a specific interval from the set of possible pull-out positions of the receptacle.

23. The system of claim 22, wherein at least two memory locations are provided, with a first interval that is assigned to a first one of the at least two memory locations comprising a maximum pull-out position of the receptacle and having a first width, and with a second interval that is assigned to a second one of the at least two memory locations adjoining the first interval and having a second width, wherein the first width is narrower than the second width.

24. The system of claim 16, wherein the camera is configured to capture an image stream of the receptacle, wherein two temporally successive images in the image stream have a predetermined temporal spacing, said control apparatus configured to determine a movement of the receptacle in dependence upon the two successive images of the image stream and to store one of the two successive images in dependence upon the determined movement of the receptacle.

25. The system of claim 16, wherein the image analysis unit is configured to determine the loading of the receptacle in dependence upon a capturing time that is assigned to a respective one of the stored images and the pull-out positions that are assigned to the memory location, wherein a first one of the images, in which a specific partial region of the receptacle is visible and whose capturing time is more recent than the capturing time of a second one of the images in which the specific partial region of the receptacle is also visible is used to determine a loading for the specific partial region of the receptacle.

26. The system of claim 16, wherein the control apparatus is configured to delete an image whose capturing time is earlier than a predetermined threshold value.

27. The system of claim 16, wherein the control apparatus is configured to set the camera into a sleep mode at an end of a predetermined time interval in which a state of the receptacle is constant.

28. The system of claim 16, wherein the image analysis unit is arranged in a facility that is external to the dishwasher, with the control apparatus and the image analysis unit being configured to transmit data to each other.

29. The system of claim 16, wherein the image analysis unit is configured to determine a type, a quantity, an orientation, an arrangement and/or a material of the item to be washed that is arranged in the receptacle.

30. A method for operating a dishwasher, said method comprising: capturing an image of at least one partial region of a receptacle for items to be washed of the dishwasher in dependence upon a pull-out position of the receptacle; determining the pull-out position of the receptacle at a time of capturing the image; storing the image in a memory location of a number of memory locations that is assigned to the pull-out position, wherein each of the memory locations is assigned a set of pull-out positions; determining a loading of the receptacle in dependence upon images that are stored in the memory locations; and executing a wash program in dependence upon the determined loading.

31. The method of claim 30 for operating a household dishwasher.

32. A computer program product, embodied on a non-transitory computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform a method as set forth in claim 30.

Description

[0048] Further advantageous embodiments and aspects of the invention are the subject matter of the subordinate claims as well as the exemplary embodiments of the invention that are described below. In the following, the invention will be explained in more detail with the aid of preferred embodiments with reference to the accompanying figures.

[0049] FIG. 1 shows a schematic perspective view of an embodiment of a system comprising a household dishwasher;

[0050] FIG. 2 shows an exemplary image of a receptacle for items to be washed;

[0051] FIG. 3 shows a schematic side view of a system having a household dishwasher;

[0052] FIG. 4 shows a further schematic side view of a system having a household dishwasher;

[0053] FIG. 5 shows a schematic view of a receptacle for items to be washed with partial regions;

[0054] FIG. 6 shows a diagram of a time sequence;

[0055] FIG. 7 shows schematically a determined loading based on various images;

[0056] FIG. 8 shows a schematic block diagram of a further embodiment of a system; and

[0057] FIG. 9 shows a schematic block diagram of an exemplary embodiment of a method for operating a dishwasher.

[0058] In the figures, identical or functionally identical elements have been given the same reference characters unless otherwise indicated.

[0059] FIG. 1 shows a schematic perspective view of an embodiment of a system 100 having a dishwasher 1, which is designed as a household dishwasher. In this example, all components of the system 100 are integrated in the household dishwasher 1. The household dishwasher 1 comprises a dishwasher cavity 2 which can be closed by a door 3, in particular in a watertight manner. For this purpose, a sealing facility can be provided between the door 3 and the dishwasher cavity 2. The dishwasher cavity 2 is preferably cuboidal. The dishwasher cavity 2 can be arranged in a housing of the household dishwasher 1. The dishwasher cavity 2 and the door 3 can form a dishwasher interior 4 for washing items to be washed.

[0060] The door 3 is shown in FIG. 1 in its open position. By pivoting about a pivot axis 5 that is provided at a lower end of the door 3, the door 3 can be closed or opened. With the aid of the door 3, a loading opening 6 of the dishwasher cavity 2 can be closed or opened. The dishwasher cavity 2 has a floor 7, a ceiling 8 that is arranged opposite the floor 7, a rear wall 9 that is arranged opposite the closed door 3 and two side walls 10, 11 that are arranged opposite each other. The floor 7, the ceiling 8, the rear wall 9 and the side walls 10, 11 can, for example, be made of a stainless steel sheet. Alternatively, for example, the floor 7 can be made of a plastic material.

[0061] The household dishwasher 1 further comprises at least one receptacle 12 to 14 for items to be washed. It is preferred that multiple, for example three, receptacles 12 to 14 for items to be washed can be provided, wherein the receptacle 12 for items to be washed can be a lower receptacle for items to be washed or a lower basket, the receptacle 13 for items to be washed can be an upper receptacle for items to be washed or an upper basket, and the receptacle 14 for items to be washed can be a cutlery drawer. As FIG. 1 further shows, the receptacles 12 to 14 for items to be washed are arranged one above the other in the dishwasher cavity 2. Each receptacle 12 to 14 for items to be washed can be moved optionally into or out of the dishwasher cavity 2. In particular, each receptacle 12 to 14 for items to be washed can be pushed into or moved into the dishwasher cavity 2 in an insertion direction E and can be pulled out of or moved out of the dishwasher cavity 2 in an extraction direction A opposite to the insertion direction E.

[0062] A camera 20 is arranged at the front edge of the ceiling 8. The field of view FOV (see FIG. 3 or 4) of the camera 20 is designed in such a way that it can fully capture a fully pulled out receptacle 12 to 14 for items to be washed. A control apparatus 15 and an image analysis unit 25 are also arranged on the door 3. The control apparatus 15 comprises a number of memory locations (not shown) for storing images IMG that are captured by the camera 20 (see FIG. 2) and is configured so as to execute a wash program in dependence upon a determined loading of the household dishwasher 1. The image analysis unit 25 is configured so as to determine the loading in dependence upon images of the receptacles 12 to 14 for items to be washed that are captured by the camera 20.

[0063] In the following, the functions of the control apparatus 15, the camera 20 and the image analysis unit 25 are explained in detail.

[0064] FIG. 2 shows an exemplary image IMG of a receptacle 12 for items to be washed that has been captured, for example, by the camera 20 of FIG. 1. In the image IMG, the door 3 and the receptacle 12 for items to be washed, which is fully pulled out in this case, is visible. In addition, a handle 12A of the receptacle 12 for items to be washed is visible. For reasons of clarity, the receptacle 12 for items to be washed is shown here without items to be washed. Two coordinate axes X, Y are also shown. The image IMG has a number of pixels whose image position is uniquely determined based on the coordinates X, Y. The coordinates X1, X2, Y1, Y2 define a region of interest ROI in the image. The region of interest ROI is selected such that the handle 12A of the receptacle 12 for items to be washed moves in this region of the image IMG when the receptacle 12 for items to be washed is moved.

[0065] The control apparatus 15 (see FIG. 1) is configured so as to determine an image position of the handle 12A in the image IMG by analyzing the region of interest ROI. A respective image position or interval Z1-Z4 (see FIGS. 3-6) is assigned a memory location on which the captured image is stored. This is explained in detail below with reference to FIGS. 3-6.

[0066] FIG. 3 shows a schematic side view of a system 100 having a household dishwasher 1, for example the household dishwasher of FIG. 1. The door 3 is shown in the open position and the lower receptacle 12 for items to be washed is shown in the fully pulled out position (maximum pull-out position). The field of view FOV of the camera 20 is indicated in this side view. The field of view FOV preferably achieved a vertical angle of up to 120° (in the illustration of FIG. 3, the field of view FOV has a vertical angle of about 75°) and a horizontal angle of up to 180° (the horizontal angle is not visible in FIG. 3).

[0067] Below the door 3, a Y-axis is shown as an example, with a number of specific points Y1-Y5. The points Y1 and Y2, for example, correspond to those of FIG. 2. Two points each form an interval Z1-Z4, wherein the intervals Z1-Z4 do not overlap: Z1=(Y3; Y2], Z2=(Y4; Y3], Z3=(Y5; Y4], Z4=[Y1; Y5], The summation of the intervals Z1-Z4 includes all pull-out positions between Y1 and Y2. It should be noted that more than four intervals Z1-Z4 or fewer than intervals Z1-Z4 can also be provided. It is apparent that the position of the handle 12A of the receptacle 12 for items to be washed with respect to the Y-axis corresponds directly to the pull-out position of the receptacle 12 for items to be washed.

[0068] In FIG. 3, the receptacle 12 for items to be washed is in the maximum pull-out position and the handle 12A is in the interval Z1=(Y3; Y2], In FIG. 4, which shows the same system 100 as FIG. 3, the receptacle 12 for items to be washed is in a middle pull-out position and the handle 12A is in the interval Z3=(Y5; Y4).

[0069] Depending upon in which interval Z1-Z4 or at which image position the handle 12A is determined, it can be concluded which partial region W1-W4 (see FIG. 5) of the receptacle 12 for items to be washed is visible on the image IMG (see FIG. 2) of the camera 20.

[0070] It should be noted that the above information applies accordingly to the other receptacles 13 and 14 for items to be washed (see FIG. 1).

[0071] FIG. 5 shows a schematic view of a receptacle 12 for items to be washed with partial regions W1-W4. The partial regions W1-W4 show which part of the receptacle 12 for items to be washed is visible on an image IMG (see FIG. 2) when the handle 12A is in a respective interval Z1-Z4 (see FIG. 3, 4 or 6). When the handle 12A is in the interval Z1, the entire receptacle 12 for items to be washed is visible (partial region W1). When the handle 12A is in the interval Z2, the receptacle 12 for items to be washed is visible, for example, between 90%-50% (partial region W2), wherein the rearmost region of the receptacle 12 for items to be washed (the region opposite the handle 12A) is no longer visible. When the handle 12A is in the interval Z3, the receptacle 12 for items to be washed is visible, for example, between 50%-30% (partial region W3), wherein the rearmost half of the receptacle 12 for items to be washed is no longer visible. When the handle 12A is in the interval Z4, the receptacle 12 for items to be washed is visible, for example, between 30%-0% (partial region W4), wherein the rear half of the receptacle 12 for items to be washed is no longer visible.

[0072] It should be noted that the above percentage information is merely for illustrative purposes and is not to be interpreted in a restrictive manner. Which portions of the receptacle 12 for items to be washed are visible on a respective image IMG depends on the one hand on the design of the camera 20, in particular its field of view FOV (see FIG. 3 or 4), and on the other hand on the definition of the intervals Z1-Z4.

[0073] FIG. 6 shows a diagram of a time sequence before the start of a wash program. In the example of FIG. 6, it is assumed that four memory locations are provided for storing four images IMG for a respective receptacle 12 to 14 for items to be washed. Each memory location is assigned an interval Z1-Z4, for example as explained with reference to FIGS. 3 and 4. In further embodiments, more memory locations can be provided.

[0074] At an initial time tO, no image IMG is stored at any of the memory locations. At a time t1, a first image IMG1 is captured. The control apparatus 15 (see FIG. 1) determines that the image position of the handle 12A in the captured image IMG1 is in the interval Z3, for this reason the image IMG1 is stored in the memory location that is assigned to the interval Z3. At a time t2, a second image IMG2 is captured. The control device 15 determines that the image position of the handle 12A in the captured image IMG2 is in the interval Z1, for this reason the image IMG2 is stored in the memory location that is assigned to the interval Z1. At a time t3, a third image IMG3 is captured. The control apparatus 15 determines that the image position of the handle 12A in the captured image IMG3 is in the interval Z3, for this reason the image IMG3 is stored in the memory location that is assigned to the interval Z3. In this case, the previously stored image IMG1 is overwritten. At a time t4, a fourth image IMG4 is captured. The control apparatus 15 determines that the image position of the handle 12A in the captured image IMG4 is in the interval Z1, for this reason the image IMG4 is stored in the memory location that is assigned to the interval Z1. In this case, the previously stored image IMG2 is overwritten. At a time t5, a fifth image IMG5 is captured. The control apparatus 15 determines that the image position of the handle 12A in the captured image IMG5 is in the interval Z2, for this reason the image IMG5 is stored in the memory location that is assigned to the interval Z2.

[0075] It should be noted that the camera 20 (see FIG. 1, 3 or 4) captures and outputs an image stream, in other words, captures and outputs images at a predetermined frame rate. For example, the frame rate is between 1 FPS (FPS: frame per second) to 30 FPS, preferably between 10 FPS and 15 FPS. If the receptacle 12 for items to be washed remains at a specific pull-out position for a longer time, for example 10 seconds, for example with the handle 12A in the interval Z1, then the currently captured image is stored in each case. In the 10 seconds, a total of 100 images are captured at a frame rate of, for example, 10 FPS. Of these 100 images, only the most recently captured image is permanently stored. It is possible to provide that each of the images is initially stored and then overwritten by the following image, or the images are initially held in a buffer memory until the receptacle 122 for items to be washed is moved further and thus the most current image for the respective pull-out position is known.

[0076] After the time t5, the wash program is started. Then the loading of the receptacle 12 for items to be washed (and of the further receptacles 13, 14, if present), for items to be washed is determined first. If images with different time stamps, i.e. different actuality, are available for different partial regions W1-W4 (see FIG. 5), the respectively most recent image IMG is analyzed for a respective partial area W1-W4.

[0077] In the present example, three images IMG4, IMG5 and IMG3 are stored at this time. The most recent image IMG5 (time t5), which is stored in the memory location that is assigned to the interval Z2, shows, for example, a partial region W2 of the receptacle 12 for items to be washed. The rearmost region of the receptacle 12 for items to be washed is not visible in this image IMG5. The image IMG4 (time t4), which is stored in the memory location that is assigned to interval Z1, shows the entire receptacle 12 for items to be washed (partial region W1). The image IMG3 (time t3), which is stored in the memory location that is assigned to the interval Z3, shows a partial area W3 of the receptacle 12 for items to be washed. This partial region W3 is encompassed by both the partial region W2 and the partial region W1. Since the capturing time t3 of image IMG 3 is earlier than the capturing time of images IMG4 and IMG5, the image IMG3 is not analyzed because it has outdated information in comparison. The image IMG5 is used to determine the loading for the partial region W2 of the receptacle 12 for items to be washed and the image IMG4 is used to determine the loading for the partial region (W1-W2) not visible in the image IMG5, as explained below with reference to FIG. 7.

[0078] The loading is preferably determined by analyzing the entire image in each case and then extracting the relevant regions. It is also possible to extract the partial regions and combine them into one image, which is then analyzed, but this can result in artifacts and distortions because the images have been captured at different times, which can affect the analysis.

[0079] FIG. 7 schematically shows a result RES of a loading determination, wherein different images IMG4, IMG5 (see FIG. 6) have been used for different partial regions W2, W1-W2. The result RES comprises two partial results RES1 and RES2. The first partial result RES1 results from an analysis of the image IMG5 on which only the partial region W2 is visible. The part of the result of the analysis of the image IMG5 which does not refer to the receptacle 12 for items to be washed but refers to the door 3, for example, is discarded. The second partial result RES2 results from an analysis of the image IMG4. The parts of the result of the analysis of the image IMG4 that relate to the partial region W2 are discarded because there is a more recent image IMG4 for this partial region, and only the part of the result that relates to the region W1-W2 is used. The overall result RES is composed of the partial results RES1, RES2.

[0080] FIG. 8 shows a schematic block diagram of a further exemplary embodiment of a system 100. The system 100 comprises an image analysis unit 25 that is arranged externally to a dishwasher 1 (for example, the household dishwasher of FIG. 1 without the image analysis unit shown there). The dishwasher comprises a communication facility 16, by means of which the control device 15 (see FIG. 1) is configured so as to transmit images IMG to the image analysis unit 25 and to receive a result of a load determination RES from the image analysis unit 25. The image analysis unit 25 in turn has a corresponding communication device 26.

[0081] The communication facilities 16, 26 are designed, for example, as a modem, as a WLAN adapter, and/or as a mobile radio modem.

[0082] FIG. 9 shows a schematic block diagram of an exemplary embodiment of a method for operating a dishwasher 1, for example the household dishwasher of FIG. 1. In a first step S1, an image IMG (see FIG. 2) of at least one partial region W1-W4 (see FIG. 5) of a receptacle 12 for items to be washed (see FIGS. 1-4) of the dishwasher 1 is captured in dependence upon a pull-out position Z1-Z4 (see FIGS. 3-5) of the dish receptacle 12. In a second step S2, the pull-out position Z1-Z4 of the receptacle 12 for items to be washed is determined at the time t1-t5 (see FIG. 5) of image capturing. In a third step S3, the image IMG is stored in a memory location of a number of memory locations that is assigned to the pull-out position Z1-Z4, wherein each memory location of the number is assigned a set of pull-out positions Z1-Z4. In a fourth step S4, a loading RES (see FIG. 7 or 8) of the receptacle 12 for items to be washed is determined in dependence upon the images IMG that are stored in the memory locations. In a fifth step S5, a wash program is executed in dependence upon the determined loading RES.

[0083] The method can include various additional steps, such as, for example, determining an image position of a feature of the receptacle 12 for items to be washed, such as a handle 12A, in the captured image IMG, overwriting and/or deleting stored images IMG, discarding captured images IMG based on a detected movement of the receptacle 12 for items to be washed in the captured image IMG, and other steps of the like.

[0084] Although the present invention has been described with reference to exemplary embodiments, it can be modified in a variety of ways.

REFERENCE CHARACTERS USED

[0085] 1 Dishwasher [0086] 2 Dishwasher interior [0087] 3 Door [0088] 4 Wash cavity [0089] 5 Pivot axis [0090] 6 Loading opening [0091] 7 Floor [0092] 8 Ceiling [0093] 9 Rear wall [0094] 10 Side wall [0095] 11 Side wall [0096] 12 Receptacle for items to be washed [0097] 12A Feature [0098] 13 Receptacle for items to be washed [0099] 14 Receptacle for items to be washed [0100] 15 Control apparatus [0101] 16 Communication facility [0102] 20 Camera [0103] 25 Image analysis unit [0104] 26 Communication facility [0105] 100 System [0106] A Pull-out direction [0107] E Push-in direction [0108] FOV Field of view [0109] IMG Image [0110] IMG1 Image [0111] IMG2 Image [0112] IMG3 Image [0113] IMG4 Image [0114] IMG5 Image [0115] RES Loading [0116] RES1 Partial loading [0117] RES2 Partial loading [0118] ROI Region of interest [0119] t Time axis [0120] tO Time [0121] t1 Time [0122] t2 Time [0123] t3 Time [0124] t4 Time [0125] t5 Time [0126] W1 Partial region [0127] W1-W2 Region [0128] W2 Partial region [0129] W3 Partial region [0130] W4 Partial region [0131] X Coordinate axis [0132] XI Position [0133] X2 Position [0134] Y Coordinate axis [0135] Y1 Position [0136] Y2 Position [0137] Y3 Position [0138] Y4 Position [0139] Y5 Position [0140] Z1 Interval [0141] Z2 Interval [0142] Z3 Interval [0143] Z4 Interval