FITTING WITH AUTOMATIC OBJECT RECOGNITION AND METHOD FOR CONTROLLING A FITTING BY MEANS OF AUTOMATIC OBJECT RECOGNITION

Abstract

A fitting, in particular an outlet fitting, for a sanitary installation having a control unit and a sanitary element controllable with the control unit is disclosed. The fitting has at least one imaging sensor. The control unit is adapted to trigger an action of the sanitary element depending on an object recognition based on at least one output signal from the at least one imaging sensor. A fitting arrangement having a fitting as well as a method for controlling a fitting is also disclosed.

Claims

1. A fitting (1) for a sanitary installation (2), comprising a control unit (3) and a sanitary element (4) controllable with the control unit (3), characterized in that the fitting (1) comprises at least one imaging sensor (5), wherein the control unit (3) is adapted to trigger an action of the sanitary element (4) depending on an object recognition based on at least one output signal from the at least one imaging sensor (5).

2. The fitting (1) according to claim 1, characterized in that the at least one imaging sensor (5) is a 2D or 3D camera, a thermal imaging camera, an ultrasonic sensor, a radar sensor or a laser distance measuring sensor, in particular a LIDAR or ToF sensor or a laser scanner, or a combination of said sensor types.

3. The fitting (1) according to claim 1, characterized in that the fitting (1) additionally comprises an illumination unit (6) for illuminating objects (7), wherein the illumination unit (6) comprises in particular one or more LEDs or laser diodes or a UV or IR light source.

4. The fitting (1) according to claim 1, characterized in that the fitting (1) additionally comprises an image processing unit (8) to perform the object recognition.

5. The fitting (1) according to claim 1, characterized in that the fitting (1) additionally comprises a communication unit (9) for sending the at least one output signal from the at least one imaging sensor (5) to a remote image processing unit (8), e.g. a server, in particular in the cloud (10), for object recognition and for receiving a result of the object recognition.

6. The fitting (1) according to claim 1, characterized in that the image processing unit (8) is adapted to perform the object recognition by means of a neural network, in particular by means of a neural network trained before the fitting (1) is put into operation.

7. The fitting (1) according to claim 1, characterized in that the image processing unit (8) is adapted to perform an object classification as part of the object recognition in order to assign an object (7) to a specific class from a plurality of predefined classes.

8. The fitting (1) according to claim 7, characterized in that by means of the object classification different classes of kitchen utensils, such as plates, glasses, cups, cutlery, cooking pots, pans, etc., or of limbs, such as hands, arms, feet or legs, of a user of the fitting or of cleaning utensils, such as a cleaning brush, a cleaning sponge, steel wool or a cleaning cloth, can be recognized.

9. The fitting (1) according to claim 1, characterized in that the image processing unit (8) is adapted to determine at least one property of an object (7), such as transparency, color, size or degree of soiling.

10. The fitting (1) according to claim 1, characterized in that the image processing unit (8) is adapted to determine a position, in particular relative to a reference position, and/or a movement of an object (7).

11. The fitting (1) according to claim 7, characterized in that the control unit (3) is adapted to trigger a specific action of the sanitary element (4) depending on the object classification and/or the property of the object (7) and/or the position of the object (7) and/or the movement of the object (7).

12. The fitting (1) according to claim 11, characterized in that one or more of the following actions are triggerable: selecting a preset of the sanitary element (4) and executing a behavior of the sanitary element (4) defined according to the preset; dispensing a specified amount of a fluid (11); dispensing a specified amount of a fluid (11) per unit of time; dispensing a fluid (11) with a preset maximum, minimum or preferred temperature of the fluid (11) to be dispensed; dispensing a specific fluid (11) from a plurality of different fluids, in particular depending on a position of the object (7); switching on and/or off the delivery of a fluid (11).

13. A fitting arrangement (12), comprising a sanitary installation (2) and a fitting (1) according to claim 1, wherein the imaging sensor (5) is arranged in such a way that an inlet region (13), in particular for dispensing a fluid (11), and/or an outlet region (14), in particular for leading away the dispensed fluid (11), of the fitting (1) is/are detectable by the imaging sensor (8).

14. The fitting arrangement (12) according to claim 13, wherein the sanitary installation (2) is a sink or washtub, bidet, shower, bathtub, soap dispenser, lotion dispenser, sanitizer dispenser, hand dryer, hair dryer, toilet, shower toilet, urinal, or wash station.

15. A method for triggering an action of a sanitary element (4) of a fitting (1) according to claim 1, comprising the steps of: detecting a scene at the sanitary element (4) with at least one imaging sensor (5), in particular in an inlet region (13) and/or an outlet region (14) of the fitting (1), wherein the scene is detected by means of one or more images as output of the at least one imaging sensor (8), wherein in particular in the case of several images these are recorded staggered in time, e.g. as a film or video, or from different viewing angles; performing object recognition for recognizing at least one object (7), such as a hand of a user (15), of the fitting (1), a kitchen utensil, or a cleaning utensil; and triggering an action of the sanitary element (4) depending on the object recognition.

16. The method according to claim 15, characterized in that the scene is illuminated by means of an illumination unit (6), in particular one or more LEDs or laser diodes or a UV or IR light source.

17. The method according to claim 15, characterized in that the scene detected with the at least one imaging sensor (5) is sent as an output of the at least one imaging sensor (5) to a remote image processing unit (8), e.g. a server, in particular in the cloud (10), for object recognition, and a result of the object recognition is received from the remote image processing unit (8).

18. The method according to claim 15, characterized in that the object recognition is performed by means of a neural network, in particular by means of a neural network trained before the fitting (1) is put into operation.

19. The method according to claim 15, characterized in that as part of the object recognition, an object classification is performed to assign an object (7) to a particular class from a plurality of predefined classes.

20. The method according to claim 15, characterized in that different classes of kitchen utensils, such as plates, glasses, cups, cutlery, cooking pots, pans, etc., or of limbs, such as hands, arm, feet and legs, of a user of the fitting or of cleaning utensils, such as a cleaning brush, a cleaning sponge, steel wool or a cleaning cloth, are recognized by means of the object classification.

21. The method according to claim 15, characterized in that at least one property of an object (7), such as transparency, color, size or degree of contamination, is determined.

22. The method according to claim 15, characterized in that a position, in particular relative to a reference position, and/or a movement of an object (7) is determined.

23. The method according to claim 20, characterized in that a specific action of the sanitary element (4) is triggered depending on the object classification and/or the property of the object (7) and/or the position of the object (7) and/or the movement of the object (7).

24. The method according to claim 23, characterized in that the action comprises at least one of the following: selecting a preset of the sanitary element (4) and executing a behavior of the sanitary element (4) defined according to the preset; dispensing a specified amount of a fluid (11); dispensing a specified amount of a fluid (11) per unit of time; dispensing a fluid (11) with a preset maximum, minimum or preferred temperature of the fluid (11) to be dispensed; dispensing a specific fluid (11) from a plurality of different fluids, in particular depending on a position of the object (7); switching on and/or off the delivery of a fluid (11).

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Non-limiting exemplary embodiments of the present invention are explained in further detail below with reference to figures, wherein:

[0059] FIG. 1 shows a schematic representation of a first exemplary embodiment of a fitting for a sanitary installation according to the invention;

[0060] FIG. 2a) shows a representation in a side view of a second exemplary embodiment of a fitting according to the invention for a sanitary installation, such as a wash basin or sink;

[0061] FIG. 2b) shows a representation in a bottom view of the second exemplary embodiment;

[0062] FIG. 3 shows a schematic representation of a third exemplary embodiment of a fitting according to the invention for a sanitary installation, such as a washbasin in a bathroom or public washroom;

[0063] FIG. 4 shows a schematic representation of a fourth exemplary embodiment of a fitting according to the invention for a sanitary installation, such as a washbasin in a bathroom or public washroom;

[0064] FIG. 5 shows a representation of the second exemplary embodiment illustrating a use case for hand washing;

[0065] FIG. 6 shows a representation of the second exemplary embodiment illustrating a use case for washing dirty dishes;

[0066] FIG. 7 shows a representation of the second exemplary embodiment illustrating a use case for filling a tea cup with hot water;

[0067] FIG. 8 shows a representation of the second exemplary embodiment illustrating a use case for filling a measuring cup with cold water;

[0068] FIG. 9 shows a schematic representation of a fifth exemplary embodiment of a fitting according to the invention for a sanitary installation, such as a shower in a bathroom or public shower room; and

[0069] FIG. 10 shows a schematic representation of a sixth exemplary embodiment of a fitting according to the invention for a sanitary installation with two outlets for two different liquids.

[0070] In the figures, the same reference numerals stand for the same elements.

DETAILED DESCRIPTION OF THE INVENTION

[0071] In order to explain the basic principle of the present invention, FIG. 1 shows in a schematic representation a first simple exemplary embodiment of a fitting 1 according to the invention for a sanitary installation, such as a washbasin for a bathroom or a sink for a kitchen. In the present case, the fitting 1 is by way of example a spout fitting (also called a faucet). The fitting 1 comprises a sanitary element 4 in the form of a pipe with an outlet 17 for a liquid 11, for example water. The flow rate of the liquid 11 exiting the outlet 17 is regulated by a valve (not shown in FIG. 1). The valve in turn is adjusted by a control unit 3. Normally, the valve is controlled by a manual control element which has to be operated by a user. Alternatively, the valve is triggered automatically, for example by means of a proximity sensor, such as an infrared/IR sensor. According to the present invention, an imaging sensor 5, such as a camera, is now used to trigger the valve via the control unit 3. For this purpose, the output signal of the imaging sensor 5, e.g. one (or more) 2D image(s) or a film or video (i.e. a temporal sequence of images) is analyzed and an object recognition is performed, for example to detect the presence of hands 7 under the outlet 17. Depending on the result of the object recognition, the valve is opened via the control unit 3 to allow the user to wash the hands 7. As soon as the user leaves the hands 7 from the inlet region 13, which is detected and monitored by the imaging sensor 5, this is detected by the object recognition system and the valve is closed again via the control unit 3 so that no more liquid 11 escapes from the outlet 17.

[0072] The advantage of object recognition is now that different actions can be performed depending on the detected object and its movement(s). For example, the flow rate of water can be controlled by the imaging sensor 5 depending on the distance of the object 7 (e.g., the hands)—e.g., the further away the hands are from the camera 5, the more water is dispensed per unit time. In addition, for example, in the case of a mixing faucet, i.e., a combined hot- and cold-water outlet valve, the water temperature can be adjusted depending on a movement of the object 7, for example—e.g., a movement/shift of the hands 7 to the right leads to colder water and a movement/shift of the hands 7 to the left leads to warmer water.

[0073] FIG. 2a) shows a side view and FIG. 2b) shows a bottom view of a second exemplary embodiment of a fitting 1 according to the invention for a sanitary installation, such as a wash basin or sink. As in the example according to FIG. 1, the imaging sensor 5 (e.g. camera) is also arranged here on the outlet pipe near the outlet 17 of the sanitary element 4—in FIGS. 2a) and 2b) above the outlet 17 and in FIG. 1 below the outlet 17, so that the inlet region 13 can be completely covered by the imaging sensor 5 (e.g. camera).

[0074] FIG. 3 shows a schematic representation of a third exemplary embodiment of a fitting 1 according to the invention for a sanitary installation, such as a washbasin 2 in a bathroom. The signal (e.g. image or film or video) recorded by the imaging sensor 5 (e.g. camera) is transmitted to an image processing unit 8 for object recognition. The image processing unit 8 may be located directly at the imaging sensor 5 (e.g., in a common electronic module) or it may be a separate unit located, e.g., remote from the outlet tube. The signal/data transmission between the imaging sensor 5 (e.g. camera) and the image processing unit 8 can be via cable or wireless (shown as “radio flash” in FIG. 3). For example, the image processing unit 8 can be located at the control unit 3 (e.g., in a common electronic module or both are implemented by a common processor). Alternatively, however, the object recognition can also be performed, for example, in a remote server, such as a cloud server 10. The connection there can be partly wireless (e.g. via IR, Bluetooth, WLAN, mobile radio) or also wired (e.g. via Controller Area Network (CAN) bus, Ethernet), in particular via Internet. To enable wireless connection of the imaging sensor 5 (e.g. camera), the fitting 1 additionally comprises a wireless communication unit 9 (transmitter/receiver, transceiver).

[0075] FIG. 3 shows a schematic representation of a fourth exemplary embodiment of a fitting 1 according to the invention for a sanitary installation, such as a washbasin 2 in a bathroom. Compared to the third exemplary embodiment according to FIG. 3, here the imaging sensor 5 (e.g. camera) is not arranged on the outlet pipe 4, but is mounted on a wall and oriented in the direction of the inlet region 13 (as well as the outlet region 14). In addition, the fitting 1 comprises an illumination unit 6, which is arranged on the outlet pipe 4. This serves to illuminate the inlet region 13 (as well as the outlet region 14) so that the imaging sensor 5 can record an image with as high a contrast as possible, which facilitates object recognition. The light emitted by the illumination unit 6 is adapted to the detection area of the imaging sensor 5. However, the illumination unit 6 can also serve as working illumination for the user as well as, for example, as a display/indicator—e.g. red light means hot water and blue light means cold water, and a flashing light indicates a malfunction.

[0076] For example, as part of object recognition, object classification is performed to assign an object to a particular class from a variety of predefined classes. Each object class comprises a certain type of object, such as kitchen utensils (e.g. plates, glasses, cups, cutlery, cooking pots, pans, etc.) or limbs (e.g. hands, fingers, arms, feet or legs) of a user of the fitting or cleaning utensils (e.g. cleaning brush, cleaning sponge, steel wool or cleaning cloth). Depending on the detected object, various actions of the sanitary element can then be triggered, i.e., each object (or each type of object) is assigned a specific action.

[0077] Object recognition can be carried out in particular by means of a neural net or network. The neural network was trained for this purpose before the fitting was put into operation (e.g. by the manufacturer of the fitting). This means that the settings of the neural network were determined e.g. with the help of training data. The training data for training the neural network consist of output signals/data of the imaging sensor 5 as well as an assignment to a specified (known) object (called “data labeling” in the field of machine learning). The training of the neural network is performed offline as mentioned above and is typically carried out using powerful computers (e.g. in the cloud) and specialized software tools. Local object recognition (e.g., by “inference” using the neural network) at the fitting has the advantage that the recorded images do not have to be transmitted to an external server, which is preferred especially for data protection reasons as well as to preserve privacy. It is conceivable that the user records training data for new objects to be recognized by means of the imaging sensor and transmits these to an (automatic) service, which sends back to the user new settings (e.g. in the form of a “TensorFlow Lite” model) for the neural network or new firmware for object recognition or classification.

[0078] FIGS. 5 to 8 now show how various objects can be detected with the aid of the fitting according to the embodiment variant shown in FIG. 2, which then lead to correspondingly assigned actions of the sanitary element.

[0079] In FIG. 5, a user holds his hands 7 under the water faucet 4. As soon as the camera 5 detects the hands 7 in the inlet region 13, the water 11 is turned on. When the user removes his hands 7 from the inlet region 13, the water 11 is turned off again. Hand movements (e.g., certain gestures) can also be used to trigger various additional actions, such as setting water warmer/cooler, increasing/decreasing the strength of the water delivery, turning the shower on/off. Gesture evaluation recognizes certain movement patterns (direction/speed) in particular. For example, opening the hand/hands (from first to outstretched fingers) can trigger the action of dispensing a portion of soap or disinfectant.

[0080] In FIG. 6, a user holds a dirty plate 7 under the faucet 4. As soon as the camera 5 detects the dirty plate 7 in the inlet region 13, (instead of cold) warm/hot water 11 is switched on. Additionally, dishwashing detergent can be added to the water 11—especially if the plate 7 is detected as (heavily) soiled, because the object recognition also allows to determine properties of an object, such as the degree of soiling. Again, the water 11 is switched off as soon as the user removes the (now clean) plate 7 from the inlet region 13.

[0081] In FIG. 7, a user holds a tea cup 7 under the faucet 4. As soon as the camera 5 detects the tea cup 7 in the inlet region 13, hot/boiling water 11 is turned on. The object recognition also makes it possible to determine properties of the object, such as its transparency, color and/or size. On the one hand, this makes it possible to distinguish between cups of different sizes and colors, e.g. made of glass or ceramic, so that the corresponding amount of hot/boiling water is also filled. The filling can also be interrupted by pulling the cup 7 away from the water jet. As soon as the fitting 1 registers that the distance between the water jet and the cup rim changes or that this distance falls below a certain value, the water 11 is automatically switched off. For safety reasons, the boiling water 11 can also be switched off immediately as soon as a user's hand is detected in the inlet region 13 (->emergency shutdown safety function).

[0082] In FIG. 8, a user holds a measuring cup 7 under the water faucet 4. As soon as the camera 5 detects the measuring cup 7 in the inlet region 13, cold water 11 is switched on. In addition, the user can indicate how much the measuring cup 7 should be filled with water 11 (e.g., one-quarter full, one-half full, three-quarters full, or completely full) by holding out his hand with, for example, 1 to 4 fingers extended. If the user instead holds a cooking pot with both hands under the water tap 4 in the inlet region 13, this is filled with water 11 until the cooking pot is pulled away from the water jet again.

[0083] FIG. 9 shows an application of the fitting 1 according to the invention in a shower 2. As soon as the camera 5 recognizes a person 15 or, in particular his head or upper body 7 under the shower in the inlet region 13, water 11 is let run. The person 15 can adjust the temperature as well as the strength of the water jet, for example, by hand gestures. It is also possible to adjust the position (e.g. height above the head) and/or the orientation of the shower head to the person and, if necessary, also to track them.

[0084] Analogous to the embodiments for the application for a shower, the fitting according to the invention can also be used for a hand or hair dryer mounted on a wall, wherein the air flow can be switched on and off and the temperature and strength (as well as direction) can be regulated depending on the object recognition.

[0085] FIG. 10 shows a schematic representation of a sixth exemplary embodiment of a fitting 1 according to the invention for a sanitary installation. In an extension of the first exemplary embodiment according to FIG. 1, this variant has two liquid outlets 17, 17′ for two different liquids 11, 11′, wherein the first liquid 11 can be, for example, water and the second liquid 11′ can be, for example, soap or an (alcoholic) hand disinfectant. If the user holds the hands 7 in the inlet region 13, which is detected and monitored by the imaging sensor 5, under the first outlet 17 at the position P1, this is detected by the object recognition system and the valve for the first liquid 11 is opened via the control unit 3, so that water 11 flows, for example, to wash the hands 7. If the user now moves the hands 7 within the inlet region 13 from the position P1 to the position P2 below the second outlet 17′, this is detected by the image processing unit, whereupon the control unit 3 immediately closes the valve for the first liquid 11 and briefly opens the valve for the second liquid 11′, e.g. to dispense a specified amount of soap or disinfectant. The second liquid 11′ can also be dispensed, for example, via a second pipe with the second liquid outlet 17′, which is arranged, for example, laterally (and towards the rear) offset with respect to the pipe for the first liquid 11. The image processing unit is thereby able to determine the positions P1 and P2 of the object 7 (e.g. hands), and the control unit 3 performs different actions depending on the respective position P1, P2 of the object 7 (e.g. hands).

[0086] If a certain (new, previously unknown) object is to be recognized, corresponding images (or film or videos) can be taken by the user as training data with the camera of the fitting (or with another camera, e.g. of a smartphone). This data can be transmitted to a server, as mentioned above, which then returns an appropriately trained neural network to the fitting (e.g., in the form of a “TensorFlow Lite” model). The assignment of a certain object (or a gesture concerning this object) to a certain action can be done or changed by the user himself, for example by means of a web app or an app on his smartphone.

[0087] Further areas of application of the fitting according to the invention are conceivable, for example, in beverage dispensers or automatic washing systems. Depending on the detected container (large, small, transparent, of a certain color), different beverages are dispensed in the beverage vending machines, e.g. pure water, sparkling water, tea, coffee, soup, etc.

[0088] The proposed fitting can be used both in the private sector and in public sanitary facilities, although for the private sector application a much higher degree of personalization is possible, as well as a greater variety of different actions. For the public sector, however, the proposed fitting also has the advantage that an installer, plumber or service technician can configure the system without a control element simply by using the built-in imaging sensor (e.g. camera), and does not need a special configuration/programming device to do so.

LIST OF REFERENCE NUMERALS

[0089] 1 Fitting, outlet fitting, water faucet, shower inlet fitting [0090] 2 Sanitary installation, sink, shower [0091] 3 Control unit, processor [0092] 4 Sanitary element, (outlet) pipe with valve [0093] 5 Imaging sensor, (2D/3D) camera [0094] 6 Illumination unit, display/indicator [0095] 7 Object, hands, dishes, kitchen utensils, head [0096] 8 Image processing unit, processor [0097] 9 Communication unit, transmitter/receiver (transceiver) [0098] 10 (Cloud) server [0099] 11, 11′ Liquid (water, disinfectant, soap) [0100] 12 Fitting arrangement [0101] 13 Inlet region [0102] 14 Outlet region [0103] 15 User [0104] 16 Valve [0105] 17, 17′ (Liquid) outlet [0106] P1, P2 Position