METHOD AND DEVICE FOR DETECTING SMOKE

Abstract

The invention relates to a machine learning system (10) which is configured, on the basis of a plurality of images captured in succession, to detect smoke (12) within the images. The machine learning system (10) comprises a convolutional recurrent neural network. The invention also relates to a method for detecting smoke by means of this machine learning system.

Claims

1. A machine learning system (10) comprising: a convolutional recurrent neural network and wherein the machine learning system is parameterized, on the basis of successive images of a sequence of images, to detect smoke (12) in the images

2. The machine learning system (10) as claimed in claim 1, which is further parameterized to output an output variable that characterizes whether smoke (12) is represented within the images.

3. The machine learning system (10) as claimed in claim 1 which is parameterized, on the basis of each of the individual images, to output a matrix as the output variable, wherein the elements of the matrix are each assigned to a section of a predetermined plurality of sections of the respective image, and wherein the elements of the matrix characterize whether smoke (12) is represented within this assigned section of the respective image.

4. The machine learning system (10) as claimed in claim 1, which comprises a plurality of concatenated convolutional LSTM modules (20), wherein a number of filters that the respective convolutional LSTM module (20) comprises increases with increasing depth of a position of the respective convolutional LSTM module, wherein a pooling layer is positioned between each of the convolutional LSTM modules (20), wherein the final convolutional LSTM module (20) of the plurality of the concatenated convolutional LSTM modules (20) is connected to a fully connected neural network, which is parameterized to output the output variable of the machine learning system (10).

5. The machine learning system (10) as claimed in claim 1, further comprising vith an input, wherein the input is configured to provide the plurality of images captured by a camera to the machine learning system (10).

6. A method for detecting smoke (12) within an image by means of a machine learning system (10), which is parameterized, on the basis of a plurality of images captured in direct succession to detect smoke (12) in the images, wherein the machine learning system (10) comprises a plurality of layers connected in a specified sequence and at least one of the layers comprises a convolutional recurrent neural network, said method comprising the following steps: obtaining a plurality of images captured in direct succession; propagating the plurality of the captured images through the machine learning system (10) in succession, wherein during the propagation, the images are processed successively by the layers of the machine learning system (10) and the final layer of the sequence of layers outputs the output variable.

7. The method as claimed in claim 6, wherein the machine learning system (10) is trained on the basis of a plurality of training data, comprising training images (x) and respectively assigned training output variables (y.sub.s), wherein the training images (x) are propagated through the machine learning system (10) during the training and, on the basis of the determined output variables of the machine learning system (10) and the respectively assigned training output variables of the training images, a parameterization of the machine learning system (10) is adjusted in such a way that a deviation between the determined output variables and the training output variables becomes a minimum.

8. The method as claimed in claim 6, wherein a smoke detector is activated on the basis of the determined output variable of the machine learning system (10).

9. A non-transitory, computer-readable medium that contains instructions that when executed on a computer cause said computer to detect smoke (12) within an image by obtaining a plurality of images captured in direct succession; and propagating the plurality of the captured images through a machine learning system (10) in succession, wherein the machine learning system (10) comprises a plurality of layers connected in a specified sequence and at least one of the layers comprises a convolutional recurrent neural network, and wherein during the propagation, the images are processed successively by the layers of the machine learning system (10) and the final layer of the sequence of layers outputs an output variable.

10. (canceled)

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In the following, exemplary embodiments are described in more detail by reference to the accompanying drawings. In the drawings:

[0039] FIG. 1 shows a schematic structure of a device for detecting smoke;

[0040] FIG. 2 shows a schematic drawing of a machine learning system which is parameterized to detect smoke;

[0041] FIG. 3 shows a schematic drawing of a flowchart of a method for detecting smoke by means of the machine learning system;

[0042] FIG. 4 shows a schematic drawing of a device which can be used for training the machine learning system.

DETAILED DESCRIPTION

[0043] FIG. 1 shows an exemplary embodiment of a device for detecting smoke (12). The device in this embodiment comprises a camera (11) which is connected to a machine learning system (10). The camera (11) captures a plurality of images from an environment of the camera (11). For example, if smoke develops in the area surrounding the camera (11) because an object (13) is burning or overheating, the camera images will contain smoke (12). The plurality of images from the camera are processed in succession by the machine learning system (10). The machine learning system (10) is parameterized in such a way that, on the basis of the images processed in succession, it outputs an output variable which characterizes whether smoke (12) is represented within the respective image. On the basis of one or a plurality of the output variables of the machine learning system, the smoke can be detected.

[0044] In a further embodiment of the device, the machine learning system (10) is connected to a smoke detector, for example. The smoke detector determines whether or not the machine learning system should trigger an alarm, depending on the output variable of the machine learning system.

[0045] FIG. 2 shows a schematic structure of the machine learning system (10). In FIG. 2, the machine learning system (10) is provided by a neural network as an example. In the embodiment illustrated in FIG. 2, the machine learning system (10) comprises a plurality of layers which are connected to each other in a predetermined sequence. The layers are either ConvLSTM (20) or pooling layers, which perform e.g. an average/max-pooling. The sequence shown in FIG. 2 is such that a pooling layer follows each ConvLSTM (20). In addition, the final layer of the machine learning system (10) can be connected to a fully connected neural network (21). Preferably, the fully connected neural network (21) is a classifier that outputs a classification as to whether the image processed by the machine learning system (10) contains smoke. Alternatively, the fully connected neural network can output a matrix, each of the elements of which is assigned to a section of a plurality of sections of the processed image and the elements each characterize whether smoke is represented within the respective section. This then corresponds to a localization of the smoke within the image.

[0046] FIG. 2 also indicates examples of the resolutions of the input variables of the layers, shown above the respective layers. The input variable (FIG. 2: “input frame”) of the machine learning system (10), i.e. the image processed by the machine learning system (10), has a resolution of 256×256×3 in this embodiment. An example of how the respective layers are configured is also shown. For example, the first ConvLSTM layer is configured by 4 filters, each having a resolution of 3×3.

[0047] FIG. 2 also shows a schematic structure of the ConvLSTM (20).

[0048] The ConvLSTM (20) receives three different input variables (x,h,c). Based on these three input variables (x,h,c), the ConvLSTM (20) determines two output variables (c′,h′). A first input variable (x) is the input variable of the respective ConvLSTM layer, or the image. A second input variable (h) is a previously determined output variable of the ConvLSTM (20) at a processing/time step, in particular an immediately preceding one. This means that this determined output variable contains information from the previous processed input variables of this ConvLSTM. A third input variable (c) is an internal state of the ConvLSTM (20), which is updated depending on the first and second input variables (x,h), in particular at each processing/time step.

[0049] In order to determine the two output variables (h′,c′), the first and second input variables are filtered by means of different filters (fh,fx,ch,cx,ih,ix,oh,ox) and summed according to the combinations shown in FIG. 2. An activation function is then applied to the respective summed results. The activation function is formed by way of example as a sigmoid/tanh function in FIG. 2.

[0050] The results of the activation functions are then processed according to the combinations shown in FIG. 2 in a so-called input gate, forget gate and output gate to produce the output variables (c′,h′).

[0051] FIG. 3 shows a schematic flowchart (30) of a method for detecting smoke.

[0052] The method starts at step 300. In this step, the machine learning system (10) receives the plurality of the images captured from the camera (11) in succession.

[0053] In the following step 310, the individual images are propagated through the machine learning system (10) in succession.

[0054] In step 320, the machine learning system (10) outputs an output variable after each of the images has been propagated through the machine learning system.

[0055] In the following step 330 it can then be decided, based on the output values output by the machine learning system (10), whether smoke is present in the area surrounding the camera. This can be carried out, for example, by comparing the output variable of the machine learning system with a threshold value. If the output variable is greater than the threshold value, the decision can be made that smoke is present. Optionally, step 340 can be performed after completing step 330. Depending on the result of step 330, a smoke detector is activated there. For example, the smoke detector can issue a warning signal such as a warning tone if the output variable of the machine learning system has exceeded the threshold value.

[0056] In a further embodiment of the method for detecting smoke, the machine learning system (10) can be trained before step 300 is executed.

[0057] When training the machine learning system (10), an optimization method, preferably a gradient descent method such as backpropagation-through-time, is used to optimize the parameterization of the machine learning system with respect to a loss function. The loss function characterizes a difference between determined output variables and supplied training output variables based on the parameterization. To optimize the parameterization, gradients can be determined using the gradient descent method, the parameterization then being adjusted according to the gradients determined.

[0058] Training data containing training images is provided for training purposes, and the machine learning system determines the output values based on these. Furthermore, the training output variables are assigned to the training images.

[0059] The training step can be executed several times in succession until a predefined abort criterion is met, for example until the difference or a change in this difference is less than a predefined value.

[0060] FIG. 4 shows a schematic drawing of a device (40) for training the machine learning system (10). The device (40) comprises a database (41) which contains training data. The training data comprises labeled images (x), the labels (y.sub.s) of which characterize whether smoke is represented within the image. The images (x) are processed by the machine learning system (10) and provided to a difference module (42) as an output variable (y). The difference module (42) also receives the labels (y.sub.s) and determines a difference based on the labels (ys) and the output variables (y), which is then passed on to the adjustment module (43). Depending on the difference, the adaptation module then determines a change (θ′) in the parameterization (θ) of the machine learning system (10). The parameterization (θ) is then adjusted in a memory (P) based on the change (θ′).

[0061] Furthermore, the device (40) can comprise a computing unit (44) and a storage element (45).