COMPUTER-IMPLEMENTED METHOD FOR PROVIDING AN OUTPUT DATA SET

Abstract

A method for providing an output data set as a function of an input data set includes applying an algorithm for reducing image noise to the input data set or an intermediate data set determined as a function of the input data set to determine a noise-reduced signal data set. The method includes determining a noise data set as difference between the input data set or the intermediate data set and the signal data set. The method includes determining a modified noise data set by applying a noise-processing algorithm to the noise data set and/or determining a modified signal data set by applying a signal processing algorithm to the signal data set. The method includes determining an output data set by adding the modified noise data set to the signal data set or the modified signal data set, or adding the noise data set to the modified signal data set.

Claims

1. A computer-implemented method for providing an output data set as a function of an input data set that is determined by a medical imaging method, the computer-implemented method comprising: determining a noise-reduced signal data set, the determining of the noise-reduced signal data set comprising applying an algorithm for reducing image noise to the input data set or an intermediate data set determined as a function of the input data set; determining a noise data set as a difference between the input data set or the intermediate data set and the noise-reduced signal data set; determining a modified noise data set, wherein the determining of the modified noise data set comprises applying a noise-processing algorithm to the noise data set, determining a modified signal data set, wherein determining the modified signal data set comprises applying a signal processing algorithm to the signal data set, or a combination thereof; and determining an output data set, determining the output data set comprising adding the modified noise data set to the noise-reduced signal data set or the modified signal data set or adding the noise data set to the modified signal data set.

2. The computer-implemented method of claim 1, wherein the noise-processing algorithm includes scaling of the noise data set, different scaling of different frequency components of the noise data set, or filtering of the noise data set.

3. The computer-implemented method of claim 2, further comprising determining at least one measure for a noise in the input data set, the intermediate data set, or the noise data set, wherein the noise-processing algorithm is a function of the at least one measure for the noise.

4. The computer-implemented method of claim 3, further comprising determining a scaling factor for the scaling of the noise data set or scaling factors for the different scaling of the different frequency components, determining a parameter of the filtering as a function of the at least one measure for the noise, or a combination thereof.

5. The computer-implemented method of claim 1, further comprising determining the intermediate data set, the determining of the intermediate data set comprising applying a variance-stabilizing transformation to the input data set, wherein the signal processing algorithm is or includes an assigned inverse transformation.

6. The computer-implemented method of claim 1, further comprising applying, in the context of the signal processing algorithm, a non-linear function to image data from the signal data set or image data of a processing result that is determined in the context of preprocessing within the signal processing algorithm as a function of the signal data set.

7. The computer-implemented method of claim 6, wherein the non-linear function is a logarithmic function.

8. The computer-implemented method of claim 6, further comprising predicting a contribution from scattered radiation that occurs during acquisition of the input data set to the image data from the signal data set or predicting the processing result, wherein the non-linear function is a function of the predicted contribution, or the signal data set or the intermediate result is corrected before the application of the non-linear function as a function of the predicted contribution.

9. The computer-implemented method of claim 1, further comprising applying, in the context of the signal processing algorithm, a function for contrast adjustment to an image data from the signal data set or image data of one or a further processing result that is determined in the context of preprocessing within the signal processing algorithm as a function of the signal data set.

10. The computer-implemented method of claim 1, wherein the signal processing algorithm, the noise-processing algorithm, or the signal processing algorithm and the noise-processing algorithm are parameterized as a function of at least one input parameter, at least one output variable, or the at least one input parameter and the at least one output variable of a dose regulation used in the context of the medical imaging method.

11. The computer-implemented method of claim 1, wherein the algorithm used as the algorithm for reducing the image noise is trained by a machine-learning method.

12. A computer-implemented method for data processing in the context of medical subtraction imaging, wherein two input data sets that are each determined by a medical imaging method on a same examination object are processed, the computer-implemented method comprising: determining a respective output data set, determining the respective output data set comprising applying a method to the two input data sets, the method comprising: determining a noise-reduced signal data set, the determining of the noise-reduced signal data set comprising applying an algorithm for reducing image noise to the input data set or an intermediate data set determined as a function of the input data set; determining a noise data set as a difference between the input data set or the intermediate data set and the noise-reduced signal data set; determining a modified noise data set, wherein the determining of the modified noise data set comprises applying a noise-processing algorithm to the noise data set, determining a modified signal data set, wherein determining the modified signal data set comprises applying a signal processing algorithm to the signal data set, or a combination thereof; and determining an output data set, determining the output data set comprising adding the modified noise data set to the noise-reduced signal data set or the modified signal data set or adding the noise data set to the modified signal data set; providing a subtraction data set, providing the subtraction data set comprising subtracting the two output data sets from one another; and applying a function for contrast adjustment to image data from the subtraction data set.

13. A provisioning device configured to provide an output data set as a function of an input data set, the provisioning device comprising: a processor configured to: determine a noise-reduced signal data set, the determination of the noise-reduced signal data set comprising application of an algorithm for reducing image noise to the input data set or an intermediate data set determined as a function of the input data set; determine a noise data set as a difference between the input data set or the intermediate data set and the noise-reduced signal data set; determine a modified noise data set, wherein the determination of the modified noise data set comprises application of a noise-processing algorithm to the noise data set, determination of a modified signal data set, wherein the determination of the modified signal data set comprises application of a signal processing algorithm to the signal data set, or a combination thereof; and determine an output data set, the determination of the output data set comprising adding the modified noise data set to the noise-reduced signal data set or the modified signal data set or adding the noise data set to the modified signal data set.

14. A medical imaging device comprising: a provisioning device configured to provide an output data set as a function of an input data set, the provisioning device comprising: a processor configured to: determine a noise-reduced signal data set, the determination of the noise-reduced signal data set comprising application of an algorithm for reducing image noise to the input data set or an intermediate data set determined as a function of the input data set; determine a noise data set as a difference between the input data set or the intermediate data set and the noise-reduced signal data set; determine a modified noise data set, wherein the determination of the modified noise data set comprises application of a noise-processing algorithm to the noise data set, determination of a modified signal data set, wherein the determination of the modified signal data set comprises application of a signal processing algorithm to the signal data set, or a combination thereof; and determine an output data set, the determination of the output data set comprising adding the modified noise data set to the noise-reduced signal data set or the modified signal data set or adding the noise data set to the modified signal data set.

15. The medical imaging device of claim 14, wherein the medical imaging device is an X-ray device.

16. In a non-transitory computer-readable storage medium that stores instructions executable by one or more processors to provide an output data set as a function of an input data set that is determined by a medical imaging method, the instructions comprising: determining a noise-reduced signal data set, the determining of the noise-reduced signal data set comprising applying an algorithm for reducing image noise to the input data set or an intermediate data set determined as a function of the input data set; determining a noise data set as a difference between the input data set or the intermediate data set and the noise-reduced signal data set; determining a modified noise data set, wherein the determining of the modified noise data set comprises applying a noise-processing algorithm to the noise data set, determining a modified signal data set, wherein determining the modified signal data set comprises applying a signal processing algorithm to the signal data set, or a combination thereof; and determining an output data set, determining the output data set comprising adding the modified noise data set to the noise-reduced signal data set or the modified signal data set or adding the noise data set to the modified signal data set.

17. The non-transitory computer-readable storage medium of claim 16, wherein the noise-processing algorithm includes scaling of the noise data set, different scaling of different frequency components of the noise data set, or filtering of the noise data set.

18. The non-transitory computer-readable storage medium of claim 17, wherein the instructions further comprise determining at least one measure for a noise in the input data set, the intermediate data set, or the noise data set, wherein the noise-processing algorithm is a function of the at least one measure for the noise.

19. The non-transitory computer-readable storage medium of claim 18, wherein the instructions further comprise determining a scaling factor for the scaling of the noise data set or scaling factors for the different scaling of the different frequency components, determining a parameter of the filtering as a function of the at least one measure for the noise, or a combination thereof.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0059] FIG. 1 shows a flowchart of an exemplary embodiment of a computer-implemented method for providing an output data set;

[0060] FIG. 2 shows exemplary algorithms and data structures used in the context of the method shown in FIG. 1;

[0061] FIG. 3 shows an interaction of an exemplary embodiment of a provisioning device with a medical imaging device;

[0062] FIG. 4 shows exemplary algorithms and data structures used in the context of an exemplary embodiment of a computer-implemented method for data processing in the context of medical subtraction imaging;

[0063] FIG. 5 shows an exemplary embodiment of a training method for training an algorithm for reducing image noise that may be used in the context of the computer-implemented methods by a machine-learning method; and

[0064] FIGS. 6 and 7 show exemplary structures of algorithms that may be trained to reduce image noise.

DETAILED DESCRIPTION

[0065] FIG. 1 shows a flowchart of one embodiment of a computer-implemented method for providing an output data set as a function of an input data set determined by a medical imaging method. The method is explained in the following with additional reference to FIG. 2, which is a schematic depiction of algorithms and data structures used in the context of the method. For reasons of clarity, in each case, two-dimensional individual image data sets are depicted as an input data set or as processing results. However, the described method is also suitable for processing three-dimensional image data sets. It is also possible for the input data set to include a plurality of image data sets recorded in sequence so that the processing results may accordingly also include a plurality of image data sets. This is also not depicted for reasons of clarity.

[0066] In act S1, initially, an input data set 33 determined by a medical imaging method (e.g., as an X-ray recording with a flat-panel detector) is provided. The acquisition of the image data from the input data set may, for example, take place in an upstream step independently of the computer-implemented method explained. However, in principle, it would also be possible to integrate the acquisition in the method.

[0067] In act S2, a variance-stabilizing transformation 34 is applied to the input data set 33 in order to generate an intermediate data set 35. The variance-stabilizing transformation 34 serves to modify a noise component of the input data set such that the noise is substantially independent of the measuring signal (e.g., to transfer the noise from a Poisson distribution to a Gaussian distribution). Different approaches that are known per se may be used for the variance stabilization. For example, a root function or an Anscombe transformation may be applied to the image data from the input data set 33.

[0068] In act S3, an algorithm 36 for reducing image noise is applied to the intermediate data set 35 in order to determine the noise-reduced signal data set 38. Due to the upstream variance stabilization, it is possible to use any algorithm 36 suitable for reducing signal-independent noise. As will be explained later, the algorithm 36 may, for example, be trained by machine learning. For this purpose, the algorithm 36 may include a large number of free parameters 37, which are learned in the context of training. One option for training such an algorithm 36 will be explained later with reference to FIG. 5, and possible structures for corresponding algorithms will be explained later with reference to FIGS. 6 and 7.

[0069] In act S4, a noise data set is determined by subtracting the signal data set 38 from the intermediate data set 35. This substantially maps the noise in the intermediate data set 35 or the variance-stabilized noise in the input data set 33.

[0070] This is followed by parallel processing of the noise data set 39 in act S5 by a noise-processing algorithm 41 in order to provide a modified noise data set 43 and of the signal data set 38 in acts S6 and S7 in order to provide a modified signal data set 42. In act S8, the modified signal data set 42 and the modified noise data set 43 are subsequently added together in order to provide the output data set 44.

[0071] The application of the noise-processing algorithm 41 in act S5 may cause a scaling or shaping of the noise to take place such that the actual noise spectrum approximates a desired noise spectrum. In the simplest case, scaling of the entire noise data set would be possible. However, in one embodiment, different scaling of different frequency components of the noise data set 7 or filtering of the noise data set 39 takes place in order to bring the noise spectrum into line with a desired shape. For example, coloration of the noise that occurs due to the measuring electronics used may be eliminated or the like.

[0072] In the example, for this, different scaling factors 53 are used for the scaling of the frequency components. The noise data set 39 may be divided into frequency components, for example, via a Fourier transformation or by constructing a Gauss-Laplace pyramid with which Gaussian filters of different sizes are applied in different positions after which the differences of positions correspond to different frequency components.

[0073] For the adjustment of the noise, for example, at least one measure 51 for the noise is determined in the intermediate data set 35 or alternatively also in the input data set 33 or in the noise data set 39. For example, noise amplitudes may be determined for different frequency components. Approaches for determining such a measure 51 have already been discussed in the general part of the description.

[0074] Additionally, the scaling factors 53 or other parameters parameterizing the noise-processing algorithm 41 may be functions of additional information 52, which, for example, may relate to a user-defined function for noise adjustment, a water equivalent for the examination object, or other parameters relating to the imaging or evaluation of the image data and were already discussed above. The scaling factors 53 or parameters of the noise-processing algorithm 41 may, for example, be taken from a lookup table as a function of the parameters.

[0075] In act S6, initially, an inverse transformation 45 for variance-stabilizing transformation 34 is performed in the context of the signal processing algorithm 40. The variance-stabilizing transformation 34 causes the noise in the intermediate data set 35 to be substantially frequency-independent. However, in this case, the variance-stabilizing transformation 34 falsifies the signal component. This is compensated by applying the inverse transformation 45; as a result of this, the intermediate result 46 substantially corresponds to the signal component of the input data set 33.

[0076] In an alternative embodiment, the acts S2 and S6, the variance-stabilizing transformation 34 and the inverse transformation 45, may also be dispensed with. In this case, the input data set 33 may be used in each case instead of the intermediate data set 35, and the processing in act S7 may process the signal data set 38 directly instead of the intermediate result 46.

[0077] In act S7, the contrast processing of the intermediate result 46 or, if variance stabilization is dispensed with, of the signal data set 38 takes place. In this case, gray level transformations and spatial frequency operations may be applied, for example. For example, lookup tables, bandpass filters, multi-scalar filters, etc. may be used. In the example shown, the application of a non-linear function 47 in order to determine a further intermediate result 49 and the subsequent application of a function 50 for contrast adjustment to the further intermediate result 49 to determine the modified signal data set 42 are performed in sequence.

[0078] The non-linear function 47 is, for example, modified as a function of a predicted contribution 48 of scattered radiation to the image data from the signal data set 38 or the processing result 46. This, for example, serves to compensate an offset of the signal data set 38 or the intermediate result 46 due to these scattered rays. Alternatively, it would also be possible to modify the intermediate result 46 before applying the non-linear function 15. The application of a non-linear function (e.g., a logarithmic function) may serve to provide that that the same thickness differences of the mapped object with the same materials always result in the same changes in contrast, and this may greatly facilitate the evaluation of the image data.

[0079] The function 50 for contrast adjustment may, for example, be a function of parameters 54 that may, for example, be functions of output variables 55 of dose regulation used in the context of the medical imaging method. The parameters 54 may, for example, be determined using a lookup table, a prespecified functional connection, or the like.

[0080] The function 50 may, for example, weight different frequency components in the further intermediate result differently in order, for example, to highlight fine structures or the like.

[0081] In the example discussed with reference to FIGS. 1 and 2, both the signal data set 38 and the noise data set 39 are modified before addition to the output data set 44. In principle, it would also be possible only to modify one of these data sets and to leave the other one unchanged.

[0082] FIG. 3 shows one embodiment of a medical imaging device 56 (e.g., an X-ray device), by which the input data set 33 in the previously explained method, which maps the examination object 57, may be acquired. A control device 58 (e.g., a controller) controls the operation of the imaging device 56. The control device 58 may, for example, implement dose regulation 65, which controls the X-ray dose irradiated onto the examination object 57 as a function of acquired measurement data (e.g., as a function of a contrast-to-noise ratio for a defined object).

[0083] The control device 58 may in principle provide the input data set determined directly (e.g., via the network 59) to an evaluation device 60 (e.g., a workstation computer). However, in this case, there may be significant differences with regard to the contrasts or noise that occur for different examination objects, different angulation angles, etc., for example, due to the use of dose regulation 65, thus complicating evaluation of the image data.

[0084] Therefore, the input data set 33 acquired is initially transferred to the provisioning device 61, which implements the method explained above with reference to FIGS. 1 and 2. This may, for example, take place in that a data processing device 62 is appropriately programed by an appropriate computer program. The resulting output data set 44 may then, for example, be provided to the evaluation device 60.

[0085] The input data sets 33 or output data sets 44 may also be initially buffered in a database 63, thus enabling the described data processing to take place independently of the acquisition of the measurement data by the medical imaging device 56. If, in this case, as explained above, operating parameters of the medical imaging device 56 (e.g., parameters of dose regulation 65) are taken into account, the operating parameters may be filed jointly with the input data set 33 in the database 63.

[0086] For example, the depicted use of a provisioning device 61 embodied separately from the medical imaging device 56 enables the explained method to be offered as a service. However, in some cases, it may also be advantageous, instead of a separate provisioning device 61, to use a provisioning device 64 integrated in the medical imaging device 56, which may, for example, be implemented by the control device 58. This enables the advantages of the explained method to be used independently of external devices.

[0087] FIG. 4 shows an exemplary embodiment of a computer-implemented method for data processing in the context of medical subtraction imaging. In this case, two input data sets 66, 67 are each processed by the method 68 already explained with reference to FIGS. 1 and 2 for providing a respective output data set 69,70. The output data sets 69, 70 are then subtracted from one another in order to provide the subtraction data set 71. Then, a function 72 for contrast adjustment is applied to the image data from the subtraction data set 71 in order to a provide a contrast-adjusted subtraction data set 73.

[0088] In the method 68, in contrast to the variant depicted in FIG. 2, in act S7, the function 50 for contrast adjustment may not be applied; instead, the function 72 is used for contrast adjustment and is only applied when the output data sets 69, 70 have been subtracted from one another. This may avoid different impacts of the function 50 on the output data sets 69, 70 that falsify the subtraction data set 71. In this case, it is accepted that, in the context of contrast adjustment, the function 72 for contrast adjustment in the exemplary embodiment shown in FIG. 4 also varies the noise component of the subtraction image 71.

[0089] Apart from the fact that it is used at a different point in time in the method, the function 72 for contrast adjustment may correspond to the function 50 for contrast adjustment already discussed with reference to FIG. 2. Accordingly, the parameters of function 72 may also be varied as a function of imaging parameters or other parameters, as has already been explained with regard to FIG. 2 and function 50.

[0090] FIG. 5 shows an exemplary training method for training the algorithm 36 for reducing the image noise, which may be used in the context of the computer-implemented method according to the present embodiments, using a machine-learning method. The algorithm is trained using training data sets 74 each including a first image data set 75 and a second image data set 76. The first image data set 75 and the second image data set 76 are acquired on the same examination object with the same acquisition parameters.

[0091] For the training of the algorithm 36, a cost function 80 is minimized by varying the parameters 37 of the algorithm 36. The cost function 80 is a function of a difference between a measure 79 for the noise in the second image data set 76 and a measure 78 for the noise in a sum data set 77. The sum data set 77 is determined in that the algorithm to be trained 36 is applied to the first image data set 75 and the resulting result data set is added to the second image data set 76. The variation of the parameters 37 of the algorithm 36 may, for example, take place using an error feedback method that is known per se.

[0092] With reference to FIGS. 6 and 7, the following uses the example of a neural network or a convolutional neural network to explain structures of algorithms that may be trained by machine learning in order to provide the algorithm 36 for reducing image noise. The structures are described using simple examples that may be expanded appropriately for real applications.

[0093] FIG. 6 shows an exemplary embodiment of an artificial neural network 1. Other English expressions for artificial neural network 1 are “neural network”, “artificial neural net”, or “neural net”.

[0094] The artificial neural network 1 includes nodes 6 to 18 and edges 19 to 21, where each edge 19 to 21 is a directional link from a first node 6 to 18 to a second node 6 to 18. Generally, the first node 6 to 18 and the second node 6 to 18 are different nodes 6 to 18; however, the first node 6 to 18 and the second node 6 to 18 may be identical. For example, in FIG. 6, the edge 19, a directional link from the node 6 to node 9 and the edge 21, is a directional link from the node 16 to the node 18. An edge 19 to 21 from a first node 6 to 18 to a second node 6 to 18 is referred to as an ingoing edge for the second nodes 6 to 18 and as an outgoing edge for the first node 6 to 18.

[0095] In this exemplary embodiment, the nodes 6 to 18 of the artificial neural network 1 may be arranged in layers 2 to 5, where the layers 2 to 5 may have an intrinsic order, which is introduced through the edges 19 to 21 between the nodes 6 to 18. For example, edges 19 to 21 may only be provided between adjacent layers of nodes. In the exemplary embodiment depicted, there is an input layer 2 that only has the nodes 6, 7, 8, in each case without an ingoing edge. The output layer 5 includes only the nodes 17, 18, in each case without outgoing edges, where there are hidden layers 3 and 4 between the ingoing layer 2 and the output layer 5. Generally, the number of hidden layers 3, 4 may be chosen as desired. The number of nodes 6, 7, 8 in the input layer 2 may correspond to the number of input values into the neural network 1, and the number of nodes 17, 18 in the output layer 5 may correspond to the number of output values of the neural network 1.

[0096] For example, a real number may be assigned to the nodes 6 to 18 of the neural network 1. Herein, x.sup.(n).sub.i designates the value of the i-th nodes 6 to 18 in the n-th layer 2 to 5. The values of the nodes 6, 7, 8 in the input layer 2 are equivalent to the input values of the neural network 1, while the values of the nodes 17, 18 in the output layer 113 are equivalent to the output values of the neural network 1. Moreover, a weight in the form of a real number may be assigned to each edge 19, 20, 21. For example, the weight is a real number in the interval [−1, 1] or in the interval [0, 1,]. Herein, w.sup.(m,n).sub.i,j designates the weight of the edge between the i-th nodes 6 to 18 in the m-th layer 2 to 5 and the j-th nodes 6 to 18 in the n-th layer 2 to 5. Further, the abbreviation w.sub.i,j.sup.(n) is defined for the weight w.sub.i,j.sup.(n,n+1).

[0097] In order to calculate output values of the neural network 1, the input values are propagated by the neural network 1. For example, the values of the nodes 6 to 18 in the (n+1)-th layer 2 to 5 may be calculated based on the values of the nodes 6 to 18 of the n-th layer 2 to 5 by

x.sub.j.sup.(n+1)=f(Σ.sub.ix.sub.i.sup.(n).Math.w.sub.i,j.sup.(n)).

[0098] Herein, f is a transfer function, which may also be designated an activation function. Known transfer functions are step functions, sigmoid functions (e.g., the logistic function, the generalized logistic function, the hyperbolic tangent, the arcus tangent, the error function, the smoothstep function), or rectifier functions. The transfer function is substantially used for purposes of standardization.

[0099] For example, the values are propagated in layers by the neural network 1, where values of the ingoing layer 2 are defined by the input data of the neural network 1. Values of the first hidden layer 3 may be calculated based on the values of the ingoing layer 2 of the neural network 1, values of the second hidden layer 4 may be calculated based on the values in the first hidden layer 3, and so on.

[0100] In order to be able to establish the values w.sub.i,j.sup.(n) the edges 19 to 21, the neural network 1 is to be trained using training data. For example, training data includes input training data and output training data, which are designated t.sub.i below. For a training step, the neural network 1 is applied to the input training data in order to determine calculated output data. For example, the output training data and the calculated output data include a number of values, where the number is determined as the number of nodes 17, 18 in the output layer 5.

[0101] For example, a comparison between the calculated output data and the output training data is used in order to adjust the weights within the neural network 1 recursively (e.g., back propagation algorithm). For example, the weights may be changed according to

w′.sub.i,j.sup.(n)=w.sub.i,j.sup.(n)−γ.Math.δ.sub.j.sup.(n).Math.x.sub.i.sup.(n),

where γ is a learning rate, and the numbers δ.sub.j.sup.(n) may be calculated recursively as

δ.sub.j.sup.(n)=(Σ.sub.kδ.sub.k.sup.(n+1).Math.w.sub.j,k.sup.(n+1)).Math.f′(Σ.sub.ix.sub.i.sup.(n).Math.w.sub.i,j.sup.(n))

based on δ.sub.j.sup.(n+1), when the (n+1)-th layer is not the output layer 5 and

δ.sub.j.sup.(n)=(x.sub.k.sup.(n+1)−t.sub.j.sup.(n+1)).Math.f′(Σ.sub.ix.sub.i.sup.(n).Math.w.sub.i,j.sup.(n))

if the (n+1)-th layer is the output layer 5, where f is the first derivation of the activation function and y.sub.j.sup.(n+1) is the comparison training value for the j-th node 17, 18 in the output layer 5.

[0102] The following also defines an example of a convolutional neural network (CNN) with respect to FIG. 7. Herein, it should be noted that the expression “layer” is used there in a slightly different manner than for conventional neural networks. For a conventional neural network, the expression “layer” only refers to the set of nodes that forms a layer (e.g., a specific generation of nodes). For a convolutional neural network, the expression “layer” is often used as an object that actively changes data (e.g., a set of nodes of the same generation and either the set of ingoing edges or the set of outgoing edges).

[0103] FIG. 7 shows an exemplary embodiment of a convolutional neural network 22. In the exemplary embodiment, the convolutional neural network 22 includes an ingoing layer 23, a convolutional layer 24, a pooling layer 25, a fully connected layer 26, and an output layer 27. In alternative embodiments, the convolutional neural network 22 may contain a plurality of convolutional layers 24, a plurality of pooling layers 25, and a plurality of fully connected layers 26, plus other types of layers. The sequence of layers may be selected at will, where fully connected layers 26 form the last layers before the output layer 27.

[0104] For example, within a convolutional neural network 22, the nodes 28 to 32 in one of the layers 23 to 27 may be arranged as d-dimensional matrix or as d-dimensional image. For example, in the two-dimensional case, the value of a node 28 to 32 with the indices i, j may be designated x.sup.(n)[i,j] in the n-th layer 23 to 27. Reference is made to the fact that the arrangement of the nodes 28 to 31 in a layer 23 to 27 has no impact on the calculations within the convolutional neural network 22 as such since this impact is exclusively defined by the structure and weights of the edges.

[0105] A convolutional layer 24 is, for example, characterized in that the structure and the weights of the ingoing edges form a convolution operation based on a specific number of kernels. For example, the structure and weights of the ingoing edges may be selected such that the values x.sub.k.sup.(n) of the nodes 29 in the convolutional layer 24 are determined as a convolution x.sub.k.sup.(n)=K.sub.k*x.sup.(n−1) based on the values x.sup.(n−1) of the nodes 28 in the preceding layer 23, where, in the two-dimensional case, the convolution * may be defined as

x.sub.k.sup.(n)[i,j]=(K.sub.k*x.sup.(n−1))[i,j]=Σ.sub.i′Σ.sub.j′K.sub.k[i′,j′].Math.x.sup.(n−1)[i−i′,j−j′],

[0106] Here, the k-th kernel K.sub.k is a d-dimensional matrix (e.g., a two-dimensional matrix) that is usually small compared to the number of nodes 28 to 32 (e.g., a 3×3-matrix or a 5×5-matrix). For example, this implies that the weights of the ingoing edges are not independent, but are selected such the weights generate the above convolution equation. In the example of a kernel that forms a 3×3-matrix, there are only nine independent weights (e.g., where each entry in the kernel matrix corresponds to an independent weight), regardless of the number of nodes 28 to 32 in the corresponding layer 23 to 27. For example, for a convolutional layer 24, the number of nodes 29 in the convolutional layer 24 is equivalent to the number of nodes 28 in the preceding layer 23 multiplied by the number of convolution kernels.

[0107] If the nodes 28 in the preceding layer 23 are arranged as a d-dimensional matrix, the use of the plurality of kernels may be understood to be the addition of a further dimension, which is also designated a depth dimension, so that the nodes 29 in the convolutional layer 24 are arranged as a (d+1)-dimensional matrix. If the nodes 28 in the preceding layer 23 are already arranged as a (d+1)-dimensional matrix with a depth dimension, the use of a plurality of convolution kernels may be understood as an expansion along the depth dimension so that the nodes 29 in the convolutional layer 221 are likewise arranged as a (d+1)-dimensional matrix, where the size of the (d+1)-dimensional matrix in the depth dimension is greater than in the preceding layer 23 by the factor formed by the number of kernels.

[0108] The advantage of using convolutional layers 24 is that the spatially local correlation of the input data may be exploited in that a local connection pattern between nodes in adjacent layers is created (e.g., due to the fact that each node only has connections to a small region of the nodes in the preceding layer).

[0109] In the exemplary embodiment, the ingoing layer 23 includes nodes 28 arranged as a two-dimensional 6×6-matrix. The convolutional layer 24 includes 72 nodes 29 arranged as two two-dimensional 6×6-matrices, where each of the two matrices is the result of a convolution of the values of the ingoing layer 23 with a convolution kernel. In the same way, the nodes 29 in the convolutional layer 24 may be understood as being arranged in a three-dimensional 6×6×2-matrix, where the last-named dimension is the depth dimension.

[0110] A pooling layer 25 is characterized in that that the structure and weights of the ingoing edges and the activation function of their nodes 30 define a pooling operation based on a non-linear pooling function f. For example, in the two-dimensional case, the values x.sup.(n) of the nodes 30 in the pooling layer 25 may be calculated based on the values x.sup.(n+1) of the nodes 29 in the preceding layer 24 as

x.sup.(n)[i,j]=f(x.sup.(n−1)[id.sub.1jd.sub.2], . . . ,x.sup.(n−1)[id.sub.1+d.sub.1−1,jd.sub.2+d.sub.2−1]).

[0111] In other words, the use of a pooling layer 25 may reduce the number of nodes 29, 30 in that a number of d.sub.1*d.sub.2 adjacent nodes 29 in the preceding layer 24 is replaced by one single node 30, which is calculated as a function of the values of the stated number of adjacent nodes 29. For example, the pooling function f may be a maximum function, an averaging, or the L2 norm. For example, for a pooling layer 25, the weights of the ingoing edges may be fixed and not modified by training.

[0112] The advantage of using a pooling layer 25 is that the number of nodes 29, 30 and the number of parameters is reduced. This results in a reduction in the number of calculations required within the convolutional neural network 22 and hence to suppression of overfitting.

[0113] In the exemplary embodiment depicted, the pooling layer 25 is a max pooling layer, in which four adjacent nodes are replaced by one single node, the value of which is formed by the maximum of the values of the four adjacent nodes. The max pooling is applied to each d-dimensional matrix of the previous layer; in this exemplary embodiment, the max pooling is applied to each of the two two-dimensional matrices so that the number of nodes is reduced from 72 to 18.

[0114] A fully connected layer 26 is characterized in that that a plurality (e.g., all) of the edges between the nodes 30 in the previous layer 25 and the nodes 31 in the fully connected layer 26 are present, where the weight of each of the edges may be adjusted individually. In this exemplary embodiment, the nodes 30 of the layer 25 preceding the fully connected layer 26 are shown as both two-dimensional matrices and as non-linked nodes (e.g., depicted as a row of nodes, where the number of nodes has been reduced to improve the depiction). In this exemplary embodiment, the number of nodes 31 in the fully connected layer 26 is equal to the number of nodes 30 in the preceding layer 25. In alternative embodiments, the number of nodes 30, 31 may differ.

[0115] In this exemplary embodiment, the values of the nodes 32 in the output layer 27 are determined in that the softmax function is applied to the values of the nodes 31 in the preceding layer 26. As a result of the application of the softmax function, the sum of the values of all nodes 32 in the output layer 27 is one and all values of all the nodes 32 in the output layer are real numbers between 0 and 1. If the convolutional neural network 22 is used to classify input data, for example, the values of the output layer 27 may be interpreted as a probability that the input data falls into one of the different classes.

[0116] A convolutional neural network 22 may also have a ReLU layer, where ReLU stands for “rectified linear units”. For example, the number of nodes and the structure of the nodes within a ReLU layer is equivalent to the number of nodes and the structures of the nodes in the preceding layer. The value of each node in the ReLU layer may, for example, be calculated by applying a rectifier function to the value of the corresponding nodes in the preceding layer. Examples of rectifier functions are f(x)=max(0,x), hyperbolic tangent, or the sigmoid function.

[0117] Convolutional neural networks 22 may, for example, be trained based on the back-propagation algorithm. To avoid overfitting, regularization methods may be used (e.g., dropout of individual nodes 28 to 32, stochastic pooling, the use of artificial data, weight decay based on the L1 or L2 norm, or maximum norm restrictions).

[0118] Although the invention has been illustrated and described in detail by the exemplary embodiments, the invention is not restricted by the disclosed examples, and other variations may be derived herefrom by the person skilled in the art without departing from the scope of protection of the invention.

[0119] The elements and features recited in the appended claims may be combined in different ways to produce new claims that likewise fall within the scope of the present invention. Thus, whereas the dependent claims appended below depend from only a single independent or dependent claim, it is to be understood that these dependent claims may, alternatively, be made to depend in the alternative from any preceding or following claim, whether independent or dependent. Such new combinations are to be understood as forming a part of the present specification.

[0120] While the present invention has been described above by reference to various embodiments, it should be understood that many changes and modifications can be made to the described embodiments. It is therefore intended that the foregoing description be regarded as illustrative rather than limiting, and that it be understood that all equivalents and/or combinations of embodiments are intended to be included in this description.