Method and device for verifying a neuron function in a neural network

Abstract

A method for verifying a calculation of a neuron value of multiple neurons of a neural network, including: carrying out or triggering a calculation of neuron functions of the multiple neurons, in each case to obtain a neuron value, the neuron functions being determined by individual weightings for each neuron input; calculating a first comparison value as the sum of the neuron values of the multiple neurons; carrying out or triggering a control calculation with one or multiple control neuron functions and with all neuron inputs of the multiple neurons, to obtain a second comparison value as a function of the neuron inputs of the multiple neurons and of the sum of the weightings of the multiple neurons assigned to the respective neuron input; and recognizing an error as a function of the first comparison value and of the second comparison value.

Claims

1. A method for verifying a calculation of a neuron value of multiple neurons of a neural network, the method comprising: performing or triggering a calculation of neuron functions of each of the multiple neurons, to obtain neuron values, the neuron functions being determined by individual weightings for each neuron input; calculating a first comparison value as a sum of the neuron values of the multiple neurons; performing or triggering a control calculation with one or multiple control neuron functions and with all neuron inputs of the multiple neurons, to obtain a second comparison value as a function of the neuron inputs of the multiple neurons and of a sum of the weightings of the multiple neurons assigned to the respective neuron input; and recognizing an error as a function of the first comparison value and of the second comparison value.

2. The method of claim 1, wherein the one or the multiple control neuron functions are performed by one or multiple additional provided control neurons.

3. The method of claim 1, wherein the neuron functions are each determined as a function of a bias value, the neuron values of the multiple neurons being calculated with the aid of the neuron functions as a function of the bias values, the control calculation of the second comparison value with the one or the multiple control neuron functions being performed as a function of a sum of all bias values of the multiple neuron functions.

4. The method of claim 1, wherein the multiple neurons are parts of multiple kernels for calculating a convolutional neural network based on a multi-dimensional data matrix having multiple channels, the control calculation being performed based on a control kernel, the weightings of the neuron functions of the control kernel being determined by a sum of the weightings of the multiple kernels assigned to a neuron input.

5. The method of claim 4, wherein the sum of all correlating data points within the multi-dimensional data matrix across all channels is formed as first comparison value, the second comparison value being ascertained by applying the control calculation with the control kernel for a matrix position of the respectively correlating data points.

6. The method of claim 1, wherein the multiple neurons are parts of a kernel for calculating a convolutional neural network based on a multi-dimensional data matrix, the control calculation being performed based on a number of sums of data values of the data matrix in a verification dimension, the second comparison value being determined as the sum of the products of the weightings of the kernel, in each case with one of the sums of the data values.

7. A verification system for verifying a calculation of neuron values of multiple neurons to be verified of a neural network, comprising: a summing circuit for calculating a first comparison value as a sum of the neuron values of the multiple neurons, wherein the multiple neurons perform calculations of neuron functions for each of the multiple neurons, to obtain the neuron values, the neuron functions being determined by individual weightings for each neuron input, and wherein at least one control neuron, which is provided to perform a control calculation with one or multiple control neuron functions and with all neuron inputs of the multiple neurons, to obtain a second comparison value as a function of the neuron inputs of the multiple neurons and of a sum of the weightings of the multiple neurons assigned to the respective neuron input; and a comparison circuit to recognize an error as a function of the first comparison value and of the second comparison value.

8. A non-transitory computer readable medium having a computer program, which is executable by a processor, comprising: a program code arrangement having program code for verifying a calculation of a neuron value of multiple neurons of a neural network, by performing the following: performing or triggering a calculation of neuron functions of each of the multiple neurons, to obtain neuron values, the neuron functions being determined by individual weightings for each neuron input; calculating a first comparison value as a sum of the neuron values of the multiple neurons; performing or triggering a control calculation with one or multiple control neuron functions and with all neuron inputs of the multiple neurons, to obtain a second comparison value as a function of the neuron inputs of the multiple neurons and of a sum of the weightings of the multiple neurons assigned to the respective neuron input; and recognizing an error as a function of the first comparison value and of the second comparison value.

9. The computer readable medium of claim 8, wherein the one or the multiple control neuron functions are performed by one or multiple additional provided control neurons.

10. The computer readable medium of claim 8, wherein the neuron functions are each determined as a function of a bias value, the neuron values of the multiple neurons being calculated with the aid of the neuron functions as a function of the bias values, the control calculation of the second comparison value with the one or the multiple control neuron functions being performed as a function of a sum of all bias values of the multiple neuron functions.

11. The computer readable medium of claim 8, wherein the multiple neurons are parts of multiple kernels for calculating a convolutional neural network based on a multi-dimensional data matrix having multiple channels, the control calculation being performed based on a control kernel, the weightings of the neuron functions of the control kernel being determined by a sum of the weightings of the multiple kernels assigned to a neuron input.

12. The computer readable medium of claim 11, wherein the sum of all correlating data points within the multi-dimensional data matrix across all channels is formed as first comparison value, the second comparison value being ascertained by applying the control calculation with the control kernel for a matrix position of the respectively correlating data points.

13. The computer readable medium of claim 8, wherein the multiple neurons are parts of a kernel for calculating a convolutional neural network based on a multi-dimensional data matrix, the control calculation being performed based on a number of sums of data values of the data matrix in a verification dimension, the second comparison value being determined as the sum of the products of the weightings of the kernel, in each case with one of the sums of the data values.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 schematically shows a representation of a neuron.

(2) FIG. 2 schematically shows a representation of a functional diagram for illustrating the functionality of a control neuron.

(3) FIG. 3 shows a representation of the calculation paths of the comparison variables for a line of an input feature map.

(4) FIG. 4 shows the application of the neuron verification for an input feature map having multiple channels.

DETAILED DESCRIPTION

(5) The core process of neural networks is the neural function. A neuron 1 for constructing a neural network is schematically depicted in FIG. 1. Neuron 1 carries out the neuron function, which includes an addition of a sum of input values x.sub.i weighted with weightings w.sub.n,i with a bias value b.sub.n, in order to generate a neuron value:

(6) $o_n=\underset{i=1}{\overset{z}{.Math.}}{x}_i{w}_{n,i}+b_n$
z corresponds to the number of neuron inputs. The provision of the bias value b.sub.n may, if necessary, be optional. The weightings w.sub.n,i and the bias values b.sub.n represent the parameters of the neuron function.

(7) The neuron value is, if necessary, provided with an activation function in order to obtain a neuron output. The activation function is handled separately in hardware implementations and is not further considered here.

(8) Thus, each neuron 1 is defined by a number of weighting values w.sub.n,i assigned to neuron 1 and to the respective neuron inputs and by an assigned bias value b.sub.n. Neurons having such a neuron function are generally implemented by a plurality of multiply-accumulate elements (MAC) in an integrated manner.

(9) To verify the functional capability of neurons, specific embodiments are described below, which utilize the associative law of mathematics, according to which the sum of the neuron values of the neurons in question equals the sum of the products of each neuron input with the sum of the weightings assigned to the neurons plus the sum of the bias values. The following applies:

(10) $\underset{n=1}{\overset{m}{.Math.}}\left(\underset{i=1}{\overset{z}{.Math.}}{x}_i{w}_{n,i}+b_n\right)=\underset{i=1}{\overset{z}{.Math.}}\left(x_i\underset{n=1}{\overset{m}{.Math.}}{w}_{n,i}\right)+\underset{n=1}{\overset{m}{.Math.}}{b}_n$
where m>1 of the number of the neurons to be verified. The right portion of the equation may be implemented by a control neuron 2.

(11) In FIG. 2, a functional diagram for implementing such a control neuron 2 for two neurons 1 to be verified is schematically depicted. Here, two neuron inputs x.sub.1, x.sub.2 are each added to neurons 1 to be verified, multiplied there in each case with an assigned weighting W.sub.1,1, W.sub.1,2, W.sub.2,1, W.sub.2,2 and additively provided with a bias value b.sub.1, b.sub.2. Each neuron output o.sub.1, o.sub.2 is added to a summing element 3 in order to obtain a first comparison value o.sub.n.

(12) The sums of the weightings

(13) $\underset{n=1}{\overset{2}{.Math.}}{w}_{n,1},\underset{n=1}{\overset{2}{.Math.}}{w}_{n,2},$
assigned to a respective neuron input, with which the respective neuron input x.sub.1, x.sub.2 is provided, are calculated in second summing elements 4 and added to control neuron as control neuron 2 weightings w.sub.c,1, W.sub.c,2. Alternatively, the sum of the weightings may be calculated in advance, since the weightings are fixed after the training, and correspondingly provided from a suitable memory.

(14) In addition, the sums of the bias values b.sub.1, b.sub.2 assigned to a respective neuron input, with which the respective neuron input x.sub.1, x.sub.2 is provided, are calculated in a third summing element 5 and added to control neuron 2 as control neuron bias value b.sub.c. Alternatively, the sum of the weightings may be calculated in advance, since the bias values b.sub.1, b.sub.2 are fixed after the training, and correspondingly provided from a suitable memory.

(15) In control neuron 2, the sum of the products is calculated from the control weightings w.sub.c,1, w.sub.c,2 with the respectively assigned neuron inputs and additively provided with a control neuron bias value b.sub.c in order to obtain a second comparison value o.sub.c.

(16) In a comparison block 6, first comparison value o.sub.n and second comparison value o.sub.c are added in order to obtain a comparison result V. No error is determined in the case of identical comparison values and an error is determined in the case of unequal comparison results V. In this way, it is possible to find errors in the calculation of a neuron value, the cause of the error potentially existing in the calculation hardware of a neuron or in the memories for the storing of the neuron parameters, such as weightings and bias value. In this way, a control neuron 2 may be used in order to recognize an error in one of the calculations of neurons 1.

(17) Another use of the above described control method is in the case of convolutional (folding) neural networks (CNN). These are frequently used for image processing. In general, any type of tensors and matrices having data points may be processed. For ease of understanding, the individual data points are referred to as pixels of a pixel matrix and the values of the data points are referred to as pixel values.

(18) To process a pixel matrix PM, which may correspond to the original data matrix or to a feature map generated from the data matrix, the feature map is scanned by a so-called kernel with the dimension D×D (frequently 3×3, 5×5 or 7×7), and corresponding pixel values processed by the kernel are written into a feature map MK formed therefrom. This means, each kernel forms a resultant pixel value of feature map MK from D×D pixel values of the pixel matrix.

(19) A kernel/filter is part of a CNN. For each calculation step D.sub.2, input values, which are multiplied by an assigned weighting, respectively and subsequently added, are added to a D×D kernel. A bias value assigned to the kernel is also added.

(20) A layer of a neural network is calculated by the repeated use of the kernel, in each case on one portion of the input data. The kernel is applied multiple times for this purpose along the x-dimension and y-dimension across pixel matrix PM. Compared to a conventional neural network, the multiple application of the kernel thus corresponds to a large quantity set of neurons, which are consistently applied with the same parameters.

(21) The pixel matrix is scanned at multiple positions set in x-dimension and y-dimension. By pre-assuming edge pixels, so-called paddings, the dimension of feature map MK may correspond to the dimension of original pixel matrix PM.

(22) This approach is illustrated in FIG. 3 for only one verification dimension and one single line/column of pixel matrix PM. In this case, kernel K includes the dimension D=3 and consistently processes adjoining input values i (at the edge areas, including a so-called padding P, in order to ensure for this exemplary embodiment that feature map MK has the same x-dimension and y-dimension as pixel matrix PM) to form a corresponding neuron value. Kernel K exhibits a constant parameterization per se (weightings and bias values) for the same pixel matrix PM and is moved across the entire pixel matrix for evaluating pixel groups of the dimension D=3, in order in each case to obtain a pixel value in feature map MK.

(23) To verify the calculation of feature map MK, a number of verification pixels UP.sub.1 . . . UP.sub.D, in addition to paddings P, is also added to the line and/or to the column (x-direction, y-direction) of pixel matrix PM, which corresponds to the dimension of kernel K in the corresponding matrix direction (x or y). Thus, when scanning pixel matrix PM, verification pixel UP in the exemplary embodiment shown is scanned last and from this a control pixel KP corresponding to the kernel function is determined. First control pixels KP1 may be situated at the beginning or at the end of a row or column of the matrix, may be assigned otherwise to the matrix or may also be provided embedded in the row or column.

(24) In this way, the resulting feature map is provided with a first control pixel KP at the end of each row and/or of each column of the verification dimension as first comparison value V1, which result from the processing of verification pixel UP by assigned kernel K. In addition, the sum of the regular pixel values of resulting feature map MK may be included in the relevant row or the relevant column, in each case as second comparison value V2. For the example of a dimension of the kernel of three shown in FIG. 3, the following sums result for the three verification pixels UP.sub.1, UP.sub.2, UP.sub.3 from pixel values i.sub.j of the lines of pixel matrix PM in question:

(25) ${\mathrm{UP}}_1=\underset{j=1}{\overset{4}{.Math.}}{i}_j{\mathrm{UP}}_2=\underset{j=1}{\overset{5}{.Math.}}{i}_j{\mathrm{UP}}_3=\underset{j=2}{\overset{5}{.Math.}}{i}_j{o}_1=0.Math.k_1+i_1.Math.k_2+i_2.Math.k_3{o}_2=i_1.Math.k_1+i_2.Math.k_2+i_3.Math.k_3{o}_3=i_2.Math.k_1+i_3.Math.k_2+i_4.Math.k_3{o}_4=i_3.Math.k_1+i_4.Math.k_2+i_5.Math.k_3{o}_5=i_4.Math.k_1+i_5.Math.k_2+0.Math.k_{3}V1={\mathrm{UP}}_1.Math.k_1+{\mathrm{UP}}_2.Math.k_2+{\mathrm{UP}}_3.Math.k_{3}V2=\underset{j=1}{\overset{5}{.Math.}}{o}_j\underset{j=1}{\overset{5}{.Math.}}{o}_j=\underset{j=1}{\overset{4}{.Math.}}{i}_j.Math.k_1+\underset{j=1}{\overset{5}{.Math.}}{i}_j.Math.k_2+\underset{j=2}{\overset{5}{.Math.}}{i}_j.Math.k_3$

(26) Normally, the pixel matrix and the feature map correspond to a p-dimensional tensor and may be considered as a set of pixels in p-dimensional space. For better illustration, a three-dimensional pixel matrix is assumed below. As illustrated in FIG. 4, such a three-dimensional pixel matrix is made up of multiple channels, for example, color channels RGB. Pixel matrix PM may then be considered as a set of pixels in a three-dimensional space. Pixels having a different x-coordinate and y-coordinate, but the same z-coordinate belong to one channel. In turn, each of the n kernels in this case generates a two-dimensional feature map and all n feature maps concatenated result in a three-dimensional feature map having n channels.

(27) In the exemplary embodiment shown in FIG. 4, a pixel matrix having m channels is indicated. Also indicated are n kernels, each of which are made up of m.Math.D.sup.2 weightings and, if necessary, a bias value. The kernels have the same dimensions. A feature map having n channels is generated from these n kernels via the convolution of the pixels of the pixel matrix with kernel K, in each case assigned to the channel.

(28) The pixel matrix having the n channels results in:

(29) $o_l\left(r,s\right)=\underset{z=1}{\overset{m}{.Math.}}\underset{x=1}{\overset{3}{.Math.}}\underset{y=1}{\overset{3}{.Math.}}i\left(r+x-a,s+y-a,z\right).Math.w_l\left(x,y,z\right)$
i referring to a three-dimensional pixel matrix, i(x,y,z) referring to a pixel value of the three-dimensional pixel matrix, and w.sub.1(x,y,z) referring to the weightings of kernel k.sub.1.

(30) For the purpose of functional verification, the feature map may include one or multiple control channels, which include first comparison values o.sub.Σ(r,s) for each point r,s determined by the x-coordinate and the y-coordinate. These are calculated as a pixel-wise sum of the feature map

(31) $o_{.Math.}\left(r,s\right)=\underset{l=1}{\overset{n}{.Math.}}{o}_l\left(r,s\right)=\underset{l=1}{\overset{n}{.Math.}}\underset{z=1}{\overset{m}{.Math.}}\underset{x=1}{\overset{3}{.Math.}}\underset{y=1}{\overset{3}{.Math.}}i\left(r+x-a,s+y-a,z\right).Math.w_l\left(x,y,z\right)$
a=(D+1)/2 and, therefore, corresponding to the relative position of the tile relative to the assigned query point x,y.

(32) Also provided is a control kernel, which is ascertained by the weightings of the kernel as follows:

(33) $o_c\left(r,s\right)=\underset{z=1}{\overset{m}{.Math.}}\underset{x=1}{\overset{3}{.Math.}}\underset{y=1}{\overset{3}{.Math.}}i\left(r+x-a,s+y-a,z\right).Math.w_c\left(x,y,z\right)\mathrm{with}{w}_c\left(x,y,z\right)=\underset{l=1}{\overset{n}{.Math.}}{w}_l\left(x,y,z\right),\forall x,yϵ\left[1;3\right],\forall z\in \left[1;m\right]$

(34) Thus, a control kernel is formed for creating the control channel, the weightings of the neurons of the control kernel corresponding to the sum of the weightings of the relevant neurons of the individual kernels for the channels of the pixel matrices.

(35) Thus:

(36) $o_c\left(r,s\right)=\underset{z=1}{\overset{m}{.Math.}}\underset{x=1}{\overset{3}{.Math.}}\underset{y=1}{\overset{3}{.Math.}}i\left(r+x-a,s+y-a,z\right).Math.w_c\left(x,y,z\right)=\underset{z=1}{\overset{m}{.Math.}}\underset{x=1}{\overset{3}{.Math.}}\underset{y=1}{\overset{3}{.Math.}}i\left(r+x-a,s+y-a,z\right).Math.\underset{l=1}{\overset{n}{.Math.}}{w}_l\left(x,y,z\right)=\underset{l=1}{\overset{n}{.Math.}}\underset{z=1}{\overset{m}{.Math.}}\underset{x=1}{\overset{3}{.Math.}}\underset{y=1}{\overset{3}{.Math.}}i\left(r+x-a,s+y-a,z\right).Math.w_l\left(x,y,z\right)$

(37) The sum of all bias values of the kernels is added to the value o.sub.c(r,s) thus obtained as a control bias value.

(38) The application of the control kernel to the pixel matrix results in a feature map having a control channel with two comparison values V2 (o.sub.c). Second comparison values V2 are compared with first comparison values V1, in order to establish an error in the case of a disparity.

(39) For the purpose of error recognition, the weightings and the bias value of the control kernel are calculated so that each corresponds to the sum of the n existing kernel weightings and bias values. This ensures that in an error-free case, the sum of the n channels of the output feature map must correspond to the control channel.

(40) The method described above allows for the one-error-recognition. If the method is applied along the x-dimension or y-dimension of pixel matrix PM, the requisite redundancy corresponds to a dimension increase of the PM from (x,y,z) to (x+D,y,z) or (x,y+D,z). If the method is applied along the z-dimension, the requisite redundancy corresponds to a dimension increase of the PM of (x,y,z) to (x,y,z+1). Moreover, the number of the control pixels for the x-dimension and the y-dimension or the number of control channels for the z-dimension are freely selectable, as a result of which an error correction may also be enabled.

(41) For the purpose of error correction, the erroneously calculated pixel value must be identified on the one hand, and the correct pixel value must be restored on the other hand. With suitably selected control kernels or control pixels, it is possible to identify the erroneous pixel value. Restoration of the correct pixel value is enabled by subtracting the correctly calculated pixel values from the erroneous control pixel. The number of the required control pixels for a one-error-correction is determined by:

(42) $\underset{i}{\min }\left(i+.Math.\frac{n}{i}.Math.\right)$
n corresponding to the number of pixels in the dimension in question of the feature map without control pixels. This means that with i control pixels, it is possible to limit the erroneously calculated value to a set of n/i pixels. With

(43) 0$.Math.\frac{n}{i}.Math.$
additional control pixels, each of which compares the j value of the row, it is possible to identify the exact position of the erroneous pixel.