DEVICE AND METHOD FOR RECOGNIZING IMAGE USING BRAIN-INSPIRED SPIKING NEURAL NETWORK AND COMPUTER READABLE PROGRAM FOR THE SAME

Abstract

Disclosed are an image recognition device and method using a brain-inspired spiking neural network and a computer-readable program for the same. The image recognition device using a brain-inspired spiking neural network according to the present disclosure includes an input unit configured to receive an input image made up of at least one pixel, and a spiking neural network unit configured to recognize the input image, the spiking neural network unit including a plurality of neurons corresponding to the pixels of the image to generate spike signals when a membrane potential state value exceeds a preset threshold, and synapses connecting the plurality of neurons.

Claims

1. An image recognition device using a brain-inspired spiking neural network, comprising: an input unit configured to receive an input image made up of at least one pixel; and a spiking neural network unit configured to recognize the input image, the spiking neural network unit including a plurality of neurons corresponding to the pixels of the image to generate spike signals when a membrane potential state value exceeds a preset threshold, and synapses connecting the plurality of neurons.

2. The image recognition device using a brain-inspired spiking neural network according to claim 1, further comprising: an encoding unit configured to perform neural coding to provide to the plurality of neurons based on a luminance of the pixels in the image, wherein the encoding unit performs firing rate coding to determine a firing rate of the spike signals according to the luminance, and spike timing coding to determine a spike timing according to the luminance.

3. The image recognition device using a brain-inspired spiking neural network according to claim 2, wherein the spiking neural network unit includes: an input layer configured to receive the input neural code by assigning one neuron to each pixel of the image; a hidden layer configured to receive the signals from some of the plurality of neurons of the input layer, the hidden layer including an excitatory hidden layer containing excitatory neurons and an inhibitory hidden layer containing inhibitory neurons; and an output layer configured to receive the signals from some of the plurality of neurons of the excitatory hidden layer, the output layer including an excitatory output layer containing excitatory neurons and an inhibitory output layer containing inhibitory neurons.

4. The image recognition device using a brain-inspired spiking neural network according to claim 3, wherein the input layer is connected to each of the excitatory hidden layer and the inhibitory hidden layer via excitatory synapses, wherein the excitatory hidden layer and the inhibitory hidden layer are interconnected via the excitatory synapses and inhibitory synapses on a same layer, wherein the excitatory hidden layer is connected to each of the excitatory output layer and the inhibitory output layer via the excitatory synapses, and wherein the excitatory output layer and the inhibitory output layer are interconnected via the excitatory synapses and the inhibitory synapses on a same layer.

5. The image recognition device using a brain-inspired spiking neural network according to claim 4, wherein the inhibitory neurons include parvalbumin (PV) expressing inhibitory neurons and somatostatin (SST) expressing inhibitory neurons.

6. The image recognition device using a brain-inspired spiking neural network according to claim 2, wherein the firing rate coding calculates a ratio of a luminance value of a present pixel to a mean luminance value of the pixels in the image as the firing rate and performs neural coding to increase the firing rate with the increasing luminance, and wherein the spike timing coding performs neural coding to make the spike timing earlier with the increasing luminance by subtracting a percentage of the luminance value of the present pixel to a maximum luminance value from a preset reference spike timing.

7. The image recognition device using a brain-inspired spiking neural network according to claim 4, further comprising: a learning unit configured to modify synaptic weights by applying a Spike Timing-Dependent plasticity (STDP) learning rule to the synapses between the excitatory neurons to allow the neurons of the output layer to selectively generate the spike signals according to the image.

8. The image recognition device using a brain-inspired spiking neural network according to claim 7, wherein the learning unit includes a supervised learning unit configured to determine a target neuron of the output layer according to the image, and induce synaptic potentiation or depression by the STDP learning rule through a rise or fall in membrane potential of the target neuron to allow the target neuron to generate the spike signals.

9. The image recognition device using a brain-inspired spiking neural network according to claim 7, wherein the learning unit includes an unsupervised learning unit configured to modify the synaptic weights according to the STDP learning rule based on the output spike signals from the output layer according to the image.

10. An image recognition method in an image recognition device using a brain-inspired spiking neural network, the method comprising: receiving, by an input unit, an input image made up of at least one pixel; and recognizing, by a spiking neural network unit, the input image, the spiking neural network unit including a plurality of neurons corresponding to the pixels of the image to generate spike signals when a membrane potential state value exceeds a preset threshold, and synapses connecting the plurality of neurons.

11. The image recognition method using a brain-inspired spiking neural network according to claim 10, further comprising: performing, by an encoding unit, neural coding to provide to the plurality of neurons based on a luminance of the pixels in the image, wherein the encoding step comprises performing firing rate coding to determine a firing rate of the spike signals according to the luminance, and spike timing coding to determine a spike timing according to the luminance.

12. The image recognition method using a brain-inspired spiking neural network according to claim 11, wherein the spiking neural network unit includes: an input layer configured to receive the input neural code by assigning one neuron to each pixel of the image; a hidden layer configured to receive the signals from some of the plurality of neurons of the input layer, the hidden layer including an excitatory hidden layer containing excitatory neurons and an inhibitory hidden layer containing inhibitory neurons; and an output layer configured to receive the signals from some of the plurality of neurons of the excitatory hidden layer, the output layer including an excitatory output layer containing excitatory neurons and an inhibitory output layer containing inhibitory neurons.

13. The image recognition method using a brain-inspired spiking neural network according to claim 12, further comprising: learning, by a learning unit, by modifying synaptic weights by applying a Spike Timing-Dependent plasticity (STDP) learning rule to the synapses between the excitatory neurons to allow the neurons of the output layer to selectively generate the spike signals according to the image.

14. The image recognition method using a brain-inspired spiking neural network according to claim 13, wherein the learning step comprises: supervised learning to determine a target neuron of the output layer according to the image, and induce synaptic potentiation or depression by the STDP learning rule through a rise or fall in membrane potential of the target neuron to allow the target neuron to generate the spike signals, or unsupervised learning to modify the synaptic weights according to the STDP learning rule based on the output spike signals from the output layer according to the image.

15. A computer-readable program stored in a computer-readable recording medium configured to perform the image recognition method using a brain-inspired spiking neural network defined in claim 10.

Description

DESCRIPTION OF DRAWINGS

[0027] FIG. 1 is a diagram showing an example of application of an image recognition device using a brain-inspired spiking neural network according to an embodiment of the present disclosure.

[0028] FIG. 2 is a block diagram showing a detailed configuration of the image recognition device using a brain-inspired spiking neural network shown in FIG. 1.

[0029] FIG. 3 is a diagram showing a detailed configuration of a spiking neural network unit shown in FIG. 2.

[0030] FIGS. 4A to 4C are diagrams showing a computational model based on experimental data of excitatory neurons and inhibitory neurons shown in FIG. 3.

[0031] FIGS. 5A and 5B are diagrams showing synapses and a synaptic plasticity computational model based on experimental data of inhibitory neurons shown in FIG. 3.

[0032] FIGS. 6A and 6B are diagrams showing an example of a result of neural coding by an encoding unit shown in FIG. 2.

[0033] FIGS. 7A and 7B are diagrams showing the concept of supervised learning and unsupervised learning in a learning unit shown in FIG. 2.

[0034] FIG. 8 is a flowchart showing an image recognition method using a brain-inspired spiking neural network according to another embodiment of the present disclosure.

BEST MODE

[0035] The following detailed description of the present disclosure is made with reference to the accompanying drawings, in which particular embodiments for practicing the present disclosure are shown for illustrative purposes. These embodiments are described in sufficient detail for those skilled in the art to practice the present disclosure. It should be understood that various embodiments of the present disclosure are different but do not need to be mutually exclusive. For example, particular shapes, structures and features described herein in connection with one embodiment may be embodied in other embodiment without departing from the spirit and scope of the present disclosure. It should be further understood that changes may be made to the positions or placement of individual elements in each disclosed embodiment without departing from the spirit and scope of the present disclosure. Accordingly, the following detailed description is not intended to be taken in limiting senses, and the scope of the present disclosure, if appropriately described, is only defined by the appended claims along with the full scope of equivalents to which such claims are entitled. In the drawings, similar reference signs indicate same or similar functions in many aspects.

[0036] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.

[0037] FIG. 1 is a diagram showing an example of application of an image recognition device using a brain-inspired spiking neural network according to an embodiment of the present disclosure, FIG. 2 is a block diagram showing a detailed configuration of the image recognition device using a brain-inspired spiking neural network shown in FIG. 1, FIG. 3 is a diagram showing a detailed configuration of a spiking neural network unit shown in FIG. 2, FIGS. 4A to 4C are diagrams showing a computational model based on experimental data of excitatory neurons and inhibitory neurons shown in FIG. 3, FIGS. 5A and 5B are diagrams showing synapses and a synaptic plasticity computational model based on experimental data of inhibitory neurons shown in FIG. 3, FIGS. 6A and 6B are diagrams showing an example of a result of neural coding by an encoding unit shown in FIG. 2, and FIGS. 7A and 7B are diagrams showing the concept of supervised learning and unsupervised learning in a learning unit shown in FIG. 2.

[0038] As shown in FIG. 1, the image recognition device 100 using a brain-inspired spiking neural network according to this embodiment may be used to classify digits in each digit image upon receiving an input of a random digit image dataset. To this end, the image recognition device 100 according to this embodiment may be implemented as a computer. Here, the computer includes, for example, a server, a notebook computer, a desktop computer and a laptop computer having a web browser installed, but is not limited thereto, and any device that can operate according to a pre-stored computer program upon receiving arbitrary input data may be used for the image recognition device 100.

[0039] Hereinafter, the detailed configuration of the image recognition device will be described with reference to FIGS. 2 to 7.

[0040] The image recognition device 100 using a brain-inspired spiking neural network according to this embodiment includes an input unit 110, an encoding unit 120, a spiking neural network unit 130, a learning unit 140 and an evaluation unit 150.

[0041] The input unit 110 receives an input image made up of at least one pixel from an external device. The input unit 110 according to this embodiment may receive the input digit image, but is not limited thereto, and the input image received by the input unit 110 may include any image made up of at least one pixel that may be classified into at least one class by the spiking neural network unit 130.

[0042] Meanwhile, the input unit 110 may receive the input of the Modified National Institute of Standards and Technology (MNIST) dataset. Here, the MNIST dataset refers to the handwritten dataset for digit image classification, and is the typical benchmark dataset in the field of machine learning. Each digit image included in the MNIST dataset may be used to classify the images of digits through the spiking neural network unit 130. For example, the input unit 110 according to this embodiment may receive the input of the MNIST dataset including a total of 70,000 handwritten digit images, and 60,000 images in the MNIST dataset may be used for the learning unit 140 to train the spiking neural network unit 130, and the remaining 10,000 images may be used for the evaluation unit 150 to evaluate the trained spiking neural network unit 130.

[0043] The encoding unit 120 performs neural coding based on the luminance of the pixels in the image. The neural code may be input to a single neuron assigned to each pixel. That is, the encoding unit 120 may performs neural coding to provide to the plurality of neurons.

[0044] In particular, the encoding unit 120 according to this embodiment may perform firing rate coding to determine a firing rate of a spike signal according to the luminance, and spike timing coding to determine spike timing according to the luminance.

[0045] First, the firing rate coding may perform neural coding as shown in the following Equation 1.

[00001]$\text{Firing}\text{rate}=\frac{\text{The}\text{luminance}\text{value}\text{of}\text{a}\text{pixel}}{\text{The}\text{mean}\text{luminance}\text{value}}$

Here, the firing rate denotes the firing rate of the individual neuron, the luminance value of a pixel denotes the luminance value of the corresponding pixel assigned to the individual neuron, and the mean luminance value denotes the mean luminance value in the entire image.

[0046] That is, the firing rate coding calculates the ratio of the luminance value of the present pixel to the mean luminance value of the pixels in the image as the firing rate and performs neural coding such that the firing rate increases with the increasing luminance, and as shown in FIG. 6A, since the luminance value of the pixel corresponding to the digit in the given input image is high, encoding is performed to increase the firing rate of the neuron associated with the corresponding pixel.

[0047] The spike timing coding may perform neural coding as shown in the following Equation 2.

[00002]$\text{Spike}\text{timing}=\text{Base}\text{delay}-100*\frac{\text{The}\text{luminance}\text{value}\text{of}\text{a}\text{pixel}}{\text{The}\text{maximum}\text{luminance}\left(\text{255}\right)}$

[0048] Here, the spike timing denotes the spike timing of the individual neuron, the base delay denotes the reference spike timing, the luminance value of a pixel denotes the luminance value of the corresponding pixel of the individual neuron, and the maximum luminance denotes the maximum luminance value of the individual pixel in the whole dataset. For example, the reference spike timing may be set to 100 and the maximum luminance value may be set to 255, but is not limited thereto.

[0049] That is, the spike timing coding performs neural coding to make the spike timing earlier with the increasing luminance by subtracting the percentage of the luminance value of the present pixel to the maximum luminance value from the preset reference spike timing, and as shown in FIG. 6B, since the luminance value of the pixel corresponding to the digit in the given input image is high, encoding is performed for the earlier spike timing of the neuron associated with the corresponding pixel.

[0050] The spiking neural network unit 130 receives the input neural code and recognizes and classifies the input image from the input unit 110.

[0051] To this end, the spiking neural network unit 130 may be a brain-inspired spiking neural network including a plurality of neurons that corresponds to the pixels of the input image and generates spike signals when a membrane potential state value exceeds a preset threshold, and synapses connecting the plurality of neurons.

[0052] Meanwhile, the spiking neural network (SNN) is a neural network that is closer to natural neural networks to mimic the biological brain than the existing neural networks, and processes information by generating the spike signals when the membrane potential state value of each nerve cell or neuron exceeds the threshold by the summation of presynaptic inputs. The spiking neural network may include the temporal concept in the input value.

[0053] First, the spiking neural network according to this embodiment is a model of neurons or nerve cells based on the Hodgkin-Huxley model, and as shown in FIGS. 4A to 4C, to simulate the intrinsic properties and spiking patterns measured through experiments in the cortical area of the biological brain, may define each neuron model of excitatory neurons, parvalbumin (PV) expressing inhibitory neurons and somatostatin (SST) expressing inhibitory neurons.

[0054] First, the equation for determining the membrane potential state value according to the electrical conductivity of ion channels of these neuron models follows the following Equation 3.

[00003]$\begin{array}{l}\frac{C_{m}d{V}_m}{dt}=-\left(I_{leak}+I_{Na}+I_{Kd}+I_{Ka}+I_{Km}+I_h\right)\\ I_{ion}=g_{ion}\left(V_m-E_{ion}\right),ion\in \left(leak,Na,Ka,Ka,Km,h\right)\end{array}$

[0055] Here, C.sub.mdV.sub.m is the total current passing through the membrane, V.sub.m is the resting potential, I.sub.leak is the leakage current, I.sub.Na is the current passing through the sodium channel, I.sub.Kd is the current passing through the delayed rectifier potassium channel, I.sub.Ka is the current passing through the A-type potassium channel, I.sub.Km is the current passing through the M-type potassium channel, I.sub.h is the current passing through the h-current channel, g.sub.ion is the conductance of the given ion channel, and E.sub.ion is the reversal potential of the corresponding ion channel.

[0056] The parameter setting values for defining each of excitatory neurons EX, PV expressing inhibitory neurons IN.sub.PV and SST expressing inhibitory neurons IN.sub.SST follow the following Table 1.

TABLE-US-00001 EX IN.sub.PV IN.sub.SST V.sub.m(mV) -63.85 -59.51 -57.19 V.sub.m(mV) -41.77 -41.45 -52.70 R (MΩ) 98.8 88.8 215.8 ISI.sub.initial (ms) 24.22 13.60 10.55 ISI.sub.final (ms) 32.10 13.99 24.07 Spike adapataion ratio 0.75 0.98 0.44 (ISI.sub.initial/ISI.sub.final) g.sub.leak(.Math.S/cm.sup.2) 76.9 200 76.9 g.sub.Na(mS/cm.sup.2) 50.0 65.0 70.0 g.sub.Kd(mS/cm.sup.2) 6.0 10.0 3.5 g.sub.Ka(mS/cm.sup.2) 1.0 1.0 0.5 g.sub.Km(.Math.S/cm.sup.2) 20 5 138 g.sub.h(.Math.S/cm.sup.2) 10 50 10 E.sub.leak(mV) -64.0 -60.0 -57.0 E.sub.Na(mV) 50.0 E.sub.Kd(mV) -77.0 E.sub.Ka.sub.s(mV) -77.0 E.sub.Km (mV) -82.3 E.sub.h(mV) 0

[0057] The spiking neural network according to this embodiment is based on the double exponential synapse model that describes the activity of the synapse as a link between neurons based on two exponential terms, and as shown in FIGS. 5A and 5B, follows the synapses and the synapse model that simulates short term synaptic plasticity measured through experiments in the cortical area of the biological brain.

[0058] The synapse model defines the equation for determining the electric current to be provided to the postsynaptic neuron by the synaptic response as shown in the following Equation 4.

[00004]$\begin{array}{l}{I}_{syn}\left(t\right)=g_{max}\times factor\times \left(-e\frac{t}{t_{rise}}+-e\frac{t}{t_{decay}}\right)\times \left(V_m-E_{syn}\right)\hfill \\ factor=1/\left(-e\frac{t_p}{t_{rise}}+-e\frac{t_p}{t_{decay}}\right)\hfill \\ t_p=t_{rise}\times t_{decay}\times log\left(t_{rise}/t_{decay}\right)/\left(t_{rise}-t_{decay}\right)\hfill \end{array}$

[0059] Here, I.sub.s.sub.yn denotes the total synaptic current, g.sub.max denotes the maximum value of the synaptic conductance, factor denotes the normalized factor, τ.sub.rise denotes the rise time constant, τ.sub.decay denotes the decay time constant, V.sub.m denotes the membrane potential, and E.sub.syn denotes the reversal potential.

[0060] Additionally, the parameters to be applied to the synapses connecting the above-defined excitatory neurons EX, PV expressing inhibitory neurons IN.sub.PV and SST expressing inhibitory neurons IN.sub.SST may be defined as shown in the following Table 2.

TABLE-US-00002 EX-EX synapse EX-IN.sub.pv synapse EX-IN.sub.SST synapse IN.sub.pv-EX or IN.sub.SST-EX synapse IN.sub.pvIN.sub.pv synapse Amplitude (mV) 0.95 2.00 0.40 -0.40 -1.00 g.sub.max (mS/cm.sup.2) 2.390 4.980 0.487 0.777 1.500 τ.sub.rise (ms) 0.2 0.1 0.2 1.0 0.5 τ.sub.decay(ms) 0.4 0.2 0.4 4.0 2.0 E.sub.syn (mV) 0 0 0 -80 -80

[0061] The synapse model according to this embodiment also defines changes in synaptic response by the short term synaptic plasticity as shown in the following Equation 5.

[00005]$\begin{array}{l}\begin{array}{l}\\ \\ I_{sym}\left(t\right)=A\times g_{max}\times factor\times \left(-e^{-t/t_{rise}}+e^{-t/t_{decay}}\right)\times \left(V_m-E_{sym}\right),\end{array}\hfill \\ A=A_{0}DF,\hfill \\ D.fwdarw.Dd,t_{P}dD/dt=1-D,\hfill \\ F.fwdarw.F+f,t_{p}dF/dt=1-F.\hfill \end{array}$

[0062] Here, A denotes the ratio of synapse response amplitudes, A.sub.o denotes the initial response amplitude, D denotes the dynamic variable representing depression, d denotes the depression factor, τ.sub.d denotes the depression time constant, F denotes the dynamic variable representing facilitation, f denotes the facilitation factor, τ.sub.F denotes the facilitation time constant, I.sub.syn denotes the total synaptic current, g.sub.max denotes the maximum value of the synaptic conductance, factor denotes the normalized factor, τ.sub.decaydenotes the decay time constant, V.sub.m denotes the membrane potential, and E.sub.syn denotes the reversal potential.

[0063] Additionally, the parameters to be applied to the synapses as a link between the above-defined excitatory neurons EX, PV expressing inhibitory neurons IN.sub.pv and SST expressing inhibitory neurons IN.sub.SST may be defined as shown in the following Table 3.

TABLE-US-00003 EX-IN.sub.pv synapse EX-IN.sub.SST synapse d 0.74 0.70 τ.sub.D (ms) 100 50 ƒ - 1.40 τ.sub.p (ms) - 250

[0064] The spiking neural network unit 130 builds a spiking neural network having a plurality of layers based on the neuron model and the synapse model as described above.

[0065] As shown in FIG. 3, the spiking neural network unit 130 includes an input layer 10 containing a plurality of neurons, each neuron assigned to each pixel of the input image, hidden layers 21, 22 which receive signals from some of the plurality of pixels of the input layer 10, and output layers 31, 32 which produce output signals to the signals received from some of the plurality of neurons of the hidden layers 21, 22.

[0066] More specifically, the input layer 10 contains the number of neurons corresponding to the number of pixels of the input image. Each neuron of the input layer 10 receives the input of the neural code corresponding to the corresponding pixel. For example, as shown in FIG. 3, when the input image has 784 pixels, the input layer may contain 784 neurons which is equal to the number of pixels.

[0067] The hidden layers 21, 22 include the excitatory hidden layer 21 containing excitatory neurons and the inhibitory hidden layer 22 containing inhibitory neurons. Each of the excitatory hidden layer 21 and the inhibitory hidden layer 22 is connected to the input layer 10 via excitatory synapses. Accordingly, the excitatory hidden layer 21 and the inhibitory hidden layer 22 receive the signals from some of the plurality of neurons of the input layer 10, and preferably, may receive the signals from about 20% of the neurons of the input layer 10. Additionally, the excitatory hidden layer 21 and the inhibitory hidden layer 22 are interconnected via the excitatory synapses and the inhibitory synapses on the same layer.

[0068] The output layers 31, 32 include the excitatory output layer 31 containing excitatory neurons and the inhibitory output layer 32 containing inhibitory neurons. Each of the excitatory output layer 31 and the inhibitory output layer 32 may be connected to the excitatory hidden layer 21 via the excitatory synapses, and the excitatory output layer and the inhibitory output layer may be interconnected via the excitatory synapses and the inhibitory synapses on the same layer.

[0069] As described above, the spiking neural network unit 130 according to this embodiment does not use lateral inhibition, and uses the separate layer containing inhibitory neurons on the same layer to induce the specific excitatory neurons to selectively respond to the specific input image, thereby building the spiking neural network that is closer to the biological brain.

[0070] The learning unit 140 modifies the synaptic weights by applying the Spike Timing-Dependent plasticity (STDP) learning rule to the synapses between the excitatory neurons to allow the neurons of the output layer to selectively generate the spike signals according to the input image.

[0071] Here, the STDP learning rule updates the synaptic weight using a time difference between the spike signal that the presynaptic neuron sends and the spike signal that the postsynaptic neuron sends in the similar way to the human body. For example, when the time difference between the time when the presynaptic neuron sends the spike signal and the time when the postsynaptic neuron sends the spike signal is larger than ‘0’, the weight is increased according to Long Term Potentiation (LTP), and when the time difference is smaller than ‘0’, the weight is decreased according to Long Term Digression (LTD). The values of the synaptic weights to change according to LTP and LTD may be defined as the following Equation 6 over Δt.

[00006]$\frac{dW}{dt}=\left\{\begin{array}{l}p\times \left(\text{wmax}-\text{W}\right)\times ex{p}^{\frac{-\Delta t}{{\tau }_p}},0<\Delta t\le 100\hfill \\ d\times \left(\text{W}-\text{wmin}\right)\times ex{p}^{\frac{-\Delta \left|t\right|}{{\tau }_d}},\Delta t<0or\Delta t>100\hfill \end{array}\right)$

[0072] Here, W denotes the synaptic weight, p denotes the potentiation factor, d denotes the depression factor, wmax denotes the maximum value, wmin denotes the minimum value, τ.sub.p and τ.sub.d denote the efficiency time constant of potentiation/depression, and Δt denotes the time difference between presynaptic and postsynaptic spikes. Meanwhile, the learning unit 140 according to this embodiment learns with the parameters p set to 0.1, d to 0.1, wmax to 1, wmin to -1, and τ.sub.p and τ.sub.d to 20, but is not limited thereto.

[0073] That is, the learning unit 140 updates the synaptic weights by adding the change value in weight determined by the above-described STDP learning rule to the previous weight values.

[0074] Meanwhile, the learning unit 140 according to this embodiment may include a supervised learning unit to determine a target neuron of the output layer according to the image, and induce synaptic potentiation or depression by the STDP learning rule through the rise or fall in membrane potential of the target neuron to allow the target neuron to generate the spike signal. That is, when a digit image is provided as the input image as shown in FIG. 7A, the supervised learning unit determines the target neuron 2 on the output layer that appropriately classifies the corresponding digit image, and induces synaptic potentiation or depression for the corresponding target neuron through spike induction by the rise or fall in membrane potential of the corresponding target neuron 2. More specifically, the supervised learning unit may increase the probability of generating spikes by increasing the membrane potential of the target neuron or reduce the probability of generating spikes by reducing the membrane potential of other neuron.

[0075] Additionally, the learning unit 140 may include an unsupervised learning unit to modify the synaptic weights by the STDP learning rule based on the output spike signals from the output layer according to the image. That is, as shown in FIG. 7B, the unsupervised learning unit learns by regarding spike signals naturally occurring in the neural network according to the input image as correct answer signals without intervention such as induction or inhibition of spikes for specific neurons on the output layer, and modifying the synaptic weights according to the STDP learning rule by the spike signals.

[0076] The evaluation unit 150 evaluates the output result by applying the test image to the spiking neural network unit 130 trained by the learning unit 140. For example, the evaluation unit 150 may evaluate the performance of the corresponding spiking neural network unit 130 based on the digit classification results obtained from the input of 10,000 digit images from the input unit 110 to the spiking neural network unit 130.

[0077] Meanwhile, the evaluation unit 150 may differently evaluate the performance depending on the learning method of the learning unit 140.

[0078] More specifically, the evaluation unit 150 may evaluate the performance of the spiking neural network unit 130 trained by the supervised learning unit based on whether the number of the neuron generating the spike signal first among the neurons on the output layer is consistent with the digit on the image.

[0079] Additionally, the evaluation unit 150 may evaluate the performance of the spiking neural network unit 130 trained by the unsupervised learning unit by regarding the number of the neuron generating the spike signal first among the neurons on the output layer as the correct answer for the image.

[0080] FIG. 8 is a flowchart showing an image recognition method using a brain-inspired spiking neural network according to another embodiment of the present disclosure.

[0081] The image recognition method using a brain-inspired spiking neural network according to this embodiment is the image recognition method in the image recognition device using a brain-inspired spiking neural network, and includes receiving, by the input unit, the input image made up of at least one pixel (S10), performing, by the encoding unit, neural coding to provide to the plurality of neurons based on the luminance of the pixels in the image (S20), training, by the learning unit, the spiking neural network unit (S30), recognizing, by the trained spiking neural network unit, the input image (S40) and evaluating, by the evaluation unit, based on the recognized image (S50).

[0082] Meanwhile, the step S20 of encoding may include performing firing rate coding to determine the firing rate of the spike signals according to the luminance and spike timing coding to determine the spike timing according to the luminance.

[0083] Additionally, the spiking neural network unit includes the plurality of neurons corresponding to the pixels of the image to generate the spike signals when the membrane potential state value exceeds the preset threshold, and the synapses connecting the plurality of neurons.

[0084] More specifically, the spiking neural network unit may include the input layer to receive the input neural code by assigning one neuron to each pixel of the image, the hidden layer that receives the signals from some of the plurality of neurons of the input layer, the hidden layer including the excitatory hidden layer containing excitatory neurons and the inhibitory hidden layer containing inhibitory neurons, and the output layer that receives the signals from some of the plurality of neurons of the excitatory hidden layer, the output layer including the excitatory output layer containing excitatory neurons and the inhibitory output layer containing inhibitory neurons.

[0085] Additionally, the step S30 of training may include modifying the synaptic weights by applying the STDP learning rule to the synapses between the excitatory neurons to allow the neurons of the output layer to selectively generate the spike signals according to the image.

[0086] More specifically, the step S30 of training may include determining the target neuron of the output layer according to the image, and performing supervised learning that induces synaptic potentiation or depression by the STDP learning rule through the rise or fall in membrane potential of the target neuron to allow the target neuron to generate the spike signals, or unsupervised learning that modifies the synaptic weights by the STDP learning rule based on the output spike signals from the output layer according to the image.

[0087] The others are the same as the description in FIGS. 1 to 7, and its description is omitted.

[0088] According to the present disclosure described above, it is possible to use the learning method that can be applied to the spiking neural network using the STDP learning rule that is seen as the learning mechanism of the biological brain rather than error backpropagation learning and reduce the computational complexity for learning.

[0089] Additionally, the integrated framework for supervised and unsupervised learning is created so that supervised learning solves the lack in biological plausibility and unnatural application problems based on STDP, and unsupervised learning can determine the final output only using the computation results of the neural network without an additional external module.

[0090] Additionally, since the encoding method for providing as inputs to the spiking neural network employs not only firing rate coding but also spike timing coding, it is possible to reduce the computational complexity necessary for the learning and classification tasks of the neural network and increase the coordinated effect with various biology based mechanisms including STDP.

[0091] Additionally, in the spiking neural network architecture, lateral inhibition is not used and the separate layer containing inhibitory neurons having the same connectivity is used to induce the specific excitatory neurons to selectively respond to the specific input image, thereby building the spiking neural network that is closer to the biological brain.

[0092] The operation by the image recognition method using a brain-inspired spiking neural network according to the above-described embodiments may be, at least in part, incorporated into a computer program and recorded in a computer-readable recording medium. The computer-readable recording medium having a program recorded thereon for implementing the operation by the image recognition method using a brain-inspired spiking neural network according to embodiments includes any type of recording device in which computer-readable data is stored. Examples of the computer-readable recording medium include read-only memory (ROM), random-access memory (RAM), compact disc read-only memory (CD-ROM), magnetic tape, floppy disk and an optical data storage device. Additionally, the computer-readable recording medium is distributed over computer systems connected via a network, and may store and execute a computer-readable code in a distributed manner. Additionally, a functional program, code and a code segment for realizing this embodiment will be easily understood by persons having ordinary skill in the technical field to which this embodiment belongs.

[0093] While the present disclosure has been hereinabove described with reference to the embodiments, those skilled in the art will understand that various modifications and change may be made thereto without departing from the spirit and scope of the present disclosure defined in the appended claims.

Description of Reference Numbers

[0094] 110: Input unit [0095] 120: Encoding unit [0096] 130: Spiking neural network unit [0097] 140: Learning unit [0098] 150: Evaluation unit