Artificial neuron
11580374 · 2023-02-14
Assignee
Inventors
- Alain Cappy (Genech, FR)
- Francois Danneville (Lille, FR)
- Virginie Hoel (Lomme, FR)
- Christophe Loyez (Festubert, FR)
Cpc classification
G06N3/049
PHYSICS
International classification
G06N3/06
PHYSICS
Abstract
An artificial neuron including: a membrane capacitor; an input of an external synaptic excitation in current, the membrane capacitor integrating the input current; a negative-feedback impulse circuit, supplied by a power supply at a negative voltage between −200 mV and 0 mV and at a positive voltage between 0 mV and +200 mV, including: a bridge based on pMOS and nMOS transistors in series and linked by a midpoint to the membrane capacitor, the midpoint defining the output of the artificial neuron, at least one delay capacitor between the gate and the source of one of the transistors of the bridge, at least two CMOS inverters between the membrane capacitor and the gates of the transistors of the bridge.
Claims
1. An artificial neuron comprising: a capacitor as a membrane capacitor; an input as an external synaptic excitation current, the membrane capacitor integrating the input current; a negative feedback pulse circuit supplied by a power supply at a negative voltage between −200 mV and 0 mV and at a positive voltage between 0 mV and +200 mV, consisting of: a bridge based on PMOS and NMOS transistors in series and connected, at drains thereof, by a midpoint to the membrane capacitor, the midpoint defining an output of the artificial neuron; and at least one capacitor as a delay capacitor, between a gate and a source of one of the transistors of the bridge; only two CMOS inverters connected in a cascade manner, each including two transistors, an input of a first one of the two CMOS inverters being directly connected to the membrane capacitor and an output of the first one of the two CMOS inverters being connected to an input of a second one of the two CMOS inverters and to a gate of one of the transistors of the bridge, an output of the second one of the two CMOS inverters being connected to a gate of the other transistor of the bridge; or only three CMOS inverters, with two of the three CMOS inverters being connected in a cascade manner, each including two transistors, an input of first one of the three CMOS inverters being connected to the membrane capacitor and an output of the first one of the three CMOS inverters being connected to an input of a second one of the three CMOS inverters, an output of the second one of the three CMOS inverters being connected to a gate of one of the transistors of the bridge, an input of a third one of the three CMOS inverters being connected to the membrane capacitor and an output of the third one of the three CMOS inverters being connected to a gate of the other transistor of the bridge.
2. The artificial neuron as claimed in claim 1, charging of the membrane capacitor being provided by the PMOS transistor of the bridge and discharging thereof being provided by the NMOS transistor.
3. The artificial neuron as claimed in claim 1, comprising two delay capacitances, a delay capacitance connected to the PM.OS transistor being lower than a delay capacitance connected to the NMOS transistor, with optimum being zero.
4. The artificial neuron as claimed in claim 3, the delay capacitance connected to the NMOS transistor being greater than the membrane capacitance.
5. The artificial neuron as claimed in claim 1, operating in stable mode, the PMOS and NMOS transistors of the bridge having different conductance values.
6. The artificial neuron as claimed in claim 1, operating in relaxation oscillator mode, the PMOS and NMOS transistors of the bridge having conductance values.
7. The artificial neuron as claimed in claim 1, the negative feedback pulse circuit being supplied by the power supply with a negative voltage between −100 mV and −50 mV and the positive voltage between +50 and +100 mV.
8. The artificial neuron as claimed in claim 1, a difference of the positive voltage minus negative voltage causing a voltage gain of each CMOS inverter to be greater than or equal to 2.
9. The artificial neuron as claimed in claim 1, a threshold voltage of at least one of the CMOS inverters being different from 0 V.
10. The artificial neuron as claimed in claim 1, comprising a leak resistor parallel to the membrane capacitor.
11. The artificial neuron as claimed in claim 1, the transistors of the bridge being produced using FD-SOI (Fully Depleted Silicon On Insulator) technology using possibility of control by a substrate, allowing maximum current of the transistors to be controlled by a substrate electrode.
12. The artificial neuron as claimed in claim 1, comprising an additional excitation circuit integrating a potential of the membrane and reinjecting a current resulting from the integration into the membrane capacitor, which additional excitation circuit has the midpoint of the transistors of the bridge as an input and output and comprises a follower amplifier, a delay line, an integrating amplifier, and a transconductance allowing the current to be obtained that is to be reinjected into the membrane capacitor on the basis of an output voltage of the integrating amplifier.
13. The artificial neuron as claimed in claim 1, using a stochastic resonance phenomenon through reception of an external excitation of two different currents: a periodic current with insufficient amplitude for generating spikes and a random noise current.
14. An artificial central pattern generator, comprising at least two artificial neurons as claimed in claim 1, as a pre-neuron and a post-neuron respectively, and an inhibitory synapse.
15. The artificial central pattern generator as claimed in claim 14, the inhibitory synapse including first and second synaptic inputs and including two transistors connected in series by drains thereof, at least one of the transistors being of NMOS type controlled by a gate potential corresponding to the first synaptic input, a gate of a second one of the transistors corresponding to the second synaptic input being connected to an output of two inverters in series, an input of the first inverter being subject to a membrane potential of the pre-neuron, an output of the synapse corresponding to the source of the NMOS transistor being connected to an output potential of the post-neuron.
16. The artificial central pattern generator as claimed in claim 14, the inhibitory synapse including first and second synaptic inputs and including two transistors connected in series by drains thereof, at least one of the transistors being of NMOS type controlled by a gate potential corresponding to the first synaptic input, a gate of a second one of the transistors corresponding to the second synaptic input being connected to a gate of the NMOS transistor of the bridge of the pre-neuron, an output of the synapse corresponding to the source of the NMOS transistor being connected to the output potential of the post-neuron.
17. The artificial central pattern generator as claimed in claim 14, comprising a pre-neuron operating in a burst mode and a post-neuron operating in an oscillating mode, associated by an inhibitory synapse.
18. The artificial central pattern generator as claimed in claim 14, comprising two artificial neurons both operating in an oscillating mode, and mutually coupled by two inhibitory synapses, so that each of the neurons is both a pre- and post-neuron.
19. The artificial central pattern generator as claimed in claim 14, comprising two artificial neurons both operating in a burst mode, and mutually coupled by two inhibitory synapses, so that each of the neurons is both a pre- and post-neuron.
20. A neural network comprising at least two artificial neurons, a pre-neuron and a post-neuron respectively, each comprising: a capacitor as a membrane capacitor; an input as an external synaptic excitation current, the membrane capacitor integrating the input current; a negative feedback pulse circuit supplied by a power supply at a negative voltage between −200 mV and 0 mV and at a positive voltage Vd between 0 mV and +200 mV, comprising: a bridge based on PMOS and NMOS transistors in series and connected, at drains thereof, by a midpoint to the membrane capacitor, the midpoint defining an output of the artificial neuron; at least one capacitor as a delay capacitor, between a gate and a source of one of the transistors of the bridge; only two CMOS inverters connected in a cascade manner, each including two transistors, an input of the first inverter being connected to the membrane capacitor and an output of the first inverter being connected to an input of the second inverter and to a gate of one of the transistors of the bridge, an output of the second inverter being connected to a gate of the other transistor of the bridge; or only three CMOS inverters, with two of the inverters being connected in a cascade manner, each including two transistors, an input of the first inverter being connected to the membrane capacitor and an output of the first inverter being connected to an input of the second inverter, an output of the second inverter being connected to a gate of one of the transistors of the bridge, an input of the third CMOS inverter being connected to the membrane capacitor and an output of the third CMOS inverter being connected to a gate of the other transistor of the bridge, the pre-neuron and the post-neuron being connected together by a synaptic circuit, and the synaptic circuit including two inputs and including two transistors connected in series by drains thereof, at least one of the transistors being of NMOS type controlled by a gate potential corresponding to a first input of the synaptic circuit, a gate of a second one of the transistors corresponding to a second input of the synaptic circuit, an output of the synaptic circuit corresponding to a source of the NMOS transistor being connected to an output potential of the post-neuron.
21. The neural network as claimed in claim 20, the synaptic circuit corresponding to an excitatory synapse, wherein a second input of the synaptic circuit is connected to an output of an inverter, having a membrane potential of the pre-neuron as an input.
22. The neural network as claimed in claim 20, the synaptic circuit corresponding to an inhibitory synapse, wherein a second input of the synaptic circuit is connected to an output of two inverters in series, with an input of the first inverter being subject to a membrane potential of the pre-neuron.
23. The neural network as claimed in claim 20, the synaptic circuit corresponding to an inhibitory synapse, wherein the second input of the synaptic circuit is connected to a gate of the NMOS transistor of the bridge of the pre-neuron.
24. The neural network as claimed in claim 20, comprising two artificial neurons, including a first neuron oscillating at a higher frequency and a second neuron oscillating at lower frequency, the first neuron operating in a burst mode by being coupled to the second neuron by two synapses, one of which is an excitatory synapse from the first neuron to the second neuron, and the other one of which is an inhibitory synapse from the second neuron to the first neuron.
25. The neural network as claimed in claim 24, membrane and delay capacitances of the second neuron being at least 100 times larger than those of the first neuron.
26. A data processing method, wherein a neural network is used as defined in claim 20.
27. The method as claimed in claim 26, wherein an intrinsic thermal noise of the artificial neuron is injected at the input thereof.
28. The method as claimed in claim 27, wherein multiple frequency spikes at an output of the artificial neuron are applied to an input of an integrating circuit, an output of which is connected to a first input of an excitatory synapse, a second input thereof heing connected to the artificial neuron.
29. The method as claimed in claim 28, wherein the integrating circuit comprises an NMOS transistor, a source of which, corresponding to an output of the integrating circuit, is connected to a capacitor, a gate and drain of the transistor being connected together and corresponding to an input of the integrating circuit.
30. An artificial neuron comprising: a capacitor as a membrane capacitor; an input as an external synaptic excitation current, the membrane capacitor integrating the input current; a negative feedback pulse circuit supplied by a power supply at a negative voltage between −200 mV and 0 mV and at a positive voltage Vd between 0 mV and +200 mV, comprising: a bridge based on PMOS and NMOS transistors in series and connected, at drains thereof, by a midpoint to the membrane capacitor, the midpoint defining an output of the artificial neuron; at least one capacitor as a delay capacitor, between a gate and a source of one of the transistors of the bridge; only three CMOS inverters, with two of the inverters being connected in a cascade manner, each including two transistors, an input of the first inverter being connected to the membrane capacitor and an output of the first inverter being connected to an input of the second inverter, an output of the second inverter being connected to a gate of one of the transistors of the bridge, an input of the third CMOS inverter being connected to the membrane capacitor and an output of the third CMOS inverter being connected to a gate of the other transistor of the bridge.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The invention will be able to be better understood upon reading the following description of non-limiting embodiments thereof, and with reference to the appended drawings, in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
(26)
(27)
(28)
(29)
(30)
(31)
(32)
DETAILED DESCRIPTION OF THE INVENTION
(33)
(34) In
(35) The midpoint 9, defining the connection of the drains of the transistors of the bridge, is connected to a terminal of a membrane capacitor Cm. The other terminal of the membrane capacitor Cm is connected to ground 0 V. Otherwise, by way of a variation, this terminal may be connected either to Vs or to Vd.
(36) In this example, Vs=<0 and Vd>=0.
(37) A capacitor Ck is connected between Vs and the gate of the NMOS transistor 7. The terminal of the capacitor Ck connected to Vs otherwise may be connected to ground.
(38) A capacitor Cna is connected between Vd and the gate of the PMOS transistor 8. The terminal of the capacitor Cna connected to Vd otherwise may be connected to ground.
(39) I.sub.ex denotes the external excitation current, for example, originating from the synapses (not shown in
(40) When the membrane potential Cm reaches the threshold voltage of the first inverter 5, a corresponding potential is then transmitted, after a first inversion by the inverter 5, to the gate of the PMOS transistor 8, activating said transistor after a delay that is defined by the capacitance Cna. Thus, the membrane capacitor Cm charges via the open conduction channel of the PMOS. This charge corresponds to the rising edge of the output spike.
(41) When the threshold voltage of the second inverter 6 is reached, a corresponding potential is transmitted to the gate of the NMOS transistor 7, activating said transistor after a delay that is defined by the delay capacitance Ck, which delay is, in the example, considered to be longer than the delay for activating the PMOS, due to the selection of Ck>Cna. Thus, after having had time to charge, the membrane capacitor Cm begins to discharge when the conduction channel of the NMOS is opened. This discharge corresponds to the falling edge of the output spike.
(42) The power supply voltages Vd and Vs in this case are the respective equivalents of the Nernst potentials of sodium and potassium by analogy with biology.
(43) The PMOS 8 and NMOS 7 transistors of the bridge respectively represent the sodium and potassium channels.
(44) The delay capacitances Cna and Ck represent the time constants required to open the sodium and potassium channels, respectively, as suggested by the aforementioned Morris-Lecar model.
(45) In general, in accordance with biology, the channels K are slower than the channels Na, which leads to Ck>Cna. Furthermore, in the considered example, the conductance of the NMOS transistor 7 of the bridge is greater than that of the PMOS transistor 8, which induces a membrane resting potential that is close to Vs, when the excitation current I.sub.ex is zero.
(46) The capacitance Cna can be equal to 0, as shown in
(47)
(48) The transistors 8 and 7 of the bridge respectively correspond to the transistors denoted M3 and M6 in
(49) The inverter 5 is composed of two transistors M2 and M5, the gate widths of which are equal to 0.3 μm.
(50) The inverter 6 is composed of two transistors M1 and M4, the gate width of which is equal to 0.3 μm and 0.05 μm, respectively.
(51) The gate length of the transistors is 22 nm. The negative power supply voltage is −100 mV and the positive power supply voltage is +100 mV.
(52) The capacitances Ck and Cm equal 50 fF and 10 fF, respectively.
(53) The external excitation current is constant, equal to 30 pA.
(54)
(55)
(56) These wave forms of the membrane potential and of ionic currents are similar to those encountered in the living entity, as shown in
(57) A qualitative match of the wave forms between
(58) In a variation where Ck<Cna and where the PMOS transistor of the bridge is more conducting than the NMOS transistor, the resting potential of the membrane is close to Vd and the outputs of the two inverters shown in
(59) A Spice digital simulation circuit according to this third embodiment of the invention is shown in
(60)
(61)
(62) The Spice digital simulation in this example has been completed using the STM 28 nm FD-SOI components library.
(63) The power supply voltages Vs=−60 mV and Vd=60 mV in this example allow a peak-to-peak amplitude of 100 mV to be obtained. It is possible to obtain pulses at lower power supply voltages, but the peak-to-peak amplitude of 100 mV is then no longer reached.
(64) If the gate widths of the transistors of the bridge are equal, the PMOS transistor has a lower drain current than that of the NMOS transistor. Hence, the gate width of the PMOS transistor preferably is adjusted so as to balance these current values, with the gate width of the NMOS transistor being equal to 80 nm and that of the PMOS transistor being equal to 450 nm, for example.
(65)
(66)
(67)
(68)
(69) These results demonstrate that the excitation current I.sub.ex and the values of the capacitances Ck and Cm influence the frequency of the pulses, which demonstrates the flexibility of the neural circuit in the sense that these various parameters may be used to optimize output pulses.
(70) In
(71)
(72)
(73) It can be seen that the energy consumed per pulse hardly changes (from 1.1 to 3.2 fJ/pulse), in a quasi-linear manner, in the variation range [2 fF, 50 fF] of Cm.
(74)
(75) The curves of
(76) It can be seen that it is possible to obtain very low energy efficiency values, which may equal 0.3 fJ per pulse for Ck=5 fF and Cm=2 fF.
(77)
(78) The inputs of the inverters 10 and 11 are connected to the midpoint 9 of the bridge and to the membrane capacitor, and the input of the inverter 12 is connected to the output of the inverter 11.
(79) With some approximations, the output voltage V.sub.out of an inverter below the threshold, supplied in a symmetrical manner (Vs=−Vd), is provided by:
V.sub.out=−Vd tanh [V.sub.in/(nV.sub.t)+0.5 Ln(I.sub.n0/I.sub.p0)].
(80) Where V.sub.in is the input voltage of the inverter, I.sub.n0/I.sub.p0 is the ratio of the maximum currents of the NMOS and PMOS transistors, V.sub.t is the thermal potential k.sub.BT/q (with k.sub.B being the Boltzmann constant, T the temperature and q the charge of an electron) and n is the ideality coefficient, being greater than 1.
(81) The expression of V.sub.out shows that the maximum voltage gain of the inverter is −Vd/(nV.sub.t) and that the threshold voltage is provided by (−nV.sub.t/2) Ln(I.sub.n0/I.sub.p0).
(82) Therefore, the voltage gain that is provided by the inverters with the power supply voltage may be adjusted, and the threshold voltage may be modified by several nV.sub.t by adjusting maximum currents of the NMOS and PMOS transistors (by modifying the gate width W of the transistors, for example).
(83)
(84) The transistors 7 and 8 of the bridge may be produced using FD-SOI technology using the possibility of control by a substrate electrode. In this case, the maximum current of the transistors is not only controlled by their gate width W, but also by the substrate electrode. Hence, an action on the substrate voltage V.sub.BB allows the temporal (charging time of the capacitors) and energy properties of the circuit to be modified.
(85)
(86)
(87)
(88)
(89)
(90)
(91) In order to obtain burst mode operation of an artificial neuron according to the Morris-Lecar model, one possibility is that the relationship between the excitation current I.sub.ex and the membrane potential V.sub.mem is governed by an equation of the following type:
dI.sub.ex/dt=ε(V.sub.0−V.sub.mem(t−T),
where ε, V.sub.0 and T are constants to be defined according to the desired properties.
(92) This equation is the equation of an inductive circuit, where the excitation current is proportional to the whole of the membrane potential.
(93) Indeed, when the resting membrane potential is close to Vs, the derivative dI.sub.ex/dt of the excitation current is positive. The excitation current increases and V.sub.mem reaches the oscillation threshold. The oscillations increase the average value of the membrane potential, resulting in a negative current derivative dI.sub.ex/dt and a reduction in the excitation current, which then drops below a threshold, stopping the burst.
(94) The principle of the burst mode operation therefore involves increasing, and respectively decreasing, the excitation current when the membrane potential is below, and respectively above, a certain threshold.
(95) As a function of the constants ε, V.sub.0 and T, the burst mode may be obtained without excitation current I.sub.ex when the circuit is unstable.
(96) The excitation circuit 60 of
(97)
(98) An example of the integration of an artificial neuron according to the invention into a neuromorphic system is schematically shown in
(99) The interconnection of two neurons, schematically shown in
(100) As the synapses are plastic (their effect varies as a function of the pre- and post-neuron activities), their plasticity is represented by a synaptic weight (modeled by the “weight” gate potential on an NMOS control transistor). This synaptic weight is assumed known, being defined by a learning sequence or generated by an appended circuit (Spike Timing Dependent Plasticity circuit, for example).
(101)
(102)
(103)
(104) The considered case is that of HF LTP (High-Frequency Long Term Potentiation), where multiple very high frequency spikes are applied to the input of an integrator, the output of which is connected to the input of the weight of a synaptic circuit.
(105)
(106)
(107)
(108) Another possibility for obtaining the burst mode is to rely on this type of coupling between neurons 1, 1′ with different oscillation frequencies. The first neuron 1, which must oscillate at a higher frequency than the second neuron 1′, has membrane C.sub.m1 and delay C.sub.k1 capacitance values, for example, with two orders of magnitude that are lower than those C.sub.m1′ and C.sub.k1′ of the second neuron 1′.
(109) A step current applied to the input of the first neuron generates a pulse train. These pulses generate an excitatory synaptic current at the input of the second neuron via the excitatory synapse 2, thus depolarizing the second neuron.
(110) When the second neuron depolarizes, it generates a pulse train generating an inhibitor synaptic current at the input of the first neuron via the inhibitory synapse 3, thus hyperpolarizing the first neuron and stopping its pulses.
(111) Given the various oscillation frequencies of the two neurons, high-frequency oscillations are obtained at the output of the first neuron, corresponding to pulse bursts.
(112)
(113)
(114) The pre-neuron 1 controls the post-neuron 1′. Indeed, when it emits pulses, the pre-neuron 1 creates an inhibitor synaptic current at the input of the post-neuron 1′, so as to prevent said neuron from oscillating. During the hyperpolarization phase of the pre-neuron 1 (absence of pulses), with the inhibitor synaptic current being insufficient, the post-neuron 1′ oscillates normally as if it was isolated.
(115)
(116)
(117)
(118)
(119) The present invention may be used in at least two fields.
(120) The artificial neuron according to the invention may be used as a building block in neuroinspired systems for data processing, particularly for processing images, video and for facial recognition. In this case, the elements of the neural circuit will be optimized for high speed and/or very low dissipated power.
(121) Furthermore, the neuron according to the invention may be used in biomedical applications, as an artificial biological neuron (implant). In this case, the elements of the circuit are optimized to faithfully reproduce the spike of the biological neurons.