Neuron circuit
11334787 · 2022-05-17
Assignee
Inventors
Cpc classification
International classification
Abstract
Disclosed is a neuron circuit in which an overflow signal before fire is retained after the fire. The neuron circuit according to an embodiment of the inventive concept includes a synapse element, a synaptic integration unit and a pulse generation unit. The synapse element receives output signals of a pre-neuron circuit and a post-neuron circuit. The synaptic integration unit includes a capacitor charged by the current flowing into the synapse element depending on the output signals of the pre-neuron circuit and the post-neuron circuit. The pulse generation unit generates an output pulse from the charging voltage of the capacitor. The pulse generation unit includes a pulse generation circuit generating the output pulse depending on the charging voltage of the capacitor and an overflow signal retaining unit connected between the capacitor and the pulse generation circuit and retaining an overflow signal, which exceeds a threshold voltage among the charging voltage of the capacitor after the pulse generation unit fires.
Claims
1. A neuron circuit comprising: a synapse element configured to receive output signals of a pre-neuron circuit and a post-neuron circuit, a weight of the synapse element being changed depending on the output signals of the pre-neuron circuit and the post-neuron circuit; a synaptic integration unit including a capacitor charged by a current flowing through the synapse element depending on the output signals of the pre-neuron circuit and the post-neuron circuit; and a pulse generation unit configured to generate an output pulse from a charging voltage of the capacitor, the output pulse being fed back to the synapse element as the output signal of the post-neuron circuit, wherein the pulse generation unit includes: a pulse generation circuit configured to generate the output pulse depending on the charging voltage of the capacitor; and an overflow signal retaining unit connected between the capacitor and the pulse generation circuit, the overflow signal retaining unit being configured to retain an overflow signal exceeding a threshold voltage among the charging voltage of the capacitor after the pulse generation unit fires.
2. The neuron circuit of claim 1, wherein the overflow signal retaining unit is reduced from the charging voltage of the capacitor by the threshold voltage upon firing of the pulse generation unit.
3. The neuron circuit of claim 1, wherein the overflow signal retaining unit includes: a first inverter circuit configured to receive the charging voltage of the capacitor; and a second inverter circuit configured to receive an output signal of the first inverter circuit, wherein the overflow signal retaining unit is configured to copy the charging voltage of the capacitor to store the copied charging voltage in an output terminal of the second inverter circuit.
4. The neuron circuit of claim 3, wherein the overflow signal retaining unit further includes a capacitor element connected between the output terminal of the second inverter circuit and a ground.
5. The neuron circuit of claim 3, wherein the pulse generation circuit includes: a first transistor having a gate, the charging voltage of the capacitor copied to the output terminal of the second inverter circuit being input to the gate of the first transistor; a first inverter configured to invert the charging voltage of the capacitor copied to the output terminal of the second inverter circuit and output the inverted charging voltage; a second inverter configured to invert an output signal of the first inverter and to output the inverted output signal to a drain or a source of the first transistor; and a second transistor having a gate, an output signal of the second inverter being input to the gate of the second transistor, and the second transistor discharging an output terminal of the second inverter circuit depending on the output signal of the second inverter.
6. The neuron circuit of claim 5, wherein the pulse generation circuit further includes: a third transistor having a gate, an output pulse of the second inverter being input to the gate of the third transistor, wherein one of a drain and a source of the third transistor is connected to a charging terminal of the capacitor and the other is grounded.
7. The neuron circuit of claim 1, wherein the synapse element includes an excitation synapse element and an inhibition synapse element, wherein the excitation synapse element includes a first transistor element, wherein the inhibition synapse element includes a second transistor element, wherein the output signal of the pre-neuron circuit is input to a first gate of each of the first transistor element and the second transistor element, and wherein the output signal of the post-neuron circuit is input to a second gate of each of the first transistor element and the second transistor element.
8. The neuron circuit of claim 1, wherein the synaptic integration unit further includes a current mirror connected between the synapse element and the capacitor, wherein the current mirror is connected to the excitation synapse element and the inhibition synapse element such that a current constantly flows through the excitation synapse element or the inhibition synapse element depending on the output signals of the pre-neuron circuit and the post-neuron circuit, and outputs a differential current between the excitation synapse element and the inhibition synapse element to the capacitor, and wherein the capacitor is isolated from the current flowing through the excitation synapse element and the inhibition synapse element by the current mirror and is charged by the differential current.
Description
BRIEF DESCRIPTION OF THE FIGURES
(1) The above and other objects and features of the inventive concept will become apparent by describing in detail exemplary embodiments thereof with reference to the accompanying drawings.
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
DETAILED DESCRIPTION
(12) According to an embodiment of the inventive concept, other advantages and features and methods of accomplishing the same may be understood more readily with reference to the following detailed description of an embodiment and the accompanying drawings. However, the inventive concept is not limited to the embodiments set forth herein; the inventive concept will only be defined by the appended claims. Even though it is not defined, all terms (including technical or scientific terms) used herein, all terms have the same meaning as being generally accepted by the general technology in the art belonging to inventive concept. General descriptions about the well-known configurations may be omitted so as not to obscure the subject matter of the inventive concept. A configuration which is equal to or corresponds to drawings in the inventive concept is possible using the same reference numerals. For the purpose of giving the understanding of the inventive concept, some configurations in the drawings may be illustrated to be somewhat exaggerated or reduced.
(13) The terminology used herein to describe embodiments is not intended to limit the scope of the inventive concept. The articles “a,” “an,” and “the” are singular in that they have a single referent, however, the use of the singular form in the inventive concept should not preclude the presence of more than one referent. It will be further understood that the terms “comprises,” “comprising,” “includes,” and/or “including,” when used herein, specify the presence of stated features, items, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, items, steps, operations, elements, components, and/or groups thereof.
(14)
(15) The synapse element 10 is configured to receive the output signal of the pre-neuron circuit 20 and to receive the output signal of the post-neuron circuit 30, which has been fed back. The weight of the synapse element 10 may be changed by the timing difference between the output signals of the pre-neuron circuit 20 and the post-neuron circuit 30.
(16) In an embodiment, the synapse element 10 may be provided as a transistor element; the drain or source of the transistor element is connected to the gate of transistor element. The transistor element may be provided as a double-gate transistor.
(17) The weight of the synapse element 10 may increase or decrease depending on the timing difference between the output signal of the pre-neuron circuit 20 input to a first gate and the output signal of the post-neuron circuit 30 fed back to a second gate.
(18) Because the structures of the pre-neuron circuit 20 and the post-neuron circuit 30 are provided to be the same as each other, the neuron circuit will be described below based on the structure of the post-neuron circuit 30.
(19)
(20) The synaptic integration unit 130 may accumulate the current flowing into the synapse element 10 in the capacitor C.sub.1 in response to the output signals of the pre-neuron circuit 20 and the post-neuron circuit 30. The charging voltage (cumulative voltage) accumulated at the accumulation terminal N1 of the capacitor C.sub.1 is input to the pulse generation unit 140.
(21) The pulse generation unit 140 generates an output pulse V.sub.OUT from the signal accumulated in the capacitor C.sub.1 by the synaptic integration unit 130. In the embodiment illustrated in
(22) As illustrated in
(23) The synapse element 10 may include an excitation synapse element 110 and an inhibition synapse element 120. In an embodiment, the synapse element 10 may be provided as a 4-terminal element. The excitation synapse element 110 includes one or more first transistor elements.
(24)
(25) Returning to
(26) Each of the excitation synapse element 110 and the inhibition synapse element 120 may be provided as the structure in which the drain (or source) and the gate of the transistor element are connected to each other; the weight may be changed depending on the timing difference between output signals of the pre-neuron circuit 20 and the post-neuron circuit 30.
(27)
(28) The process in which the weight of the synapse element is changed by the timing difference between the pre-neuron signal and the post-neuron signal may be referred to as Spike Timing Dependent Plasticity (STDP). When the pre-neuron signal is first input to the synapse element and then the post-neuron signal is input, the weight of the synapse element increases by potentiation. On the other hand, when the post-neuron signal is first input to the synapse element and then the pre-neuron signal is input, the weight of the synapse element decreases by depression.
(29) As illustrated in
(30) Returning to
(31) The current mirror 132 is connected to the drain or the source of the excitation synapse element 110 and the inhibition synapse element 120 such that the current constantly flows into the excitation synapse element 110 or the inhibition synapse element 120 depending on the output signals of the pre-neuron circuit 20 and the post-neuron circuit 30 regardless of the change of the voltage charged in the capacitor C.sub.1.
(32) The current mirror 132 may output the differential current I.sub.E-I.sub.I between the excitation synapse element 110 and the inhibition synapse element 120 to the capacitor C.sub.1. The capacitor C.sub.1 is isolated by the current mirror 132 from the current I.sub.E flowing into the excitation synapse element 110 and the current I.sub.1 flowing into the inhibition synapse element 120. The capacitor C.sub.1 may be charged by the differential current I.sub.E-I.sub.I generated by the current mirror 132.
(33) The capacitor C.sub.1 may be charged by the differential current corresponding to the difference between the current I.sub.E flowing into the excitation synapse element 110 and the current I.sub.1 flowing into the inhibition synapse element 120. Accordingly, the current flowing into the capacitor C.sub.1 may be reduced and the size of a circuit may be reduced by reducing the capacity of the capacitor C.sub.1. The amount of current flowing into the capacitor C.sub.1 may be adjusted by designing the channel width and the channel distance of the current mirror 132, and thus the capacity and the size of the capacitor C.sub.1 may be adjusted.
(34) The pulse generation unit 140 includes pulse generation circuits 141 to 145 and an overflow signal retaining unit 150. The pulse generation circuits 141 to 145 operate depending on the output terminal signal of the overflow signal retaining unit 150 to generate an output pulse. The pulse generation circuits 141 to 145 may include the two inverters 141 and 142, which are serially connected to each other, the first transistor 143, the second transistor 144, and the third transistor 145.
(35) The first inverter 141 inverts the output signal of the overflow signal retaining unit 150 that is a virtual membrane signal and then outputs the inverted output signal. The output terminal of the first inverter 141 is connected to the input terminal of the second inverter 142. The second inverter 142 inverts the output signal of the first inverter 141 and then outputs the inverted output signal to the output terminal.
(36) A first capacitor element C.sub.2 may be provided between the output terminal of the first inverter 141 and the ground. A second capacitor element C.sub.3 may be connected between the output terminal of the second inverter 142 and the ground.
(37) The output signal of the second inverter 142 corresponds to the output pulse V.sub.OUT of the pulse generation unit 140. The output pulse V.sub.OUT generated at an output terminal N3 of the pulse generation unit 140 is fed back as the output signal of the post-neuron circuit 30 and then is input to the second gate of the synapse element 10.
(38) The output signal of a virtual membrane node N2 output by the overflow signal retaining unit 150 is input to the gate of the first transistor 143. A third capacitor element C5 may be connected between the virtual membrane node N2 that is the output terminal of the overflow signal retaining unit 150 and the ground.
(39) The second capacitor element C.sub.3 may be connected to the drain (or source) of the first transistor 143. The operating voltage (e.g., V.sub.DD or −V.sub.DD) may be applied to the source (or drain) of the first transistor 143.
(40) The gate of the second transistor 144 may be connected to the output terminal N3 of the second inverter 142; the source (or drain) thereof may be grounded or the operating voltage may be applied to the source (or drain); the drain (or source) may be connected to the gate of the first transistor 143. The second transistor 144 may discharge the third capacitor element C5 depending on the output pulse V.sub.OUT that is the output signal of the second inverter 142.
(41) The first transistor 143 and the second transistor 144 may be provided as the transistor of a complementary type. In the embodiment of
(42) The output pulse V.sub.OUT of the second inverter 142 is input to the gate of the third transistor 145. The drain (or source) of the third transistor 145 may be connected to the charging terminal of the capacitor C.sub.1; the source (or drain) thereof may be grounded or the operating voltage may be applied to the source (or drain). The third transistor 145 discharges the capacitor C.sub.1 in response to the signal of the output pulse V.sub.OUT of the second inverter 142.
(43) The overflow signal retaining unit 150 may be provided such that the overflow signal before fire is retained by the output pulse the pulse generation unit 140 after the fire. The overflow signal retaining unit 150 may be connected between an accumulation terminal N1, which is the charging node of the capacitor C.sub.1, and the virtual membrane node N2, which is the input terminal of the first inverter 141.
(44) The overflow signal retaining unit 150 is configured to copy the signal of the accumulation terminal N1 of the capacitor C.sub.1 to the output terminal (virtual membrane node), to drive an output pulse generation circuit, and to generate an output pulse. The overflow signal retaining unit 150 may include two inverter circuits serially connected to each other.
(45) The overflow signal retaining unit 150 may include a first inverter circuit 151 and a second inverter circuit 152. The first inverter circuit 151 inverts the charging voltage of the capacitor C.sub.1 to output the inverted charging voltage. The output terminal of the first inverter circuit 151 is connected to the input terminal of the second inverter circuit 152.
(46) The second inverter circuit 152 inverts the output signal of the first inverter circuit 151 and then outputs the inverted output signal to the output terminal. The output signal of the second inverter circuit 152 is input to the input terminal of the first inverter 141. A fourth capacitor element C.sub.4 may be connected between the output terminal of the first inverter circuit 151 and the ground.
(47) Hereinafter, according to an embodiment of the inventive concept, the procedure in which an output pulse is generated and the principle by which the overflow signal before fire is retained after the fire will be described. The current from the synapse element 10 charges the capacitor C.sub.1 via the current mirror 132.
(48) When the voltage of a specific level or more is charged in the capacitor C.sub.1, the virtual membrane signal is accumulated at the virtual membrane node N2 and then the first transistor 143 is turned on by the virtual membrane signal. As a result, the first transistor 143 reduces the voltage of the output pulse V.sub.OUT of the output terminal N3 of the pulse generation unit 140 from 0 V to −V.sub.DD.
(49) While the voltage of the output terminal of the pulse generation unit 140 is reduced, the two inverters 141 and 142 operate. Due to the delay of the inverters 141 and 142, the voltage of the output terminal V.sub.OUT is dropped by the operation of the first transistor 143 to −V.sub.DD and then the charging voltage of the capacitor C.sub.1 is output to the output terminal N3 of the second inverter 142, and thus the voltage of the output terminal V.sub.OUT increases from −V.sub.DD to +V.sub.DD again.
(50) Lastly, when the voltage of the output terminal V.sub.OUT of the pulse generation unit 140 increases to +V.sub.DD, the third transistor 145 operates and then the capacitor C.sub.1 is discharged. Accordingly, the voltage of the output terminal V.sub.OUT of the pulse generation unit 140 returns to 0 V that is the original state.
(51) According to an embodiment of the inventive concept, the asymmetrical pulse generation unit 140 composed of six transistors may generate an asymmetrical output pulse, and the number of transistors necessary to make the asymmetrical output pulse may be reduced. Accordingly, the neuron circuit of low-power and low-area may be implemented.
(52) Unlike Von Neumann architecture, in the neuron circuit according to an embodiment of the inventive concept, as training is performed, the weight is changed simultaneously. Furthermore, as the output signal is fed back to the 4-terminal synapse element 10, the weight is changed automatically. Accordingly, there is no need for an additional controller or there is no need for the process of updating the weight.
(53) Furthermore, a neuron circuit according to an embodiment of the inventive concept may have the structure in which a memory and a processor are integrated because the weight is stored in the 4-terminal synapse element and may be configured to have a great parallel structure. Accordingly, it is expected that the large power consumption in a flexible structure may be reduced.
(54)
(55) Accordingly, the capacitor C.sub.1 independently operates without being affected by the operations of the first transistor 143 and the second transistor 144 for generating the output pulse V.sub.OUT. While the capacitor C.sub.1 is discharged, the voltage of the membrane node of the accumulation terminal N1 of the capacitor C.sub.1 may be reduced by a specific level (e.g., a threshold voltage) and may retain an overflow signal V.sub.OV after fire, thereby improving the accuracy of pattern recognition.
(56)
(57) As illustrated in
(58) For example, when the threshold voltage is 1 V and the voltage of the accumulation terminal of the capacitor is 1.2 V, after the neuron circuit according to an embodiment of the inventive concept fires, the overflow signal of 0.2 V is retained at the accumulation terminal of the capacitor. On the other hand, in the case of the neuron circuit not having a function of retaining an overflow signal, the overflow signal of 0.2 V is not retained at the accumulation terminal of the capacitor after the fire.
(59) Afterward, when the procedure in which the same signal is accumulated and fired is repeated four times, the neuron circuit not having a function of retaining an overflow signal performs a fire operation five times. On the other hand, the neuron circuit according to an embodiment of the inventive concept performs the fire operation six times. Accordingly, the number of fires increases by 20%.
(60)
(61) As illustrated in
(62) In the case of an embodiment of the inventive concept, as illustrated in
(63) According to an embodiment of the inventive concept, the error rate of pattern recognition of the neuron circuit is 736/10000; the error rate of pattern recognition of the neuron circuit not having a function of retaining an overflow signal is 763/10000. In the case of the neuron circuit according to an embodiment of the inventive concept, the error rate of pattern recognition may be reduced by about 5%.
(64)
(65) In the neuron circuit according to an embodiment of the inventive concept, the overflow signal is capable of being retained after fire, thereby reducing the error rate of pattern recognition and improving the accuracy of pattern recognition. The neuron circuit according to an embodiment of the inventive concept may be applied to various technologies that mimic biological neurons as a circuit, such as nervous system imitation AI, hardware the implementation of nervous systems, and hardware-based AI systems.
(66) The above embodiment is provided to give an understanding of the inventive concept, the scope and spirit of the inventive concept is not be limited thereto, and various modifications possible embodiments therefrom are also understood within the scope of the inventive concept. The technical protection scope of the inventive concept will be defined by the technical spirit of the appended claims, the scope and spirit of the inventive concept is not limited to the wording of the claims, and it is to be understood that the technical value substantially affects the equivalent scope of the inventive concept.
(67) According to an embodiment of the inventive concept, a neuron circuit in which the overflow signal before fire is retained after the fire may be provided.
(68) The effect of the inventive concept is not limited to the above-mentioned effects. Other effects which are not mentioned will be clearly understood from the following description and accompanying drawings to those skilled in the art.
(69) While the inventive concept has been described with reference to exemplary embodiments thereof, it will be apparent to those of ordinary skill in the art that various changes and modifications may be made thereto without departing from the spirit and scope of the inventive concept as set forth in the following claims.