Weight matrix circuit and weight matrix input circuit

Abstract

Provided are a weight matrix circuit and a weight matrix input circuit. The weight matrix circuit includes a memory array including n input lines, m output lines, and n×m resistive memory devices each connected to the n input lines and the m output lines and each having a non-linear current-voltage characteristic, an input circuit connected to each of the input lines, and an output circuit connected to each of the output lines. The input circuit is connected to the resistive memory devices such that the weight matrix circuit has a linear current-voltage characteristic.

Claims

1. A weight matrix circuit comprising: a memory array configured to include n input lines, m output lines, and nxm resistive memory devices each connected to the n input lines and the m output lines and each having a non-linear current-voltage characteristic; an input circuit configured to be connected to each of the input lines; and an output circuit configured to be connected to each of the output lines, wherein the input circuit is connected to the resistive memory devices such that the weight matrix circuit has a linear current-voltage characteristic, and wherein the input circuit comprises: an operational amplifier (OP-amp) having an inverting input electrode, a non-inverting input electrode, and an output electrode; a resistor connected to the inverting input electrode; and a resistive memory device having ends which are connected to the inverting input electrode and the output electrode.

2. The weight matrix circuit of claim 1, wherein the resistive memory devices are any one selected from the group consisting of a resistive random access memory (RRAM), a magnetic random access memory (MRAM), a phase-change random access memory (PCRAM), and a ferroelectric random access memory (FeRAM).

3. The weight matrix circuit of claim 1, wherein the non-linear current-voltage characteristic of the resistive memory devices is modeled as a hyperbolic sine function (sinhx) with respect to a provided voltage.

4. The weight matrix circuit of claim 1, wherein the input circuit is modeled as an inverse function of a current-voltage characteristic of the resistive memory devices.

5. A weight matrix input circuit for providing an input to a weight matrix circuit which stores weight values between neurons of a neural network, the weight matrix input circuit comprising: an operational amplifier (OP-amp) having an inverting input electrode, a non-inverting input electrode, and an output electrode; a resistor connected to the inverting input electrode; and a resistive memory device having ends connected to the inverting input electrode and the output electrode, wherein the weight matrix circuit connected to the weight matrix input circuit has a linear current-voltage characteristic.

6. The weight matrix input circuit of claim 5, wherein the current-voltage characteristic of the weight matrix circuit is modeled as a hyperbolic sine function, and a current-voltage characteristic of the weight matrix input circuit is modeled as an inverse function of the hyperbolic sine function.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The above and other objects, features and advantages of the present invention will become more apparent to those of ordinary skill in the art by describing in detail exemplary embodiments thereof with reference to the accompanying drawings, in which:

(2) FIG. 1 is a diagram showing an overview of a weight matrix circuit according to an exemplary embodiment of the present invention;

(3) FIG. 2 is a diagram showing an overview of a conversion circuit included in an output circuit according to an exemplary embodiment of the present invention;

(4) FIG. 3 is a diagram showing an overview of a weight matrix input circuit included in an input circuit according to an exemplary embodiment of the present invention;

(5) FIG. 4A is a graph showing a current-voltage characteristic of resistive memory devices included in a memory array;

(6) FIG. 4B is a graph schematically showing a current-voltage characteristic of an input circuit; and

(7) FIGS. 5A, 5B and 5C illustrate a set of graphs in which a weight matrix circuit including an input circuit according to an exemplary embodiment of the present invention is compared with a weight matrix circuit according to related art.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

(8) The following descriptions of the present invention are merely exemplary embodiments for structural or functional descriptions, and the scope of the present invention should not be construed as being limited to the exemplary embodiments set forth herein. In other words, exemplary embodiments may be diversely changed and have various forms, and the scope of the present invention should be understood as including equivalents thereto to realize the technical idea.

(9) Meanwhile, terminology described in this application is to be understood as follows.

(10) It is to be understood that the singular forms include the plural forms unless the context clearly indicates otherwise. The terms “include,” “have,” or the like when used in this specification, specify the presence of stated features, numbers, operations, elements, parts, or combinations thereof but do not preclude the presence or addition of one or more other features, numbers, operations, elements, parts, or combinations thereof.

(11) Unless a specific sequence is clearly mentioned in the context, respective steps may be performed out of the order noted in a flowchart. In other words, respective steps may be performed in the same order as stated, substantially concurrently, or in the reverse order.

(12) Unless otherwise defined, all terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. Terms generally used, like those defined in a dictionary, should be interpreted to match the contextual meaning of related technology and should not be interpreted in an idealized or overly formal sense unless clearly so defined herein.

(13) Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a diagram showing an overview of a weight matrix circuit 1 according to an exemplary embodiment of the present invention. Referring to FIG. 1, the weight matrix circuit 1 includes a memory array 100 including n input lines, m output lines, and n×m resistive memory devices RM which are each connected to the n input lines and the m output lines and have a non-linear current-voltage characteristic, an input circuit 200 connected to each of the input lines, and an output circuit 300 connected to each of the output lines. The input circuit is connected to the resistive memory devices RM such that the weight matrix circuit 1 has a linear current-voltage characteristic.

(14) The memory array 100 includes the resistive memory devices RM connected to the plurality of input lines and the plurality of output lines, and the resistive memory devices RM are arranged in an array form. The resistive memory devices RM store weight values w11, w21, . . . , and wnm which indicate strengths of connections between neurons. Therefore, the memory array 100 exemplified in FIG. 1 may store weight values for connecting n neurons and m neurons. A larger weight value may indicate a stronger connection between neurons, and a smaller weight value may indicate a weaker connection between neurons.

(15) For example, when a voltage input signal Vi2 is provided to a second row of the memory array 100, currents I1, I2, I3, . . . , and Im are output according to the weight values w21, w22, w23, . . . , and w2m stored in memory devices, and the current values output from the memory array 100 correspond to products of the weight values w21, w22, w23, . . . , and w2m stored in the memory devices and the input.

(16) According to an exemplary embodiment, the resistive memory devices RM may be any one selected from among the group of a resistive random access memory (RRAM), a magnetic random access memory (MRAM), a phase-change random access memory (PCRAM), and a ferroelectric random access memory (FeRAM).

(17) FIG. 2 is a diagram showing an overview of a conversion circuit 310 included in the output circuit 300 according to an exemplary embodiment of the present invention. Referring to FIG. 2, the conversion circuit 310 includes an operational amplifier (OP-amp) 312 and a resistor RF having one end and the other end each connected to an inverting input and an output of the OP-amp 312. A non-inverting input of the OP-amp 312 is connected to a reference voltage. An input current IK is provided through a node at which the OP-amp 312 and the resistor RF are connected, and the OP-amp 312 converts the input current IK into a corresponding voltage signal Vok and outputs the voltage signal Vok.

(18) FIG. 3 is a diagram showing an overview of a weight matrix input circuit 210 included in the input circuit 200 according to an exemplary embodiment of the present invention. Referring to FIG. 3, the weight matrix input circuit 210 includes an OP-amp 212 having an inverting input electrode, a non-inverting input electrode, and an output electrode, a resistor IR connected to the inverting input electrode, and a resistive memory device RM having ends each connected to the inverting input electrode and the output electrode.

(19) According to an exemplary embodiment, the memory device RM included in the weight matrix input circuit 210 may be any one selected from among the group of an RRAM, an MRAM, a PCRAM, and a FeRAM and may be the same device as the resistive memory devices included in the memory array 100.

(20) Operation of the weight matrix circuit 1 having the above-describe configuration will be described from now. FIG. 4A is a graph showing a current-voltage characteristic of resistive memory devices RM included in the memory array 100. Referring to FIG. 4A, a current-voltage characteristic of the resistor which is a passive device is shown as a straight line, but a current-voltage characteristic of the resistive memory devices RM is not linear. The current-voltage characteristic of the resistive memory devices RM may be mathematically modeled as a hyperbolic sine function as shown in Equation 1 below.
I=G sinh (BV)  [Equation 1]

(21) (I: current, V: voltage, B: constant, G: conductance)

(22) A current-voltage characteristic of the weight matrix input circuit 210 shown in FIG. 3 is as follows. When an arbitrary input voltage Vit is given as an input and a conductance of the resistive memory device RM is Gt, a voltage Vot of the output terminal obtained according to characteristics of the circuit 210 employing the input resistor IR and the OP-amp 212 is determined to be an inverse function of the current-voltage characteristic of the resistive memory device RM. This may be represented as shown in Equation 2 below and schematically depicted as shown in FIG. 4B.

(23) $\begin{array}{cc}\mathrm{Vot}=\frac{1}{B}\mathrm{arc}\sin h\left(\frac{V\mathrm{it}}{G_{t}R}\right)& \left[\mathrm{Equation}2\right]\end{array}$

(24) Therefore, when the output voltage Vot is input to another resistive memory device RM which has been set to have a conductance G, a current flowing through the memory device RM may be calculated as shown in Equation 3 below, and it is possible to obtain a linear current-to-voltage characteristic as shown in Equation 3.

(25) $\begin{array}{cc}\mathrm{Iit}=\frac{G}{G_{t}R}V\mathrm{it}& \left[\mathrm{Equation}3\right]\end{array}$

Implementation Example and Experimental Example

(26) FIGS. 5A, 5B and 5C illustrate a set of graphs in which a weight matrix circuit including an input circuit according to an exemplary embodiment of the present invention is compared with a weight matrix circuit according to related art. Multi-layer perception (MLP) training was performed to classify a modified national institute of standards and technology (MNIST) data set such that effects of the exemplary embodiment of the present invention were verified. Training and determination were performed with MATLAB. FIG. 5A is a graph for comparing accuracy of an ideal case Ideal, accuracy of a case Naïve of the related art, and an accuracy of a case Proposed of the exemplary embodiment of the present invention. While there was an accuracy of 88% in the case of the related art, there was an accuracy of 94.81% in the case of the exemplary embodiment of the present invention.

(27) Convolutional neural networks (CNNs) were used to classify complex images such that classification accuracy results of the three cases were compared. FIGS. 5B and 5C show calculation simulation results of LeNet and AlexNet for classifying the Canadian institute for advanced research (CIFAR)-10 and ImageNet data sets.

(28) Referring to FIGS. 5B and 5C, when non-linearity increases, accuracy in data set classification of the related art was reduced (black). On the other hand, according to the exemplary embodiment of the present invention, accuracy in data set classification was maintained at a fixed level regardless of an increase in non-linearity.

(29) An input circuit according to an exemplary embodiment of the present invention is designed using a resistive memory device to have an output voltage which corresponds to an inverse function of a non-linear current-voltage characteristic of the resistive memory device with respect to an arbitrary input voltage. Therefore, the input circuit according to an exemplary embodiment of the present invention compensates for a non-linear current-voltage characteristic of resistive memory devices included in a memory array, and accordingly, it is possible to improve calculation accuracy of an artificial neural network circuit employing the memory device.

(30) According to an exemplary embodiment of the present invention, it is possible to provide a linear current-voltage characteristic of a weight matrix such that a weight value can be easily controlled. Due to this characteristic, it is possible to improve accuracy in training and inference and speed of training and inference and to lower power consumption.

(31) Although exemplary embodiments of the present invention have been described in detail with reference to the accompanying drawings, the embodiments are merely examples, and those of ordinary skill in the art will appreciate that various modifications and equivalents may be made from the exemplary embodiments. Therefore, the technical scope of the present invention should be determined by the following claims.