Unit element for performing multiply-accumulate operations

Abstract

The present invention provides an analog-digital hybrid architecture, which performs 256 multiplications and additions at a time. The system comprises 256 Processing Elements (PE) (108), which are arranged in a matrix form (16 rows and 16 columns). The digital inputs (110) are converted to analog signal (114) using digital to analog converters (DAC) (102). One PE (108) produces one analog output (115) which is nothing but the multiplication of the analog input (114) and the digital weight input (112). The implementation of PE is done by using i) capacitors and switches and ii) resistor and switches. The outputs from multiple PEs (108) in a column are connected together to produce one analog MAC output (116). In the similar manner, the system produces 16 MAC outputs (118) corresponding to 16 columns. Analog to digital converters (ADC) (104) are used to convert the analog MAC output (116) to digital form (118).

Claims

1. A processing element (PE) comprising: a PH1 clock and a PH2 clock, the PH1 clock and PH2 clock enabled during mutually exclusive intervals of time; a VinP input, VinN input, and a common terminal; a first plurality of binary inputs and a second plurality of binary inputs; a first PE terminal coupled to the VinP input when PH1 and a most significant bit of the second plurality of binary inputs are both asserted; the first PE terminal coupled to the VinN input when PH1 and a most significant bit of the first plurality of binary inputs are both asserted; a plurality of elements, each element coupled to the first PE terminal and the common terminal, each element comprising: a binary weighted capacitor having a first capacitor terminal coupled to the common terminal, and a second capacitor terminal coupled to the first PE terminal when PH2 is asserted, the second capacitor terminal also coupled to a first switch which closes when the PH1 is asserted, the first switch in series with a second switch which closes when a unique bit of the plurality of first input bits is asserted, a connection between the first switch and second switch connected to the common terminal when a bit from the second plurality of input bits corresponding to a bit from the first plurality of input bits is asserted; the PE having an output coupled to the first PE terminal when the PH2 clock is asserted.

2. The PE of claim 1 where the plurality of first input bits is seven.

3. The PE of claim 1 where the plurality of second input bits is seven.

4. The PE of claim 1 where each element of the plurality of elements is coupled to one of the plurality of first input bits and also coupled to one of the plurality of second input bits.

5. The PE of claim 1 where the binary weighted capacitor has a value which corresponds to a first input bit position or a second input bit position.

6. The PE of claim 1 where each binary weighted capacitor of the plurality of elements has a unique value 1C, 2C, 4C, 8C, 16C, 32C, and 64C.

7. The PE of claim 1 where an output is asserted when the PH2 is active.

8. The PE of claim 1 where PH1 and PH2 are continuous clocks.

9. The PE of claim 1 where PH2 is enabled for a shorter duration than PH1 is enabled.

10. A multiplier-accumulator (MAC) comprising: a PH1 clock and a PH2 clock, the PH1 clock and PH2 clock enabled during mutually exclusive intervals of time; a VinP input, VinN input, and a common terminal; a first plurality of binary inputs and a second plurality of binary inputs; a positive processing element (PE) comprising: a first PE terminal coupled to the VinP input when PH1 and a most significant bit of the second plurality of binary inputs are both asserted; the first PE terminal coupled to the VinN input when PH1 and a most significant bit of the first plurality of binary inputs are both asserted; a plurality of elements, each element coupled to the first PE terminal and the common terminal, each element comprising: a binary weighted capacitor having a first capacitor terminal coupled to the common terminal, and a second capacitor terminal coupled to the first PE terminal when PH2 is asserted, the second capacitor terminal also coupled to a first switch which closes when the PH1 is asserted, the first switch in series with a second switch which closes when a unique bit of the plurality of first input bits is asserted, a connection between the first switch and second switch connected to the common terminal when a bit from the second plurality of input bits corresponding to a bit from the first plurality of input bits is asserted; a negative processing element (PE) comprising: a first PE terminal coupled to the VinN input when PH1 and a most significant bit of the second plurality of binary inputs are both asserted; the first PE terminal coupled to the VinP input when PH1 and a most significant bit of the first plurality of binary inputs are both asserted; a plurality of elements, each element coupled to the first PE terminal and the common terminal, each element comprising: a binary weighted capacitor having a first capacitor terminal coupled to the common terminal, and a second capacitor terminal coupled to the first PE terminal when PH2 is asserted, the second capacitor terminal also coupled to a first switch which closes when the PH1 is asserted, the first switch in series with a second switch which closes when a unique bit of the plurality of first input bits is asserted, a connection between the first switch and second switch connected to the common terminal when a bit from the second plurality of input bits corresponding to a bit from the first plurality of input bits is asserted; the multiplier-accumulator generating a positive output coupled to the positive PE when PH2 is asserted and also generating a negative output coupled to the negative PE when PH2 is asserted.

11. The MAC of claim 10 where the plurality of first input bits is seven.

12. The MAC of claim 10 where the plurality of second input bits is seven.

13. The MAC of claim 10 where each element of the plurality of elements is coupled to one of the plurality of first input bits and also coupled to one of the plurality of second input bits.

14. The MAC of claim 10 where the binary weighted capacitor has a value which corresponds to a first input bit position or a second input bit position.

15. The MAC of claim 10 where each binary weighted capacitor of the plurality of elements has a unique value 1C, 2C, 4C, 8C, 16C, 32C, and 64C.

16. The MAC of claim 10 where an output is asserted when the PH2 is active.

17. The PE of claim 10 where PH1 and PH2 are continuous clocks.

18. The PE of claim 10 where PH2 is enabled for a shorter duration than PH1 is enabled.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Features, aspects, and advantages of the present invention will be better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings.

(2) FIGS. 1A and 1B illustrate a schematic diagram of a 16×16 MAC array, in accordance with an embodiment of the present invention.

(3) FIG. 1C illustrates a schematic diagram of a Processing Element (PE) unit, in accordance with an embodiment of the present invention.

(4) FIG. 1D illustrates a flow diagram of a method for saving power in a real time hardware processing unit, in accordance with an embodiment of the present invention.

(5) FIGS. 2A and 2B illustrate a schematic diagram of multiplier implementation using capacitors, in accordance with an embodiment of the present invention.

(6) FIG. 2C is a waveform plot for FIGS. 2A and 2B.

(7) FIG. 2D shows inputs to FIGS. 2A and 2B.

(8) FIGS. 3A and 3B illustrate a schematic diagram of resistive string-based multiplier using resistors, in accordance with an embodiment of the present invention.

(9) FIG. 3C shows a schematic diagram of decoders for use in FIGS. 3A and 3B.

(10) FIG. 3D shows a waveform plot for FIGS. 3A and 3B.

(11) It should be understood that the drawings are an aid to understand certain aspects of the present invention and are not to be construed as limiting.

DETAILED DESCRIPTION OF THE INVENTION

(12) While system and method are described herein by way of example and embodiments, those skilled in the art recognize that a method and system for saving power in a real time hardware processing unit are not limited to the embodiments or drawings described. It should be understood that the drawings and description are not intended to be limiting to the particular form disclosed. Rather, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the appended claims. Any headings used herein are for organizational purposes only and are not meant to limit the scope of the description or the claims. As used herein, the word “may” is used in a permissive sense (i.e., meaning having the potential to) rather than the mandatory sense (i.e., meaning must). Similarly, the words “include”, “including”, and “includes” mean including, but not limited to.

(13) The following description is full and informative description of the best method and system presently contemplated for carrying out the present invention which is known to the inventors at the time of filing the patent application. Of course, many modifications and adaptations will be apparent to those skilled in the relevant arts in view of the following description in view of the accompanying drawings and the appended claims. While the system and method described herein are provided with a certain degree of specificity, the present technique may be implemented with either greater or lesser specificity, depending on the needs of the user. Further, some of the features of the present technique may be used to advantage without the corresponding use of other features described in the following paragraphs. As such, the present description should be considered as merely illustrative of the principles of the present technique and not in limitation thereof, since the present technique is defined solely by the claims.

(14) It is worth noting that the present discussion relates to exemplary embodiments, and the appended claims should not be limited to the embodiments discussed herein. Disclosed embodiments provide a method and system of saving power in a real time hardware processing circuit.

(15) FIGS. 1A and 2B illustrate a schematic diagram of 16×16 MAC array, in accordance with an embodiment of the present invention. The present invention provides a hardware accelerator for Convolution Neural Network (CNN). The basic operation of a CNN involves dot product or Multiplication and Accumulation (MAC). In this invention, the MAC operation is performed by using analog-digital hybrid architecture, as shown in FIGS. 1A and 1B, which performs 256 multiplications and additions at a time (within one clock period).

(16) The system comprises 256 (PE1-PE256) Processing Elements (PE) (108), which are arranged in a matrix form (16 rows and 16 columns). The digital input (110) is common for each PE in a row and hence requiring 16 digital inputs (110). However, the digital weights (112) are unique to each PE cell (108).

(17) The inputs, which are provided to the system, are clock (106), 16 digital inputs of 8-bit width (110) and 8-bit digital weights (minimum of 256 weights from a memory) (112). Each of the 16 digital input is common to all PE cells in a row. The 8-bit width for digital input is taken or mentioned here for explanation purpose only. However, the present invention is also applicable to n-bit width. By taking the inputs, the system produces 16 MAC outputs in digital form (8-bit) (118) as shown in the FIG. 1. The operational part of the system for producing the output by taking the inputs is as follows:

(18) At first, the digital inputs (110) are converted to analog signal (114) using digital to analog converters (DAC) (102). Each row uses one DAC (DAC1-DAC16) (102) and converts the digital input to the analog output and then the analog output (116) obtained from the DAC (102) is connected to all PE cells (108) in the row.

(19) The processing element (PE) (108) is the primary component in the accelerator system. As shown in the FIG. 1C, each PE (108) comprises a multiplier (122) and a small amount of storage memory (120). The digital weights (8-bit digital number) (112) are stored in the storage memory (120) and are used for several times. The multiplier (122) takes an analog input (114) and a digital weight (digital input) (112) and multiplies both. In other words, one processing element (108) produces one analog output (116) which is nothing but the multiplication of the analog input (114) and the digital weight input (112). The outputs from multiple PEs (108) are connected together, which is equivalent to accumulation, to produce one analog MAC output (116).

(20) The system or architecture produces 16 (8-bit) MAC outputs (118) in single clock (106) cycle and each MAC output (118) is computed by performing 16 multiplications and adding all the 16 multiplication outputs. In other words, the accumulation operation is performed by combining the outputs of all the 16 PEs in a column. In the similar manner, the system produces 16 MAC outputs (118) corresponding to 16 columns.

(21) Accumulation operation is performed in column basis, thus producing 16 analog outputs (116). Analog to digital converters (ADC) (104) are used to convert the analog MAC output (116) back to digital form (118). There are 16 ADCs (ADC1-ADC16) present in the system, which are used for converting the analog outputs (116) to digital form (118), wherein one ADC (104) is used for one column respectively.

(22) FIG. 1D illustrates a flow diagram of a method for saving power in a real time hardware processing unit, in accordance with an embodiment of the present invention. At step 130, connect a plurality of processing elements (PEs) (108) in a matrix form. A first set of the processing elements (PEs) (108) comprises a multiple sets of the processing elements (PEs) (108); wherein each set of the multiple sets of the processing elements (PEs) (108) of the first set comprises a plurality of the processing elements (PEs) (108) being adapted to connect in a row. A second set of the processing elements (PEs) (108) comprises a multiple sets of the processing elements (PEs) (108); wherein each set of the multiple sets of the processing elements (PEs) (108) of the second set comprises a plurality of the processing elements (PEs) (108) being adapted to connect in a column.

(23) At step 132, connect a plurality of digital to analog converters (DAC) (102) to each set of the multiple sets of the processing elements (PEs) (108) of the first set. At step 134, convert a plurality of digital inputs (110) to a first analog output (114) using each digital to analog converter (DAC) (102). At step 136, send the first analog output (114) to the plurality of the processing elements (PEs) (108), which are adapted to connect in the row, using each the digital to analog converters (DAC) (102) such that the first analog output of the digital to analog converters (DAC) (102) being an input to the plurality of the processing elements (PEs) (108), which are adapted to connect in the row.

(24) At step 138, generate a second analog output (115) corresponding to each the processing element (PE) (108), wherein the second analog output of each the processing element (PE) (108) is a product of the first analog output of the digital to analog converter (DAC) and a digital weight input (112).

(25) At step 140, generate an analog dot product output (116) using each set of the multiple sets of the processing elements (PEs) (108) of the second set. At step 142, connect each the set of the multiple sets of the processing elements (PEs) (108) of the second set using an analog to digital converter (ADC). At step 144, convert the analog dot product output (116) to a digital output (118) using each the analog to digital converter (ADC) (104).

(26) FIGS. 2A and 2B illustrate a schematic diagram of multiplier implementation using capacitors (210), in accordance with an embodiment of the present invention. As shown in the figure, the circuit implementation of one multiplication operation between an analog input (202a/202b) and a digital input (204) is done by using capacitors (210) and switches (212). The circuit takes a fully differential analog input (Vinp (202a), or Vinn (202b)), a 8-bit digital input [b7: b0] (204) in sign-magnitude form where Most Significant Bit (MSB) represents sign, and a two phase non-overlapping clock signal (ph1 (214a), ph2 (214b)). The 8-bit width for digital input is taken or mentioned here for explanation purpose only. However, the present invention is also applicable to n-bit width.

(27) The circuit comprises a set of binary weighted capacitors (210) and multiple switches (212). The set of binary weighted capacitors, namely 1C, 2C, 4C, 8C, 16C, 32C and 64C, are connected with multiple switches (212), which are controlled by clock (214a/214b) and digital input (204), in a particular pattern. Here, “C” is the value of a unit capacitor. To realize differential implementation, this multiplier has two copies of same circuit wherein the difference between them are the analog input (202a or 202b) and analog output (206a or 206b). Hereafter, one half circuit is explained exclusively and, the other half circuit performs similarly with the complementary analog input and produces complementary analog output.

(28) As shown in the clock waveforms of FIG. 2C, with respect to the sign bit of the digital input, one half of the differential analog input i.e. either Vinp (202a) or Vinn (202b), is sampled to the circuit during the “ON” period of the “ph1” clock (214a). If sign-bit is “0”, “Vinp” (202a) is sampled to the circuit during the “ON” period of the “ph1” clock (214a). If sign-bit is “1” then “Vinn” (202b) is sampled to the circuit during the “ON” period of the “ph1” clock (214a).

(29) The pattern of arranging the capacitors (210), switches (212) is shown in FIGS. 2A and 2B, wherein each capacitor is connected to the analog input (202a/202b) through two switches (212), each switch is controlled by the “ph1” clock (214a) of FIG. 2C and the corresponding digital input (204) bit respectively of FIG. 2D. The digital input bit is one among “b6” to “b0” bits. In addition to that, each capacitor (210) is short circuited through two switches, (212), wherein each switch is controlled by the “ph1” clock (214a) and the complement of the corresponding digital input (208) bit. Hence, with the same clock (“ph1”), the analog input (204a/204b) is sampled into the capacitors (210) corresponding to the digital input (204) bits which are “1”, and rest of the capacitors are reset. In other words, both sampling of analog input and reset operation are performed in the same clock cycle. For example, if the digital input bit “b5” is “1” and “ph1” is “ON”, then “32C” is connected to “Vinp(202a)/Vinn(202b)”. If “b5” is “0” and “ph1” is “ON” then “32C” is reset. Advantages of this arrangement/pattern are, (i) no additional clock phase is required to reset the capacitors, (ii) the existing charge on the capacitors are reused in the next operation so that the power requirement for driving the analog inputs (202a/202b) in each cycle is reduced.

(30) Operation of the Circuit:

(31) During “ON” time of “ph1” clock (214a) of FIG. 2C, the analog input (Vinp (202a) or Vinn (202b) which is decided by the sign bit (MSB)) is sampled into the capacitors (210) corresponding to the digital input (204) bits which are “1”. For example, if digital input is “01011011(2)” then the analog input “Vinp” is sampled into 64C, 16C, 8C, 2C and 1C capacitors through the switches (212) controlled by corresponding digital input (204) bits namely b6, b4, b3, b1, b0. Hence the total charge is Qin=Vinp*91C (91C=64C+16C+8C+2C+1C). At the same time the capacitors namely 32C and 4C are reset through the switches controlled by the complements of the corresponding digital input bits namely b5b and b2b. Hence the charge on the two capacitors is zero.

(32) During the “ON” time of “ph2” clock (214b) of FIG. 2C, all the capacitors are connected in parallel through the switches (212). Since all the capacitors are connected in parallel, the charge stored during the “ON” time of “ph1” clock (214a) is shared among all the capacitors. Hence generated voltage output (206a) during the “ON” time of “ph2” (214b) is corresponding to the product of analog and digital inputs.

(33) For example, as stated in the previous example, if the stored charge is Qin=Vinp*91C, the generated output voltage is Voutp=Vinp*91C/127C=Vinp*91/127, wherein 127C is the total capacitance (64C+32C+16C+8C+4C+2C+1C).

(34) At the same time, the other half circuit takes the analog input “Vinn” (202b) and computes the analog output “Voutn” (Voutn=Vinn*91/127) (206b) in a similar manner.

(35) FIGS. 3A and 3B show a schematic diagram of resistive string-based multiplier using resistors (300), in accordance with an embodiment of the present invention. As shown in the figure (300), the differential version of the 8-bit resistive string multiplier comprises of two main resistive strings (306a, 306b), two sub resistive strings (308a,308b), four decoders (312a,312b,312c,312d), switching mechanisms (S00 to S77) and two capacitors (314,316).

(36) Each of the main resistive string 306a (or 306b) comprises of 16 resistors (R0 to R15) and divide the input voltage Vinp (or Vinn) into 16 distinct values from 0 to input voltage vinp (or vinn) (302a/302b) with step vinp/16 (or vinn/16). Each of the sub resistive string 308a (or 308b) comprises of 16 resistors (R16 to R31) and divide the vinp/16 (or Vinn/16) value of the main resistive string voltage 306a (or 306b) into again 16 distinct values with step vinp/256 (or vinn/256).

(37) The coefficient 8 bits (B) (304) of FIG. 3C is subdivided into four most significant bits (4 MSBs) (304b) and four least significant bits (4 LSBs) (304a). Based on the 4 MSB (304b) values, during the “ON” period of the CLK (322a), one of the node voltage of the main resistive strings 306a (or 306b) is sampled on the top plate 314a(or 316a) of the capacitor 314 (or 316) through a first clock switch (SC1). Similarly, based on the 4 LSB values, during the “ON” period of the CLK (322a), one of the node voltage of sub resistive strings 308a (or 308b) is sampled on the bottom plate 314b (or 316b) of the capacitor 314 (or 316) through a second clock switch (SC2). The first clock switch (SC1) and the second clock switch (SC2) are closed during ‘ON’ period of the clock. The switches (S00-S03, S10-S13, S20-S23, S30-S33, col0, col1, col2 and col3; S44-S47, S54-S57, S64-S67, S74-S77, col4, col5, col6 and col7) and decoders (312a, 312b; 312c, 312d) for their control are arranged, to maintain the voltage across the capacitor, proportional to the product of input voltage (vinp or vinn) (302a, 302b) and the digital coefficient (B) (304). The third clock switch (SC3) is closed and the bottom plate of the first capacitor is connected to the common mode voltage during “off” period of the clock such that the analog voltage output is available at the top plate of the capacitor, which is proportional to the product of input voltage (vinp or vinn) (302a, 302b) and the digital coefficient (B) (304). The 8-bit width for digital input is taken or mentioned here for explanation purpose only. However, the present invention is also applicable to n-bit width.

(38) During the “ON” period of the CLK, the bottom plate 314b (or 316b) of the capacitor 314 (or 316) gets connected to common mode voltage (VCM) (310) and also the top plate 314a (or 316a) voltage is available as an output voltage.

(39) Hence, the output voltage 320a (or 320b) of the Multiplier is proportional to the product of input voltage Vinp (or Vinn) and the digital coefficient (B) (304). For example, if vinp=400 mV, Vinn=0V, Vcm=200 mV and B=10101001(2) then during the positive half cycle of CLK/the “ON” period of the “ph1” CLK, 24 the capacitor C1 top plate gets a voltage equlal to 1011(2)*(vinp-Vcm)/16 (i.e., 11*(Vinp-Vcm)/16 and the bottom plate capacitor gets the voltage equal to (1111(2)-1001(20-1)*(Vinp-Vcm)/256 (i.e., 5*(vinp-vcm)/256). Finally, the difference between the two plates voltage is equal to 10101001(2)*(Vinp-Vcm)/256. The differential output, Vout is equal to vin*B/256. The operation of the circuit explained hereinabove with respect to the positive side input and output signals Vinp and Voutp. The present system multiplies the vinp and Vinn analog input signals with the digital coefficient (B) and produces analog outputs Voutp and Voutn respectively.

(40) Advantages:

(41) 1. One multiplication and addition operation is performed within one clock period. Hence the circuit is faster for a specific power.

(42) 2. No pipelined operation is performed in the circuit and hence no latency at the output.

(43) 3. Each DAC and ADC are shared by multiple PE cells.

(44) 4. Number of DACs and ADCs used in the present invention are equal to number of rows (“m”) and columns (“n”) in the PE matrix respectively.

(45) 5. The present invention is scalable to different operating clock speeds, different input and output sizes.

(46) 6. In the capacitor-based multiplier, the input sampling and reset of capacitors are performed in the same clock cycle.

(47) 7. No extra cycle is required to reset the capacitors.

(48) 8. The charge stored on the capacitors is partially reused in the next multiplication cycle, so that the energy required to charge the capacitors in the next computation cycle is reduced. Hence additional power saving is achieved.

(49) 9. The resistor-based multiplier generates fully monotonic output.

(50) Having described and illustrated the principles of the invention with reference to described embodiments, it will be recognized that the described embodiments may be modified in arrangement and detail without departing from such principles.

(51) In view of the many possible embodiments to which the principles of the invention may be applied, we claim the invention as all such embodiments may come within the scope and spirit of the claims and equivalents thereto.

(52) While the present invention has been related in terms of the foregoing embodiments, those skilled in the art will recognize that the invention is not limited to the embodiments depicted. The present invention may be practiced with modification and alteration within the spirit and scope of the appended claims. Thus, the description is to be regarded as illustrative instead of restrictive on the present invention.

(53) The detailed description is presented to enable a person of ordinary skill in the art to make and use the invention and is provided in the context of the requirement for obtaining a patent. The present description is the best presently-contemplated method for carrying out the present invention. Various modifications to the preferred embodiment will be readily apparent to those skilled in the art and the generic principles of the present invention may be applied to other embodiments, and some features of the present invention may be used without the corresponding use of other features. Accordingly, the present invention is not intended to be limited to the embodiment shown but is to be accorded the widest scope consistent with the principles and features described herein.