An instruction to perform a recurrent neural network cell activation is executed. The executing includes performing a plurality of operations of the recurrent neural network cell activation to provide a result of the recurrent neural network cell activation. The plurality of operations is performed in a single invocation of the instruction. The recurrent neural network cell activation is, for instance, a long short-term memory cell activation or a gated recurrent unit cell activation.

An instruction to perform a recurrent neural network cell activation is executed. The executing includes performing a plurality of operations of the recurrent neural network cell activation to provide a result of the recurrent neural network cell activation. The plurality of operations is performed in a single invocation of the instruction. The recurrent neural network cell activation is, for instance, a long short-term memory cell activation or a gated recurrent unit cell activation.

According to one embodiment, in a semiconductor integrated circuit, the plurality of storage devices are arranged in a form of a plurality of rows. Each of the storage devices are configured to store a bit position value of a weight of multiple bits. The plurality of multiplication circuits are arranged in a form of a plurality of rows and are configured to multiply a plurality of input voltages by the weight of multiple bits to generate a plurality of multiplication results. The one or more capacitive devices are configured to accumulate charges corresponding to the plurality of multiplication results. The adder circuit are configured to generate an output voltage corresponding to the total value of the charges accumulated in the one or more capacitive devices. The plurality of input voltages have different amplitudes. Each of the input voltages is associated with a corresponding bit position of the weight.

According to one embodiment, in a semiconductor integrated circuit, the plurality of storage devices are arranged in a form of a plurality of rows. Each of the storage devices are configured to store a bit position value of a weight of multiple bits. The plurality of multiplication circuits are arranged in a form of a plurality of rows and are configured to multiply a plurality of input voltages by the weight of multiple bits to generate a plurality of multiplication results. The one or more capacitive devices are configured to accumulate charges corresponding to the plurality of multiplication results. The adder circuit are configured to generate an output voltage corresponding to the total value of the charges accumulated in the one or more capacitive devices. The plurality of input voltages have different amplitudes. Each of the input voltages is associated with a corresponding bit position of the weight.

Aspects for Long Short-Term Memory (LSTM) blocks in a recurrent neural network (RNN) are described herein. As an example, the aspects may include one or more slave computation modules, an interconnection unit, and a master computation module collectively configured to calculate an activated input gate value, an activated forget gate value, a current cell status of the current computation period, an activated output gate value, and a forward pass result.

Aspects for Long Short-Term Memory (LSTM) blocks in a recurrent neural network (RNN) are described herein. As an example, the aspects may include one or more slave computation modules, an interconnection unit, and a master computation module collectively configured to calculate an activated input gate value, an activated forget gate value, a current cell status of the current computation period, an activated output gate value, and a forward pass result.

A semiconductor device executes the processing of a neural network. The memory MEM1 holds a plurality of pixel values and j compressed weighting factors. The decompressor DCMP restores the j compressed weighting factors to the uncompressed k (k≥j) weighting factors. The DMA controller DMAC1 reads the j compressed weighting factors from the memory MEM1 and transfers them to the decompressor DCMP. The n (n>k) accumulators in the accumulator unit ACCU multiply a plurality of pixel values and k uncompressed weighting factor to accumulate and add the multiplication results to the time series. A switch circuit SW1 provided between the decompressor DCMP and the accumulator unit ACCU transfers the k uncompressed weighting factors restored by the decompressor DCMP to n accumulators based on the correspondence represented by the identifier.

A semiconductor device executes the processing of a neural network. The memory MEM1 holds a plurality of pixel values and j compressed weighting factors. The decompressor DCMP restores the j compressed weighting factors to the uncompressed k (k≥j) weighting factors. The DMA controller DMAC1 reads the j compressed weighting factors from the memory MEM1 and transfers them to the decompressor DCMP. The n (n>k) accumulators in the accumulator unit ACCU multiply a plurality of pixel values and k uncompressed weighting factor to accumulate and add the multiplication results to the time series. A switch circuit SW1 provided between the decompressor DCMP and the accumulator unit ACCU transfers the k uncompressed weighting factors restored by the decompressor DCMP to n accumulators based on the correspondence represented by the identifier.

A non-volatile memory device for performing compute in memory operations for a neural network uses a three dimensional NOR architecture in which vertical NOR strings are formed of multiple memory cells connected in parallel between a source line and a bit line. Weights of the neural network are encoded as threshold voltages of the memory cells and activations are encoded as word line voltages applied to the memory cells of the NOR strings. The memory cells are operated in the subthreshold region, where the word line voltages are below the threshold voltages. The NOR structure naturally sums the resultant subthreshold currents of the individual memory cells to generate the product of the activations and the weights of the neural network by concurrently applying input voltages to multiple memory cells of a NOR string.

A non-volatile memory device for performing compute in memory operations for a neural network uses a three dimensional NAND architecture. Multi-bit weight values are stored encoded as sets of threshold voltages for sets of memory cells. A weight value is stored in multiple memory cells on the same word line and connected between a bit line and a source line, each of the memory cells programmed to one of multiple threshold voltages. When multiplying an input value with the weight value, the word line is biased so that, for at least one of the threshold voltages, the memory cell will be in the linear operation region. Input values are encoded as a set of one or more voltage levels applied to a corresponding set of bit lines, each bit line connected memory cells also storing the weight value, connected to the word line, and connected to the source line.