Systems and methods that provide reconfigurable shifter configurations supporting multiple instruction, multiple data (MIMD) are described. Shifters implemented according to embodiments support multiple data shifts with respect to an instance of data shifting, wherein multiple individual different data shifts are implemented at a time in parallel. Reconfigurable segmented scalable shifters of embodiments, in addition being reconfigurable for scalability in supporting data shifting with respect to various bit lengths of data, are configured to support data shifting of differing bit lengths in parallel. The data shifters of embodiments implement segmentation for facilitating data shifting with respect to differing bit lengths. Different data shift commands may be provided with respect to each such segment, thereby facilitating multiple data shifts in parallel with respect to various bit lengths of data. Reconfigurable segmented scalable shifter configurations provide for fully reconfigurable data width and shift command of each message of input data.

Systems and methods that provide reconfigurable shifter configurations supporting multiple instruction, multiple data (MIMD) are described. Shifters implemented according to embodiments support multiple data shifts with respect to an instance of data shifting, wherein multiple individual different data shifts are implemented at a time in parallel. Reconfigurable segmented scalable shifters of embodiments, in addition being reconfigurable for scalability in supporting data shifting with respect to various bit lengths of data, are configured to support data shifting of differing bit lengths in parallel. The data shifters of embodiments implement segmentation for facilitating data shifting with respect to differing bit lengths. Different data shift commands may be provided with respect to each such segment, thereby facilitating multiple data shifts in parallel with respect to various bit lengths of data. Reconfigurable segmented scalable shifter configurations provide for fully reconfigurable data width and shift command of each message of input data.

Hardware structures for check pointing a main shift register one or more times which include a circular buffer used to store the data elements most recently shifted onto the main shift register which has an extra data position for each check point and an extra data position for each restorable point in time; an update history shift register which has a data position for each check point which is used to store information indicating whether the circular buffer was updated in a particular clock cycle; a pointer that identifies a subset of the data positions of the circular buffer as active data positions; and check point generation logic that derives each check point by selecting a subset of the active data positions based on the information stored in the update history shift register.

Hardware structures for check pointing a main shift register one or more times which include a circular buffer used to store the data elements most recently shifted onto the main shift register which has an extra data position for each check point and an extra data position for each restorable point in time; an update history shift register which has a data position for each check point which is used to store information indicating whether the circular buffer was updated in a particular clock cycle; a pointer that identifies a subset of the data positions of the circular buffer as active data positions; and check point generation logic that derives each check point by selecting a subset of the active data positions based on the information stored in the update history shift register.

Hardware structures for check pointing a main shift register one or more times which include a circular buffer used to store the data elements most recently shifted onto the main shift register which has an extra data position for each check point and an extra data position for each restorable point in time; an update history shift register which has a data position for each check point which is used to store information indicating whether the circular buffer was updated in a particular clock cycle; a pointer that identifies a subset of the data positions of the circular buffer as active data positions; and check point generation logic that derives each check point by selecting a subset of the active data positions based on the information stored in the update history shift register.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting neural networks with an external memory. One of the methods includes receiving a plurality of high-dimensional data items; generating a circulant embedding matrix for the high-dimensional data items, wherein the circulant embedding matrix is a matrix that is fully specified by a single vector; for each high-dimensional data item, generating a compact representation of the high-dimensional data item, comprising computing a product of the circulant embedding matrix and the high dimensional data item by performing a circular convolution of the single vector that fully specifies the circulant embedding matrix and the high dimensional data item using a Fast Fourier Transform (FFT); and generating a compact representation of the high dimensional data item by computing a binary map of the computed product.

A hardware structure provides a way for check pointing a main shift register one or more times. The hardware structure includes an extended shift register used to store the data elements most recently shifted onto the main shift register which has an extra data position for each check point. An update history shift register has a data position for each check point which is used to store information indicating whether the extended shift register was updated. Check point generation logic derives each check point by selecting a subset of the data elements stored in the extended shift register based on the information stored in the update history shift register.

Hardware structures for check pointing a main shift register one or more times which include a circular buffer used to store the data elements most recently shifted onto the main shift register which has an extra data position for each check point and an extra data position for each restorable point in time; an update history shift register which has a data position for each check point which is used to store information indicating whether the circular buffer was updated in a particular clock cycle; a pointer that identifies a subset of the data positions of the circular buffer as active data positions; and check point generation logic that derives each check point by selecting a subset of the active data positions based on the information stored in the update history shift register.

The present specification discloses a computing device architecture and method for executing computationally intensive attention mechanisms, as commonly utilized in transformer models for artificial intelligence. In an embodiment, the device includes a row of interconnected processing elements (PEs), each linked to dedicated coefficient memory units (CRAMs) and controlled by a central controller. By employing a diagonal-offset and transposition technique for matrix coefficients, the architecture enables efficient execution of Generalized Matrix-Vector (GEMV) operations.

A decoding method includes: using a syndrome calculation check circuit to calculate a syndrome value of variable node V.sub.i of a parity check matrix based on multiple input bits; rotating the syndrome value of variable node V.sub.i into the direction of variable node V.sub.i+1 to generate an estimated syndrome value of variable node V.sub.i+1; rotating the syndrome value of variable node V.sub.i1 into the direction of variable node V.sub.i to generate a rotated syndrome information; performing a weighted-sum calculation based on multiple rotated syndrome information to generate a flipping function value; and comparing the flipping function value with a flipping threshold to generate a flipping result; the syndrome calculation check circuit determines whether to modify/flip a specific bit according to the flipping result.