A training method, system, and computer program product include computing a matrix norm over a product of a weight matrix and a transpose of the weight matrix and using the matrix norm to constrain the L2 non-expansive neural network.

Method decrypting and/or encrypting an input message: providing at least five of sixteen first order logic functions; and decrypting and/or encrypting the input message based on the at least five first order logic functions.

Method decrypting and/or encrypting an input message: providing at least five of sixteen first order logic functions; and decrypting and/or encrypting the input message based on the at least five first order logic functions.

Aspects of the disclosure provide for methods, systems, and apparatuses, including computer-readable storage media, for sparse matrix multiplication. A system for matrix multiplication includes an array of sparse shards. Each sparse shard can be configured to receive an input sub-matrix and an input sub-vector, where the input sub-matrix has a number of non-zero values equal to or less than a predetermined maximum non-zero threshold. The sparse shard can, by a plurality of multiplier circuits, compute one or more products of vector values multiplied with respective non-zero values of the input sub-matrix. The sparse shard can generate, as output to the sparse shard and using the one or more products, a shard output vector that is the product of applying the shard input vector to the shard input matrix.

Aspects of the disclosure provide for methods, systems, and apparatuses, including computer-readable storage media, for sparse matrix multiplication. A system for matrix multiplication includes an array of sparse shards. Each sparse shard can be configured to receive an input sub-matrix and an input sub-vector, where the input sub-matrix has a number of non-zero values equal to or less than a predetermined maximum non-zero threshold. The sparse shard can, by a plurality of multiplier circuits, compute one or more products of vector values multiplied with respective non-zero values of the input sub-matrix. The sparse shard can generate, as output to the sparse shard and using the one or more products, a shard output vector that is the product of applying the shard input vector to the shard input matrix.

Numerous examples of a precision programming apparatus are disclosed for precisely and quickly depositing the correct amount of charge on the floating gate of a non-volatile memory cell within a vector-by-matrix multiplication (VMM) array in an artificial neural network. In one example, a neuron output circuit for providing a current to program as a weight value in a selected memory cell in a vector-by-matrix multiplication array is disclosed, the neuron output circuit comprising a first adjustable current source to generate a scaled current in response to a neuron current to implement a positive weight, and a second adjustable current source to generate a scaled current in response to a neuron current to implement a negative weight.

Numerous examples of a precision programming apparatus are disclosed for precisely and quickly depositing the correct amount of charge on the floating gate of a non-volatile memory cell within a vector-by-matrix multiplication (VMM) array in an artificial neural network. In one example, a neuron output circuit for providing a current to program as a weight value in a selected memory cell in a vector-by-matrix multiplication array is disclosed, the neuron output circuit comprising a first adjustable current source to generate a scaled current in response to a neuron current to implement a positive weight, and a second adjustable current source to generate a scaled current in response to a neuron current to implement a negative weight.

A computing method, suitable for computing a transformer model, include following steps. An input matrix corresponding to an input sequence of feature vectors is projected into a query matrix according to first learnable weights. The input matrix is projected into a value matrix according to second learnable weights. A factorized matrix is generated by an incomplete Cholesky factorization according to the query matrix and a transpose of the query matrix. An intermediate matrix is calculated according to a product between a transpose of the factorized matrix and the value matrix. An output matrix is calculated according to a product between the factorized matrix (H) and the intermediate matrix.

A computing method, suitable for computing a transformer model, include following steps. An input matrix corresponding to an input sequence of feature vectors is projected into a query matrix according to first learnable weights. The input matrix is projected into a value matrix according to second learnable weights. A factorized matrix is generated by an incomplete Cholesky factorization according to the query matrix and a transpose of the query matrix. An intermediate matrix is calculated according to a product between a transpose of the factorized matrix and the value matrix. An output matrix is calculated according to a product between the factorized matrix (H) and the intermediate matrix.

A learning method implemented by a computer, includes: creating an input data tensor including a local dimension and a universal dimension by partitioning series data into local units, the series data including a plurality of elements, each of the plurality of elements in the series data being logically arranged in a predetermined order; and performing machine learning by using tensor transformation in which a transformation data tensor obtained by transforming the input data tensor with a transformation matrix is outputted using a neural network, wherein the learning includes rearranging the transformation matrix so as to maximize a similarity to a matching pattern serving as a reference in the tensor transformation regarding the universal dimension of the input data tensor, and updating the matching pattern in a process of the machine learning regarding the local dimension of the input data tensor.