An apparatus has matrix processing circuitry to perform a matrix processing operation on first and second input operands to generate a 2D result matrix; operand storage circuitry to store information for forming the first and second input operands for the matrix processing circuitry; and position shifting circuitry to apply a variable position shift to vary which row/column of the result matrix is updated based on a given element of one of the first and second input operands stored in the operand storage circuitry during a given matrix processing operation. The variable position shift is based on one of a plurality of alternative shift amounts, each alternative hift amount corresponding to a position shift of the one of the first and second input operands relative to the result matrix by a different umber of rows/columns. This is useful for performing 2D convolution operations.

An apparatus has matrix processing circuitry to perform a matrix processing operation on first and second input operands to generate a 2D result matrix; operand storage circuitry to store information for forming the first and second input operands for the matrix processing circuitry; and position shifting circuitry to apply a variable position shift to vary which row/column of the result matrix is updated based on a given element of one of the first and second input operands stored in the operand storage circuitry during a given matrix processing operation. The variable position shift is based on one of a plurality of alternative shift amounts, each alternative hift amount corresponding to a position shift of the one of the first and second input operands relative to the result matrix by a different umber of rows/columns. This is useful for performing 2D convolution operations.

A normalizer receives input data including first exponent data and first mantissa data and generates normalized output data. The normalizer includes a mantissa alignment circuit, a “1” search circuit, an exponent addition circuit, a normalization circuit. The mantissa alignment circuit outputs second mantissa data including a binary point shifted by one bit to the left in comparison to a binary point of the first mantissa data. The “1” search circuit searches for an uppermost bit position of a leading “1” in the second mantissa data to output shift data. The exponent addition circuit performs an addition operation on the shift data and the first exponent data and configured to perform a “+1” operation on a result of the addition operation to output addition data. The normalization circuit performs normalization when the addition data corresponds to a normalization condition, the normalization being performed by outputting the addition data as exponent data of the output data and outputting result data that is obtained by shifting the second mantissa data by the number of bits that correspond to an absolute value of the shift data as mantissa data of the output data.

Processors and methods for neural network processing are provided. A method includes receiving a subset of data corresponding to a layer of a neural network for processing using the processor. The method further includes during a forward propagation pass: (1) normalizing the subset of the data corresponding to the layer of the neural network based on an average associated with the subset of the data and a variance associated with the subset of the data, where the normalizing the subset of the data comprises dynamically updating the average and dynamically updating the variance, to generate normalized data and (2) applying a transformation to the normalized data using a fixed scale parameter corresponding to the subset of the data and a fixed shift parameter corresponding to the subset of the data such that during the forward propagation pass neither the fixed scale parameter nor the fixed shift parameter is updated.

A method of maintaining a distributed ledger at a client node includes: storing a distributed ledger defining a plurality of records each containing a set of values; storing (i) a local voting weight corresponding to the client node, and (ii) respective remote voting weights for a plurality of remote client nodes; obtaining a proposed update to a record of the distributed ledger; generating a local vote to apply or discard the proposed update and transmitting the local vote to the remote client nodes; receiving remote votes to apply or discard the proposed update from the remote client nodes; determining whether to permit the proposed update based on (i) the local vote and the local voting weight, and (ii) the remote votes and the corresponding remote voting weights; and according to the determination, applying the proposed update to the distributed ledger or discarding the proposed update.

A method to implement a fixed-point batchnorm layer in a neural network for data processing is provided in the present disclosure. The method includes: receiving fixed-point input data over a channel of a standalone floating-point batchnorm layer, and converting the floating-point input data into fixed-point input data of the standalone floating-point batchnorm layer; obtaining fixed-point quantization parameters in each channel based on the input data and floating-point parameters μ.sub.i, σ.sub.i, ε.sub.i in each channel; converting the standalone floating-point batchnorm layer based on the fixed-point quantization parameters into a fixed-point batchnorm layer for processing the fixed-point input data to generate fixed-point output data; and mapping the fixed-point batchnorm layer to a fixed-point convolution layer and the computation of convolution is done by matrix multiplication that can be executed on a GEMM engine.

A processor-implemented data processing method includes encoding a plurality of weights of a filter of a neural network using an inverted two's complement fixed-point format; generating weight data based on values of the encoded weights corresponding to same filter positions of a plurality of filters; and performing an operation on the weight data and input activation data using a bit-serial scheme to control when to perform an activation function with respect to the weight data and input activation data.

A processor-implemented data processing method includes encoding a plurality of weights of a filter of a neural network using an inverted two's complement fixed-point format; generating weight data based on values of the encoded weights corresponding to same filter positions of a plurality of filters; and performing an operation on the weight data and input activation data using a bit-serial scheme to control when to perform an activation function with respect to the weight data and input activation data.

Provided is a method for normalizing embeddings for cross-embedding alignment. The method may include applying mean centering to the at least one embedding set, applying spectral normalization to the at least one embedding set, and/or applying length normalization to the at least one embedding set. Spectral normalization may include decomposing the at least one embedding set, determining an average singular value of the at least one embedding set, determining a respective substitute singular value for each respective singular value of a diagonal matrix, and/or replacing the at least one embedding set with a product of the at least one embedding set, a right singular vector, and an inverse of the substitute diagonal matrix. The mean centering, spectral normalization, and/or length normalization may be iteratively repeated for a configurable number of iterations. A system and computer program product are also disclosed.

Provided is a method for normalizing embeddings for cross-embedding alignment. The method may include applying mean centering to the at least one embedding set, applying spectral normalization to the at least one embedding set, and/or applying length normalization to the at least one embedding set. Spectral normalization may include decomposing the at least one embedding set, determining an average singular value of the at least one embedding set, determining a respective substitute singular value for each respective singular value of a diagonal matrix, and/or replacing the at least one embedding set with a product of the at least one embedding set, a right singular vector, and an inverse of the substitute diagonal matrix. The mean centering, spectral normalization, and/or length normalization may be iteratively repeated for a configurable number of iterations. A system and computer program product are also disclosed.