A method comprising storing a plurality of entries, each entry of the plurality of entries associated with a portion of a range of input values, each entry of the plurality of entries comprising a set of coefficients defining a power series approximation; selecting first entry of the plurality of entries based on a determination that a floating point input value is within a portion of the range of input values that is associated with the first entry; and calculating an output value by evaluating the power series approximation defined by the set of coefficients of the first entry at the floating point input value.

The present disclosure concerns a method of executing a SoftMax function, the method comprising: (i) pre-storing in memory M fraction components (fc.sub.j) in binary form, derived from the expression 2.sup.(j/M), said fc.sub.j forming a lookup table (T) of size M; (ii) calculating, for each z.sub.i, an element y.sub.i of a number of the form 2.sup.y.sup.i; (iii) separating y.sub.i into an integral part (int.sub.i) and a fractional part (fract.sub.i); (iv) determining a lookup index (ind.sub.i) that corresponds to fract.sub.i scaled by the size M; (v) retrieving a fraction component fc.sub.i from T with ind.sub.i; (vi) generating, in a result register, a binary number representative of the exponential value of said z.sub.i, by combining said fc.sub.i retrieved from T and said int.sub.i; (v) adding the K result registers corresponding to z.sub.i into a sum register R7; and (vi) determining the K probability values p.sub.i from the K result registers and the sum register.

A processor determines an exponent common to a plurality of numerical values, determines a mantissa for each of the plurality of numerical values based on the determined exponent, and performs four arithmetic operations using a sign, the determined exponent, and the determined mantissa.

A processor-implemented includes receiving a first floating point operand and a second floating point operand, each having an n-bit format comprising a sign field, an exponent field, and a significand field, normalizing a binary value obtained by performing arithmetic operations for fields corresponding to each other in the first and second floating point operands for an n-bit multiplication operation, determining whether the normalized binary value is a number that is representable in the n-bit format or an extended normal number that is not representable in the n-bit format, according to a result of the determining, encoding the normalized binary value using an extension bit format in which an extension pin identifying whether the normalized binary value is the extended normal number is added to the n-bit format, and outputting the encoded binary value using the extended bit format, as a result of the n-bit multiplication operation.

An artificial intelligence (AI) accelerator includes memory circuits configured to output weight data and vector data, a multiplication circuit/adder tree performing a multiplying/adding calculation on the weight data and the vector data to generate multiplication/addition result data, a first accumulator synchronized with an odd clock signal to perform an accumulative adding calculation on odd-numbered multiplication/addition result data of the multiplication/addition result data and a first latched data, and a second accumulator synchronized with an even clock signal to perform an accumulative adding calculation on even-numbered multiplication/addition result data of the multiplication/addition result data and a second latched data.

An information processing apparatus that performs deep learning using a neural network includes a memory, and an arithmetic processing device that performs a process for layers of the neural network in a predetermined direction. The process for the layers includes: pre-determining a decimal point position of a fixed-point number of an intermediate data obtained by an operation of each of the layers; performing the arithmetic operation for each layer with the pre-determined decimal point position to obtain the intermediate data and acquiring first statistical information of a distribution of bits of the intermediate data; determining a decimal point position of the intermediate data based on the statistical information; and performing the arithmetic operation for each layer with the determined decimal point position again when the difference of the determined decimal point position and the pre-determined decimal point position is greater than a threshold value.

An information processing apparatus that performs deep learning using a neural network includes a memory, and an arithmetic processing device that performs a process for layers of the neural network in a predetermined direction. The process for the layers includes: pre-determining a decimal point position of a fixed-point number of an intermediate data obtained by an operation of each of the layers; performing the arithmetic operation for each layer with the pre-determined decimal point position to obtain the intermediate data and acquiring first statistical information of a distribution of bits of the intermediate data; determining a decimal point position of the intermediate data based on the statistical information; and performing the arithmetic operation for each layer with the determined decimal point position again when the difference of the determined decimal point position and the pre-determined decimal point position is greater than a threshold value.

Certain aspects of the present disclosure provide techniques for adaptively reducing the bit size of features in a training data set used to train a machine learning model. An example method generally includes receiving a data set to be used in training a machine learning model and a definition of the machine learning model to be trained. A reduced number of bits to represent features in the data set is determined based on values of each feature in the data set and the definition of the machine learning model. A reduced bit-size data set is generated by reducing a bit size of each feature in the data set according to the reduced number of bits, and the reduced bit-size data set is encrypted using a homomorphic encryption scheme. A machine learning model is trained based on the encrypted reduced bit-size data set.

Certain aspects of the present disclosure provide techniques for adaptively reducing the bit size of features in a training data set used to train a machine learning model. An example method generally includes receiving a data set to be used in training a machine learning model and a definition of the machine learning model to be trained. A reduced number of bits to represent features in the data set is determined based on values of each feature in the data set and the definition of the machine learning model. A reduced bit-size data set is generated by reducing a bit size of each feature in the data set according to the reduced number of bits, and the reduced bit-size data set is encrypted using a homomorphic encryption scheme. A machine learning model is trained based on the encrypted reduced bit-size data set.

Systems and methods for compression of data that exhibits mixed compressibility, such as floating-point data, are provided. As one example, aspects of the present disclosure can be used to compress floating-point data that represents the values of parameters of a machine-learned model. Therefore, aspects of the present disclosure can be used to compress machine-learned models (e.g., for reducing storage requirements associated with the model, reducing the bandwidth expended to transmit the model, etc.).