A technique utilizing speculative execution and rollback for performing data parallel training of a neural network model is disclosed. Activations for a layer of the neural network model are normalized during a speculative normalization operation using estimated normalization parameters associated with a partial population of a set of training data allocated to a particular processor. Normalization parameters associated with the total population of the set of training data are generated by a distributed reduce operation in parallel with the speculative normalization operation. An optional rollback operation can revert the activations to a pre-normalization state if the estimated normalization parameters for the partial population are subsequently determined to be inaccurate compared to the normalization parameters for the population of the set of training data distributed across a plurality of processors.

A method for quantizing a machine learning model during an inference phase, including determining a normalization factor using a set of floating-point values and a damped value of a damped value sequence; and assigning a quantized value for each floating-point value of the set of floating-point values based on the damped value sequence and the normalization factor.

A data processing apparatus is provided that includes input circuitry to receive a signal corresponding to a square root instruction that identifies an input value. Processing circuitry performs an iterative square root operation on the input value and includes digit determination circuitry to determine, for a current iteration, a next digit of an least partial result of the square root operation and remainder determination circuitry that determines, for the current iteration, an at least partial remainder of the square root operation. The next digit for the current iteration is determined based on an least partial remainder of the square root operation from a previous iteration. The at least partial remainder for the current iteration is determined based on the at least partial remainder and the at least partial result of the square root operation from the previous iteration and the processing circuitry is adapted to speculatively generate a set of candidate at least partial remainders of the square root operation for the current iteration prior to the at least partial result of the square root operation for the current iteration being determined.

An arithmetic unit includes a multiplier multiplying first and second inputs to output a multiplication result, an adder adding the third input to the multiplication result to output a multiplication addition result, a normalization shift circuit shifting the multiplication addition result left with a left shift amount, and a left shift amount prediction circuit. The adder includes a carry-save adder adding a first addition value and a first carry value to the third input and a full adder outputting the multiplication addition result. The left shift amount prediction circuit includes a leading zero count circuit generating a leading zero count, a leading one count circuit generating a leading one count, and a correction circuit correcting the leading one count to zero when NOR of respective least significant bits of the M upper order bits of the second addition value and the second carry value of the full adder is true.

A system for block floating point computation in a neural network receives a plurality of floating point numbers. An exponent value for an exponent portion of each floating point number of the plurality of floating point numbers is identified and mantissa portions of the floating point numbers are grouped. A shared exponent value of the grouped mantissa portions is selected according to the identified exponent values and then removed from the grouped mantissa portions to define multi-tiered shared exponent block floating point numbers. One or more dot product operations are performed on the grouped mantissa portions of the multi-tiered shared exponent block floating point numbers to obtain individual results. The individual results are shifted to generate a final dot product value, which is used to implement the neural network. The shared exponent block floating point computations reduce processing time with less reduction in system accuracy.

The disclosed computer-implemented method may include receiving an input value and a floating-point scaling factor and determining (1) an integer scaling factor based on the floating-point scaling factor, (2) a pre-scaling adjustment value representative of a number of places by which to shift a binary representation of the input value prior to a scaling operation, and (3) a post-scaling adjustment value representative of a number of places by which to shift the binary representation of the input value following the scaling operation. The method may further include calculating a scaled result value by (1) shifting rightwards the binary representation of the input value by the pre-scaling adjustment value, (2) scaling the shifted binary representation of the input value by the integer scaling factor, and (3) shifting rightwards the shifted and scaled binary value by the post-scaling adjustment value. Various other methods, systems, and computer-readable media are also disclosed.

The invention relates to a method for determining a value of an integer scaling in a linking of input sets to output sets, wherein the linking comprises operators, each of which has operator inputs and operator outputs that are at least partially linked to one another or to the input sets or to the output sets, by using a computer device having a processing unit, a memory unit, and an output unit. Representations of set objects are used to efficiently carry out rescaling operations within the linking, with up to infinitely large resolution sets. This procedure makes it possible to calculate resource-conserving integer scalings for a target system while taking secondary conditions into account.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for a circuit configured to perform computations using multiple inputs. The circuit includes multiple adder circuits and a selection circuit that includes multiple input selector. Each adder circuit performs an addition operation using sets of inputs derived from the multiple inputs. The input selectors are configured to select one or more inputs from a set of inputs derived from the multiple inputs based on a sign bit for an input in the set and pass the selected inputs to an adder circuit that generates a sum using the selected inputs. The circuit determines a routing of the sum to another adder circuit based in part on a sign bit for the input in the set of inputs.

A process for performing vector dot products receives a row vector and a column vector as floating point numbers in a format of sign plus exponent bits plus mantissa bits. The process generates a single dot product value by separately processing the sign bits, exponent bits, and mantissa bits to form a sign bit, a normalized mantissa formed by multiplying pairs multiplicand elements, and exponent information including MAX_EXP and EXP_DIFF. A second pipeline stage receives the multiplied pairs of normalized mantissas, optionally performs an exponent adjustment, pads, complements and shifts the normalized mantissas, and the results are added in a series of stages until a single addition result remains, which is normalized using MAX_EXP to form the floating point output result.

A system for block floating point computation in a neural network receives a plurality of floating point numbers. An exponent value for an exponent portion of each floating point number of the plurality of floating point numbers is identified and mantissa portions of the floating point numbers are grouped. A shared exponent value of the grouped mantissa portions is selected according to the identified exponent values and then removed from the grouped mantissa portions to define multi-tiered shared exponent block floating point numbers. One or more dot product operations are performed on the grouped mantissa portions of the multi-tiered shared exponent block floating point numbers to obtain individual results. The individual results are shifted to generate a final dot product value, which is used to implement the neural network. The shared exponent block floating point computations reduce processing time with less reduction in system accuracy.