Disclosed herein is a method for measuring the weight of a discrete entity, performed in a neural network model configured with multiple layers, the method including receiving data configured with the indices of discrete entities, converting the data into embedding vectors corresponding to respective indices through an embedding layer, generating a masked vector through element-wise multiplication between a mask vector and the embedding vector, calculating a loss using output based on the masked vector, and training the model based on the loss.

A device for deep learning acceleration with mixed precision may include a precision mode port configured to receive an indication of an output precision mode, a data input port configured to receive an input value, and a truncation component configured to truncate the input value into a keep segment value and a truncate segment value. The device may be configured to add the keep segment value and a carry bit to generate a rounded keep segment value, and to generate a rounded output based on the rounded keep segment value and the output precision mode. The rounded output generation component may be configured to generate the rounded output to include a sign bit of the keep segment value and either a first quantity or a second quantity of lower bits of the keep segment value based on the output precision mode being either a first value or a second value.

A device for deep learning acceleration with mixed precision may include a first precision mode port to receive an indication of an input precision mode and a second precision mode port to receive an indication of an output precision mode. The device may include a first data port to receive map data and a second data port to receive kernel data. The device may include multiply-accumulate (MAC) components that are each configured to generate a MAC output based on the input precision mode, the map data, and the kernel data. The device may include an adder component to generate an adder component output based on the input precision mode and one or more MAC outputs. The device may include a rounding component to round the adder component output, based on the output precision mode, to generate a rounded output, and an output port to output the rounded output.

An information processing device that executes calculation of a neural network, includes a memory; and a processor coupled to the memory and the processor configured to: set a division position for quantization of a variable to be used for the calculation so that a quantization error based on a difference between the variable before the quantization and the variable after the quantization is reduced; and quantize the variable based on the division position set.

A data processing apparatus for performing rounding on an input value to produce a rounded form output value includes floor calculation circuitry that receives the input value in redundant-representation and generates two candidates of a floor of the input value in non-redundant representation. Ceiling calculation circuitry receives the input value in redundant-representation and generates two candidates of a ceiling of the input value in non-redundant representation. Selection circuitry outputs one of the two candidates of the floor of said input value and the two candidates of the ceiling of said input value as the rounded form output value, based on a sign of a residual value associated with the input value. Each of the two candidates of the floor of the input value correspond with different values of the sign of the residual value and each of the two candidates of the ceiling of said input value correspond with different values of the sign of said residual value.

Devices for adding floating point numbers, devices for multiplying floating point numbers, devices for floating-point fused multiply-add operations, devices for performing fixed point number operations, and associated converters thereof. A preprocessed fixed point format is a fixed point format wherein the LSD of all numbers exactly represented in said format is equal to B/2 (i.e. one for binary radix), and the rest are rounded to one of these numbers. A preprocessed floating point format is a floating point format wherein the significand is a preprocessed fixed point number.

In one embodiment, the present invention includes a method for receiving a rounding instruction and an immediate value in a processor, determining if a rounding mode override indicator of the immediate value is active, and if so executing a rounding operation on a source operand in a floating point unit of the processor responsive to the rounding instruction and according to a rounding mode set forth in the immediate operand. Other embodiments are described and claimed.

Embodiments are directed to a computer implemented method for executing machine instructions in a central processing unit. The executing includes loading a first operand into a first operand register, and loading a second operand into a second operand register. The executing further includes shifting either the first operand or the second operand to form a shifted operand. The executing further includes adding or subtracting the first operand and the second operand to obtain a sum or a difference, and loading the sum or the difference having a least significant bit into a third register or a memory. The executing further includes performing a probability analysis on least significant bits of the shifted operand or the non-shifted operand, and initiating a rounding operation on the least significant bit of the sum or the difference based at least in part on the probability analysis.

Embodiments are directed to a computer implemented method for executing machine instructions in a central processing unit. The method includes obtaining, by a processor system, a machine instruction for execution, the machine instruction being defined for computer execution according to a computer architecture. The method further includes executing the machine instruction, wherein the executing includes loading a multiplicand into a multiplicand register, and loading a multiplier into a multiplier register. The executing further generates an intermediate product having least significant bits by multiplying the multiplicand and the multiplier. The executing further includes generating a rounded product by performing a probability analysis on the least significant bits of the intermediate product, and initiating a rounding operation on the intermediate product to produce the rounded product based at least in part on the probability analysis.

An integrated circuit is provided with a modular multiplication circuit. The modular multiplication circuit includes an input multiplier for computing the product of two input signals, truncated multipliers for computing another product based on a modulus value and the product, and a subtraction circuit for computing a difference between the two products. An error correction circuit uses the difference to look up an estimated quotient value and to subtract out an integer multiple of the modulus value from the difference in a single step, wherein the integer multiple is equal to the estimated quotient value. A final adjustment stage is used to remove any remaining residual estimation error.