The embodiments herein describe a conversion engine that converts floating point data into integer data using a dynamic scaling factor. To select the scaling factor, the conversion engine compares a default (or initial) scaling factor value to an exponent portion of the floating point value to determine a shift value with which to bit shift a mantissa of the floating point value. After bit shifting the mantissa, the conversion engine determines whether the shift value caused an overflow or an underflow and whether that overflow or underflow violates a predefined policy. If the policy is violated, the conversion engine adjusts the scaling factor and restarts the conversion process. In this manner, the conversion engine can adjust the scaling factor until identifying a scaling factor that converts all the floating point values in the batch without violating the policy.

A multiple-input floating-point number processing method is provided. The method includes: acquiring a plurality of floating-point numbers corresponding to a target task; extracting an exponential value of an exponent part and a mantissa value of a mantissa part in each of the floating-point numbers respectively; sorting, according to a magnitude of the exponential value of each of the floating-point numbers, the plurality of floating-point numbers to obtain a sorting result; allocating, based on the sorting result, a shifter for each of the floating-point numbers from a plurality of shifters with different preset bits; performing, for each of the floating-point numbers, shift processing on the mantissa value of the corresponding floating-point number through the shifter allocated for the floating-point number to obtain a shift result; and determining a floating-point number processing result corresponding to the target task based on each of shift results.

A processing device comprising: a control register configured to store a scaling factor; at least one execution unit configured to execute instructions to perform arithmetic operations on input floating-point numbers provided according to a first floating-point format, wherein each of the input floating-point numbers provided according to the first floating-point format comprises a predetermined number of bits, wherein the at least one execution unit is configured to, in response to execution of an instance of a first of the instructions: perform processing of a first set of the input floating-point numbers to generate a result value, the result value provided in a further format and comprising more the predetermined number of bits, enabling representation of a greater range of values than is representable in the first floating-point format; and apply the scaling factor specified in the control register to increase or decrease an exponent of the result value.

A binary logic circuit converts a number in floating point format having an exponent E, an exponent bias B=2.sup.ew-1−1, and a significand comprising a mantissa M of mw bits into a fixed point format with an integer width of iw bits and a fractional width of fw bits. The circuit includes an offset unit configured to offset the exponent of the floating point number by an offset value equal to (iw−1−s.sub.y) to generate a shift value s.sub.v of sw bits given by s.sub.v=(B−E)+(iw−1−s.sub.y), the offset value being equal to a maximum amount by which the significand can be left-shifted before overflow occurs in the fixed point format; a right-shifter operable to receive a significand input comprising a formatted set of bits derived from the significand, the shifter being configured to right-shift the input by a number of bits equal to the value represented by k least significant bits of the shift value to generate an output result, where bitwidth[min(2.sup.ew-1−1, iw−1−s.sub.y)+min(2.sup.ew-1−2, fw)]≤k≤sw, where s.sub.y=1 for a signed floating point number and s.sub.y=0 for an unsigned floating point number.

A graphics processor is described that includes a single instruction, multiple thread (SIMT) architecture including hardware multithreading. The multiprocessor can execute parallel threads of instructions associated with a command stream, where the multiprocessor includes a set of functional units to execute at least one of the parallel threads of the instructions. The set of functional units can include a mixed precision tensor processor to perform tensor computations to generate loss data. The loss data is stored as a first floating-point data type and scaled by a scaling factor to enable a data distribution of a gradient tensor generated based on the loss data to be represented by a second floating point data type.

Apparatus and methods are disclosed for performing block floating-point (BFP) operations, including in implementations of neural networks. All or a portion of one or more matrices or vectors can share one or more common exponents. Techniques are disclosed for selecting the shared common exponents. In some examples of the disclosed technology, a method includes producing BFP representations of matrices or vectors, at least two elements of the respective matrices or vectors sharing a common exponent, performing a mathematical operation on two or more of the plurality of matrices or vectors, and producing an output matrix or vector. Based on the output matrix or vector, one or more updated common exponents are selected, and an updated matrix or vector is produced having some elements that share the updated common exponents.

A graphics processor is described that includes a single instruction, multiple thread (SIMT) architecture including hardware multithreading. The multiprocessor can execute parallel threads of instructions associated with a command stream, where the multiprocessor includes a set of functional units to execute at least one of the parallel threads of the instructions. The set of functional units can include a mixed precision tensor processor to perform tensor computations to generate loss data. The loss data is stored as a first floating-point data type and scaled by a scaling factor to enable a data distribution of a gradient tensor generated based on the loss data to be represented by a second floating point data type.

A processor-implemented method for fast floating point simulations with learnable parameters includes receiving a single precision input. An integer quantization process is performed on the input. Each element of the input is scaled based on a scaling parameter to generate an m-bit floating point output, where m is an integer.

A fused dot-product multiply-accumulate (MAC) circuit may support variable precisions of floating-point data elements to perform computations (e.g., MAC operations) in deep learning operations. An operation mode of the circuit may be selected based on the precision of an input element. The operation mode may be a FP16 mode or a FP8 mode. In the FP8 mode, product exponents may be computed based on exponents of floating-point input elements. A maximum exponent may be selected from the one or more product exponents. A global maximum exponent may be selected from a plurality of maximum exponents. A product mantissa may be computed and aligned with another product mantissa based on a difference between the global maximum exponent and a corresponding maximum exponent. An adder tree may accumulate the aligned product mantissas and compute a partial sum mantissa. The partial sum mantissa may be normalized using the global maximum exponent.

A graphics processor is described that includes a single instruction, multiple thread (SIMT) architecture including hardware multithreading. The multiprocessor can execute parallel threads of instructions associated with a command stream, where the multiprocessor includes a set of functional units to execute at least one of the parallel threads of the instructions. The set of functional units can include a mixed precision tensor processor to perform tensor computations. The functional units can also include circuitry to analyze statistics for output values of the tensor computations, determine a target format to convert the output values, the target format determined based on the statistics for the output values and a precision associated with a second layer of the neural network, and convert the output values to the target format.