A computing methodology in digital systems for performing computationally expensive operations while lowering the required computing resources, the power consumed to accomplish the computation, and maximizing the system throughput. Intermediate computations within the operation may be analyzed and those that have low gain values are identified and may be either removed from the computation or calculated with lower precision.

Circuits and methods for determining a maximum bias for computing softmax on a tensor include a processor circuit configured to transform in parallel, elements of each group of a plurality of groups of elements of a tensor X into respective power-of-two elements. The respective power-of-two element from element x.sub.t of the tensor is p.sub.t, p.sub.t=(x.sub.t*log.sub.2e), and p.sub.t has an integer part and a fraction part. A first comparison circuit (204) is configured to determine respective group-level biases for the groups. The group-level bias of group.sub.m is d.sub.m, and d.sub.m is an integer part of a maximum of the power-of-two elements of group.sub.m. A second comparison circuit is configured to determine a greatest one of the respective group-level biases to be a tensor-level bias, d.sub.max.

An adder circuit inhibiting overflow is provided. A first memory, a second memory, a third memory, and a fourth memory are included. A step of supplying first data with a sign to the first memory and supplying the first data with a positive sign stored in the first memory, to the second memory; a step of supplying the first data with a negative sign stored in the second memory, to the third memory; a step of generating second data by adding the first data with a positive sign stored in the second memory and the first data with a negative sign stored in the third memory; and a step of storing the second data in the fourth memory are included. When the second data stored in the fourth memory are all second data with a positive sign or all second data with a negative sign, all the second data stored in the fourth memory are added.

An apparatus and method are provided for processing floating point values using an intermediate representation which has significand, exponent and shadow sections. A less significant portion of the exponent of the floating point value defines a range of positions within the significand section where the representation of the significand is to be held. The exponent section holds a representation of a more significant portion of the exponent indicating a selected window of multiple contiguous windows spanning a value range of a format of the floating point value. A first portion of the significand section corresponds to the selected window and a second portion corresponds to an overlap into a further window which is adjacent to and lower in the value range. The shadow section holds values for populating the second portion of the significand section when the representation of the significand of the floating point value is moved to a higher window which is adjacent to and higher in the value range than the selected window. The shadow section allows the selected window to be shifted, such that the summation of multiple values produces the same result independent of the order in which the values are summed.

A processing device comprises a register configured to store a count value indicating a number of times overflow events have resulted from arithmetic operations performed by the processing device. An execution unit of the device, in response to performing an arithmetic operation having a result which extends beyond one of the predefined limit values for the floating-point format, stores a result value that is within the predefined limit values, and cause the count value to be incremented. The count value provides a performant way of determining the number of overflow events that have occurred during the arithmetic processing performed by the execution unit. The count value provides a metric that provides a measure of the inaccuracy imparted into the results of the application processing by overflow events.

Execution of a machine instruction in a central processing unit. A perform floating-point operation instruction and a test bit are obtained. If the test bit has a first value, a specified floating-point operation function is performed, and a condition code is set to a value determined by the specified function. If the test bit has a second value, a check is made to determine if the specified function is valid and installed on the machine. If the specified function is valid and installed on the machine, the condition code is set to one code value, and if the specified function is either not valid or not installed on the machine, the condition code is set to a second code value.

Execution of a machine instruction in a central processing unit. A perform floating-point operation instruction and a test bit are obtained. If the test bit has a first value, a specified floating-point operation function is performed, and a condition code is set to a value determined by the specified function. If the test bit has a second value, a check is made to determine if the specified function is valid and installed on the machine. If the specified function is valid and installed on the machine, the condition code is set to one code value, and if the specified function is either not valid or not installed on the machine, the condition code is set to a second code value.

An image processing device including a calculation processing device is provided. The calculation processing device includes: a preprocessor for extracting an effective divisor, an effective dividend, and an overflow bit for correcting a calculation result from a divisor and a dividend, based on a maximum bit number of the calculation result; a calculator for outputting a division value as a result obtained by performing a comparison calculation operation by a number of times, which is determined according to the maximum bit number, based on the effective divisor and the effective dividend; and a postprocessor for correcting the division value, based on the overflow bit.

Embodiments of the present disclosure include a tininess prediction and handler engine for handling numeric underflow while streamlining the data path for handling normal range cases, thereby avoiding flushes, and reducing the complexity of a scheduler with respect to how dependent operations are handled. A preemptive tiny detection logic section can detect a potential tiny result for the function or operation that is being performed, and can produce a pessimistic tiny indicator. The tininess prediction and handler engine can further include a subnormal post-processing pipe, which can denormalize and round one or more subnormal operations while in a post-processing mode. A schedule modification logic section can reschedule in-flight operations. The schedule modification logic section can issue dependent operations optimistically assuming that a producing operation will not produce a tiny result, and so will not incur extra latency associated with fixing the tiny result in the post-processing pipe.

Processing within a computing environment is facilitated. An operand of an instruction is obtained, which includes decimal floating point data encoded in a compressed format. An operation is performed on the operand absent decompressing a source value of a trailing significand of the decimal floating point data in the compressed format.