A method and computer system are provided for performing a comparison computation, e.g. for use in a check procedure for a reciprocal square root operation. The comparison computation compares a multiplication of three values with a predetermined value. The computer system performs the multiplication using multiplier logic which is configured to perform multiply operations in which two values are multiplied together. A first and second of the three values are multiplied to determine a first intermediate result, w.sub.1. The digits of w.sub.1 are separated into two portions, w.sub.1,1 and w.sub.1,2. The third of the three values is multiplied with w.sub.1,2 and the result is added into a multiplication of the third of the three values with w.sub.1,1 to thereby determine the result of multiplying the three values together. In this way the comparison is performed with high accuracy, while keeping the area and power consumption of the multiplier logic low.

A method and computer system are provided for performing a comparison computation, e.g. for use in a check procedure for a reciprocal square root operation. The comparison computation compares a multiplication of three values with a predetermined value. The computer system performs the multiplication using multiplier logic which is configured to perform multiply operations in which two values are multiplied together. A first and second of the three values are multiplied to determine a first intermediate result, w.sub.1. The digits of w.sub.1 are separated into two portions, w.sub.1,1 and w.sub.1,2. The third of the three values is multiplied with w.sub.1,2 and the result is added into a multiplication of the third of the three values with w.sub.1,1 to thereby determine the result of multiplying the three values together. In this way the comparison is performed with high accuracy, whilst keeping the area and power consumption of the multiplier logic low.

A method and computer system are provided for performing a comparison computation, e.g. for use in a check procedure for a reciprocal square root operation. The comparison computation compares a multiplication of three values with a predetermined value. The computer system performs the multiplication using multiplier logic which is configured to perform multiply operations in which two values are multiplied together. A first and second of the three values are multiplied to determine a first intermediate result, w.sub.1. The digits of w.sub.1 are separated into two portions, w.sub.1,1 and w.sub.1,2. The third of the three values is multiplied with w.sub.1,2 and the result is added into a multiplication of the third of the three values with w.sub.1,1 to thereby determine the result of multiplying the three values together. In this way the comparison is performed with high accuracy, while keeping the area and power consumption of the multiplier logic low.

Methods and systems for determining whether an infinitely precise result of a reciprocal square root operation performed on an input floating point number is greater than a particular number in a first floating point precision. The method includes calculating the square of the particular number in a second lower floating point precision; calculating an error in the calculated square due to the second floating point precision; calculating a first delta value in the first floating point precision by calculating the square multiplied by the input floating point number less one; calculating a second delta value by calculating the error multiplied by the input floating point number plus the first delta value; and outputting an indication of whether the infinitely precise result of the reciprocal square root operation is greater than the particular number based on the second delta term.

A function estimation hardware logic unit may be implemented as part of an execution pipeline in a processor. The function estimation hardware logic unit is arranged to calculate, in hardware logic, an improved estimate of a function of an input value, d, where the function is given by $1/\sqrt[i]{d}.$
The hardware logic comprises a plurality of multipliers and adders arranged to implement a m.sup.th-order polynomial with coefficients that are rational numbers, where m is not equal to two and in various examples m is not equal to a power of two. In various examples i=1, i=2 or i=3. In various examples m=3.

A function estimation hardware logic unit may be implemented as part of an execution pipeline in a processor. The function estimation hardware logic unit is arranged to calculate, in hardware logic, an improved estimate of a function of an input value, d, where the function is given by

[00001]$1/\sqrt[i]{d}.$

The hardware logic comprises a plurality of multipliers and adders arranged to implement a m.sup.th-order polynomial with coefficients that are rational numbers, where m is not equal to two and in various examples m is not equal to a power of two. In various examples i=1, i=2 or i=3. In various examples m=3.

Techniques for performing square root or reciprocal square root calculations on BF16 data elements in response to an instruction are described. An example of an instruction is one that includes fields for an opcode, an identification of a location of a packed data source operand, and an identification of a packed data destination operand, wherein the opcode is to indicate that execution circuitry is to perform, for each data element position of the packed data source operand, a calculation of a square root value of a BF16 data element in that position and store a result of each square root into a corresponding data element position of the packed data destination operand.