HARDWARE MODULE FOR CONVERTING NUMBERS

Abstract

A hardware module comprising circuity configured to: store a sequence of n bits in a register of the hardware module; generate a signed integer comprising a magnitude component and a sign bit by: if the most significant bit of the sequence of n bits is equal to one: set each of the n1 of the most significant bits of the magnitude component to be equal to the corresponding bit of the n1 least significant bits of the sequence of n bits; and set the sign bit to be zero; if the most significant bit of the sequence of n bits is equal to zero: set each of the n1 of the most significant bits of the magnitude component to be equal to the inverse of the corresponding bit of the n1 least significant bits of the sequence of n bits; and set the sign bit to be one.

Claims

1. A hardware module comprising circuity configured to: store a sequence of n bits in a register of the hardware module; and generate a signed integer comprising a magnitude component and a sign bit by: detecting a binary state of the most significant bit of the sequence of n bits; setting the sign bit as the inverse the of most significant bit of the sequence of n bits; and setting each of the n1 of the most significant bits of the magnitude component to an output binary state in dependence on the detected binary state of the most significant bit of the sequence of n bits wherein responsive to detecting that the binary state of the most significant bit of the sequence of n bits is one, the output binary state equals the corresponding bit of the n1 least significant bits of the sequence of n bits.

2. (canceled)

3. A hardware module as claimed in claim 1, further comprising a NOT gate configured to receive the most significant bit of the sequence of n bits and to output the sign bit.

4. A hardware module as claimed in claim 1, further comprising a plurality of XNOR gates, wherein each XNOR gate is configured to: receive as an input, the most significant bit of the sequence of n bits; receive as an input, one of the n1 least significant bits of the sequence of n bits; and provide as an output, the corresponding bit of the n1 most significant bits of the magnitude component.

5. A hardware module as claimed in claim 1, further comprising a random number generator configured to provide the sequence of n bits as a sequence of randomly generated bits.

6. A hardware module as claimed in claim 1, further configured to generate a floating point number from the signed integer.

7. A hardware module as claimed in claim 6, further configured to generate a mantissa of the floating point number by: setting the sign bit of the signed integer equal to zero to produce an unsigned bit sequence; performing a left shift on the unsigned bit sequence by an amount equal to the number of leading zeros of the unsigned bit sequence; and truncating the shifted unsigned bit sequence to a number of bits specified for the mantissa.

8. A hardware module as claimed in claim 7, further configured to set an exponent of the floating point number in dependence upon the number of leading zeros.

9. A hardware module as claimed in claim 6, further configured to set an exponent of the floating number equal to 1.

10. A hardware module as claimed in claim 1, further configured to generate a plurality of signed integers in a distribution centered on zero.

11. A hardware module as claimed in claim 10, further configured to generate a plurality of floating point numbers in a distribution centered on zero.

12. A hardware module as claimed in claim 1, wherein the circuity comprises at least one item selected from a list consisting of: an application specific integrated circuit and a field programmable gate array.

13. A hardware module as claimed in claim 1, further configured to calculate the magnitude component by: multiplying the sequence of bits by two to generate a first intermediate; adding one to the first intermediate to generate a second intermediate; and subtracting two to the power of n from the second intermediate.

14. A method for generating a signed integer comprising a magnitude component and a sign bit, the method comprising: storing a sequence of n bits in a register of the hardware module; and generating the signed integer by: detecting a binary state of the most significant bit of the sequence of n bits; setting the sign bit as the inverse the of most significant bit of the sequence of n bits; and setting each of the n1 of the most significant bits of the magnitude component to an output binary state in dependence on the detected binary state of the most significant bit of the sequence of n bits wherein responsive to detecting that the most significant bit of the sequence of n bits is equal to zero, the output binary state equals the inverse of the corresponding bit of the n1 least significant bits of the sequence of n bits.

15. (canceled)

16. A method according to claim 14, wherein the method comprises providing the sequence of n bits as a sequence of randomly generated bits.

17. A method according to claim 14, further comprising generating a floating point number from the signed integer.

18. A method according to claim 14, further comprising generating a mantissa of a floating point number from the signed integer by: setting the sign bit of the signed integer equal to zero to produce an unsigned bit sequence; performing a left shift on the unsigned bit sequence by an amount equal to the number of leading zeros of the unsigned bit sequence; and truncating the shifted unsigned bit sequence to a number of bits specified for the mantissa.

19. A method according to claim 14, further comprising calculating the magnitude component by: multiplying the sequence of bits by two to generate a first intermediate; adding one to the first intermediate to generate a second intermediate; and subtracting two to the power of n from the second intermediate.

20. A non-transitory computer readable medium comprising program instructions for causing a hardware module to carry out a method, the method comprising: storing a first sequence of n bits on a register of the hardware module; generating a first signed integer having a first magnitude component and a first sign bit by: setting each of the n1 of the most significant bits of the first magnitude component to be equal to a corresponding bit of the n1 least significant bits of the first sequence of n bits; and setting the first sign bit to be equal to zero; and storing a second sequence of n bits on the register of the hardware module; and generating a second signed integer having a second magnitude component and a second sign bit by: setting each of the n1 of the most significant bits of the second magnitude component to be equal to an inverse of a corresponding bit of the n1 least significant bits of the second sequence of n bits; and setting the second sign bit to be equal to one.

21. The non-transitory computer readable medium of claim 20, wherein the most significant bit of the first sequence of n bits is one.

22. The non-transitory computer readable medium of claim 20, wherein the most significant bit of the second sequence of n bits is zero.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG. 1 is a highly simplified schematic view of a neural net;

[0035] FIG. 1A is a highly simplified schematic view of a neuron;

[0036] FIG. 2 shows an example non-symmetric distribution of values;

[0037] FIG. 3 shows an example symmetric distribution of values;

[0038] FIG. 4 shows an example of circuitry converting a value to floating point format;

[0039] FIG. 5 shows an example of the circuity for converting a value to floating point format;

[0040] FIG. 6 shows an example of a scaled distribution;

[0041] FIG. 7 is a schematic diagram of a processing unit in accordance with an embodiment of the invention; and

[0042] FIG. 8 is a schematic diagram illustrating the process of conversion from a random bit sequence to a floating point number.

DETAILED DESCRIPTION

[0043] A pseudo random number generator (PRNG) generates uniformly distributed random bits. Calculations often need random floating point numbers in the range of 0 to 1 to represent probabilities. A su-from-ui circuit described herein provides conversion from a random unsigned integer to a signed integer. The signed integers are then suitable for conversion by the circuit to floating point numbers in the range 0.5 to 0.5.

[0044] Overall an n bit floating point number may be converted from an unsigned integer using the formula:

float=((2*uint)+12.sup.n)/2.sup.(n+1)

where n=the number of bits representing the mantissa of the floating point number. The largest representable magnitude with this scheme is 0.5(2.sup.(n+1)). Embodiments of the application provide a hardware implementation of symmetric and uniform random floats.

[0045] A challenge is to convert bit sequences (which may be taken to represent unsigned integers) to a distribution of signed integers centred on zero. Such signed integers are then suitable for conversion to the floating point format to produce a distribution of floating point numbers centred on zero. One proposal to produce a signed integer is to make use of subtraction (or addition) functions. However, these are expensive. Some embodiments also provide, therefore, that subtraction is achieved by novel bit manipulation.

[0046] Embodiments of the application relate to a hardware module for converting a sequence of bits into a signed integer. The hardware module is further configured to convert the signed integer to a floating point number. Each bit sequence may represent a randomly generated value. The conversion process is such that a distribution of numbers that each undergo the conversion process will form a new distribution of numbers that are centred at zero.

[0047] In order to produce a floating point number, the hardware module is configured to first convert a randomly generated sequence of bits (which represents an unsigned integer) to a signed integer. The signed integer is calculated such that a distribution of signed integers that are calculated in this way is centred on zero. The signed integer is then converted to a floating point number.

[0048] The hardware module comprises one or more of: at least one application specific integrated circuit (ASIC) and at least one field programmable gate array (FPGA) or circuits/gates. The hardware module comprises a circuit, referred to as the symmetric uniform from unsigned integer circuit, configured to calculate a floating point number from an unsigned integer according to embodiments of the application. The symmetric uniform from unsigned integer circuit comprises circuitry for converting a bit sequence representing an unsigned integer to a signed integer. The symmetric uniform from unsigned integer circuit also comprises circuitry for converting a bit sequence representing a signed integer to a floating point number.

[0049] These hardware elements (i.e. the FPGA and/or ASIC or circuits/gates) are configured to receive a sequence of n bits and manipulate the bits in such a way as to shift them to an appropriate position in the new distribution. The sequence of bits may represent an unsigned integer which is converted to a floating point format. A number in the floating point format comprises a sign bit, a mantissa, and an exponent. No limitation as to the number of bits used to represent the number is intended by the use of the term floating point format. In some embodiments, the number may be represented in single-precision floating point format and, therefore, consists of 32 bits. In other embodiments, the number may be represented in half-precision floating point format and, therefore, consists of 16 bits.

[0050] The hardware elements (i.e. the FPGA and/or ASIC or circuits/gates) are configured to determine the n1 most significant bits (MSBs) of the magnitude component of the signed integer in dependence upon the n1 least significant bits (LSBs) of the sequence of n bits provided to the hardware module.

[0051] In the case that the MSB of the sequence of n bits is equal to one, the hardware elements are configured to set each of the n1 MSBs of the magnitude component of the signed integer equal to their corresponding bit of the n1 LSBs of the sequence of n bits. In this case, the sign bit of the signed integer is set equal to one.

[0052] In the case, that the MSB of the sequence of n bits is equal to zero, the hardware elements are configured to set each of the n1 MSBs of the magnitude component of the signed integer equal to the inverse of their corresponding bit of the n1 LSBs of the sequence of n bits. In this case, the sign bit of the signed integer is set equal to zero.

[0053] In either case, the LSB of the signed integer is set equal to one.

[0054] The sequence of n bits that are received at the hardware module may be randomly generated by a random number generator. The randomly generated numbers received at the hardware module may form a uniform distribution of random numbers or may form an approximate Gaussian distribution. By modifying the randomly generated numbers in manner described above, a uniform, Gaussian, or some other form of distribution that is centred at zero may be formed. Techniques requiring a distribution of numbers centred on zero have applications in neural networks. Techniques have recently been devised for improving the performance of neural networks by adding random noise to weights or activations. Gaussian noise has been explored as a possibility in this respect. Techniques described herein for generating random numbers can be used to generate Gaussian noise as described in our earlier U.S. application Ser. No. 15/886,505, the contents of which are herein incorporated by reference. To provide the required noise, the Gaussian distribution in this example, is normalised and centred on zero.

[0055] Reference is made to FIG. 7 which illustrates an example of the use of such a hardware module 24 for generating the floating point numbers as described. FIG. 7 shows a schematic block diagram of an execution unit 2 arranged to execute a single instruction for generating a floating point number from an unsigned integer.

[0056] The execution unit 2 forms part of a pipeline 4 in a processing unit. The processing unit comprises an instruction fetch unit 6 which fetches instruction from an instruction memory 10. The processing unit also comprises a memory access stage 8 which is responsible for accessing a data memory 12 for loading data from the memory or storing data into the memory. A set of registers 14 is provided for holding source and destination operands for the instructions being executed at any instance by the pipeline 4. It will readily be understood that the pipeline 4 may contain many different types of execution unit for executing a variety of different instructions, for example for performing mathematical operations. One type of processing unit which may be useful with the present invention is a processing unit using barrel-threaded time slots, in which a supervisor thread may allocate different worker threads to different time slots for their execution.

[0057] The execution unit 2 comprises an instruction execution module 23 and an output buffer 27 for holding the result of execution of the instruction. The instruction execution module 23 is configured to execute an instruction to cause the hardware module 24 to generate a floating point number. The hardware module 24 is configured in response to such an instruction to convert a sequence of n bits received from a pseudo random number generator 22 and to provide the floating point number to the output buffer 27.

[0058] Through the execution of multiple instructions, multiple floating point numbers may be generated in a distribution centred at zero.

[0059] The sequence of n bits that are received at the hardware module may be understood to represent unsigned integers. These unsigned integers form an initial distribution. Reference is made to FIG. 2, which illustrates an initial distribution 210 of unsigned integers that may be received at the hardware module. The initial distribution 210 is a uniform distribution, reflecting the uniform distribution of numbers output by the random number generator.

[0060] In this example, n=4 (i.e. each unsigned integer is represented by 4 bits). Therefore, the maximum value that can be represented is 15, which is given by the bit sequence: 1111. The minimum value that can be represented is 0, which is given by the bit sequence: 0000. This is reflected in the distribution 210 which extends between 0 and 15.

[0061] Reference is made to FIG. 3, which illustrates a distribution 310 of floating point numbers that is centred at zero. The distribution 310 represents the result of the sequence of n bits received at the hardware module as described. The range of the distribution, as for the initial distribution is 15. However, now the distribution extends between 7.5 and 7.5.

[0062] Reference is made to FIG. 4, which illustrates circuity of the hardware module used for generating a signed integer from an unsigned integer. The signed integers are generated by this circuit in such a way that a resulting distribution is centred at zero.

[0063] In the example shown, the sequence of n bits 405 comprises the bit sequence 0101. FIG. 4 shows at 410 that this represents an unsigned integer of value 5. The magnitude 415 of the signed integer also comprises 4 bits. The signed integer additionally includes a sign bit 420.

[0064] To determine the value of the sign bit 420, the MSB of the sequence 405 of n bits is used an input to a NOT gate 425. The output is the sign bit 420. This has the effect of making the signed integer negative if the unsigned integer is less than half the maximum that is representable by the sequence of n bits. In this example, the sign bit 420 is set equal to 1 (representing a negative number), because the unsigned integer has a value (5) that is less than half (7.5) the maximum (15) that is representable by 4 bits.

[0065] FIG. 4 also shows the circuitry used to construct the n1 MSBs 430 of the magnitude of the signed integer. These are calculated in dependence upon the n1 LSBs 435 of the sequence of n bits.

[0066] The hardware elements comprise a plurality of XNOR (exclusive NOR) gates 440. There may be n1 XNOR gates 440 provided. Each XNOR gate 440 is also configured to receive as an input one of the n1 LSBs of the sequence of n bits and to provide its output to the corresponding bit of the n1 MSBs 430 of the magnitude of the signed integer. Therefore, a first XNOR gate 440a is configured to receive a MSB of the n1 LSBs 435 of the sequence of n bits and to output a MSB of the magnitude of the signed integer. A second XNOR gate 440b is configured to receive a 2.sup.nd MSB of the n1 LSBs 435 of the sequence of n bits and to output a 2.sup.nd MSB of the magnitude of the signed integer. A third XNOR gate 440c is configured to receive a 3.sup.rd MSB of the n1 LSBs 435 of the sequence of n bits and to output a 3.sup.rd MSB of the magnitude of the signed integer.

[0067] Each of the XNOR gates 440 is also configured to receive as an input, the MSB 450 of the sequence of n bits. By doing so, the XNOR gates 440 determine whether or not, the inputs of the n1 LSBs 435 are to be inverted. In the example in FIG. 4, since the MSB 450 is equal to zero, each of n1 LSBs 435 are inverted to form the n1 MSBs of the magnitude of the signed integer. In other words, the inversion takes place since the value of the unsigned integer represented by the sequence of n bits is less than half the maximum possible value.

[0068] FIG. 4 also shows that the LSB 445 of the magnitude of the signed integer is set to 1. This is true irrespective of the values of sequence of n bits.

[0069] Reference is made to FIG. 5, which illustrates how the circuity for generating the signed integer is applied to a different sequence of n bits.

[0070] In the example shown, the sequence of n bits 505 comprises the bit sequence 1100. The Figure shows at 510 that this represents an unsigned integer of value 12. The magnitude of the signed integer 515 of the floating point number also comprises 4 bits. The signed integer additionally includes a sign bit 520.

[0071] To determine the value of the sign bit 520, the MSB of the sequence of n bits is used an input to a NOT gate 525. In this example, the sign bit 520 is set equal to 0 (representing a positive number), since the unsigned integer has a value (12) that is less than half (7.5) the maximum (15) that is representable by 4 bits.

[0072] FIG. 5 also shows how the circuitry is used to construct the n1 MSBs 530 of the magnitude of the signed integer. As before, each of the XNOR gates 540 receive as an input, the MSB 550 of the sequence of n bits. In the example in FIG. 4, since the MSB 550 is equal to one, each of the n1 MSBs 530 of the magnitude of the signed integer are set equal to the corresponding bit in the n1 LSBs 535 of the sequence of n bits. In other words, no inversion takes place since the value of the unsigned integer represented by the sequence of n bits is greater than half the maximum possible value.

[0073] Again, FIG. 5 also shows that the LSB 545 of the magnitude of the signed integer is set to 1. This is true irrespective of the values of sequence of n bits.

[0074] Therefore, by manipulating the bits as described to generate suitable signed integers, the hardware module is capable of generating a distribution centred on zero.

[0075] The signed integers are then suitable for conversion to floating point numbers to form a distribution of floating point numbers centred on zero. The hardware module comprises a floating point processing unit for converting the signed integers to floating point numbers.

[0076] Reference is made to FIG. 8, which illustrates the overall process carried out by the hardware module for converting the unsigned integers to floating point numbers.

[0077] Initially, a randomly generated bit sequence 810 is provided to the hardware module by a PRNG. The bit sequence 810 is converted to a bit sequence 820 representing a signed integer in accordance with the techniques described above with respect to FIGS. 5 and 6.

[0078] The sign bit of the signed integer 820 is set equal to zero to produce the bit sequence 830. The hardware module is then configured to count the number of leading zeros and perform a left shift on the bit sequence 830 until all of the leading zeros are discarded and the MSB is equal to 1. This produces the bit sequence 840.

[0079] Since the mantissa of a floating point number typically has fewer bits compared to an input integer. For example, when converting a 32 bit integer to a single precision float, the mantissa has 24 bits. Therefore, 8 bits will be truncated from the 32 bit integer to form the 24 bit mantissa.

[0080] The bits removed from the bit sequence 830 are used to round the LSB of the mantissa 840. Any suitable rounding scheme may be applied to achieve this. For example, the rounding scheme round to nearest, even (RTNE) scheme may applied. With this scheme, the LSB of the mantissa is rounded up if the bits that are removed are greater than half the maximum value of the LSB and rounded down if the bits that are removed are less than half the maximum value of the LSB. In the event that the bits removed are equal to half the maximum value of the LSB, the LSB is rounded to the nearest even, i.e. the LSB is rounded up if equal to 1 and rounded down if equal to 0.

[0081] The exponent and the sign bit of the floating point number 860 are also calculated by the hardware module. The exponent is determined in dependence upon the number of leading zeros (and therefore the amount of left shift that was performed) of the bit sequence 830. The exponent may also be determined in dependent upon any rounding carry that may result from the rounding process of the mantissa 850. Such a rounding carry will result in the case that the pre-rounding mantissa consists of all ones, and therefore the rounding is propagated along the mantissa to the MSB of the mantissa.

[0082] The exponent is calculated using the following formula:

exponent=expBias+expMaxleadingZeroCount+roundingCarry

[0083] expBias is equal to the exponent bias (i.e. 127 in single-precision format and 15 in the half-precision format). expMax is an offset to provide the maximum possible exponent attainable based on the length of the bit sequence 810. If the bit sequence 810 consists of 32 bits, then expMax is equal to 31. leadingZerocount is the number of leading zeros of bit sequence 830. roundingCarry is the rounding carry that may result if the mantissa 850 consists of all ones.

[0084] In other examples, the exponent is set equal to 1, so as to obtain a uniform distribution with a range equal to 1. If the exponent is set equal to 1, this would lead to a range of 0.5 to 0.5 for the distribution. FIG. 6 illustrates such a distribution.

[0085] The sign bit of the floating point number 860 is set equal to the sign bit of the signed integer.

[0086] Hence, embodiments of the application provide an apparatus and method for producing signed integers in a distribution centred on zero. The signed integers are suitable for conversion to floating point numbers to obtain a distribution of floating point numbers centred on zero.