A processor comprising a register file comprising a bias register for holding a bias and a plurality of operand registers each for holding a respective number which together with the bias represents a respective value in a logarithmic number system; and an execution unit configured to, in response to receiving a logarithmic addition opcode: retrieve first and second numbers from first and second sources respectively; subtract the first number from the second number to determine a difference; and if the determined difference is less than or equal to a predetermined number, retrieve, from a look-up table, a third number mapped to the determined difference, and add the third number to the first number to determine a result; if the determined difference is greater than the predetermined number, determine the result to be the greatest of the first and second numbers; and store the result.

Methods of nonlinear differentiation and nonlinear differentiators are described. A log-sign nonlinear differentiator and an adaptive gain log-sign differentiator for signal tracking in a digital signal processor receive an input signal, u(t), estimates a filtered first state, x.sub.1(t) of the input signal, estimates second state signal, x.sub.2(t), and receive parameters which cause the filtered first state, x.sub.1(t), to converge asymptotically to the input signal, u(t), and the second state signal, x.sub.2(t), to converge asymptotically to the first derivative {dot over (u)}(t) of the input signal, u(t), such that a first output, y.sub.1(t), of the log-sign nonlinear differentiator, is an estimate of the input signal, u(t), and a second output, y.sub.2(t) equals the first derivative, {dot over (u)}(t) of the input signal, u(t), tracked by the log-sign nonlinear differentiator. The adaptive log-sign differentiator includes a signal path which includes calculating a deadzone function at the input of the first differentiator.

A processor or other device, such as a programmable and/or massively parallel processor or other device, includes processing elements designed to perform arithmetic operations (possibly but not necessarily including, for example, one or more of addition, multiplication, subtraction, and division) on numerical values of low precision but high dynamic range (“LPHDR arithmetic”). Such a processor or other device may, for example, be implemented on a single chip. Whether or not implemented on a single chip, the number of LPHDR arithmetic elements in the processor or other device in certain embodiments of the present invention significantly exceeds (e.g., by at least 20 more than three times) the number of arithmetic elements, if any, in the processor or other device which are designed to perform high dynamic range arithmetic of traditional precision (such as 32 bit or 64 bit floating point arithmetic).

The present invention provides a logarithm calculation method, wherein the logarithm calculation method includes the steps of: (a) selecting a first parameter, a second parameter, a third parameter and a fourth parameter corresponding to an i-th iteration operation; (b) determining whether an input value is greater than the third parameter or smaller than the fourth parameter (c) if the input value is greater than the third parameter, updating the input value by multiplying the first parameter, and updating an output value by subtracting a logarithmic value of the first parameter; if the input value is less than the fourth parameter, updating the input value by multiplying the second parameter, and updating the output value by subtracting a logarithmic value of the second parameter (d) adding one to ‘i’ and return to step (a); (e) when ‘i’ is equal to a predetermined value, outputting the current output value.

An arithmetic logic unit, comprising an addition unit for determining a sum of a first input and a second input; and a logarithmic addition unit for determining an output using the sum and a third input. The output is a multiply-accumulate output represented in a logarithmic domain when the first, second and third inputs are represented in the logarithmic domain.

A method of training a neural network including multiple neural network weights and multiple neurons, and the method includes using floating-point signed digit numbers to represent each of the multiple neural network weights, wherein a mantissa of each of the multiple neural network weights is represented by multiple mantissa signed digit groups and an exponent of each of the multiple neural network weights is represented by an exponent digit group; and using the exponent digit group and at least one of the multiple mantissa signed digit groups to perform weight adjustment computation and neural network inference computation.

Low precision computers can be efficient at finding possible answers to search problems. However, sometimes the task demands finding better answers than a single low precision search. A computer system augments low precision computing with a small amount of high precision computing, to improve search quality with little additional computing.

A processor or other device, such as a programmable and/or massively parallel processor or other device, includes processing elements designed to perform arithmetic operations (possibly but not necessarily including, for example, one or more of addition, multiplication, subtraction, and division) on numerical values of low precision but high dynamic range (“LPHDR arithmetic”). Such a processor or other device may, for example, be implemented on a single chip. Whether or not implemented on a single chip, the number of LPHDR arithmetic elements in the processor or other device in certain embodiments of the present invention significantly exceeds (e.g., by at least 20 more than three times) the number of arithmetic elements, if any, in the processor or other device which are designed to perform high dynamic range arithmetic of traditional precision (such as 32 bit or 64 bit floating point arithmetic).

A processing unit for multiplying a first value by a first multiplicand, or for multiplying the first value by, in each instance, a second and third multiplicand. The processing unit receives the multiplicands in a logarithmic number format, so that the multiplicands are each present in the form of at least one exponent at a specifiable base. The processing unit includes a first register, in which either two exponents of the first multiplicand or the exponent of the second and the exponent of the third multiplicand are stored. A set configuration bit indicates whether either the two exponents of the first multiplicand or the exponent of the second and the exponent of the third multiplicand are stored in the first register. The processing unit includes at least two bitshift operators. A method and a computer program for multiplying the value by the multiplicand are also described.

An end-to-end low-precision training system based on a multi-base logarithmic number system and a multiplicative weight update algorithm. The multi-base logarithmic number system is applied to update weights of the neural network, with different bases of the multi-base logarithmic number system utilized between calculation of weight updates, calculation of feed-forward signals, and calculation of feedback signals. The LNS expresses a high dynamic range and computational energy efficiency, making it advantageous for on-board training in energy-constrained edge devices.