Apparatus and methods for training a neural network accelerator using quantized precision data formats are disclosed, and in particular for storing activation values from a neural network in a compressed format having lossy or non-uniform mantissas for use during forward and backward propagation training of the neural network. In certain examples of the disclosed technology, a computing system includes processors, memory, and a compressor in communication with the memory. The computing system is configured to perform forward propagation for a layer of a neural network to produced first activation values in a first block floating-point format. In some examples, activation values generated by forward propagation are converted by the compressor to a second block floating-point format having a non-uniform and/or lossy mantissa. The compressed activation values are stored in the memory, where they can be retrieved for use during back propagation.

Apparatus and methods for training a neural network accelerator using quantized precision data formats are disclosed, and in particular for storing activation values from a neural network in a compressed format having lossy or non-uniform mantissas for use during forward and backward propagation training of the neural network. In certain examples of the disclosed technology, a computing system includes processors, memory, and a compressor in communication with the memory. The computing system is configured to perform forward propagation for a layer of a neural network to produced first activation values in a first block floating-point format. In some examples, activation values generated by forward propagation are converted by the compressor to a second block floating-point format having a non-uniform and/or lossy mantissa. The compressed activation values are stored in the memory, where they can be retrieved for use during back propagation.

A method includes inputting a decompressing compressed image in a computing device. The method also includes applying one or more delta images by a processor to reduce a transfer time for the inputted and decompressed patch image. The method also includes performing one or more calls to a system of memory caches for reads and writes of the decompressed patch image and additional input and output due to a shortage of space in an internal random-access memory (RAM). The method also includes locating arbitrary storage to redirect the decompressed patch image and the additional input and output. The method also includes redirecting the inputted decompressed patch image and the additional input and output to the arbitrary storage.

Described herein is a graphics processing unit (GPU) configured to receive an instruction having multiple operands, where the instruction is a single instruction multiple data (SIMD) instruction configured to use a bfloat16 (BF16) number format and the BF16 number format is a sixteen-bit floating point format having an eight-bit exponent. The GPU can process the instruction using the multiple operands, where to process the instruction includes to perform a multiply operation, perform an addition to a result of the multiply operation, and apply a rectified linear unit function to a result of the addition.

Described herein is a graphics processing unit (GPU) configured to receive an instruction having multiple operands, where the instruction is a single instruction multiple data (SIMD) instruction configured to use a bfloat16 (BF16) number format and the BF16 number format is a sixteen-bit floating point format having an eight-bit exponent. The GPU can process the instruction using the multiple operands, where to process the instruction includes to perform a multiply operation, perform an addition to a result of the multiply operation, and apply a rectified linear unit function to a result of the addition.

A method and apparatus of a device that compresses an object stored in memory is described. In an exemplary embodiment, the device receives an indication that the object is to be compressed. The device further selects one of a plurality of compression algorithms based on at least a characteristic of the object. In addition, the device compresses the object in-memory using the selected compression algorithm.

Circuits, methods, and apparatus for efficiently implementing encoding and decoding between binary and multilevel data.

To enable lossless compression, an auxiliary bitmap is used to provide side information about the graph bitmap. Each bit in the auxiliary bitmap represents a word in the graph bitmap. A zero bit in the auxiliary bitmap means that the corresponding word in the graph bitmap is not transmitted. Therefore, it is set to the default value, λ, during decompression. This default value could be either an all-zeros word, or all-ones word depending on the BFS step. A one bit in the auxiliary bitmap means that the corresponding word in the graph bitmap is transmitted.

To enable lossless compression, an auxiliary bitmap is used to provide side information about the graph bitmap. Each bit in the auxiliary bitmap represents a word in the graph bitmap. A zero bit in the auxiliary bitmap means that the corresponding word in the graph bitmap is not transmitted. Therefore, it is set to the default value, λ, during decompression. This default value could be either an all-zeros word, or all-ones word depending on the BFS step. A one bit in the auxiliary bitmap means that the corresponding word in the graph bitmap is transmitted.

Described herein is a graphics processing unit (GPU) comprising a first processing cluster to perform parallel processing operations, the parallel processing operations including a ray tracing operation and a matrix multiply operation; and a second processing cluster coupled to the first processing cluster, wherein the first processing cluster includes a floating-point unit to perform floating point operations, the floating-point unit is configured to process an instruction using a bfloat16 (BF16) format with a multiplier to multiply second and third source operands while an accumulator adds a first source operand with output from the multiplier.