A method for compressing a first application file and second application file includes accessing the first and the second application files, the first application file being in a first language and the second application being in a second language and being a counterpart of the first application file, decompressing the first and second application files to access internal files for the first and the second application files, comparing one of the first internal files to one of the second internal files, upon determining that the first internal file is identical to the second internal file, copying one of the internal files to an output folder, and upon determining that the files are not identical, copying both of the internal files to the output folder, or executing a differencing procedure on the first and second internal files to identify differences between them, storing data about the differences in the output folder, and compressing the output folder into one output file.

A method of compressing weights of a neural network includes compressing a weight set including the weights of a the neural network, determining modified weight sets by changing at least one of the weights, calculating compression efficiency values for the determined modified weight sets based on a result of compressing the weight set and results of compressing the determined modified weight sets, determining a target weight of the weights satisfying a compression efficiency condition among the weights based on the calculated compression efficiency values, and determining a final compression result by compressing the weights based on a result of replacing the determined target weight.

Aspects of the invention include identifying a first subsystem and a second subsystem of a plurality of subsystems respectively storing a first compressed data and a second compressed data, wherein the first compressed data and the second compressed data are fragments of a requested data. A compression method used to compress the first compressed data and second compressed data is identified. A first accelerator of first subsystem and a second accelerator of the second subsystem is identified. The first compressed data from a first local memory of the first subsystem is offloaded to the first accelerator, and the second compressed data from a second local memory of the second subsystem is offloaded to the second accelerator, wherein offloading comprises provided a decompression method for the first compressed data and the second compressed data.

A compression algorithm based on Huffman coding is disclosed that is adapted to be readily implemented using VLSI design. A data file may be processed to replace duplicate data with a copy commands including an offset and length, such as according to the LV algorithm. A Huffman code may then be generated for parts of the file. The Huffman code may be generated according to a novel method that generates Huffman code lengths for literals in a data file without first sorting the literal statistics. The Huffman code lengths may be constrained to be no longer than a maximum length and the Huffman code may be modified to provide an acceptable overflow probability and be in canonical order. Literals, offsets, and lengths may be separately encoded. The different values for these data sets may be assigned to a limited number of bins for purpose of generating usage statistics used for generating Huffman codes.

The entropy coding of a current part of a predetermined entropy slice is based on, not only, the respective probability estimations of the predetermined entropy slice as adapted using the previously coded part of the predetermined entropy slice, but also probability estimations as used in the entropy coding of a spatially neighboring, in entropy slice order preceding entropy slice at a neighboring part thereof. Thereby, the probability estimations used in entropy coding are adapted to the actual symbol statistics more closely, thereby lowering the coding efficiency decrease normally caused by lower-delay concepts. Temporal interrelationships are exploited additionally or alternatively.

In one embodiment, an apparatus comprises a decompression engine to perform a non-speculative decode operation on a first portion of a first compressed payload comprising a first plurality of codes; and perform a speculative decode operation on a second portion of the first compressed payload, wherein the non-speculative decode operation and the speculative decode operation share at least one decode path and the non-speculative decode operation is to utilize bandwidth of the at least one decode path that is not used by the non-speculative decode operation.

A method of compressing weights of a neural network includes compressing a weight set including the weights of a the neural network, determining modified weight sets by changing at least one of the weights, calculating compression efficiency values for the determined modified weight sets based on a result of compressing the weight set and results of compressing the determined modified weight sets, determining a target weight of the weights satisfying a compression efficiency condition among the weights based on the calculated compression efficiency values, and determining a final compression result by compressing the weights based on a result of replacing the determined target weight.

Methods and apparatus to parallelize data decompression are disclosed. An example method selecting initial starting positions in a compressed data bitstream; adjusting a first one of the initial starting positions to determine a first adjusted starting position by decoding the bitstream starting at a training position in the bitstream, the decoding including traversing the bitstream from the training position as though first data located at the training position is a valid token; outputting first decoded data generated by decoding a first segment of the bitstream starting from the first adjusted starting position; and merging the first decoded data with second decoded data generated by decoding a second segment of the bitstream, the decoding of the second segment starting from a second position in the bitstream and being performed in parallel with the decoding of the first segment, and the second segment preceding the first segment in the bitstream.

An apparatus including a Huffman decoder circuit is described. In a first embodiment, the Huffman decoder circuit includes a register file with simultaneous parallel load capability. The register file is to keep multiple copies of same decoded values in different entries of the register file. The different entries are to be addressed by respective addresses having a same leading edge encoded symbol. The parallel load capability is to simultaneously load a same decoded value for those register file addresses having a same leading edge encoded symbol. In a second embodiment, the Huffman decoder circuit includes a CAM circuit coupled to a register file, wherein respective match lines of the CAM circuit are coupled to respective entries of the register file. The CAM circuit is to keep encoded symbols. The register file is to keep decoded values of the encoded symbols.

To decompress encoded data, a Huffman code tree stored in a data header may need to be decompressed and rebuilt. A bit length histogram table is used in a hardware design to more efficiently decompress the Huffman code tree. The bit length histogram table relates each bit length used by the Canonical Huffman Code (CHC) symbols to a corresponding number of symbols in the encoding that have that bit length. Performing decompression using bit length histogram table allows part of the Huffman tree decompression to be performed in a single pass.