Data compression is continuously optimized using frequency-based dictionary lookup tables for communication between a transmitter in a vehicle and a receiver. In a transmitter, first and second frequency-based dictionary lookup tables are provided. The transmitter receives a data block and compresses it using the first table. Using the data block, the second table is updated and a difference between a compression efficiency of the first table and a compression efficiency of the second table for the data block is calculated. The compressed data block is transmitted to the receiver. When the difference is more than a pre-defined threshold, the content of the first table is replaced with the content of the updated second table in the transmitter. The content of the updated second table is then also transmitted to the receiver.

Methods, systems, and apparatus, including computer-readable storage media for hardware compression and decompression. A system can include a decompressor device coupled to a memory device and a processor. The decompressor device can be configured to receive, from the memory device, compressed data that has been compressed using an entropy encoding, process the compressed data using the entropy encoding to generate uncompressed data, and send the uncompressed data to the processor. The system can also include a compressor device configured to generate, from uncompressed data, a probability distribution of codewords, generate a code table from the probability distribution, and compress incoming data using the generated code table.

Selecting tables for compression by threshold statistical values. Identified tables are reordered according to fields having the lowest cardinality to increase the size of character strings replaced by keys during compression. Field locations are mapped between the original table and the reordered table. Dictionary-based compression is performed on reordered tables.

A dictionary configuration method, a buffer control method, a network side device and a UE are provided. The buffer control method for the network side device includes: receiving, by the network side device, first check information transmitted by the UE; determining, by the network side device, whether content in a compression buffer of the UE is identical to content in a decompression buffer of the network side device; and when the content in the compression buffer is different from the content in the decompression buffer, transmitting, by the network side device, buffer resetting information to the UE.

A compression algorithm based on Huffman coding is disclosed that is adapted to be readily implemented using VLSI design. A data file may be processed to replace duplicate data with a copy commands including an offset and length, such as according to the LV algorithm. A Huffman code may then be generated for parts of the file. The Huffman code may be generated according to a novel method that generates Huffman code lengths for literals in a data file without first sorting the literal statistics. The Huffman code lengths may be constrained to be no longer than a maximum length and the Huffman code may be modified to provide an acceptable overflow probability and be in canonical order. Literals, offsets, and lengths may be separately encoded. The different values for these data sets may be assigned to a limited number of bins for purpose of generating usage statistics used for generating Huffman codes.

The present disclosure is directed to systems and methods for providing fast and efficient data compression using a combination of content dependent, content estimation, and content independent data compression. In one aspect of the disclosure a method for compressing data comprises the steps of: analyzing a data block of an input data stream to identify a data type of the data block, the input data stream comprising a plurality of disparate data types; performing content dependent data compression on the data block, if the data type of the data block is identified; performing content estimation data compression if the content is estimable; and performing content independent data compression on the data block, if the data type of the data block is not identified or estimable. In another aspect of the present invention LZDR compression is applied to simultaneously perform one method of compression while computing statistics useful in estimating the optimal form of compression to be applied.

A system having multiple devices that can host different versions of an artificial neural network (ANN) as well as different versions of a feature dictionary. In the system, encoded inputs for the ANN can be decoded by the feature dictionary, which allows for encoded input to be sent to a master version of the ANN over a network instead of an original version of the input which usually includes more data than the encoded input. Thus, by using the feature dictionary for training of a master ANN there can be reduction of data transmission.

Apparatus and method for dictionary accelerator compression. For example, one embodiment of an apparatus comprises: a plurality of cores; a compression/decompression accelerator coupled to or integral to one or more of the plurality of cores, the compression/decompression accelerator to perform decompression and compression operations in response to read and write operations, respectively, wherein responsive to notification of a compression job to compress a memory page or a portion thereof, a history buffer associated with the compression/decompression accelerator to is to be initialized with pre-configured dictionary data, the compression/decompression accelerator to match portions of the pre-configured dictionary data with portions of the memory page to generate compressed output data.

Described embodiments provide systems and methods for stream-based compression. An encoder of a first device receives an input stream of bytes including a first byte preceded by one or more second bytes. The encoder may determine to identify a prefix code for the first byte. The encoder may select a prefix code table using the one or more second bytes. The encoder may identify, from the selected prefix code table, the prefix code of the first byte. The encoder may generate an output stream of bytes by replacing the first byte in the input stream with the prefix code of the first byte. The encoder may transmit the output stream from the encoder of the first device to a decoder of a second device. The output stream may have a fewer number of bits than the input stream.

Methods and apparatus for compressing data streams. In an embodiment, a method includes calculating a decomposition of matrix data to generate eigenvectors and associated eigenvalues, determining clusters of the eigenvectors based on weighting the eigenvalues, calculating an eigenvector centroid for each cluster so that a dictionary of centroids is generated, and tagging the eigenvectors with tags, respectively, that identify an associated eigenvector centroid for each eigenvector. The method also includes counting a number of eigenvectors associated with each eigenvector centroid to construct a probability distribution function (PDF) of centroids, matching the PDF of centroids to PDF templates to determine a closest matching PDF template, determining an encoder corresponding to the closest matching PDF template wherein a corresponding encoder identifier is identified, encoding the tags with the encoder to generate an encoded data stream, and transmitting the encoded data stream, the encoder identifier, the dictionary of centroids, and the eigenvalues.