Aspects of dynamic data compression selection are presented. In an example method, as uncompressed data chunks of a data stream are compressed, at least one performance factor affecting selection of one of multiple compression algorithms for the uncompressed data chunks of the data stream may be determined. Each of the multiple compression algorithms may facilitate a different expected compression ratio. One of the multiple compression algorithms may be selected separately for each uncompressed data chunk of the data stream based on the at least one performance factor. Each uncompressed data chunk may be compressed using the selected one of the multiple compression algorithms for the uncompressed data chunk.

Methods, devices and systems enhance compression and decompression of data blocks of data values by selecting the best suited compression method and device among two or a plurality of compression methods and devices, which are combined together and which said compression methods and devices compress effectively data values of particular data types; said best suited compression method and device is selected using as main selection criterion the dominating data type in a data block by predicting the data types within said data block.

A computer system includes a hardware controller and an internal millicode storage area. The controller includes an accelerator that decompresses a data stream requested by an application. The internal millicode storage area can store a compression dictionary library including a plurality of different pre-defined compression dictionaries. A host system includes a dictionary manager that determines a compression dictionary from the plurality of different pre-defined compression dictionaries included in the dictionary library to decompress the data stream. The accelerator can access the internal millicode storage area to obtain the determined compression dictionary, and to decompress the data stream according to the determined compression dictionary.

The transmission of broadcast data, such as financial data and news feeds, is accelerated over a communication channel using data compression and decompression to provide secure transmission and transparent multiplication of communication bandwidth, as well as reduce the latency. Broadcast data may include packets having fields. Encoders associated with particular fields may be selected to compress those particular fields.

A processing device is provided which includes a plurality of encoders each configured to compress a portion of data using a different compression algorithm. The processing device also includes one or more processors configured to cause an encoder, of the plurality of encoders, to compress the portion of data when it is determined that the portion of data, which is compressed by another encoder configured to compress the portion of data prior to the encoder in an encoder hierarchy, is not successfully compressed according to a compression metric by the other encoder in the encoder hierarchy. The one or more processors are also configured to prevent the encoder from compressing the portion of data when it is determined that the portion of data is successfully compressed according to the compression metric by the other encoder in the encoder hierarchy.

Data compression (and corresponding decompression) is used to compress blocks of data values involving processes including one or more of color decorrelation, spatial decorrelation, entropy encoding and packing. The entropy encoding generates encoded data values which have variable sizes (in terms of the number of bits). The entropy encoding uses size indications for respective sets of data values to indicate the number of bits used for the encoded data values of the set. The size indications allow the encoded data values to be parsed quickly (e.g. in parallel).

A device that includes a node engine configured to emulate a first node, a second node, and a third node. The first node is configured to receive a first correlithm object, fetch a second correlithm object based on the first correlithm object, and output the second correlithm object to the second node and the third node. Each correlithm object is a point in an n-dimensional space represented by a binary string. The second node is configured to receive the second correlithm object, fetch a third correlithm object based on the second correlithm object, and output the third correlithm object to the third node. The third node is configured to receive the second correlithm object, receive the third correlithm object, fetch a fourth correlithm object based on the second correlithm object and the third correlithm object, and output the fourth correlithm object.

An encoder for encoding input data to generate corresponding encoded data is provided. The encoder (10) is operable to process a sequence of elements in the input data from a first element thereof to a last element thereof. The elements have corresponding symbols. The encoder is operable to compute probabilities of symbols present in the sequence, wherein the probabilities of the symbols are computed while disregarding those elements of the sequence that have already been encoded into the encoded data. Moreover, the probabilities of the symbols are adaptively changed as the sequence of elements is progressively encoded into the encoded data. Furthermore, information describing the probabilities is delivered; optionally, the probabilities of the symbols are accompanied with additional information indicating how the probabilities are adaptively changing. There is also provided a decoder for performing an inverse of encoding performed by the encoder.

Systems, apparatuses, and methods for efficiently moving data for storage and processing. A compression unit within a processor includes multiple hardware lanes, selects two or more input words to compress, and for assigns them to two or more of the multiple hardware lanes. As each assigned input word is processed, each word is compared to an entry of a plurality of entries of a table. If it is determined that each of the assigned input words indexes the same entry of the table, the hardware lane with the oldest input word generates a single read request for the table entry and the hardware lane with the youngest input word generates a single write request for updating the table entry upon completing compression. Each hardware lane generates a compressed packet based on its assigned input word.

This application sets forth methods and apparatus to parallelize data decompression. An example method selecting initial starting positions in a compressed data bitstream; adjusting a first one of the initial starting positions to determine a first adjusted starting position by decoding the bitstream starting at a training position in the bitstream, the decoding including traversing the bitstream from the training position as though first data located at the training position is a valid token; outputting first decoded data generated by decoding a first segment of the bitstream starting from the first adjusted starting position; and merging the first decoded data with second decoded data generated by decoding a second segment of the bitstream, the decoding of the second segment starting from a second position in the bitstream and being performed in parallel with the decoding of the first segment, and the second segment preceding the first segment in the bitstream.