The transmission of broadcast data, such as financial data and news feeds, is accelerated over a communication channel using data compression and decompression to provide secure transmission and transparent multiplication of communication bandwidth, as well as reduce the latency. Broadcast data may include packets having fields. Encoders associated with particular fields may be selected to compress those particular fields.

One embodiment provides a method comprising receiving an input data stream, partitioning the input data stream into a plurality of data blocks, and compressing the data blocks utilizing a plurality of processor sets. Each processor set is assigned a data block to compress. The processor sets compress in parallel to exploit inter-block parallelism. Each processor set comprises one or more processors that collaborate in compressing an assigned data block to exploit intra-block parallelism. The method further comprises writing a plurality of compressed data blocks resulting from the compressing to a storage device in encoded form.

Systems, apparatuses, and methods for efficiently moving data for storage and processing. A compression unit within a processor includes multiple hardware lanes, selects two or more input words to compress, and for assigns them to two or more of the multiple hardware lanes. As each assigned input word is processed, each word is compared to an entry of a plurality of entries of a table. If it is determined that each of the assigned input words indexes the same entry of the table, the hardware lane with the oldest input word generates a single read request for the table entry and the hardware lane with the youngest input word generates a single write request for updating the table entry upon completing compression. Each hardware lane generates a compressed packet based on its assigned input word.

This application sets forth methods and apparatus to parallelize data decompression. An example method selecting initial starting positions in a compressed data bitstream; adjusting a first one of the initial starting positions to determine a first adjusted starting position by decoding the bitstream starting at a training position in the bitstream, the decoding including traversing the bitstream from the training position as though first data located at the training position is a valid token; outputting first decoded data generated by decoding a first segment of the bitstream starting from the first adjusted starting position; and merging the first decoded data with second decoded data generated by decoding a second segment of the bitstream, the decoding of the second segment starting from a second position in the bitstream and being performed in parallel with the decoding of the first segment, and the second segment preceding the first segment in the bitstream.

Aspects of dynamic data compression selection are presented. In an example method, as uncompressed data chunks of a data stream are compressed, at least one performance factor affecting selection of one of multiple compression algorithms for the uncompressed data chunks of the data stream may be determined. Each of the multiple compression algorithms may facilitate a different expected compression ratio. One of the multiple compression algorithms may be selected separately for each uncompressed data chunk of the data stream based on the at least one performance factor. Each uncompressed data chunk may be compressed using the selected one of the multiple compression algorithms for the uncompressed data chunk.

Systems, apparatuses, and methods for compression of frequent data values across narrow links are disclosed. In one embodiment, a system includes a processor, a link interface unit, and a communication link. The link interface unit is configured to receive a data stream for transmission over the communication link, wherein the data stream is generated by the processor. The link interface unit determines if blocks of data of a first size from the data stream match one or more first data patterns and the link interface unit determines if blocks of data of a second size from the data stream match one or more second data patterns. The link interface unit sends, over the communication link, only blocks of data which do not match the first or second data patterns.

Data weighted averaging of a thermometric coded input signal is accomplished by controlling the operation of a crossbar switch matrix to generate a current cycle of a data weighted averaging output signal using a control signal generated in response to feedback of a previous cycle of the data weighted averaging output signal. The control signal specifies a bit location for a beginning logic transition of the data weighted averaging output signal in the current cycle based on detection of an ending logic transition of the data weighted averaging output signal in the previous cycle.

Aspects of dynamic data compression selection are presented. In an example method, as uncompressed data chunks of a data stream are compressed, at least one performance factor affecting selection of one of multiple compression algorithms for the uncompressed data chunks of the data stream may be determined. Each of the multiple compression algorithms may facilitate a different expected compression ratio. One of the multiple compression algorithms may be selected separately for each uncompressed data chunk of the data stream based on the at least one performance factor. Each uncompressed data chunk may be compressed using the selected one of the multiple compression algorithms for the uncompressed data chunk.

Technologies for efficiently compressing data with run detection include a compute device. The compute device is to produce a hash as a function of a symbol at a present position and a predefined number of symbols after the present position in an input stream, determine whether the symbol at the present position is part of a run, obtain, from a hash table, a chain of pointers to previous positions in the input stream associated with the hash, determine, as a function of whether the symbol is part of a run and to identify a matched string, a number of strings referenced by the chain of pointers to compare to a string associated with the present position in the input stream, and output, in response to an identification of a matched string, a reference to the matched string in a set of compressed output data.

Technologies for high-performance single-stream data compression include a computing device that updates an index data structure based on an input data stream. The input data stream is divided into multiple chunks. Each chunk has a predetermined length, such as 136 bytes, and overlaps the previous chunk by a predetermine amount, such as eight bytes. The computing device processes multiple chunks in parallel using the index data to generate multiple token streams. The tokens include literal tokens and reference tokens that refer to matching data from earlier in the input data stream. The computing device thus searches for matching data in parallel. The computing device merges the token streams to generate a single output token stream. The computing device may merge a pair of tokens from two different chunks to generate one or more synchronized tokens that are output to the output token stream. Other embodiments are described and claimed.