Aspects for generating compressed data streams with lookback pre-fetch instructions are disclosed. A data compression system is provided and configured to receive and compress an uncompressed data stream as part of a lookback-based compression scheme. The data compression system determines if a current data block was previously compressed. If so, the data compression system is configured to insert a lookback instruction corresponding to the current data block into the compressed data stream. Each lookback instruction includes a lookback buffer index that points to an entry in a lookback buffer where decompressed data corresponding to the data block will be stored during a separate decompression scheme. Once the data blocks have been compressed, the data compression system is configured to move a lookback buffer index of each lookback instruction in the compressed data stream into a lookback pre-fetch instruction located earlier than the corresponding lookback instruction in the compressed data stream.

In a data storage system, a prior set S of prefix codes for pseudo-dynamic compression as well as data compressed utilizing prior set S are stored. While data compressed utilizing prior set S are stored in the data storage system, the number of prefix codes utilized by the data storage system for pseudo-dynamic compression are augmented. Augmenting the number of codes includes determining a new set S′ of prefix codes for pseudo-dynamic compression from a training data set selected from a workload of the data storage system and storing the new set S′ in the data storage system with the prior set S.

Technologies for data decompression include a computing device that reads a symbol tag byte from an input stream. The computing device determines whether the symbol can be decoded using a fast-path routine, and if not, executes a slow-path routine to decompress the symbol. The slow-path routine may include data-dependent branch instructions that may be unpredictable using branch prediction hardware. For the fast-path routine, the computing device determines a next symbol increment value, a literal increment value, a data length, and an offset based on the tag byte, without executing an unpredictable branch instruction. The computing device sets a source pointer to either literal data or reference data as a function of the tag byte, without executing an unpredictable branch instruction. The computing device may set the source pointer using a conditional move instruction. The computing device copies the data and processes remaining symbols. Other embodiments are described and claimed.

Systems and methods relating generally to data storage, and more particularly to systems and methods for encoding to modify the size of an information set.

A mechanism is provided for high-throughput compression of data. Responsive to receiving an indication of a match of a current 4-byte sequence from an incoming data stream to stored hash values in a set of hash tables, numerous variables are set to initial values. Responsive to receiving a subsequent 4-byte sequence from the incoming data stream and determining that an active match variable is set to one, the subsequent 4-byte sequence is compared to data in a copy of the incoming data stream in memory at an active position with a predefined length offset. A constraint variable is set to a number of bytes for which the match is to be extended. Responsive to the constraint variable being below a predetermined number, a length, distance pair is output indicating a match to a previous pattern in the incoming data stream.

Massively parallel, block-based encoding and decoding technology that includes an encoded block format uses a plurality of processing cores to perform block-based encoding and decoding operations. The encoded block format includes a header and a payload. The encoded block format's headers represent unique single-Byte and multi-Byte event parameters that occur in the original data block from which each encoded block was generated. The encoded block format's payloads represent a sequence of single-Byte and multi-Byte events using tokens that associate each event with its corresponding parameter(s). Metadata can include an array of encoded block sizes that support random access.

Embodiments may be generally direct to apparatuses, systems, method, and techniques to determine a configuration for a plurality of connectors, the configuration to associate a first interconnect protocol with a first subset of the plurality of connectors and a second interconnect protocol with a second subset of the plurality of connectors, the first interconnect protocol and the second interconnect protocol are different interconnect protocols and each comprising one of a serial link protocol, a coherent link protocol, and an accelerator link protocol, cause processing of data for communication via the first subset of the plurality of connectors in accordance with the first interconnect protocol, and cause processing of data for communication via the second subset of the plurality of connector in accordance with the second interconnect protocol.

A processing device comprising compression circuitry to: determine a compression configuration to compress source data; generate a checksum of the source data in an uncompressed state; compress the source data into at least one block based on the compression configuration, wherein the at least one block comprises: a plurality of sub-blocks, wherein the plurality of sub-block includes a predetermined size; a block header corresponding to the plurality of sub-blocks; and decompression circuitry coupled to the compression circuitry, wherein the decompression circuitry to: while not outputting a decompressed data stream of the source data: generate index information corresponding to the plurality of sub-blocks; in response to generating the index information, generate a checksum of the compressed source data associated with the plurality of sub-blocks; and determine whether the checksum of the source data in the uncompressed format matches the checksum of the compressed source data.

A compression scheme can be selected for an input data stream based on characteristics of the input data stream. For example, when the input data stream is searched for pattern matches, input stream characteristics used to select a compression scheme can include one or more of: type and size of an input stream, a length of a pattern, a distance from a start of where the pattern is to be inserted to the beginning of where the pattern occurred previously, a gap between two pattern matches (including different or same patterns), standard deviation of a length of a pattern, standard deviation of a distance from a start of where the pattern is to be inserted to the beginning of where the pattern occurred previously, or standard deviation of a gap between two pattern matches. Criteria can be established whereby one or more characteristics are used to select a particular encoding scheme.

A method for sharing a hardware decompression engine, including performing a compression type check on a first data stream to determine a compression type of the first data stream, wherein the first data stream is compressed using one selected from a group consisting of a first compression type and a second compression type; wherein, when the first data stream is compressed with the second compression type: receiving the second compression type at a selector; converting the first data stream compressed with the second compression type into a second data stream of the first compression type; inputting the converted second data stream into the selector; and decompressing the converted second data stream using the hardware decompression engine capable of decompressing a data stream compressed using the first compression type. In other aspects, a system for sharing a hardware decompression engine and a computing system are provided.