A system and methods for bandwidth-efficient encoding of genome and bioinformatic sequence datasets comprising a sequence analyzer configured to: analyze a received sequence dataset to determine a sequence dataset file type, scan the sequence dataset to maintain a count of unique characters contained therein, identify positions where the unique character count increases by a power of two, deconstruct the sequence dataset into a plurality of sourceblocks at the identified positions, and encode the plurality of sourceblocks using a data deconstruction engine and library management module to assign each sourceblock a reference code.

An encoding method is illustrated. The method includes receiving data to be encoded onto a storage media, wherein the data corresponds to an item and is assigned to a data category. Further, the method includes parsing data into a plurality of data portions, based on one or more first characteristics associated with each of one or more characters in the data. The method further includes encoding, by the processor, the plurality of data portions using a plurality of encoding schemes, to generate a data packet, such that a first data portion of the plurality of data portions is encoded using a first encoding scheme of the plurality of encoding schemes and a second data portion of the plurality of data portions is encoded using a second encoding scheme of the plurality of encoding schemes, wherein the first encoding scheme is different from the second encoding scheme. Furthermore, the method includes transmitting the data packet, wherein the data packet is configured to be stored in the storage media.

Techniques are provided for implementing additional compression for existing compressed data. Format information stored within a data block is evaluated to determine whether the data block is compressed or uncompressed. In response to the data block being compressed according to a first compression format, the data block is decompressed using the format information. The data block is compressed with one or more other data blocks to create compressed data having a second compression format different than the first compression format.

A method for managing data includes obtaining a compression algorithm selection request for a data object, wherein the data object is generated by a production host, identifying, in response to the compression algorithm selection request, a set of production host performance objectives of the production host, performing a compression algorithm selection analysis using the set of production host performance objectives and a compression selection model to obtain a compression algorithm selection for a compression algorithm, specifying the compression algorithm to the production host using a data agent, wherein the data agent is operatively connected to the production host, initiating a compression on the data object using the data agent by applying the compression algorithm to obtain a compressed data object, and initiating a storage of the compressed data object.

A system and methods for bandwidth-efficient encoding of genome and bioinformatic sequence datasets comprising a sequence analyzer configured to: analyze a received sequence dataset to determine a sequence dataset file type, scan the sequence dataset to maintain a count of unique characters contained therein, identify positions where the unique character count increases by a power of two, deconstruct the sequence dataset into a plurality of sourceblocks at the identified positions, and encode the plurality of sourceblocks using a data deconstruction engine and library management module to assign each sourceblock a reference code.

A method of sending blocks of data from a client to be stored at a storage server, wherein for each block compression and encryption is performed at the client, and deduplication is performed at the server. Security is thus enhanced as the block is compressed and encrypted when it is sent over an unsecured network and when it is stored in potentially a third-party backup system. Provisions are made to enable addition of new compression algorithms and for retirement of old compression algorithms, while ensuring that a client would not receive a block which was compressed using an unsupported, e.g., retired, compression algorithm. In some examples a compression algorithm ID is tied to an encryption key version to enable refresh of blocks compressed with old algorithm.

An encoding method is illustrated. The method includes receiving data to be encoded onto a storage media, wherein the data corresponds to an item and is assigned to a data category. Further, the method includes parsing data into a plurality of data portions, based on one or more first characteristics associated with each of one or more characters in the data. The method further includes encoding, by the processor, the plurality of data portions using a plurality of encoding schemes, to generate a data packet, such that a first data portion of the plurality of data portions is encoded using a first encoding scheme of the plurality of encoding schemes and a second data portion of the plurality of data portions is encoded using a second encoding scheme of the plurality of encoding schemes, wherein the first encoding scheme is different from the second encoding scheme. Furthermore, the method includes transmitting the data packet, wherein the data packet is configured to be stored in the storage media.

An information handling system for compressing data includes multiple compression engines, a source data buffer to provide compression data to the compression engines, at least one destination data buffer to receive compressed data from the compression engines, and a compression engine driver. Each compression engine is configured to provide a different compression function. The compression engine driver directs each compression engine to compress data from the source data buffer, and retrieves select compressed data from a first one of the compression engines from the at least one destination data buffer. The selection is based upon a selection criterion.

A data compression method includes: obtaining a to-be-compressed object; searching a recommendation record for a recommended compression coding rule that meets a compression rate condition, the recommendation record being configured to record a compression coding rule of a historical compressed object and corresponding compression rate information, and the historical compressed object being of a same type as the to-be-compressed object; and if the recommended compression coding rule that meets the compression rate condition is found, compressing the to-be-compressed object by using the recommended compression coding rule; and if the recommended compression coding rule that meets the compression rate condition is not found, starting a regular compression coding process to obtain estimated compression rates of a plurality of compression coding rules for the to-be-compressed object, selecting a target compression coding rule based on at least the estimated compression rates, and compressing the to-be-compressed object by using the target compression coding rule.

A lossless data compressor of an aspect includes a first lossless data compressor circuitry coupled to receive input data. The first lossless data compressor circuitry is to apply a first lossless data compression approach to compress the input data to generate intermediate compressed data. The apparatus also includes a second lossless data compressor circuitry coupled with the first lossless data compressor circuitry to receive the intermediate compressed data. The second lossless data compressor circuitry is to apply a second lossless data compression approach to compress at least some of the intermediate compressed data to generate compressed data. The second lossless data compression approach different than the first lossless data compression approach. Lossless data decompressors are also disclosed, as are methods of lossless data compression and decompression.