A system includes application of respective compression types to first data associated with each of a plurality of columns to generate compressed column data, determination of a first compression ratio for each of the plurality of columns based on the compressed column data, storage of the determined first compression ratios, application, for each of the plurality of columns, of the determined compression type to second data associated with the column to generate second compressed column data, determination of a second compression ratio for each of the plurality of columns based on the second compressed column data, determination of a value for each column based on the stored first compression ratio and the second compression ratio determined for the column, determination of a representative value of the determined values, and determination, based on the representative value, whether to re-determine a compression type for each of the plurality of columns.

Tiers of compression algorithms may be determined using compression information collected regarding compression ratios achieved for data sets using compression algorithms. Each tier may meet specified criteria regarding expected compression ratios achieved for a specified portion or number of data sets. Compression algorithms of each tier may be implemented by a different hardware device that may include hardware accelerators for the algorithms of the tier. Different tiers, and thus different hardware devices, achieve different levels of compression. A recommendation may be provided using compression information collected, such as from one of the hosts, regarding which hardware device to use for compression. The recommendation may be to purchase a license to use or whether to purchase a particular hardware device for compression. Compression information may be collected by a host that issues tagged I/Os providing a hint regarding what compression algorithm to use for the particular I/O operation data.

An information handling system includes a processor configured to process a training data file to determine an optimal data compression algorithm. The processor may also perform a compression ratio analysis that includes compressing the training data file using data compression algorithms, calculating a compression ratio associated with each of the data compression algorithms, determining an optimal compression ratio from the compression ratio associated with the each data compression algorithm; and determining a desirable data compression algorithm associated with the training data file based on the optimal compression ratio. The processor may also perform a probability analysis that includes generating a symbol transition matrix based on the desirable data compression algorithm, extracting statistical feature data based on the symbol transition matrix, and generating probability matrices based on the statistical feature data to determine the optimal data compression algorithm for each segment of a working data file.

Provided is a data processing system comprising a compression/expansion unit configured by including a compressor which compresses data, and an expander which expands the data compressed by the compressor, wherein the compression/expansion unit comprises a first interface unit capable of outputting configuration information of the compressor, and a second interface unit capable of outputting the data compressed by the compressor.

A framework and the associated method, schema and design for processing digital data, whether random or not, through encoding and decoding losslessly and correctly for purposes including the purposes of encryption/decryption or compression/decompression or both. There is no assumption of the digital information to be processed before processing. An universal coder is invented and now pigeonhole meets blackhole.

A method of compressing data is described in which the compressed data is generated by either or both of a primary compression unit or a reserve compression unit in order that a target compression threshold is satisfied. If a compressed data block generated by the primary compression unit satisfies the compression threshold, that block is output. However, if the compressed data block generated by the primary compression unit is too large, such that the compression threshold is not satisfied, a compressed data block generated by the reserve compression unit using a lossy compression technique, is output.

A method of compressing data is described in which the compressed data is generated by either or both of a primary compression unit or a reserve compression unit in order that a target compression threshold is satisfied. If a compressed data block generated by the primary compression unit satisfies the compression threshold, that block is output. However, if the compressed data block generated by the primary compression unit is too large, such that the compression threshold is not satisfied, a compressed data block generated by the reserve compression unit using a lossy compression technique, is output.

Embodiments of the present disclosure measure a state of a storage group within a storage array. The embodiments also increase or decrease a compression ratio corresponding to input/output (I/O) operations on the storage group based on a target data reduction ratio (DRR) of the storage array, an expected performance envelope, and a compressibility factor of the storage group.

A method for compressing data in a local cache of a web server is described. A local cache compression engine accesses values in the local cache and determines a cardinality of the values of the local cache. The local cache compression engine determines a compression rate of a compression algorithm based on the cardinality of the values of the local cache. The compression algorithm is applied to the cache based on the compression rate to generate a compressed local cache.

Certain aspects of the present disclosure provide techniques for committing log data in an application to a log data repository. An example method generally includes receiving, from an application, data to be committed to a remote storage location. A type of the received data is determined. The type of the received data is generally associated with a prioritization level and a compression mechanism to be used in committing the data to the remote storage location. An application execution context associated with the received data is determined. At a dispatch time associated with the prioritization level of the received data and the application execution context associated with the received data, a compressed data payload is generated and transmitted to the remote storage location. Generally, to compress the data payload, at least the received data is generally compressed based on the determined compression mechanism.