A method, computer program product, and computer system for identifying, by a computing device, storage containers that contain cold data. At least a portion of the storage containers may be processed to determine whether a first compression technique will result in a higher level of compression above a threshold level of compression than a second compression technique. The storage containers may be processed using the first compression technique based upon, at least in part, determining that the first compression technique will result in the higher level of compression above the threshold level of compression than the second compression technique.

One example method includes file specific compression selection. Compression metrics are generated for a chunk of a file using a reference compressor. Compression metrics for other compressors are determined from the metrics of the reference compressor. A compressor is then selected to compress the file.

Tiers of compression algorithms may be determined using compression information collected regarding compression ratios achieved for data sets using compression algorithms. Each tier may meet specified criteria regarding expected compression ratios achieved for a specified portion or number of data sets. Compression algorithms of each tier may be implemented by a different hardware device that may include hardware accelerators for the algorithms of the tier. Different tiers, and thus different hardware devices, achieve different levels of compression. A recommendation may be provided using compression information collected, such as from one of the hosts, regarding which hardware device to use for compression. The recommendation may be to purchase a license to use or whether to purchase a particular hardware device for compression. Compression information may be collected by a host that issues tagged I/Os providing a hint regarding what compression algorithm to use for the particular I/O operation data.

The present disclosure provides a data processing system and method. The method may include obtaining preliminary data. The method may also include generating first encoded data by compressing the preliminary data in a first encoding procedure. The method may further include determining whether a compression ratio of the first encoded data is below a compression threshold. The method may still further include generating second encoded data by compressing the preliminary data in a second encoding procedure in response to determining that the compression ratio of the first encoded data is below the compression threshold, wherein the first encoding procedure and the second encoding procedure are lossless compressions.

An apparatus, method and computer program is described comprising: initialising weights of a target encoder based on a source encoder; initialising weights of a target discriminator associated with the target encoder such that the target discriminator is initialised to match a source discriminator associated with the source encoder; applying some of a target data set to the target encoder to generate target encoder outputs; applying the target encoder outputs to the target discriminator to generate a first local loss function output; training the target encoder to seek to increase the first local loss function output; training the target discriminator to seek to decrease the first local loss function output; and synchronising weights of the target discriminator and the source discriminator.

A method for compressing is provided. The method includes compressing, via a processor, a portion of a first data packet to generate a second data packet having a compressed portion. The method includes transmitting the second data packet having the compressed portion via an interface to a co-processor. The processor and the co-processor are communicatively coupled via the interface. The method also includes unpacking, via the co-processor, the compressed portion of the second data packet to restore the first data packet.

A first storage system compresses data relating to read and write by a primary site and stores the data in a first physical volume. A second storage system compresses data relating to read and write by a secondary site and stores the data in a second physical volume. When performing replication for transferring the data stored in the first physical volume of the first storage system to the second storage system and storing the data in the second physical volume, the first storage system and the second storage system determine, based on a compression scheme executable by the first storage system and a compression scheme executable by the second storage system, a compression scheme to be applied to transfer target data and transfer the transfer target data compressed by the determined compression scheme.

Aspects of the invention include identifying a first subsystem and a second subsystem of a plurality of subsystems respectively storing a first compressed data and a second compressed data, wherein the first compressed data and the second compressed data are fragments of a requested data. A compression method used to compress the first compressed data and second compressed data is identified. A first accelerator of first subsystem and a second accelerator of the second subsystem is identified. The first compressed data from a first local memory of the first subsystem is offloaded to the first accelerator, and the second compressed data from a second local memory of the second subsystem is offloaded to the second accelerator, wherein offloading comprises provided a decompression method for the first compressed data and the second compressed data.

A system includes application of respective compression types to first data associated with each of a plurality of columns to generate compressed column data, determination of a first compression ratio for each of the plurality of columns based on the compressed column data, storage of the determined first compression ratios, application, for each of the plurality of columns, of the determined compression type to second data associated with the column to generate second compressed column data, determination of a second compression ratio for each of the plurality of columns based on the second compressed column data, determination of a value for each column based on the stored first compression ratio and the second compression ratio determined for the column, determination of a representative value of the determined values, and determination, based on the representative value, whether to re-determine a compression type for each of the plurality of columns.

A method of data reduction in a block-based data storage system includes selecting a starting position in a block based on a deterministic function of block data content. Then for an unaligned block beginning at the selected starting position, a block digest (e.g., block hash) is generated and compared with stored block digests of stored data blocks. If there is a match, and the stored block matches the unaligned block, then a reference to the stored block is stored in place of the unaligned block, and otherwise the unaligned block and a corresponding digest are stored. The storing of references to already stored blocks, without the constraint of observing aligned-block boundaries, realizes increased savings of physical storage space.