Techniques are provided for implementing additional compression for existing compressed data. Format information stored within a data block is evaluated to determine whether the data block is compressed or uncompressed. In response to the data block being compressed according to a first compression format, the data block is decompressed using the format information. The data block is compressed with one or more other data blocks to create compressed data having a second compression format different than the first compression format.

A method, computer program product, and computing system for receiving a candidate data portion; calculating a distance-preserving hash for the candidate data portion; and performing an entropy analysis on the distance-preserving hash to generate a hash entropy for the candidate data portion.

A method of sending blocks of data from a client to be stored at a storage server, wherein for each block compression and encryption is performed at the client, and deduplication is performed at the server. Security is thus enhanced as the block is compressed and encrypted when it is sent over an unsecured network and when it is stored in potentially a third-party backup system. Provisions are made to enable addition of new compression algorithms and for retirement of old compression algorithms, while ensuring that a client would not receive a block which was compressed using an unsupported, e.g., retired, compression algorithm. In some examples a compression algorithm ID is tied to an encryption key version to enable refresh of blocks compressed with old algorithm

Provided are a data transmission method and device. The method includes: processing a first data packet to be sent by using a compression strategy obtained in advance from a receiving end, deleting specified duplicated data comprised in the compression strategy in the first data packet; generating a second data packet to be sent from the processed first data packet, where the second data packet includes a modification record field for indicating the deleted duplicated data; and sending the second data packet to the receiving end.

A system comprises a computer including a processor and a memory. The memory includes instructions such that the processor is programmed to: receive metadata and results including at least one data transmission constraint, perform a search of a k-d tree based on the at least one data transmission constraint, and select at least one data compression process from the k-d tree based on the search. The k-d tree includes a plurality of nodes corresponding to data compression solutions.

The present disclosure relates to a system and method for compressing a dataset. The dataset can be divided in to a plurality of groups. Each group can be converted independently into corresponding text file using dictionary coding technique. The corresponding text files can be compressed independently into corresponding compressed files. Finally, all the corresponding compressed files can be combined together to generate a complete compressed data.

Methods, apparatus, systems and articles of manufacture to compress data are disclosed. An example apparatus includes a data slicer to split a dataset into a plurality of blocks of data; a data processor to select a first compression technique for a first block of the plurality of blocks of data based on first characteristics of the first block; and select a second compression technique for a second block of the plurality of blocks of data based on second characteristics of the second block; a first compressor to compress the first block using the first compression technique to generate a first compressed block of data; a second compressor to compress the second block using the second compression technique to generate a second compressed block of data; and a header generator to generate a first header identifying the first compression technique and a second header identifying the second compression technique.

The disclosed technology is generally directed to the compression of inverted indexes. In one example of the technology, an inverted index that includes a plurality of posting lists and metadata is provided. The inverted index indicates compression settings that are associated with the plurality of posting lists. At periodic scheduled times, a regeneration is performed on the inverted index. The regeneration includes decompressing the inverted index. The decompressing uses the compression settings indicated by the inverted index. The regeneration further includes determining compression settings to use during a next periodic scheduled time of the plurality of periodic scheduled times, such that at least a first posting list of the plurality of posting lists uses a different compression setting than a second posting list of the plurality of posting lists.

A data storage system in which different copies of a data object (e.g., a file) can be compressed using different compression processes (e.g. different compression algorithms/processes and/or compression parameters), with some favoring faster decompression, while others favoring storage space savings. When a data object needs to be accessed, the copy of the data object that can be decompressed using minimal resource (computing and/or time) can be located and retrieved.

An image processing apparatus includes a memory configured to store compressed data; and frame buffer compressing circuitry which includes encoder circuitry configured to compress at least some of source data to generate the compressed data and transmit the compressed data to the memory, and decoder circuitry configured to read and decompress the compressed data from the memory, in which the compressed data includes a payload and a header including actual compressed data and flag, the frame buffer compressing circuitry is configured to reflect a result obtained by comparing an accumulated compressibility corresponding to the compressed data with a reference compressibility in the flag, and is configured to perform compression or decompression in a lossy mode or a lossless mode depending on the flag.