Techniques are disclosed relating to data compression. In some embodiments, compression circuitry determines, at least partially in parallel for multiple different lossless compression techniques, a number of bits needed to represent a least compressible pixel, using that technique, in a set of pixels being compressed. The compression techniques may include neighbor, origin, and gradient techniques, for example. The compression circuitry may select one of the compression techniques based on the determined numbers of bits for the multiple compression techniques and corresponding header sizes. In some embodiments, the compression circuitry determines, for multiple regions of pixels in the set of pixels, for ones of the compression techniques, a region number of bits needed to represent a least compressible pixel, using that technique. The selection of a compression technique may be further based on region numbers of bits.

A data compression technique involves: selecting, from a data block, a plurality of bit strings as sample data. The technique further involves: determining a set of characters included in the sample data. Each character in the set of characters is represented by at least one bit string among the plurality of bit strings. The technique further involves: compressing the data block if statistical characteristics of the set of characters match predetermined statistical characteristics. Accordingly, incompressible data can be filtered out efficiently and accurately, so as to utilize computing resources more efficiently.

A method for managing data includes obtaining a compression algorithm selection request for a data object, wherein the data object is generated by a production host, identifying, in response to the compression algorithm selection request, a set of production host performance objectives of the production host, performing a compression algorithm selection analysis using the set of production host performance objectives and a compression selection model to obtain a compression algorithm selection for a compression algorithm, specifying the compression algorithm to the production host using a data agent, wherein the data agent is operatively connected to the production host, initiating a compression on the data object using the data agent by applying the compression algorithm to obtain a compressed data object, and initiating a storage of the compressed data object.

One example method includes file specific compression selection. Compression metrics are generated for a chunk of a file. Using a set of training data, the compression metrics are corrected using a correction factor to determine estimated file compression metrics. A compressor is then selected to compress the file based on at least the estimated file compression metrics.

An embodiment of the invention may include a method, computer program product and system for saving data received from a host computing device to a storage system. The storage system includes at least one processor and at least one storage. An embodiment may include storing the received data to the storage on a record basis. A record includes a record header including information indicative of an implemented compression method of the record. An embodiment may include monitoring a processing load of the at least one processor. In response to the processing load being less than a predetermined level, an embodiment may include further compressing the record utilizing a high-ratio compression method based on the record requiring further compression. An embodiment may include updating the record header information to reflect details of the utilized a high-ratio compression method. An embodiment may include storing the further compressed record to the storage.

Techniques are provided for implementing additional compression for existing compressed data. Format information stored within a data block is evaluated to determine whether the data block is compressed or uncompressed. In response to the data block being compressed according to a first compression format, the data block is decompressed using the format information. The data block is compressed with one or more other data blocks to create compressed data having a second compression format different than the first compression format.

A decoder includes circuitry configured to receive a bitstream identify, in the bitstream, a current frame, wherein the current frame includes a first region and a third region, detect, in the bitstream, an indication that the first region is encoded according to a lossless encoding protocol, and decode the current frame, wherein decoding the current frame further comprises decoding the first region using a lossless decoding protocol corresponding to the lossless encoding protocol.

Systems and methods for dynamic and automatic data storage scheme switching in a distributed data storage system. A machine learning-based policy for computing probable future content item access patterns based on historical content item access patterns is employed to dynamically and automatically switch the storage of content items (e.g., files, digital data, photos, text, audio, video, streaming content, cloud documents, etc.) between different data storage schemes. The different data storage schemes may have different data storage cost and different data access cost characteristics. For example, the different data storage schemes may encompass different types of data storage devices, different data compression schemes, and/or different data redundancy schemes.

Techniques are disclosed relating to data compression. In some embodiments, compression circuitry determines, at least partially in parallel for multiple different lossless compression techniques, a number of bits needed to represent a least compressible pixel, using that technique, in a set of pixels being compressed. The compression techniques may include neighbor, origin, and gradient techniques, for example. The compression circuitry may select one of the compression techniques based on the determined numbers of bits for the multiple compression techniques and corresponding header sizes. In some embodiments, the compression circuitry determines, for multiple regions of pixels in the set of pixels, for ones of the compression techniques, a region number of bits needed to represent a least compressible pixel, using that technique. The selection of a compression technique may be further based on region numbers of bits.

This application provides a data compression method and apparatus, and relates to the field of storage technologies. The method includes: after receiving to-be-stored first data, a storage device may determine whether the first data is hot write data, and compress the first data using a selected compression algorithm if the first data is not hot write data.