A decoder includes circuitry configured to receive a bitstream identify, in the bitstream, a current frame, wherein the current frame includes a first region and a third region, detect, in the bitstream, an indication that the first region is encoded according to a lossless encoding protocol, and decode the current frame, wherein decoding the current frame further comprises decoding the first region using a lossless decoding protocol corresponding to the lossless encoding protocol.

Techniques are provided for compression of log data using field types. An exemplary method comprises: obtaining at least one log message, wherein the at least one log message comprises a message template and at least one message variable, wherein the message template comprises at least one field corresponding to the at least one message variable; obtaining a compression index that maps a plurality of message templates to a corresponding message signature; identifying a predefined field type of the at least one field; selecting a compression technique to apply to the at least one message variable based on the predefined field type to obtain a compressed message variable; and writing the compressed message variable and a message signature corresponding to the message template of the at least one log message to a log file. The at least one log message can be a historical log message or part of a real-time stream of log messages.

Provided are a data transmission method and device. The method includes: processing a first data packet to be sent by using a compression strategy obtained in advance from a receiving end, deleting specified duplicated data comprised in the compression strategy in the first data packet; generating a second data packet to be sent from the processed first data packet, where the second data packet includes a modification record field for indicating the deleted duplicated data; and sending the second data packet to the receiving end.

Methods, apparatus, systems and articles of manufacture to compress data are disclosed. An example apparatus includes an off-chip memory to store data; a data slicer to split a dataset into a plurality of blocks of data; a data processor to select a first compression technique for a first block of the plurality of blocks of data based on first characteristics of the first block; and select a second compression technique for a second block of the plurality of blocks of data based on second characteristics of the second block; a first compressor to compress the first block using the first compression technique to generate a first compressed block of data; a second compressor to compress the second block using the second compression technique to generate a second compressed block of data; a header generator to generate a first header identifying the first compression technique and a second header identifying the second compression technique; and an interface to transmit the first compressed block of data with the first header and the second compressed block of data with the second header to be stored in the off chip memory.

Systems and methods for dynamic and automatic data storage scheme switching in a distributed data storage system. A machine learning-based policy for computing probable future content item access patterns based on historical content item access patterns is employed to dynamically and automatically switch the storage of content items (e.g., files, digital data, photos, text, audio, video, streaming content, cloud documents, etc.) between different data storage schemes. The different data storage schemes may have different data storage cost and different data access cost characteristics. For example, the different data storage schemes may encompass different types of data storage devices, different data compression schemes, and/or different data redundancy schemes.

An encoding method is illustrated. The method includes receiving data to be encoded onto a storage media, wherein the data corresponds to an item and is assigned to a data category. Further, the method includes parsing data into a plurality of data portions, based on one or more first characteristics associated with each of one or more characters in the data. The method further includes encoding, by the processor, the plurality of data portions using a plurality of encoding schemes, to generate a data packet, such that a first data portion of the plurality of data portions is encoded using a first encoding scheme of the plurality of encoding schemes and a second data portion of the plurality of data portions is encoded using a second encoding scheme of the plurality of encoding schemes, wherein the first encoding scheme is different from the second encoding scheme. Furthermore, the method includes transmitting the data packet, wherein the data packet is configured to be stored in the storage media.

Techniques for genetic programming based compression determination are described herein. An aspect includes adding a first plurality of randomly generated compression algorithms to a first set of compression algorithms. Another aspect includes determining a respective mutated version of each of the first plurality of randomly generated compression algorithms. Another aspect includes adding the determined mutated versions to the first set of compression algorithms. Another aspect includes evaluating and ranking the first set of compression algorithms based on respective achieved degrees of compression.

Methods and systems for storing data include compressing data inflated from a first compression format into a second format using a processor and verifying contents of the data concurrently with compressing the data. Compression is aborted responsive to a failure of the content verification, but an output of the compression is stored to a tape drive until the compression is aborted. The tape drive is rolled back to a file start position after the compression is aborted and compression of any remaining uncompressed data is skipped after the compression is aborted. The data is stored to the tape drive after rolling the tape drive back.

Embodiments of the invention are directed to a DEFLATE compression accelerator and to a method for verifying the correctness of the DEFLATE compression accelerator. The accelerator includes an input buffer and a Lempel-Ziv 77 (LZ77) compressor communicatively coupled to an output of the input buffer. A switch is communicatively coupled to the output of the input buffer and to the output of the LZ77 compressor. The switch is configured to bypass the LZ77 compressor during a compression test. The accelerator further includes a deflate Huffman encoder communicatively coupled to an output of the switch and an output buffer communicatively coupled to the deflate Huffman encoder. When the switch is not bypassed, the compressor can be modified to produce repeatable results.

Disclosed is an operating method of a semiconductor device, including acquiring resource information on a plurality of hardware resources, receiving a compression request or a decompression request for data, acquiring context information on the semiconductor device, in response to receiving the compression request or the decompression request for the data, selecting a compression algorithm for compressing or decompressing the data, based on the context information, selecting, among the plurality of hardware resources, a hardware resource for performing the selected compression algorithm, based on the acquired resource information, and compressing or decompressing the data using the selected compression algorithm and the selected hardware resource.