One aspect of the present disclosure relates to a data compression method. The method includes generating, by one or more processors, compressed data from data, wherein the compressed data includes one or more unduplicated values of the data and generating, by the one or more processors, index data from the data, wherein the index data includes indices indicative of storage locations for the unduplicated values.

The embodiments described herein relate to managing compressed data to optimize file compression. A compression is performed on a first set of data to create a first set of compressed data partitions in a compression group. A partition table is constructed, and partition entries for the first data block are added to the table in conjunction with the first set. A current size of the compression group is assessed. In response to a compression ratio being greater than a target compression ratio and internal fragmentation of the compression group being smaller than a threshold, the compression group is dynamically completed. The dynamic completion decides a size for the compression group. The partition table is added to the compression group by assigning space within the first compression group for the table. The compression group is written to persistent storage.

Method and apparatus for processing data are disclosed. The method may include: sorting to-be-compressed at least one piece of data in a predetermined order; for a sorted piece of data, in response to the data having a common prefix with a previous piece of data of the piece of data, adding the common prefix to a common prefix set; dividing the sorted at least one piece of data into at least one group based on the common prefix set and determining a common prefix of each group; and storing, for a group in the at least one group, a common prefix of the group and a non-common prefix portion of each piece of data in the group, and recording a common prefix corresponding to each piece of data in the each group and a shared length between the each piece of data and the common prefix.

Provided is a storage system that can store various types of and large amounts of sensor data while accurately compressing the sensor data without wasting storage resources. The storage system includes: a storage that records sensor data output from a plurality of sensors; a processor that controls recording of sensor data in the storage; and a memory that records parameters of the plurality of sensors. The processor reads parameters assigned to the sensors that output the sensor data from the memory, normalizes the sensor data based on the parameters, compresses the normalized sensor data, and records the compressed sensor data in the storage.

A memory system includes a nonvolatile memory, an interface circuit, and a controller configured to upon receipt of a plurality of write commands for storing write data in the nonvolatile memory via the interface circuit, acquire compression-ratio information about the write data associated with each write command, determine a compression ratio of each write data based on the acquired compression-ratio information, and determine an execution order of the write commands based on the determined compression ratio.

A system and method for a non-transient computer readable medium containing program instructions for causing a computer to perform a method for compressing data comprising the steps of receiving a data string for compression, the data string including a plurality of data elements, creating a template based on processing the data string, the template including common information across all data elements of the data string, creating one or more entries, wherein the one or more entries include information that is different to the template, and storing the template and the one or more entries.

Various embodiments include methods and devices for implementing compression of high dynamic ratio fields. Various embodiments may include receiving a compression block having data units, receiving a mapping for the compression block, wherein the mapping is configured to map bits of each data unit to two or more data fields to generate a first set of data fields and a second set of data fields, compressing the first set of data fields together to generate a compressed first set of data fields, and compressing the second set of data fields together to generate a compressed second set of data fields.

Various embodiments are provided for length-limited Huffman encoding in a data compression accelerator in a computing environment by a processor. Symbol counts of a plurality of symbols in compressed data may be normalized and manipulated according to a maximum code length limiting operation such that those of the plurality of symbols having a least frequent symbol count have a symbol count equal to a maximum code length of a Huffman tree.

A radar data processing method is provided. The method includes dividing to-be-compressed radar data into groups, determining encoding parameters of each group according to at least one radar data in each group, and encoding each radar data in each group according to the encoding parameters of each group, to obtain encoded radar data.

A data storage system performs partial compression and decompression of a set of memory items. The memory items each include a data block and a tag with a prefix making up at least part of the tag. The memory items are ordered based on the prefixes. A code word is created containing compressed information representing values of the prefixes for the set of memory items. The code word and block data for each of the memory items are stored in a memory. The code word is decompressed to recover the prefixes.