Techniques are provided herein for more efficiently storing images that have a common subject, such as product images that share the same product in the image. Each image undergoes an adaptive tiling procedure to split the image into a plurality of tiles, with each tile identifying a region of the image having pixels with the same content. The tiles across multiple images can then be clustered together and those tiles having identical content are removed. Once all duplicate tiles have been removed from the set of all tiles across the images, the tiles are once again clustered based on their encoding scheme and certain encoding parameters. Tiles within each cluster are compressed using the best compression technique for the tiles in each corresponding cluster. By removing duplicative tile content between numerous images of the same subject, the total amount of data that needs to be stored is reduced.

According to one embodiment, a memory system includes a first compression unit, a second compression unit, a non-volatile memory, a first decoding unit, a conversion unit and an output unit. The first compression unit is configured to output second data obtained by compressing first data. The second compression unit is configured to output third data obtained by compressing the second data. Fourth data based on the third data is written to the non-volatile memory. The first decoding unit is configured to decode the third data based on the fourth data to the second data. The conversion unit is configured to acquire fifth data by converting a format of the second data. The output unit is configured to output the fifth data to a host.

Described are systems and methods for lossy compression and restoration of data. The raw data is first truncated. Then the truncated data is compressed. The compressed truncated data can then be efficiently stored and/or transmitted using fewer bits. To restore the data, the compressed data is then decompressed and restoration bits are concatenated. The restoration bits are selected to compensate for statistical biasing introduced by the truncation.

There is provided a system and method for compression and decompression of a data stream used by machine learning networks. The method including: encoding each value in the data stream, including: determining a mapping to one of a plurality of non-overlapping ranges, each value encoded as a symbol representative of the range and a corresponding offset; and arithmetically coding the symbol using a probability count; storing a compressed data stream including the arithmetically coded symbols and the corresponding offsets; and decoding the compressed data stream with arithmetic decoding using the probability count, the arithmetic decoded symbols use the offset bits to arrive at a decoded data stream; and communicating the decoded data stream for use by the machine learning networks.

The inventor has conceived, and reduced to practice, a system and method for data compaction using that applies delta encoding methods to entropy encoding methods to improve data compaction of entropy encoding methods under certain conditions and when compacting data having certain characteristics. Delta encoding may be applied to entropy encoding methods to further compact data sets by reducing the number of sourceblocks included in a codebook to those most commonly encountered in data to be encoded and, where mismatches occur during encoding, using delta encoding of bit differences with existing sourceblocks in the codebook rather than adding new sourceblocks to the codebook.

Methods, systems, and devices for wireless communications are described. A user equipment (UE) may encode channel state feedback (CSF) information to compress the CSF information to a first encoding output associated with a first dimensional space, and apply entropy coding to the first encoding output of the channel state feedback information. The entropy coding may transform the first encoding output to a second encoding output associated with a second dimensional space that is smaller than the first dimensional space of the first encoding output. The UE may transmit a CSF message comprising the second encoding output. A network device may receive the CSF message and apply entropy decoding to the compressed CSF information to partially decompress the compressed CSF information to a first decoding output. The network device may decode the first decoding output to completely decompress the compressed CSF information to a second decoding output.

Methods, systems, and computer programs for compressing nucleic acid sequence data. A method can include obtaining nucleic acid sequence data representing: (i) a read sequence, and (ii) a plurality of quality scores, determining whether the read sequence includes at least one “N” base, based on a determination that the read sequence does not include at least one “N” base, generating a first encoded data set by using a first encoding process to encode each of the quality scores of the read sequence using a base-(x minus 1) number, where x is an integer representing a number of different quality scores used by the nucleic acid sequencing device, and using a second encoding process to encode the first encoded data set, thereby compressing the data to be compressed.

Techniques for generating data sets may include: receiving an initial buffer that achieves a compression ratio responsive to compression processing using a compression algorithm, the initial buffer including first content located at a first position in the initial buffer and including second content located at a second position in the initial buffer; and generating a data set of buffers using the initial buffer. The data set may be expected to achieve a specified deduplication ratio responsive to deduplication processing and to achieve the compression ratio responsive to compression processing using the compression algorithm. Generating the data set may include generating a first plurality of buffers where each buffer of the first plurality is not a duplicate of another buffer in the first plurality, and generating a second plurality of duplicate buffers. Each duplicate buffer may be a duplicate of a buffer in the first plurality of buffers.

Embodiments of the present disclosure include a digital circuit and method for multi-stage compression. Digital data values are compressed using a multi-stage compression algorithm and stored in a memory. A decompression circuit receives the values and performs a partial decompression. The partially compressed values are provided to a processor, which performs the final decompression. In one embodiment, a vector of N length compressed values are decompressed using a first bit mask into two N length sets having non-zero values. The two N length sets are further decompressed using two M length bit masks into M length sparse vectors, each having non-zero values.

The present invention realizes a storage device that has a high data reduction effect without decreasing I/O performances. The storage device includes a processor, an accelerator, a memory, and a storage medium, the processor specifies data to be compressed that is data stored in the storage medium from data stored in the memory and transmits a compression instruction including information relating to the data to be compressed to the accelerator, and the accelerator reads the plurality of continuous items of data from the memory and compresses the plurality of items of data to be compressed obtained by excluding data that is not to be compressed from the plurality of items of data, based on the information relating to the data to be compressed received from the processor, to generate compressed data stored in the storage device.