A method for compressing digital data, including: extrapolating a value of each sample of data to be compressed as a function of a value of at least one preceding sample, to produce an extrapolated sample; differentiating between each extrapolated sample and the corresponding sample of data to be compressed, to produce a differentiated sample; and deleting redundancy between successive differentiated samples produced by the differentiating stage.

A framework and the associated method, schema and design for processing digital data, whether random or not, through encoding and decoding losslessly and correctly for purposes including the purposes of encryption/decryption or compression/decompression or both. There is no assumption of the digital information to be processed before processing. An universal coder is invented and now pigeonhole meets blackhole.

The systems and methods described herein may improve the rendering of computer-generated three-dimensional models using progressive mesh compression. In various implementations, an initial mesh may be obtained and encoded into a data stream. Subsequent meshes may then be encoded based on a superset relationship between consecutive meshes. If the vertices of the mesh are not a superset of a prior mesh, the mesh may be encoded within an intermediate symbol stream using a non-incremental mesh compression technique. If the vertices of the mesh are a superset of a prior mesh, a sequence of per-triangle operators may be applied to the mesh to produce a progressive mesh. The mesh may then be encoded by encoding the operators applied to the mesh in sequence. When encoding the mesh, coordinates of vertices may be defined based on the difference between the coordinates and predicted values generated using a prediction function.

According to one embodiment, a code table generation device includes a table generation unit, a merge unit and a tree generation unit. The table generation unit generates a frequency table including symbols and frequencies of occurrence respectively associated with the symbols, based on a frequency of occurrence for each symbol of input symbols. The merge unit acquires top K symbols in descending order of the frequencies of occurrence and remaining symbols from the symbols, divides the remaining symbols into one or more symbol sets, and determines a frequency of occurrence associated with a root node of each of subtrees correspond to the respective symbol sets. The tree generation unit generates a Huffman tree using the K symbols and the root node of each of the subtrees.

A FRAMEWORK and the associated method, schema and design for processing digital data, whether random or not, through encoding and decoding losslessly and correctly for purposes including the purposes of encryption/decryption or compression/decompression or both. There is no assumption of the digital information to be processed before processing. A Universal Coder is invented and now Pigeonhole meets Blackhole.

Methods for estimating cost savings in a storage system using an external host system. One method includes accessing over a communication network data from a unit of storage of a data storage system, wherein each of the blocks of data is uncompressed. A plurality of blocks is parsed from the data. A plurality of fingerprints is generated from the blocks using a hash algorithm. A deduplication ratio is estimated for the plurality of blocks stored in the unit of storage using a hyperloglog algorithm and a first plurality of buckets compartmentalizing the plurality of blocks, wherein the first plurality of buckets is defined by precision bits of the plurality of fingerprints. An effective compression ratio is estimated for the plurality of blocks stored in the unit of storage using the hyperloglog algorithm and a second plurality of buckets compartmentalizing the plurality of blocks, wherein the second plurality of buckets is defined by ranges of compression ratios.

Techniques for handling data compression in which metadata that indicates which portions of data are compressed are which portions of data are not compressed are disclosed. Segments of a buffer referred to as block groups store compressed blocks of data along with uncompressed blocks of data and hash blocks. If a block group includes a block that is a hash of another block in the block group, then the other block is considered to be compressed. If the block group does not include a block that is a hash of another block in the block group, then the blocks in the block group are uncompressed. The hash function to generate the hash is selected to prevent collisions, which occur when the data being stored in the buffer is such that it is possible for a hash block and an uncompressed block to be the same.

A method for compressing columnar data may include generating, for a data column included in a data chunk, a dictionary enumerating, in a sorted order, a first set of unique values included in the first data column. A compression technique for generated a compressed representation of the data column having a fewest quantity of bytes may be identified based at least on the dictionary. The compression technique including a dictionary compression applying the dictionary and/or another compression technique. A compressed data chunk may be generated by applying the compression technique to compress the data column included in the data chunk. The compressed data chunk may be stored at a database in a variable-size persistent page whose size is allocated based on the size of the compressed representation of the data column. Related systems and articles of manufacture are also provided.

The disclosure is directed at a method of data compression. The method includes creating a set of single composite data structures and then calculating a set of bit probabilities based on the set of single data structures. The bit probabilities are then used to create a set of intermediate buffers which are then sorted and traversed for data compression.

A data processing method includes: acquiring, by one or more processors, compressed data generated from data, wherein values of the compressed data are stored at first storage locations, values of the data are stored at second storage locations; acquiring, by the one or more processors, index data includes indices indicative of the first storage locations; acquiring, by the one or more processors, at least two packed indices from the index data, the at least two packed indices being generated from the index data; and inputting, by the one or more processors, the at least two packed indices into at least two selectors.