A storage infrastructure, method and encoder device for implementing low complexity encoding, The described encoder includes: a preprocessing system that assigns a code length to each unique symbol based on the frequency without performing a sort operation and determines maximum and minimum occurrence frequencies of symbols of each given code length, and the maximum and minimum code length among all the symbols; and a post processing system that cycles through each code length, determines if a maximum occurrence frequency of a current code length, associated with a first symbol, is greater than a minimum occurrence frequency of an adjacent code length, associated with a second symbol, and if greater, swaps code lengths of the first and second symbols.

An example method of compressing a data set includes determining whether individual values from a data set correspond to a first category or a second category of values. Based on one of the values corresponding to the first category, the value is added to a compressed data set. Based on one of the values corresponding to the second category, the value is excluded from the compressed data set, and a statistical distribution of values of the second category is updated based on the value. During a first phase, the determining is performed for a plurality of values from a first portion of the data set based on comparison of the values to criteria. During a second phase, the determining is performed for a plurality of values from a second portion of the data set based on the statistical distribution.

Embodiments of the present disclosure include techniques for compressing data using a tree encoded bit mask that may result in higher compression ratios. In one embodiment, an input vector having a plurality of values is received by a first plurality of switch circuits. Selection of the input values is controlled by sets of bits from the bit mask. The sets of bits specify locations of portions of the input vector where particular value of interest reside. The switch circuits output multiple values of the input vector, which include the particular value of interest. A second stage of switch circuits is controlled by logic circuit that detects values on the outputs of the first stage of switch circuits and outputs the values of interest. In some embodiments, the values of interest may be non-zero values of a sparse input vector, and the switch circuits may be multiplexers.

A system and method for compacting data that uses mismatch probability estimation to improve entropy encoding methods to account for, and efficiently handle, previously-unseen data in data to be compacted. Training data sets are analyzed to determine the frequency of occurrence of each sourceblock in the training data sets. A mismatch probability estimate is calculated comprising an estimated frequency at which any given data sourceblock received during encoding will not have a codeword in the codebook. Entropy encoding is used to generate codebooks comprising codewords for data sourceblocks based on the frequency of occurrence of each sourceblock. A mismatch codeword is inserted into the codebook based on the mismatch probability estimate to represent those cases when a block of data to be encoded does not have a codeword in the codebook. During encoding, if a mismatch occurs, a secondary encoding process is used to encode the mismatched sourceblock.

Methods and devices for encoding a point cloud. A bit sequence signalling an occupancy pattern for sub-volumes of a volume is coded using binary entropy coding. For a given bit in the bit sequence, a context may be based on a sub-volume neighbour configuration for the sub-volume corresponding to that bit. The sub-volume neighbour configuration depends on an occupancy pattern of a group of sub-volumes of neighbouring volumes to the volume, the group of sub-volumes neighbouring the sub-volume corresponding to the given bit. The context may be further based on a partial sequence of previously-coded bits of the bit sequence.

An encoder includes a processor, a buffer, and a memory. The memory includes code as instructions that cause the processor to perform a number of steps. The steps include quantizing geometric data associated with a geometric construct, partitioning the geometric construct, determining a number of points in the partition, generating a deviation value based on the number of points in the partition, storing the deviation value in the buffer, and entropy encoding the deviation value.

The present invention relates to a method and apparatus for compression and decompression of a numerical file. The compression method comprises: read a numerical file, convert each numerical element into a 32-bit floating point number; combine all the numbers to form a binary numerical file; group the binary numerical file into a n-bit sequence pattern; generate a Huffman tree based on frequency of occurrences of a plurality of unique bit patterns present in the binary numerical file; generate codewords and replace unique bit patterns with codewords so that a compressed binary numerical file is generated. A method for decompression comprises: read a compressed binary numerical file having codewords; fetch a part or entire compressed binary numerical file using an address dictionary; replace the codewords with unique bit patterns using a Huffman tree such that a decompressed binary numerical file being generated.

A system and method for high-speed transfer of small data sets, that provides near-instantaneous bit-level lossless compression, that is ideal for communications environments that cannot tolerate even small amounts of data corruption, have very low latency tolerance, where data has a low entropy rate, and where every bit costs the user bandwidth, power, or time so that deflation is worthwhile. Where some loss of data can be tolerated, the system and method can be configured for use as lossy compression.

A system and method for codebook-based data encoding. Portions of the data are encoded by different encoding libraries, depending on which library provides the greatest compaction for a given portion of the data. This methodology not only provides substantial improvements in data compaction over use of a single data compaction algorithm with the highest average compaction, but provides substantial additional security in that multiple decoding libraries must be used to decode the data. In some embodiments, each portion of data may further be encoded using different sourceblock sizes, providing further security enhancements as decoding requires multiple decoding libraries and knowledge of the sourceblock size used for each portion of the data. In some embodiments, encoding libraries may be randomly or pseudo-randomly rotated to provide additional security.

A system and method for encoding personal health monitor data using a plurality of encoding libraries. Portions of the data are encoded by different encoding libraries, depending on which library provides the greatest compaction or on some other criteria for a given portion of the data. This methodology not only provides substantial improvements in data compaction over use of a single data compaction algorithm with the highest average compaction, but provides substantial additional security in that multiple decoding libraries must be used to decode the data. In some embodiments, each portion of data may further be encoded using different sourceblock sizes, providing further security enhancements as decoding requires multiple decoding libraries and knowledge of the sourceblock size used for each portion of the data. In some embodiments, encoding libraries may be randomly or pseudo-randomly rotated to provide additional security.