A system and method for compacting data that uses mismatch probability estimation to improve entropy encoding methods to account for, and efficiently handle, previously-unseen data in data to be compacted. Training data sets are analyzed to determine the frequency of occurrence of each sourceblock in the training data sets. A mismatch probability estimate is calculated comprising an estimated frequency at which any given data sourceblock received during encoding will not have a codeword in the codebook. Entropy encoding is used to generate codebooks comprising codewords for data sourceblocks based on the frequency of occurrence of each sourceblock. A “mismatch codeword” is inserted into the codebook based on the mismatch probability estimate to represent those cases when a block of data to be encoded does not have a codeword in the codebook. During encoding, if a mismatch occurs, a secondary encoding process is used to encode the mismatched sourceblock.

Methods and devices for coding point clouds using direct coding mode to code coordinates of a point within a sub-volume associated with a current node instead of a pattern of occupancy for child nodes. Eligibility for use of direct coding is based on occupancy data from another node. If eligible, then a flag is represented in the bitstream to signal whether direct coding is applied to points in the sub-volume or not.

A device configured to compress a tensor including a plurality of cells includes: a quadtree generator configured to generate a quadtree searching for a non-zero cell included in the tensor and extract at least one parameter value from the quadtree; a mode selector configured to determine a compression mode based on the at least one parameter; and a bitstream generator configured to generate a bitstream by compressing the tensor based on the compression mode.

A system and method for encoding data using a plurality of encoding libraries. Portions of the data are encoded by different encoding libraries, depending on which library provides the greatest compaction for a given portion of the data. This methodology not only provides substantial improvements in data compaction over use of a single data compaction algorithm with the highest average compaction, but provides substantial additional security in that multiple decoding libraries must be used to decode the data. In some embodiments, each portion of data may further be encoded using different sourceblock sizes, providing further security enhancements as decoding requires multiple decoding libraries and knowledge of the sourceblock size used for each portion of the data. In some embodiments, encoding libraries may be randomly or pseudo-randomly rotated to provide additional security.

A method includes receiving an input data stream at a processor, and for each byte sequence from a plurality of byte sequences of the input data stream, a hash is generated and compared to a hash table to determine whether a match exists. If a match exists, that byte sequence is incrementally expanded to include one or more additional adjacent bytes from the input data stream, to produce multiple expanded byte sequences. Each of the expanded byte sequences is compared to the hash table to identify a maximum-length matched byte sequence from a set that includes the byte sequence and the plurality of expanded byte sequences. A representation of the maximum-length matched byte sequence is stored in the memory. If a match does not exist, a representation of that byte sequence is stored as a byte sequence literal in the memory.

The disclosure relates to compressing strings by reducing the number of string characters that are stored. For example, a system may generate a first radix tree for a set of strings and a second radix tree for a reverse of each of the set of strings. The system may merge nodes of the first radix tree and/or second radix tree based on a tuning parameter. The system may identify, based on the first radix tree, beginning portions of at least two strings that match and identify, based on the second radix tree, ending portions of at least two strings that match. The system may use the matching beginning portions, the unique portions, and/or the matching ending portions to generate a pattern that matches the two or more strings. The system may store the two or more strings in association with the generated pattern without their matching beginning and/or ending portions.

A method for a compression scheme comprising encryption, comprising: receiving, as input, data comprising a plurality of data elements; constructing a Huffman tree coding representation of the input data based on a known encryption key, wherein the Huffman tree comprises nodes that are compression codes having compression code lengths corresponding to respective occurrence probabilities of the data elements, and wherein the encryption key specifies a subset of the nodes to be selected for an encryption process; selecting the subset of nodes in the Huffman tree for the encryption process, based on the encryption key; applying, to each sub-tree of the Huffman tree that is rooted at one of the nodes in the subset, a specified transformation, based on the encryption key; and generating an output data file based, at least in part, on the applying.

Methods and devices for coding point clouds using direct coding mode to code coordinates of a point within a sub-volume associated with a current node instead of a pattern of occupancy for child nodes. Eligibility for use of direct coding is based on occupancy data from another node. If eligible, then a flag is represented in the bitstream to signal whether direct coding is applied to points in the sub-volume or not.

A computer memory compression method involves analyzing (1210) computer memory content with respect to occurrence of duplicate memory objects as well as value redundancy of data values in unique memory objects. The computer memory content is encoded (1220) by eliminating the duplicate memory objects and compressing each remaining unique memory object by exploiting data value locality of the data values thereof. Metadata (500) is provided (1230) to represent the memory objects of the encoded computer memory content. The metadata reflects eliminated duplicate memory objects, remaining unique memory objects as well as a type of compression used for compressing each remaining unique memory object. A memory object in the encoded computer memory content is located (1240) using the metadata (500).

A literal element that has a plurality of bits is received. The plurality of bits in the literal element is divided into a first sub-literal comprising a first set of bits and a second sub-literal comprising a second set of bits. The first sub-literal is encoded using a first Huffman code tree to obtain a first sub-literal codeword; the second sub-literal is encoded using a second Huffman code tree to obtain a second sub-literal codeword. Encoded data that includes information associated with the first Huffman code tree, information associated with the second Huffman code tree, the first sub-literal codeword, and the second sub-literal codeword is output.