An accelerator or system including an accelerator can include an input interface to receive input data to be compressed and user application parameters for invocation of compression. The accelerator can include circuitry to identify a compression algorithm from configuration data provided with the input data. The user application parameters may not include parameters specifying entropy thresholds for compression of the input data. The circuitry can generate headers specific to the compression algorithm. The circuitry can generate uncompressed data blocks comprising blocks of the input data and corresponding headers. The circuitry can determine whether to provide the uncompressed data blocks or compressed data blocks based at least in part on entropy of the input data. Other methods, systems, and apparatuses are described.

A system and method for encoding data utilizing mismatch probability estimates. A training data set can be statistically analyzed to calculate a mismatch probability estimate which is the estimated frequency at which a data packet received during system runtime is not part of (i.e., a mismatch) the training data set. A plurality of tokens may be created, based on the mismatch probability estimate, to represent potential mismatched data that may be encountered during runtime, and an entropy encoder may generate codewords for the tokens using the mismatch probability estimate. An opcode, indicating a mismatch, may be generated and appended to the generated codewords to form a mismatch codeword. During runtime when a mismatch occurs, the system can retrieve a mismatch codeword and assign it to the mismatched data, making the encoder system robust against previously unencountered data.

A method of encoding image data, including: frequency-transforming input image data to generate an array of frequency-transformed input image coefficients by a matrix-multiplication process, according to a maximum dynamic range of the transformed data and using transform matrices having a data precision; and selecting the maximum dynamic range and/or the data precision of the transform matrices according to the bit depth of the input image data.

A system and method for encoding data using a plurality of encoding libraries. Portions of the data are encoded by different encoding libraries, depending on which library provides the greatest compaction for a given portion of the data. This methodology not only provides substantial improvements in data compaction over use of a single data compaction algorithm with the highest average compaction, but provides substantial additional security in that multiple decoding libraries must be used to decode the data. In some embodiments, each portion of data may further be encoded using different sourceblock sizes, providing further security enhancements as decoding requires multiple decoding libraries and knowledge of the sourceblock size used for each portion of the data. In some embodiments, encoding libraries may be randomly or pseudo-randomly rotated to provide additional security.

A system and method for highly efficient encoding of data that includes extended functionality for asymmetric encoding/decoding and network policy enforcement. In the case of asymmetric encoding/decoding the original data is encoded by an encoder according to a codebook and sent to a decoder, but the output of the decoder depends on data manipulation rules applied at the decoding stage to transform the decoded data, into a different data set from the original data. In the case of network pokey enforcement, a behavior appendix into the codebook, such that the encoder and/or decoder at each node of the network comply with network behavioral rules, limits, and policies during encoding and decoding.

According to one embodiment, a compression device includes a coding information generation unit. The unit determines code lengths that are respectively associated with a plurality of symbols, based on a frequency of occurrence of each of the plurality of symbols. When the plurality of symbols include one or more first symbols that are respectively associated with one or more first code lengths exceeding an upper limit, the unit changes the first code lengths to the upper limit, selects, from one or more second symbols of the plurality of symbols that are respectively associated with one or more second code lengths shorter than the upper limit, at least one symbol in descending associated code length order, changes at least one code length associated with the symbol to the upper limit.

A communication node processing a communication signal in a distributed antenna system includes a data appearance frequency monitor configured to receive a communication signal including a sign bit string, a count leading zero bit string, and an additional data bit string, and to monitor data appearance frequency in the count leading zero bit string included in the received communication signal, a Huffman encoder configured to encode the count leading zero bit string into a corresponding codeword according to a Huffman encoding algorithm based on the data appearance frequency in the count leading zero bit string and an additional bit allocator configured to allocate additional bits to the additional data bit string when the number of bits in the count leading zero bit string decreases during the encoding process.

Methods and systems for processing telemetry data that contains multiple data types is disclosed. Optimum multimodal encoding approaches can be used which can achieve data-specific compression performance for heterogeneous datasets by distinguishing data types and their characteristics at real-time and applying most effective compression method to a given data type. Using an optimum encoding diagram for heterogeneous data, a data classification algorithm classifies input data blocks into predefined categories, such as Unicode, telemetry, RCS and IR for telemetry datasets, and a class of unknown which includes non-studied data types, and then assigns them into corresponding compression models.

One embodiment provides a computer implemented method of data compression using a hardware accelerator. A first thread pool for compression jobs, and a first polling thread is allocated for polling the status of a hardware accelerator. A compression thread is retrieved from the first thread pool in response to a compression request from a file system. Multiple source data buffers from the file system are aggregated into a compression unit, and a scatter gather list and destination buffer are submitted to the hardware accelerator. A checksum of result data is calculated from the destination buffer. A zlib header is added to the result data, and the checksum is added as a zlib footer to the result data.

An instruction to perform a function of a plurality of functions is obtained. The instruction is a single architected instruction of an instruction set architecture that complies to an industry standard for compression. The instruction is executed, and the executing includes performing the function specified by the instruction. The performing includes, based on the function being a compression function or a decompression function, transforming state of input data between an uncompressed form of the input data and a compressed form of the input data to provide a transformed state of data accessing. During performing the function, history relating to the function is accessed. The history is to be used in transforming the state of input data between the uncompressed form and the compressed form.