A method for a compression scheme comprising encryption, comprising: receiving, as input, data comprising a plurality of data elements; constructing a Huffman tree coding representation of the input data based on a known encryption key, wherein the Huffman tree comprises nodes that are compression codes having compression code lengths corresponding to respective occurrence probabilities of the data elements, and wherein the encryption key specifies a subset of the nodes to be selected for an encryption process; selecting the subset of nodes in the Huffman tree for the encryption process, based on the encryption key; applying, to each sub-tree of the Huffman tree that is rooted at one of the nodes in the subset, a specified transformation, based on the encryption key; and generating an output data file based, at least in part, on the applying.

A method for a compression scheme comprising encryption, comprising: receiving, as input, data comprising a plurality of data elements; constructing a Huffman tree coding representation of the input data based on a known encryption key, wherein the Huffman tree comprises nodes that are compression codes having compression code lengths corresponding to respective occurrence probabilities of the data elements, and wherein the encryption key specifies a subset of the nodes to be selected for an encryption process; selecting the subset of nodes in the Huffman tree for the encryption process, based on the encryption key; applying, to each sub-tree of the Huffman tree that is rooted at one of the nodes in the subset, a specified transformation, based on the encryption key; and generating an output data file based, at least in part, on the applying.

According to one embodiment, a data decompression device includes: a detection circuit configured to detect a boundary between a header and a payload in a compressed stream, based on boundary information in the header; a separation circuit configured to separate the header and the payload; a first decompression circuit configured to decompress a compressed coding table in the header; and a second decompression circuit configured to decompress the payload, based on an output of the first decompression circuit.

A method for compressing columnar data may include generating, for a data column included in a data chunk, a dictionary enumerating, in a sorted order, a first set of unique values included in the first data column. A compression technique for generated a compressed representation of the data column having a fewest quantity of bytes may be identified based at least on the dictionary. The compression technique including a dictionary compression applying the dictionary and/or another compression technique. A compressed data chunk may be generated by applying the compression technique to compress the data column included in the data chunk. The compressed data chunk may be stored at a database in a variable-size persistent page whose size is allocated based on the size of the compressed representation of the data column. Related systems and articles of manufacture are also provided.

The present disclosure provides a data encoding method, a decoding method, a related device, and a storage medium. The data encoding method first passes a first bit stream of an original encoded data through a logical operation to obtain a second bit stream. Then, through signal determination, negating processing, and insertion of corresponding flag bit, encoded data having a certain jump amplitude is obtained. A problem that signal is prone to error in transmission process is solved, reliability of coding is improved, and signal transmission is facilitated.

Embodiments of the subject matter described herein relate to a wireless programmable media processing system. In the media processing system, a processing unit in a computing device generates a frame to be displayed based on a graphics content for an application running on the computing device. The frame to be displayed is then divided into a plurality of block groups which are compressed. The plurality of compressed block groups are sent to a graphics display device over a wireless link. In this manner, both the generation and the compression of the frame to be displayed may be completed at the same processing unit in the computing device, which avoids data copying and simplifies processing operations. Thereby, the data processing speed and efficiency is improved significantly.

Updating cumulative distribution functions (CDFs) during arithmetic encoding can be a challenge because the final element of the CDF should remain fixed during the update calculations. If the probabilities were floating-point numbers, this would not be too much of a challenge; nevertheless, the probabilities and hence the CDFs are represented as integers to take advantage of infinite-precision arithmetic. Some of these difficulties may be alleviated by introducing a “mixing” CDF along with the active CDF being updated; the mixing CDF provides nonlocal context for updating the CDF due to the introduction of a particular symbol in the encoding. Improved techniques of performing arithmetic encoding include updating the CDF using two, one-dimensional mixing CDF arrays: a symbol-dependent array and a symbol-dependent array. The symbol-dependent array is a sub array of a larger, fixed array such that the sub array selected depends on the symbol being used.

Methods, apparatus, systems, and software for implementing self-checking compression. A byte stream is encoded to generate tokens and selected tokens are encoded with hidden parity information in a compressed byte stream that may be stored for later streaming or streamed to a receiver. As the compressed byte stream is received, it is decompressed, with the hidden parity information being decoded and used to detect for errors in the decompressed data, enabling errors to be detected on-the-fly rather than waiting to perform a checksum over an entire received file. In one embodiment the byte stream is encoded using a Lempel-Ziv 77 (LZ77)-based encoding process to generate a sequence of tokens including literals and references, with all or selected references encoded with hidden parity information in a compressed byte stream having a standard format such as DEFLATE or Zstandard. The hidden parity information is encoded such that the compressed byte stream may be decompressed without parity checks using standard DEFLATE or Zstandard decompression schemes. Dictionary coders such as LZ78 and LZW may also be used.

Methods, apparatus, systems, and software for implementing self-checking compression. A byte stream is encoded to generate tokens and selected tokens are encoded with hidden parity information in a compressed byte stream that may be stored for later streaming or streamed to a receiver. As the compressed byte stream is received, it is decompressed, with the hidden parity information being decoded and used to detect for errors in the decompressed data, enabling errors to be detected on-the-fly rather than waiting to perform a checksum over an entire received file. In one embodiment the byte stream is encoded using a Lempel-Ziv 77 (LZ77)-based encoding process to generate a sequence of tokens including literals and references, with all or selected references encoded with hidden parity information in a compressed byte stream having a standard format such as DEFLATE or Zstandard. The hidden parity information is encoded such that the compressed byte stream may be decompressed without parity checks using standard DEFLATE or Zstandard decompression schemes. Dictionary coders such as LZ78 and LZW may also be used.

According to one embodiment, a compression circuit generates substrings from input data for (3+M) cycles, the input data being N bytes per cycle, a byte length of each substring being greater than or equal to (N×(1+M)+1); obtains a set of matches, each of the matches including at least one past input data which input past and corresponds to at least a part of each of the substrings; selects a subset of matches from the set of matches including the input data of one cycle; and outputs the subset of matches. M is zero or a natural number. N is a positive integer which is two or more.