A computerized method of compressing symbolic information organized into a plurality of documents, each document having a plurality of symbols, includes: (i) automatically identifying a plurality of sequential and non-sequential symbol pairs in an input document; (ii) counting the number of appearances of each unique symbol pair; and (iii) producing a compressed document that includes a replacement symbol at each position associated with one of the plurality of symbol pairs, at least one of which corresponds to a non-sequential symbol pair. For each non-sequential pair the compressed document includes corresponding indicia indicating a distance between locations of the non-sequential symbols of the pair in the input document. In some instances the plurality of symbol pairs includes only those pairs of non-sequential symbols for which the distance between locations of the non-sequential symbols of the pair in the input document is less than a numeric distance cap.

A data processing platform, method, and program product perform compression and decompression of a set of data items. Suffix data and a prefix are selected for each respective data item in the set of data items based on data content of the respective data item. The set of data items is sorted based on the prefixes. The prefixes are encoded by querying multiple encoding tables to create a code word containing compressed information representing values of all prefixes for the set of data items. The code word and suffix data for each of the data items are stored in memory. The code word is decompressed to recover the prefixes. The recovered prefixes are paired with their respective suffix data.

Methods, systems, and apparatus, including computer-readable storage media for hardware compression and decompression. A system can include a decompressor device coupled to a memory device and a processor. The decompressor device can be configured to receive, from the memory device, compressed data that has been compressed using an entropy encoding, process the compressed data using the entropy encoding to generate uncompressed data, and send the uncompressed data to the processor. The system can also include a compressor device configured to generate, from uncompressed data, a probability distribution of codewords, generate a code table from the probability distribution, and compress incoming data using the generated code table.

A system collects statistical data for a data page, divides the data page into parts, analyzes the data page and the statistical data, based on compression efficiency of one or more compression methods for each part of each page, to determine a compression method for each part of page, and compresses, based on the analyzing, the parts of the data page.

A buffer control method, a UE, and a non-volatile computer-readable storage medium are provided. The buffer control method for the UE includes: transmitting, by the UE, first check information to a network side device; receiving, by the UE, first buffer resetting information from the network side device; releasing or emptying a current compression buffer of the UE in accordance with content in the first buffer resetting information; and when the UE receives a dictionary activating or enabling indication: storing, by the UE, a dictionary into the compression buffer; compressing, by the UE through UDC, an uncompressed data packet to obtain a compressed data packet; and transmitting, by the UE, the compressed data packet, wherein the compression buffer is continuously updated by the UE in accordance with the uncompressed data packet.

Embodiments of this application disclose example data compression methods and example base stations. One example method includes obtaining, by a base station, identification information of at least one of a data compression device or a data decompression device. A dictionary can then be generated based on the identification information. The dictionary can then be sent to the data compression device and the data decompression device to enable the data compression device and the data decompression device to perform data compression and data transmission based on the dictionary.

Methods, systems, and apparatus, including computer-readable storage media for hardware compression and decompression. A system can include a decompressor device coupled to a memory device and a processor. The decompressor device can be configured to receive, from the memory device, compressed data that has been compressed using an entropy encoding, process the compressed data using the entropy encoding to generate uncompressed data, and send the uncompressed data to the processor. The system can also include a compressor device configured to generate, from uncompressed data, a probability distribution of codewords, generate a code table from the probability distribution, and compress incoming data using the generated code table.

A system collects statistical data for a data page, divides the data page into parts, analyzes the data page and the statistical data, based on compression efficiency of one or more compression methods for each part of each page, to determine a compression method for each part of page, and compresses, based on the analyzing, the parts of the data page.

A pre-shared compression dictionary is received. The pre-shared compression dictionary was generated based on an analysis of sample data for use in compression of other data. A compressed version of a batch of machine-generated data is received. The batch of machine-generated data has been compressed at least in part using the pre-shared compression dictionary and a batch-specific compression dictionary. The received compressed batch is uncompressed using the batch-specific compression dictionary to determine an intermediate version. The intermediate version is uncompressed using the pre-shared compression dictionary to determine an uncompressed version of the batch of machine-generated data.

According to one embodiment, a code table generation device includes a table generation unit, a merge unit and a tree generation unit. The table generation unit generates a frequency table including symbols and frequencies of occurrence respectively associated with the symbols, based on a frequency of occurrence for each symbol of input symbols. The merge unit acquires top K symbols in descending order of the frequencies of occurrence and remaining symbols from the symbols, divides the remaining symbols into one or more symbol sets, and determines a frequency of occurrence associated with a root node of each of subtrees correspond to the respective symbol sets. The tree generation unit generates a Huffman tree using the K symbols and the root node of each of the subtrees.