A method, article comprising machine-readable instructions and apparatus that processes data systems for encoding, decoding, pattern recognition/matching and data generation is disclosed. State subsets of a data system are identified for the efficient processing of data based, at least in part, on the data system's systemic characteristics.

A computer generates a plurality of pieces of syntax information respectively corresponding to a plurality of words in a compression target document by analyzing relationships between the plurality of words. Next, the computer assigns a plurality of compression codes to the plurality of words and to the plurality of pieces of syntax information. Then, the computer outputs the plurality of compression codes with an arrangement of a specific order.

A method for encoding text includes grouping text as a sequence of bytes, the text comprising a string of characters, each byte corresponding to a character in the text. For each byte of the sequence of bytes: (a) each bit is processed from most significant bit to least significant bit to generate a context; and (b) a subsequent bit is predicted, using a prediction model, based on the context generated based on previously processed bits, prediction of the prediction model being a combination of predictions of a plurality of sub-models. An encoded bitstream is output based on the predicted bits. The encoded bitstream includes encoded data corresponding to the text.

A non-transitory machine-readable medium of an electronic device storing computer-executable instructions for updating a probability estimator value during entropy decoding for a bitstream representing a set of video pictures is provided. The computer-executable instructions, when executed by a processor of the electronic device, update a probability estimator value by (i) performing an initial right bit-shifting operation on the probability estimator value to reduce a length, in bits, of the probability estimator value, (ii) multiplying the right bit-shifted probability estimator value by a range value representing an interval, (iii) performing another right bit-shifting operation on a result of the multiplication generated by multiplying the right bit-shifted probability estimator value by the range value, and (iv) adding a constant value to a result of the other right bit-shifting operation, wherein the probability estimator value is associated with a probability of a bin having a particular value.

A method and device for compressing and decompressing data information, a drive compensation method and device, and a display device. The method for compressing data information includes: acquiring data information corresponding to a sub pixel unit; establishing a distribution function model according to the data information; obtaining a valid option value section according to the distribution function model and a valid threshold value; and dividing the valid option value section into N compression sections, and compressing data information corresponding to each of the compression sections to M times of data information corresponding to all the sub pixel units according to a storage length P of the data information corresponding to the sub pixel unit to obtain N compressed data information blocks.

A method, computer program product, and system includes a processor obtaining data including values and generating a value conversion dictionary by applying a parse tree based compression algorithm to the data, where the value conversion dictionary includes dictionary entries that represent the values. The processor obtains a distribution of the values and estimates a likelihood for each based on the distribution. The processor generates a code word to represent each value, a size of each code word is inversely proportional to the likelihood of the word. The processor assigns a rank to each code word, the rank for each represents the likelihood of the value represented by the code word; and based on the rank associated with each code word, the processor reorders each dictionary entry in the value conversion dictionary to associate each dictionary entry with an equivalent rank, the reordered value conversion dictionary comprises an architected dictionary.

A method, computer program product, and system includes a processor obtaining data including values and generating a value conversion dictionary by applying a parse tree based compression algorithm to the data, where the value conversion dictionary includes dictionary entries that represent the values. The processor obtains a distribution of the values and estimates a likelihood for each based on the distribution. The processor generates a code word to represent each value, a size of each code word is inversely proportional to the likelihood of the word. The processor assigns a rank to each code word, the rank for each represents the likelihood of the value represented by the code word; and based on the rank associated with each code word, the processor reorders each dictionary entry in the value conversion dictionary to associate each dictionary entry with an equivalent rank, the reordered value conversion dictionary comprises an architected dictionary.

A runtime bit-plane data-format optimizer for a processing element includes a sparsity-detector and a compression-converter. The sparsity-detector selects a bit-plane compression-conversion format during a runtime of the processing element using a performance model that is based on a first sparsity pattern of first bit-plane data stored in a memory exterior to the processing element and a second sparsity pattern of second bit-plane data that is to be stored in a memory within the processing element. The second sparsity pattern is based on a runtime configuration of the processing element. The first bit-plane data is stored using a first bit-plane compression format and the bit-plane second data is to be stored using a second bit-plane compression format. The compression-conversion circuit converts the first bit-plane compression format of the first data to be the second bit-plane compression format of the second data.

A system and method for lossy precompression for data compaction using automated model monitoring and training, wherein statistical analyses of test datasets are used to determine if the probability distribution of two datasets are within a pre-determined range, and responsive to that determination new encoding and decoding algorithms may be retrained in order to produce new data sourceblocks, and pre-compression of data prior to processing and statistical analysis allows for the compaction of already compressed data into highly dense formats. The new data sourceblocks may then be processed and assigned new codewords which are compiled into an updated codebook which may be distributed back to encoding and decoding systems and devices.

A runtime data-format optimizer for a processing element includes a sparsity-detector and a compression-converter. The sparsity-detector selects a first compression-conversion format during a runtime of the processing element based on a performance model that is based on a first sparsity pattern of first data stored in a first memory that is exterior to the processing element and a second sparsity pattern of second data that is to be stored in a second memory within the processing element. The second sparsity pattern is based on a runtime configuration of the processing element. The first data is stored in the first memory using a first compression format and the second data is to be stored in the second memory using a second compression format. The compression-conversion circuit converts the first compression format of the first data to be the second compression format of the second data based on the first compression-conversion format.