A system and method for compacting data that uses mismatch probability estimation to improve entropy encoding methods to account for, and efficiently handle, previously-unseen data in data to be compacted. Training data sets are analyzed to determine the frequency of occurrence of each sourceblock in the training data sets. A mismatch probability estimate is calculated comprising an estimated frequency at which any given data sourceblock received during encoding will not have a codeword in the codebook. Entropy encoding is used to generate codebooks comprising codewords for data sourceblocks based on the frequency of occurrence of each sourceblock. A “mismatch codeword” is inserted into the codebook based on the mismatch probability estimate to represent those cases when a block of data to be encoded does not have a codeword in the codebook. During encoding, if a mismatch occurs, a secondary encoding process is used to encode the mismatched sourceblock.

A method for transmitting data collected by at least one sensor to a monitoring device. The method includes, upon acquisition of a new piece of data by the at least one sensor, acts of calculating a deviation indicator indicating a deviation between the value of the new piece of data and a value predicted for this piece of data by a prediction model representative of previously acquired data, and transmitting the new piece of data to the monitoring device when the deviation indicator is higher than a threshold. Also provided are a monitoring method on a monitoring device, a terminal implementing the transmission method and a server implementing the monitoring method.

A method is performed by a network element (NE) in a network implementing an Interior Gateway Protocol (IGP). The method comprises generating a message comprising a header and data, wherein the header comprises a length of the data prior to compressing the data, a length of the data after compressing the data, and a compression identifier, compressing the data based on a compression scheme identified by the compression identifier to obtain compressed data, and forwarding a compressed message comprising the header and the compressed data to another NE in the network.

A system and method for file type identification involving extraction of a file-print of a file, the file-print being a unique or practically-unique representation of statistical characteristics associated with the distribution of bits in the binary contents of the file, similar to a fingerprint. The file-print is then passed to a machine learning algorithm that has been trained to recognize file types from their file-prints. The machine learning algorithm returns a predicted file type and, in some cases, a probability of correctness of the prediction. The file may then be encoded using an encoding algorithm chosen based on the predicted file type.

A method, computer program product, and system includes a processor obtaining data including values and generating a value conversion dictionary by applying a parse tree based compression algorithm to the data, where the value conversion dictionary includes dictionary entries that represent the values. The processor obtains a distribution of the values and estimates a likelihood for each based on the distribution. The processor generates a code word to represent each value, a size of each code word is inversely proportional to the likelihood of the word. The processor assigns a rank to each code word, the rank for each represents the likelihood of the value represented by the code word; and based on the rank associated with each code word, the processor reorders each dictionary entry in the value conversion dictionary to associate each dictionary entry with an equivalent rank, the reordered value conversion dictionary comprises an architected dictionary.

A method, article comprising machine-readable instructions and apparatus that processes data systems for encoding, decoding, pattern recognition/matching and data generation is disclosed. State subsets of a data system are identified for the efficient processing of data based, at least in part, on the data system's systemic characteristics.

Methods and devices for lossy encoding of point clouds. Rate-distortion optimization is used in coding an occupancy pattern for a sub-volume to determine whether to invert any of the bits of the occupancy pattern. The assessment may be a greedy evaluation of whether to invert bits in the coding order. Inverting a bit of the occupancy pattern amounts to adding or removing a point from the point cloud. A distortion metric may measure distance between the point added or removed and its nearest neighbouring point.

A method and device for compressing and decompressing data information, a drive compensation method and device, and a display device. The method for compressing data information includes: acquiring data information corresponding to a sub pixel unit; establishing a distribution function model according to the data information; obtaining a valid option value section according to the distribution function model and a valid threshold value; and dividing the valid option value section into N compression sections, and compressing data information corresponding to each of the compression sections to M times of data information corresponding to all the sub pixel units according to a storage length P of the data information corresponding to the sub pixel unit to obtain N compressed data information blocks.

A compression scheme can be selected for an input data stream based on characteristics of the input data stream. For example, when the input data stream is searched for pattern matches, input stream characteristics used to select a compression scheme can include one or more of: type and size of an input stream, a length of a pattern, a distance from a start of where the pattern is to be inserted to the beginning of where the pattern occurred previously, a gap between two pattern matches (including different or same patterns), standard deviation of a length of a pattern, standard deviation of a distance from a start of where the pattern is to be inserted to the beginning of where the pattern occurred previously, or standard deviation of a gap between two pattern matches. Criteria can be established whereby one or more characteristics are used to select a particular encoding scheme.

A method performed in real-time includes receiving and storing time-based data over a specific time period and dividing the specific time period into a plurality of time windows. The method further includes determining that data associated with two or more proximate time windows are within a predetermined variance of one another and responsive to the determination: generating a mathematical function representative of the data associated with the two or more proximate time windows, deleting the data associated with the two or more proximate time windows, and generating a representation of the deleted data from the mathematical function. In certain embodiments, the data comprises empirical network telemetry data.