An encoder includes processing circuitry, a block memory, and a frame memory. The processing circuitry defines at least one parameter for each of plural types of segment_ids, splits an image into blocks, assigns, to each of the blocks, segment_id according to a type of the block, among the plural types of segment_ids, and sequentially encodes the blocks. In encoding the blocks, the processing circuitry identifies segment_id of a current block to be encoded, and encodes the current block using the at least one parameter defined for identified segment_id. The at least one parameter includes seg_context_idx for identifying probability information associated with context used in context-based adaptive binary arithmetic coding (CABAC).

A framework and the associated method, schema and design for processing digital data, whether random or not, through encoding and decoding losslessly and correctly for purposes including the purposes of encryption/decryption or compression/decompression or both. There is no assumption of the digital information to be processed before processing. An universal coder is invented and now pigeonhole meets blackhole.

Lossless compression of a content item using a neural network trained on content item cohorts. A computing system includes a neural network that is used to train a plurality of symbol prediction models. Each symbol prediction model is trained based on a corresponding cohort of content items. A particular symbol prediction model of the models trained is selected based on an intrinsic characteristic of a particular content item to be losslessly compressed such as, for example, the type or file extension of the content item. The content item is then losslessly compressed based on a set of symbol predictions fed to an arithmetic coder that are generated using the particular symbol prediction model selected.

A system and method for extracting data from a compressed and encrypted data stream employing lossless compression and a codeword-to-byte mapping enables selective random extraction from compressed and encrypted data. Initially, data is compressed to reduce its size without information loss. A codeword-to-byte mapping table is created, associating codewords with specific positions in the compressed data. When a request for a specific byte range is received, the system uses the mapping to identify the codewords linked to that range within the compressed data. The system then selectively decompresses the identified codewords and subsequently decrypts the decompressed data. This process ensures that only the relevant data is processed, eliminating the need to decompress and decrypt the entire dataset. This approach provides efficient and secure random access to specific data segments within the compressed and encrypted dataset, making it ideal for scenarios requiring selective data retrieval while maintaining data integrity and confidentiality.

According to an embodiment of the present disclosure, an electronic device may comprise a memory and a processor configured to produce compressed data by compressing data including a first block and a second block stored in the memory, wherein the processor may be configured to include a first replacement data table corresponding to first sub-data in the compressed data, the first sub-data included in the first block, and the first replacement data table produced based on, at least, rankings of first frequencies for the first sub-data, and include information for reference to the first replacement data table, corresponding to the second block, when second sub-data included in the second block and rankings of second frequencies for the second sub-data meet a designated condition with respect to the first sub-data included in the first block and the rankings of the first frequencies. Other embodiments are also possible.

A FRAMEWORK and the associated method, schema and design for processing digital data, whether random or not, through encoding and decoding losslessly and correctly for purposes including the purposes of encryption/decryption or compression/decompression or both. There is no assumption of the digital information to be processed before processing. A Universal Coder is invented and now Pigeonhole meets Blackhole.

A technique for selecting context models (CMs) for a CM ensemble (CME) in a context mixing compressor includes measuring compression ratios (CRs) of the compressor on a dataset for each CM included in a base set of CMs. A first CM that has a maximum CR for the dataset is added to the CME. In response to a desired number of the CMs not being in the CME, subsequent CRs for the compressor are measured on the dataset for each of the CMs in the base set of CMs that are not in the CME in conjunction with one or more CMs in the CME. In response to a desired number of the CMs not being in the CME, subsequent CMs that in conjunction with the one or more CMs in the CME result in a maximum subsequent CR for the dataset are added to the CME.

Methods for estimating cost savings in a storage system using an external host system. One method includes accessing over a communication network data from a unit of storage of a data storage system, wherein each of the blocks of data is uncompressed. A plurality of blocks is parsed from the data. A plurality of fingerprints is generated from the blocks using a hash algorithm. A deduplication ratio is estimated for the plurality of blocks stored in the unit of storage using a hyperloglog algorithm and a first plurality of buckets compartmentalizing the plurality of blocks, wherein the first plurality of buckets is defined by precision bits of the plurality of fingerprints. An effective compression ratio is estimated for the plurality of blocks stored in the unit of storage using the hyperloglog algorithm and a second plurality of buckets compartmentalizing the plurality of blocks, wherein the second plurality of buckets is defined by ranges of compression ratios.

An video decoding apparatus including a parser which obtains bit strings corresponding to current transformation coefficient level information by arithmetic decoding a bitstream based on a context model; a parameter determiner which determines a current binarization parameter by updating or maintaining a previous binarization parameter based on a comparison of a threshold and a size of a previous transformation coefficient; a syntax element restorer which obtains the current transformation coefficient level information by performing de-binarization of the bit strings using the determined current binarization parameter and generates a size of a current transformation coefficient using the current transformation coefficient level information, wherein the current binarization parameter has a value equal to or smaller than a predetermined value.

An video decoding apparatus including a parser which obtains bit strings corresponding to current transformation coefficient level information by arithmetic decoding a bitstream based on a context model; a parameter determiner which determines a current binarization parameter by updating or maintaining a previous binarization parameter based on a comparison of a threshold and a size of a previous transformation coefficient; a syntax element restorer which obtains the current transformation coefficient level information by performing de-binarization of the bit strings using the determined current binarization parameter and generates a size of a current transformation coefficient using the current transformation coefficient level information, wherein the current binarization parameter has a value equal to or smaller than a predetermined value.