Methods and systems for encoding of integers are discussed. For example, various methods and systems may utilize Huffman coding, Tunstall coding, Arithmetic Coding, LZ77 coding, LZ78 coding, LW coding, or Shannon Fano Elias coding to encode the integers.

A transcoder is disclosed. The transcoder may comprise a buffer to store input encoded data. An index mapper may map an input dictionary to an output dictionary. A current encode buffer may store a modified current encoded data, which may be responsive to the input encoded data, the input dictionary, and the map from the input dictionary to the output dictionary. A previous encode buffer may store a modified previous encoded data, which may be responsive to the input encoded data, the input dictionary, and the map from the input dictionary to the output dictionary. A rule evaluator may generate an output stream responsive to the modified current encoded data in the current encode buffer, the modified previous encoded data in the previous encode buffer, and transcoding rules.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for compressing and decompressing data. In one aspect, an encoder neural network processes data to generate an output including a representation of the data as an ordered collection of code symbols. The ordered collection of code symbols is entropy encoded using one or more code symbol probability distributions. A compressed representation of the data is determined based on the entropy encoded representation of the collection of code symbols and data indicating the code symbol probability distributions used to entropy encode the collection of code symbols. In another aspect, a compressed representation of the data is decoded to determine the collection of code symbols representing the data. A reconstruction of the data is determined by processing the collection of code symbols by a decoder neural network.

A storage device is disclosed. The storage device may comprise storage for input encoded data. A controller may process read requests and write requests from a host computer on the data in the storage. An in-storage compute controller may receive a predicate from the host computer to be applied to the input encoded data. A transcoder may include an index mapper to map an input dictionary to an output dictionary, with one entry in the input dictionary mapped to an entry in the output dictionary, and another entry in the input dictionary mapped to a “don't care” entry in the output dictionary.

Methods and devices for encoding or decoding messages, each message including a list of information items. The encoding method comprises determining a first list of indexes associated with information items that are already indexed in a local indexing table and a second list of literal values of other information items not yet indexed in said indexing table; encoding the indexes of the first list; binary compressing at least a serialized binary representation of the literal values of the second list; and concatenating the first list and the second list together to obtain an encoded bitstream of the information items. When the messages are sent over a plurality of connections, a global table is shared between the connections to store the indexed items of information; and a local indexing table for each connection associates indexes with references to an entry of the shared global table.

Certain implementations of the disclosed technology may include methods and computing systems for performing high-density data compression, particularly on numerical data that demonstrates various patterns, and patterns of patters. According to an example implementation, a method is provided. The method may include extracting a data sample from a data set, compressing the data sample using a first compression filter configuration, and calculating a compression ratio associated with the first compression filter configuration. The method may also include compressing the data sample using a second compression filter configuration and calculating a compression ratio associated with the second compression filter configuration. A particular compression filter configuration to utilize in compressing the entire data set may be selected based on a comparison of the compression ratio associated with the first compression filter configuration and a compression ratio associated with the second compression filter configuration.

Encoded (compressed) data in an encoded block format output by a data encoder, having an input including digital input elements from an input stream or file that are divided into blocks that contain Nb individual data elements per block. The encoded block format can include an index that, during decoding, supports random access. A decoder can losslessly decode the encoded block format. Each encoded block contains a header and a payload. The header specifies the unique characteristics of elements used by the payload to describe single-Byte or multi-Byte events that occurred in the input block. The encoded block format is generated by one or more block-oriented encoders (compressors). Blocks having the encoded block format may be consumed by one or more decoders (decompressors) that regenerate Nb elements from each decoded (uncompressed) block.

Embodiments of the present disclosure provide systems, methods, and computer storage media for mitigating latencies associated with the encoding of digital assets. Instead of waiting for codebook generation to complete in order to encode a digital asset for storage, embodiments described herein describe a shifting codebook generation and employment technique that significantly mitigates any latencies typically associated with encoding schemes. As a digital asset is received, a single codebook is trained based on each portion of the digital asset, or in some instances along with each portion of other digital assets being received. The single codebook is employed to encode subsequent portion(s) of the digital asset as it is received. The process continues until an end of the digital asset is reached or another command to terminate the encoding process is received. To encode an initial portion of the digital asset, a bootstrap codebook can be employed.

A method for compressing semi-structured data is discussed. The method includes accessing semi-structured data, the semi-structured data comprising a plurality of elements. The method includes determining a plurality of unique elements of the plurality of elements, each of the plurality of unique elements associated with a respective unique index of a plurality of unique indexes. Each of the unique index can indicate a position in one of a plurality of data stores. The method includes generating a sequence of encoded representations corresponding to the plurality of elements, the generating based on the plurality of unique indexes.

Aspects of the present disclosure provide techniques for variable spreading factor codes for non-orthogonal multiple access (NOMA). In an exemplary method, a base station assigns, from a first codebook of N short code sequences of length K, a subset of the short code sequences to a number of user equipments (UEs); receives a signal including uplink data or control signals from two or more of the UEs, wherein a first uplink data or control signal is sent using a first subsequence of one of the assigned short code sequences, and a second uplink data or control signal is sent using a second subsequence of one of the assigned short code sequences or using one of the assigned short code sequences; and decodes each uplink data or control signal in the signal based on the assigned short code sequences and subsequences of the assigned the short code sequences.