A packet processing system having a barrel compactor that extracts a desired data subset from an input dataset (e.g. an incoming packet). The barrel compactor is able to selectively shift one or more of the input data units of the input dataset based on individual shift values for those data units. Additionally, in some embodiments one or more of the data units are able to be logically combined to produce a desired logical output unit.

A multichannel data packer includes a plurality of two-input multiplexers and a controller. The plurality of two-input multiplexers is arranged in 2.sup.N rows and N columns in which N is an integer greater than 1. Each input of a multiplexer in a first column receives a respective bit stream of 2.sup.N channels of bit streams. Each respective bit stream includes a bit-stream length based on data in the bit stream. The multiplexers in a last column output 2.sup.N channels of packed bit streams each having a same bit-stream length. The controller controls the plurality of multiplexers so that the multiplexers in the last column output the 2.sup.N channels of bit streams that each has the same bit-stream length.

Techniques are disclosed relating to detecting matching datasets using encode values. In various embodiments, a data monitoring system may perform encoding operations on a first dataset to generate a first encode value that corresponds to a particular one of one or more fields included in the first dataset. The data monitoring system may then determine whether the first dataset matches a previously analyzed dataset. For example, in some embodiments, data monitoring system may compare the first encode value to a previous encode value that corresponds to a second field of the previously analyzed dataset. Based on this comparison, the data monitoring system may generate an output value that is indicative of a similarity between the first encode value and the previous encode value. The data monitoring system may then determine whether the first dataset matches the previously analyzed dataset based on this output value.

Multi-byte compressed string representation embodiments define a String class control field identifying compression as enabled/disabled, and another control field, identifying a decompressed string created when compression enabled. On pattern matching by a compiler, noping tests based on null setting of stringCompressionFlag and registering a class loading assumption on a nop location. When arguments to a String class constructor are not compressible, a decompressed String is created and stringCompressionFlag initialized. Endian-aware helper methods for reading/writing byte and character values and helper methods for widening, narrowing, truncation, conversion and masking are defined. Enhanced String class constructors, when characters are not compressible, create a decompressed String, and initialize stringCompressionFlag triggering class load assumptions, overwriting all nopable patch points. A String object sign bit is set to one for decompressed strings when compression enabled, and masking/testing this flag bit is noped using stringCompressionFlag and associated class load assumption. Alternative package protected string constructors and operations are provided. Checking a predetermined location to determine whether supplied arguments to a String class constructor are compressible is performed.

Text can be encoded into DNA sequences. Each word from a document or other text sample can be encoded in a DNA sequence or DNA sequences and the DNA sequences can be stored for later retrieval. The DNA sequences can be stored digitally, or actual DNA molecules containing the sequences can be synthesized and stored. In one example, the encoding technique makes use of a polynomial function to transform words based on the Latin alphabet into k-mer DNA sequences of length k. Because the whole bits required for the DNA sequences are smaller than the actual strings of words, storing documents using DNA sequences may compress the documents relative to storing the same documents using other techniques. In at least one example, the mapping between words and DNA sequences is one-to-one and the collision ratio for the encoding is low.

A system is provided for electronic data compression by automated time-dependent compression algorithm. In particular, the system may track instances in which a particular dataset is used, copied, or accessed over time. For certain datasets (e.g., datasets that have not been accessed for a threshold amount of time), the system may use a time-based compression algorithm that progressively removes the least significant bits of such datasets as time passes. The compression of the datasets may continue until the system detects that further compression would cause the dataset to be unreadable or unrecoverable. In this way, the system may minimize the computing resources allocated to storing datasets that are not frequently accessed.

A system comprises an encoder configured to entropy encode a bitstream comprising both compressible and non-compressible symbols. The encoder parses the bitstream into a compressible symbol sub-stream and a non-compressible sub-stream. The non-compressible symbol sub-stream bypass an entropy encoding component of the encoder while the compressible symbol sub-stream is entropy encoded. When a quantity of bytes of entropy encoded symbols and bypass symbols is accumulated a chunk of fixed or known size is formed using the accumulated entropy encoded symbol bytes and the bypass bytes without waiting on the full bitstream to be processed by the encoder. In a complementary manner, a decoder reconstructs the bitstream from the packets or chunks.

Provided is a technique for converting an integer value sequence for encoding/decoding which allows an integer value sequence having a distribution including small values other than a zero value and greatly biased to small values to be encoded with a small average bit number. Provided are: a unary coding unit which subjects an input sequence of non-negative integer values to unary coding to obtain a unary code sequence; a bit reversing unit which replaces a bit value ‘0’ with a bit value ‘1’ and a bit value ‘1’ with a bit value ‘0’ in the bits in the unary code sequence to obtain a replaced code sequence; and a unary decoding unit which subjects the replaced code sequence to unary decoding to obtain a sequence of non-negative integer values.

A storage device is disclosed. The storage device may comprise storage for input encoded data. A controller may process read requests and write requests from a host computer on the data in the storage. An in-storage compute controller may receive a predicate from the host computer to be applied to the input encoded data. A transcoder may include an index mapper to map an input dictionary to an output dictionary, with one entry in the input dictionary mapped to an entry in the output dictionary, and another entry in the input dictionary mapped to a “don't care” entry in the output dictionary.

A method of data compression in which the total size of the compressed data is determined and based on that determination, the bit depth of the input data may be reduced before the data is compressed. The bit depth that is used may be determined by comparing the calculated total size to one or more pre-defined threshold values to generate a mapping parameter. The mapping parameter is then input to a remapping element that is arranged to perform the conversion of the input data and then output the converted data to a data compression element. The value of the mapping parameter may be encoded into the compressed data so that it can be extracted and used when subsequently decompressing the data.