Accesses between a processor and its external memory is reduced when the processor internally maintains a compressed version of values stored in the external memory. The processor can then refer to the compressed version rather than access the external memory. One compression technique involves maintaining a dictionary on the processor mapping portions of a memory to values. When all of the values of a portion of memory are uniform (e.g., the same), the value is stored in the dictionary for that portion of memory. Thereafter, when the processor needs to access that portion of memory, the value is retrieved from the dictionary rather than from external memory. Techniques are disclosed herein to extend, for example, the capabilities of such dictionary-based compression so that the amount of accesses between the processor and its external memory are further reduced.

Systems, apparatuses, and methods for implementing flexible dictionary sharing techniques for caches are disclosed. A set-associative cache includes a dictionary for each data array set. When a cache line is to be allocated in the cache, a cache controller determines to which set a base index of the cache line address maps. Then, a selector unit determines which dictionary of a group of dictionaries stored by those sets neighboring this set would achieve the most compression for the cache line. This dictionary is then selected to compress the cache line. An offset is added to the base index of the cache line to generate a full index in order to map the cache line to the set corresponding to this chosen dictionary. The compressed cache line is stored in this set with the chosen dictionary, and the offset is stored in the corresponding tag array entry.

According to one embodiment, a buffer stores first hash values and first complementary data. A first conversion unit converts consecutive characters in a second character string into second hash values and second complementary data. A search unit searches for consecutive first hash values from the buffer, and output a pointer. A selection unit selects consecutive first hash values and pieces of first complementary data from the buffer. A second conversion unit converts the consecutive first hash values into a third character string using the pieces of first complementary data. A comparison unit compares the second character string with the third character string to acquire a matching length. An output unit output the matching length with the pointer.

Embodiments of the present disclosure provide systems and methods for reverse decompression. According to one embodiment of the present disclosure, the method for reverse decompression includes receiving encoded and compressed input data in a form of one or more data blocks and locating an end of block marker for a last block of the one or more data blocks of the input data. The method also includes traversing the input data, bit by bit, in a reverse direction starting from a last bit of the end of block marker of the last block of the one or more data blocks of the input data towards a beginning of the input data, determining if one block of the one or more blocks of the input data can be designated as a valid block, designating the one block as a valid block and decompressing the valid block in a forward direction.

Examples described herein relate to performing data compression by performing dictionary matching of data using hardware circuitry to generate dictionary matched results and post-processing of dictionary matched results using software executed by a processor. In some examples, dictionary matching includes LZ77 dictionary matching. In some examples, dictionary matching occurs on multiple segments of data in parallel.

A system and method for creating firmware patch files is disclosed. The method utilizes the Executable Linkable Format file that is created when the firmware image is created. By analyzing the ELF file, the patch creation software is able to identify functions and other data in the new firmware image. The patch creation software then compares these functions to corresponding functions in the old firmware image. The method then creates an edit sequence that may be used to transform the old firmware image into the new firmware image. The edit sequence is then converted into a series of opcodes, where each opcode is followed by at least one parameter. A patch program, disposed on a network device, is able to apply the patch file to update its firmware. This method creates a smaller patch file than other popular tools.

Dictionary-based compression is performed to compress data units using a similar data unit as the base unit (i.e., dictionary) for each candidate data unit. Similarity may be determined between data units by applying a locality-sensitive hashing scheme to each candidate data unit to produce a hash value, and by determining whether there is a matching value in a hash index of hash values for existing data units on the system. If there is a matching hash value, the candidate data unit may be compressed using the data unit corresponding to the matching hash value as the dictionary. Only a representative portion of the data unit may be hashed to produce the hash value, the portion comprised of chunks of the data unit, where each chunk is a continuous, uninterrupted section of data. The chunks themselves may not be (in some embodiments likely are not) contiguous to one another.

A method for compressing columnar data may include generating, for a data column included in a data chunk, a dictionary enumerating, in a sorted order, a first set of unique values included in the first data column. A compression technique for generated a compressed representation of the data column having a fewest quantity of bytes may be identified based at least on the dictionary. The compression technique including a dictionary compression applying the dictionary and/or another compression technique. A compressed data chunk may be generated by applying the compression technique to compress the data column included in the data chunk. The compressed data chunk may be stored at a database in a variable-size persistent page whose size is allocated based on the size of the compressed representation of the data column. Related systems and articles of manufacture are also provided.

Selecting tables for compression by threshold statistical values. Identified tables are reordered according to fields having the lowest cardinality to increase the size of character strings replaced by keys during compression. Field locations are mapped between the original table and the reordered table. Dictionary-based compression is performed on reordered tables.

According to one embodiment, a compression circuit generates substrings from input data for (3+M) cycles, the input data being N bytes per cycle, a byte length of each substring being greater than or equal to (N×(1+M)+1); obtains a set of matches, each of the matches including at least one past input data which input past and corresponds to at least a part of each of the substrings; selects a subset of matches from the set of matches including the input data of one cycle; and outputs the subset of matches. M is zero or a natural number. N is a positive integer which is two or more.