Methods, apparatus and design structures are provided for improving resource utilization by data compression accelerators. An exemplary apparatus for compressing data comprises a plurality of hardware data compression accelerators and a hash table shared by the plurality of hardware data compression accelerators. Each of the plurality of hardware data compression accelerators optionally comprises a first-in-first-out buffer that stores one or more input phrases. The hash table optionally records a location in the first-in-first-out buffers where a previous instance of an input phrase is stored. The plurality of hardware data compression accelerators can simultaneously access the hash table. For example, the hash table optionally comprises a plurality of input ports for simultaneous access of the hash table by the plurality of hardware data compression accelerators. A design structure for a data compression accelerator system is also disclosed.

A data reduction for reducing highly correlated data (e.g., highly correlated data streams) is provided. Correlated data of a plurality of data streams are identified, and a spectral dimensional decomposition is performed. In this way, information from the data of the data streams may be exploited, and this information may be used in order to achieve a highly efficient reduction of the data. In this way, the compression ratio of the data may be enhanced or the data loss of the reduce data compression may be minimized.

Methods and systems are provided for the compression and decompression of data. The compression and decompression of data may include partitioning the data into chunks, analyzing the individual chunks to determine the best compression and decompression encoders to utilize for the next data chunk of a data file. In compressing and decompressing using the mentioned technique, the data is delivered to the requesting client in an efficient and speedy manner.

An amount of memory needed to hold prime data elements during reconstitution may be determined by examining the creation and usage of prime data elements and their spatial and temporal characteristics during data distillation.

Data records may be managed in a relational database by monitoring, a record length for a first data record in a page of memory, an amount of free space in the page, and a page length. In response to receiving an operator command to replace the first data record with a second data record, a database management system may determine whether an estimated record length of a compressed second data record is outside of the amount of free space in the page. In response to determining the estimated record length of a compressed second data record is outside of the amount of free space in the page, the database management system may determine whether an estimated length of a compressed page is outside of the page length. In response to determining the estimated length of a compressed page is within the page length, the page may be compressed.

Data records may be managed in a relational database by monitoring, a record length for a first data record in a page of memory, an amount of free space in the page, and a page length. In response to receiving an operator command to replace the first data record with a second data record, a database management system may determine whether an estimated record length of a compressed second data record is outside of the amount of free space in the page. In response to determining the estimated record length of a compressed second data record is outside of the amount of free space in the page, the database management system may determine whether an estimated length of a compressed page is outside of the page length. In response to determining the estimated length of a compressed page is within the page length, the page may be compressed.

Data records may be managed in a relational database by monitoring, a record length for a first data record in a page of memory, an amount of free space in the page, and a page length. In response to receiving an operator command to replace the first data record with a second data record, a database management system may determine whether an estimated record length of a compressed second data record is outside of the amount of free space in the page. In response to determining the estimated record length of a compressed second data record is outside of the amount of free space in the page, the database management system may determine whether an estimated length of a compressed page is outside of the page length. In response to determining the estimated length of a compressed page is within the page length, the page may be compressed.

Data records may be managed in a relational database by monitoring, a record length for a first data record in a page of memory, an amount of free space in the page, and a page length. In response to receiving an operator command to replace the first data record with a second data record, a database management system may determine whether an estimated record length of a compressed second data record is outside of the amount of free space in the page. In response to determining the estimated record length of a compressed second data record is outside of the amount of free space in the page, the database management system may determine whether an estimated length of a compressed page is outside of the page length. In response to determining the estimated length of a compressed page is within the page length, the page may be compressed.

The present disclosure is directed to memory saving systems and methods for buffer overflow that occurs during image compression. In example embodiments, when an overflow occurs during image compression, the overflow data is written to an allocated designated overflow memory by an overflow handler. This memory can be designed to be rewritten multiple times during the image compression process, and can therefore occupy less memory than would be required in possible alternative solutions that comprise expanding the size of each unit of compression destination memory to account for the worst case compression scenario in each instance. Various embodiments that utilize the designated overflow memory to conserve memory when buffer overflow occurs during image compression are illustrated and described.

A data compression/decompression methodology and system stores in a compression-side dictionary the data that has been transmitted and additionally applies to such data in the compression-side dictionary a status of being invalid for purposes of use as a reference in data compression until an acknowledgement signal has been received from the decompression side indicating that such data has been received. Once the compression side has received the acknowledgement signal indicating that the data in the form of a compressed data packet has been received on the decompression side, the status is changed from being invalid into being valid, i.e., into being usable as reference data for use in compressing further data elements in the data stream. Each data packet includes a stream index which is representative of the memory location in the compression-side dictionary where the first data element of the uncompressed data set of the data packet is stored.