A method, non-transitory computer readable medium, and device that assists with reducing memory fragmentation in solid state devices includes identifying an allocation area within an address range to write data from a cache. Next, the identified allocation area is determined for including previously stored data. The previously stored data is read from the identified allocation area when it is determined that the identified allocation area comprises previously stored data. Next, both the write data from the cache and the read previously stored data are written back into the identified allocation area sequentially through the address range.

A method for storing a data file (DF) on a storage entity (SE) includes receiving, by a proxy (PE) and from a computing entity (CE), a plurality of hash values corresponding to a plurality of blocks of the DF. The PE may check whether the plurality of blocks of the DF are stored in the SE based on the plurality of hash values. Based on determining that at least a subset of the plurality of blocks of the DF are not being stored in the SE, the PE may compute a secret associated with an encryption key. The PE may transmit, to the CE, the secret. The PE may receive, from the CE, information including storage locations of the subset of the plurality of blocks within the SE and one or more hash values, of the plurality of hash values, associated with the subset of the plurality of blocks.

In some examples, a system performs data deduplication using a deduplication fingerprint index in a hash data structure comprising a plurality of blocks, wherein a block of the plurality of blocks comprises fingerprints computed based on content of respective data values. The system merges, in a merge operation, updates for the deduplication fingerprint index to the hash data structure stored in a persistent storage. As part of the merge operation, the system mirrors the updates to a cached copy of the hash data structure in a cache memory, and updates, in an indirect block, information regarding locations of blocks in the cached copy of the hash data structure.

An example method for storing data includes providing a plurality of physical storage pools, each storage pool including a plurality of storage nodes coupled to a network. The method also includes mapping a partition of a plurality of partitions to a set of physical storage pools, where each physical storage pool of the set of physical storage pools is located in a different availability zone, and the storage nodes within an availability zone are subject to a correlated loss of access to stored data. The method further includes receiving a data management request over the network, the data management request being associated with a data object. The method also includes identifying a first partition of the plurality of partitions corresponding to the received data management request and manipulating the data object in the physical storage pools mapped to the first partition in accordance with the data management request.

A method for moving files in a hierarchical storage system having a primary storage and a secondary storage including a sequential storage device from the primary storage to the secondary storage includes obtaining a predetermined file size to be written to the secondary storage, extracting, from a plurality of files in the primary storage, a file not stored in the secondary storage and having the oldest last access time, estimating a file size of the file having the oldest last access time on the secondary storage if the file having the oldest last access time is written to the secondary storage, and selecting the file having the oldest last access time as a file to be moved to the secondary storage as long as the estimated file size does not exceed the predetermined file size to be written to the secondary storage.

A storage management method and a storage management apparatus are provided. In some embodiments, the method includes: detecting, during a preset length of time, a writing amount per time unit of service data of a target network service in a target storage; retrieving a correspondence relationship between the writing amount per time unit and an amount of a redundant storage, wherein the relationship indicates the amount of the redundant storage increases with the increasing of the writing amount per time unit; determining a first amount of the redundant storage corresponding to the first writing amount per time unit according to the correspondence relationship; and configuring the redundant storage for the target network service in accordance with the first amount of the redundant storage.

A method includes determining a length of a file and storing the length of the file in a first memory location. An endpoint of a last complete record within the file is determined and the endpoint is stored in a second memory location. The length of the file stored in the first memory location is compared to a current length of the file, and a data structure associated with the file is updated beginning at the endpoint if the current length of the file exceeds the length of the file stored in the first memory location.

A method and system for eliminating the redundant allocation and deallocation of special data on disk, wherein the redundant allocation and deallocation of special data on disk is eliminated by providing an innovate technique for specially allocating special data of a storage system. Specially allocated data is data that is pre-allocated on disk and stored in memory of the storage system. “Special data” may include any pre-decided data, one or more portions of data that exceed a pre-defined sharing threshold, and/or one or more portions of data that have been identified by a user as special. For example, in some embodiments, a zero-filled data block is specially allocated by a storage system. As another example, in some embodiments, a data block whose contents correspond to a particular type document header is specially allocated.

The over-provisioning (OP) of a physical storage device (PSD) may be increased, and the useful life of the PSD increased, by converting uncompressed data stored on the PSD to compressed data. It may be determined that increasing the useful life of the PSD, and the data reduction resulting from the compression, outweigh the benefit of faster I/O response times if the data remains uncompressed. A first portion of the PSD may be initially reserved for compression. A second portion of the PSD may store compressed data. It may be determined whether it is desirable to increase the OP of the PSD to thereby reduce the effective write rate on the PSD. If compression is determined to be desirable, the dynamic portion may be compressed, thereby reducing the amount of storage space consumed by the data, and freeing up storage space that can be used by the PSD for OP.

In a hierarchical storage memory (HSM), a file recalled by a specific application is migrated as soon as possible after completion of the application process. Specifically, the effective UID of a specific process is preregistered on an HSM client. After a recall operation is performed on a certain file from the user ID, when there is no access from the UID to the file for a given length of time, the file is migrated. This prevents files premigrated by access from any application other than the specific one from being handled in the same way, resolving a disadvantageous problem caused when these (premigrated) files are not desired to be migrated preferentially.