An information processing apparatus performs a backup process to store backup target data on a deduplication storage device configured to eliminate duplicate storage by referring to previously stored data having the same content. The apparatus includes a calculation unit configured to calculate the capacity after deduplication that is performed by storing the backup target data in the deduplication storage device, each time the backup process is performed, and a determination unit configured to determine whether the backup target data is normal or abnormal, based on the capacity calculated each time the backup process is performed.

A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

In one embodiment, a method includes receiving data comprising a plurality of data elements; creating a binary sequence comprising a plurality of bonus bits using a first binary sequence generator; using a first exclusive-or module to provide a XOR calculation using bits of each data element of the data with a bonus bit from the binary sequence; passing each data element along with its corresponding parity bit to an input of a data path; receiving each data element along with its corresponding parity bit at an output of the data path; creating the binary sequence using a second binary sequence generator; using a second XOR module to XOR together bits of each data element along with its corresponding parity bit and a bonus bit from the binary sequence to produce a result; and analyzing the result to determine whether an error has occurred to the data in the data path.

Techniques for supporting large segments when issuing writes to an erasure coded storage object in a distributed storage system are provided. In one set of embodiments, a node of the system can pre-allocate a segment of space in a capacity object of the storage object, receive a write request for updating a logical data block of the storage object, write data/metadata for the block to a record in a data log of a metadata object of the storage object, place the block in an in-memory bank, and determine whether the in-memory bank has become full. If so, the node can compute/fill-in one or more parity blocks for each stripe of the storage object in the in-memory bank and write, based on a next sub-segment pointer pointing to a free sub-segment of the pre-allocated segment, the contents of the in-memory bank via a full stripe write to the free sub-segment.

The data storage system is a RAID-based data storage system in which resources are globally shared. This storage system includes the first number of disks, and the RAID mechanism is used to store data on each disk. The blocks on different disks form stripes, and at least one of the blocks on the stripe stores the parity information, wherein the width of the stripe is less than the first number. The data layout of the data storage system satisfies the following characteristics: any two physical blocks in the stripe are distributed on different disks; the data blocks distributed on each disk are the same, and the distributed parity blocks are also the same; other data in the stripe associated with any piece of disk data is evenly distributed across all the remaining disks. Normal data layout and degraded data layout can be implemented by orthogonal Latin squares. This system can remove the limitation that the number of disks in the normal data storage system is equal to the stripe width, and break the resource isolation between the disk groups. And in the event of a disk failure, this invention can achieve a complete equalization of the reconstructed read load.

The system receives a request to write data and associated metadata. The system determines a key associated with the data, wherein the key corresponds to an entry in a data structure maintained by a first storage system. The system writes the metadata to a first non-volatile memory of a first set of storage drives of the first storage system by updating the entry with a logical block address for the data and a physical location in a second set of storage drives of a second storage system. The system writes the key and the data to a second non-volatile memory of the second set of storage drives based on the physical location, wherein the first non-volatile memory is of a lower density than the second non-volatile memory.

A method, computer program product, and computing system for remotely storing first content received on a first processing node of a clustered computing environment onto a storage platform, wherein the clustered computing environment includes a plurality of processing nodes; locally storing metadata that identifies the location of the first content within the storage platform on the first processing node, thus defining first locally-stored differential metadata; and after the occurrence of a storage trigger event, instructing the first processing node write the first locally-stored differential metadata to a cluster metadata pool within the storage platform.

Storage blocks are managed. For instance, a set of write parameters and a set of deletion parameters are obtained related to a target storage block. In response to the set of write parameters matching the set of deletion parameters, a first data length is obtained for the target storage block, the first data length being determined in response to receiving a write request for the target storage block. Further, reclaim information is determined related to the target storage block based on the first data length and the set of deletion parameters. It is thus possible to reduce times of scanning the entire object table to determine whether there is an object referring to the storage block, thereby reducing the time consumed by the verification process and improving the reclaiming speed of storage blocks.