A method begins with a processing module of a distributed storage network (DSN), receiving a request to access a data object stored in the DSN and identifying a first data descriptor associated with the data object, identifying a second data descriptor associated with the data object, identifying a first data index key and a first index structure for the first data descriptor and identifying a second data index key and a second index structure for the second data descriptor. The method continues with the processing module accessing the first index structure for the first data descriptor , based on the first and second data index keys, to retrieve a first and second set of data identifiers, respectively and based on one or more data identifiers being common to the first set of data identifiers and the second set of data identifiers, creating a superset of data identifiers.

A storage apparatus includes a plurality of drives and has a parity group constituted by a plurality of drives. The storage apparatus stores a hash management table to manage hash values of a prescribed data unit of data of the drives constituting the parity group and a hash value of a prescribed data unit of data stored in another drive other than the drives constituting the parity group. A processor is configured to determine whether a same data unit as a data unit included in data stored in a replacement target drive exists in the other drive on a basis of the hash values, and copy the same data unit of the other drive to a replacement destination drive when the same data unit exists.

Transferring data between memories may include reading data associated with a memory transfer transaction from a first memory, determining whether a bypass indication associated with the memory transfer transaction is asserted, and transferring the data from the first memory to a second memory. The transferring may include bypassing the first-processing if the bypass indication is asserted. The transferring may further include bypassing second-processing the data if the bypass indication is asserted. Following bypassing the second-processing, the data may be stored in the second memory.

An all flash array storage device includes a flash memory array including multiple flash memories and a microprocessor. The flash memories correspond to multiple logical aggregation units. Each logical aggregation unit includes multiple stripes. Each stripe includes multiple storage units, including multiple data units and at least one parity unit. The microprocessor detects a status of the flash memories. In response to a detection result indicating that one of the flash memories has been removed from the flash memory array, the microprocessor sequentially performs a repair operation on the stripes comprised in one or more logical aggregation units that have been written with data. In the repair operation of one stripe, the microprocessor recalculates protection information of the stripe according to content stored in a portion of data units of the stripe and writes the recalculated protection information in one or more storage units of the stripe.

Systems, methods, and circuitries are provided for supporting distributed erasure coding in a shared file system. In one example, a method is provided to be performed by an initiator device configured to read and write data in files stored in a plurality of storage nodes that are controlled by a file management system. The method includes generating a stripe by identifying data to be stored in a storage system; dividing the data into K data segments; and performing an erasure encoding operation on the K data segments to generate a parity segment, wherein the stripe includes the K data segments and the parity segment. The method includes requesting, from the file management system, respective memory allocations in the storage devices for storing respective segments of the stripe; and transmitting, to each of the plurality of storage nodes, a respective instruction to store a respective data or parity segment in the memory allocation on the respective storage device.

A distributed storage places data units and parity units constituting a stripe formed by divided data into storage nodes in a distributed manner. In reference to determination formulas, either a full-stripe parity calculation method or an RPM parity calculation method is selected so as to minimize an amount of network traffic.

A write hole protection method and system for a RAID, and a storage medium. The method comprises: presetting a log area, and after a RAID is degraded, setting the log area to be in an enabled state; when the log area is in the enabled state, determining, before each stripe write operation, whether a data block of a failed member disk of the RAID in a stripe is a check data block; if the data block is not the check data block, determining whether data blocks to be written of the stripe comprise a data block to be written into the failed member disk; if yes, backing up the data block to be written into the failed member disk in the log area; if not, calculating the data block of the failed member disk and backing up the data block in the log area, or backing up the data blocks to be written in the log area; and when the degraded RAID is started after a failure, performing data recovery using the log area. By using the present solution, the write hole issue of the RAID is avoided.

A data recovery method for a storage medium, a data recovery system, and a related device are provided. The method includes: when a write failure occurs during writing data into the storage medium, extracting original data corresponding to the data failed to be written and original verification data in an original RAID stripe to which the original data belongs; obtaining new verification data by performing calculation on the original data and the original verification data by using a preset verification algorithm; forming a new RAID stripe by using valid data other than the original data in the original RAID stripe and the new verification data, and recording data relations in the new RAID stripe; and when a reading failure occurs during reading the data in the new RAID stripe, recovering, according to the data relations in the new RAID stripe, the data failed to be read.

First data is received. First error-checking data generated based on a cyclic redundancy check (CRC) operation of the first data is received. Second data is generated by combining the first data with a first data pattern. Second error-checking data of the second data is generated by using a combination of the first error-checking data and a second data pattern. The second data pattern has a size that is based on the first data pattern.

According to one embodiment, a memory system includes a non-volatile memory, and a controller configured to control the non-volatile memory. The controller is configured to write data to the non-volatile memory, read the written data from the non-volatile memory after writing of the data is completed, generate parity data corresponding to the read data, and write the generated parity data to a memory for parity storage.