A storage system has a plurality of storage nodes having equal non-volatile storage capacity that is subdivided into equal size cells. Host application data that is stored in the cells is protected using RAID or EC protection groups each having members stored in ones of the cells and distributed across the storage nodes such that no more than one member of any single protection group is stored by any one of the storage nodes. Spare cells are maintained for rebuilding protection group members of a failed one of the storage nodes on remaining non-failed storage nodes so full data access is possible before replacement or repair of the failed storage node.

A method for rebuilding data, comprising: obtaining, from a metadata node, a source file data layout for a source file and a target file data layout for a target file, wherein the source file is associated with a degraded mapped RAID group and the target file is associated with a new mapped RAID group; generating, by the client application node, a plurality of input/output (I/O) requests to read a portion of the data associated with the source file using the source file data layout; obtaining, in response to the plurality of I/O requests, the portion of the data associated with the source file; rebuilding a second portion of the data associated with source file using the portion of the data; and initiating, storage of at least the second portion of the data associated with the source file in the storage pool using the target file data layout.

Methods and systems are provided for rapid failure recovery for a distributed storage system for failures by one or more nodes.

A method for execution by a dispersed storage network (DSN) managing unit includes receiving access information from a plurality of distributed storage and task (DST) processing units via a network. Cache memory utilization data is generated based on the access information. Configuration instructions are generated for transmission via the network to the plurality of DST processing units based on the cache memory utilization data.

Systems and methods are described for providing rapid access to data objects stored in a cache. Rather than storing data objects directly, each object can be broken into a number of parts via erasure coding, which enables the object to be generated from less than all parts. When servicing a request for the data object, a device can attempt to retrieve all parts, but begin to generate the data object as soon as a sufficient number of parts is retrieved, even if requests for other parts are outstanding. In this way, the data object can be retrieved without delay due to the slowest requests. For example, where one or more requests timeout, such as due to failure of cache devices, this timeout may have no effect on time required to retrieve the data object from the cache.

Event records from multiple computing devices are received at a managing unit. Individual event records include an event identifier field including an event identifier identifying a first event associated with a particular computing device, a parent event identifier field identifying a parent event that initialized the first event, and an entity identifier field including an entity identifier identifying the particular computing device. The managing unit generates log records associated with event identifiers included in the event records. The log records include state fields indicating a state of a particular event associated with a particular event identifier. Based on a correlation of the event and log records, the managing unit determines at least two computing devices associated with events resulting in an error state, and identifies parent events that initialized the events with errors. The managing unit generates a report linking the parent events to the events having an error state.

Systems and methods for replacing and testing a data storage device are disclosed. In disclosed embodiments, a system including a data storage array (DSA) including a plurality of data storage devices (DSDs) in an enclosure. The system further includes an I/O server coupling the DSA to a client node and configured to provide data access between the client node and the DSA. The system further includes a management server coupled to the DSA, configured to detect a failed DSD in the DSA, detect a replacement DSD in the enclosure that replaces the failed DSD, and add the replacement DSD to a logical path of the DSA. The management server is further configured to display an indication of a state of the DSA based on the comparing.

A method for execution in a storage network, the method begins by determining a user device group content preference, wherein the user group content includes target content for a user device group and the determining includes predicting future target content for the user group. The method continues by selecting a plurality of network edge units for staging encoded data slices, identifying target content for partial download to the plurality of network edge units and dispersed error encoding the target content to generate a set of encoded data slices. The method then continues by identifying encoded data slices from the set of encoded data slices corresponding to the target content for partial download and determining a partial downloading schedule for sending the encoded data slices for partial download to each network edge unit of the plurality of network edge units. The method continues by facilitating partial downloading of the target content by sending the encoded data slices for partial download to each network edge unit of the plurality of network edge units.

A secure data parser is provided that may be integrated into any suitable system for securely storing and communicating data. The secure data parser parses data and then splits the data into multiple portions that are stored or communicated distinctly. Encryption of the original data, the portions of data, or both may be employed for additional security. The secure data parser may be used to protect data in motion by splitting original data into portions of data, that may be communicated using multiple communications paths.

A storage network processing system includes a processor, a network interface and memory that stores operational instructions. The operation instructions enable the processor to receive a data object for storage and dispersed error encode the data object in accordance with dispersed error encoding parameters to produce a plurality of encoded data slices. The operation instructions further enable the processor to generate to determine a plurality of site slice sets from the plurality of encoded data slices, where each site slice set of the plurality of site slice sets includes a number of unique encoded data slices of the plurality of encoded data slices that is greater than or equal to a site write threshold value. The operation instructions further enable the processor to a designate one of a plurality of storage sites for each of the plurality of site slice sets and transmit each of the plurality of site slice sets to a corresponding designated one of the plurality of storage sites via the network.