A method for surveying a data storage subsystem for latent errors before a failing disk drive of the data storage subsystem fails and recovering unreadable data usable to reconstruct data of the failing disk drive. The method includes determining that a disk drive of a plurality of disk drives of the data storage subsystem meets a threshold for being identified as a failing disk drive, and prior to failure of the failing disk drive, surveying at least a portion of the data on the remaining plurality of disk drives to identify data storage areas with latent errors. The identified data storage areas may be reconstructed utilizing, at least in part, data stored on the failing disk drive.

An improved system is disclosed for ensuring the integrity of data stored on a dispersed data storage network. Checksums are used to ensure integrity of both data segments and data slices. Checksums appended to data slices are checked by receiving slice servers to ensure that no errors occurred during transmission. Slice servers also periodically recalculate checksums for stored data slices to ensure that data slices have not been corrupted during storage. Checksums appended to data segments are checked when data segments are read from the storage network.

A scrubbing controller of a semiconductor memory device includes a scrubbing address generator and a weak codeword address generator. The scrubbing address generator generates a scrubbing address for all codewords in a first bank array of a plurality of bank arrays in a first scrubbing mode. The scrubbing address is associated with a normal scrubbing operation and changes in response to an internal scrubbing signal and a scrubbing command. The weak codeword address generator generates a weak codeword address for weak codewords in the first bank array in a second scrubbing mode. The weak codeword address is associated with a weak scrubbing operation and is generated in response to the internal scrubbing signal.

Provided are techniques for capturing compression efficiency metrics for processing data. In response to retrieving native data for a first operation, perform the first operation; perform a second operation to generate a compression efficiency metric from the native data based on a ratio of the native data to compressed native data; and store the compression efficiency metric persistently for subsequent use in prioritizing compression of the native data.

For an antivirus scan during a data scrub operation, the antivirus scan is concurrently performed as an overlap with the data scrub operation, wherein the data scrub operation periodically inspects and corrects memory errors. The antivirus scan concurrently performing as an overlap with the data scrub operation is increased if a reduction in disk access by a host application is detected.

Provided are techniques for capturing compression efficiency metrics for processing data. In response to retrieving native data for a first operation, perform the first operation; perform a second operation to generate a compression efficiency metric from the native data based on a ratio of the native data to compressed native data; and store the compression efficiency metric persistently for subsequent use in prioritizing compression of the native data.

For an antivirus scan during a data scrub operation, an antivirus scan is concurrently performed as an overlap with the data scrub operation, wherein the data scrub operation periodically inspects and corrects memory errors. The antivirus scan concurrently performing as an overlap with the data scrub operation is increased if a reduction in disk access by a host application is detected. A number of antivirus scan input/output (I/O) operations and data scrub I/O operations is reduced.

For an antivirus scan during a data scrub operation, an antivirus scan is concurrently performed as an overlap with the data scrub operation, wherein the data scrub operation periodically inspects and corrects memory errors. The antivirus scan concurrently performing as an overlap with the data scrub operation is increased if a reduction in disk access by a host application is detected. A number of antivirus scan input/output (I/O) operations and data scrub I/O operations is reduced.

Disclosed is the technology for data scrubbing in a cluster-based storage system. This technology allows protecting data against failures of storage devices by periodically reading data object replicas and data object hashes stored in a plurality of storage devices and rewriting those data object replicas that have errors. The present disclosure addresses aspects of writing data object replicas and hashes, checking validity of data object replicas, and performing data scrubbing based upon results of the checking.

For an antivirus scan during a data scrub operation, an antivirus scan is concurrently performed as an overlap with the data scrub operation, wherein the data scrub operation periodically inspects and corrects memory errors. The antivirus scan concurrently performing as an overlap with the data scrub operation is increased if a reduction in disk access by a host application is detected. A number of antivirus scan input/output (I/O) operations and data scrub I/O operations is reduced.