Described are methods, systems and computer readable media for performance logging of complex query operations.

Methods for balancing storage data traffic in a system in which at least one computing device (server) coupled to a converged network accesses at least one storage device coupled (by at least one adapter) to the network, systems configured to perform such methods, and devices configured to implement such methods or for use in such systems. Typically, the system includes servers and adapters, and server agents implemented on the servers and adapter agents implemented on the adapters are configured to detect and respond to imbalances in storage and data traffic in the network, and to redirect the storage data traffic to reduce the imbalances and, thereby to improve the overall network performance (for both data communications and storage traffic). Typically, each agent operates autonomously (except in that an adapter agent may respond to a request or notification from a server agent), and no central computer or manager directs operation of the agents.

Variables utilized in device firmware that provides various boot and runtime services are repaved in a fault-tolerant manner within a secure store in a durable, non-volatile device memory during an FOTA update process. A spare region in the secure store is utilized to temporarily hold a back-up of a primary region in which the firmware variables are written. Using a transaction-based fault-tolerant write (FTW) process, the variables in the primary region can be repaved with variables contained in a firmware update payload that is delivered from a remote service. In the event of a fault in the variable region repaving process, either the primary or spare region will remain valid so that firmware in a known good state can be utilized to enable the device to boot successfully and the variable region repaving in the FOTA update process may be restarted.

To identify objects shared by entities and to, in turn, identify free space in nonvolatile storage, a computer system uses a probabilistic data structure which tests whether an element is a member of a set. Such probabilistic data structures are created for entities in the storage system that share objects. The probabilistic data structure for an entity represents the objects that are used by that entity. When an entity is deleted, each object used by that entity is compared to the probabilistic data structures of other entities to determine if there is a likelihood that the object is used by one or more of the other entities. If the likelihood determined for an object is above an acceptable threshold, then the object is not deleted. If the likelihood determined for an object is below the set threshold, then the object can be deleted and the corresponding storage locations can be marked as free.

Peer-to-peer redundant file server system and methods include clients that determine a target storage provider to contact for a particular storage transaction based on a pathname provided by the filesystem and a predetermined scheme such as a hash function applied to a portion of the pathname. Servers use the same scheme to determine where to store relevant file information so that the clients can locate the file information. The target storage provider may store the file itself and/or may store metadata that identifies one or more other storage providers where the file is stored. A file may be replicated in multiple storage providers, and the metadata may include a list of storage providers from which the clients can select (e.g., randomly) in order to access the file.

A method and system for eliminating the redundant allocation and deallocation of special data on disk, wherein the redundant allocation and deallocation of special data on disk is eliminated by providing an innovate technique for specially allocating special data of a storage system. Specially allocated data is data that is pre-allocated on disk and stored in memory of the storage system. “Special data” may include any pre-decided data, one or more portions of data that exceed a pre-defined sharing threshold, and/or one or more portions of data that have been identified by a user as special. For example, in some embodiments, a zero-filled data block is specially allocated by a storage system. As another example, in some embodiments, a data block whose contents correspond to a particular type document header is specially allocated.

As disclosed herein a computer system for secure database backup and recovery in a secure database network has N distributed data nodes. The computer system includes program instructions that include instructions to receive a database backup file, fragment the file using a fragment engine, and associate each fragment with one node, where the fragment is not stored on the associated node. The program instructions further include instructions to encrypt each fragment using a first encryption key, and store, randomly, encrypted fragments on the distributed data nodes. The program instructions further include instructions to retrieve the encrypted fragments, decrypt the encrypted fragments using the first encryption key, re-encrypt the decrypted fragments using a different encryption key, and store, randomly, the re-encrypted fragments on the distributed data nodes. A computer program product and method corresponding to the above computer system are also disclosed herein.

Nonsequential readahead for deep learning training that includes: receiving an indication of a list of batch storage locations for a batch of data objects; prefetching, for each storage location in the list of batch storage locations, storage content corresponding to the batch of data objects; and storing the storage content corresponding to the batch of data objects within a cache accessible to an artificial intelligence workflow.

A distributed storage system includes multiple resource nodes each having associated storage media. The resource nodes are configured to operate a first protocol between the resource nodes that exchanges availability and performance information for storage elements in the associated storage media. The resource nodes also operate a second protocol that dynamically distributes and redistributes data between the different resource nodes based on the availability and performance information for the storage elements. Relative distances may be identified between the different resource nodes and the second protocol may weight the availability and performance information based on the relative distances. The second protocol also may identify types of unshared use, shared use, and concurrent use for different portions of the data and distribute the portions of the data to other resource nodes based on the identified types of use.

Multiple data paths may be available to a data management system for transferring data between a primary storage device and a secondary storage device. The data management system may be able to gain operational advantages by performing load balancing across the multiple data paths. The system may use application layer characteristics of the data for transferring from a primary storage to a backup storage during data backup operation, and correspondingly from a secondary or backup storage system to a primary storage system during restoration.