A method for enabling pruning of a blockchain of a blockchain network includes creating an active blocks commitments Merkle tree from hashes of active blocks and creating an active smart contracts commitments Merkle tree from hashes of active smart contracts. The Merkle trees are created after an amount of blocks created in the blockchain has reached a threshold set by a pruning threshold parameter stored in the blockchain network. Hashes of the roots of the Merkle trees are stored in a header of a new block as a new genesis block. The new genesis block is broadcast to the blockchain network. A set of the active blocks and active smart contracts used respectively to create the active blocks commitments Merkle tree and the active smart contracts commitments Merkle tree are committed to upon the blockchain network reaching consensus on the new genesis block.

Data transformation caching in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: identifying, in dependence upon one or more machine learning models to be executed on the GPU servers, one or more transformations to apply to a dataset; generating, in dependence upon the one or more transformations, a transformed dataset; storing, within one or more of the storage systems, the transformed dataset; receiving a plurality of requests to transmit the transformed dataset to one or more of the GPU servers; and responsive to each request, transmitting, from the one or more storage systems to the one or more GPU servers without re-performing the one or more transformations on the dataset, the transformed dataset.

In a storage system such as a SAN, NAS, or storage array that implements hierarchical performance tiers based rated drive access latency, on-drive compression is used on data stored on a first tier and off-drive compression is used on data stored on a second tier. Off-drive compression is more processor intensive and may introduce some data access latency but reduces storage requirements. On-drive compression is performed at or near line speed but generally yields lower size reduction ratios than off-drive compression. On-drive compression may be implemented at a higher performance tier whereas off-drive compression may be implemented at a lower performance tier. Further, space saving realized from on-drive compression may be applied to over-provisioning.

Systems and methods are described for enabling garbage collection on data storage systems. Traditional garbage collection often attempts to track use of data items on an individual level, deleting each item when it is no longer used. In distributed systems, tracking use on an individual level is difficult, and may require centralized knowledge across the system with respect to individual data items. Provided herein is a “coarse-grained” garbage collection mechanism, which divides objects into logical groups referred to as “roots.” Each root has a life cycle. While active, new data can be stored in a root. While inactive, use of data within a root can cause that date to be copied to a different, active root. When the system detects that data hasn't been used in an inactive root for a threshold period, the root can be considered “dead” and data within the root may be deleted.

Various implementations described herein relate to systems and methods for managing metadata for conditional update, including adding conditional entry to a list in an in-memory journal for a conditional update associated with a garbage collection write, configuring a base entry in the list to point to the conditional entry, and in response to determining that the conditional update is resolved such that a physical location identified in the conditional entry is valid, freeing the conditional entry.

An integrated circuit is provided that includes compression or decompression circuitry along a datapath. An integrated circuit system may include first memory to store data, data utilization circuitry to operate on the data, and a configurable data distribution path to transfer data between the first memory and the data utilization circuitry. Compression or decompression circuitry may be disposed along the data distribution path between the first memory and the data utilization circuitry to enable the first memory to store the data in compressed form and to enable the data utilization circuitry to operate on the data in uncompressed form. The compression or decompression circuitry may use lossless sparse encoding, lossless multi-precision encoding, lossless prefix lookup table-based encoding, Huffman encoding, selective compression, or lossy compression.

An illustrative storage management appliance is interposed between client computing devices and one or more cloud storage resources. The appliance uses cloud storage resources in conjunction with a network attached storage device configured within the appliance to provide to the client computing devices seemingly unlimited network attached storage on respective network shares. The storage management appliance monitors data objects on the network shares and when a data object meets one or more criteria for archiving, the storage management appliance archives the data object to a cloud storage resource and replaces it with a stub and preview image on the network share. When access to the stub and/or preview image is detected, the storage management appliance restores the data object from the cloud storage resource. The criteria for archiving flexibly allow individual data objects to be archived to cloud storage without archiving frequently-accessed “neighboring” data objects on the same network share.

Techniques are provided for serializing replication operations. A plurality of operations are implemented upon a first storage object and are replicated as a plurality of replication operations. An order with which the plurality of replication operation are to be executed upon a second storage object is determined. Execution of the plurality of replication operations upon the second storage object is serialized according to the order.

Example implementations relate to metadata operations in a storage system. An example method includes receiving, from a first stream process, a first write request for a first container index in memory. The method further includes, in response to a receipt of the first write request, sending a first token to the first stream process without writing the first container index to a persistent storage. The method further includes receiving, from a second stream process, a first completion request for the first container index. The method further includes, in response to a receipt of the first completion request, writing the first container index from the memory to the persistent storage.

A method for mapping LUNs (logical unit numbers) in storage memory, performed by a storage system, is provided. The method includes determining a set of LUNs in the storage memory and generating a mapping from a logical address space to all of the LUNs in the set, based on the determining, so that each logical address in the logical address space maps to one LUN in the set. The method includes accessing one or more of the LUNs in accordance with the mapping.