Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.

A method of performing partial redundant array of independent disks (RAID) stripe parity calculations is disclosed. The method includes receiving a last portion of a RAID stripe among multiple portions of the RAID stripe, all portions for a successful write of the RAID stripe being previously received except for the last portion. The method also includes calculating a parity value based on the last portion of the RAID stripe and a previous parity value without calculating the parity value using a previous portion of the RAID stripe. The method further includes writing of the RAID stripe.

Physical storage devices (PSDs) of a protection group cluster (PGC) may be represented by a protection group matrix (PGM) having a plurality of rows and a plurality of columns, where each row corresponds to a PSD of the PGC, and each column corresponds to a partition of each PSD. The value specified in each cell at an intersection of a row and column specifies the protection group of the PGC to which the partition of the PSD represented by the column and row, respectively, is (or will be) assigned. In response to one or more of PSDs being added to a PGC, the PGM may be reconfigured, including adding new rows, and transposing portions of columns to the new rows, or transposing portions of rows to portions of columns of the new rows. Protection members of the PGC may be re-assigned based on the reconfiguration.

Systems and methods for supporting dynamic disk growth within a virtual storage appliance are provided. According to one embodiment, a portion of a logical size of respective hyperscale disks provided by a hyperscaler are provisioned for use by a virtual storage system as backing for respective file system disks. To accommodate growth, block numbers for the file system disks are pre-allocated within a sparse space of a contiguous sequence of block numbers corresponding to a number of blocks represented by the logical size. Metadata is maintained for the file system disks regarding a range of the pre-allocated block numbers that are available for use. Responsive to a triggering condition, the provisioned portion of a hyperscale disk is increased and subsequently, responsive to detecting a change in a size of the hyperscale disk by the virtual storage system, a size of the corresponding file system disk is updated within the metadata.

Techniques for provisioning storage may include: initially provisioning storage for a storage group of logical devices; tagging the storage group to enable autonomous storage provisioning; receiving a plurality of parameters used in connection with performing autonomous storage provisioning for the storage group, wherein the plurality of parameters includes a first parameter denoting a threshold amount of consumed storage of the storage group, a second parameter denoting a storage capacity expansion amount by which to expand the storage capacity of the storage group, and a third parameter denoting a system-wide threshold of consumed backend non-volatile storage; determining, in accordance with the plurality of parameters, whether to expand a current storage capacity of the storage group; and responsive to determining to expand the current storage capacity of the storage group, performing first processing to automatically expand the current storage capacity of the storage group in accordance with the second parameter.

A technique for correcting errors in a data storage system operates while the data storage system remains online. The technique includes identifying an object for validation, scanning a plurality of pointers, and counting a number of pointers that point to the object. The technique further includes repairing a discrepancy between the count of pointers and a reference count stored in connection with the object.

Provided are a cluster arbitration method and system based on heterogeneous storage, and a device and a storage medium. The method comprises: configuring heartbeat between every two adjacent nodes in a cluster (S1); in response to the occurrence of disconnection of the heartbeat between nodes, dividing the nodes in the cluster into a plurality of sub-clusters on the basis of the disconnected heartbeat, and determining whether a sub-cluster with the largest number of nodes is unique (S2); in response to the sub-cluster with the largest number of nodes not being unique, selecting a node from each sub-cluster with the largest number of nodes and also sending a reservation request to a third-party storage logical volume (S3); in response to there being a successfully reserved node, determining whether arbitration data in the third-party storage logical volume is valid (S4); and in response to the arbitration data in the third-party storage logical volume being invalid, writing arbitration information of the successfully reserved node into the third-party storage logical volume, such that the sub-cluster, where the node is located, obtains a control right of the cluster (S5). By means of an existing configuration, storage cluster arbitration is realized without occupying additional network bandwidths.

Managing input/output (‘I/O’) queues in a data storage system, including: receiving, by a host that is coupled to a plurality of storage devices via a storage network, a plurality of I/O operations to be serviced by a target storage device; determining, for each of a plurality of paths between the host and the target storage device, a data transfer maximum associated with the path; determining, for one or more of the plurality of paths, a cumulative amount of data to be transferred by I/O operations pending on the path; and selecting a target path for transmitting one or more of the plurality of I/O operations to the target storage device in dependence upon the cumulative amount of data to be transferred by I/O operations pending on the path and the data transfer maximum associated with the path.

Storage devices can be configured to desirably reduce the number of times a zone reset or erasure occur via the use of one or more paired overwrite memory blocks. These storage devices can include a plurality of memory devices with some of these memory devices designated as overwrite memory devices. A controller within the storage device can be configured to direct the storage device to generate one or more subsets within the memory devices such as zones, pair each of subsets with at least one or more overwrite memory devices, store data sequentially within the subset of memory devices, and store any received overwrite data in the overwrite memory devices in chronological order. Data stored within the subsets of memory devices are not erased and instead of being overwritten directly, are instead pointed via a control table to a location in the overwrite memory devices storing the corresponding overwrite data.

A system and method is disclosed for managing data in a non-volatile memory. The system may include a non-volatile memory having multiple non-volatile memory sub-drives. A controller of the memory system is configured to route incoming host data to a desired sub-drive, keep data within the same sub-drive as its source during a garbage collection operation, and re-map data between sub-drives, separate from any garbage collection operation, when a sub-drive overflows its designated amount logical address space. The method may include initial data sorting of host writes into sub-drives based on any number of hot/cold sorting functions. In one implementation, the initial host write data sorting may be based on a host list of recently written blocks for each sub-drive and a second write to a logical address encompassed by the list may trigger routing the host write to a hotter sub-drive than the current sub-drive.