A method and system for backing up and restoring data includes: determining, at a server storage area network (SAN) node, an LUN required for protection within a server SAN node based on communication between a backup agent and a logic unit number (LUN) agent in the server SAN node; determining a target backup device based on communication between the backup agent and respective target agent of a plurality of backup devices, the plurality of backup devices being connected with the server SAN node; and transmitting a virtual disk control command to the target backup device in response to the backup or restoration being triggered, such that data in the LUN required for protection is backed up to the target backup device or restored from the target backup device via an activated virtual disk interface of the target backup device.

Determining active membership among a set of storage systems, including: determining, by a cloud-based storage system among the set of storage systems, that a membership event corresponds to a change in membership to the set of storage systems synchronously replicating the dataset; applying, in dependence upon the membership event, one or more membership protocols to determine a new set of storage systems to synchronously replicate the dataset; and for one or more I/O operations directed to the dataset, applying the one or more I/O operations to the dataset synchronously replicated by the new set of storage systems.

Techniques for determining replication modes may include: issuing, while unsolicited data transfer mode is enabled for a first link, first write operations over the first link; issuing, while unsolicited data transfer mode is disabled for the first link, second write operations over the first link; determining a first performance metric for the first link in accordance with the first write operations; determining a second performance metric for the first link in accordance with the second write operations; and determining, in accordance with the first and second performance metrics whether to enable or disable unsolicited data transfer mode for the first link for a first time period. The first and second performance metrics may be response times. Unsolicited data transfer mode, when enabled, allows write data payload to be sent to a target without receiving an acknowledgement regarding receipt of a write command block for the write operation from the target.

Replicated instances in a distributed computing environment provide for automatic failover and recovery. A component monitors the status of event processors in a set or bucket and handles the failure of an event processor. For a large number of instances, the data environment can be partitioned such that each monitoring component is assigned a partition of the workload. At intervals, each event processor sends a “heartbeat” message to the event processors in the bucket covering the same workload partition, to inform the other event processors of the status of the event processor sending the heartbeat. If it is determined that a heartbeat is received from each event processor in the bucket, a current process can continue. In the event of monitoring component failure, the instances can be repartitioned, and the remaining monitoring components can be assigned to the new partitions to substantially evenly distribute the workload.

A scale-out storage system includes a plurality of computer nodes each of which has a memory and a processor, and a storage apparatus. The computer nodes have one or more redundancy groups each of which is a group for metadata protection. Each of the one or more redundancy groups includes two or more of the computer nodes including a primary node being a primary computer node and a secondary node being a secondary computer node, and a failover is performed from the primary node to the secondary node. The memory of the primary node has stored therein metadata related to the redundancy group and to be accessed for control. The metadata is redundantly stored in the memory of the primary node and the memory of the secondary node.

In an example embodiment, a solution is used to provide container volume replication via a container storage replication log and volume buffer synchronization, which is built on top of a container cloud platform whose container metadata and replication runtime configuration are all managed by a storage manager (a service orchestrated by its job scheduler and service orchestrator). This container volume replication ensures the data security for a long-running service in the container. In the case of any disaster, the in-memory database and application data inside of the container can be recovered via volume replication. This provides container volume replication for long-running containerized applications whose states keep changing.

Methods and systems for using a hierarchical consistency group (CG) in a storage system are provided. A parent CG is associated with at least a first child CG having a plurality of storage volumes. An atomic application programming interface (API) provisions the parent CG and the first child CG by allocating storage and storing policies for the parent CG and the first CG. A storage service selected from a backup service, a replication service and a cloning service for the parent CG and the first CG is executed based on the stored policies.

In an example embodiment, a solution is used to provide container volume replication via a container storage replication log and volume buffer synchronization, which is built on top of a container cloud platform whose container metadata and replication runtime configuration are all managed by a storage manager (a service orchestrated by its job scheduler and service orchestrator). This container volume replication ensures the data security for a long-running service in the container. In the case of any disaster, the in-memory database and application data inside of the container can be recovered via volume replication. This provides container volume replication for long-running containerized applications whose states keep changing.

A storage node that maintains a replica of a logical volume for use in response to a failover trigger includes a data node with volatile memory in which a filesystem and its metadata and a VDM and its metadata associated with the replica are maintained prior to the failover trigger. The storage node also includes a SAN node in which data associated with the replica is maintained. The data is maintained in a RW (read-write) state by the SAN node prior to the failover trigger. However, the replica is presented in a RO (read-only) state by the storage node prior to the failover trigger. The storage node changes the in-memory state of the filesystem and VDM to RW responsive to the failover trigger. Because the filesystem and its metadata and VDM and its metadata are already in memory and the data is in a RW state in block storage the failover is completed relatively quickly.

Methods and systems for using a hierarchical consistency group (CG) in a storage system are provided. A parent CG is associated with at least a first child CG having a plurality of storage volumes. An atomic application programming interface (API) provisions the parent CG and the first child CG by allocating storage and storing policies for the parent CG and the first CG. A storage service selected from a backup service, a replication service and a cloning service for the parent CG and the first CG is executed based on the stored policies.