An apparatus comprises at least one processing device configured to identify a snapshot lineage comprising snapshots of a storage volume, the snapshot lineage comprising (i) a local snapshot lineage stored on a storage system and (ii) cloud snapshot lineages stored on cloud storage external to the storage system, to select at least one snapshot that is to be copied from the local snapshot lineage, to determine at least two of the cloud snapshot lineages as destinations for the selected snapshot, to generate a snapshot copy job for copying the selected snapshot to the at least two cloud snapshot lineages, and to process the snapshot copy job by reading data of the selected snapshot stored in the local snapshot lineage once and writing the data of the selected snapshot to the at least two cloud snapshot lineages.

Systems and methods including a framework for migration of live data. The method may comprised, by one or more hardware processors executing program instructions, receiving, at a migration proxy of the framework, code for reading data and writing data compatible with each of a plurality of states of a migration of data in a data store, wherein a service is at least intermittently reading data from and writing data to the data store; determining, by a migration runner of the framework, to perform the migration of the data; initiating, by the migration runner, the migration of the data, wherein the migration comprises a plurality of stages; storing, as the migration progresses through the plurality of stages, and at a migration data store of the framework, a current stage of the migration; and during the migration, using the migration proxy to read data from and write data to the data store.

Data recovery systems and methods utilize object-based storage for providing a data protection and recovery methodology with low recovery point objectives, and for enabling both full recovery and point-in-time based recovery. Data generated at a protected site (e.g., via one or more virtual machines) is intercepted during write procedures to primary storage. The intercepted data is replicated via a replication log, provided as data objects, and transmitted to an object based storage system. During recovery, data objects may be retrieved through point-in-time based recovery directly by the systems of the protected site, and/or data objects may be provided via full recovery, for example, within a runtime environment of a recovery site, with minimal data loss and operation interruption by rehydrating data objects within the runtime environment via low-latency data transfer and rehydration systems.

An interface for requesting, and technique for generation of, a backup of a past state of a database table are provided. Changes made to a database table are accumulated, in durable storage, and snapshots of partitions of the table are obtained. The accumulated changes and the successive partition snapshots are used to generate a past state of the database at any point in time across a continuum between successive snapshots. Although each partition of the table may have a snapshot that was generated at a time different from when other partition snapshots were generated, changes from respective change logs may be selectively log-applied to distinct partitions of a table to generate backup in the past of the entire table at common point-in-time across partitions.

Methods and systems are provided for rapid failure recovery for a distributed storage system for failures by one or more nodes.

A computer-implemented system for dynamic aggregation of data and minimization of data loss is disclosed. The system may be configured to perform instructions for: aggregating information from a plurality of networked systems by collecting a set of data from the networked systems, the set of data comprising data associated with a predetermined period of time and comprising one or more central variables that are included in data associated with more than one networked systems of the plurality of networked systems and one or more associated variables that describe one or more aspects of the central variables; retrieving one or more data transformation rules based on a relational map among the central variables and the associated variables; and aggregating the first set of data into one or more master data structures corresponding to the central variables based on the data transformation rules.

A method for synchronous replication of stream data includes receiving a stream of data blocks for storage at a first storage location associated with a first geographical region and at a second storage location associated with a second geographical region. The method also includes synchronously writing the stream of data blocks to the first storage location and to the second storage location. While synchronously writing the stream of data blocks, the method includes determining an unrecoverable failure at the second storage location. The method also includes determining a failure point in the writing of the stream of data blocks that demarcates data blocks that were successfully written and not successfully written to the second storage location. The method also includes synchronously writing, starting at the failure point, the stream of data blocks to the first storage location and to a third storage location associated with a third geographical region.

A system includes a multi-process application that runs. A multi-process application runs on primary hosts and is checkpointed by a checkpointer comprised of at least one of a kernel-mode checkpointer module and one or more user-space interceptors providing at least one of barrier synchronization, checkpointing thread, resource flushing, and an application virtualization space. Checkpoints may be written to storage and the application restored from said stored checkpoint at a later time. Checkpointing may be incremental using Page Table Entry (PTE) pages and Virtual Memory Areas (VMA) information. Checkpointing is transparent to the application and requires no modification to the application, operating system, networking stack or libraries. In an alternate embodiment the kernel-mode checkpointer is built into the kernel.

A method, computer program product, and computer system are provided. A message storage area of an adjunct processor (AP) crypto adapter is filled with a plurality of command request messages sufficient to maximize utilization and performance of the AP crypto adapter. In response to detecting an error during execution of one of the plurality of command request messages, generating an AP crypto adapter command reply message. The AP crypto adapter command reply message includes the error. In response to the error being a non-recoverable failure, determining a state of the command request message, wherein the state of the command request message comprises an in-process state or a request-pending state. The AP crypto adapter command reply message is formatted, wherein the formatted AP crypto adapter command reply message is stored in a message queue in the AP crypto adapter pending completion of machine failure recovery. The AP crypto adapter is recovered.

The subject technology determines, using a connection to an external data source, a set of shards stored in an external data source, the connection to the external data source being established using an external integration, the external integration including security and configuration information. The subject technology determines a set of offsets of each shard of the set of shards. The subject technology generates a query plan indicating a degree of parallelism based at least in part on a size of the set of offsets. The subject technology, based on the set of shards and the set of offsets, performs an operation on the external data source by performing, using the connection to the external data source, a write operation from a query statement on the external data source, the external data source being different than a storage platform associated with the system.