Provided are techniques for role reversal of primary and secondary sites with minimal replication delay. During a backup phase, an Input/Output (I/O) operation selected from a group comprising a create operation and a rename operation is received with one or more local attributes of a local file. The I/O operation is processed to generate one or more remote attributes. The one or more local attributes are stored with the one or more remote attributes. During a reversed phase, a write operation for the remote file is processed, and, the write operation is sent to the primary replication system with the one or more local attributes, where the primary replication system uses a local file handle in the one or more local attributes to access the local file for processing the write operation to sync the local file with the remote file.

File system management is disclosed. File system metadata is stored in local memory of a client. The portion of local memory storing the file system metadata is under control of the metadata server. The construction and management of the metadata stored in the local memory of the client is driven, initiated and managed by the remote metadata server.

Provided are techniques for role reversal of primary and secondary sites with minimal replication delay. During a backup phase, an Input/Output (I/O) operation selected from a group comprising a create operation and a rename operation is received with one or more local attributes of a local file. The I/O operation is processed to generate one or more remote attributes. The one or more local attributes are stored with the one or more remote attributes. During a reversed phase, a write operation for the remote file is processed, and, the write operation is sent to the primary replication system with the one or more local attributes, where the primary replication system uses a local file handle in the one or more local attributes to access the local file for processing the write operation to sync the local file with the remote file.

A high frequency snapshot technique improves data replication in a disaster recovery (DR) environment. A base snapshot is generated from failover data at a primary site and replicated to a placeholder file at a secondary site. Upon commencement of the base snapshot generation and replication, incremental light weight snapshots (LWSs) of the failover data are captured and replicated to the secondary site. A staging file at the secondary site accumulates the replicated LWSs (“high-frequency snapshots”). The staging file is populated with the LWSs in parallel with the replication of the base snapshot at the placeholder file. At a subsequent predetermined time interval, the accumulated LWSs are synthesized to capture a “checkpoint” snapshot by applying and pruning the accumulated LWSs at the staging file. Once the base snapshot is fully replicated, the pruned LWSs are merged to the base snapshot to synchronize the replicated failover data.

The present technology pertains to a organization directory hosted by a synchronized content management system. The corporate directory can provide access to user accounts for all members of the organization to all content items in the organization directory on the respective file systems of the members' client devices. Members can reach any content item at the same path as other members relative to the organization directory root on their respective client device. In some embodiments novel access permissions are granted to maintain path consistency.

In some embodiments, during synchronizing of files in a source data set to a destination data set, a method receives a set of events that occurred at the source data set after replicating an image of the source data set to the destination data set. The method analyzes the set of events to determine if an exception to a first set of rules for performing a set of operators on the destination data set for the set of events occurs. A second set of rules for the exception is selected based on analyzing the set of events. The method processes the set of operators for the set of events according to the second set of rules to synchronize data from the first data set to the second data set based on the set of events. The processing of the set of operators uses the second set of rules.

One example method includes receiving a write request that includes a data structure version to be written, wherein the data structure version is associated with a unique identifier, storing the data structure version in association with the unique identifier, receiving a read request for a most recent version of the data structure and, when the stored data structure version is not the most recent version of the data structure, examining respective unique identifiers of each of a group of other stored data structure versions to determine which stored data structure version is the most recent. Finally, the example method includes returning the most recent data structure version, notwithstanding that one or more other data structure versions existed at the time that the read request was received.

Techniques are provided for dependency aware parallel splitting of operations. For example, a count of pending data operations being executed by a first node and replicated in parallel to a second node are tracked. A metadata operation is executed at the first node based upon the count being less than a threshold (e.g., the count being zero). A first list of affected inodes modified by the metadata operation is identified. A dependency of the metadata operation with respect to pending metadata operations replicated to the second node is determined. The metadata operation is dispatched to the second node based upon the dependency indicating that the metadata operation is independent of the pending metadata operations.

Presented herein are methods of replicating versioned and hierarchical data structures, as well as data structures representing complex transactions. Due to interdependencies between data entities and a lack of guaranteed message ordering, simple replication methods employed for simple data types cannot be used. Operations on data structures exhibit dependencies between the messages making up the operations. This strategy can be extended to various types of complex transactions by considering certain messages to depend on other messages or on the existence of other entries at the data store. Regardless of origin, these dependencies can be enforced by suspending the processing of messages with unsatisfied dependencies until all of its dependencies have been met. Alternately, transactions can be committed immediately, creating entities that include versioned identifiers for each of their dependencies. These entities can then be garbage collected of the parent objects are not subsequently created.

Distributed ledgered data is stored within a distributed persistent storage system comprising multiple persistent storage systems as distributed ledgered participants. In various embodiments, the distributed ledgered data is maintained using the native capabilities of a persistent storage system. The distributed ledgered data is replicated as persistent data objects in a “ledgered repository of objects” that are replicated at each of the persistent storage systems. Changes at one persistent storage system are recorded within a block in a distributed blockchain that is distributed across each of the other distributed ledgered participants. The other distributed ledgered participants read the changes from the blockchain and apply the changes to the respective replicas at each of the other distributed ledgered participants. Hence, this approach is referred to as blockchain apply. Blockchain apply may be used to replicate the repository objects of various forms of PSSs. In a DBMS, a repository of objects is a table, where each record or row is an object in the repository. In a file system, a repository of objects is a directory, where each directory and file therein is an object in the repository. In a document storage system (DOCS), a repository of objects is a collection of documents, where each document is an object in the repository.