In connection with a data distribution architecture, client-side “deduplication” techniques may be utilized for data transfers occurring among various file system nodes. In some examples, these deduplication techniques involve fingerprinting file system elements that are being shared and transferred, and dividing each file into separate units referred to as “blocks” or “chunks.” These separate units may be used for independently rebuilding a file from local and remote collections, storage locations, or sources. The deduplication techniques may be applied to data transfers to prevent unnecessary data transfers, and to reduce the amount of bandwidth, processing power, and memory used to synchronize and transfer data among the file system nodes. The described deduplication concepts may also be applied for purposes of efficient file replication, data transfers, and file system events occurring within and among networks and file system nodes.

The present technology pertains to a organization directory hosted by a synchronized content management system. The corporate directory can provide access to user accounts for all members of the organization to all content items in the organization directory on the respective file systems of the members' client devices. Members can reach any content item at the same path as other members relative to the organization directory root on their respective client device. In some embodiments novel access permissions are granted to maintain path consistency.

In connection with a data distribution architecture, client-side “deduplication” techniques may be utilized for data transfers occurring among various file system nodes. In some examples, these deduplication techniques involve fingerprinting file system elements that are being shared and transferred, and dividing each file into separate units referred to as “blocks” or “chunks.” These separate units may be used for independently rebuilding a file from local and remote collections, storage locations, or sources. The deduplication techniques may be applied to data transfers to prevent unnecessary data transfers, and to reduce the amount of bandwidth, processing power, and memory used to synchronize and transfer data among the file system nodes. The described deduplication concepts may also be applied for purposes of efficient file replication, data transfers, and file system events occurring within and among networks and file system nodes.

An example file server manager updates a selected share of a destination distributed file server based on a snapshot of at least a portion of a selected share of a source distributed file server. The selected share of the destination distributed file server is updated while the source distributed file server serves client requests for storage items of the selected share of the source distributed file server. The file server manager receives a request to failover from the source distributed file server to the destination distributed file server and configures the destination distributed file server to service read and write requests for storage items of the selected share of the destination distributed file server. The file server manager further redirects client requests for storage items of the selected share of the source distributed file server to the destination distributed file server by updating active directory information.

Presented herein are methods of replicating versioned and hierarchical data structures, as well as data structures representing complex transactions. Due to interdependencies between data entities and a lack of guaranteed message ordering, simple replication methods employed for simple data types cannot be used. Operations on data structures exhibit dependencies between the messages making up the operations. This strategy can be extended to various types of complex transactions by considering certain messages to depend on other messages or on the existence of other entries at the data store. Regardless of origin, these dependencies can be enforced by suspending the processing of messages with unsatisfied dependencies until all of its dependencies have been met. Alternately, transactions can be committed immediately, creating entities that include versioned identifiers for each of their dependencies. These entities can then be garbage collected of the parent objects are not subsequently created.

Techniques and structures to synchronize hierarchical data, comprising. A conflict between data stored at a computing device and data stored at a service provider database is detected. A synchronization operation is performed to synchronize a plurality of objects within the hierarchical entity data stored at the computing device with hierarchical entity data stored at the service provider database.

Providing continuous data protection includes maintaining a database having substantially all data modifications made to a primary volume over a recovery interval. The database is maintained in conjunction with a copying operation where the data of the primary volume are mirrored to a remote volume to permit recovery of mirrored data in the event of loss of primary volume data. The contents of the remote volume generally lag behind the contents of the primary volume by substantially the recovery interval. Providing continuous data protection also includes providing data roll-back to a precise point in time within the recovery interval by applying, to the contents of the remote volume, all data modifications in the database that occurred between the latest data modification to the remote volume and the precise point in time within the recovery interval. A time stamp mechanism of sufficient precision and granularity may be used.

Expandable cache management dynamically manages cache storage for multiple network shares configured in a file server. Once a file is written to a directory or folder on a specially designated network share, such as one that is configured for “infinite backup,” an intermediary pre-backup copy of the file is created in an expandable cache in the file server that hosts the network share. On write operations, cache storage space can be dynamically expanded or freed up by pruning previously backed up data. This advantageously creates flexible storage caches in the file server for each network share, each cache managed independently of other like caches for other network shares on the same file server. On read operations, intermediary file storage in the expandable cache gives client computing devices speedy access to data targeted for backup, which is generally quicker than restoring files from backed up secondary copies.

Techniques are described for providing a multi-service storage layer in a cloud provider network for applications and workloads that are highly sensitive to outages affecting “mission critical” data or other resources. A multi-service storage layer is designed to provide additional resiliency against various types of correlated failures among existing geographic regions by enabling the storage of data using a plurality of separate storage services and storage resource types and across a plurality of regions of the cloud-provider network. A multi-service storage layer provides an application programming interface (API) with actions for storing, retrieving, and querying data stored in a highly available storage resource across a selection of underlying storage services. The ability to store critical data and other resources using a multi-service storage layer provides additional resiliency while also providing a single interface that applications can use to interact with the data stored across any number of underlying storage services.

In some embodiments, a system for synchronizing content with client devices receives a request from a client device to synchronize operations pertaining to content items associated with a user account registered at the system. The request can include the operations and a cursor identifying a current position of the client in a journal of revisions on the system. Based on the operations, the system generates linearized operations associated with the content items. The linearized operations can include a respective operation derived for each of the content items from one or more of the operations. The system converts each respective operation in the linearized operations to a respective revision for the journal of revisions and, based on the cursor, determines whether the respective revision conflicts with revisions in the journal. When the respective revision does not conflict with revisions in the journal, the system adds the respective revision to the journal.