This invention is intended to process data synchronization more efficiently. Disclosed is a data synchronization system comprising: an all records fetching unit that fetches all records of synchronization target data, i.e., data specified as a target of synchronization, from a first device that is a source of synchronization; one or more storage units to prestore synchronization destination data, namely, data that is now retained on a second device that is a destination of synchronization and store synchronization target data fetched by the all records fetching unit; and a difference extraction unit that identifies difference to be reflected in the data on the second device by using the synchronization destination data and the synchronization target data, makes identified difference reflected in the data on the second device, and, after the reflection, updates the synchronization destination data based on the synchronization target data.

A system, a method and a computer program product for storing data, which include receiving a data stream having a plurality of transactions that include at least one portion of data, determining whether at least one portion of data within at least one transaction is substantially similar to at least another portion of data within at least one transaction, clustering together at least one portion of data and at least another portion of data within at least one transaction, selecting one of at least one portion of data and at least another portion of data as a representative of at least one portion of data and at least another portion of data in the received data stream, and storing each representative of a portion of data from each transaction in the plurality of transactions, wherein a plurality of representatives is configured to form a chain representing the received data stream.

Techniques for replicating data involve: acquiring a first snapshot of a data block set, the first snapshot being a snapshot before a first subset of the data block set starts to be replicated; acquiring a second snapshot of the data block set, the second snapshot being a snapshot of the data block set when replication of the first subset is completed; and determining, based on a difference between the second snapshot and the first snapshot, a second subset of the data block set, the second subset being different from the first subset. Accordingly, such techniques can improve data protection efficiency in asynchronous replication.

Techniques are provided for dependency aware parallel splitting of operations. For example, a first operation and a second operation may be replicated in parallel from a first device to a second device if the operations only target a single common inode that is an access control list inode referenced by the operations. An operation that dereferences the access control list inode can be replicated in parallel with other operations if the operation does not have the potential to delete the access control list inode from the second device. In another example, operations may be replicated to the second device in parallel if the operations only affect a single common parent directory inode and where timestamps are only moved forward in time at the second device.

A data handling or patching system operates to reduce redundant data within a data storage. Data files are compared, and a data difference is output. In files that have very similar data, the data difference between values is relatively small, and thus when a basic data file and a data difference file are stored, data redundancy is significantly reduced. The data difference of “diff” file can then be used to losslessly reproduce the data of either of the compared files. A first or primary port is dedicated for base quality transmissions on which the broadcasting client sends the lowest available quality. Additional or secondary ports enable transmission of higher than base quality data, sent as “diff's” of the base quality. The receiving client combines the “diff” data on the secondary ports with the base data of the first port, to produce higher quality media.

Methods are disclosed for providing alternate reality (e.g., virtual reality) representations to users. Exemplary methods employ data collections (e.g., stacks) which affect the virtual representations of baseline virtual models. Data collections contain layers which contain deltas. The deltas specify modifications to the baseline virtual reality world or model. The deltas may be geocoded, while the layers that contain them may not be geocoded. Separately selectable layers are used to temporarily modify or substitute baseline data or virtual elements (e.g., virtual objects) that are ultimately presented to a user on an output device. Conflict resolution algorithms harmonize conflicts between layers of a collection.

Techniques and mechanisms for incremental data ingestion are disclosed. Raw data is received from multiple disparate sources to be consumed in an environment for collecting unformatted raw data. The environment has at least a delta data table and a delta notification table. A write to an entry in the delta data table is attempted. Entries to the delta data table specify at least records indicating changes to objects in the environment. A write a corresponding entry to the delta notification table is attempted in response to a successful write attempt to the delta data table. The delta notification table entry includes information about delta data table entries for a specified period. At least one data consumer is notified that the delta data table has been modified.

A mechanism is provided for storing data files in a file system. The file system provides a plurality of reference data files, where each reference data file in the plurality of data files represents a group of similar data files. The mechanism creates a new data file and associated the new data file with one reference data file in the plurality of data files thus defining an associated reference data file of the plurality of reference data files. The mechanism informs the file system about the association of the new data file with the associated reference data file. The mechanism compresses the new data file using the associated reference data file thereby forming a compressed data file. The mechanism stores the compressed data file together with information about the association of the new data file with the associated reference data file.

Embodiments of the present invention relate to a method, system and computer program product for compacting data in a distributed storage system. According to the method, a query request is received from a client, wherein the query request comprises information of a previous query request of the client. A first query result of the query request and a second query result of the previous query request are retrieved. A delta data and a first indicator are sent to the client in response to the first query result being different from the second query result, wherein the delta data is the data of the first query result excluding the data comprised in both first and second query results, and the first indicator indicates the data comprised in both the first and the second query results.

A system, a method, and a computer program product for performing deduplication of data using a scalable deduplication grid are disclosed. A listing of a plurality of zone stamps is generated, where each zone stamp represents a zone in the plurality of zones in a data stream. The listing contains a logical arrangement of the plurality of zone stamps obtained from each storage location and being accessible by a plurality of servers. A first zone stamp in the listing is compared to a second zone stamp in the listing. The first and second zones are delta-compressed based on a determination that the first zone stamp is substantially similar to the second zone stamp. A server is selected to perform the comparison and delta-compression.