SYSTEM AND METHOD FOR SYNCHRONIZING AND RECONCILING DATA FROM EDGE NODE TO CLOUD NODE

Abstract

A processor-implemented method for synchronizing and reconciling data stream in real-time between an edge node and a cloud node is provided. The method includes (i) generatin classified data by classifying data of the data stream that is obtained from a plurality of edge nodes in real-time to synchronize in a location at a plurality of cloud nodes, (ii) synchronizing the classified data in the plurality of cloud nodes by detecting a first location and a type of database in the plurality of cloud nodes and storing the classified data in the first location in the plurality of cloud, (iii) replicating the classified data in a plurality of second locations in the plurality of cloud nodes, and (iv) receiving, from the plurality of edge nodes, a request signal for synchronizing the classified data from the plurality of cloud nodes to the plurality of edge nodes when the first location in the plurality of cloud nodes fails to transfer the classified data during a transaction.

Claims

1. A processor-implemented method for synchronizing and reconciling data stream in real-time between an edge node and a cloud node, the method comprising: generating classified data by classifying data [[for]] of the data stream that is obtained from a plurality of edge nodes in real-time to synchronize in a location at a plurality of cloud nodes; queuing the classified data associated with the plurality of edge nodes and tagging the classified data with (i) a time stamp of the data, (ii) a location of the plurality of edge nodes, (iii) an edge identifier of the plurality of edge nodes, and (iv) an internet protocol address; automatically determining a frequency count to synchronize the classified data associated with a real-time execution at the plurality of edge nodes with the plurality of cloud nodes regularly, wherein the frequency count is determined based on a quantity of the data obtained from the plurality of edge nodes for a predefined time period; synchronizing the classified data associated with the plurality of edge nodes in the plurality of cloud nodes by detecting a first location and a type of database in the plurality of cloud nodes and storing the classified data in the first location in the plurality of cloud nodes; replicating the classified data associated with the plurality of edge nodes in a plurality of second locations in the plurality of cloud nodes; receiving from the plurality of edge nodes, a request signal for synchronizing the classified data from the plurality of cloud nodes to the plurality of edge nodes when the first location in the plurality of cloud nodes fails to transfer the classified data during a transaction; upon receiving, from the plurality of cloud nodes, a response signal comprising the plurality of second locations at which the classified data is stored in the plurality of cloud nodes, updating a synchronization table in the plurality of edge nodes with the plurality of second locations at which the classified data is stored in the plurality of cloud nodes; and synchronizing the classified data from the plurality of second locations in the plurality of cloud nodes to the plurality of edge nodes without duplicating the data based on the synchronization table that is updated when the request signal is received from the plurality of edge nodes.

2. The processor-implemented method of claim 1, wherein the data stream comprises at least one of numerical data, string-based data, or image data.

3. The processor-implemented method of claim 1, wherein the frequency count is automatically determined based on (i) an amount of data, (ii) a location of the plurality of edge nodes, and (iii) a type of application running in the plurality of edge nodes.

4. The processor-implemented method of claim 1, wherein when the synchronization table is not synchronized then the data from the plurality of edge nodes is synchronized in a next cycle, wherein a cycle is a time period computed to synchronize the data and is pre-determined by the user.

5. The processor-implemented method of claim 1, wherein each edge node is synchronized with the synchronization table to synchronize the data obtained from the real-time execution at the plurality of edge nodes.

6. The processor-implemented method of claim 1, wherein the synchronization table stored in each edge node comprises metadata of that edge node, wherein the metadata comprises the location of each cloud node, frequency count, wherein the size of the metadata is in a range of Kilobytes (Kb) to 10 Kb.

7. The processor-implemented method of claim 1, wherein the response signal comprises a location of the data in the plurality of cloud nodes.

8. A system for synchronizing and reconciling data stream in real-time between an edge node and a cloud node, the system comprises: a memory that stores a set of instructions; a processor that is configured to execute the set of instructions and is configured to generate classified data by classifying data of the data stream that is obtained from a plurality of edge nodes in real-time to synchronize in a location at a plurality of cloud nodes; queue the classified data associated with the plurality of edge nodes and tagging the classified data with (i) a time stamp of the data, (ii) a location of the plurality of edge nodes, (iii) an edge identifier of the plurality of edge nodes, and (iv) an internet protocol address; automatically determine a frequency count to synchronize the classified data associated with a real-time execution at the plurality of edge nodes with the plurality of cloud nodes regularly, wherein the frequency count is determined based on a quantity of the data obtained from the plurality of edge nodes for a predefined time petiod; synchronize the classified data associated with the plurality of edge nodes in the plurality of cloud nodes by detecting a first location and a type of database in the plurality of cloud nodes and store the classified data in the first location in the plurality of cloud nodes; replicate the classified data associated with the pluralit of edge nodes in a plurality of second locations in the plurality of cloud nodes; receive a request signal from the plurality of edge nodes for synchronizing the classified data from the plurality of cloud nodes to the plurality of edge nodes when the first location in the plurality of cloud nodes fails to transfer the classified data durin a transaction; upon receiving a response signal comprising the plurality of second locations at which the classified data is stored in the plurality of cloud nodes from the plurality of cloud nodes, update a synchronization table in the plurality of edge nodes with the plurality of second locations at which the classified data is stored in the plurality of cloud nodes; and synchronize the classified data from the plurality of second locations in the plurality of cloud nodes to the plurality of edge nodes without duplicating the data based on the synchronization table that is updated when the request signal is received from the plurality of edge nodes.

9. The system of claim 8, wherein the data stream comprises at least one of numerical data, sting-based data, or image data.

10. The system of claim 8, wherein the frequency count is automatically determined based on (i) an amount of data, (ii) a location of the plurality of edge nodes, and (iii) a type of application running in the plurality of edge nodes.

11. The system of claim 8, wherein when the synchronization table is not synchronized then the data from the plurality of edge nodes is synchronized in a next cycle, wherein a cycle is a time period computed to synchronize the data and is pre-determined by the user.

12. The system of claim 8, wherein each edge node is synchronized with the synchronization table to synchronize the data obtained from the real-time execution at the plurality of edge nodes.

13. The system of claim 8, wherein the synchronization table stored in each edge node comprises metadata of that edge node, wherein the metada.ta comprises the location of each cloud node, frequency count, wherein the size of the metadata is in a range of 1 Kilobytes (Kb) to 10 Kb.

14. The system of claim 8, wherein the response signal comprises a location of the data in the plurality of cloud nodes.

15. One or more non-transitory computer-readable storage mediums storing one or more sequences of instructions, which when executed by one or more processors, causes a method for synchronizing and reconciling data stream in real-time between an edge node and a cloud node performing steps of generating classified data by classifying data of the data stream that is obtained from a plurality of edge nodes in real-time to synchronize in a location at a plurality of cloud nodes; queuing the classified data associated with the plurality of edge nodes-and tagging the classified data with (i) a time stamp of the data, (ii) a location of the plurality of edge nodes, (iii) an edge identifier of the plurality of edge nodes. and (iv) an internet protocol address; automatically determining a frequency count to synchronize the classified data associated with a real-time execution at the plurality of edge nodes with the plurality of cloud nodes regularly, wherein the frequency count is determined based on a quantity of the data obtained from the plurality of edge nodes for a predefined time period; synchronizing the classified data associated with the plurality of edge nodes in the plurality of cloud nodes by detecting a first location and a type of database in the plurality of cloud nodes and storing the classified data in the first location in the plurality of cloud nodes; replicating the classified data associated with the plurality of edge nodes in a plurality of second locations in the plurality of cloud nodes: receiving, from the plurality of edge nodes, a request signal for synchronizing the classified data from the plurality of cloud nodes to the plurality of edge nodes when the first location in the plurality of cloud nodes fails to transfer the classified data during a transaction; upon receiving, from the plurality of cloud nodes, a response signal comprising the plurality of second locations at which the classified data is stored in the plurality of cloud nodes, updating a synchronization table in the plurality of edge nodes with the plurality of second locations at which the classified data is stored in the plurality of cloud nodes; and synchronizing the classified data from the plurality of second locations in the plurality of cloud nodes to the plurality of edge nodes without duplicating the data based on the synchronization table that is updated when the request signal is received from the plurality of edge nodes.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:

[0024] FIG. 1 is a block diagram that illustrates a system for synchronizing and reconciling data stream from an edge node to a cloud node in real-time according to an embodiment herein;

[0025] FIG. 2 is a block diagram that illustrates a process of synchronizing and reconciling data stream from the edge node to the cloud node in real-time of FIG. 1 according to some embodiments herein;

[0026] FIG. 3 is an interaction diagram that illustrates a method for synchronizing and reconciling data stream from an edge node to a cloud node in real-time according to an embodiment herein;

[0027] FIGS. 4A-4B are flow diagrams that illustrate a method for synchronizing and reconciling data stream from the edge node to the cloud node in real-time according to an embodiment herein; and

[0028] FIG. 5 is a block diagram of a schematic diagram of a device used in accordance with embodiments herein.

DETAILED DESCRIPTION

[0029] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and. to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

[0030] Various embodiments disclosed herein provide a system and a method for a. system for synchronizing and reconciling data stream in real-time between an edge node and a. cloud node. Referring now to the drawings, and more particularly to FIGS. 1 through 5, where similar reference characters denote corresponding features consistently throughout the figures, preferred embodiments are shown.

[0031] FIG. 1 is a block diagram that illustrates a system 100 for synchronizing and reconciling data stream from an edge node 102 to a cloud node 112 in real-time according to an embodiment herein. The system 100 synchronizes the data stream from a plurality of edge nodes 102A-N (distributed system) to a central location in the cloud node 112. The cloud node may be a cloud server. In some embodiments, the system 100 maintains a state of the cloud server and monitors the data stream from the plurality of edge nodes 102A-N. The system 100 detects an exact location of the data stream coming from the plurality of edge nodes 102:A-N. The data stream includes a plurality of data obtained from a plurality of edge nodes 102A-N. The system 100 pushes the plurality of data received from different locations back to the determined central location in the cloud node 112 in a synchronized manner. A database access and synchronization layer identifies the plurality of data in the central location in the cloud node 112 and send it back to the requesting edge node. The system 100 includes a synchronization table in the plurality of edge nodes 102A-N to synchronize a location of data, in a central location in the cloud node 112. The system 100 includes a memory 106 and a processor 104. The memory 106 stores a set of instructions. The processor 104 executes the set of instructions to synchronize the plurality of data from a plurality of edge servers in the plurality of edge nodes 102A-N to a central location in the cloud node 112. The processor 104 classifies the plurality of data from the plurality of edge nodes 102A-N to generate a data stream in real-time. The processor 104 generates classified data by classifying the plurality of data from the plurality of edge nodes 102A-N to synchronize the plurality of data in a location at the cloud nodes 112. In some embodiments, the edge node 102A acts as an end-user portal for communication with other nodes in cluster computing. In sonic embodiments, the edge nodes are at least one of the gateway nodes or edge communication nodes. In some embodiments, the edge node 102A includes at least one of but is not limited to computer, mobile phone, tab. The processor 104 queues the classified data associated with the plurality of edge nodes 102A-N. The processor 104 obtains the exact location of the cloud node 112 or central server and tags the classified data with (i) a time stamp of the data, (ii) a location of the plurality of edge nodes 102A-N, (iii) an edge identifier of the plurality of edge nodes 102A-N, and (iv) an interact protocol address. In some embodiments, the classified data is tagged to identify the location of the classified data in the cloud node 112.

[0032] In some embodiments, the processor 104 queues the data from different locations of the plurality of edge nodes 102A-N. The information of the central location is located as metadata with every application. The metadata includes information that comprises application information, a central repository, and gets updated with respect to the frequency required for synchronizing to maintain records. The processor 104 synchronizes the data based on the frequency. The database in a local location may be bulky when the plurality of data is stored in the local locati on. in some embodiments, the local location is a plurality of edge server locations. The operation may be compute-heavy if the system continuously synchronizes the data to the database in the central location. The processor 104 creates a frequency count to automatically synchronize the plurality of data from the plurality of edge nodes 102A-N to the cloud node 112 regularly, The processor 104 determines a frequency based on the quantity of data from the plurality of cloud nodes 102A-N. In some embodiments, detection of the database is saved as metadata. The frequency of updating the information may be auto-decided by the processor 104.

[0033] The processor 104 detects a first location in the central location in the cloud node 112 and a type of database in the plurality of cloud nodes to synchronize the classified data associated with a real-time execution at the plurality of edge nodes 102A-N. In some embodiments, once the database in the cloud node 112 is detected, the Input/Output reads on the .sup.-file system are monitored and the data input is sent to the central location server in the cloud node 112 and the data is synchronized. The processor 104 receives a request signal from the plurality of edge nodes 102A-N for synchronizing the classified data from the plurality of cloud nodes to the plurality of edge nodes when the first location fails to transfer the classified data during a transaction. The processor 104 maintains the data received from the plurality of edge nodes 102A-N in a plurality of second locations in the cloud node 112 by replicating the classified data in the plurality of cloud nodes 112A-N. In some embodiments, the data is replicated to ensure fault tolerance. In some embodiments, volume-based replication of the database makes it easy to clone the same data in the plurality of edge nodes 102A-N in the plurality of locations.

[0034] The same data is maintained in the plurality of edge nodes 102A-N in the plurality of locations. In some embodiments, in case one database in the cloud node 112 is shut/down. or fails during any.sup., transaction, the other database in the cloud node 112 may store data.

[0035] The processor 104 receives a response signal from the plurality of cloud nodes 112A-N in the central location when the classified data is stored in the cloud node 112 in the central location. The processor updates the location of the classified data in the cloud node 112 to the synchronization table in the plurality of edge nodes 102A-N.

[0036] The response signal is sent back to the calling edge node location to update the synchronization table in the edge node once the data is synchronized in the cloud node 112, In some embodiments, the plurality of synchronization tables is present in the plurality of edge nodes 102A-N individually. The synchronization table is updated to denote that the synchronization is completed once the data is synchronized in the central location in the cloud node 112. If the synchronization table is not updated then the data gets synchronized in the next cycle. The replication layer determines the next cycle if the synchronization table is not updated.

[0037] The individual synchronization tables are present on the plurality of edge node locations. In some embodiments, the detection and synchronization are done in at least one of real-time or scheduled.

[0038] The processor 104 synchronizes the classified data from the plurality of the second locations in the cloud node 112 to the plurality of edge nodes I 02A-N without duplicating the data based on the synchronization table that is updated. The processor 104 synchronizes the data from the plurality of databases in the cloud node 112 to the plurality of edge nodes 102A-N when the request signal is received from the plurality of edge nodes 102A-N.

[0039] The plurality of data synchronization may be done from the plurality of edge nodes 102A-N to the cloud node without additional code from an end-user perspective. A legacy application can be deployed on edge without any extra effort.

[0040] FIG. 2 is a block diagram that illustrates a process of synchronizing and reconciling data stream from the edge node to the cloud node in real-time of FIG. 1 according to some embodiments herein. The plurality of data from the plurality of edge nodes 102A-N is synchronized with the cloud node 112 using a network 108. In some embodiments, the plurality of edge nodes 102A-N includes at least one of but is not limited to computer, mobile phone, tablet.

[0041] In some embodiments, the plurality of edge nodes 102A-N is a computer that acts as an end-user portal for communication with other nodes in cluster computing. The database classifying layer 204 generates classified data by classifying the data from the plurality of edge nodes 102A-N of the data stream in real-time to synchronize in a location at a plurality of databases in the cloud node 112. In some embodiments, the database classifying layer 204 detects the database in the cloud node 112 and data type. The database queuing agent 206 queues the classified data associated with the plurality of edge nodes 102A-N and tags the classified data. with (i) a time stamp of the data, (ii) a location of the plurality of edge nodes 102A-N, (iii) an edge identifier of the plurality of edge nodes 102A-N, and (iv) an internet protocol address. The database synchronization layer 208 synchronizes the classified data from the plurality of edge nodes 102A-N to the cloud node 112 based on the frequency count. The database synchronization layer 208 synchronizes the plurality of data to a first location and a type of database in the cloud node 112 and store the classified data in the first location in the cloud node 112. In some embodiments, the frequency count is automatically determined based on (i) an amount of data, (ii) a location of the plurality of edge nodes, and (iii) a type of application running in the plurality of edge nodes 102A-N. The database replication layer 210 replicates the classified data and ensures fault tolerance. In some embodiments, volume-based replication of the database makes it easy to clone the same data in the plurality of edge nodes 102A-N. In some embodiments, the node in cloud computing is a connection point, either a redistribution point or an endpoint for data transmissions in general. The cloud node 112 stores the plurality of data from the plurality of edge nodes 102A-N. The cloud node is connected with the database synchronization layer 208 to synchronize the location of the data in the plurality of databases in the cloud node 112 to the plurality of synchronization tables 202A-N in the plurality of edge nodes 102A-N. The plurality of synchronization tables 202A-N in the plurality of edge nodes 102A-N include the exact location of the plurality of data in the cloud node 112. The processor 104 synchronizes the data from the plurality of databases in the cloud node 112 to the plurality of edge nodes 102A-N without duplicating the data based on the synchronization table that is updated. In some embodiments, the response signal from the cloud node 112 comprises a plurality of second locations at which the classified data is stored in the cloud node 112.

[0042] In some embodiments, when the synchronization table is not synchronized then the data from the plurality of edge nodes 102A-N is synchronized in the next cycle. A cycle is a time period computed to synchronize the data and may be pre-determined by the user. In some embodiments, the synthronization table stored in each edge node comprises metadata of the edge node. In some embodiments, the metadata comprises the location of each cloud node. frequency count. The size of the metadata is in a range of 1 kilobyte (Kb) to 10 Kb.

[0043] FIG. 3 is an interaction diagram that illustrates a method for synchronizing and reconciling data stream from an edge node to a cloud node in a real-time according to an embodiment herein. At step 302, providing a plurality of data to the processor from the plurality of edge nodes 102A-N. The method 300 includes classifying data for the data stream that is obtained from a plurality of edge nodes 102A-N using the processor 104. At step 304, queuing a classified data associated with the plurality of edge nodes 102A-N using the processor 104. At step 306, tagging the classified data with (i) a time stamp of the data, (ii) a location of the plurality of edge nodes 102A-N, (iii) an edge identifier of the plurality of edge nodes 102A-N, and (iv) an internet protocol address using the processor 104. At step 308, determining a frequency count to synchronize the classified data associated with a real-time execution at the plurality of edge nodes 102A-N with the plurality of cloud nodes 112A-N using the processor 104. At step 310, synchronizing the classified data associated with the plurality of edge nodes 102.A-N in the plurality of cloud nodes 112A-N by detecting a first location using the processor 104 and a type of database in the plurality of cloud nodes 112A-N and storing the classified data in the first location in the plurality of cloud nodes 112A-N. At step 312, replicating the classified data associated with the plurality of edge nodes 102A-N in a plurality of second locations in the plurality of cloud nodes 112A-N using the processor 104. At step 314, receiving, from the plurality of edge nodes 102A-N, a request signal for synchronizing the classified data from the plurality of cloud nodes 112A-N to the plurality of edge nodes 102A-N by the processor 104 when the first location in the plurality of cloud nodes 112A-N fails to transfer the classified data during a transaction. At step 316, upon receiving, from the plurality of cloud nodes 112A-N, a response signal comprising the plurality of second locations at which the classified data is stored in the plurality of cloud nodes 112A-N using the processor 104. At step 318, updating a synchronization table in the plurality of edge nodes 102A-N with the plurality of second locations at which the classified data is stored in the plurality of cloud nodes 112A-N by the processor 104. At step 320, synchronizing the classified data from the plurality of second locations in the plurality of cloud nodes 112A-N to the plurality of edge nodes 102A-N without duplicating the data based on the synchronization table that is updated when the request signal is received from the plurality of edge nodes by the processor 104.

[0044] FIGS. 4A-4B are flow diagrams that illustrate a method for synchronizing and reconcilinu data stream from the edge node to the cloud node in real-time according to an embodiment herein. At step 402, classified data is generated by classifying data of the data stream that is obtained from a. plurality of edge nodes in real-time to synchronize in a location at a plurality of cloud nodes. At step 404, the classified data associated with the plurality of edge nodes are queued and the classified data is tagged with (i) a time stamp of the data, (ii) a location of the plurality of edge nodes, (iii) an edge identifier of the plurality of edge nodes, and (iv) an interne protocol address. At step 406, a frequency count is automatically determined to synchronize the classified data associated with a real-time execution at the plurality of edge nodes with the plurality of cloud nodes regularly. At step 408, the classified data associated. with the plurality of edge nodes is synchronized in the plurality of cloud nodes by detecting a first location and a type of database in the plurality of cloud nodes and the classified data is stored in the first location in the plurality of cloud nodes. At step 410, the classified data associated with the plurality of edge nodes is replicated in a plurality of second locations in the plurality of cloud nodes. At step 412, a request signal for synchronizing the classified data from the plurality of cloud nodes to the plurality of edge nodes is received from the plurality of edge nodes when the first location in the plurality of cloud nodes fails to transfer the classified data during a transaction. At step 414, a response signal comprising the plurality of second locations at which the classified data is stored in the plurality of cloud nodes is received from the plurality of cloud nodes and the plurality of second locations at which the classified data is stored in the plurality of cloud nodes is updated to a synchronization table in the plurality of edge nodes. At step 416, the classified data from the plurality of second locations in the plurality of cloud nodes are synchronized to the plurality of edge nodes without duplicating the data based on the synchronization table that is updated, when the request signal is received from the plurality of edge nodes.

[0045] A representative hardware environment for practicing the embodiments herein is depicted in FIG. 5, with reference to FIGS. 1 through 4A and 413. This schematic drawing illustrates a hardware configuration of a server/computer system/ user device in the plurality of edge node 102A-N in accordance with the embodiments herein. The user device includes at least one processing device 10 and a cryptographic processor 11. The special-purpose CPU 10 and the cryptographic processor (CP) 11 may be interconnected via system bus 14 to various devices such as a random-access memory (RAM) 15, read-only memory (ROM) 16, and an input/output (I/O) adapter 17. The I/O adapter 17 can connect to peripheral devices, such as disk units 12 and tape drives 13, or other program storage devices that are readable by the system. The user device can read the inventive instructions on the program storage devices and follow these instructions to execute the methodology of the embodiments herein. The user device further includes a user interface adapter 20 that connects a keyboard 18, mouse 19, speaker 25, microphone 23, and/or other user interface devices such as a touch screen device (not shown) to the bus 14 to gather user input. Additionally, a communication adapter 21 connects the bus 14 to a data processing network 26, and a display adapter 22 connects the bus 14 to a display device 24, which provides a graphical user interface (GUI) 30 of the output data in accordance with the embodiments herein, or which may be embodied. as an output device such as a monitor, printer, or transmitter, for example. Further, a transceiver 27, a signal comparator 28, and a signal converter 29 may be connected with the bus 14 for processing, transmission, receipt, comparison, and conversion of electric or electronic signals.

[0046] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications without departing from the generic concept, and, therefore, such adaptations and modifications should be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the appended claims.