Method and system for optimized routing of data streams in telecommunication networks

Abstract

A method, system and a network element for routing at least one data stream in a telecommunications network, the network having at least one transmitting node providing at least one data stream, at least one receiving node receiving the at least one data stream, and zero or more processing nodes, and at least one link connecting said nodes. All nodes communicate their capabilities to a system database on a database server which receives at least one request with at least one condition for sending at least one data stream between two nodes. For or each request, a measure is calculated for at least one route between said two nodes, and instructions are provided to use only the at least one route having measures compliant with the at least one condition in the request. An optimal route can be determined using at least one heuristic rule or linear programming.

Claims

1. A method for determining at least one route for at least one data stream in a telecommunications network having nodes comprising at least one transmitting node, having at least one receiving node receiving the at least one data stream, and having processing nodes, and at least one link connecting said nodes, the method comprising the steps of: all of said nodes communicating their processing capabilities to at least one single source or central server, the at least one single source or central server receiving at least one request with at least one condition for sending at least one data stream between the at least one transmitting node and the at least one receiving node, and for each request, using the communicated processing capabilities of all of said nodes and said at least one link stored in the single source or central server at the time of receiving the request, to calculate a measure for at least one route between said at least one transmitting and at least one receiving nodes, said at least one route comprising the at least one transmitting node, the at least one receiving node and one or more of the processing nodes, wherein the request is to send the at least one data stream from the at least one transmitting node to the at least one receiving node in a processed format, calculating an optimal location of the one or more processing nodes in relation to the at least one transmitting node or the at least one receiving node, sending the at least one data stream to the one or more processing nodes at the optimal location for processing into a processed data stream by using any of the communicated processing capabilities, whereby the processing can be any of compositing, transcoding, scaling, re-packaging, or transrating or any combination thereof, and sending the processed data stream to the at least one receiving node, and providing instructions to use only the at least one route, wherein said at least one route has measures compliant with the at least one condition in the request and said at least one condition comprises actual properties of the telecommunications network.

2. The method according to claim 1 comprising calculating an optimal route using at least one heuristic rule or linear programming.

3. The method according to claim 2, comprising for each direct pair of stateful nodes, replace hops between them with a direct link, calculate a route cost, and if the route cost is lowered and the required processing power is still provided, keep the direct link.

4. The method according to claim 2, comprising for each direct pair of stateful nodes, remove all links between the two nodes and, considering the required processing power, add the shortest path according to one of the Dijkstra algorithm, the Bellman/Ford algorithm; and the Johnson's algorithm.

5. The method according to claim 1 comprising adding or removing at least one node with processing power, calculating a route cost, and if the route cost is lower and the required processing power is still provided, keeping the direct link.

6. The method according to claim 2 comprising modelling the network with an objective function and using the request to define boundary conditions of variables in the model, and minimizing or maximizing the objective function.

7. The method according to claim 6, wherein the variables are binary.

8. The method according to claim 6, wherein the model is limited by constraints provided in the request, or the model is limited by constraints provided in the request being cost and/or latency.

9. The method according to claim 6, wherein the model is limited by actual properties of the network, or the model is limited by actual properties of the network being receiving and transmitting bandwidth and/or encoding and decoding capacity.

10. The method according to claim 6, wherein the objective function is minimized or maximized within the limits of variable constraints.

11. The method according to claim 6, wherein a solution is found by using linear programming.

12. The method according to claim 1 comprising requesting to send at least two data streams from at least two different transmitting nodes.

13. The method according to claim 1 comprising at least two of the network nodes of the calculated route using different protocols.

14. The method according to claim 1 comprising adding or removing network nodes, or a combination of both, between two events in the network.

15. The method according to claim 1 comprising for each of the at least one calculated route, modifying the settings of the at least one link before sending the at least one data stream along the calculated route.

16. The method including re-routing a data stream comprising the steps of the network being submitted to an event which triggers engagement of the method of claim 1.

17. A system for determining at least one route for at least one data stream in a first telecommunications network comprising: said first telecommunications network having nodes comprising at least one transmitting node, at least one receiving node for receiving the at least one data stream, and one or more processing nodes; at least one link connecting said nodes; at least one single source or central server connected to all of said nodes, via clients, over a second network, wherein all of said nodes are configured to communicate their processing capabilities to the at least one single source or central server, wherein for each point in time, a processing capability of each node of said nodes and said at least one link is stored on the single source or central server, so that for each point in time, a measure is calculated for the at least one route through the network from the processing capability of each node of said nodes and said at least one link belonging to said at least one route, wherein the system is adapted to receive a request to send the at least one data stream from the at least one transmitting node to the at least one receiving node in a processed format with at least one condition for sending the at least one data stream, said at least one condition comprises actual properties of the telecommunications network, calculating an optimal location of a processing node of the one or more processing nodes in relation to the at least transmitting node or the at least one receiving node, sending the at least one data stream to the processing node at the optimal location for processing into a processed data stream by using any of the communicated processing capabilities, whereby the processing can be any of compositing, transcoding, scaling, re-packaging, or transrating or any combination thereof, and sending the processed data stream to the at least one receiving node, and to provide instructions to use only the at least one route, wherein the at least one route has measures compliant with the at least one condition in the request.

18. The system according to claim 17, wherein a measure comprises any of latency, cost, computation time, quality of the data stream, frame rate, accepted loss rate, the priority of a data stream or any combination thereof.

19. A non-transitory signal storage medium storing a computer program product which when executed on a processing engine carries out the method of claim 1.

20. The method according to claim 1, wherein said measures comprise said calculated measure for at least one route between said at least one transmitting node and at least one receiving nodes.

21. The method according to claim 1, wherein the at least one data stream comprises at least one data selected from the group consisting of text, video, audio, metadata files or any digitized data, packets, and frames.

Description

BRIEF DESCRIPTION OF DRAWINGS

(1) FIG. 1 shows an embodiment of the present invention comprising different nodes of a network, connected via clients, to an application server.

(2) FIG. 2 shows an embodiment of the present invention comprising a firewall blocking a link.

(3) FIG. 3 shows an embodiment of the present invention comprising the use of a DMZ.

(4) FIG. 4 shows an embodiment of the present invention comprising rerouting via a node.

(5) FIG. 5 shows an embodiment of the present invention comprising a processing node for compositing.

(6) FIG. 6 shows an embodiment of the present invention comprising the capabilities of a processing node.

(7) FIG. 7 shows an embodiment of the present invention comprising transcoding.

(8) FIG. 8 shows an embodiment of the present invention comprising relocating a high bandwidth demand.

(9) FIG. 9 shows an embodiment of the present invention comprising differing characteristics of different links in the network.

(10) FIG. 10 shows an embodiment of the present invention comprising re-routing after a link breakdown.

(11) FIG. 11 shows an embodiment of the present invention comprising a flowchart describing re-routing.

(12) FIG. 12 shows an embodiment of the present invention comprising monitoring of a certain link.

(13) FIG. 13 shows an embodiment of the present invention comprising different nodes in a network.

(14) FIG. 14 shows a flow chart of an embodiment of the present invention comprising the application of an heuristic rule.

(15) FIG. 15 shows a flow chart of an embodiment of the present invention comprising linear programming

(16) FIG. 16 shows an embodiment of the present invention comprising several network clusters.

DEFINITIONS

(17) In the following text a data stream can comprise data including any one of, any combination of, or all of text, video, audio, metadata files or any digitized data that can be sent between- and be processed by nodes in a network. The data stream can include packets or frames with, for example, headers or preambles, mid-ambles or post-ambles, error checking data or error correction data.

(18) A hybrid network can comprise communication with different communication technology, such as e.g. wired and wireless connections. A hybrid network comprising different networks may need a conversion of a data stream from one network to another. Hence, the hybrid network can comprise one or some or all of e.g. Private Networks, Wide Area Networks (WAN) or Cloud Backbones.

(19) A hop is a part of the path that a packet, frame or stream takes when going from a source node to a destination node. When performing the hop, the packet or stream passes a network element such as e.g. a router, gateway, processing node, etc. Nodes can be configured as repeaters.

(20) Network elements such as e.g. servers such as application servers, database servers, routers, gateways, processing nodes, repeaters, encoders, decoders, transcoders, encrypters and/or decrypters, compositors, modems, transrating or resizing devices, protocol converters, buffers, etc can be devices which have a processing engine such as a microprocessor or an FPGA and also memory or are constructed as a microcontroller or comprise software which can be executed to create the relevant functionality such that digital information can be processed. Such network elements can have a network interface such as a network card which allows each of these network elements to interact with the network. Network elements may comprise logic encoded in media for performing any kind of information processing. Logic may comprise logic gates or software encoded in a disk or other computer-readable medium and/or instructions encoded in an application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other processor or hardware.

(21) It is advantageous to use a single source server for storing some information whose duplication would require synchronisation that could cause problems if the synchronisation is delayed. Such a single source server can be implemented as a central server.

(22) A link is a communication channel between two nodes in the network.

(23) A stateful node can remember one or more preceding events in a system where interactions (e.g. with a user, with an application) occur. A direct pair of stateful nodes has no other stateful nodes in the link between them.

(24) A client is referred to as a piece of software that can run on any type of computer (e.g. laptop, desktop, tablet, phone, . . . ) or is a physical digital processing device which communicates with a server and can sometimes interact with a user. It can host one or more types of nodes, e.g. source nodes, destination nodes and/or processing nodes. There are various types of client-server schemes such as a thin client, a thick client etc. which differ with respect to the level of intelligence and independence of the client device.

(25) Applications are computer programs or software that can run on either a server, or on clients that might be hosting the nodes. An application can start a stream between two nodes by submitting a request to the application server with information on the stream, together with requirements for how it should be sent and received. The nodes will use the network to send the stream to other nodes.

(26) Alternatively, the client can request a stream that is hosted on another client.

(27) Codecs and transport protocols: with reference to audio/video: there can be any combination of video codec (and/or audio codec) and transport protocol:

(28) Video Codecs: MJPEG, h.263, h.264 (=AVC), h.265 (=HEVC), VP8, VP9, VMW, . . . .

(29) Audio Codecs: mp3, flac, ogg vorbis, WAV, AAC, . . . .

(30) Transport protocols: WebRTC, RTP, RTSP, DASH, HLS, RTMP, MirrorOp, . . . .

(31) In case of other data, JSON over websockets, xml over http, . . . and many more combinations are available.

ABBREVIATIONS

(32) MJPEG, Motion JPE (JPEG: Joint Photographic Experts Group)

(33) H.263,

(34) H.264 (=AVC), AVC=Advanced Video Coding

(35) H.265 (=HEVC), HEVC=High Efficiency Video

(36) Coding

(37) VP8,

(38) VP9,

(39) VMW Windows Media Video

(40) mp3, MPEG Layer-3 (MPEG: Motion Picture Experts Group)

(41) flac, Free Lossless Audio CODEC

(42) vorbis,

(43) WAV, WAVE

(44) AAC Advanced Audio Coding

(45) WebRTC, Web Real Time Communications

(46) RTP, Real-time Transport Protocol

(47) RTSP, Real Time Streaming Protocol

(48) DASH, Dynamic Adaptive Streaming over HTTP

(49) HLS, HTTP Live Streaming

(50) RTMP, Real Time Messaging Protocol

(51) MirrorOp Mirror+Operate (app for WePresent)

DETAILED DESCRIPTION OF THE INVENTION

(52) The present invention will be described with respect to particular embodiments and with reference to certain drawings but the invention is not limited thereto but only by the claims. The drawings described are only schematic and are non-limiting.

(53) Furthermore, the terms first, second, third and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. The terms are interchangeable under appropriate circumstances and the embodiments of the invention can operate in other sequences than described or illustrated herein.

(54) Moreover, the terms top, bottom, over, under and the like in the description and the claims are used for descriptive purposes and not necessarily for describing relative positions. The terms so used are interchangeable under appropriate circumstances and the embodiments of the invention described herein can operate in other orientations than described or illustrated herein. The term comprising, used in the claims, should not be interpreted as being restricted to the means listed thereafter; it does not exclude other elements or steps. It needs to be interpreted as specifying the presence of the stated features, integers, steps or components as referred to, but does not preclude the presence or addition of one or more other features, integers, steps or components, or groups thereof. Thus, the scope of the expression a device comprising means A and B should not be limited to devices consisting only of components A and B. It means that with respect to the present invention, the only relevant components of the device are A and B. Similarly, it is to be noticed that the term coupled, also used in the description or claims, should not be interpreted as being restricted to direct connections only. Thus, the scope of the expression a device A coupled to a device B should not be limited to devices or systems wherein an output of device A is directly connected to an input of device B. It means that there exists a path between an output of A and an input of B which may be a path including other devices or means.

(55) References to software can encompass any type of programs in any language executable directly or indirectly by a processor.

(56) References to logic, hardware, processor or circuitry can encompass any kind of logic or analog circuitry, integrated to any degree, and not limited to general purpose processors, digital signal processors, ASICs, FPGAs, discrete components or transistor logic gates and so on.

(57) FIG. 1 shows an embodiment of the invention comprising a electronic digital telecommunication network with a transmitting source node 1, four processing nodes 2, 3, 4 and 5, and one receiving destination node 6. The nodes are hosted by different clients and located at different clouds. Figure shows different clients (squares) 220, 221, 222, 223 and 224 and a connection 225 to the single source or central server. The source node 1 is a local node hosted by client 220, node 2 is hosted by client 222 and located at cloud 15, node 3 and node 4 are hosted by client 221 and located at cloud 16, node 5 is hosted by client 223 and located at cloud 17 and the destination node 6 is a local node hosted by client 224. The task is to set up a link between the source node 1 and the destination node 6 for sending data. FIG. 1 shows that there are various alternatives for how to accomplish this in the connecting lines between the nodes. A certain route comprising links 7, 8 and 9 is finally selected. Embodiments of the present invention are adapted so that the transmission is able to fulfil certain requirements taking into account the current status of the network and its links. The requirements could be for example a maximum amount of latency, a minimum level of quality. In case there is a multiple of streams, there might be composition needed, etc.

(58) The nodes of a hybrid network may be distributed within one region, one country, one continent or worldwide in different types of networks, such as e.g. Private Networks, Wide Area Networks (WAN) or Cloud Backbones. The nodes are implemented as stand-alone devices or by one or several instances of virtual machines on physical servers of the cloud provider. There can be an arbitrary number of nodes for each virtual machine. All nodes are connected to an application server 10 over the networks 11, 12, 13, 14 and via a network link 225 which can be separate or connected to each other. The application server 10 is continuously receiving information from each node about its capability, thus the application server 10 is implementing a monitoring function that monitors the state of the hybrid network at each point in time. In one embodiment information can also be gathered non-continuously and upon request. For example, one request could be that there is a bandwidth test started periodically (e.g. once per hour or with any interval of preference). In another example, if a new node is added to the network, a request could be launched to test e.g. its network capabilities and/or connectivity status (for example testing a firewall) to see if it could be used for connection in the future.

(59) There can be events in the network, i.e. actions that affect the network thus changing its status. For example a broken link can appear or a channel can suddenly be heavily loaded so that the bandwidth is drastically reduced. If necessary, the single source or central server can add or remove nodes from a path in the network to increase efficiency and assure quality of service. The introduction or removal of a node is also an event.

(60) A data stream originates from a source node, which can also act as a transmitting node, and is received at its final destination by a destination node, which also acts as a receiving node. A processing node can share a physical device with the source node(s) or the destination node(s), or it can be located as a separate node between the source node(s) and the destination node(s). A processing node has computation power to apply to the stream(s) various functions. These could be for example compositing, transcoding, . . . . Additionally, a processing node can be configured to perform other functions. Some of these can be unknown to the system, e.g. because they are application specific e.g. face detection, recording, license plate recognition.

(61) The processing node(s) can also act as transmitting and receiving nodes. In case there is an event, e.g. a broken link, the network status has changed and the route calculated before might no longer be valid. In such a situation, a network element such as the application server calculates a new route which may or may not use the processing node currently hosting the stream, as a transmitting node. The network element such as an application server for doing calculations relevant to rerouting can include one or more microprocessors, processors, controllers, or a central processing unit (CPU) and/or a Graphics Processing Unit (GPU), and can be adapted to carry out functions by being programmed with software, i.e. one or more computer programmes.

(62) The network may have memory (such as non-transitory computer readable medium, RAM and/or ROM), an operating system, optionally a display such as a fixed format display, data entry devices such as a keyboard, a pointer device such as a mouse, serial or parallel ports to communicate with other devices, or network cards and connections to connect to any of the networks. The application server can consult a mix of static and dynamic knowledge on the network to find acceptable, good or better routes, e.g. correct routes. Static knowledge can e.g. be entered by the system administrator, for example to use gateway x for streaming to any destination inside a certain company network. Dynamic knowledge is knowledge added to the system by learning from previous decisions.

(63) The calculation of the route can be optimized, taking into account any, some or all of the conditions in the request, characteristics of the data to be sent and the current capabilities and/or current measured state of the network nodes. For example, if a link has full capacity of 1 Gbit per second, and 50% is used by other applications, the current measured state would be that only 500 Mbit per second can be used.

(64) FIG. 2 shows an embodiment of the present invention comprising a source node 20 and a destination node 21. The direct UDP (User Datagram Protocol) link 22 between them is blocked by a firewall 23 at the border of network 24. However, both devices are able to tunnel RTP over TCP, so the application server can select to do so over links 25 instead of selecting routing over a processing node.

(65) FIG. 3 shows an embodiment of a network according to the present invention having a source node 30 located in a network 36, a destination node 31 and a gateway node 32 in a perimeter network, or DMZ (Demilitarized Zone) 33. The direct link 34 between the source node 30 and the destination node 31 is blocked by a firewall 35. However, the perimeter network 33 comprises a gateway node 32, and the stream can be re-routed via the gateway node 32 and further to the destination node 31 via link 39. The gateway node 32 is also limited by firewalls 37 and 38, but they can for example be configured not to block the streams from this gateway personal computer. An alternative is that the gateway node 32 has a network interface to the network behind the firewall and a network interface to the network not behind the firewall (not shown).

(66) In one embodiment, a multiple of streams are to be sent over the network. FIG. 4a) shows how 2 streams from nodes 40 and 41 in network 42 are requested to be sent to node 43 in network 44. The direct routes would be 45 and 46. However, node 43 might not be able to receive one of the streams over one of the routes, e.g. route 46. For example, the destination node 43 might not have the processing power (for e.g. decoding and/or compositing, or any other function(s) needed), or there could be a line breakdown or there could be a firewall present. FIG. 4b) shows how the application server re-routes both streams to node 47 over routes 48 and 49. The streams are processed at node 47 and the resulting one stream 50 can now be sent to node 43.

(67) FIG. 5 shows an embodiment of the present invention comprising two source nodes 60 and 61 at a local site 62, two source nodes 63 and 64 in the cloud 65, one processing node 66 in the cloud 65, one processing node 67 at the local site 68, one processing node 69 at a perimeter network, or DMZ zone 70 and one destination node 71 at the local site 68 In one embodiment, the streams 72, 73, 74 and 75 from all four source nodes 60, 61, 63 and 64 should be composited to one stream and sent to the destination node 71. The three processing nodes 66, 67 and 69 make possible a variety of solutions. One condition could be that the compositing has to be performed at a processing node whose location is close or optimally spaced with respect to the source nodes. Or the optimal condition could be related to e.g. bandwidth, distance, processing power or any other characteristic. FIG. 5 shows an example where the optimal processing node is 66, which is located in the cloud 65. Thus, the streams 72, 73, 74 and 75 are sent to processing node 66 and composed to one stream 76 which is sent to the destination node 71. In the general case, the processing nodes are able to perform e.g. compositing, transcoding, scaling, re-packaging, relaying, tunnelling or transrating, or any other function. Each node can host one function or any combination of several functions.

(68) An example of one embodiment of the present invention where a node comprises several functions is shown in FIG. 6. The processing node P comprises multiple processing pipelines 81 to 85: Pipelines 81 and 82 for transcoding, pipelines 83 and 84 for relaying and pipeline 85 for compositing. The transcoding pipelines 81 and 82 each receives and transmits one stream, 86 and 87 respectively. The relay pipelines 81 and 82 each receives a stream 86 and 87 respectively which is processed and the processed stream 93 is transmitted. The relay pipeline 82 receives a stream 87 which is processed and the processed stream 82 it transmits to two destinations. The compositing pipeline receives three streams which it compositing to one stream and sends this stream to two destinations.

(69) In one embodiment of the present invention the transmitting and receiving nodes communicate with different protocols, so there is a need for transcoding. FIG. 7 shows an example where node 100 is located in network 101, using H.264 over RTP (Real Time Protocol). H.264 is a video coding format that can be carried by RTP. There is a destination node 102 located outside the network 101 but the direct link 103 is blocked since node 102 runs VP8 over Web RTC (Web Real-Time Communication). VP8 is a video compression format that can be carried by WEB RTC. WEB RTC is an application programming interface that supports browser to browser communication. Node 104 is located in the cloud 105 and comprises a transcoder. The application server (not shown) can then decide to reroute the stream to node via node 104 in link 107. The stream is transcoded to VP8 over WebRTC and can now be sent to the destination node 102 via link 106.

(70) FIG. 8 shows an embodiment where 5 streams are to be sent from one node 110 located in a network 111 to destinations 112-116 located outside 111. This might give rise to a bottle neck 117 near the location of node S. The application server (not shown) can then re-route all streams to processing node 118 located in the cloud 118, which in turn is relaying the streams to their destination nodes 112-116. In this way the high bandwidth demand can be moved to a less sensitive place in the network.

(71) FIG. 9 shows an embodiment of the present invention being a hybrid network comprising a source node 120, two processing nodes 121 and 122, and two destination nodes 123 and 124. One stream will be sent from the source node 120 to the two destination nodes 123 and 124. The different links will have different properties. E.g. the cloud backbone 125 between the processing nodes 121 and 122 is normally fast and reliable while e.g. the link 126 between the source node 120 and the first processing node 121 can exhibit variable performance. Also links 127 and 128 between the processing node 122 and the destination nodes 123 and 124 can be less reliable than 125. By choosing appropriate link settings for each link, the delivery of the stream(s) can be further optimized. For example, if the stream should be sent to multiple destinations but in different codecs and resolutions, it is possible to choose resolution and codec in the intermediate steps, or hops, to minimize bandwidth and processing power (to reduce cost) and/or processing needs (to reduce latency). In another example a link which is more reliable is used so that a protocol with less overhead cost used for error recovery can be chosen. Choosing the link settings can be incorporated in the overall route calculation since it might impact the route (for example, there is a need for a certain processing node that might or might not be available at a certain location).

(72) FIG. 10 shows an embodiment of the present invention comprising a source node 130, processing nodes 131-135 in different locations and a destination node 136. For a certain data stream, the application server has calculated a route over links 137, 138 and 139. If the link 139 suddenly fails due to an event 140, the application server (not shown) is notified and triggered to re-route the data stream. The application server collects all the capabilities of all nodes between the source node 130 and the destination node 136 and calculates a new route. The calculation can be made from the source node 130 or any other non-replaceable (or stateful) node, or from the processing node 135 just before the broken link. In this example a route over nodes 131 and 132 was chosen. The present invention can manage live streams where there can be a continuous transmission form the source. Since one node might send out a multiple of streams to several nodes, only the data streams being sent over a link that fails would need to be re-routed while ongoing routing over functioning links can be left undisturbed. The cache time in the nodes is limited to a few milliseconds.

(73) The procedure is shown as a flow chart in FIG. 11. In step 150 at least one stream flows along at least one calculated route, from at least one source node to at least one destination node in a network. In step 151, an event occurs in the network wherein at least one link gets broken. In step 152 the event triggers the application server to sample the capability of each node and re-calculate a route for each of the at least one stream that is send along a route with a broken link. Each of the at least one stream that is sent along a route with a broken link is then re-routed over its recalculated route.

(74) Empirical models can be made to predict for example how much of each type of capacity that is needed for certain inputs/outputs. This can be implemented by using abstract descriptions of components and continuously monitoring these components. This big data can then be used to model the components. These models can be used to predict the behaviour of components and to detect anomalies and/or problems. For example, on a certain type of machine, a transcoder of type x consumes 10% of the central processing unit for a 1080p video stream.

(75) In one embodiment the application server collects the capabilities of certain nodes and links at each point in time. By considering e.g. the actual loads on the network, the server can build a database with statistics of the network behaviour. FIG. 12 shows the logging of packet loss in links between two processing nodes 160 and 161 at a certain point in time. Each link provides its packet loss percentage 163-167 to the application server 162. An example of actual packet losses could be that 163 is 0.1%, 164 is 0%, 165 is 0.2%, 166 is 0.15% and 167 is 0.09%. The application server 162 can then learn that the average loss of packets between node 160 and 161 is ca 0.1%. Such logged data can be stored in databases to be used for predictions.

(76) The predictions described above might show that there is a need for additional nodes or that some nodes can be removed. In one embodiment the application server can add or remove nodes depending on the current need. FIG. 13 shows an example of routes in a network at a certain point in time. The network has source-destination- and processing nodes. There are some nodes 170-172 that are currently not used. The application server can then decide to remove these nodes to reduce cost. Should the demand change at a later point in time so that additional nodes are needed, the application server can add nodes to the network. The network is an overlay network comprising nodes that are implemented as virtual machines that can be added or removed depending on the need. The calculations described in FIG. 14 can be used to calculate predictions to determine the need of amount and type of nodes at various conditions. Such conditions can for example be at a certain time of the day/week or year or the request of sending a certain type of data stream or combination of certain types of data streams. Together with information on the processing power of each node, the actual need of amount of nodes can be predicted.

(77) For a network comprising a source- and destination node and at least one processing node, there can be at least two possible ways for how to route a stream from the source node to the destination node. The number of possible routes increases with the number of available processing nodes between the source node and the destination node. Each node and link in a network can be associated with a cost, for example monetary cost for using a leased network, or it can be cost measured in latency; cost measures other than these are also possible. Thus different routes can be associated with different costs. Since each request comes with specific requirements (need for relaying, compositing, transcoding, etc.) there will be different routes possible for each request and a cost related to each route. It is therefore advantageous to find an optimal route for the request that minimize the cost. This can comprise to minimize e.g. monetary cost, latency cost or cost for number of changes, or any other type cost, or any combination thereof.

(78) The optimization can be performed by just performing all possible calculations using brute force, however this is often time consuming an inefficient. Alternatively, a Monte Carlo method can be used. Heuristic rules can be used to limit the solution space.

(79) Embodiments of the present invention include applying heuristic rules according to the following method described in FIG. 14. FIG. 14 shows a flow chart of an embodiment of the present invention where a route is calculated. The process can be performed in two main steps. In step 180 a first step is performed in that a functional graph is built between the source node and the destination node according to the received request and taking into account the available nodes. The graph is built with an ad-hoc approach so that that all requested functions are present but there is not yet any consideration of optimization. Step 180 is a fast step that can be performed within time frames such as 800 ms or even 50 ms, e.g. less than 800 ms down to 30 ms. In a second step 181, an optimization process is started. The step comprises for each direct pair of stateful nodes, that hops are removed and replaced with a direct link, and if the request is still fulfilled (or improved), the direct link is kept, and if the request is not fulfilled, the change is reverted back to the previous state. Step 182 is a second part of the optimization process comprising removing all links between the two nodes and adding the shortest path according to the Dijkstra algorithm (see for example A Note on Two Problems in Connexion with Graphs, Numerische Mathematik, 1959). This step considers that the links have different costs. Steps 181 and 182 can be performed in a longer time frame, for example several minutes. The process can also work in a multiple of such time frames where it at the end of each time frame provides intermediate results that can be accepted or rejected.

(80) Although the Dijkstra algorithm is preferred other algorithms can be used. The Bellman-Ford algorithm is an algorithm that computes shortest paths from a single source vertex to all of the other vertices in a weighted digraph. It is slower than Dijkstra's algorithm for the same problem, but more versatile, as it is capable of handling graphs in which some of the edge weights are negative numbers. See for example https://en.wikipedia.org/wiki/Bellman%E2%80%93Ford_algorithm

(81) As the negative cost or latency is not so useful this algorithm can be used instead of Dijkstra, but may be slower

(82) Johnson's algorithm works by using the Bellman-Ford algorithm to compute a transformation of the input graph that removes all negative weights, allowing Dijkstra's algorithm to be used on the transformed graph.

(83) See for example https://en.wikipedia.org/wiki/Johnson%27s_algorithm

(84) FIG. 15 shows another embodiment of the present invention where step 190 is a first fast step that can be performed within time frames such as 800 ms or even 50 ms e.g. less than 800 ms down to 30 ms. Step 191 starts an optimization process which comprises linear programming. The step 191 comprises setting up an objective function taking into account all nodes in the network and using known limitations of the network and data stream, and requested requirements of the routing, to define boundary conditions. In step 192 the equation is solved, i.e. optimized, with an appropriate solver. Step 191 and 192 can be performed on a longer time frame for example several minutes. The process can also work in a multiple of such time frames whereby at the end of each time frame intermediate results are provided that can be accepted or rejected. The time to reach a solution can depend on the complexity of the ad-hoc network, the solver tool in use and the available computational power.

(85) In order to reduce latency it can be of interest to put in a requirement to limit the number of hop levels of the network. Here follows an example of the case when the number of levels is limited to three; each level can have an arbitrary number of nodes. The network in this example comprises sources, controllers and compositors but the equations can be extended to further types of processing.

(86) Consider the objective function

(87) $\begin{array}{cc}M=\underset{i}{\overset{N}{.Math.}}\underset{j}{\overset{n_i}{.Math.}}\left(w_x{x}_{\mathrm{ij}}+w_y\underset{k}{.Math.}{y}_{\mathrm{ijk}}+w_z\underset{k}{.Math.}{z}_{\mathrm{ijk}}\right)a_{\mathrm{ij}}& \left(1\right)\end{array}$

(88) Where x.sub.ij represents using the original source, y.sub.ijk represents letting the k.sup.th compositor do scaling, and z.sub.ijk represents letting the k.sup.th compositor do scaling and compositing. x.sub.ij, y.sub.ijk and z.sub.ijk are binary.

(89) Thus if

(90) x.sub.ij=1, x.sub.ij is the i.sup.th controller using the original source (as received) for window w.sub.ij, and if

(91) y.sub.ijk=1, y.sub.ijk is the i.sup.th controller using a scaled source for window w.sub.ij, and scaling is made by the k.sup.th compositor, and if

(92) z.sub.ijk=1, z.sub.ijk is the i.sup.th controller using the k.sup.th compositor to do scaling and compositing for the complete layout.

(93) The boundary conditions for (1) are

(94) $\begin{array}{cc}\underset{k}{.Math.}{y}_{\mathrm{ijk}}1& \left(2\right)\end{array}$
i.e. only one compositor can perform scaling, or

(95) $\begin{array}{cc}\underset{k}{.Math.}{z}_{\mathrm{ik}}1& \left(3\right)\end{array}$
i.e., the complete composition is made by only one compositor, and

(96) $\begin{array}{cc}{x}_{\mathrm{ij}}+\underset{k}{.Math.}{y}_{\mathrm{ijk}}+\underset{k}{.Math.}{z}_{\mathrm{ik}}1,i,j& \left(4\right)\end{array}$
i.e. each source is either original or scaled or part of composition, where
i=1 . . . N, j=1 . . . n.sub.i, N is the number of controllers and n.sub.i is the number of windows in the layout of controller i.

(97) w.sub.x, w.sub.y, w.sub.z are weights that can set a preference to do local compositing, scaling or cloud compositing.

(98) For example, if w.sub.x>w.sub.y>w.sub.z, any controller will prefer to receive and decode on its own (w.sub.x being largest), if this is not possible, the controller will rely on scaling (w.sub.y being second largest) and last option is that the controller will rely completely on cloud composition (w.sub.z being smallest).

(99) The following boundary conditions consider available bandwidth:

(100) $\begin{array}{cc}{b}_x\underset{j}{.Math.}{x}_{\mathrm{ij}}+b_y\underset{j}{.Math.}\underset{k}{.Math.}{y}_{\mathrm{ijk}}+b_z\underset{k}{.Math.}{z}_{\mathrm{ik}}{B}_{\mathrm{CTRL}},i& \left(5\right)\\ d_x\underset{j}{.Math.}{x}_{\mathrm{ij}}+d_y\underset{j}{.Math.}\underset{k}{.Math.}{y}_{\mathrm{ijk}}+d_z\underset{k}{.Math.}{z}_{\mathrm{ik}}{D}_{\mathrm{CTRL}},i& \left(6\right)\end{array}$
where
B.sub.CTRL is the receiving bandwidth of the controller
D.sub.CTRL is the decoding capacity of the controller.
b.sub.x is the bandwidth required to receive an original source
b.sub.y is the bandwidth required to receive a scaled source
b.sub.z is the bandwidth required to receive a composition of one or more sources.

(101) Further, there are boundary conditions for the encoding and decoding capacity:

(102) $\begin{array}{cc}{d}_x\underset{i}{.Math.}\underset{j}{.Math.}{y}_{\mathrm{ijk}}+d_x\underset{i}{.Math.}{z}_{\mathrm{ik}}{n}_i{D}_{\mathrm{COMP}},k& \left(7\right)\\ e_x\underset{i}{.Math.}\underset{j}{.Math.}{y}_{\mathrm{ijk}}+e_z\underset{i}{.Math.}{z}_{\mathrm{ik}}{E}_{\mathrm{COMP}},k& \left(8\right)\\ b_x\underset{i}{.Math.}\underset{j}{.Math.}{y}_{\mathrm{ijk}}+b_x\underset{i}{.Math.}{z}_{\mathrm{ik}}{n}_i{\mathrm{BRX}}_{\mathrm{COMP}},k& \left(9\right)\\ b_y\underset{i}{.Math.}\underset{j}{.Math.}{y}_{\mathrm{ijk}}+b_z\underset{i}{.Math.}{z}_{\mathrm{ik}}{\mathrm{BTX}}_{\mathrm{COMP}},k& \left(10\right)\end{array}$
d.sub.x, d.sub.y, d.sub.z is the decoding capacity required to decode original, scaled or composition of sources.
b.sub.x, b.sub.y, b.sub.z is bandwidth, as described above.
D.sub.comp is the decoding capacity of a compositor
E.sub.comp is the encoding capacity of a compositor
BTX.sub.comp and BRX.sub.comp are the transmitting and receiving bandwidth respectively. It is assumed that all compositors have the same limitations.

(103) The objective is to maximize M in equation (1) subject to the boundary conditions of (2) to (10).

(104) In another embodiment the nodes can be located in different networks. FIG. 16 shows a node 200 that can comprise any source or processing function. The node 200 can also be connected to clusters 201, 202 and 203 with high bandwidth connections, e.g. with speeds of 100 Mb/second. Cluster 201 can comprise a source node 204 and a compositor node 205. Cluster 202 can comprise a compositor node 206, a controller node 207 and a source node 208. Cluster 203 can comprise a source node 209 and a compositor node 210. If needed, node 200 can use the resources in clusters 201, 202 and/or 203. The bandwidths between the clusters are 211, 212 and 213 respectively. The source nodes are of the format S.sub.lm where l is the source and m is the cluster. The compositor nodes are of the format PP.sub.km where k is the compositor and m is the cluster. The controller nodes are of the format PL.sub.im where i is the compositor and m is the cluster.

(105) Further we have that

(106) $\underset{m}{.Math.}{S}_{lm}=1,l=1\mathrm{.Math.}L,m=1\mathrm{.Math.}M,\mathrm{and}$$\underset{m}{.Math.}{\mathrm{PP}}_{km}=1,l=1\mathrm{.Math.}K,m=1\mathrm{.Math.}{M}_i,\mathrm{and}$$\underset{m}{.Math.}{\mathrm{CL}}_{im}=1,i=1\mathrm{.Math.}N,m=1\mathrm{.Math.}{M}_i.$

(107) Furthermore, .sub.ijl=1 if source S.sub.l is located in window w.sub.ij and .sub.ijl=0 if source S.sub.l is not located in window w.sub.ij.

(108) With

(109) $\underset{l}{.Math.}{f}_{\mathrm{ijl}}=1,i=1\mathrm{.Math.}N,j=1\mathrm{.Math.}{n}_i.$

(110) If cluster m has bandwidth B.sub.m, then the following condition should be fulfilled for the outgoing streams

(111) $\begin{array}{cc}{b}_x\underset{i}{.Math.}\underset{j}{.Math.}\left[\underset{l}{.Math.}{f}_{\mathrm{ijl}}{S}_{lm}\right].Math.\left[x_{\mathrm{ij}}\left(1-{\mathrm{PL}}_{\mathrm{im}}\right)+\underset{k}{.Math.}\left(y_{\mathrm{ijk}}+z_{\mathrm{ik}}\right)\left(1-{\mathrm{PP}}_{km}\right)\right]+b_y\underset{i}{.Math.}\underset{j}{.Math.}\underset{k}{.Math.}{y}_{\mathrm{ijk}}{\mathrm{PP}}_{km}\left(1-{\mathrm{PL}}_{im}\right)+b_z\underset{i}{.Math.}\underset{k}{.Math.}{z}_{\mathrm{ik}}{\mathrm{PP}}_{km}\left(1-{\mathrm{PL}}_{\mathrm{im}}\right)B_m,m=1\mathrm{.Math.}M,& \left(11\right)\end{array}$
and the following condition should be fulfilled for the incoming stream

(112) 0$\begin{array}{cc}{b}_x\underset{i}{.Math.}\underset{j}{.Math.}\left[\underset{l}{.Math.}{f}_{\mathrm{ijl}}\left(1-S_{lm}\right)\right].Math.\left[x_{\mathrm{ij}}{\mathrm{PL}}_{im}+\underset{k}{.Math.}\left(y_{\mathrm{ijk}}+z_{\mathrm{ik}}\right){\mathrm{PP}}_{km}\right]+b_y\underset{i}{.Math.}\underset{j}{.Math.}\underset{k}{.Math.}{y}_{\mathrm{ijk}}{\mathrm{PL}}_{im}\left(1-{\mathrm{PP}}_{km}\right)+b_z\underset{i}{.Math.}\underset{k}{.Math.}{z}_{\mathrm{ik}}{\mathrm{PL}}_{\mathrm{im}}\left(1-{\mathrm{PP}}_{km}\right)B_m,m=1\mathrm{.Math.}M& \left(12\right)\end{array}$

(113) S.sub.lm, PL.sub.im, PP.sub.km are inputs to the model and can easily be changed. .sub.ijl are retrieved from analysing what each controller wants to display (the requested layout). Equations (11) and (12) model the flux out and in of cluster M where the links to M are limited by bandwidth B.sub.m. Adding the constraints (11) and (12) to the model allows to take into account network and limitations. Further, this model can comprise sources and controllers that are mobile since S.sub.lm and PL.sub.im are variable. The cluster size is arbitrary; it can be one single compositor or source (in which case the parentheses in (11) and/or (12) are cancelled out). The cluster definition allows to model communication over local and remote networks, such as e.g. remote collaboration.

(114) The present invention relates to a variety of digital devices with processing capability such as application servers, database servers etc. Each one of these can include one or more microprocessors, processors, controllers, or central processing unit (CPU) and/or a Graphics Processing Unit (GPU), and can be adapted to carry out their respective functions by being programmed with software, i.e. one or more computer programmes. Any, some or all of these devices may have memory (such as non-transitory computer readable medium, RAM and/or ROM), an operating system, optionally a display such as a fixed format display, data entry devices such as a keyboard, a pointer device such as a mouse, serial or parallel ports to communicate other devices, network cards and connections to connect to any of the networks.

(115) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(116) routing at least one data stream in a telecommunications network having at least one transmitting node providing at least one data stream and at least one receiving node receiving the at least one data stream, and zero or more processing nodes, and at least one link connecting said nodes.

(117) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(118) all nodes communicating their capabilities to a system database on a database server, the database server receiving at least one request with at least one condition for sending at least one data stream between two nodes, and for each request, using the capabilities of all nodes and links stored in the central database at the time of receiving the request, to calculate a measure for at least one route between said two nodes, each route comprising at least one transmitting node, at least one receiving node and zero or more processing nodes, and
providing instructions to use only the at least one route having measures compliant with the at least one condition in the request.

(119) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(120) calculating an optimal route using at least one heuristic rule or linear programming. For each direct pair of stateful nodes, replace hops between them with a direct link, calculate a route cost, and if the route cost is lowered and the required processing power is still provided, keep the direct link.

(121) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(122) For each direct pair of stateful nodes, remove all links between the two nodes and, considering the required processing power, add the shortest path according to one of the Dijkstra algorithm, the Bellman/Ford algorithm; and the Johnson's algorithm.

(123) Software code portions may allow or facilitate, when executed on a processing engine, adding or removing at least one node with processing power, calculate a route cost, if the route cost is lower and the required processing power is still provided, keep the direct link, whereby the route cost is monetary cost or latency cost.

(124) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(125) modelling the network with an objective function and use the request to define boundary conditions of the variables in the model, and minimize or maximize the objective function, whereby, the model can comprise variables for at least one processing type and/or at least one window. The variables can be binary and each variable can have an index counter giving each specific component in the network a unique variable.

(126) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(127) each processing step in the model is associated with a weight variable.

(128) The model is limited by constraints provided in the request, for example cost and/or latency.

(129) The model is limited by actual properties of the network, for example receiving and transmitting bandwidth and/or encoding and decoding capacity.

(130) The objective function can be minimized or maximized within the limits of variable constraints, whereby the solution can be found by using linear programming.

(131) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(132) requesting to send at least two data streams from at least two different transmitting nodes, whereby at least two of the network nodes can use different protocols, and at least two of the network nodes of the calculated route can use different protocols.

(133) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(134) Adding or removing network nodes, or a combination of both, between two events in the network.

(135) Sending at least one data stream via a perimeter network or by means of Transmission Control Protocol tunnelling.

(136) Requesting to send at least one data stream from at least one transmitting node to at least one receiving node in a processed format, calculating the optimal location of a processing node in relation to the transmitting node or the receiving node, sending the at least one data stream to the optimal node for processing, and sending the processed data stream to the at least one receiving node, possibly via another processing node, whereby the processing can be compositing, transcoding, scaling, re-packaging, relaying, tunnelling or transrating or any combination thereof.

(137) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(138) for each of the at least one calculated route, modifying the settings of the at least one link before sending the at least one data stream along the calculated route.

(139) The at least one single source or central server using the communicated capabilities of nodes to create an empiric model of each node and associated link.

(140) The request condition comprises any of latency, cost, computation time, quality of the data stream, frame rate, accepted loss rate, the priority of a data stream or any combination thereof.

(141) The software can be embodied in a computer program product adapted to carry out the following functions when the software is loaded onto the respective device or devices and executed on one or more processing engines such as microprocessors, ASIC's, FPGA's etc.:

(142) A measure comprises any of latency, cost, computation time, quality of the data stream, frame rate, accepted loss rate, the priority of a data stream or any combination thereof.

(143) For each point in time, there is an optimal route for one measure or for any combination of a multiple of measures.

(144) A pre-set measure is assigned to at least one link of a route.

(145) The single source or central server keeps samplings of capabilities of nodes at different points in time.

(146) Any of the above software may be implemented as a computer program product which has been compiled for a processing engine in any of the servers or nodes of the network. The computer program product may be stored on a non-transitory signal storage medium such as an optical disk (CD-ROM or DVD-ROM), a digital magnetic tape, a magnetic disk, a solid state memory such as a USB flash memory, a ROM, etc.