Facilitating real-time transport of data streams
11695818 · 2023-07-04
Assignee
- Koninklijke Kpn N.V. (Rotterdam, NL)
- Nederlandse Organisatie voor toegepast-natuurwetenschappelijk onderzoek TNO ('s-Gravenhage, NL)
Inventors
Cpc classification
H04L65/65
ELECTRICITY
H04L47/225
ELECTRICITY
H04L47/24
ELECTRICITY
International classification
H04L65/65
ELECTRICITY
H04L47/24
ELECTRICITY
Abstract
An interface may be provided between i) a selective forwarding unit (SFU) configured to, in real-time, receive a data stream from a sender via a first network link of a communication network and selectively forward the data stream to one or more receivers via respective second network links, and ii) one or more core network functions (PCF, PCRF, NSMF, CSMF) for establishing service guarantees for data flows in the communication network. In a specific example, the interface may be established as a network function (SMGF) which translates streaming requirements for one-to-many flows coming from WebRTC SFUs into appropriate QoS/network slice configurations, such that the quality of RTC flows may be increased. Accordingly, negative side-effects of conservative congestion control algorithms in WebRTC clients and static/overprovisioned QoS at network operators may be overcome.
Claims
1. A network node or a distributed system of network nodes for facilitating real-time transport of data streams in a communication network, wherein the communication network comprises: one or more nodes each comprising a network interface and a processor subsystem implementing one or more core network functions for establishing service guarantees for data flows in the communication network, the one or more core network functions comprising at least one of a policy control function and a network management function; at least one selective forwarding unit (SFU) configured to, in real-time, receive a data stream from a sender via a first network link of the communication network and selectively forward the data stream to one or more receivers via respective second network links; wherein the network node or the distributed system comprises: a network interface configured to interface with the communication network; a processor subsystem configured to interface between the selective forwarding unit and the one or more core network functions by: receiving stream information from the selective forwarding unit, the stream information being indicative of a first bandwidth associated with transport of the data stream via a select one or more network links, the select one or more network links being one or more of a group of: the first network link and the respective second network links; based on the stream information, determining a second bandwidth based on the first bandwidth and requesting the one or more core network functions to establish a service guarantee for the transport of the data stream via the select one or more network links by requesting the second bandwidth to be allocated for the transport of the data stream via the select one or more network links; and based on the stream information, providing bitrate information to the selective forwarding unit, the bitrate information being indicative of a target bitrate to be used for the transport of the data stream via the select one or more network links.
2. The network node or the distributed system of network nodes according to claim 1, wherein the one or more core network functions comprise a policy control function, and wherein the processor subsystem is configured to request the policy control function to establish a quality-of-service (QoS)-based service guarantee for the transport of the data stream via the select one or more network links.
3. The network node or the distributed system of network nodes according to claim 1, wherein the one or more core network functions comprise a network management function, and wherein the processor subsystem is configured to request the network management function to establish the service guarantee via allocation or configuration of a network slice for the transport of the data stream.
4. The network node or the distributed system of network nodes according to claim 1, wherein the processor subsystem is configured to determine the second bandwidth as: the first bandwidth; or a function of the first bandwidth and resource information indicative of a bandwidth allocation and/or bandwidth capacity of the select one or more network links.
5. The network node or the distributed system of network nodes according to claim 1, wherein the processor subsystem is configured to determine the target bitrate based on at least one of: a response received from the one or more core network functions which is indicative of a guaranteed bandwidth; and a function of the first bandwidth and resource information indicative of a bandwidth allocation and/or bandwidth capacity of the select one or more network links.
6. A communication network comprising the network node or the distributed system of network nodes according to claim 1, wherein the network node or the distributed system is configured as a network function of the communication network for providing a network function-based interface between the selective forwarding unit and the one or more core network functions.
7. A computer-implemented method for facilitating real-time transport of data streams in a communication network, wherein the communication network comprises: one or more core network functions for establishing service guarantees for data flows in the communication network; at least one selective forwarding unit (SFU) configured to, in real-time, receive a data stream from a sender via a first network link of the communication network and selectively forward the data stream to one or more receivers via respective second network links; wherein the method comprises interfacing between the selective forwarding unit and the one or more core network functions by: receiving stream information from the selective forwarding unit, the stream information being indicative of a first bandwidth associated with transport of the data stream via a select one or more network links, the select one or more network links being one or more of a group of: the first network link and the second network links; based on the stream information, determining a second bandwidth based on the first bandwidth and requesting the one or more core network functions to establish a service guarantee for the transport of the data stream via the select one or more network links by requesting the second bandwidth to be allocated for the transport of the data stream via the select one or more network links; and based on the stream information, providing bitrate information to the selective forwarding unit, the bitrate information being indicative of a target bitrate to be used for the transport of the data stream via the select one or more network links.
8. A non-transitory computer-readable medium comprising a computer program, the computer program comprising instructions for causing a processor system to perform the method according to claim 7.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter. In the drawings,
(2)
(3)
(4)
(5)
(6)
(7)
(8) SMGF receives a stream update from the SFU;
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16) It should be noted that items which have the same reference numbers in different figures, have the same structural features and the same functions, or are the same signals. Where the function and/or structure of such an item has been explained, there is no necessity for repeated explanation thereof in the detailed description.
LIST OF REFERENCE AND ABBREVIATIONS
(17) The following list of references and abbreviations is provided for facilitating the interpretation of the drawings and shall not be construed as limiting the claims. AF Application Function AMF Access and Mobility Management Function BBERF Bearer Binding and Event Reporting Function CHF Charging Function GTWY Gateway NAT Network Address Translation NEF Network Exposure Function NOP Network Operator NWDAF Network Data Analytics Function PCC Policy and Charging Control PCEF Policy and Charging Enforcement Function PCF Policy Function PCRF Policy and Charging Rules Function RCAF Radio Congestion Awareness Function RTC Real-Time Communication SCEF Service Capability Exposure Function SFU Selective Forwarding Unit SMF Session Management Function SMGF Stream Management and Guidance Function SPR Subscription Profile Repository TDF Traffic Detection Function TURN Traversal Using Relay NAT OCS Online Charging System OFCS Offline Charging System QoS Quality of Service UDR User Data Convergence UE User Equipment UPF User Plane Function N4-N40, Np, Nt, Nu, Rx, S1, Sd, Sp, Sy interfaces Gx, Gxx, Gy, Gyn, Gz, Gzn, Gw, Gwn interfaces 10-15 user equipment 20-23 network link 30-32 data stream 40 network part 50 selective forwarding unit 60 stream graph 100 mobile network 110 3GPP LTE/5G reference points 120 service guarantee management 130 SMGF 140 3.sup.rd party application interface 150 streaming guidance 160 WebRTC SFU 200 collect information on active streams 210 assign streaming bitrates 220 configure network 230 inform SFU of streaming bitrates 300 receive stream update 310 update stream graph 320 identify affected network links 325 decision 330 network links have aggregated bandwidth maximum 340 identify affected stream(s) 350 assign target bitrate(s) 360 request QoS 365 decision 370 request denied 380 inform WebRTC SFU 400 receive stream update 405 update stream graph 410 identify affected network slice(s) 415 identify affected stream(s) 420 assign (temporary) target bitrates 425 inform WebRTC SFU 430 forecast demand 435 decision 440 change triggered 445 determine required capacity 450 adapt network slice(s) 455 decision 460 request denied 465 await network slice changes 470 identify affected stream(s) 475 assign target bitrates 480 inform WebRTC SFU 500 network node configured as SMGF 510 network interface 520 processor subsystem 530 data storage 600 network node configured as SFU 605 user equipment 610 network interface 620 processor subsystem 630 data storage 700 method of facilitating real-time transport of data streams 710 receiving stream information from selective forwarding unit 720 requesting establishment of service guarantee 730 providing target bitrate to selective forwarding unit 800 method of selectively forwarding data streams 810 providing provide stream information to entity 820 obtaining target bitrate from entity 830 providing target bitrate to sender 900 computer readable medium 910 non-transitory data 1000 exemplary data processing system 1002 processor 1004 memory element 1006 system bus 1008 local memory 1010 bulk storage device 1012 input device 1014 output device 1016 network adapter 1018 application
DETAILED DESCRIPTION OF EMBODIMENTS
(18) The following embodiments are described in the context of real-time transport of data streams based on WebRTC and the transport taking place via a communication network adhering to one or more 3GPP standards. It will be appreciated, however, that the concepts described in the following embodiments may equally apply, mutatis mutandis, to any other type of real-time transport of data streams and any other type of communication network with core network functions capable of performing the functions as defined by the wording of the claims.
(19)
(20)
(21) As shown in
(22) According to 3GPP TS 23.501, the 5G QoS model is based on QoS Flows, which may support both GBR flows and non-GBR flows. Within a PDU session, QoS Flows may be identified using QoS Flow IDs (QFIs), where flows with the same QFI may receive similar treatment within a PDU session. QoS may be controlled by the Session Management Function (SMF). QoS Flows may be treated differently based on their QoS Profile, which, among other parameters, may include a 5G QoS Identifier (5QI; similar to the QCI used in LTE), and a Guaranteed Flow Bitrate (GFBR) and Maximum Flow Bitrate (MFBR) for GBR QoS Flows.
(23)
(24) In view of the similarities between the PCRF and PCF, they will be used interchangeability (as PCRF/PCF) in the following, and may in general be considered as policy control functions via which a QoS-based service guarantee may be established.
(25) Besides QoS, 5G networks may also offer WebRTC streams service guarantees and separation through so-called network slices. Network slices may offer a virtual isolated network which may be customized to the Quality of Service requirements of the applications. According to 3GPP TR 28.801 (v15.1.0), network slice management may be performed by a Communication Service Provider (CSP), for example through the following two functions that may be provided by a Network Operator (NOP): Network Slice Management Function (NSMF): The NSMF may allow the creation, configuration, and termination of Network Slice Instances (NSIs). Network Slice Subnet Management Function (NSSMF): The NSSMF may allow for the creation, configuration, and termination of Network Slice Subnet Instances (NSSIs) in which specialized network functions can be deployed.
(26) The NSMF may be available to a CSP. CSPs may offer a (limited) interface to Communication Service Customers (CSCs) in the form of the Communication Service Management Function (CSMF), see, e.g., 3GPP study item TR 28.801: Communication Service Management Function (CSMF): The CSMF may allow a CSC to specify the service requirements for a network slice and translates these service requirements into network slice related requirements. The CSMF may trigger a reconfiguration of an NSI and NSSI when the service requirements change.
(27) The relation between the different management functions related to network slices, as proposed in 3GPP TR 28.801 (v15.1.0), may be the following: a CSMF may delegate to a NSMF which may delegate to a NSSMF.
(28) In general, core network functions such as the NSMF and CSMF may be considered network management functions which enable establishments of service guarantees for the transport of data stream(s) via allocation or configuration of a network slice.
(29)
(30)
(31) 1. Manage service guarantees in the mobile networks through the reference points of the 3GPP LTE and 5G systems (depicted as ‘Service Guarantee Management 120’), and
(32) 2. Guide WebRTC clients (depicted as ‘Streaming Guidance 150’), by collecting information about the current load and requirements of RTC streams, and providing information about optimal streaming bitrates and bandwidth availability, for example through a third-party application interface 140 between the SMGF 130 and WebRTC SFUs 160.
(33)
Data Structure
(34) In some embodiments, the SMGF may maintain data structures associated with the collected information on the active streams. Such information may for example include the connected client endpoints, the currently active streams, and the current quality of service agreements per network link. The information for the connected endpoints may include, but is not limited to, on or more of the following types of information: Application-level endpoint identifier, Application-level client identifier, Network-level endpoint address (e.g., IP-address, transport protocol, and port number), (If a relay is used, see also below) Network-level address of relay (e.g., IP-address, transport protocol, and port number).
(35) If a WebRTC client receives more than one stream, or if a client is both a sender and a receiver, the client may be listed as multiple endpoints in the data structure.
(36) It is noted that a network-level address of a relay server may be specified, for example when a TURN server is used to connect an SFU to a WebRTC peer. When using a TURN server, the network-level endpoint address may be the address that the WebRTC client uses when connecting to the TURN server. The network-level address of the relay may be the address of the TURN server that is used on the connection with the SFU.
(37) Example 1. An example of an endpoint data structure in JSON format, with three endpoints “e1”, “e2”, and “e3” and where and points “e1” and “e2” belong to the same client “c1” and where endpoint “e3” is making use of a TURN server, is given below:
(38) TABLE-US-00001 { “endpoints”: [{ “id”: “e1”, “client”: “c1”, “address”: “250.142.4.116”, “port”: “UDP/49153” }, { “id”: “e2”, “client”: “c1”, “address”: “250.142.4.116”, “port”: “UDP/49268” }, { “id”: “e3”, “client”: “c2”, “address”: “233.85.25.238”, “port”: “TCP/56213” “relay”: { “address”: “128.66.222.55”, “port”: “TCP/61172” } }] }
(39) The information for the currently active streams may include: Application level stream identifier, Application level endpoint identifier of the sender List of application level endpoints of the receivers Network service requirements (e.g., minimum required bandwidth or maximum latency).
(40) The number of senders may be limited to one sender per stream. The number of receivers of the stream may be variable. Many-to-many communication may be represented as multiple streams, each with multiple receivers.
(41) Examples of network service requirements which may be specified in the data structure include, but are not limited to, the following: Minimum required bandwidth: the minimum bandwidth in bit/s of an RTC flow that may be required for the RTC application to function; Preferred bandwidth: the bandwidth in bit/s of an RTC flow such that the RTC application may optimally function; Current bandwidth: the bandwidth in bit/s that may be made available by the SMGF to an RTC flow; Maximum expected bandwidth: the maximum bandwidth in bit/s of an RTC flow that the RTC application may request for an RTC flow (e.g., to allow for dynamically changing the streaming bitrate of RTC flows);
(42) Example 2. An example of the data structure for active streams in JSON format, with two streams of which stream “s1” is a one-to-one stream and stream “s2” has multiple receivers, is given below:
(43) TABLE-US-00002 { “streams”: [{ “id”: “s1”, “sender”: “e1”, “receivers”: [“e3”], “qos”: { “min-bw”: 240000, “preferred-bw”: 1200000, “current-bw”: 1200000, “max-bw”: 2200000, } }, { “id”: “s2”, “sender”: “e2”, “receivers”: [“e1”, “e3”, “e4”], “qos”: { “min-bw”: 360000, “preferred-bw”: 1800000, “current-bw”: 1650000, “max-bw”: 1800000, } }] }
(44) It is noted that in this example, the current available bandwidth for an RTC flow does not necessarily have to match the preferred bandwidth. Namely, the SMGF may deviate from the preferred bandwidth when it may not deliver the service guarantee.
(45) The data structure with the information about the current QoS agreements per network may include, but is not limited to: Network identifier, List of endpoints that are associated with the network, QoS agreement.
(46) In some embodiments, the QoS agreement may be specified on the bases of flows. The parameters that form the specification of the QoS agreement may then become: Current maximum bandwidth per flow: The maximum bandwidth in bits/s that the SMGF may assign to a WebRTC stream at current time; Absolute Maximum bandwidth per flow: The maximum bandwidth in bits/s for WebRTC streams, specified based on an agreement between the application service and a network operator or CSP;
(47) In other embodiments, the QoS agreement may be specified on the basis total (e.g., aggregated) bandwidth usage of the service in a network. The QoS agreement specification may then include, but is not limited to: Current maximum aggregated bandwidth: The maximum aggregated bandwidth in bits/s that the SMGF may assign to all WebRTC streams in a network combined, at current time; Absolute maximum aggregated bandwidth: The maximum aggregated bandwidth in bits/s for WebRTC stream combined, specified based on an agreement between the application service and a network operator or CSP.
(48) Based on the aggregated bandwidth, the SMGF may derive the maximum bandwidth per flow. By using aggregated bandwidth instead of per-flow maximums, the SMGF may have more flexibility in dividing the available bandwidth over the active flows. When needed, the QoS agreement may also be a combination of per-flow and aggregated bandwidth.
(49) It is noted that for both the maximum per flow bandwidth and the aggregated bandwidth, the current maximum bandwidth may not exceed the absolute maximum bandwidth. Differentiation between current and absolute maxima may allow for efficient resource usage when the load on the networks is lower than agreed on with the network operator or CSP.
(50) QoS agreements for the uplink and downlink do not have to be the same, and therefore may be specified separately.
(51) Example 3. An example of the data structure for agreements between a network operator or CSP and a third-party WebRTC service in JSON format, with two agreements for networks “n1”, and “n2”, where the agreement for “n1” is specific on per-flow basis and the agreement for “n2” is specified on the basis of a maximum aggregated maximum, is:
(52) TABLE-US-00003 { “agreements”: [{ “network”: “n1”, “endpoints”: [“e1, e2”], “qos”: { “uplink”: { “stream-bw”: XX, “max-stream-bw”: XX, }, “downlink”: { “stream-bw”: XX, “max-stream-bw”: XX, } },{ “network”: “n2”, “endpoints”: [“e3”], “qos”: { “uplink”: { “aggr-bw ”: XX, “max-aggr-bw”: XX }, “downlink”: { “aggr-bw ”: XX, “max-aggr-bw”: XX } }] }
Application Interface
(53) The SMGF may provide (e.g., ‘expose’) an API to the WebRTC SFU for specifying the service requirements. The API may, for example, include the following functions: endpoint_add, this function may be used by the WebRTC SFU to add an endpoint to the list of endpoints in the SMGF. When using this function, the SFU may specify an application level endpoint identifier, a client identifier, a network level address, and optionally the network level address of a TURN server. Using this function more than once with the same endpoint identifier may update (override) the current configuration. Using this function more than once with the same client identifier may add another endpoint for the same client.
(54) Example 4. An example use of this function, where an endpoint is added that directly connects to the SFU (network-level address of the TURN server is set to NULL), is:
(55) endpoint_add(“e1”, “c1”, “250.142.4.116”, “UDP/49153”, NULL)
(56) Example 5. An example of the use of this function, where the network-level address for a TURN server is specified, is:
(57) endpoint_add (“e3”, “c2”, “233.85.25.238”, “TCP/56231”, relay)
(58) where relay is an (optional) object with the following data structure:
(59) TABLE-US-00004 “relay”: { “address”: “128.66.222.55”, “port”: “TCP/61172” }
(60) Failing to specify the endpoint identifier, client identifier, and network-level address may result in not adding the endpoint to the list of identifiers and an error response. endpoint_remove, this function may be used by the WebRTC SFU to remove an endpoint from the list of endpoints in the SMGF. The SFU may specify the endpoint identifier of the endpoint that has to be removed. Specifying a non-existing endpoint identifier may result in an error response. Endpoints that are associated as receiver of a stream may be automatically detached from the stream. When an endpoint that is a sender of a stream is removed, the stream may have to be removed first using the stream_remove function, or this function call may result in an error response.
(61) Example 6. An example of the use of this function is:
(62) endpoint remove (“e3”) client_add, this may be a convenience function that may be used by the WebRTC SFU to add multiple endpoints associated to a single client to the list of endpoints maintained in the SMGF. Using this function multiple times with the same client identifier may: (1) add all endpoints that do not yet exists in the list of endpoints maintained by the SMGF, (2) when the update flag is set, update all endpoints that do exists in the list of endpoints maintained by the SMGF, and (3) when the cleanup flag is set, remove all endpoints that do exists in the list of endpoints maintained by the SMGF but are not specified in the function call.
(63) Example 7. An example of the use of this function, where a client with identifier “c1”, two endpoints, the update flag set, and the cleanup flag no set, is:
(64) client add(“c1”, endpoints, true, false)
(65) where endpoints may be an object with the following data structure:
(66) TABLE-US-00005 “endpoints”: [{ “id”: “e1”, “address”: “250.142.4.116”, “port”: “UDP/49153” }, { “id”: “e2”, “address”: “250.142.4.116”, “port”: “UDP/49268” }]
(67) Similar to the endpoint_add function, WebRTC SFUs may also specify the network-level address of a TURN server.
(68) Example 8. An example of the use of this function, where a client with identifier “c3”, one endpoint including the network-level address of a TURN server, the update flag set, and the cleanup flag set, is:
(69) client add(“c3”, endpoints, true, true)
(70) where endpoints may be an object with the following data structure:
(71) TABLE-US-00006 “endpoints”: [{ “id”: “e3”, “address”: “233.85.25.238”, “port”: “TCP/56213” “relay”: { “address”: “128.66.222.55”, “port”: “TCP/61172” } }] client_remove, this is a convenience function that may be used by the WebRTC SFU to remove all endpoints associated with a client from the list of endpoints maintained in the SMGF. Specifying a client identifier that does not exist may result in an error response.
(72) Example 9. An example use of this function is:
(73) clinet_remove(“c2”) stream_add, this function may be used by the WebRTC SFU to add a new stream to the list of streams that is maintained by the SMGF. The WebRTC SFU may specify the sending endpoint and the service guarantees that it wishes to receive from the network. Specifying an endpoint that does not exist in the list of connected endpoints maintained by the SMGF, or specifying inconsistent service guarantees, may result in an error response.
(74) Example 10. An example use of this function is:
(75) stream add(“s1”, “e1”, qos)
(76) where qos may be an object with the following data structure:
(77) TABLE-US-00007 “qos”: { “min-bw”: 240000, “pref-bw”: 1200000, “max-bw”: 2200000, }
(78) Regarding the qos, the following conditions may apply: all values must be non-negative integers, min-bw must be smaller than or equal to max-bw, pref-bw must be greater than or equal to min-bw, and pref-bw must be smaller than or equal to max-bw. Specifying inconsistent service guarantees may result in an error response.
(79) Calling this function multiple times with the same stream identifier may update (override) the stream specification currently known by the SMGF. stream_remove, this function may be used by the WebRTC SFU to remove a stream from the list of streams maintained by the SMGF. The WebRTC SFU may specify the identifier for the stream that has to be removed. Specifying a stream identifier that does not exists in the SMGF may result in an error response.
(80) Example 11. An example use of this function is:
(81) stream remove(“s1”) stream_update_bw, this function may be used by the WebRTC SFU to change the preferred bandwidth for the specified stream. Specifying a stream that does not exists, or specifying a bandwidth that is inconsistent with the service guarantee specification (see description of the stream_add function), may result in an error response.
(82) Example 12. An example use of this function, where the preferred bandwidth of stream “s1” is updated to 2.2 Mbit/s, is:
(83) stream update bw(“s1”, 2200000) stream_attach_receivers, this function may be used by the WebRTC SFU to attach one or more receiving endpoints to a stream. The WebRTC SFU may specify the stream identifier and provide a non-empty list of endpoint identifiers. Specifying a non-existing stream, or non-existing endpoint, may result in an error response.
(84) Example 13. An example use of this function, where three endpoints are added to stream “s2”, is:
(85) stream_attach_receivers(endpoints)
(86) where endpoints may be an object with the following data structure:
(87) “endpoints”: [“e1”, “e3”, “e4”] stream_detach_receivers, this function may be used by the WebRTC SFU to remove endpoints from the list of receivers of a stream. The WebRTC SFU may specify the stream identifier, and a non-empty list of receiving endpoints that have to be removed. Specifying a non-existing stream, or non-existing endpoint, may result in an error response.
(88) Example 14. An example use of this function, where two endpoints are removed from stream “s2”, is:
(89) stream_detach_receivers(endpoints)
(90) where endpoints may be an object with the following data structure:
(91) “endpoints”: [“e1”, “e4”] streams_get, this function may be used by the WebRTC SFU to retrieve information about streaming bitrates of the WebRTC flows (e.g., the current-bw element in the active streams data structure). By default, this function may return information about all active streams. The SFU may optionally specify a list of stream identifiers to limit the information that is returned. This function may be used to request changes to the streaming bitrates, specifically after making modifications using the functions described above.
(92) Example 15. An example use of this function, where the WebRTC SFU requests information about the streaming bitrates for two streams “S1” and “s2”, is:
(93) streams_get(streams)
(94) where streams may be an optional object with the following data structure:
(95) An example response from the SMGF, which contains information about the bitrates of streams “s1” and “s2” in JSON format, is:
(96) TABLE-US-00008 { “streams”: [{ “id”: “s1”, “bitrate”: “1200000”, },{ “id”: “s2”, “bitrate”: “1650000”, }] }
Application Interface Protocol
(97) The interface between the SMGF and WebRTC SFUs may be on request/response basis, or using a bi-directional communication channel.
Request/Response Interface
(98) In some embodiments, the interface between the SMGF and WebRTC SFUs may follow a request/response-based communication scheme, for example through the implementation of an HTTP-based Representational State Transfer (REST) API. The HTTP “GET”, “POST”, “PUT”, “PATCH” and “DELETE” methods may be used for retrieving, adding, and removing data from the SMGF.
(99) Example 15. An example use of the HTTP-based application interface, where endpoints “e1”, “e3”, and “e4” are added to stream “s1”, is:
(100) TABLE-US-00009 POST /streams/s1/receivers HTTP/1.1 Host: network.example.org Content-Type: application/json Content-Length: 37 { “endpoints”: [“e1”, “e3”, “e4”] }
(101) In this example, the list of endpoints is provided in JSON format.
(102) Example 16. An example use of the HTTP-based application interface, where the WebRTC SFU updates the preferred streaming bitrate for stream “s1”, is:
(103) TABLE-US-00010 PATCH /streams/s1 HTTP/1.1 Host: network.example.org Content-Type: application/x-www-form-urlencoded Content-Length: 37 preferred-bw=1200000
(104) In this example, the preferred bandwidth is provided as form-data in the request body. This may be an alternative encoding to JSON.
(105) Example 17. An example use of the HTTP-based application interface, where the WebRTC SFU removes endpoint “e1” from the SMGF, is:
(106) TABLE-US-00011 DELETE /endpoints/e1 HTTP/1.1 Host: network.example.org Content-Length: 0
(107) The SMGF may return with an empty response on success, or a message in case of an error. WebRTC SFUs may then have to explicitly request the bitrates for active streams. Alternatively, the SMGF may return the bitrate for affected streams after making an update.
Bi-Directional Communication Channel
(108) In some embodiments, the interface between the SMGF and WebRTC SFUs may be implemented using a bi-directional communication channel, for example a WebSocket connection. When using this interface, the communication between the SMGF and WebRTC SFUs may be based on messages. WebRTC SFUs may send messages to make updates. The SMGF may send messages to provide SFUs with changes to the bitrates of the active streams. Each message may contain elements to communicate the action type and its parameters. Such messages may be considered examples of ‘stream information’ as described elsewhere.
(109) Example 18. An example of a message in JSON format, to be send over the bi-directional communication channel, where the message is used to add stream “s1” to the SMGF, is:
(110) TABLE-US-00012 { “action”: “stream_add”, “stream”: { “id”: “s1”, “sender”: “e1”, “qos”: { “min-bw”: 240000, “preferred-bw”: 1200000, “max-bw”: 2200000, } } }
Function Logic
(111) In some specific embodiments, one of the primary functionalities of the SMGF may be to translate the updates for WebRTC steams (e.g., requirements and preferences) which the SMGF receives from the WebRTC SFUs into a concrete network configuration. As part of this translation step, the SMGF may map the stream paths on the network infrastructure, such that the SMGF may determine which network links and streams are affected. It is noted that the network infrastructure may be identified by the SMGF in the form of a (static) configuration or through Software Defined Networking (SDN) techniques. Briefly speaking, based on this information, the SMGF may balance the availability of dedicated network resources, providing matching resources in the uplink and the downlink. Once resources are secured in the network, the SMGF may inform the WebRTC SFU, which, in turn, may inform the connected clients. For example, to balance the streaming bitrates and the network bandwidth guarantees, the following procedure may be used. (Step 1) For each network link, assign each stream that traverses that network link the minimum bandwidth. (Step 2) In case the network link can provide a larger guarantee than the assigned bitrates, divide the remaining capacity over the streams that have a higher preferred bandwidth, weighted by the difference between the preferred and the minimum bandwidth. (Step 3) Next, for each stream select the lowest assigned bitrate as the streaming bitrate. (Step 4) Then, for each network link compute the desired bandwidth guarantees by summing the streaming bitrates for streams that traverse the network link. (Step 5) In case the desired bandwidth guarantee for a network link are higher than the current guarantee, the bandwidth guarantees should be increased. When the desired bandwidth guarantee fora network is lower than the current bandwidth guarantee, it should be reduced. Note that when adjusting bandwidth guarantees is not instant (e.g., in the case of network slicing), a temporary bandwidth assignment can be made following the same procedure described above (i.e., steps 1, 2, and 3), using the current bandwidth guarantees as the network link maximum. It is noted that the above-described functionality of the SMGF, and various aspects thereof, are further described in the following.
Stream Graph to Network Infrastructure Mapping
(112) The SMGF may be configured based on service agreements between WebRTC service providers and network operators. This agreement may specify which network guarantees the SMGF may provide for WebRTC streams (see under “Data Structure” for an example agreement). The SMGF may use the network level addresses of the WebRTC endpoints, and possibly the network level addresses of a TURN server, to link the endpoints and relays to network links. After receiving an update to one of the streams (e.g., a new stream, changing the preferred streaming bitrate, or attaching a receiving endpoint to a stream), the SMGF may have to identify which network links are affected. Because a stream extends both in the direction of the uplink and at least one downlink, multiple network links may be affected by an update. To identify the affected network links, the SMGF may construct a stream graph, which provides a mapping between the paths of the streams and the network links.
(113) Example 19. An example stream graph and network infrastructure mapping are shown in
Balancing Uplink and Downlink Network Guarantees
(114) When both sending and receiving endpoints make use of the mobile network, and the endpoints make use of networks for which the SMGF has an agreement with the NOP or CSP, it may be desirable that the network guarantees that are provided for the uplink match those of the downlink, and vice versa. Therefore, the SMGF may continuously or periodically adjust the network configuration to balance the guarantees for uplink and downlink.
(115) The strategy for balancing the guarantees may depend on the type of agreement between the NOP/CSP and the SMGF, the availability of network resources, and the bitrate preferences for the WebRTC streams. When an agreement is based on the bandwidth per stream, without a maximum of number of streams or an aggregated (e.g., total) bandwidth usage, the SMGF may balance the uplink and downlink network guarantees based on the network link that can provide the least bandwidth. When agreements are based on aggregated bandwidth, for example when network guarantees are provided through network slicing, balancing guarantees potentially involves several streams and network links. The optimal division of bandwidth may, for example, be found using dynamic programming techniques or the procedure outlined in ‘function logic’.
(116) In both cases, the SMGF may, by being appropriately configured, try to satisfy the preferred streaming bitrates as they are specified by the WebRTC SFU. In an optimal scenario, the SMGF may configure the desired bandwidth guarantees. If the bandwidth guarantees fall within the agreement with the NOP/CSP, it is likely that the request for bandwidth guarantees is honored by the mobile network. However, in cases of high load on the network and conflicting agreements with other services, the mobile network may not be able to satisfy the request. When the request of the SMGF is denied, the SMGF may try to obtain network guarantees for a lower bandwidth. The SMGF may inform the WebRTC SFU about the current state, so that the WebRTC service may optimally use the available bandwidth. The SMGF may do so by providing a target bitrate for each stream. The target bitrate may the maximum bitrate that WebRTC clients can use, for which bandwidth guarantees are provided for in the network.
Network Configuration
(117) The role of the SMGF towards the mobile network may be to configure sufficient resources for WebRTC streams. Depending on the agreement towards the WebRTC service, the SMGF may have several options to realize the network performance guarantees, e.g.:
(118) Quality of Service: In some embodiments, the SMGF may act as a broker between the WebRTC SFU and the policy control functions, or specifically the policy and charging functions in the mobile network (e.g., the PCRF and PCF). By configuring QoS, the SMGF may provide WebRTC streams with sufficient bandwidth and enable low-latency delivery. This option may be suitable for use cases that include frequent changes to the preferred streaming bitrate and use cases in which those changes have to be effective quickly.
(119) The SMGF may select QoS mechanisms for providing network guarantees when a minimum bandwidth has to be guaranteed, but there may be no other requirement for the service to separate WebRTC traffic from other traffic in the mobile network. Depending on the type of mobile network, the SMGF may configure QoS bearers or QoS Flows. When one of the network links is LTE, the SMGF may configure a dedicated bearer, e.g., with QCI value 2 (Conversational Video/Live Streaming) and a GBR matching the target bitrate, for each of the streams that are using this network link. When a network link is 5G, a QoS Flow with similar properties as the dedicated bearer (e.g., 5QI value 2) may be configured for each stream.
(120)
(121) It is noted that in
(122) Network slicing: In some embodiments, the SMGF may act as a broker between the WebRTC SFU and the network slice management functions. Changes made to a network slice might take time to be effective. Therefore, the SMGF may be configured to proactively monitor changes in the number and bitrate of active WebRTC streams. In some embodiments, the SMGF may determine when the capacity in a network slice should be increased/decreased based on an upper- and lower-threshold. In other embodiments, the SMGF may make use of machine learning techniques (e.g., time-series forecasting) to forecast the needed capacity.
(123) The SMGF may configure network slices to provide network guarantees, for example when the WebRTC service requires separation of traffic between WebRTC streams and other traffic in the mobile network. In such cases, the WebRTC clients may be trusted to follow the instruction from the WebRTC SFU, and the streams may not need individual QoS to guarantee bandwidth within the network slice. The capacity in a network slice may be fixed, nevertheless changes to streams may occur frequently, although such frequent changes may be subject to the aggregated required bandwidth not changing rapidly (e.g., increasing the streaming bitrate of one stream while reducing the bitrate of another competing stream).
(124)
(125) Because adjusting network slices may take time (in some examples in the order of a few minutes), the SMGF may continue by operating on two sub-processes in parallel. The first sub-process may assign streaming bitrates, for example to handle the change when the network slice is not adapted, or to cover the time until the changes to the network slices are realized. The actions and events that occur in the first sub-process may include: a. Using the information about the network slices that are affected by the stream update, the SMGF may identify which streams are also making use of those network slices in a process step titled ‘Identify affected stream(s)’ 415. b. Based on the current capacity of the network slices, the SMGF may assign each stream a target bitrate in a process step titled ‘Assign (temporary) target bitrates’ 420, e.g., to satisfy preferred streaming bitrate as much as possible. c. The SMGF may inform the WebRTC SFU about the new target bitrates of the affected streams in a process step titled ‘Inform WebRTC SFU’ 425.
(126) In the second sub-process, the network slice(s) may be adapted, for example when significant changes in demand are expected. The actions and events that occur in the second sub-process may include: a. By tracking stream updates, including the latest update, the SMGF may forecast near-future demand (for example, in the order of the next minutes) in the affected network slices in a process step titled ‘Forecast demand’ 430. If the current capacity of the network slices is still appropriate for handling current and future demand, the sub-process may terminate and the target bitrates assigned by the first sub-process remain valid (see decision 335). b. When the forecast indicates a significant change in demand (see decision 335 and decision outcome titled ‘Change triggered’ 440, the SMGF may determine how the network slices have to be adjusted (e.g., increase/decrease the capacity of at least part of the affected network sliced) in a process step titled ‘Determine required capacity’ 445, e.g., taking into the expected demand and the agreements between the WebRTC service and the network operator. c. The SMGF may request adjustments of the network slices in a process step titled ‘Adapt network slice(s)’ 450, for example through the NSMF or CSMF offered by the 5G mobile network. If the request is denied (see decision 455 and decision outcome titled ‘Request denied’ 460), the SMGF may repeat the above-described step “b” until a suitable configuration is found, or may terminate the sub-process when no suitable configuration can be found. d. After requesting the changes to the network slices, the SMGF may await the network slice changes in a process step titled ‘Await network slice changes’ 365, for example by awaiting a signal from the NSMF/CSMF indicating that the network slice changes have been implemented. e. Given the changes to the network slice capacities, the SMGF may identify which streams are affected and may need to change their streaming bitrates in a process step titled ‘Identify affected stream(s)’ 470. f. Based on the new capacity of the network slices, the SMGF may assign each stream a target bitrate in a process step titled ‘Assign target bitrates’ 475, for example to satisfy the preferred streaming bitrate as much as possible. g. The SMGF may inform the WebRTC SFU about the new target bitrates of the affected streams in a process step titled ‘Inform WebRTC SFU’ 480, thereby overriding the (temporary) target bitrates from the first sub-process.
(127) Network slicing with QoS: In other embodiments, the SMGF may translate the streaming requirements into network slices, e.g., by allocating and/or configuring network slices, and may configure QoS through the PCFs of those network slices. This may be desirable when creating a dedicated WebRTC network slice, but when the clients may or cannot be trusted. This may be the case when multiple WebRTC services make use of the same network slice and QoS may be required to prevent competing services from using bandwidth that is supposed to be for another service. The process for handling a stream update may be similar to only using network slices, with the exception of an additional step of configuring QoS after assigning the target bitrates.
(128) Client Connectivity: Depending on whether the client device and/or the network makes use of firewalls or NAT, WebRTC clients may connect to the SFU either directly (see ‘direct connection’ below) or using a relay (e.g., a TURN server, see ‘relayed connection’).
(129) Direct connection: In some embodiments, WebRTC clients may connect directly to the SFU which may be located in the (LTE/5G) core network (e.g., in the form of an AF) or the SFU may be located in the NOP network (e.g., in a cloud computing facility from the NOP). In this case, the SMGF may offer network performance guarantees on the full path between the WebRTC clients and the SFU. When the SFU is located outside of the NOP network (e.g., in a public cloud or in facilities from the WebRTC service provider), the SMGF may offer network guarantees up until the gateway.
(130) Relayed connection: In some embodiments, at least one of the WebRTC clients may connect to the SFU through a TURN server. When the TURN server is located close to the client (e.g., at the multi-access edge cloud), the SMGF may configure network guarantees in the Radio Access Network, from the UE to the edge. Furthermore, the SMGF may configure network guarantees in the LTE/5G Core Network or backhaul network to facilitate the link between the TURN server and the SFU. When the TURN server is located near the SFU, the SMGF may configure network guarantees on the path from the WebRTC client to the TURN server.
(131) Use Cases: Relevant use cases include, but are not limited, to the following:
(132) Use case 1: Team Video Conferencing and Collaboration
(133) In one use case, the SMGF may support bi-directional many-to-many video communications where clients are both senders and receivers. The canonical use case is team conferencing and collaboration. Enterprises may use video conferencing between employees, partners, and customers.
(134) Use case 2: Massive Video Surveillance
(135) In another use case, the SMGF may support many-to-one unidirectional video streaming with a high number of senders. An example use case is massive video surveillance. Deploying a video surveillance infrastructure may be challenging due to high installation costs. A wireless surveillance system may provide a flexible alternative, with the surveillance cameras connected to the mobile network. Camera feeds may be uploaded to a mobile monitoring unit.
(136) In this location, the camera feeds are monitored by people.
(137) Use case 3: Autonomous Driving
(138) In another use case, the SMGF may support many-to-many real-time communication of data streams. Self-driving cars may collect an exceptional amount of data which may have to be processed. Autonomous vehicles may exchange (part of) this data to better determine location and context, improving safety and robustness.
Data Processing Entities
(139)
(140) The SMGF 500 may further comprise a processor subsystem 520 which may be configured, e.g., by hardware design or software, to perform the operations described with reference to
(141) In general, the SMGF 500 may be embodied by a (single) device or apparatus. For example, the SMGF 500 may be embodied by a single network node, e.g., a network server. The SMGF 500 may also be embodied by a distributed system of such devices or apparatuses. An example of the latter may be the functionality of the SMGF 500 being distributed over different network nodes of a network, for example using virtualization techniques.
(142)
(143) The SFU 600 may further comprise a processor subsystem 620 which may be configured, e.g., by hardware design or software, to perform the operations described with reference to
(144) In general, the SFU 600 may be embodied by a (single) device or apparatus. For example, the SFU 600 may be embodied by a single network node, e.g., a network server. The SFU 600 may also be embodied by a distributed system of such devices or apparatuses. An example of the latter may be the functionality of the SFU 600 being distributed over different network nodes of a network, for example using virtualization techniques.
(145) Although described with reference to the SFU,
(146) In general, the SMGF 500 of
(147)
(148)
(149) It will be appreciated that, in general, the operations or steps of method 700 of
(150) It is noted that any of the methods described in this specification, for example in any of the claims, may be implemented on a computer as a computer implemented method, as dedicated hardware, or as a combination of both. Instructions for the computer, e.g., executable code, may be stored on a computer readable medium 900 as for example shown in
(151) In an alternative embodiment of
(152)
(153) The data processing system 1000 may include at least one processor 1002 coupled to memory elements 1004 through a system bus 1006. As such, the data processing system may store program code within memory elements 1004. Further, processor 1002 may execute the program code accessed from memory elements 1004 via system bus 1006. In one aspect, data processing system may be implemented as a computer that is suitable for storing and/or executing program code. It should be appreciated, however, that data processing system 1000 may be implemented in the form of any system including a processor and memory that is capable of performing the functions described within this specification.
(154) Memory elements 1004 may include one or more physical memory devices such as, for example, local memory 1008 and one or more bulk storage devices 1010. Local memory may refer to random access memory or other non-persistent memory device(s) generally used during actual execution of the program code. A bulk storage device may be implemented as a hard drive, solid state disk or other persistent data storage device. The processing system 1000 may also include one or more cache memories (not shown) that provide temporary storage of at least some program code in order to reduce the number of times program code must be retrieved from bulk storage device 1010 during execution.
(155) Input/output (I/O) devices depicted as input device 1012 and output device 1014 optionally can be coupled to the data processing system. Examples of input devices may include, but are not limited to, for example, a microphone, a keyboard, a pointing device such as a mouse, or the like. Examples of output devices may include, but are not limited to, for example, a monitor or display, speakers, or the like. Input device and/or output device may be coupled to data processing system either directly or through intervening I/O controllers. A network adapter 1016 may also be coupled to data processing system to enable it to become coupled to other systems, computer systems, remote network devices, and/or remote storage devices through intervening private or public networks. The network adapter may comprise a data receiver for receiving data that is transmitted by said systems, devices and/or networks to said data and a data transmitter for transmitting data to said systems, devices and/or networks. Modems, cable modems, and Ethernet cards are examples of different types of network adapter that may be used with data processing system 1000.
(156) Memory elements 1004 may store an application 1018. It should be appreciated that data processing system 1000 may further execute an operating system (not shown) that can facilitate execution of the application. The application, being implemented in the form of executable program code, can be executed by data processing system 1000, e.g., by processor 1002. Responsive to executing the application, the data processing system may be configured to perform one or more operations to be described herein in further detail.
(157) In one aspect, for example, data processing system 1000 may represent the SMGF. In that case, application 1018 may represent an application that, when executed, configures data processing system 1000 to perform the functions described herein with reference to the SMGF. In another aspect, data processing system 1000 may represent the SFU. In that case, application 1018 may represent an application that, when executed, configures data processing system 1000 to perform the functions described herein with reference to the SFU. In another aspect, data processing system 1000 may represent a UE. In that case, application 1018 may represent an application that, when executed, configures data processing system 1000 to perform the functions described herein with reference to the UE.
(158) In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb “comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article “a” or “an” preceding an element does not exclude the presence of a plurality of such elements.
(159) Expressions such as “at least one of” when preceding a list or group of elements represent a selection of all or of any subset of elements from the list or group. For example, the expression, “at least one of A, B, and C” should be understood as including only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.