PREDICTING A RADIO SPECTRUM USAGE

20220369115 · 2022-11-17

    Inventors

    Cpc classification

    International classification

    Abstract

    A computer implemented method predicts by a neural network a radio spectrum usage in an environment comprising interferers and a plurality of wireless nodes. The nodes operate via a slotted medium access control, MAC, protocol within a radio spectrum. The method includes the following steps: obtaining per slot the radio spectrum usage by sensing the radio spectrum, labelling the slots as usable when a respective radio spectrum usage is lower than a predefined noise threshold or when a node successfully transmitted a packet through it, unusable when it is not usable and suitable to receive a packet, and calculating a loss function as a cross entropy loss with the usable and unusable slots over a total number of usable and unusable slots, and training the neural network by minimizing the loss function.

    Claims

    1-11. (canceled)

    12. A computer implemented method for obtaining a neural network for predicting usage of a radio spectrum in an environment comprising interferers and a plurality of wireless nodes, the wireless nodes operating via a slotted medium access control, MAC, protocol within the radio spectrum, the method comprising the following steps: obtaining per slot usage of the radio spectrum by sensing the radio spectrum; labelling the slots as: usable when a respective radio spectrum usage is lower than a predefined noise threshold or when a node successfully transmitted a packet through it; unusable when the slot is not labelled as usable but the slot is suitable to receive a packet according to the MAC protocol; and calculating a loss function as a cross entropy loss with the usable and unusable slots over a total number of usable and unusable slots for a certain prediction; and training the neural network to predict usable and unusable slots by minimizing the loss function.

    13. The computer implemented method according to claim 12, further comprising the step of: optimizing layers of the neural network.

    14. The computer implemented method according to claim 13, wherein the optimizing further comprises: optimizing an output layer of the neural network by a softmax activation.

    15. The computer implemented method according to claim 12, further comprising the step of: predicting by the neural network an upcoming radio spectrum usage of the interferers.

    16. The computer implemented method according to claim 12, wherein the slotted MAC protocol is a time division multiple access, TDMA, protocol, a multiple frequencies time division multiple access, MF-TDMA protocol, or a slotted carrier sense multiple access, CSMA, protocol.

    17. The computer implemented method according to claim 12, wherein the sensing is performed by a software defined radio, SDR, platform.

    18. A neural network for predicting usage of a radio spectrum in an environment comprising interferers and a plurality of wireless nodes obtained by the computer implemented method according to claim 12.

    19. A data processing system comprising means for carrying out the method according to claim 12.

    20. A computer program product comprising computer-executable instructions for performing the method according to claim 12 when the program is run on a computer.

    21. A computer readable storage medium comprising the computer program product according to claim 20.

    22. A wireless communication node configured to operate by a slotted medium access control, MAC, protocol within a radio spectrum; wherein the wireless node is further configured to predict available slots by the neural network according to claim 18.

    Description

    BRIEF DESCRIPTION OF THE DRAWINGS

    [0040] Some example embodiments will now be described with reference to the accompanying drawings.

    [0041] FIG. 1 shows an example embodiment of a ring topology of a network with a known and an unknown network;

    [0042] FIG. 2 shows another example embodiment of an access point, AP, topology of a network with a known and an unknown network;

    [0043] FIG. 3 shows an average overall throughput of a superframe in a multiple frequencies time division multiple access, MF-TDMA, mesh network and interfering networks cluster, INC, for different scheduling approaches;

    [0044] FIG. 4 shows network metrics of a superframe in a network for different scheduling approaches in an MF-TDMA mesh network;

    [0045] FIG. 5 shows an average overall throughput of a superframe in a MF-TDMA AP network and INC for different scheduling approaches;

    [0046] FIG. 6 shows network metrics of a superframe in a network for different scheduling approaches in an MF-TDMA AP network;

    [0047] FIG. 7 shows an average overall throughput of a superframe in a carrier sense multiple access, CSMA, AP network and INC for different scheduling approaches;

    [0048] FIG. 8 shows network metrics of a superframe in a network for different scheduling approaches in an CSMA AP network;

    [0049] FIG. 9 shows an example embodiment of a suitable computing system 900 for performing one or several steps in embodiments of the invention; and

    [0050] FIG. 10 shows an example embodiment an architecture of an algorithm of the invention for an own network comprising three nodes.

    DETAILED DESCRIPTION OF EMBODIMENT(S)

    [0051] In FIG. 1 a ring topology of a network is illustrated. The topology comprises a known network comprising nodes 100-105 which are configured to communicate with each other. The topology further comprises a second unknown network comprising nodes 110-11 which are likewise configured to communicate with each other, but not with the other nodes 100-105. In FIG. 2 another topology is illustrated. The topology comprises nodes 200-206 arranged in an access point, AP, topology. The other nodes 210-211 illustrate a second unknown network. The nodes 110-111 and 210-211 illustrate an interfering networks cluster, INC. The INC interferers with the network of nodes 100-105 respectively nodes 200-206. Thus, a known network comprises a number of known nodes N, representing the own network, and at least a second unknown set of networks, the INC. The INC may comprise noise from other sources, for example microwaves.

    [0052] The nodes 100-105 respectively 200-206 of the own networks have access to C channels and use a slotted medium access control, MAC, protocol. The MAC protocol is, for example, a multiple frequencies time division multiple access, MF-TDMA, protocol, or a slotted random- access protocol. A superframe used in the slotted MAC protocol comprises S time slots. The superframe could thus be equal to the MF-TDMA superframe when using an MF-TDMA protocol, or it could be an artificial created window of S time slots if a slotted random-access protocol is used.

    [0053] Further, t is defined as a global reference of time, wherein t is the number of slots executed after the start of the first radio in the environment. Since all the nodes 100-105 respectively 200-206 use a slotted MAC protocol, all the nodes 100-105 respectively 200-206 execute a slot at exactly the same time, because this is a general property of a slotted MAC protocol. A node n of the nodes 100-105 respectively 200-206 execute an action A.sub.t,c.sup.n ∈{Idle, TX, RX} at time t for every channel c. Furthermore, TX.sub.t,c.sup.n and RX.sub.t,c.sup.n are denoted as:

    [00001] TX t , c n { 1 if A t , c n = TX 0 otherwise , ( Eq . 1 ) RX t , c n { 1 if A t , c n = RX 0 otherwise . ( Eq . 2 )

    [0054] There are no constraints about the number of actions a node can execute at a certain time t. As long as they are executed on different channels. This is denoted as:


    ∀n∈[1,N],∀C∈[1,C], ∀t:TX.sub.t,c.sup.n+RX.sub.t,c.sup.n≤1  (Eq. 3).

    [0055] Further, INC is completely unknown, thus there can be an unknown number of nodes in the INC, using an unknown protocol, or even noise generators could be part of the INC. For the INC P.sub.t,c is defined as the usage of channel c at time t by the INC, where P.sub.t,c=1 if a source in the INC used channel c at time t, and P.sub.t,c=0 otherwise. Due to the fact of fading, power control, modulation and coding scheme, MCS, setting, etc., the interference of a generated signal could be different for different nodes at different positions. Therefore, P.sub.t,c.sup.n is defined as the usage of channel c at time t by the INC with energy above a predefined noise threshold in the interpretation of node n. Where P.sub.t,c.sup.n=1 if a source from the INC produces energy that is the interpretation of node n at time t on channel c is above the noise threshold.

    [0056] To create a better and smarter use of the spectrum, the MAC protocol in the own network avoid collisions and for every transmission there is at least one node of the nodes 100-105 respectively 200-206 listening. This could, for example, be realised by a centralised MF-TDMA algorithm. Communication between the different nodes in the own network is successful if at a given time slot t at least one node is in a RX state and only received a message from one other node of the own network at a given channel c:

    [00002] c [ 1 , C ] , t : .Math. n [ 1 , N ] TX t , c n 1. ( Eq . 4 )

    [0057] Thus, a communication of the own network at time t on channel c is successful if one own node is transmitting on channel c where the destination node will not have any influence of the INC. Parameter y.sub.t,c is used to determine if an own packet could be delivered successfully:

    [00003] γ t , c = { 1 if .Math. n [ 1 , N ] TX t , c n = 1 .Math. .Math. n [ 1 , N ] RX t , c n ( 1 - P t , c n ) = 0 0 otherwise . ( Eq . 5 )

    [0058] Because the INC is unknown and since there is, from an external point of view, no way to know if a transmission interferers with a packet of the INC, it can be assumed that a communication in the INC is successful if y′.sub.t,c=1, where:

    [00004] γ t , c = { 1 if .Math. n [ 1 , N ] TX t , c n = 0 .Math. P t , c n = 0 0 otherwise . ( Eq . 6 )

    [0059] Further, the maximum network packet throughput, Γ, is the sum of all successful transmissions:

    [00005] Γ ^ = .Math. t .Math. c [ 1 , C ] γ t , c . ( Eq . 7 )

    [0060] Likewise, in a same way, the maximum network packet throughput of the INC is:

    [00006] Γ ^ = .Math. t .Math. c [ 1 , C ] γ t , c . ( Eq . 8 )

    [0061] Further, Γ and Γ′ are defined as the actual packet throughput for the own network respectively INC.

    [0062] By the invention, the overall network throughput of all the network in maximised, both the own network as the INC, by choosing the actions A as optimally possible for all channel-time pairs for all the nodes n in the own network. The overall network throughput of the superframe is defined as:


    F(α)=αΓ+(1−α)Γ′with αϵ[0,1]  (Eq. 9),

    where α denotes the relative priority between the own network and the INC.

    [0063] To increase the throughput αand α′, either the amount of transmissions (TX/RX actions) is increased, more data is generated, or the number of collisions is decreased between the INC and the own network. Since there is no control about the data generation and only control of the nodes 100-105 respectively 200-206 in the own network, the throughput is increased by reducing the collisions.

    [0064] Parameter ω.sub.t,c , is defined to be one if there was a potential collision between the INC and the own network at the slot time t on channel c, and Ω is the total of potential collisions:

    [00007] ω t , c = { 0 if .Math. n [ 1 , N ] TX t , c n + P t , c 1 1 otherwise , ( Eq . 10 ) Ω = .Math. t .Math. c [ 1 , C ] ω t , c . ( Eq . 11 )

    [0065] By Eq. 4 it can be assumed that there will be no collisions in the own network thus to maximise F(α), Ω needs to be minimised. Furthermore, every node n can measure energy on all c channels based on hardware technologies such as software defined radios, SDRs.

    [0066] Further, parameter o.sub.t,c.sup.n ϵ[0,1] is defined as the observation made at time t on channel c for node n. Parameter o.sub.t,c.sup.n=1 if the amount of energy detected on channel c on time t for node n is too high for receiving another packet. . Parameter o.sub.t,c.sup.n=0 if there is no energy or a small amount detected on the channel and there would be no influence to receive packets successfully. All the energy is detected, also energy generated by the node itself or neighbouring nodes of the own network.

    [0067] Each node comprises network information and statistics, this is, a node knows if a current slot is allocated inside the own network where TX.sub.t,c=1 if slot at time t on channel c will be used by any node in the own network as described in Eq. 12 below. This may, for example, be implemented by Request-to-Send/Clear-to-Send, RTS/CTS, strategies. Further, each node knows if packets are received correctly ζ.sub.t,c.sup.n=1 when RX.sub.t,c.sup.n=1 and node n received a packet correctly at time t on channel c:

    [00008] TX t , c = n [ 1 , N ] TX t , c n . ( Eq . 12 )

    [0068] As illustrated in FIG. 10, the approach of the invention comprises five components: a spectrum monitor 1000, a pre-processing unit 1001, a predictor unit 1002, a probability matrix on the nodes 1003 and an overall, either centralised or decentralised, scheduler 1010. The components are present for each of the nodes, for example a node A, a node B 1005 and a node C 1006. The components will now be further discussed with reference to node A.

    [0069] The spectrum monitor 1000 captures the energy of the overall spectrum. The captured spectrum monitor 100 information is forwarded to the pre-processing unit 1001. The pre-processing unit 1001 of node A creates the correct observation values O.sub.t,c.sup.n, together with the network schedule TX.sub.t,c, the node schedule A.sub.t,c.sup.n, and success information ζ.sub.t,c.sup.n for node A, and all t in the last superframe on each channel c.

    [0070] For every superframe, the predictor unit 1002 predicts the upcoming spectrum usage that will be generated by the INC. Parameter p.sub.t,c.sup.n is defined as the result of the prediction unit 1002 where p.sub.t,c.sup.n ϵ[0,1] which describes if the slot will be predicted as highly used—a value close to zero—or will be predicted as free—a value close to one. Parameters p.sub.t,c.sup.n are called values of a probability matrix. The probability matrix 1003 is used by the scheduler 1010 to select slots that would be free to avoid collisions. Since the scheduler 1010 can be a centralised or decentralised MF-TDMA scheduler, or a slotted CSMA AP that will define when nodes could transmit. This can be realised in a MF-TDMA network where a sender node makes a request for slots to the receiver, the receiver replies with a best available slot based on the prediction.

    [0071] Thus, the predictor unit 1002 predicts the used slots in an upcoming superframe used by the INC. These predictions could be used as an action by the scheduler 1010 to select slots. The prediction is defined as π.sub.t.sup.n(O.sup.n, TX):custom-character.sup.H×S×C, {0,1}.sup.H×S×Ccustom-charactercustom-character.sup.S×C as the policy, predicted by the prediction unit 1002 of node A at time t based on the observation matrix O.sup.n and the schedule matrix TX where H defines a number of history frames. Additionally, π.sup.+η(O.sup.n, TX) is defined as the theoretically optimal policy.

    [0072] The problem may further be modelled as a reinforcement learning, RL, problem, wherein the order of slots is the action space. This way, the scheduler 1010 may decided how many slots will be selected to deliver a correct amount of packets to a correct destination and can select slots in a given order. The problem is described by using a continuous action space representing a stochastic game with continuous action space thereby avoiding a large amount of possible actions. The problem is further solved by combining supervised learning with RL techniques for solving it. Therefore, three components are described, namely a state, a label and loss function, and a neural network, NN.

    [0073] The state or input features represents that state of the environment in the last H superframes. A state is described as [O.sup.n, TX], where O.sup.n ∈custom-character.sup.H×S×C, where each element in the matrix corresponds to o.sub.t,c.sup.n respectively, and the network schedule information TX ∈{0,1}.sup.H×S×C where each element in the matrix corresponds to TX.sub.t,c.sup.n respectively. By using this representation, it may be indicated when observations could be used to recognise patterns. If TX.sub.t,c=1, a node is using the spectrum at time t in channel c, implying that the observation O.sub.t,c.sup.n may comprise energy from a node in the network and may give a wrong impression about the patterns to be recognised.

    [0074] Further to construct a label, a slot is defined as usable, unusable, or unknown. Therefore, ϕ is defined as a noise threshold. Usable and unusable slots are defined in Eq. 13 and Eq. 14:


    Usable.sub.t,c.sup.n=o.sub.t,c.sup.n<ϕvξ.sub.t,c.sup.n  (Eq. 13),


    Unusable.sub.t,c.sup.n=)¬Usable.sub.t,c.sup.n(¬TX.sub.t,cvRX.sub.t,c.sup.n)  (Eq. 14).

    [0075] In other words, in Eq. 13 usable slots are slots where the amount of energy detected is lower than the noise threshold ϕ or if there was a successful transmission. Otherwise, as stated in Eq. 14, if a slot is not usable and not part of the overall schedule but may be part of the own node receiving schedule, a slot is defined as unusable. The slots in the superframe that are not part of the set usable or unusable are labelled as unknown, or not labelled at all.

    [0076] The loss is further described in Eq. 15:

    [00009] = - .Math. t , c [ Usable t , c n log ( π t , c n ) + Unusable t , c n log ( 1 - π t , c n ) ] .Math. t , c [ Usable t , c n + Unusable t , c n ] , ( Eq . 15 )

    where a cross entropy loss with usable and unusable labels is calculated and divided by the number of slots that may be updated. The loss of unknown slots will be zero, and the NN will try to keep the output values of these slots. To make sure that every sample used for training is as important as all the others, the cross entropy is divided by the number of slots that may be updated. This way, it is prevented that the NN finds a way to start using all slots to decrease the loss by labelling as many slots as possible as unknown.

    [0077] The NN is used to optimise the prediction. As an illustrative embodiment, a NN may be used comprising eight fully connected layers with for each layer

    [00010] S × C × ( 8 - l ) H 8 × 2 neurons ,

    where l is the number of the layer between [1,8]. A swish activation function on each layer may be used, except for the last output layer, where a softmax activation is applied. Further, because data generation, schedule policies, network topologies, and the environment that may change drastically, online learning is preferred. After every δ steps, a training step is executed where at random samples out of the replay buffer are selected. To optimise the weights in the NN, a RMSProp optimiser may be used. As an illustrative example, the following hyperparameter values may be used:

    TABLE-US-00001 TABLE 1 Hyperparameter Value Minibatch size   256 Optimiser RMPSProp RMSProp momentum    0 RMSProp ϵ    10.sup.−10 RMSProp decay    0.9 Learning rate    10.sup.−10−4 Experience replay buffer size 20,000 δ    10

    [0078] Besides different topologies, as illustrated in FIG. 1 and FIG. 2, different approaches for communication between the nodes 100-105 respectively 200-206 may be implemented. As illustrative embodiments, five approaches may be distinguished. A regular approach is used in most systems. In order to reach a high throughput, the regular approach uses all free slots in the internal schedule by the scheduler 1010. In other words, it uses only a traditional MF-TDMA schedule algorithm or CSMA strategy without having collisions in the own network. Secondly, no sending is an extreme approach where the own network will not send any packet at all. This is an extreme case but illustrates an optimal performance of the INC 110-111 respectively 210-211. Fourthly, an optimal approach is the hypothetical best solution to solve the problem when α=0.5. In the optimal approach the own network knows behaviour of the INC upfront, which is not possible in real cases. Lastly, in a deep Q-network, DQN, approach each node of the nodes 100-105 respectively 200-206 uses a DQN, wherein, as an illustrative embodiment, a superframe of size one is used. Finally, the fifth approach is the approach as solved by the invention and which is nominated as the prediction approach.

    [0079] In FIG. 3 the average overall throughput F (α) 300 of a superframe is illustrated for the topology of FIG. 1. It is observed that in the no sending approach, the overall throughput decreases with increasing α 301. On the other hand, in the regular approach the overall throughput increases with increasing α 301. The no sending and regular approach are the two most extreme approaches to indicate the performance of the INC and the own network. Further, as illustrated, the DQN approach underperforms, while the prediction approach by the invention is performing at least as good as the regular approach, when α<0.8, or the optimal approach, when α>0.8.

    [0080] Further, in FIG. 4 different network metrics of the superframe in the network for the different scheduling approaches in the MF-TDMA mesh network are illustrated. In 405 it is illustrated that the prediction approach of the invention reduces the average number of collisions for each superframe with 30% in comparison with the regular scheduling approach. Further, in 400 the throughput is illustrated, in 401 packet success ratio is illustrated, in 402 missed opportunities is illustrated, in 403 noise throughput is illustrated, and in 404 INC packet success ratio is illustrated. It is observed that the prediction approach of the invention is always between the performance of the regular approach and the optimal approach. Since the prediction approach is less aggressive than the regular approach, the throughput is reduced a bit. This leads to less collisions if the module predicts a false positive.

    [0081] In FIG. 5 the average overall throughput F(α) 300 of a superframe is illustrated for the topology of FIG. 2. In this topology the prediction approach outer performs the regular approach when α<0.7. Further, the number of collisions is reduced with a factor 1.3, which is illustrated in FIG. 6 reference 605. FIG. 6 illustrates network metrics of the superframe in the network for the different scheduling approaches in the MF-TDMA AP network of FIG. 2. The throughput 600 of the own network is similar and the throughput 600 of the INC is increased with a factor 1.17 as illustrated in 600. In FIG. 6 the metrics packet success ratio 601, missed opportunities 602, INC throughput 603 and INC Packet success ratio 604 are further illustrated.

    [0082] In FIG. 7 the average overall throughput F(α) 300 of a superframe is illustrated for a CSMA AP topology. In FIG. 8 network metrics related to this topology are illustrated.

    [0083] FIG. 9 shows a suitable computing system 900 enabling to implement embodiments of the method for predicting a radio spectrum usage in an environment according to the invention.

    [0084] Computing system 900 may in general be formed as a suitable general-purpose computer and comprise a bus 910, a processor 902, a local memory 904, one or more optional input interfaces 914, one or more optional output interfaces 916, a communication interface 912, a storage element interface 906, and one or more storage elements 908. Bus 910 may comprise one or more conductors that permit communication among the components of the computing system 900. Processor 902 may include any type of conventional processor or microprocessor that interprets and executes programming instructions. Local memory 904 may include a random-access memory (RAM) or another type of dynamic storage device that stores information and instructions for execution by processor 902 and/or a read only memory (ROM) or another type of static storage device that stores static information and instructions for use by processor 902. Input interface 914 may comprise one or more conventional mechanisms that permit an operator or user to input information to the computing device 900, such as a keyboard 920, a mouse 930, a pen, voice recognition and/or biometric mechanisms, a camera, etc. Output interface 916 may comprise one or more conventional mechanisms that output information to the operator or user, such as a display 940, etc. Communication interface 912 may comprise any transceiver-like mechanism such as for example one or more Ethernet interfaces that enables computing system 900 to communicate with other devices and/or systems, for example with other one or more of the nodes 100-105 or 200-206. The communication interface 912 of computing system 900 may be connected to such another computing system by means of a local area network (LAN) or a wide area network (WAN) such as for example the internet. Storage element interface 906 may comprise a storage interface such as for example a Serial Advanced Technology Attachment (SATA) interface or a Small Computer System Interface (SCSI) for connecting bus 910 to one or more storage elements 908, such as one or more local disks, for example SATA disk drives, and control the reading and writing of data to and/or from these storage elements 908. Although the storage element(s) 908 above is/are described as a local disk, in general any other suitable computer-readable media such as a removable magnetic disk, optical storage media such as a CD or DVD, -ROM disk, solid state drives, flash memory cards, . . . could be used. Computing system 900 could thus correspond to a node in the embodiments illustrated by FIG. 1 or FIG. 2.

    [0085] As used in this application, the term “circuitry” may refer to one or more or all of the following:

    [0086] (a) hardware-only circuit implementations such as implementations in only analog and/or digital circuitry and

    [0087] (b) combinations of hardware circuits and software, such as (as applicable): [0088] (i) a combination of analog and/or digital hardware circuit(s) with software/firmware and [0089] (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and

    [0090] (c) hardware circuit(s) and/or processor(s), such as microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g. firmware) for operation, but the software may not be present when it is not needed for operation.

    [0091] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and/or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in a server, a cellular network device, or other computing or network device.

    [0092] Although the present invention has been illustrated by reference to specific embodiments, it will be apparent to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied with various changes and modifications without departing from the scope thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the scope of the claims are therefore intended to be embraced therein.

    [0093] It will furthermore be understood by the reader of this patent application that the words “comprising” or “comprise” do not exclude other elements or steps, that the words “a” or “an” do not exclude a plurality, and that a single element, such as a computer system, a processor, or another integrated unit may fulfil the functions of several means recited in the claims. Any reference signs in the claims shall not be construed as limiting the respective claims concerned. The terms “first”, “second”, third”, “a”, “b”, “c”, and the like, when used in the description or in the claims are introduced to distinguish between similar elements or steps and are not necessarily describing a sequential or chronological order. Similarly, the terms “top”, “bottom”, “over”, “under”, and the like are introduced for descriptive purposes and not necessarily to denote relative positions. It is to be understood that the terms so used are interchangeable under appropriate circumstances and embodiments of the invention are capable of operating according to the present invention in other sequences, or in orientations different from the one(s) described or illustrated above.