Online learning based smart steering system for wireless mesh networks
10952120 ยท 2021-03-16
Assignee
Inventors
Cpc classification
H04W24/10
ELECTRICITY
International classification
Abstract
An online learning-based smart steering system and method comprises an online kernel perceptron classifier. The smart steering approach/method performs learning sequentially at the cloud from the entire data of multiple mesh networks, and, operates at APs for steering; both of which are executed in real-time. The system and method uses network features such as current RSSI (received signal strength indicator), current cost, target RSSI and target cost for every steering action resulting in either success of failure. The system and method uses a classifier based on this data for obtaining a model in the feature space which predicts whether a steering action can succeed or not before it is issued.
Claims
1. A method comprising: observing steering actions employed in a cloud-based network traffic steering system for clients in a large-scale wireless mesh network; data collection; where data pertaining to instances of steering actions are collected in the cloud, feature selection; where a plurality of descriptive operational network features are determined at a time of each respective steering action instance, model training; where a classification problem with successful steering actions and unsuccessful steering actions is formulated based on the descriptive operational network features, and at least one classifier model is trained to learn nonlinear critical regions of successful steering actions in the collected data; and issuing a steering action request to a client device in the large-scale wireless mesh network at an instance where a steering action is taken by the client device, the steering action request issued based on identification, with the trained at least one classifier model, of a non-linear critical region of successful steering actions being present in the large-scale wireless mesh network during the instance.
2. The method set forth in claim 1, wherein the classification problem of said model training step is formulated as a support vector machine algorithm such that nonlinearity is achieved.
3. The method as set forth in claim 1, wherein the classification problem of said model training step is formulated as an online data driven kernel perceptron algorithm whereby nonlinearity and computational scalability are achieved.
4. The method as set forth in claim 1, wherein said outcomes of said model training step, namely online kernel perceptron and support vector machine models are switchable in the cloud.
5. The method as set forth in claim 1, wherein said plurality of descriptive operational network features in said feature selection step are selected from a group including, but not limited to, current received signal strength indicator (RSSI), current connection cost, target RSSI and target connection cost.
6. The method as set forth in claim 1, wherein said data in the data collection step is configured to include a result of a steering action taken by a device utilizing said method enabling sequential and real-time training of said at least one classifier, whereby adaptiveness to network conditions is achieved.
7. The method of claim 1, wherein the cloud comprises a server computer accessible over the internet.
8. A system comprising: a processor configured to execute instructions stored in non-transitory memory to observe steering actions employed in a cloud-based network traffic steering system for clients in a large-scale wireless mesh network; the processor further configured to execute instructions stored in memory to collect data; where data pertaining to instances of steering actions are collected in the cloud; the processor further configured to execute instructions stored in memory to select features; where a plurality of descriptive operational network features are determined at a time of each respective steering action instance; the processor further configured to execute instructions stored in memory to train a model; where a classification problem with successful steering actions and unsuccessful steering actions is formulated based on the descriptive operational network features, and at least one classifier model is trained to learn nonlinear critical regions of successful steering actions in the collected data; and the processor further configured to execute instructions stored in memory to issue a steering action request to a client device in the large-scale wireless mesh network at an instance where a steering action is taken by the client device, the steering action request issued based on identification, with the trained at least one classifier model, of a non-linear critical region of successful steering actions being present in the large-scale wireless mesh network during the instance.
9. The system of claim 8, wherein the cloud comprises a server computer accessible over the internet.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Accompanying drawings are given solely for the purpose of exemplifying an adaptive, data-driven, highly scalable and cloud-based online smart steering method for wireless mesh networks, whose advantages over prior art were outlined above and will be explained in brief hereinafter.
(2) The drawings are not meant to delimit the scope of protection as identified in the claims nor should they be referred to alone in an effort to interpret the scope identified in said claims without recourse to the technical disclosure in the description of the present invention.
(3)
(4)
DETAILED DESCRIPTION
(5) A cloud-based online system and method assesses the success or failure chances of the steering actions for clients in Wi-Fi mesh networks using a cloud based smart steering algorithm (SSA). Said system and method includes circuitry and logic which operates based on a computationally efficient model based upon an algorithm, stored in the cloud, that treats conditions as a learnable classification problem with successful and unsuccessful attempts at steering clients.
(6) As used herein, the term cloud refers to a server computer accessible over a network. The server computer may include circuitry, such as a processor and memory storing instructions executable by the circuitry. In an example, the cloud refers to a server accessible over the internet. The cloud may be included as a network resource of the cloud-based online client steering system or may be part of the cloud-based online client steering system. Alternatively or in addition, the cloud may be provided by a third party provider of services accessible over a network by the cloud-based online client steering system.
(7) The cloud-based online client steering system and method targets the sticky client problem which occurs in wireless mesh networks comprising multiple clients and multiple access points (APs), when a network system performs certain probing functions to observe link metrics and executes steering actions (guiding the client onto a different access point) but client(s) corresponding to the executed action(s) refuse(s) to roam to the new AP.
(8) Steering actions in wireless mesh networks refer to requesting clients to change their respective access points to better exploit mesh networks capabilities and achieve higher quality connections, as well as utilizing network recourses ultimately more efficiently. Especially in the sticky client issue, steering actions do not always produce the intended outcome. This proves to be extremely detrimental especially when the number of clients is large in which case a severely non-linear classification problem between the success and failure in the space of network metrics is encountered regarding the steering decisions. Technical problem therefore of the proposed method is to solve, by a detailed and data-driven success/failure analysis of the steering actions, this sticky client issue for large scale wireless mesh networks in a computationally highly efficient and online manner through robust and powerful online kernel expansion while ensuring the real time adaptability to non-stationarities of the network conditions.
(9) The present system and method is structured around the approach that steering module outcomes may be modeled as a binary classification problem, i.e. a success in steering or a failure in steering. Guidance of a client to another access point or interface, as a sequence of events, constitutes a data set of said steering actions. Typically, whenever a client is considered to be better off if it was connected to another access point or interface, then, an appropriate steering action (guidance) is issued. Such an action does not always necessarily result in the intended outcome. The present system and method affords solution thereto by proposing, in a set of known steering actions of number N, successful and unsuccessful steering action labels based on the features pertaining to the conditions a client is particularly subject to. Such features regard the network at the time of a steering action. In an embodiment of the disclosed system and method, said features are: received signal strength indicator (RSSI) and cost of the current as well as the targeted connections of the client.
(10) According to an embodiment of the disclosed system and method, besides received signal strength indicator (RSSI), cost is an end-to-end metric that provides additional information about the connection quality. Measured between the client and the associated access point (AP) or between the root AP with gateway and the other APs, the cost metric per link is computed periodically by checking all possible steering connections, to evaluate initiating a steering action. Cost is therefore considered as a parameter summarizing the environmental, network- or vendor-specific effects on steering actions that are not counted in the remaining features. According to an embodiment of the disclosed system and method, cost per link is calculated as:
(11)
where R is the physical layer link (data) rate.
(12) Current and target signal strength as well as the costs for the current and targeted connections constitute altogether a four-dimensional feature vector for each action, generating the data:
xR.sup.N4
along with the feedbacks, i.e. labels:
Y{,1}.sup.N1
(13) The approach of the disclosed system and method is to train a classifier based on this data for obtaining a model in the four-dimensional feature space which predicts whether a steering action can succeed or not, before it is issued. In different embodiments, feature space may have a greater dimensionality, and said feature vector(s) may concurrently contain a greater number of features. This classifier suggests a steering outcome on each single point in the feature space, even it is not attempted before by the given steering module. This enables the exploration of the entire space and, also, if need be, the correction of the failing decision points of the existing steering module. This consequently enables decision making as to when exactly a steering action is to be requested. Namely, a decision can be made to not follow the existing module; and instead rely on the ones predicted by the model in the disclosed system and method which readily defines a novel and data-driven, adaptive steering solution that is, due to training with feedback, superior against rule-based non-adaptive versions thereof.
(14) According to an embodiment of the disclosed system and method, a central agent responsible for requesting the steering actions may be simultaneously observing a large number of mesh networks, e.g. a mesh network multiplicity of different apartments in a building, each of which may be serving multiple clients. The devised model f(x) shall be trainable such that it is computationally scalable to such large scale networks with relatively small space complexity and adaptive to non-stationarity, i.e., possible changes in the network behavior. This realistic processing requirement hinders the use of batch techniques and makes online algorithms more favorable. Therefore, the disclosed system and method introduces a classification problem in the online processing framework in which the data is received sequentially in time (then the data is time-indexed as xt, yt without a known horizon like T). At each time, an instance of steering action is received and then the current model is updated based on the feedback. Afterwards, the received instance is discarded without being stored so the storage complexity is significantly small. In this framework of online processing, the computational complexity scales only linearly with the number of processed instances, and thus, scalability can be achieved with limited storage needs.
(15) As a result, steering decision outcomes are sequentially observed in the 4-dimensional vector space, and the goal is to learn an online classifier that is sequentially updated at each time with respect to the observed loss i.e., cross entropy loss, induced by the feedback:
y.sub.t{1,1}
(16) In addition, constant magnitude updates are used, rather than the conventional update diminishing at a 1/t rate, for adaptation to non-stationarity or drifting network parameters. The proposed solution is a data-driven, adaptive and online, essentially a smart steering module which is a significantly novel contribution to the literature and expected to open-up new directions in managing large scale mesh networks.
(17) In the envisioned system model shown in
(18) Batch analysis leading to the development of batch ML for smart steering enables development of the online learning approach, which is named as Online Machine Learning (ML) for Smart Steering. To this end, disclosed system and method exploits a stochastic gradient descent based minimization of the cross entropy loss that leads to Rosenblatt's perceptron in a compact kernel space for online nonlinear classification, which is called online kernel perceptron. The online kernel perceptron algorithm, which is again executed at each AP but trained at the cloud based on the entire data, considers only instantaneous values of steering data from all clients (the entire data at the cloud is processed by the online technique sequentially in an online manner without storing, whereas the batch technique stores the entire data and uses it repeatedly). Note that in a mesh network, an AP is able to continuously monitor all of the clients and their features. Therefore, in both cases of the envisioned batch or online scenarios, an AP can decide when exactly to issue a successful steering request (for its own clients), while monitoring the clients based on the classifier model it received from the cloud. This gives the disclosed system and method the merit of smart steering.
(19) The method for learning complex, i.e., nonlinear, decision boundaries in the online setting as disclosed in the present system and method exploits one of the central ideas in machine learning for solving nonlinear classification problems, that is, to lift, i.e., transform, the data to a high dimensional space in which the data is linearly separable and hence linear methods are applicable. In addition, methods of the disclosed system does not need to explicitly construct that high dimensional space except the pairwise affinities after the transform, rather, it is sufficient to know the inner products that are encoded by function known as the kernel. Nevertheless, as kernel techniques (especially in the case of certain kernels such as RBF) in its original form require to compute all of the pairwise kernel evaluations (from test to train) every time an instance is tested, which is usually prohibitively computationally and space-wise complex with even more problematic aspect of training, explicitly but also compactly constructing (though not needed in principle) the high dimensional space through random projections in a compact manner emerges as a way to go.
(20) An example objective of the disclosed system and method is to guarantee online processing, therefore the transformation (i.e. lifting to high dimension) as well as the classifier are both designed online. According to an embodiment of the present system and method, perceptron is selected for the online classifier. For the transformation, a compact kernel expansion via a random set of Fourier projections is selected, which does not disturb online processing.
(21) According to an embodiment of the disclosed system and method, for any two instances such as:
x.sub.i,x.sub.jR.sup.d1
from the feature space, we produce the corresponding set of transformed instances such that:
x.sub.i,x.sub.j.fwdarw.z.sub.i,z.sub.jR.sup.2D1
k(x.sub.i,x.sub.j)
can be approximated arbitrarily well by:
z.sub.i.sup.Tz.sub.j
as D increases. This approximation allows exploitation of computationally efficient online linear learners after the transformation, which is equivalent in principle to training a highly powerful kernel classifier, e.g., SVM with the radial basis function (RBF) kernel, in the original space. The motive behind using the kernel is to replace the conventional similarity, i.e., dot products in the original space since most linear techniques do only rely on dot products, with an appropriate similarity measure encoded by a kernel. This approximation directly embeds that information into an explicitly constructed known space.
(22) According to an embodiment of the disclosed system and method, radial basis function kernel is chosen such that:
k(x.sub.i,x.sub.j)=exp(gx.sub.ix.sub.j.sup.2)(1)
where g is the bandwidth parameter. In the batch classification analysis part of the disclosed system and method, the bandwidth parameter g of (1) is empirically determined with 5-fold cross validation in the SVM part and used in the online classification part as is without a need to update. Moreover, any symmetric shift invariant kernel under Mercer's conditions can be used straightforwardly in the presented system and method. In this current embodiment, the rbf kernel is used for only a demonstration with no loss in generality.
(23) The approximation that:
k(x.sub.i,x.sub.j)z.sub.i.sup.Tz.sub.j
is based on random Fourier projections as an application of the Bochner's theorem known in the art. The radial basis function kernel is shift-invariant and symmetric. Considering this, Bochner's theorem reads as follows:
(24)
p() is a proper probability density, thus E(.) is the expectation with respect to the multivariate Gaussian density:
p()=N(; ,)
with zero mean and covariance of 2 gI, where I is the identity matrix of the appropriate size. Hence,
(25)
is the transform with the desired approximation holding due to the law of large numbers (sample mean approximating the expectation), and
{.sub.1,.sub.2, . . . ,.sub.D}
is a set of independent and identically distributed samples generated from said multivariate Gaussian density. Said density is chosen Gaussian for the reason by Bochner's theorem that it is the Fourier transform of the RBF kernel, and as such makes the approach working for any shift-invariant and symmetric kernel selectable under Mercer's condition.
(26) Hence, what is obtained explicitly and in a randomized fashion, is the high dimensional space (which is in fact infinite dimensional) implied by the RBF kernel; wherein dot products approximate kernel values in the original space so that training a linear classifier in the high dimensional space will correspond to nonlinear modeling in the original space as desired. This construct is advantageous for following cases: One can truncate the dimensionality of the transformation to a desired degree so that it fits to the computational requirements of the application at hand; and the rate of convergence is fast, so the quality of the approximation gets better at an exponential rate in D in accordance with Hoeffding's inequality. Thus, the dimension of the transform (D) may be chosen relatively small enabling efficiency regarding the computation of transformation for different embodiments.
(27) Having the kernel space explicitly and compactly constructed via the transform for each instance in a given data stream (without a definitive end), then a linear classifier in the z-space may solve any non-linear classification problem provided that the kernel is chosen suitably, e.g. rbf kernel with an appropriate kernel parameter g. For this purpose, a perceptron which trains a linear classifier in the online setting through stochastic updates is utilized.
(28) A time-indexed online linear classifier as:
f.sub.t(x)=w.sub.t1.sup.Tx+b.sub.t1
parametrized over
(w.sub.t1,b.sub.t1)(2)
suffers the loss
l(f.sub.t1(x.sub.t),y.sub.t)
at time t. If the loss is chosen as the cross-entropy loss such that
(29)
where
(30)
then the stochastic gradient update on the model parameters as given in expression (2), yields
(w.sub.t,b.sub.t)
This essentially defines the perceptron algorithm provided that the sigmoid steepness, v, in expression (3) is sufficiently large.
(31) According to an embodiment of the disclosed system and method, perceptron is applied, after the random kernel expansion, to the transformed stream for online non-linear modeling. Resulting online kernel perceptron is computationally greatly superior over support vector machine algorithm since it sequentially processes data with linear complexity, whereas SVM is a batch algorithm of at least quadratic complexity. Performance-wise, online kernel perceptron is comparable to SVM. From an empirical standpoint, online kernel perceptron is much stronger since it is able to learn even complex separations in the sequential setting in a computationally superior manner.
(32) Online kernel perceptron as disclosed in the system and method can be used as Online Machine Learning (ML) for Smart Steering for large scale networks in the envisioned system model with reference to
(33) With reference to
(34) According to an embodiment of the disclosed system and method, a step of data collection is proposed; where data pertaining to instances of steering actions are collected in the cloud. (204) (200)
(35) According to an embodiment of the disclosed system and method, a step of feature selection is proposed; where a plurality of descriptive network features is determined. (206)
(36) According to an embodiment of the disclosed system and method, a step of model training is proposed; where a classification problem is formulated based on the features and at least one classifier is trained to learn the nonlinear critical regions of successful steering actions in collected data. (208) Such training of the at least one classifier may occur in the cloud. (200)
(37) According to an embodiment of the disclosed system and method, a step of steering action taking; where a steering action is taken by a device based on the trained model, is proposed. Thus, a steering action request is issued to the cloud based on the trained model. (210) (200)
(38) According to an embodiment of the disclosed system and method, the classification problem of said model training step is formulated as a support vector machine algorithm such that nonlinearity is achieved.
(39) According to an embodiment of the disclosed system and method, the classification problem of said model training step is formulated as an online data driven kernel perceptron algorithm whereby nonlinearity and computational scalability are achieved.
(40) According to an embodiment of the disclosed system and method, said outcomes of said model training step, namely online kernel perceptron and support vector machine models are switchable in the cloud.
(41) According to an embodiment of the disclosed system and method, said plurality of descriptive network features in said feature selection step are selected from a group including, but not limited to, current RSSI, current connection cost, target RSSI and target connection cost.
(42) According to an embodiment of the disclosed system and method, said data in the data collection step is configured to include a result of a steering action taken by a device utilizing said method enabling sequential and real-time training of said classifiers, whereby adaptiveness to network conditions is achieved.
(43) According to an embodiment of the disclosed system and method, a steering module for clients within a large-scale wireless mesh network characterized employing a client steering method as set forth is proposed.
(44) The methods, devices, processing, circuitry, and logic described above may be implemented in many different ways and in many different combinations of hardware and software. For example, all or parts of the implementations may be circuitry that includes an instruction processor, such as a Central Processing Unit (CPU), microcontroller, or a microprocessor; or as an Application Specific Integrated Circuit (ASIC), Programmable Logic Device (PLD), or Field Programmable Gate Array (FPGA); or as circuitry that includes discrete logic or other circuit components, including analog circuit components, digital circuit components or both; or any combination thereof. The circuitry may include discrete interconnected hardware components or may be combined on a single integrated circuit die, distributed among multiple integrated circuit dies, or implemented in a Multiple Chip Module (MCM) of multiple integrated circuit dies in a common package, as examples.
(45) Accordingly, the circuitry may store or access instructions for execution, or may implement its functionality in hardware alone. The instructions may be stored in a tangible storage medium that is other than a transitory signal, such as a flash memory, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM); or on a magnetic or optical disc, such as a Compact Disc Read Only Memory (CDROM), Hard Disk Drive (HDD), or other magnetic or optical disk; or in or on another machine-readable medium. A product, such as a computer program product, may include a storage medium and instructions stored in or on the medium, and the instructions when executed by the circuitry in a device may cause the device to implement any of the processing described above or illustrated in the drawings.
(46) The implementations may be distributed. For instance, the circuitry may include multiple distinct system components, such as multiple processors and memories, and may span multiple distributed processing systems. Parameters, databases, and other data structures may be separately stored and managed, may be incorporated into a single memory or database, may be logically and physically organized in many different ways, and may be implemented in many different ways. Example implementations include linked lists, program variables, hash tables, arrays, records (e.g., database records), objects, and implicit storage mechanisms. Instructions may form parts (e.g., subroutines or other code sections) of a single program, may form multiple separate programs, may be distributed across multiple memories and processors, and may be implemented in many different ways. Example implementations include stand-alone programs, and as part of a library, such as a shared library like a Dynamic Link Library (DLL). The library, for example, may contain shared data and one or more shared programs that include instructions that perform any of the processing described above or illustrated in the drawings, when executed by the circuitry.
(47) The subject-matter of the disclosure may also relate, among others, to the following aspects:
(48) 1) A cloud-based network traffic steering method for clients in a large-scale wireless mesh network to be employed in a steering module, comprising steps of;
(49) data collection; where data pertaining to instances of steering actions are collected in the cloud,
(50) feature selection; where a plurality of descriptive network features is determined,
(51) model training; where a classification problem is formulated based on the features and at least one classifier is trained to learn the nonlinear critical regions of successful steering actions in collected data; and,
(52) steering action taking; where a steering action is taken by a device based on the trained model.
(53) 2) A cloud-based network traffic steering method for wireless mesh networks as set forth in aspect 1, characterized in that the classification problem of said model training step is formulated as a support vector machine algorithm such that nonlinearity is achieved.
(54) 3) A cloud-based network traffic steering method for wireless mesh networks as set forth in aspects 1 and 2, characterized in that the classification problem of said model training step is formulated as an online data driven kernel perceptron algorithm whereby nonlinearity and computational scalability are achieved.
(55) 4) A cloud-based network traffic steering method for wireless mesh networks as set forth in aspects 1 to 3, characterized in that said outcomes of said model training step, namely online kernel perceptron and support vector machine models are switchable in the cloud.
(56) 5) A cloud-based network traffic steering method for wireless mesh networks as set forth in any preceding aspect 1-4, characterized in that said plurality of descriptive network features in said feature selection step are selected from a group including, but not limited to, current RSSI, current connection cost, target RSSI and target connection cost.
(57) 6) A cloud-based network traffic steering method for wireless mesh networks as set forth in any preceding aspect 1-5, characterized in that said data in the data collection step is configured to include a result of a steering action taken by a device utilizing said method enabling sequential and real-time training of said classifiers, whereby adaptiveness to network conditions is achieved.
(58) 7) A steering module for clients within a large-scale wireless mesh network characterized in that said module is configured to execute a client steering method as set forth in aspect 1.
(59) While various embodiments have been described, it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible. Accordingly, the embodiments described herein are examples, not the only possible embodiments and implementations.