A METHOD AND SYSTEM FOR CONTROLLING THE FLIGHT OF A PLURALITY OF QUADCOPTERS

Abstract

A method and system for controlling the flight of a plurality of quadcopters includes communicating a flight instruction from a user to a leader quadcopter. The method includes calculating a leader formation maneuver and a follower formation maneuver with a leader-follower formation controller configured to use affine transformations and stress matrices to convert a flight instruction into a leader formation maneuver and a follower formation maneuver. The method may include communicating the follower formation maneuver from the leader quadcopter to a follower quadcopter. The method may include executing the leader formation maneuver on the leader quadcopter. The method may include executing the follower formation maneuver on the follower quadcopter.

Claims

1. A method for controlling the flight of a plurality of quadcopters, comprising: communicating a flight instruction from a user to a leader quadcopter; calculating a leader formation maneuver and a follower formation maneuver with a leader-follower formation controller configured to use affine transformations and stress matrices to convert the flight instruction into a leader formation maneuver and a follower formation maneuver; communicating the follower formation maneuver from the leader quadcopter to a follower quadcopter; executing the leader formation maneuver on the leader quadcopter; and executing the follower formation maneuver on the follower quadcopter.

2. The method of claim 1, wherein the leader-follower formation controller is further configured to utilize a barrier function to calculate the leader formation maneuver and the follower formation maneuver.

3. The method of claim 2, wherein the barrier function is a Lyapunov candidate function which trends to infinity at a predetermined constraint value.

4. The method of claim 2, wherein the leader-follower formation controller is further configured to use an actor-critic learning mechanism.

5. The method of claim 4, wherein the actor-critic learning mechanism is a machine learning algorithm.

6. The method of claim 1, wherein the leader-follower formation controller is further configured to use a distributed sliding mode control.

7. The method of claim 6, wherein the distributed sliding mode control is configured to use a control law to protect at least one of the plurality of quadcopters from a malicious cyber-attack.

8. The method of claim 7, wherein the malicious cyber-attack is a distributed denial of service attack, a sensor deception attack, or an actuator injection attack.

9. The method of claim 7, wherein the distributed sliding mode control is configured to use a Nussbaum gain function to mitigate the effects of input gain on the follower formation maneuver, the input gain being created by a malicious cyber-attack.

10. The method of claim 1, wherein the follower formation maneuver comprises: a scaling maneuver; a shearing maneuver; a translation maneuver; and a collinearity maneuver.

11. The method of claim 10, wherein the position of each of the plurality of quadcopters within the follower formation maneuver is defined by: an x position; a y position; a z position; a roll angle; a yaw angle; and a pitch angle.

12. The method of claim 1, wherein the leader-follower formation controller is configured to use a radial basis function neural network to smooth a flight instruction after the flight instruction has been communicated from a user to a leader quadcopter.

13. A method for controlling the flight of a plurality of quadcopters, comprising: communicating a flight instruction from a user to a leader quadcopter; calculating a leader formation maneuver and a follower formation maneuver with a leader-follower formation controller configured to use a distributed sliding mode control and an actor-critic learning mechanism to convert a flight instruction into a leader formation maneuver and a follower formation maneuver; communicating the follower formation maneuver from the leader quadcopter to a follower quadcopter; executing the leader formation maneuver on the leader quadcopter; and executing the follower formation maneuver on the follower quadcopter.

14. The method of claim 13, wherein the sliding mode control is configured to convert the flight instruction into a follower formation maneuver with affine transformations and stress matrices.

15. The method of claim 13, wherein the actor-critic learning mechanism is a machine learning algorithm and the machine learning algorithm is further configured to use a radial basis function to smooth a flight instruction after the flight instruction has been communicated from a user to a leader quadcopter.

16. The method of claim 13, wherein the sliding mode control and the actor-critic learning mechanism are configured to use a control law to protect at least one of the plurality of quadcopters from a malicious cyber-attack.

17. The method of claim 16, wherein the distributed sliding mode control and the actor-critic learning mechanism are configured to use a Nussbaum gain function to mitigate the effects of input gain on a follower formation maneuver, the input gain being created by a malicious cyber-attack.

18. The method of claim 16, wherein the malicious cyber-attack is a distributed denial of service attack, a sensor deception attack, or an actuator injection attack.

19. A system for controlling a plurality of quadcopters, the system comprising: a user input device configured to receive a flight instruction from a user and deliver the flight instruction to a leader quadcopter; a leader quadcopter configured to: receive a flight instruction from a user input device; calculate a leader formation maneuver; calculate a follower formation maneuver with a leader-follower formation controller configured to use affine transformations and stress matrices to calculate the follower formation maneuver; communicate the follower formation maneuver to a follower quadcopter; and execute the leader formation maneuver; and a follower quadcopter configured to: receive the follower formation maneuver; and execute the follower formation maneuver.

20. The method of claim 19, wherein the leader-follower formation controller is further configured to use an actor-critic machine learning algorithm to calculate the follower formation maneuver.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] A more complete appreciation of this disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:

[0012] FIG. 1A depicts a block diagram of a system for controlling a plurality of quadcopters, according to certain embodiments.

[0013] FIG. 1B depicts several formation shape maneuvers of a plurality of quadcopters, according to certain embodiments.

[0014] FIG. 2 depicts a nominal formation of the quadrotors, according to certain embodiments.

[0015] FIG. 3A depicts a graph of a leader-follower time-varying position trajectories in x-axis, according to certain embodiments.

[0016] FIG. 3B depicts a graph of a leader-follower time-varying position trajectories in y-axis, according to certain embodiments.

[0017] FIG. 4A depicts a graph of tracking errors of followers in the x-axis, according to certain embodiments.

[0018] FIG. 4B depicts a graph of tracking errors of followers in the y-axis, according to certain embodiments.

[0019] FIG. 5A depicts a graph of actor-critic learning control protocols of the followers in the x-axis, according to certain embodiments.

[0020] FIG. 5B depicts a graph of actor-critic learning control protocols of the followers in the y-axis, according to certain embodiments.

[0021] FIG. 6A-FIG. 6H depicts a graph of Norm-2 of actor weights, according to certain embodiments.

[0022] FIG. 7A-FIG. 7H depicts a graph of Norm-2 of critic weights, according to certain embodiments.

[0023] FIG. 8 depicts a graph of leader-follower affine formation maneuvers, according to certain embodiments.

[0024] FIG. 9 depicts a flowchart of a method for controlling the flight of a plurality of quadcopters, according to certain embodiments.

[0025] FIG. 10 depicts a flowchart of a method for controlling the flight of a plurality of quadcopters, according to certain other embodiments.

[0026] FIG. 11 depicts a block diagram of a computer and/or networked device configured to communicate with at least one quadcopter of the plurality of quadcopters according to certain embodiments.

[0027] FIG. 12 depicts a block diagram of networked device configured to communicate with at least one quadcopter of the plurality of quadcopters according to certain other embodiments.

[0028] FIG. 13 depicts a block diagram of a central processing unit configured to process information related to the controlling of at least one quadcopter of the plurality of quadcopters according to certain embodiments.

[0029] FIG. 14 depicts a network diagram of a communication network configured to relay information to and from at least one networked device configured to communicate with at least one quadcopter of the plurality of quadcopters according to certain embodiments.

DETAILED DESCRIPTION

[0030] In the drawings, like reference numerals designate identical or corresponding parts throughout the several views. Further, as used herein, the words a, an and the like generally carry a meaning of one or more, unless stated otherwise.

[0031] Furthermore, the terms approximately, approximate, about and similar terms generally refer to ranges that include the identified value within a margin of 20%, 10%, or preferably 5%, and any values therebetween.

[0032] The term affine formation maneuver refers to a maneuver of a plurality of objects in formation which retain a spatial relationship consistent with an affine transformations. The term affine transformation may refer to the class of linear mapping methods which preserves points, straight lines, and planes, but do not necessarily preserve Euclidian distances or angles. Therefore, the geometry of a formation at a first point in time will have the same points, straight lines, and planes of the geometry of the formation after an affine formation maneuver has been performed.

[0033] The terms leader and follower may describe the independently controlled components of a leader-follower control scheme respectively. In a leader-follower control scheme, at least one member of the control scheme is chosen as a leader and the leader(s) may dictate or decide the whole formation group's moving trajectory, typically by communicating commands to the non-deciding follower members. In one embodiment of a leader-follower control scheme a user may send user commands to leader vehicles, thereby externally controlling the leaders. The leaders may then send information about the user commands and/or send follower commands to the follower vehicles which may act on those instructions, thereby internally controlling the followers. In another embodiment, the user commands may be additionally or alternatively be sent to the followers by a user or a separate computer and/or networked device.

[0034] According to an embodiment, an actor-critic learning scheme for safe leader-follower affine formation maneuver control of networked quadrotor unmanned aerial vehicles (UAVs) is described in the face of external disturbances, sensor deception attacks, and injection attacks on the actuators. Typically, the follower quadrotors (followers) aim to track the formation maneuvers such as scaling, shearing, translation, and rotation decided by the leader quadrotors (leaders). Motivated by increasing safety and performance requirements during formation maneuvering, the dynamic states of the quadrotor UAVs are constrained within the prescribed safety region. The term dynamic states may refer to the full range of actual positions/formations that the UAVs may be arranged in during operation.

[0035] To guarantee that the safety constraints are not violated, a barrier Lyapunov function may be used. A distributed sliding mode control with actor-critic learning is also formulated and implemented to facilitate accurate leader-follower affine formation maneuvers and reject malicious cyber-attack signals. A sliding mode control may refer to any nonlinear control method that alters the dynamics of a nonlinear system, such as a flight path, by applying a discontinuous control signal which forces the system to slide along a cross-section of the system's normal behavior thereby regulating the system. The term actor-critic learning may refer to a machine reinforcement learning system wherein software is trained on the outcomes of actions its decisions have informed such that it will make more effective decisions in the future.

[0036] Additionally, input gains that arise due to the attacks might corrupt the control direction and so in the present disclosure, a Nussbaum gain function is coupled to the controller to address input gain corruption. The actor system may be responsible for estimating uncertain dynamics together with the malicious attack signals, while the critic network evaluates the control performance through an estimated long-term performance index. In the method and system of the present disclosure, the overall stability of the closed-loop system can be uniformly bounded using a Lyapunov stability function.

[0037] In view of the foregoing discussion, the present disclosure provides a solution for leader-follower affine formation maneuver control of networked quadrotor unmanned aerial vehicles (UAVs) subjected to external disturbances, sensor deception attacks, and actuator injection attacks. By considering the safety and physical limitations of the quadrotor UAVs, their dynamic states are constrained to operate within a safe workspace. To prevent violation of the safety constraints, in the method and system of the present disclosure, a barrier Lyapunov function is involved to improve the safety of the system. The safety of the system may refer to the operational conditions under which the formation may maneuver without risking a loss of control of an individual UAV, collisions between UAVs, or some other undesirable flight path for the formation. Then, the actor-critic learning-based distributed sliding mode control technique is designed to aid the leader-follower formation maneuvers of the quadrotor UAVs within the prescribed workspace while mitigating cyber-attacks. The present disclosure provides a method and system that utilizes the properties of affine transformation and stress matrices, and various collective affine formation maneuvers of the networked quadrotor UAVs such as scaling, translation, rotation, shearing, and collinearity. Moreover, the quadrotors in this technology have improved freedom of maneuverability. The present disclosure addresses a safety-guaranteed leader-follower affine formation maneuver control problem associated with multiple quadrotor UAVs under sensor deception attacks and actuator injection attacks. Compared with the conventional systems, where the problem of input gains induced by attack signals was not considered, the present disclosure addresses this problem by integrating the Nussbaum function into the controller of the present disclosure. As such, the controller of the present disclosure does not require any prior information about the signs of the input gains induced by the attack signals to counteract them. In conventional systems the learning-based controllers are in the form of linear quadratic regulators, in contrast, the present disclosure utilizes a distributed sliding mode control approach integrated with a learning mechanism.

[0038] According to an embodiment, the quadrotor of the present disclosure is a system having six degrees of freedom with four control inputs. Assuming that the quadrotor framework is rigid and symmetric, and the center of gravity coincides with the body-frame origin, the dynamic equations of the displacement and rotation of the quadrotor are as given by equations (1) to (6):

[00001]$\begin{array}{cc}{\stackrel{`}{}}_i=\frac{1}{I_x}\left[\left(I_y-I_z\right){\stackrel{`}{}}_i{\stackrel{`}{}}_i-a_1{\stackrel{`}{}}_i^2-I_p{\stackrel{`}{}}_i+{\mathrm{lu}}_{2i}\right]+{}_i& \left(1\right)\end{array}$$\begin{array}{cc}{\stackrel{`}{}}_i=\frac{1}{I_y}\left[\left(I_z-I_x\right){\stackrel{`}{}}_i\stackrel{`}{}-a_2{\stackrel{`}{}}_i^2+I_p{\stackrel{`}{}}_i+{\mathrm{lu}}_{3i}\right]+{}_i& \left(2\right)\end{array}$$\begin{array}{cc}{\stackrel{`}{}}_i=\frac{1}{I_z}\left[\left(I_x-I_y\right){\stackrel{`}{}}_i{\stackrel{`}{}}_i-a_3{\stackrel{`}{}}_i^2+{\mathrm{lu}}_{4i}\right]+{}_i& \left(3\right)\end{array}$$\begin{array}{cc}{\stackrel{`}{x}}_i=-\frac{a_4{\stackrel{`}{x}}_i}{m}+\left(\cos {}_i\sin {}_i\cos {}_i+\sin {}_i\sin {}_i\right)\frac{u_{1i}}{m}+{}_{\mathrm{xi}}& \left(4\right)\end{array}$$\begin{array}{cc}{\stackrel{`}{y}}_i=-\frac{a_5{\stackrel{`}{y}}_i}{m}+\left(\cos {}_i\sin {}_i\sin {}_i-\sin {}_i\cos {}_i\right)\frac{u_{1i}}{m}+{}_{\mathrm{yi}}& \left(5\right)\end{array}$$\begin{array}{cc}{\stackrel{`}{z}}_i=-\frac{a_6{\stackrel{`}{z}}_i}{m}+\left(\cos {}_i\cos {}_i\right)\frac{u_{1i}}{m}-\frac{g}{m}+{}_{\mathrm{zi}}& \left(6\right)\end{array}$

wherein x, y, and z denote the position of the quadrotors in the earth frame, , , and represent the roll, yaw, and pitch angles, respectively, .sub.i, .sub.i, .sub.i, .sub.xi, .sub.yi, and .sub. stand for the unknown time-varying external disturbances, l is the length from the center of each actuator to the center of mass, I.sub.p is the propeller inertia, I.sub.x, I.sub.y, I.sub.z are the moment of inertia along the x, y, and z axes, respectively, g represents gravitational acceleration, a.sub.1, a.sub.2, a.sub.3, a.sub.4, a.sub.5, and a.sub.6 stand for the drag coefficients, u.sub.1i, u.sub.2i, u.sub.3i, and u.sub.4i denote the control inputs.

[0039] The three-degree-of-freedom translational equations of the N quadrotors are given by equations (7) to (9):

[00002]$\begin{array}{cc}{\stackrel{`}{x}}_i=-\frac{a_4{\stackrel{`}{x}}_i}{m}+u_{\mathrm{xi}}+{}_{\mathrm{xi}}& \left(7\right)\end{array}$$\begin{array}{cc}{\stackrel{`}{y}}_i=-\frac{a_5{\stackrel{`}{y}}_i}{m}+u_{\mathrm{yi}}+{}_{\mathrm{yi}}& \left(8\right)\end{array}$$\begin{array}{cc}{\stackrel{`}{z}}_i=-\frac{a_6{\stackrel{`}{z}}_i}{m}+u_{\mathrm{zi}}+{}_{\mathrm{zi}}\mathrm{where}{u}_{\mathrm{xi}}=\left(\cos {}_i\sin {}_i\cos {}_i+\sin {}_i\sin {}_i\right)\frac{u_{1i}}{m}{u}_{\mathrm{yi}}=\left(\cos {}_i\sin {}_i\sin {}_i-\sin {}_i\cos {}_i\right)\frac{u_{1i}}{m}{u}_{\mathrm{zi}}=\left(\cos {}_i\cos {}_i\right)\frac{u_{1i}}{m}-\frac{g}{m}& \left(9\right)\end{array}$

[0040] Assuming that the quadrotors are flying at the same altitude, the translational equations of the quadrotors in two-dimensional space can be written as:

[00003]$$\begin{gathered} \begin{array}{cc}{\stackrel{`}{q}}_i={\stackrel{}{}}_i\left(q_i,{\stackrel{`}{q}}_i\right)+u_i+{}_i \\ \mathrm{where}{q}_i={\left[\begin{array}{cc}{x}_i& y_i\end{array}\right]}^T{}^2,u_i={\left[\begin{array}{cc}{u}_{\mathrm{xi}}& u_{\mathrm{yi}}\end{array}\right]}^T{}^2,{\stackrel{}{}}_i\left(q_i,{\stackrel{`}{q}}_i\right)={\left[-\frac{a_4{\stackrel{`}{x}}_i}{m}-\frac{a_5{\stackrel{`}{y}}_i}{m}\right]}^T{}^2,{}_i={\left[\begin{array}{cc}{}_{\mathrm{xi}}& {}_{\mathrm{yi}}\end{array}\right]}^T{}^2.& \left(10\right)\end{array} \end{gathered}$$

[0041] The dynamic states of the quadrotors in equation (10) are constrained in the following compact sets:

[00004]$\begin{array}{cc}.Math.q_i.Math.=\left[\begin{array}{c}.Math.x_i.Math.\\ .Math.y_i.Math.\end{array}\right]<\left[\begin{array}{c}{k}_{\mathrm{ci}}\\ k_{\mathrm{ci}}\end{array}\right]& \left(11\right)\end{array}$

where k.sub.ci>0 is a constant. The compact set (11) specifies the region of safety operation of the quadrotors.

[0042] Assumption 1: Let q.sub.i* be the target position of each quadrotor. It is assuming that

[00005]$q_i^{*}$

is bounded as:

[00006]$\begin{array}{cc}.Math.q_i^{*}.Math.=\left[\begin{array}{c}.Math.x_i.Math.\\ .Math.y_i.Math.\end{array}\right]<\left[\begin{array}{c}{.Math.}_i\\ {.Math.}_i\end{array}\right]<\left[\begin{array}{c}{k}_{\mathrm{ci}}\\ k_{\mathrm{ci}}\end{array}\right]& \left(12\right)\end{array}$

and its 1st and 2nd derivatives are bounded as follows:

[00007]$\begin{array}{cc}.Math.{\stackrel{`}{q}}_i.Math.=\left[\begin{array}{c}.Math.{\stackrel{`}{x}}_i.Math.\\ .Math.{\stackrel{`}{y}}_i.Math.\end{array}\right]<\left[\begin{array}{c}{.Math.}_i^{}\\ {.Math.}_i^{}\end{array}\right]<\left[\begin{array}{c}{k}_{\mathrm{ci}}\\ k_{\mathrm{ci}}\end{array}\right]& \left(13\right)\end{array}$$$\begin{gathered} \begin{array}{cc}.Math.{\stackrel{`}{q}}_i.Math.=\left[\begin{array}{c}.Math.{\stackrel{`}{x}}_i.Math.\\ .Math.{\stackrel{`}{y}}_i.Math.\end{array}\right]<\left[\begin{array}{c}{.Math.}_i^{}\\ {.Math.}_i^{}\end{array}\right]<\left[\begin{array}{c}{k}_{\mathrm{ci}}\\ k_{\mathrm{ci}}\end{array}\right] \\ \mathrm{where} \\ {.Math.}_i>0,{.Math.}_i^{}>0\mathrm{and}{.Math.}_i^{}>0& \left(14\right)\end{array} \end{gathered}$$

are constants,

[0043] The two-dimension translational equations of the quadrotors in the face of cyber-attacks are given as:

[00008]$\begin{array}{cc}\{\begin{array}{c}{\stackrel{}{q}}_i={\stackrel{}{}}_i\left(q_i,{\stackrel{}{q}}_i\right)+{\stackrel{}{u}}_i+{}_i\\ {\stackrel{}{q}}_i=q_i+{}_i\left(t,q_i\right)\\ {\stackrel{}{u}}_i={\underset{\_}{h}}_i{u}_i+{\stackrel{}{h}}_i\left(t,q_i,{\stackrel{}{q}}_i\right)\end{array}& \left(15\right)\end{array}$

wherein {grave over (q)}.sub.i.sup.2 is the vector of the compromised states, .sub.i.sup.2 is the vector of the compromised control inputs, .sub.i(t, q.sub.i).sup.2 is the vector of the deception attack signals, .sub.i=[].sup.T.sup.2 is the vector of time-varying injected attack signals, and h.sub.i=dia g{h.sub.xih.sub.yi}.sup.22

[0044] The deception attack signals can be described by state-dependent function as .sub.i(t, q.sub.i)=.sub.i(t)q.sub.i since they are mimicking the state variables, where .sub.i(t).sup.22 is a time-varying weight. Thus, the quadrotor states corrupted by the deception attacks are given as:

[00009]$\begin{array}{cc}{\stackrel{}{q}}_i=q_i+{}_i\left(t\right)q_i={}_i\left(t\right)q_i& \left(16\right)\end{array}$

where .sub.i(t)=(1+.sub.i(t))

[0045] As a result of the attack, the actual state variables q.sub.i are no longer available. Therefore, the contaminated state variables q.sub.i will be utilized for control design.

[0046] Deriving from (16):

[00010]$\begin{array}{cc}{\stackrel{.}{\stackrel{}{q}}}_l={\stackrel{.}{}}_l{q}_i+{}_i{\stackrel{.}{q}}_l& \left(17\right)\end{array}$$\begin{array}{cc}{\stackrel{.Math.}{\stackrel{}{q}}}_l={\stackrel{.Math.}{}}_l{q}_i+2{\stackrel{.}{}}_l{\stackrel{.}{q}}_l+{}_i{\stackrel{.Math.}{q}}_l& \left(18\right)\end{array}$

[0047] Noting that

[00011]$q_i=\frac{{\stackrel{}{q}}_i}{{}_i}\mathrm{and}{\stackrel{}{q}}_i=\frac{1}{{}_i}\left[{\stackrel{.}{\stackrel{}{q}}}_l-{\stackrel{}{}}_i\frac{{\stackrel{}{q}}_i}{{}_i}\right]$

and 15, (18) becomes:

[00012]$\begin{array}{cc}{\stackrel{}{q}}_i={\stackrel{}{}}_i\frac{{\stackrel{}{q}}_i}{{}_i}+2\frac{{\stackrel{}{}}_i}{{}_i}\left[{\stackrel{}{\stackrel{}{q}}}_i-{\stackrel{}{}}_i\frac{{\stackrel{}{q}}_i}{{}_i}\right]+{}_i{\stackrel{}{}}_i+{}_i{}_i+{}_i{\underset{\_}{h}}_i{u}_i+{}_i{\stackrel{}{h}}_i& \left(19\right)\end{array}$

[0048] The quadrotor systems under the cyber-attacks could be expressed as:

[00013]$\begin{array}{cc}{\stackrel{}{q}}_i={}_i+g_i{u}_i\mathrm{where}{}_i={\stackrel{}{}}_i\frac{{\stackrel{}{q}}_i}{{}_i}+2\frac{{\stackrel{}{}}_i}{{}_i}\left[{\stackrel{}{\stackrel{.fwdarw.}{q}}}_i-{\stackrel{}{}}_i\frac{{\stackrel{}{q}}_i}{{}_i}\right]+{}_i{\stackrel{}{}}_i+{}_i{}_i+{}_i{\stackrel{}{h}}_i,\mathrm{and}{g}_i={}_i{\underset{\_}{h}}_i.& \left(20\right)\end{array}$

[0049] The N quadrotors in (20) can be partitioned into two groups: the N.sub.l leaders and N.sub.f=(NN.sub.l) followers.

[0050] Let {grave over (q)}.sub.l and {grave over (q)}.sub.f be the vectors of the N.sub.l leaders and N.sub.f followers, respectively. Then:

[00014]$\begin{array}{cc}{\stackrel{}{q}}_l={\left[\begin{array}{cc}{\stackrel{}{q}}_1^T& {\stackrel{}{q}}_2^T.Math.{\stackrel{}{q}}_{N_l}^T\end{array}\right]}^T& \left(21\right)\end{array}$$\begin{array}{cc}{\stackrel{}{q}}_f={\left[\begin{array}{cc}{\stackrel{}{q}}_{N_l+1}^T& {\stackrel{}{q}}_{N_l+2}^T.Math.{\stackrel{}{q}}_{N_l+N_f}^T\end{array}\right]}^T& \left(22\right)\end{array}$

[0051] The configuration of the N quadrotors which consists of their positions in two-dimensional space under cyberattacks is thus:

[00015]$\begin{array}{cc}\stackrel{}{q}={\left[\begin{array}{cc}{\stackrel{}{q}}_l^T& {\stackrel{}{q}}_f^T\end{array}\right]}^T={\left[\begin{array}{cccc}{\stackrel{}{q}}_1^T& {\stackrel{}{q}}_2^T& .Math.& {\stackrel{}{q}}_N^T\end{array}\right]}^T{}^{2N}& \left(23\right)\end{array}$

[0052] It is advantageous to provide a control law that can maintain healthy quadrotors states within the safety constraints (11) even in the presence of cyberattacks.

[0053] Assumption 2: Considering that the N.sub.l leader quadrotors are already controlled to acquire the desired formation maneuvers. In this sense, the control procedure for the leader quadrotors will not be considered.

[0054] Definition 1. Given a continuous function H():.fwdarw. that satisfies:

[00016]$\begin{array}{cc}\{\begin{array}{c}\underset{t.fwdarw.}{}\sup \frac{1}{t}{}_0^{t}N\left(\right)d=+\\ \underset{t.fwdarw.}{}\inf \frac{1}{t}{}_0^{t}N\left(\right)d=-\end{array}& \left(24\right)\end{array}$

then H(): .fwdarw. is a Nussbaum function. Many functions satisfy the condition in (24), for instance, .sup.2 cos (), e.sup.2 cos (1/2) and .sup.2 sin().

[0055] Lemma 1. Defining the smooth functions L(t) and s(t) over the range [0, t.sub.f) with L(t)0, H(s) as a smooth Nussbaum function, the following inequality holds:

[00017]$\begin{array}{cc}L\left(t\right)r_1+e^{-r_{2}t}{}_0^t\left[g(.)H\left(\right)+1\right]\stackrel{}{}{e}^{r_{2}t}\mathrm{dt}& \left(25\right)\end{array}$

where r.sub.1>0, r.sub.2>0 are constants, g()isatimevarying function, then, L(t), (t), and

[00018]${}_0^{t}g\left(\right)N.\left(\right)\stackrel{}{}\mathrm{dt}$

are bounded over [0, t.sub.f].

[0056] According to an embodiment, radial basis function neural networks are commonly employed to approximate any smooth and continuous unknown term as:

[00019]$\begin{array}{cc}=W^T\left(\right)+\left(\right)& \left(26\right)\end{array}$

where is the input vector, W=[w.sub.1 w.sub.2 . . . w.sub.k].sup.T.sup.k is the weight vector, k is the number of nodes in the hidden layer, () is the approximation error and (x) with >0, ()=[.sub.1().sub.2() . . . .sub.k()].sup.T represent the basis function vector with entries .sub.i() given by

[00020]$\begin{array}{cc}{}_i\left(\right)=\exp \left(-\frac{{.Math.\left(-{\stackrel{}{}}_i\right).Math.}^2}{{\underset{\_}{}}_i^2}\right),i=1,2,.Math.,k& \left(27\right)\end{array}$

where and .sub.i denote the center and width of the Gaussian function, respectively.

[0057] According to an embodiment, a barrier function is widely employed to guarantee the safety of dynamic systems. The barrier function tends to infinity when it reaches the barriers of the specified safety constraints, but never violates the constraints. This property is utilized to design control laws that keep dynamic systems within the safety barriers.

[0058] Lemma 2. A Lyapunov candidate function L(z) complying L(z).fwdarw. as |z|.fwdarw.c, is deployed to ensure that the state-variables boundaries are not transgressed. The Lyapunov candidate function is as follows:

[00021]$\begin{array}{cc}L\left(z\right)=\frac{1}{2}\log \frac{c^2}{c^2-z^2}& \left(28\right)\end{array}$

where c is the constraint on z. Thus, L is positive definite and continuous within the set |z|c. The control policy is developed to meet {grave over (L)}0.

[0059] Lemma 3. For any constant c>0 and z meeting |z|c, one gets

[00022]$\begin{array}{cc}\log \frac{c^2}{c^2-z^2}<\frac{z^2}{c^2-z^2}& \left(29\right)\end{array}$

[0060] See Y.-J. Liu, J. Li, S. Tong, C. L. P. Chen, Neural network control based adaptive learning design for nonlinear systems with full state constraints, IEEE Transactions on Neural Networks and Learning Systems 27 (2016) 1562-1571. doi 10.1109, incorporated herein by reference in its entirety.

[0061] The exchange of information among the quadrotors is modeled by an undirected graph =(, , ) consisting of N vertices. Define ={.sub.1, . . . , .sub.N} and .Math. as the sets of vertices and edges, respectively. Then, the set of neighbors of the vertex i is defined by =j:(i, j).

[0062] The formation of the quadrotors (, {grave over (q)}) is defined as the graph of the quadrotors =(, , ) with their corresponding configuration {grave over (q)}.

[0063] The scalar weight allotted to each edge of the formation (, q) is termed as the stress of the edge and it can be positive or negative. The set of the scalar weights is defined by {s.sub.ij}.sub.(i,j) with s.sub.ij=s.sub.ji. The stress matrix S.sup.NN is described as:

[00023]$\begin{array}{cc}{\left[S\right]}_{\mathrm{ij}}=\{\begin{array}{cc}0,& ij,\left(i,j\right).Math.{}_i\\ -s_{\mathrm{ij}},& ij,\left(i,j\right){}_i\\ {.Math.}_{k{}_i}{s}_{\mathrm{ik}},& ij\end{array}.& \left(30\right)\end{array}$

[0064] When the stress satisfies s.sub.ij({grave over (q)}.sub.j{grave over (q)}.sub.i)=0 or (S.Math.I.sub.2){grave over (q)}=0 in compact form, it is referred to as equilibrium stress.

[0065] Thus, (30) can be rewritten as:

[00024]$\begin{array}{cc}\stackrel{}{S}=S.Math.I_2=\left[\begin{array}{cc}{S}_{\mathrm{ll}}& S_{\mathrm{lf}}\\ S_{\mathrm{fl}}& S_{\mathrm{ff}}\end{array}\right].Math.I_2=\left[\begin{array}{cc}{\stackrel{}{S}}_{\mathrm{ll}}& {\stackrel{}{S}}_{\mathrm{lf}}\\ {\stackrel{}{S}}_{\mathrm{fl}}& {\stackrel{}{S}}_{\mathrm{ff}}\end{array}\right].& \left(31\right)\end{array}$

where S.sub.ll.sup.2N.sup.l.sup.2N.sup.l, S.sub.ff.sup.2N.sup.f.sup.2N.sup.f and S.sub.fl.sup.2N.sup.l.sup.2N.sup.f

[0066] Let

[00025]$={\left[\begin{array}{cccc}{}_1^T& {}_2^T& .Math.& {}_N^T\end{array}\right]}^T={\left[\begin{array}{cc}{}_l^T& {}_f^T\end{array}\right]}^T={}^{2N}$

be a constant configuration. In view of the graph , a nominal formation (, n) is formed. Afterward, the time-varying reference formation is:

[00026]$\begin{array}{cc}q*\left(t\right)=\left[I_N.Math.A\left(t\right)\right]+1_N.Math.b\left(t\right)& \left(32\right)\end{array}$

where A(t).sup.22 and b(t).sup.2 are time-varying. The affine transformation of can be achieved by specifying the entries of A(t) and b(t).

[0067] The affine image of is defined by:

[00027]$\begin{array}{cc}\left(\right)=\left\{\stackrel{`}{q}{}^{2N}:\stackrel{`}{q}=\left(I_N.Math.A\right)+1_N.Math.b\right\}.& \left(33\right)\end{array}$

[0068] If q(), the N agents acquire the affine formation maneuvers.

[0069] Definition 2. Affine localizability: If for any

[00028]$\stackrel{`}{q}={\left[\begin{array}{cc}{\stackrel{`}{q}}_f^T& {\stackrel{`}{q}}_l^T\end{array}\right]}^T\left(\right),{\stackrel{`}{q}}_l$

determine {grave over (q)}.sub.f, then (, ) is affinely localizable by the leaders.

[0070] Assumption 3. The stress matrix S is positive semidefinite, and ran k(S)=Ndim1 holds, where dim is the dimension of the systems in the Euclidean space. In this work, two-dimensional systems are considered, i.e., dim=2. This assumption implies that the matrix S.sub.ff is positive definite and invertible.

[0071] By virtue of Definition 2 and Assumption 3, for any

[00029]$\stackrel{`}{q}={\left[\begin{array}{cc}{\stackrel{`}{q}}_f^T& {\stackrel{`}{q}}_l^T\end{array}\right]}^T\left(\right),$

{grave over (q)}.sub.l can uniquely determine

[00030]${\stackrel{`}{q}}_f^{*}$

as:

[00031]$\begin{array}{cc}{\stackrel{`}{q}}_f^{*}=-\left(S_{\mathrm{ff}}^{-1}{S}_{\mathrm{fl}}.Math.I_2\right){\stackrel{`}{q}}_l.& \left(34\right)\end{array}$

where

[00032]$q_f^{*}$

is the target position of the followers.

[0072] The aforementioned mathematics support an intelligent learning control method to ensure the safe formational maneuvering of a system of quadrotor UAVs and shield them against harmful cyber-attacks. Considering Assumption 2, the control of the leaders will not be considered. The distributed sliding mode surface of the followers' affine formation maneuver is designed as:

[00033]$\begin{array}{cc}{}_i={.Math.}_{j=1}^N{s}_{\mathrm{ij}}\left({\stackrel{\stackrel{`}{`}}{q}}_i-{\stackrel{\stackrel{`}{`}}{q}}_j\right)-{.Math.}_{j=1}^N{s}_{\mathrm{ij}}\left({\stackrel{\stackrel{`}{`}}{q}}_i\left(0\right)-{\stackrel{\stackrel{`}{`}}{q}}_j\left(0\right)\right),i{}_f+{}_1{.Math.}_{j=1}^N{s}_{\mathrm{ij}}\left({\stackrel{`}{q}}_i-{\stackrel{`}{q}}_j\right)+{}_2{}_0^t{s}_{\mathrm{ij}}\left({}_i-{}_j\right)\mathrm{dt}& \left(35\right)\end{array}$

where .sub.1>0, .sub.2>0 are constant diagonal matrices, the function .sub.i(t) is designed as:

[00034]$\begin{array}{cc}{}_i\left(t\right)=\left[{.Math.}_{j=1}^N{s}_{\mathrm{ij}}{\tanh }^{{\mathrm{.Math.}}_1}\left({\stackrel{`}{q}}_i-{\stackrel{`}{q}}_j\right)+{.Math.}_{j=1}^N{s}_{\mathrm{ij}}\left({\stackrel{\stackrel{`}{.fwdarw.}}{q}}_i-{\stackrel{\stackrel{`}{.fwdarw.}}{q}}_j\right)\right]& \left(36\right)\end{array}$

where 1<.Math..sub.1<2, and .Math..sub.2>.Math..sub.1

[0073] The compact form of (35) can be expressed as:

[00035]$\begin{array}{cc}{}_f={\stackrel{}{S}}_{\mathrm{ff}}\left({\stackrel{`}{}}_f-q_f^{*}\right)-{\stackrel{`}{z}}_f\left(0\right)+{}_1{z}_f+{}_2{}_0^t{}_f\mathrm{dt}\mathrm{where}{z}_f={\stackrel{}{S}}_{\mathrm{ff}}\left({\stackrel{`}{q}}_f-q_f^{*}\right),q_f^{*}=-\left(S_{\mathrm{ff}}^{-1}{S}_{\mathrm{fl}}.Math.I_2\right){\stackrel{`}{q}}_l\mathrm{and}{\stackrel{`}{q}}_f^{*}=-\left(S_{\mathrm{ff}}^{-1}{S}_{\mathrm{fl}}.Math.I_2\right){\stackrel{\stackrel{`}{`}}{q}}_l& \left(37\right)\end{array}$

represent the target formation of the followers and its first-order derivative, respectively, and

[00036]${}_f={\left[\begin{array}{cccc}{}_{N_l+1}^T& {}_{N_l+1}^T& .Math.& {}_{N_l+N_f}^T\end{array}\right]}^T.$

[0074] The main objective here is to achieve:

[00037]$\begin{array}{cc}\underset{t.fwdarw.}{\lim }\left({\stackrel{`}{q}}_f\left(t\right)-q_f^{*}\left(t\right)\right)=0& \left(38\right)\end{array}$

[0075] Differentiating (37) with respect to time gives:

[00038]$$\begin{gathered} \begin{array}{cc}\begin{array}{c}{\stackrel{`}{}}_f={\stackrel{}{S}}_{\mathrm{ff}}\left({\stackrel{\stackrel{`}{`}}{q}}_f-{\stackrel{`}{q}}_f^{*}\right)+{}_1{\stackrel{`}{z}}_f+{}_2{}_f\\ ={\stackrel{}{S}}_{\mathrm{ff}}\left({}_f+{}_f{u}_f-{\stackrel{`}{q}}_f^{*}+{}_1{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{z}}_f+{}_2{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{}_f\right)\end{array} \\ \mathrm{Where}{}_f=[\begin{array}{cccc}{}_{N_l+1}^T& {}_{N_l+1}^T& .Math.& {{}_{N_l+N_f}^T]}^T\end{array},z_f=[\begin{array}{cccc}{z}_{N_l+1}^T& z_{N_l+1}^T& .Math.& {z_{N_l+N_f}^T]}^T\end{array},{}_f={\left[\begin{array}{cccc}{}_{N_l+1}^T& {}_{N_l+2}^T& .Math.& {}_{N_l+N_f}^T\end{array}\right]}^T,{}_f=\mathrm{diag}\left\{\begin{array}{cccc}{}_{N_l+1}& {}_{N_l+1}& .Math.& {}_{N_l+N_f}\end{array}\right\},\mathrm{and}u=[\begin{array}{cccc}{u}_{N_l+1}^T& u_{N_l+2}^T& .Math.& {u_{N_l+N_f}^T]}^T\end{array}.& \left(39\right)\end{array} \end{gathered}$$

From (35), it is known that .sub.i=[.sub.xi.sub.yi].sup.T. Then, a barrier Lyapunov function for the tracking error system (39) is defined as follows:

[00039]$\begin{array}{cc}{L}_1=\mathrm{Trace}({\stackrel{}{S}}_{\mathrm{ff}}^{-1}{.Math.}_{i=N_l+1}^{N_l+N_f}\frac{1}{2}\left(\frac{c_{xi}^2}{c_{xi}^2-{}_{xi}^2}+\frac{c_{yi}^2}{c_{yi}^2-{}_{yi}^2}\right))& \left(40\right)\end{array}$

where c.sub.xi>0 and c.sub.yi>0 are chosen to make sure that the states' constraints are not violated. From Remark 11 and Assumption 1, setting c.sub.xi=k.sub.ci.sub.i and c.sub.yi=k.sub.ci.sub.i.

[0076] Differentiating L.sub.1 with respect to time gives:

[00040]$\begin{array}{cc}\begin{array}{c}{\stackrel{`}{L}}_1=\mathrm{Trace}\left({\stackrel{}{S}}_{\mathrm{ff}}^{-1}{.Math.}_{i=N_l+1}^{N_l+N_f}\left(\frac{{}_{xi}{}_{xi}}{c_{xi}^2-{}_{xi}^2}+\frac{{}_{yi}{}_{yi}}{c_{yi}^2-z_{yi}^2}\right)\right)\\ =\mathrm{Trace}\left({\stackrel{}{S}}_{\mathrm{ff}}^{-1}{.Math.}_{i=N_l+1}^{N_l+N_f}\left(\left[\frac{{}_{xi}}{c_{xi}^2-{}_{xi}^2}\frac{{}_{yi}}{c_{yi}^2-{}_{yi}^2}\right]\left[\begin{array}{c}{\stackrel{`}{}}_{xi}\\ {\stackrel{`}{}}_{yi}\end{array}\right]\right)\right)\\ =\mathrm{Trace}\left({\stackrel{}{S}}_{\mathrm{ff}}^{-1}{.Math.}_{i=N_l+1}^{N_l+N_f}{O}_i^T{\stackrel{`}{}}_i\right)\\ \mathrm{where}{O}_i={\left[\frac{{}_{xi}}{c_{xi}^2{}_{xi}^2}\frac{{}_{yi}}{c_{yi}^2-{}_{yi}^2}\right]}^T\end{array}& \left(41\right)\end{array}$

[0077] Equation (41) can be rewritten as:

[00041]$\begin{array}{cc}\begin{array}{c}{\stackrel{`}{L}}_1=\mathrm{Trace}\left(O_f^T{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{}}_f\right)\\ =O_f^T\left({}_f+{}_f{u}_f-{\stackrel{`}{q}}_f^{*}+{}_I{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{z}}_f+{}_2{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{}_f\right)\\ \mathrm{where}{O}_f={\left[O_{N_l+1}^T{O}_{N_l+2}^T.Math.O_{N_l+N_f}^T\right]}^T.\end{array}& \left(42\right)\end{array}$

[0078] The control protocol for the quadrotors with loss of direction resulting from an actuator attack can be designed as:

[00042]$\begin{array}{cc}\begin{array}{c}{\stackrel{}{u}}_f={}_f^{-1}\left(\begin{array}{c}{\stackrel{`}{}}_f+{}_3{}_f+\frac{1}{2}{O}_f+{}_1{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{z}}_f\\ +{}_2{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{}_f-{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{q}}_f^{*}\end{array}\right)\\ u_f=N\left({}_f\right){\stackrel{}{u}}_f\\ {\stackrel{`}{}}_f=O_f^T\left(\begin{array}{c}{\stackrel{`}{}}_f+{}_3{}_f+\frac{1}{2}{O}_f+{}_1{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{z}}_f\\ +{}_2{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{}_f-{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{q}}_f^{*}\end{array}\right)\end{array}& \left(43\right)\end{array}$

where .sub.3 is a constant positive definite diagonal matrix.

[0079] Remark 6. the control input suffered from a loss of direction as a result of the attack gain .sub.f. The Nussbaum function N(.sub.f) is employed to tackle this problem. The control input u.sub.f is the corrected signal.

[0080] Remark 7. The control protocol (43) cannot be applied to the quadrotors directly because of the loss of control direction and the unknown value of vector .

[0081] The critic network is designed to assess the performance of the present control action and produce punishment/reward signals for adaptive learning. The critic network is designed using the neural network. A long-term cost function is defined as:

[00043]$\begin{array}{cc}J\left(t\right)={}_c^{}{e}^{\frac{-t}{a}}\left(\right)d& \left(44\right)\end{array}$$\mathrm{where}J={\left[J_{N_l+1}^T{J}_{N_l+1}^T.Math.J_{N_l+N_f}^T\right]}^T,>0$

is a constant used to discount the future cost, (t) is the current cost function expressed as:

[00044]$\begin{array}{cc}\left(t\right)={}_f^{T}Q{}_f+u_f^{T}R{u}_f& \left(45\right)\end{array}$

where Q and R are constant positive definite matrices.

[0082] The long-term cost function (44) contains future system information coupled with the compromised variables, so the solution is difficult to calculate. Thus, the critic neural network will be deployed to approximate J as

[00045]$\begin{array}{cc}J=W_c^T{}_c\left({}_c\right)+{}_c& \left(46\right)\end{array}$

[0083] The critic neural network estimate of J is given by:

[00046]$\begin{array}{cc}\stackrel{`}{J}={\stackrel{`}{W}}_c^T{}_c\left({}_c\right)& \left(47\right)\end{array}$

where {grave over (W)}.sub.c is the weight estimate of W.sub.c.

[0084] The continuous-time temporal difference error is obtained as:

[00047]$\begin{array}{cc}{c}_{}\left(t\right)=\left(t\right)-\frac{1}{}\stackrel{`}{J}\left(t\right)+\stackrel{`}{\stackrel{`}{J}}\left(t\right)& \left(48\right)\end{array}$

[0085] For large , i.e .fwdarw., (48) simplifies to:

[00048]$\begin{array}{cc}{c}_{}\left(t\right)=\left(t\right)+\stackrel{`}{\stackrel{`}{J}}\left(t\right)=\left(t\right)+\stackrel{`}{J}\left(t\right){\stackrel{`}{}}_c& \left(49\right)\end{array}$

where stands for the gradient operator.

[0086] Define the temporal difference error objective function as:

[00049]$\begin{array}{cc}{E}_c=\frac{1}{2}{c}_T^{}{c}_{}& \left(50\right)\end{array}$

[0087] The weight update rule of the critic network is derived as follows:

[00050]$\begin{array}{cc}\begin{array}{c}{\stackrel{`}{\stackrel{`}{W}}}_c=-{}_c\frac{E_c}{W_c}=-{}_c{c}_{}\left(t\right)\frac{c_{}}{W_c}\\ =-{}_c{c}_{}\left(t\right)\frac{\left[\left(t\right)-\left(1/\right)\stackrel{`}{J}\left(t\right)+\stackrel{`}{J}\left(t\right)\right]}{W_c}\\ =-{}_c{c}_{}\left(t\right)\left[-\frac{1}{}\frac{\stackrel{`}{J}}{W_c}+\frac{}{W_c}\left(\frac{\stackrel{`}{J}}{{}_c}\right)\right]\\ =-{}_c\left(\left(t\right)+W_C^T\right)\end{array}& \left(51\right)\end{array}$

where .sub.c>0 is the learning rate,

[00051]$=-\frac{{}_c}{}+{}_c{\stackrel{`}{}}_c.$

[0088] Design a Lyapunov function L.sub.c for the critic network as:

[00052]$\begin{array}{cc}{L}_c=\frac{1}{2}{\stackrel{`}{W}}_C^T{\stackrel{`}{W}}_c& \left(52\right)\end{array}$

[0089] Using (51) and (52), one gets:

[00053]$\begin{array}{cc}\begin{array}{c}{\stackrel{`}{L}}_c={\stackrel{`}{W}}_c^T{\stackrel{\stackrel{`}{`}}{W}}_c={\stackrel{`}{W}}_c^T{\stackrel{\stackrel{`}{`}}{W}}_c\\ =-{}_c{\stackrel{`}{W}}_c^T\left(\left(t\right)+W_c^T\right)\end{array}& \left(53\right)\end{array}$

[0090] At the end of the reinforcement learning, the weight update law (51) will minimize the objective function (50) such that .sub.c.fwdarw.0 and

[00054]$\frac{J}{}-j.$

It follows that:

[00055]$\begin{array}{cc}\begin{array}{c}\left(t\right)=W_c^T\frac{{}_c}{}+\frac{{}_c}{}-I{\stackrel{`}{}}_c\\ =W_c^T\frac{{}_c}{}+\frac{{}_c}{}-\left(W_c^T{}_c\left({}_c\right)+{}_c\right){\stackrel{`}{}}_c\\ =-W_c^T+{}_c\end{array}& \left(54\right)\end{array}$

where .sub.c=.sub.c+.sub.c{grave over ()}.sub.c, .sub.c.sub.c,max, with >0 being the upper-bound.

[0091] Putting (54) into (53), one achieves:

[00056]$\begin{array}{cc}\begin{array}{c}{\stackrel{`}{L}}_c=-{}_c{\stackrel{`}{W}}_c^T\left({\stackrel{`}{W}}_c{}_c+{}_c\right)\\ =-{}_c{}^T{\stackrel{`}{W}}_c^T{\stackrel{`}{W}}_c-{}_c{\stackrel{`}{W}}_c^T{}_c\\ -\frac{{}_c{}^T}{2}{\stackrel{`}{W}}_c^T{\stackrel{`}{W}}_c+\frac{{}_c}{2}{}_c^T{}_c\\ -\frac{{}_c{}^T}{2}{\stackrel{`}{W}}_c^T{\stackrel{`}{W}}_c+\frac{{}_c}{2}{.Math.{}_{c,m\mathrm{ax}}.Math.}^2\end{array}& \left(55\right)\end{array}$

[0092] Therefore, (55) represents a Lyapunov function for the critic network which has been modified by the weight update law (51) to minimize the objective function (50). This function (55) may be utilized by the actor-critic learning scheme to protect the UAVs from external disturbances, sensor deception attacks, and injection attacks on the actuators.

[0093] FIG. 1A depicts a block diagram of a system 100 for controlling a plurality of quadcopters, according to certain embodiments. The system 100 includes a user input device 102, a leader quadcopter 104, and a follower quadcopter 106. The user input device 102 is configured to receive a flight instruction from a user and deliver the flight instruction to a leader quadcopter. The leader quadcopter 104 is configured to receive a flight instruction from a user input device, calculate a leader formation maneuver, calculate a follower formation maneuver with a leader-follower formation controller configured to use affine transformations and stress matrices to calculate the follower formation maneuver, and communicate the follower formation maneuver to a follower quadcopter and execute the leader formation maneuver. The leader quadcopter 104 is a multirotor drone with four arms or booms, each with a rotor. Multirotor drones are unmanned aerial vehicles (UAV) with multiple rotors that are used to generate lift to enable the aircraft to fly. The follower quadcopter 106 is configured to receive the follower formation maneuver and execute the follower formation maneuver. In an implementation, the leader-follower formation controller is further configured to use an actor-critic machine learning algorithm to calculate the follower formation maneuver.

[0094] In the actor-critic control scheme, information about the lumped function is unavailable due to the existence of cyber-attacks, unknown external disturbances, and uncertain dynamics. The actor neural network may be deployed to approximate as follows:

[00057]$\begin{array}{cc}{}_f=W_a^T{}_c\left({}_a\right)+{}_a& \left(56\right)\end{array}$

[0095] The actor network's estimation of is thus:

[00058]$\begin{array}{cc}{\stackrel{`}{}}_f={\stackrel{`}{W}}_a{}_a\left({}_a\right)& \left(57\right)\end{array}$

[0096] The current weight estimation error of the actor network is:

[00059]$\begin{array}{cc}{\stackrel{`}{}}_f={\stackrel{`}{W}}_a^T{S}_a\left({}_a\right)& \left(58\right)\end{array}$

where {grave over (W)}.sub.a=W.sub.A{grave over (W)}.sub.a.

[0097] Let J.sub.d=0 be the desired cos-to-go in the actor network. The error between {grave over (J)} and J.sub.d is:

[00060]$\begin{array}{cc}{\stackrel{`}{J}}_d=\stackrel{`}{J}-J_d=\stackrel{`}{J}& \left(59\right)\end{array}$

[0098] The total of the errors in the actor system is expressed as:

[00061]$\begin{array}{cc}{}_a={\stackrel{`}{}}_f+K_a\stackrel{`}{J}& \left(60\right)\end{array}$$\begin{array}{cc}{\stackrel{`}{\stackrel{`}{W}}}_a=-{}_a\frac{E_a}{W_a}& \left(61\right)\end{array}$

where K.sub.a is a diagonal gain matrix. The weight update law of the actor network,
where .sub.a>0 is the learning rate, is:

[00062]$E_a=\frac{1}{2}{}_a^T{}_a\frac{E_a}{{\stackrel{`}{W}}_a}=\frac{E_a}{{}_a}\frac{{}_a}{{\stackrel{`}{}}_f}\frac{{\stackrel{`}{}}_f}{{\stackrel{`}{W}}_a}=\left({\stackrel{`}{}}_f+K_a\stackrel{`}{J}\right){}_a$

[0099] Because is unknown, the weight update law is rewritten as:

[00063]$\begin{array}{cc}{\stackrel{`}{\stackrel{`}{W}}}_a=-{}_a\left({\stackrel{`}{W}}_a^T{}_a\left({}_a\right)+K_a\stackrel{`}{J}\right){}_a& \left(62\right)\end{array}$

[0100] A candidate Lyapunov function for the actor-network is selected as follows:

[00064]$\begin{array}{cc}{L}_a=\frac{1}{2}{\stackrel{`}{W}}_a^T{\stackrel{`}{W}}_a& \left(63\right)\end{array}$

[0101] The time-derivative of L.sub.a yields:

[00065]$\begin{array}{cc}{\stackrel{`}{L}}_a=-{}_a{\stackrel{`}{W}}_a^T{}_a\left({\stackrel{`}{W}}_a^T{}_a\left({}_a\right)+K_a\stackrel{`}{J}\right)& \left(64\right)\end{array}$

[0102] Considering that {grave over (J)}=W.sub.c.sup.T.sub.c(.sub.c)+{grave over (W)}.sub.c.sup.T.sub.c(.sub.c), gets:

[00066]$\begin{array}{cc}{\stackrel{`}{J}}^T\stackrel{`}{J}2{\left(W_c^T{}_c\right)}^T{W}_c{}_c+2{\left({\stackrel{`}{W}}_c^T{}_c\right)}^T{\stackrel{`}{W}}_c^T{}_c& \left(65\right)\end{array}$

[0103] Inserting (65) into (64):

[00067]$\begin{array}{cc}\begin{array}{c}{\stackrel{`}{L}}_a=-{}_a{\stackrel{`}{W}}_a^T{}_a\left(W_a^T{}_a\left({}_a\right)-{\stackrel{`}{W}}_a^T{}_a\left({}_a\right)+K_a\stackrel{`}{J}\right)\\ =-{}_a{\stackrel{`}{W}}_a^T{}_a{W}_a^T{}_a\left({}_a\right)+{}_a{\stackrel{`}{.Math.W}}_a{.Math.}^2{.Math.{}_a.Math.}^2\\ -{}_a{\stackrel{`}{W}}_a^T{}_a{K}_c\stackrel{`}{J}\\ -{}_a{\stackrel{`}{W}}_a^T{}_a{W}_a^T{}_a\left({}_a\right)+{}_a{.Math.{\stackrel{`}{W}}_a.Math.}^2{.Math.{}_a.Math.}^2\\ -\frac{{}_a}{2}{.Math.{\stackrel{`}{W}}_a.Math.}^2{.Math.{}_a.Math.}^2+\frac{{}_a}{2}{K}_a^T{K}_a.Math.\stackrel{`}{J}.Math.{.Math.}^2\\ -\frac{{}_a}{2}{.Math.{\stackrel{`}{W}}_a.Math.}^2{.Math.{}_a.Math.}^2+\frac{{}_a}{2}{.Math.W_a.Math.}^2{.Math.{}_a.Math.}^2\\ -\frac{{}_a}{2}{.Math.{\stackrel{`}{W}}_a.Math.}^2{.Math.{}_a.Math.}^2\\ +{}_a{K}_a^T{K}_a\left({.Math.W_c.Math.}^2{.Math.{}_c.Math.}^2+{.Math.{\stackrel{`}{W}}_c.Math.}^2{.Math.{}_c.Math.}^2\right)\end{array}& \left(66\right)\end{array}$

[0104] Using the actor estimation in (57), (43) becomes:

[00068]$\begin{array}{cc}\begin{array}{c}{\stackrel{}{u}}_f={}_f^{-1}\left(\begin{array}{c}{\stackrel{`}{W}}_a^T{}_a\left({}_a\right)+{}_3{}_f+\frac{1}{2}{O}_f+{}_1{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{z}}_f+\\ {}_2{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{}_f-{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{q}}_f^{*}\end{array}\right)\\ u_f=N\left({}_f\right){\stackrel{}{u}}_f\\ {\stackrel{`}{}}_f=O_f^T\left(\begin{array}{c}{\stackrel{`}{W}}_a^T{}_a\left({}_a\right)+{}_3{}_f+\frac{1}{2}{O}_f+\\ {}_1{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{z}}_f+{}_2{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{}_f-{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{q}}_f^{*}\end{array}\right)\end{array}& \left(67\right)\end{array}$

[0105] Equation (42) can be evaluated as follows:

[00069]$\begin{array}{cc}\begin{array}{c}{\stackrel{`}{L}}_1=O_f^T{\stackrel{`}{}}_f={\stackrel{`}{}}_f-{\stackrel{`}{}}_f+O_f^T{}_f{u}_f\\ +O_f^T\left(\begin{array}{c}{W}_a^T\left({}_a{}_a\right)+{}_a-{\stackrel{`}{q}}_f^{*}+{}_1{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{z}}_f+\\ {}_2{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{}_f\end{array}\right)\\ ={\stackrel{`}{S}}_f+{}_{f}N\left({}_f\right){\stackrel{`}{S}}_f\\ +O_f^T\left(\begin{array}{c}{W}_a^T\left({}_a{}_a\right)+{}_a-{\stackrel{`}{q}}_f^{*}+{}_1{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{z}}_f+\\ {}_2{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{}_f\end{array}\right)\\ -O_f^T\left(\begin{array}{c}{\stackrel{`}{W}}_a^T{}_a\left({}_a\right)+{}_3{}_f+\frac{1}{2}{O}_f+\\ {}_1{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{z}}_f+{}_2{\stackrel{}{S}}_{\mathrm{ff}}^{-1}-{\stackrel{}{S}}_{\mathrm{ff}}^{-1}{\stackrel{`}{q}}_f^{*}\end{array}\right)\\ =-O_f^T{}_3{}_f-\frac{1}{2}{O}_f^T{O}_f+O_f^T{\stackrel{`}{W}}_a^T{}_a\left({}_a\right)b\\ +O_f^T{}_a+\left[{}_{f}N\left({}_f\right)+1\right]{\stackrel{`}{}}_f\end{array}& \left(68\right)\end{array}$