VALIDATING AND COMPUTING STABILITY LIMITS OF HUMAN-IN-THE-LOOP ADAPTIVE CONTROL SYSTEMS

Abstract

Systems and methods for implementing and/or validating a model reference adaptive control (MRAC) for human-in-the-loop control of a vehicle system. A first operator model is applied to a first feedback-loop-based MRAC scheme, wherein the first operator model is configured to adjust a control command provided as an input to the MRAC scheme based at least in part on an actual action of the vehicle system and a reference action for the vehicle system with a time-delay. A stability limit of a first operating parameter is determined for the MRAC scheme based on the application of the first operator model to the first feedback-loop-based MRAC scheme. The MRAC scheme is validated in response to determining that expected operating conditions of the first operating parameter are within the determined stability limit of the first operating parameter.

Claims

1. A method of implementing a model reference adaptive control (MRAC) for a vehicle system, the method comprising: defining a first feedback-loop-based MRAC scheme, wherein the first feedback-loop based MRAC scheme is configured to receive a control command, apply a reference model to determine a desired action for the vehicle system based on the control command, determine an actuator command based on the control command, transmit the actuator command to at last one actuator of the vehicle system, monitor a sensor to determine an actual action of the vehicle system in response to application of the actuator command by the at least one actuator, determine a system error based on a difference between the desired action determined by the reference model and the actual action, and adjust at least one adaptive parameter used to determine the actuator control command based on the determined system error; applying a first operator model to the first feedback-loop-based MRAC scheme, wherein the first operator model is configured to adjust the control command based at least in part on the actual action of the vehicle system and a reference action for the vehicle system with a time-delay; determining a stability limit of a first operating parameter of the first feedback-loop-based MRAC scheme based on the application of the first operator model to the first feedback-loop-based MRAC scheme; and validating the first feedback-loop-based MRAC scheme in response to determining that expected operating conditions of the first operating parameter are within the determined stability limit of the first operating parameter.

2. The method of claim 1, further comprising: receiving, by an electronic process, the control command from a user control; and controlling the vehicle system by an electronic processor configured to apply the first feedback-loop-based MRAC scheme to generate the actuator command in response to a control command received from a user control.

3. The method of claim 2, wherein receiving the control command from a user control includes receiving a control command from a steering wheel, wherein the control command is indicative of a rotational position of the steering wheel.

4. The method of claim 2, wherein determining the stability limit of the first operating parameter includes determining whether the first feedback-loop-based MRAC scheme will cause the system error to approach zero regardless of variations in the first operating parameter due to human operator-based manipulations of the user control.

5. The method of claim 1, wherein controlling the vehicle system by the electronic processor further includes: determining, by the electronic processor, the actuator command based on the control command received from the user control and a previous actuator command value to ensure that the first operating parameter remains within the determined stability limit of the first operating parameter.

6. The method of claim 1, further comprising: determining that the expected operating conditions of the first operating parameters are not within the determined stability limit of the first operating parameter and, in response, adjusting at least one parameter of the first feedback-loop-based MRAC scheme.

7. The method of claim 1, further comprising: determining that the expected operating conditions of the first operating parameters are not within the determined stability limit of the first operating parameter and, in response, defining a second feedback-loop-based MRAC scheme and applying the first operator model to the second feedback-loop-based MRAC scheme.

8. The method of claim 1, wherein the first operating parameter of the first feedback-loop-based MRAC scheme includes a time-delay indicative of a period of time between the occurrence of the actual action and a corresponding corrective action applied by an operator to a user control.

9. The method of claim 8, wherein determining the stability limit of the first operating parameter of the first feedback-loop-based MRAC scheme includes determining whether the feedback-loop-based MRAC scheme will ensure that operation of the vehicle system remains stable regardless of a value of the time-delay parameter.

10. The method of claim 7, wherein determining the stability limit of the first operating parameter of the first feedback-loop-based MRAC scheme includes determining range of time-delay values for which the first feedback-loop-based MRAC scheme will ensure that operation of the vehicle system remains stable, and wherein validating the first feedback-loop-based MRAC scheme includes determining that a range of expected time-delay values for the operator is within the determine range of time-delay values.

11. The method of claim 1, wherein the vehicle system includes an airplane control system and wherein the first feedback-loop-based MRAC scheme is configured to adjust the actuator to counteract an external force acting on the airplane and to maintain a desired path of travel.

12. The method of claim 11, wherein the external force acting on the airplane includes turbulence.

13. The method of claim 1, wherein the vehicle system includes an automobile system and wherein the first feedback-loop-based MRAC scheme is configured to regulate operation of at least one selected from a group consisting of an automobile steering system and an automobile braking system.

14. The method of claim 1, wherein applying the first operator model to the first feedback-loop-based MRAC scheme includes determining a mathematical model representative of the first operator model and apply the mathematical model of the first operator model to a mathematical model representative of the first feedback-loop-based MRAC scheme to determine an overall mathematical model representative of system operation under parallel control of both a human operator and the first feedback-loop-based MRAC scheme.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] FIG. 1 is a block diagram of a control system for a vehicle according to one embodiment.

[0015] FIG. 2 is a schematic diagram of a method for controlling an operation of a vehicle using a model reference adaptive control (MRAC) in the system of FIG. 1.

[0016] FIG. 3 is a schematic diagram of a method for controlling an operation of a vehicle in order to validate a MRAC in the system of FIG. 1.

[0017] FIG. 4 is a schematic diagram of a method for validating an MRAC for use in the system of FIG. 1 using modeling.

[0018] FIG. 5 is a graph of the location of the right most pole (RMP) of a characteristic equation representative of a human-in-the-loop MRAC with respect to a control penalty variable p for different pilot reaction times.

[0019] FIG. 6 is a graph of tracking and control signal curves for two different values of the pilot reaction times in the example of FIG. 5.

[0020] FIG. 7 is a graph of the location of the right-most pole (RMP) of the characteristic equation representative of the human-in-the-loop MRAC with respect to a control penalty variable for different pilot transfer function pole locations.

[0021] FIG. 8 is a graph of tracking and control signal curves for two different values of pilot transfer function pole locations.

[0022] FIG. 9 is a graph of the location of the right-most pole (RMP) of the characteristic equation representative of the human-in-the-loop MRAC with respect to a control penalty variable for different pilot transfer function zero locations.

[0023] FIG. 10 is a graph of tracking and control signal curves for two different values of pilot transfer function zero locations.

[0024] FIG. 11 is a graph of the location of the right-most pole (RMP) of the characteristic equation representative of the human-in-the-loop MRAC with respect to a control penalty variable for different pilot transfer function gain values.

[0025] FIG. 12 is a graph of tracking and control signal curves for two different values of the pilot transfer function gain.

[0026] FIG. 13 is a flowchart of a method for validating the stability of an MRAC for use in the system of FIGS. 1 and 2.

[0027] FIG. 14 is a flowchart of a method for tuning a controller based only determined stability limits of an MRAC for use in the system of FIGS. 1 and 2.

DETAILED DESCRIPTION

[0028] Before any embodiments of the invention are explained in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the following drawings. The invention is capable of other embodiments and of being practiced or of being carried out in various ways.

[0029] FIG. 1 illustrates an example of a human-in-the-loop, feedback-based control system. A controller 101 includes an electronic processor 103 that is communicatively coupled to a computer-readable, non-transitory memory 105. The memory 105 stores instructions that, when executed by the electronic processor 103, causes the controller to provide various functionality of the system including certain functionality as described herein. The controller 101 is communicatively coupled to one or more actuators and/or vehicle systems 107 and is configured to provide a control signal to the actuator(s)/vehicle system 107 to dictate the operation of the actuator(s)/vehicle system 107. The controller 101 is also communicatively coupled to one or more sensors 109 that monitor one or more performance variables/states of the system being controlled by the system of FIG. 1. The controller 101 is also configured to receive a user control input from one or more user control(s) 111.

[0030] For example, the system illustrated in FIG. 1 may be implemented in an automobile where the user control 111 includes an steering wheel of the automobile, the actuator(s)/vehicle system 107 includes a steering and/or braking system, and the sensor 109 includes a yaw sensor configured to monitor turning characteristics of the automobile. In another example, the system may be configured to control the steering of an airplane. In still other examples, the system may be configured to control other aspects and/or of other systems.

[0031] FIG. 2 illustrates an example of a method implemented by the controller 101 for controlling the operation of the vehicle using a model reference adaptive controller (MRAC) mechanism. The controller 101 receives a control command from the user control 111 indicative of a desired task to be performed (e.g., adjusting the steering of an airplane or an automobile). The controller 101 applies the control command as an input to a reference model 201 to determine a target behavior or action to be performed in response to the control command from the user control 111. The controller 101 also applies the control command as an input to a vehicle system control 203 that determines an appropriate actuator command, which is then transmitted as a control signal from the controller 101 to the actuator 107. The controller 101 then monitors an output from a sensor 109 to determine how the system responded to the actuator command. The actual performance of the system, as indicated by the output of the sensor 109, is then compared to the expected performance as determined by the reference model 201. The difference between the actual performance and the expected performance is identified as a system error. The controller 101 is configured to apply a parameter adjustment 205 that subsequently adjusts the actuator command provided by the vehicle system control 203 to the actuator 107. In this way, the controller 101 is configured to use feedback to minimize the system error and, as a consequence, the actual performance approaches the expected performance.

[0032] The control architecture illustrated in FIG. 2 enables the controller 101 to adapt to changes in the actuator/system itself due to degradation and to adapt to external forces that influence the actual performance of the vehicle in such a way that the system continues to operate as expected (i.e., actual performance matches/approaches expected performance). For example, in an implementation where the system of FIGS. 1 and 2 is provided to control the steering of an airplane, the control command from the user control 111 may indicate that the operator intends to continue to fly straight. However, turbulence or a strong wind may cause the airplane to diverge from its intended course. As a result, the expected performance (i.e., straight travel) will not match/equal the actual performance. Using the MRAC control architecture of FIG. 2, the controller 101 uses feedback from the sensor 109 to adjust the actuator command 107 that is provided to the actuator 107 and, thereby, adapt to account for the strong wind or turbulence.

[0033] However, before the controller 101 is able to adjust the actuators 107 in such a way that the actual performance is corrected to match the expected performance, the pilot of the airplane may also notice that the path of travel of the airplane is deviating from its intended straight path. In response, the pilot may adjust the position of the user control 111 in a way intended to offset/correct for the deviation in the path of travel. Accordingly, the controller 101 and the human operator (via the user control 111) both attempt to correct for the system error. However, the human-induced correction may inadvertently affect the ability of the controller 101 to correct the system error and, in some cases, the interference of the human-induced correction and the MRAC implemented by the controller 101 may, not only prevent the controller 101 from correcting the system error, but may also cause the steering of the airplane to become unstable.

[0034] To study the effect of human interactions with the MRAC control architecture, the system may be adjusted to apply an additional modeled feedback loop mechanism. For example, a human dynamics model 301, as discussed in further detail below, may be provided as a control model designed to represent an expected human response to detecting an actual performance that does not match the expected performance. In this way, the control architecture provided by the system of FIG. 3 includes an inner loop 311, in which the MRAC is applied to correct for deviations between actual performance and expected performance and also includes an outer loop 313 in which the control command provided to the vehicle system control 307 is adjusted based on an expected human response. The human dynamics model 301 in the example of FIG. 3 can, in some implementations be implemented on the same controller 101 that is actively applying the MRAC. In other implementations, the human dynamics model 301 is simulated and/or represented mathematically on a separate computer system.

[0035] Furthermore, in still other examples, the performance capabilities of the MRAC can be evaluated through modeling instead of through observation of actual system performance. For example, we start with the block diagram configuration given by FIG. 4. In FIG. 4, the outer loop 413 architecture includes the reference that is fed into the human dynamics model 401 to generate a command for the inner loop 411 architecture in response to the variations resulting from the uncertain dynamical system 407. In this setting, the reference input is what the human aims to achieve in a task (e.g., the expected performance), and the uncertain dynamical system 407 represents the machine on which this task is being performed. The inner loop 411 architecture includes the uncertain dynamical system 407 as well as the model reference adaptive controller components (i.e., the reference model 403, the parameter adjustment mechanism 409, and the vehicle system control 405). Specifically, at the outer loop 413 architecture, we consider a general class of linear human models with constant time-delay given by

{dot over ()}(t)=A.sub.h(t)+B.sub.h(t),(0)=.sub.0(1)

c(t)=C.sub.h(t)+D.sub.h(t)(2)

where (t).sup.n.sup. is the internal human state vector, .sub.+ is the internal human time-delay, A.sub.h.sup.n.sup..sup.n.sup., B.sub.h.sup.n.sup..sup.n.sup.r, C.sup.h.sup.n.sup.c.sup.n.sup., D.sub.h.sup.n.sup.c.sup.n.sup.r, c(t).sup.n.sup.c is the command produced by the human, which is the input to the inner loop architecture as shown in FIG. 4. Here, input to the human dynamics is given by

(t)r(t)E.sub.h(t)(3)

where (t).sup.n.sup.r, with r(t).sup.n.sup.r being the bounded reference. Here x(t).sup.n is the state vector (further details below) and E.sub.h.sup.n.sup.r.sup.n selects the appropriate states to be compared with r(t). Note that the dynamics given by (1), (2), and (3) is general enough to capture, for example, linear time-invariant human models with time-delay including Neal-Smith model and its extensions.

[0036] Next, at the inner loop architecture, we consider the uncertain dynamical system given by

{dot over (x)}.sub.p(t)=A.sub.px.sub.p(t)+B.sub.pu(t)+B.sub.p.sub.p(x.sub.p(t)), x.sub.p(0)=x.sub.p.sub.0(4)

where x.sub.p(t).sup.n.sup.p is the accessible state vector, u(t).sup.m is the control input, .sub.p: .sup.n.sup.p.fwdarw..sup.m is an uncertainty, A.sub.p.sup.n.sup.p.sup.n.sup.p is a known system matrix, B.sub.p.sup.n.sup.p.sup.m is a known control input matrix, and .sub.+.sup.mm D.sup.mm is an unknown control effectiveness matrix. Furthermore, we assume that the pair(A.sub.p, B.sub.p) is controllable and the uncertainty is parameterized as

.sub.p(x.sub.p)=W.sub.p.sup.T.sub.p(x.sub.p), x.sub.p.sup.n.sup.p(5)

where W.sub.p.sup.sm is an unknown weight matrix and .sub.p.sup.n.sup.p.fwdarw..sup.s is a known basis function of the form .sub.p(x.sub.p)=[.sub.p.sub.1(x.sub.p), .sub.p.sub.2(x.sub.p), . . . , .sub.p.sub.s(x.sub.p)].sup.T. Note for the case where the basis function .sub.p(x.sub.p) is unknown, the parameterization in (5) can be relaxed without significantly changing the results of this invention by considering

.sub.p(x.sub.p)=W.sub.p.sup.T.sub.p.sup.nn(V.sub.p.sup.Tx.sub.p)+.sub.p.sup.nn(x.sub.p),x.sub.pD.sub.x.sub.p(6)

where W.sub.p.sup.sm and V.sub.p.sup.n.sup.p.sup.s are unknown weight matrices, .sub.p.sup.nn: D.sub.x.sub.p.fwdarw..sup.s is a known basis composed of neural networks function approximators, .sub.p.sup.nn: D.sub.x.sub.p.fwdarw..sup.m is an unknown residual error, and D.sub.x.sub.p is a compact subset of .sup.n.sup.p.

[0037] To address command following at the inner loop architecture, let x.sub.c(t).sup.n.sup.c be the integrator state satisfying

{dot over (x)}.sub.c(t)=E.sub.px.sub.p(t)c(t), x.sub.c(0)=x.sub.c.sub.0(7)

where E.sub.p.sup.n.sup.c.sup.n.sup.p allows to choose a subset of x.sub.p(t) to be followed by c(t). Now, (4) can be augmented with (7) as

{dot over (x)}(t)=Ax(t)+Bu(t)+BW.sub.p.sup.T.sub.p(x.sub.p(t))+B.sub.rC(t), x(0)=x.sub.0(8)

where

[00001]$\begin{array}{cc}A.Math.\stackrel{}{=}.Math.\left[\begin{array}{cc}{A}_p& 0_{n_p{n}_c}\\ E_p& 0_{n_c{n}_c}\end{array}\right]{}^{nn}& \left(9\right)\\ B.Math.\stackrel{}{=}.Math.{\left[B_p^T,0_{n_{c}m}^T\right]}^T{}^{nm}& \left(10\right)\\ B_r.Math.\stackrel{}{=}.Math.{\left[0_{n_p{n}_c}^T,-I_{n_c{n}_c}\right]}^T{}^{n{n}_c}& \left(11\right)\end{array}$

and x(t)[x.sub.p.sup.T(t),x.sub.c.sup.T(t)].sup.T.sup.n is the augmented state vector, x.sub.0 [x.sub.p.sub.0.sup.T,x.sub.c.sub.0.sup.T].sup.T.sup.n, and n=n.sub.pn.sub.c. In this inner loop architecture setting, it is practically reasonable to set E.sub.h=[E.sub.h.sub.p,0.sub.n.sub.r.sub.n.sub.c], E.sub.h.sub.p.sup.n.sup.r.sup.n.sup.p, in (3) without loss of theoretical generality since a subset of the accessible state vector is usually available and/or sensed by the human at the outer loop (but not the states of the integrator).

[0038] Finally, consider the feedback control law at the inner loop architecture given by

u(t)=u.sub.n(t)+u.sub.a(t)(12)

where u.sub.n(t).sup.m and u.sub.a(t).sup.m are the nominal and adaptive control laws, respectively. Furthermore, let the nominal control law be

u.sub.n(t)=Kx(t)(13)

with K.sup.mn, such that A.sub.r ABK is Hurwitz. For instance, such K exists if and only if (A,B) is a controllable pair. Using (12) and (13) in (8) next yields

{dot over (x)}(t)=A.sub.rx(t)+B.sub.rc(t)+B[u.sub.a(t)+W.sup.T(x(t))](14)

where W.sup.T[.sup.1W.sub.p.sup.T,(.sup.1I.sub.mm)K].sup.(s+n)m is an unknown aggregated weight matrix and .sup.T(x(t))[.sub.p.sup.T(x.sub.p(t)),x.sup.T(t)].sup.s+n is a known aggregated basis function. Considering (14), let the adaptive control law be

u.sub.a(t)=.sup.T(t)(x(t))(15)

where (t).sup.(s+n)m is the estimate of W satisfying the parameter adjustment mechanism

{dot over ()}(t)=y(x(t))e.sup.T(t)PB, (0)=.sub.0(16)

where .sub.+ is the learning rate, and system error reads,

e(t)x(t)x.sub.r(t)(17)

with x.sub.r(t).sup.n being the reference state vector satisfying the reference system

{dot over (x)}.sub.r(t)=A.sub.rx.sub.r(t)+B.sub.rc(t),x.sub.r(0)=x.sub.r.sub.0(18)

and P.sub.+.sup.nnS.sup.nn is a solution of the Lyapunov equation

0=A.sub.r.sup.TP+PA.sub.r+R(19)

with R.sub.+.sup.nnS.sup.nn. Since A.sub.r is Hurwitz, it follows that there exists a unique P.sup.nnS.sup.nn satisfying (19) for a given R.sub.+.sup.nnS.sup.nn. Although we consider a specific yet widely studied parameter adjustment mechanism given by (16), one can also consider other types of parameter adjustment mechanisms without changing the essence of this invention.

[0039] Based on the given problem formulation, the next section analyzes the stability of the coupled inner and outer loop architectures depicted in FIG. 4 in order to establish a fundamental stability limit for guaranteeing the closed-loop system stability (when this limit is satisfied by the given human model at the outer loop and the given adaptive controller at the inner loop).

[0040] Fundamental Stability Limit Calculation

[0041] To analyze the stability of the coupled inner and outer loop architectures introduced in the previous section, we first write the system error dynamics using (14), (15), and (18) as

(t)=A.sub.re(t)B{tilde over (W)}.sup.TT(t)(x(t)),e(0)=e.sub.0(20)

where

{tilde over (W)}(t)(t)W.sup.(s+n)m(21)

is the weight error and e.sub.0x.sub.0x.sub.r.sub.0. In addition, we write the weight error dynamics using (16) as

{dot over ({tilde over (W)})}(t)=(x(t))e.sup.T(t)PB, {tilde over (W)}(0)={tilde over (W)}.sub.0(22)

where {tilde over (W)}.sub.0(0)W. The following lemma is now immediate.

[0042] Lemma 1.

[0043] Consider the uncertain dynamical system given by (4) subject to (5), the reference model given by (18), and the feedback control law given by (12), (13), (15), and (16). Then, the solution (e(t), {tilde over (W)}(t)) is Lyapunov stable for all (e.sub.0, {tilde over (W)}.sub.0).sup.n.sup.(s+n)m and t.sub.+.

[0044] Proof.

[0045] To show Lyapunov stability of the solution (e(t), {tilde over (W)}(t)) given by (20) and (22) for all (e.sub.0, {tilde over (W)}.sub.0).sup.n.sup.(s+n)m and t.sub.+, consider the Lyapunov function candidate

V(e,{tilde over (W)})=e.sup.TPe+.sup.1tr({tilde over (W)}.sup.1/2).sup.T({tilde over (W)}.sup.1/2)(23)

[0046] Note that V(0,0)=0,V(e,{tilde over (W)})>0 for all (e,{tilde over (W)})(0,0), and V(e,{tilde over (W)}) is radially unbounded. Differentiating (23) along the trajectories of (20) and (22) yields

{dot over (V)}(e(t),{tilde over (W)}(t))=e.sup.T(t)Re(t)0(24)

where the result is now immediate.

[0047] Since the solution (e(t),{tilde over (W)}(t)) is Lyapunov stable for all (e.sub.0,{tilde over (W)}.sub.0).sup.n.sup.(s+n)m and t.sub.+ from Lemma 1, this implies that e(t)L.sub. and {tilde over (W)}(t)L.sub.. At this stage in our analysis, it should be noted that one cannot use the Barbalat's lemma to conclude lim.sub.t.fwdarw. e(t)=0. To elucidate this point, one can write

{umlaut over (V)}(e(t),{tilde over (W)}(t))=2e.sup.T(t)R[A.sub.re(t)B{tilde over (W)}.sup.T(t)(e(t)+x.sub.r(t))](25)

where since x.sub.r(t) can be unbounded due to the coupling between the inner and outer loop architectures, one cannot conclude the boundedness of (25), which is necessary for utilizing the Barbalat's lemma in (24). Motivated from this standpoint, we next provide the conditions to ensure the boundedness of the reference model states x.sub.r(t), which also reveal the fundamental stability limit (FSL) for guaranteeing the closed-loop system stability. It is noted that two FSLs are provided below; namely, a delay-independent FSL and a delay-dependent FSL.

[0048] Delay-Independent FSL

[0049] A linear time invariant system subject to time delay can in some cases be stable regardless of how large the time delay is. We present the mathematical conditions under which the system at hand can be delay-independent stable. For this, start with using (2) in (18), and first write

[00002]$\begin{array}{cc}\begin{array}{c}{\stackrel{.}{x}}_r\left(t\right)=.Math.A_r.Math.x_r\left(t\right)+B_r.Math.\left(C_h.Math.\left(t\right)+D_h.Math.\left(t-\right)\right),\\ =.Math.A_r.Math.x_r\left(t\right)-B_r.Math.D_h.Math.E_h.Math.x_r\left(t-\right)+B_r.Math.C_h.Math.\left(t\right)-\\ .Math.B_r.Math.D_h.Math.E_h.Math.e\left(t-\right)+B_r.Math.D_h.Math.r\left(t-\right)\end{array}& \left(26\right)\end{array}$

[0050] Next, it follows from (1) that

{dot over ()}(t)=A.sub.h(t)B.sub.hE.sub.hx.sub.r(t)B.sub.hE.sub.he(t)+B.sub.rr(t)(27)

[0051] Finally, by letting (t)[x.sub.r.sup.T(t),.sup.T(t)].sup.T, and using (26) and (27), one can write

{dot over ()}(t)=A.sub.0(t)+A.sub.(t)+(.), (0)=.sub.0(28)

where

[00003]$\begin{array}{cc}{A}_0.Math.\stackrel{}{=}.Math.\left[\begin{array}{cc}{A}_r& B_r.Math.C_h\\ 0_{n_{}n}& A_h\end{array}\right]{}^{\left(n+n_{}\right)\left(n+n_{}\right)}& \left(29\right)\\ A_{}.Math.\stackrel{}{=}.Math.\left[\begin{array}{cc}-B_r.Math.D_h.Math.E_h& 0_{n{n}_{}}\\ -B_h.Math.E_h& 0_{n_{}{n}_{}}\end{array}\right]{}^{\left(n+n_{}\right)\left(n+n_{}\right)}& \left(30\right)\\ (.Math.).Math.\stackrel{}{=}.Math.\left[\begin{array}{c}-B_r.Math.D_h.Math.E_h.Math.e\left(t-\right)+B_r.Math.D_h.Math.r\left(t-\right)\\ -B_h.Math.E_h.Math.e\left(t-\right)+B_h.Math.r\left(t-\right)\end{array}\right]{}^{n+n_{}}& \left(31\right)\end{array}$

[0052] As a consequence of Lemma 1 and the boundedness of the reference r (t), one can conclude that (.)L.sub.. We now state the following lemma that is necessary for the main result of this invention.

[0053] LEMMA 2. Let P.sub.+.sup.(n+n.sup..sup.)(n+n.sup..sup.)S.sup.(n+n.sup..sup.)(n+n.sup..sup.) and S.sub.+.sup.(n+n.sup..sup.)(n+n.sup..sup.)S.sup.(n+n.sup..sup.)(n+n.sup..sup.) such that the linear matrix inequality (LMI)

[00004]$\begin{array}{cc}F.Math.\stackrel{}{=}.Math.\left[\begin{array}{cc}{A}_0^T.Math.P+{\mathrm{PA}}_0+S& {\mathrm{PA}}_{}\\ A_{}^T.Math.P& -S\end{array}\right]<0& \left(32\right)\end{array}$

holds. Then, (t) of the dynamical system given by (28) is bounded for any .sub.+ and for all (t).sup.n+n.sup. and .sub.+.

[0054] PROOF. Consider the Lyapunov-Krasovskii functional candidate given by

V()=.sup.TP+.sub..sup.0.sup.T(t+)d(33)

and, since (.)L.sub., let *.sub.+ be such that (.).sub.2*. Differentiating (33) along the trajectory of (28) yields

{dot over (V)}((t)).sup.T(t)F(t)+2.sub.max(P)*|(t).sub.2(34)

where (t)[.sup.T(t),.sup.T(t)].sup.T. Since (32) holds, let k.sub.+ be such that k .sub.min(F). Now, it follows from (34) that

{dot over (V)}((t))k(t).sub.2((t).sub.22k.sup.1.sub.max(P)*)(35)

and hence, there exists a compact set R({(t).sup.2(n+n.sup..sup.): (t).sub.22k.sup.1.sub.max(P)*} such that {dot over (V)}((t))0 outside of this set, which proves the boundedness of (28) for any .sub.+ and for all (0).sup.n+n.sup. and .sub.+.

[0055] Lemma 2 establishes the boundedness of not only the reference model states, the dynamics of which are given by (18), but also the internal human dynamics given by (1), and hence, x.sub.r(t)L.sub. and (t)L.sub..

[0056] Theorem 1.

[0057] Consider the uncertain dynamical system given by (4) subject to (5), the reference model given by (18), the feedback control law given by (12), (13), (15), and (16), and the human dynamics given by (1), (2), and (3). Then, e(t)L.sub. and {tilde over (W)}(t)L.sub.. If, in addition, there exist P.sub.+.sup.(n+n.sup..sup.)(n+n.sup..sup.)S.sup.(n+n.sup..sup.)(n+n.sup..sup.) and S.sub.+.sup.(n+n.sup..sup.)(n+n.sup..sup.)S.sup.(n+n.sup..sup.)(n+n.sup..sup.) such that the LMI given by (32) holds, then x.sub.r(t)L.sub., (t)L.sub., and lim.sub.t.fwdarw. e(t)=0.

[0058] Proof.

[0059] As a consequence of Lemma 1, recall that e(t)L.sub. and {tilde over (W)}(t)L.sub.. In addition, note that (.)L.sub. in (28). Next, if there exist P.sub.+.sup.(n+n.sup..sup.)(n+n.sup..sup.)S.sup.(n+n.sup..sup.)(n+n.sup..sup.) and S.sub.+.sup.(n+n.sup..sup.)(n+n.sup..sup.)S.sup.(n+n.sup..sup.)(n+n.sup..sup.) such that the LMI given by (32) holds, recall from Lemma 2 that x.sub.r(t)L.sub. and (t)L.sub.. Finally, since e(t)L.sub., x.sub.r(t)L.sub., and {tilde over (W)}(t)L.sub. ensure the boundedness of (25), it now follows from the Barbalat's lemma that lim.sub.t.fwdarw. e(t)=0.

[0060] For the boundedness of all closed-loop system signals and lim.sub.t.fwdarw. e(t)=0, Theorem 1 requires the fundamental stability limit given by the LMI (32) to hold. Note that this fundamental stability limit can be equivalently written in an equality form as

0=A.sub.0.sup.TP+PA.sub.0P+A.sub.S.sup.1A.sub..sup.TP+S+Q(36)

where P.sub.+.sup.(n+n.sup..sup.)(n+n.sup..sup.)S.sup.(n+n.sup..sup.)(n+n.sup..sup.), S.sub.+.sup.(n+n.sup..sup.)(n+n.sup..sup.)S.sup.(n+n.sup..sup.)(n+n.sup..sup.), and Q.sub.+.sup.(n+n.sup..sup.)(n+n.sup..sup.)S.sup.(n+n.sup..sup.)(n+n.sup..sup.) with A.sub.0 and A.sub. respectively given by (29) and (30). Importantly, in addition, note that A.sub.0 and A.sub. do not depend on any unknown parameters and they only depend on the given set of human model and reference model parameters. As a consequence, for a given human model of the form (1), (2), and (3), if the fundamental stability limit given by (36) (or, equivalently (32)) holds with respect to a judiciously chosen reference model parameters, then the closed-loop system trajectories are guaranteed to be stable.

[0061] Notice above that we have employed a time-domain technique based on a Lyapunov-Krasovskii functional to prove delay independent stability. A large body of literature was devoted to this effort where one main focus was to reduce the inherent conservatism imposed by the choice of candidate functionals. Another method would be to employ frequency domain tools where one instead studies the eigenvalues of the corresponding linear time invariant system with time delay. For example, consider the nominal part of (28); e.g., (.)=0, with .fwdarw.. In this case, the system will behave like an open loop system whose stability is determined by the eigenvalues of A.sub.0. For the system to be stable in this setting, A.sub.0 must be Hurwitz, which also makes it invertible. Next, we note that the characteristic function of the dynamical system

f:=det[sIA.sub.0A.sub.e.sup.s](37)

can be rearranged as

det[I(sIA.sub.0).sup.1A.sub.e.sup.s]*det[sIA.sub.0](38)

[0062] Note that for the class of time-delay systems being considered here, as a parameter of interest; e.g., delay, changes, the system can switch from a stable to unstable regime (or vice versa) if and only if the system has imaginary eigenvalues s=j. Investigation of whether or not such a switch could arise then requires studying the zeros of the system characteristic function (38) at s=j, where <0 without loss of generality. On the imaginary axis however only the first determinant can be zero since the second determinant is always non-zero owing to A.sub.0 being Hurwitz. Denoting with (.) the spectral radius and noticing that |e.sup.j|=1, we have the following theorem.

[0063] Theorem 2.

[0064] The dynamical system given by (28) with (.)=0 is asymptotically stable independent of delay if and only if

[0065] i) A.sub.0 is asymptotically stable;

[0066] ii) ((jIA.sub.0).sup.1A.sub.)<1, >0; and

[0067] iii) either a) (A.sub.0.sup.1A.sub.)<1 or b) (A.sub.0.sup.1A.sub.)=1 and det(A.sub.0+A.sub.)0.

[0068] Implementing the steps in the above theorem are straightforward. Condition i) can be checked by a standard eigenvalue computation, while condition ii) requires sweeping of the frequency >0. Here one generates the matrix (jIA.sub.0).sup.1A.sub. and for a given , computes the eigenvalues. If all these eigenvalues fall into the unit circle then condition ii) is satisfied for this . This process is repeated for all . Note that the inverse matrix operation here will guarantee that, for sufficiently large , condition ii) will always be satisfied as the spectral radius will keep shrinking. Checking of condition iii) is much simpler as it does not require parametric scanning but only computation of eigenvalues. Note that condition iii) is the special case of condition ii) computed at =0.

[0069] Corollary 1.

[0070] Let the human dynamics given by (1), (2), and (3) be a single-input single-output system (SISO) with gain k.sub.p. Then, for (28) with (.)=0 to be delay-independent stable, it is necessary that

[00005]$\begin{array}{cc}{k}_p<\frac{1}{\left(A_r^{-1}.Math.B_r.Math.E_h\right)}& \left(39\right)\end{array}$

holds.

[0071] Proof.

[0072] Start with (29) and (30) and rewrite the characteristic function (37) explicitly as

f:=det[sIA.sub.r+B.sub.r(C.sub.h(sIA.sub.h).sup.1B.sub.h+D.sub.h)E.sub.he.sup.s](40)

which simplifies to

f:=det[sIA.sub.r+B.sub.rE.sub.hG(s)e.sup.s](41)

where G(s) is the scalar transfer function corresponding to the SISO system given by (1) and (2). Note that the above expression is in the exact form as (37); hence, for (28) with (.)=0 to be delay-independent stable, it is necessary that condition i) of Theorem 2 holds, which in this case requires that A.sub.r must be Hurwitz. As per the construction in (13) this always holds. Then, invoking condition ii) in Theorem 2 at =0, and recalling that k.sub.p=G(0), we have

((A.sub.r).sup.1(B.sub.rE.sub.h)G(0))<1(42)

which gives (39), and hence, the proof is now complete.

[0073] It is worthy to note that the results in Corollary 1 can be further improved in many practical situations. For example, observe that the reference input to the human model and the human command are of dimension one in the SISO case. In addition, since generally the outer loop and inner loop command following objectives are the same, note that E.sub.h.sub.p=E.sub.p. Thus, in view of these, the following result is now immediate.

[0074] Corollary 2.

[0075] Given E.sub.h.sub.p=E.sub.p and under the conditions in Corollary 1, the necessary condition for the human-in-the loop MRAC model (28) with (.)=0 to be delay-independent stable is given by

k.sub.p<1(43)

[0076] Proof.

[0077] Note that A.sub.r.sup.1B.sub.r and E.sub.h in (39) are column vectors. Therefore, we have (A.sub.r.sup.1B.sub.rE.sub.h)=|E.sub.hA.sub.r.sup.1B.sub.r|. Since in the scalar case, E.sub.hA.sub.r.sup.1B.sub.r=1, then (43) follows.

[0078] In the above corollary, we prove that the human gain must be less than one such that (28) with (.)=0 can have a chance to be delay-independent stable. The sufficiency can be numerically checked by studying condition ii) of Theorem 2 (see the next section). What is interesting in the above analysis is that human's aggressiveness as measured by k.sub.p can be a strong limiting factor that ruins delay-independent stability. In the case when MRAC deals with a highly aggressive human behavior with k.sub.p>1, it is impossible to avoid instability for some delay values . Moreover, since by the design of stable MRAC we have zero steady-state error in tracking, the necessary condition k.sub.p<1 is solely inherent to the human's gain and holds irrespective of the controller gain K. While in many cases it is reasonable to assume that the human model can be considered as SISO dynamics; e.g., when the human produces a single output to steer a manipulator, in the case when an auto-human model is utilized in multi-input multi-output (MIMO) form, the necessary condition (42) can be revised as follows

(A.sub.r.sup.1B.sub.r|G(0)|E.sub.h)<1(44)

where [G(0)] denotes the matrix transfer function of the MIMO auto-human model with s=0 in its all entries.

[0079] It is important to note that while guaranteeing delay-independent stability in a dynamical system is attractive as this makes the system completely immune to destabilizing effects of delays, in some cases by the nature of the problem, delay-independent stability cannot be possible as is the case above for k.sub.p>1. Moreover, a trade-off in delay-independent stable cases is system's performance, which may deteriorate for large delays although stability is preserved. In light of this, we now turn our attention to the case when delay-independent stability is not possible, or not desired, and hence, system stability is affected by the numerical value of the delay in the dynamical system.

[0080] Delay Dependent FSL

[0081] Delay-independent FSL given in the previous section guarantees the boundedness of all closed loop system signals and lim.sub.t.fwdarw. e(t)=0 for any .sub.+. Since the time delay in human dynamics can in general be known in practice for certain applications, at least within a certain range, it is possible to relax these conditions by utilizing the delay information in the stability analysis. Towards this goal, we first provide the following lemma.

[0082] Lemma 3.

[0083] Consider the following system dynamics given by

(t)=Fz(t)+Gz(t)+h(t,z(t)),z(0)=z.sub.0(45)

where z(t).sup.n is the state vector, F.sup.nn and G.sup.nn are constant matrices, is the time delay, and h(t, z(t)) is piecewise constant and bounded nonlinear forcing term, which is in general a function of state z. If the homogeneous dynamical system given by

(t)=Fz(t)+Gz(t)(46)

is asymptotically stable, then the states of the original inhomogeneous dynamical system given by (45) remains bounded for all times.

[0084] Proof.

[0085] Since h(t, z(t)) is piecewise continuous and bounded, this signal can be considered as an exogenous input to the system with the transfer function

G(s)=(sI(F+Ge.sup.s)).sup.1(47)

[0086] Under the assumption that the homogeneous system (46) is asymptotically stable, then we have that all of the infinitely many roots of the characteristic equation

det(sI(F+Ge.sup.s))=0(48)

of the system (47), have strictly negative real parts. Therefore, the output z(t) of the dynamical system remains bounded.

[0087] Having established Lemma 3, we are now ready to state the second main result of this invention, which provides a more relaxed delay-dependent stability condition for the overall human-in-the-loop system and convergence of the system error, e(t), to zero.

[0088] Theorem 3.

[0089] Consider the uncertain dynamical system given by (4) subject to (5), the reference model given by (18), the feedback control law given by (12), (13), (15), and (16), and the human dynamics given by (1), (2), and (3). Then, e(t)L.sub. and {tilde over (W)}(t)L.sub.. If, in addition, the real parts of all the infinitely many roots of the following characteristic equation

det(sI(A.sub.0+A.sub.e.sup.s))=0(49)

have strictly negative real parts, then x.sub.r(t)L.sub., (t)L.sub., and lim.sub.t.fwdarw. e(t)=0.

[0090] Proof.

[0091] As a consequence of Lemma 1, recall that e(t)L.sub. and {tilde over (W)}(t)L.sub.. In addition, note that (.)L.sub. in (28). Therefore, if all of the roots of the characteristic equation given by (49) have strictly negative real parts, making the homogeneous equation

{dot over ()}(t)=A.sub.0(t)+A.sub.(t)(50)

asymptotically stable, then, per Lemma 3, (t)[x.sub.r.sup.T(t),.sup.T(t)].sup.TL.sub.. Finally, since e(t)L.sub., x.sub.r(t)L.sub., and {tilde over (W)}(t)L.sub. ensure the boundedness of (25), it now follows from the Barbalat's lemma that lim.sub.t.fwdarw. e(t)=0.

[0092] Note that there are several methods in the literature for the analysis of the root locations of (49). The four most-used methods are TRACE-DDE, DDE-BIFTOOL, QPMR, and Lambert-W function. In essence, one provides the matrices A.sub.0 and A.sub. as well as the delay to these methods, which then return the numerical values of the rightmost root locations of (49). In some sense, these methods perform a nontrivial approximation with which they are able to identify the most relevant rootsthe rightmost roots. In the illustrative numerical example provided below, we employ TRACE-DDE readily available for download at https://users.dimi.uniud.it/dimitri.breda/research/software/.

Illustrative Example

[0093] Consider the longitudinal motion of a Boeing 747 airplane linearized at an altitude of 40 kft and a velocity of 774 ft/sec with the dynamics given by

{dot over (x)}(t)=A.sub.px(t)+B.sub.p(u(t)+W.sup.T(x(t))), x(0)=x.sub.0(51)

where x(t)=[x.sub.1(t),x.sub.2(t),x.sub.3(t),x.sub.4(t)].sup.T is the state vector. Note that (51) can be equivalently written as (4) with =I. Here, x.sub.1(t), x.sub.2(t), and x.sub.3(t) respectively represent the components of the velocity along the x, z and y axes of the aircraft with respect to the reference axes (in crad/sec), and x.sub.4(t) represents the pitch Euler angle of the aircraft body axis with respect to the reference axes (in crad). Recall that 0.01 radian=1 crad (centriradian). In addition, u(t) represents the elevator control input (in crad). Finally, W.sup.3 is an unknown weighting matrix and (x(t))=[1, x.sub.1(t), x.sub.2 (t)].sup.T is a known basis function. In the following simulations, we set W=[0.1 0.3 0.3]. The dynamical system given in (51) is assumed to be controlled using a model reference adaptive controller. In addition, the aircraft is assumed to be operated by a pilot whose Neal-Schmidt Model is given by

[00006]$\begin{array}{cc}{k}_p.Math.\frac{T_p.Math.S+1}{T_z.Math.S+1}.Math.e^{-.Math..Math.s}& \left(52\right)\end{array}$

Where k.sub.p is the positive scalar pilot gain, T.sub.p and T.sub.z are positive scalar time constants, and is the pilot reaction time delay. The values of the parameters used in the simulations are provided in Table 1.

[0094] To obtain the nominal controller K, a linear quadratic regulator (LQR) approach is utilized with the following objective function to be minimized

J(.)=.sub.0.sup.(x.sup.T(t)Qx(t)+u.sup.2(t))dt(53)

where Q is a positive-definite weighting matrix of appropriate dimension and is a positive weighting scalar. Notice that the framework developed above is not limited to a particular design method for the nominal controller. To this end, this task can be handled by a number of different ways. Here LQR is utilized for convenience reasons. In this setting, the selection of the weighing matrices, as expected, will affect the resulting nominal controller gain K in (13), which in turn will determine the reference model dynamics (18). In the following simulation studies, the effect of the weighting matrices, and thus the effects of reference model parameters on system stability are investigated for various values of pilot model parameters. To facilitate the analysis, reference model parameter variations is achieved mainly by manipulating the control penalty variable .

TABLE-US-00001 TABLE 1 T.sub.p 1 T.sub.z 5 0.5 A.sub.p [0.003 0.039 0 0.322; 0.065 0.319 7.740 0; 0.020 0.101 0.429 0; 0 0 1 0] B.sub.p [0.010; 0.1800; 1.160; 0] B.sub.p [0.0100 0.1800 1.1600 0].sup.T E.sub.p [0 0 0 1] E.sub.h [0 0 0 1 0] B.sub.r [0 0 0 0 1].sup.T Q diag([0 0 0 1 2.5])

[0095] Note that the purpose of the numerical examples provided in this section is to verify the theoretical stability predictions of the proposed framework. Therefore, the simulation results are created to present the stability/instability of the closed loop system without paying attention to enhanced transient response characteristics.

[0096] Delay-Independent Stability: LMI Approach:

[0097] We set k.sub.p= without loss of generality and investigate whether or not the closed loop is delay-independent stable. Specifically, we first use the LQR control designer in MATLAB with =1.0 to design K, which returns K=[0.0185, 0.0815, 1.5809, 2.7560, 1.5811]. Next the matrices A.sub.0 and A.sub. are constructed based on the information provided on Table 1. Assigning P and S as positive definite variables greater than 0.5I.sup.(n+n.sup..sup.)(n+n.sup..sup.) while imposing the negativity constraint in (32) as F<0.1I.sup.(n+n.sup..sup.)(n+n.sup..sup.), the YALMIP LMI optimization toolbox returns a feasible set of matrices P and S, indicating that the closed-loop system is delay-independent stable.

[0098] Delay-Independent Stability: Frequency-Domain Approach:

[0099] To be consistent with the previous subsection, we set k.sub.p= and =1.0 in the LQR optimization. Based on Corollary 2, since k.sub.p<1 and A.sub.r is Hurwitz, the necessary conditions for delay-independent stability are satisfied. Next, the sufficient conditions in Theorem 2 are to be checked simply by computing the metric in condition ii)-iii) of the theorem with respect to 0. We find out that the metric value starts at k.sub.p= when =0 (condition iii)) and decreases for larger 0 (condition ii)), remaining always less than 1. That is, the closed loop system will remain stable for any choice of delay . Keeping =1.0 but letting k.sub.p=0.95 has only negligible effects on K, again with the system remaining delay independent stable under the conditions of Theorem 2. On the other hand, selecting k.sub.p=1.05 violates the theorem and the system loses its delay-independent stability characteristics.

[0100] Delay-Dependent Stability: Effect of Control Penalty on System Stability for Different Pilot Reaction Time Delays:

[0101] To investigate the effects of the reference model parameter variations on the stability of the closed loop system, the control weight is manipulated by assigning values in the range [0, 50]. Then, the rightmost pole (RMP) of the system, whose characteristic equation is given by (49), is plotted against these values. This procedure is repeated for various pilot reaction time delays and the results are presented in FIG. 5.

[0102] FIG. 5 reveals several interesting results. First, it is shown that if the reference model dynamics is not chosen carefully with an appropriate value, then the human-in-the-loop adaptive control system can be indeed unstable. Second, it is seen that the closed loop system can be stable for small and large values of the parameter.sup.1 and be unstable in between. Third, it is observed that as the pilot reaction time delay increases, the unstable region of gets larger as indicated by RMP>0.

[0103] It is predicted in FIG. 5 that for =10, pilot reaction time delays =0.2 and =0.5 results in a stable and unstable system, respectively. Time domain tracking and control signal plots presented in FIG. 6 confirm this prediction. As noted earlier, the simulation results are employed to verify the theoretical stability predictions of the proposed method and therefore controllers are not tuned to obtain the best transient response. The investigation of the effect of the human-controller interactions on the transient response will be addressed in future research.

[0104] Delay-Dependent Stability: Effect of Control Penalty on System Stability for Different Values of Pilot Model Poles:

[0105] The poles of the pilot model (52) represent how fast the pilot responds to changes in the aircraft pitch angle, which can also be interpreted as pilot aggressiveness. In this section, the effect of pilot aggressiveness on system stability is investigated while assigning values to the control penalty from 0 to 50.

[0106] FIG. 7 depicts the effect of the pilot pole locations on the RMP. The zero location and the time-delay of the pilot model, are kept at their nominal values of 1 and 0.5, respectively. It is seen from the figure that, in general, stable-unstable-stable transition is observed for increasing values of and, as expected, higher values of poles, corresponding to faster pilot response, decrease the region of stability.

[0107] FIG. 8 depicts the tracking and control signal curves for two pilot model pole locations; that is, 0.175 and 0.2, when =10. As predicted in FIG. 7, the closed loop system remains stable when the pole is located at 0.175 and becomes unstable when the pole is at 0.2.

[0108] Delay-Dependent Stability: Effect of Control Penalty on System Stability from Different Values of Pilot Model Zeros:

[0109] In this section, the effect of zeros of the pilot transfer function (52) on system stability is investigated when control penalty pi takes values in the range [0,50]. The pole location and the time delay of the pilot transfer function are kept at their nominal values of 0.2 and 0.5, respectively. Changes in the zero location of the model can be interpreted as an adjustment to the lead nature of the pilot, which is related to pilot's anticipation capabilities.

[0110] As seen in FIG. 9, stable-unstable-stable transition structure still exists, in general, for increasing values. Furthermore, it is seen that when the pilot transfer function does not have a zero, a large region of instability arises. It is noted that for the given nominal values of the system parameters, no value of zero can make the system always stable, regardless of the p value, since delay-independence is determined only by the pilot's gain k.sub.p.

[0111] FIG. 10 presents tracking and control signal curves for pilot model zero locations 0.2 and 0.909, for the case when =1. As predicted in FIG. 9, the closed loop system becomes stable for the former and unstable for latter zero value.

[0112] Delay-Dependent Stability: Effect of Control Penalty on System Stability for Different Values of Pilot Model Gains:

[0113] The pilot gain in kp in (52) determines the intensity of the response that the pilot gives to the pitch angle deviations in the aircraft. In some sense, this gain also represents the aggressiveness of the pilot.

[0114] Stability properties of the pilot-in-the-loop system depending on the nominal control penalty and the pilot gain k.sub.p is presented in FIG. 11, where the RMP vs is plotted for certain values of k.sub.p. In these analyses, the pole and zero locations and time-delay of the pilot transfer function are kept at their nominal values of 0.2, 1, and 0.5, respectively. From the figure, stable-unstable-stable stability transition is once again observed for increasing values of . On the other hand, it is seen that, similar to the trend for the pilot pole location, as the pilot gain increases, the stability region shrinks. These results confirm the well-known adverse effects of high gain of pilots on system stability, such as pilot-induced oscillations.

[0115] It is predicted in FIG. 11 that the closed loop system will be stable for k.sub.p=4 and unstable for k.sub.p=5, when =10. This is confirmed by the results presented in FIG. 12, where time domain tracking and control signal curves are plotted for these gain values.

[0116] To summarize, the presented invention analyzed human-in-the-loop model reference adaptive control architectures and explicitly derived fundamental stability limit for both delay-independent and delay-dependent stability cases. Specifically, this stability limit results from the coupling between outer and inner loop architectures, where the outer loop portion includes the human dynamics modeled as a linear dynamical system with time delay and the inner loop portion includes the uncertain dynamical system, the reference model, the parameter adjustment mechanism, and the controller. We showed that when the given set of human model and reference model parameters satisfy this stability limit, the closed-loop system trajectories are guaranteed to be stable. The theoretical stability predictions of the proposed approach were verified via several simulation studies presented above. While the main focus of this invention was to reveal and compute stability limit of human-in-the-loop model reference adaptive control architectures, the effect of the controller design parameters on the transient response is also another important research direction that will be taken into consideration as a future research direction.

[0117] The techniques described above can be applied and adapted in various ways. For example, FIG. 13 illustrates a method for using the techniques discussed above to validate a MRACthat is, to determine whether a proposed MRAC will remain stable during operation. This method may be applied, for example, by an engineer while designing the MRAC and/or by a technician tuning the system for a particular user/use. The method of FIG. 13 may be fully or semi-automated by a computer system and results in a determination of whether the MRAC is acceptably stable or whether the MRAC should be adjusted or replaced in order to ensure stability.

[0118] The method begins by applying the operator model to the MRAC (step 1301), for example, as described above in reference to FIG. 4. The interactions of the operator/human model and the MRAC are then evaluated to determine whether the MRAC provides control-variable-independent stability (e.g., whether the MRAC enables the system operation to remain stable regardless of the time delay associated with a human operator's reaction to observed conditions) (step 1303). In implementations where the MRAC is being evaluated for time-delay-independent stability, the computer system may be configured to verify the delay-independent stability of the MRAC using the techniques outlined above in Theorem 1 and/or Theorem 2.

[0119] If the selected MRAC is confirmed to provide control-variable-independent stability for a selected control variable (e.g., time-delay-independent stability), then the MRAC is validated and the MRAC is used to control the vehicle system as illustrated in the example of FIG. 2 above (step 1305). However, if the MRAC does not provide control-variable-independent stability, a determined range of control variables is identified that provides stability (for example, using the techniques associated with Theorem 3, above) (step 1307). For example, the techniques described above can be used to determine a range of time delays, a range of pilot model poles, a range of pilot model zeros, and/or a range of pilot model gains that can be confirmed to provide stability (e.g., according to the techniques associated with Theorem 3 and the Illustrative Example described above). If the expected control variables for operation of the system (e.g., for the system itself, for a particular user, or for all users) fall within the determined range (step 1309), then the MRAC is validated as providing acceptable stability and is used to control the vehicle system as illustrated in the example of FIG. 2 above (step 1305). However, if the expected control variables for operation are outside of the range of variables that has been determined to provide for stability, then the MRAC must be either adjusted or replaced (step 1311).

[0120] In some implementations, the method of FIG. 13 is applied as a loop while adjusting particular parameters of a MRAC in order to tune the MRAC for stable operation. In other implementations, the method of FIG. 13 is applied to a plurality of different MRACs to determine which ones are acceptable (i.e., stable) and which ones are not.

[0121] The techniques and framework described above can also be adapted to be govern the operation of a vehicle using the controller 101. FIG. 14 illustrates one example of a method utilizing the methods for evaluating stability and determining stability limits in order to regulate the actuator commands provided by the control signals from the controller 101 in response to particular user input commands. The human/operator model is again applied to the MRAC (step 1401) and the control-variable-independent stability of the MRAC (e.g., the time-delay-independent stability) is evaluated (step 1403). If the MRAC is determined to be stable independent of the particular control variable in question (e.g., time-delay), then the MRAC is validated and ready to use in controlling the vehicle (e.g., as illustrated in FIG. 2). However, if the MRAC is determined to not be stable for all control variables, a range of control variables is determined at which stability is expected (step 1407). Based on this determined range of stable control variables, the controller 101 (e.g., in the system of FIGS. 1 and 2) limits the actuator commands to ensure that the control variable(s) remain with the determined range (step 1409).

[0122] For example, in reference to FIGS. 7 and 8 and the associated discussion above, a MRAC has been validated as stable for particular pilot model poles indicative of levels of pilot aggressiveness. The controller 101 may be configured to adjust/regulate the actuator commands to limit the aggressiveness of aerial maneuvers to ensure that the performance of the airplane remains within the acceptable/stable range of control variables.

[0123] Thus, the invention provides, among other things, systems and methods for validating and ensuring the stability of a control architecture. Various features and advantages of the invention are set forth in the following claims.