SYSTEM, METHOD, AND COMPUTER READABLE MEDIUM FOR AFFINE FORMATION MANEUVERING OF NONLINEAR MULTI-AGENT SYSTEMS WITH FAULT-TOLERANT SECURE OPTIMIZED BACKSTEPPING CONTROL USING REINFORCEMENT LEARNING

Abstract

A system, computer readable storage medium and method for controlling a trajectory of coordinated time-varying maneuvers of a geometric formation of unmanned vehicles is disclosed. The system includes unmanned vehicles, each configured with communication circuitry to communicate between the vehicles. A subset of the unmanned vehicles function as leader vehicles, with the remaining vehicles functioning as follower vehicles for leader-follower maneuvering. The system further includes an actuator suite configured to adjust the direction and orientation of each vehicle, a sensor suite for stabilization and navigation, and a flight controller for maintaining stable maneuvering, even in the presence of actuator faults and sensor deception attacks. Processing circuitry is configured with a reinforcement learning neural network that includes identifier, actor, and critic radial basis function neural networks to estimate movement, adjust control actions, and assess vehicle performance based on feedback signals, including corrupted signals from the sensor suite due to deception attacks.

Claims

1. A system for controlling a trajectory of coordinated time-varying maneuvers of a geometric formation of unmanned vehicles, the system comprising: a plurality of the unmanned vehicles, each having communication circuitry configured to communicate between each unmanned vehicle of the plurality of the unmanned vehicles, wherein a subset of the plurality of the unmanned vehicles are leader vehicles and remaining unmanned vehicles are follower vehicles, for leader-follower maneuvering of the geometric formation of the plurality of unmanned vehicles to target positions of the trajectory; an actuator suite to maintain and adjust direction and orientation of a respective unmanned vehicle, a sensor suite to stabilize and navigate the respective unmanned vehicle, a flight controller configured to send a control signal to the actuator suite and receive a feedback signal from the sensor suite, wherein the flight controller maintains stable maneuvering of the respective unmanned vehicle while the actuator suite is subject to an actuator fault and the sensor suite is subject to a deception attack; and processing circuitry configured to control the leader-follower maneuvering of the geometric formation of the plurality of unmanned vehicles, wherein the maneuvering of the plurality of unmanned vehicles is controlled with a reinforcement learning neural network that includes an identifier radial basis function neural network to estimate nonlinear movement of the plurality of unmanned vehicles, an actor radial basis function neural network to adjust direction and orientation of the respective unmanned vehicle by the respective actuator suite based on the estimated nonlinear movement, and a critic radial basis function neural network to assess the adjusted direction and orientation of each of the unmanned vehicles based on a feedback signal by the sensor suite, wherein the feedback signal includes corrupted signals from the sensor suite due to the deception attack.

2. The system of claim 1, wherein the processing circuitry is further configured to train the reinforcement learning neural network to learn a performance function that resets a preassigned convergence time whenever a target formation maneuver changes to maintain transient states of each leader-follower tracking error within a predefined range.

3. The system of claim 2, wherein the processing circuitry is further configured to control the leader vehicles to maneuver in a coordinated time-varying formation including one or more of shape, direction, rotation, scaling, and translation, wherein the follower vehicles track positions of the leader vehicles to achieve the target formation maneuver.

4. The system of claim 1, wherein the communication circuitry uses WiFi for communication with others of the plurality of unmanned vehicles.

5. The system of claim 2, wherein the processing circuitry is further configured for controlling the leader-follower maneuvering in an affine formation of the plurality of unmanned vehicles.

6. The system of claim 5, wherein the processing circuitry is further configured to train the reinforcement learning neural network to learn the performance function using performance-constrained backstepping control, wherein an initial control by the reinforcement learning neural network is used as an intermediate control input, and wherein optimal laws for the backstepping control are obtained from an approximate solution of a Hamilton-Jacobi-Bellman equation using the reinforcement learning.

7. The system of claim 1, wherein the plurality of the unmanned vehicles are unmanned aerial vehicles, each having a plurality of top-mounted rotors to move the unmanned aerial vehicle forward, backward, left, and right by adjusting speed of each rotor, wherein the plurality of top-mounted rotors are driven by the actuator suite.

8. The system of claim 1, wherein the plurality of unmanned vehicles are unmanned aerial vehicles, each having a single rotor and a plurality of movable fins, wherein the single rotor and the plurality of movable fins are driven by the actuator suite.

9. The system of claim 1, further comprising a ground-based controller configured with the processing circuitry, for centralized control of the leader-follower maneuvering of the geometric formation.

10. The system of claim 1, wherein the flight controller of each of the unmanned vehicles executes program instructions to obtain sensor suite data and adjust the unmanned vehicle positioning and rotor speeds based on the sensor suite data.

11. The system of claim 1, wherein the sensor suite in each of the plurality of unmanned vehicles includes a gyroscope, an accelerometer, and magnetometer.

12. The system of claim 1, wherein the sensor suit in the leader vehicles includes one or more sensors for detection of obstacles.

13. A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method for controlling a trajectory of coordinated time-varying maneuvers of a geometric formation of unmanned vehicles, the method comprising: controlling a leader-follower maneuvering of the geometric formation of a plurality of unmanned vehicles with a reinforcement learning neural network, wherein a subset of the unmanned vehicles are leader vehicles and remaining unmanned vehicles are follower vehicles, for the leader-follower maneuvering of the geometric formation of the plurality of unmanned vehicles to target positions of a trajectory, including estimating, by an identifier radial basis function neural network, nonlinear movement of the plurality of unmanned vehicles, adjusting, by an actor radial basis function neural network, direction and orientation of the respective unmanned vehicle by actions that control a respective actuator suite based on the estimated nonlinear movement, wherein the actuator suite is subject to an actuator fault, and assessing, by a critic radial basis function neural network, the adjusted direction and orientation of each of the unmanned vehicles based on a feedback signal from a sensor suite, wherein the feedback signal includes corrupted signals from the sensor suite due to a deception attack.

14. The computer-readable storage medium of claim 13, further comprising resetting, by a performance function, a preassigned convergence time whenever a target formation maneuver changes to maintain transient states of each leader-follower tracking error within a predefined range.

15. The computer-readable storage medium of claim 14, further comprising: controlling the leader vehicles to maneuver in a coordinated time-varying formation including one or more of shape, direction, rotation, scaling, and translation; and controlling the follower vehicles to track positions of the leader vehicles to achieve the target formation maneuver.

16. The computer-readable storage medium of claim 14, further comprising controlling the leader-follower maneuvering in an affine formation of the plurality of unmanned vehicles.

17. The computer-readable storage medium of claim 14, further comprising: executing the performance function using performance-constrained backstepping control, wherein an initial control by the reinforcement learning neural network is used as an intermediate control input, and wherein optimal laws for the backstepping control are obtained from an approximate solution of a Hamilton-Jacobi-Bellman equation using the reinforcement learning.

18. A method for controlling a trajectory of coordinated time-varying maneuvers of a geometric formation of unmanned vehicles, the method comprising: controlling a leader-follower maneuvering of the geometric formation of a plurality of unmanned vehicles with a reinforcement learning neural network, wherein a subset of the unmanned vehicles are leader vehicles and remaining unmanned vehicles are follower vehicles, for the leader-follower maneuvering of the geometric formation of the plurality of unmanned vehicles to target positions of a trajectory, including estimating, by an identifier radial basis function neural network, nonlinear movement of the plurality of unmanned vehicles, adjusting, by an actor radial basis function neural network, direction and orientation of the respective unmanned vehicle by actions that control a respective actuator suite based on the estimated nonlinear movement, wherein the actuator suite is subject to an actuator fault, and assessing, by a critic radial basis function neural network, the adjusted direction and orientation of each of the unmanned vehicles based on a feedback signal from a sensor suite, wherein the feedback signal includes corrupted signals from the sensor suite due to a deception attack.

19. The method of claim 18, further comprising resetting, by a performance function, a preassigned convergence time whenever a target formation maneuver changes to maintain transient states of each leader-follower tracking error within a predefined range.

20. The method of claim 19, further comprising: executing the performance function using performance-constrained backstepping control, wherein an initial control by the reinforcement learning neural network is used as an intermediate control input, and wherein optimal laws for the backstepping control are obtained from an approximate solution of a Hamilton-Jacobi-Bellman equation using the reinforcement learning.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] A more complete appreciation of this disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:

[0017] FIG. 1A illustrates an unmanned vehicle system, in accordance with an exemplary aspect of the disclosure;

[0018] FIG. 1B illustrates an unmanned vehicle system configured to control the trajectory of coordinated time-varying maneuvers of a geometric formation of unmanned vehicles, in accordance with an exemplary aspect of the disclosure;

[0019] FIG. 2A illustrates a quad-rotor autonomous aerial vehicle, in accordance with an exemplary aspect of the disclosure;

[0020] FIG. 2B illustrates a single rotor unmanned aerial vehicle, in accordance with the exemplary aspect of the disclosure;

[0021] FIG. 3A depicts the behavior of the tracking error over time when a sudden change in the target trajectory occurs at t=4 seconds and t=8 seconds, in accordance with the exemplary aspect of the disclosure;

[0022] FIG. 3B depicts the state trajectory p.sub.i tracking the target trajectory p.sup.i as sudden changes in amplitude occurs at t=4 seconds and t=8 seconds, in accordance with the exemplary aspect of the disclosure;

[0023] FIG. 4A depicts the behavior of the tracking error .sub.i under a sudden change in the target trajectory at t=4 seconds, in accordance with the exemplary aspect of the disclosure;

[0024] FIG. 4B depicts the state trajectory p.sub.i tracking the target trajectory p.sub.i when sudden changes in amplitude occur at t=4 seconds, in accordance with the exemplary aspect of the disclosure;

[0025] FIG. 5 illustrates the leader-follower nominal formation topology for a multi-agent system, in accordance with the exemplary aspect of the disclosure;

[0026] FIG. 6A depicts the behavior of the optimized virtual controllers of leaders and followers of UAV formation in x-axis, in accordance with the exemplary aspect of the disclosure;

[0027] FIG. 6B depicts the behavior of the optimized virtual controllers the of leaders and followers of UAV formation in y-axis, in accordance with the exemplary aspect of the disclosure;

[0028] FIG. 7A depicts the behavior of the control inputs the of leaders and followers of UAV formation in the x-axis, in accordance with the exemplary aspect of the disclosure;

[0029] FIG. 7B depicts the behavior of the control inputs the of leaders and followers of UAV formation in the y-axis, in accordance with the exemplary aspect of the disclosure;

[0030] FIG. 8A illustrates time-varying trajectories the of leaders and followers of UAV formation in the x-axis, in accordance with the exemplary aspect of the disclosure;

[0031] FIG. 8B illustrates time-varying trajectories the of leaders and followers of UAV formation in the y-axis, in accordance with the exemplary aspect of the disclosure;

[0032] FIG. 9A illustrates tracking errors of the of leaders and followers of UAV formation in the x-axis over time in accordance with the exemplary aspect of the disclosure;

[0033] FIG. 9B illustrates tracking errors of the of leaders and followers of UAV formation in the y-axis over time, in accordance with the exemplary aspect of the disclosure;

[0034] FIG. 10 illustrates the affine formation maneuvers of the leader-follower system under adversarial conditions, in accordance with the exemplary aspect of the disclosure;

[0035] FIG. 11A presents the time-varying trajectories of the leaders and followers of UAV formation in the x-axis, in accordance with the exemplary aspect of the disclosure;

[0036] FIG. 11B presents the time-varying trajectories of the leaders and followers of UAV formation in the y-axis, in accordance with the exemplary aspect of the disclosure;

[0037] FIG. 12 illustrates the affine formation maneuvers of the leader-follower system with another existing control method, in accordance with the exemplary aspect of the disclosure;

[0038] FIG. 13 is an illustration of a non-limiting example of details of mobile computing hardware used in the computing system, according to certain embodiments; and

[0039] FIG. 14 is an illustration of a non-limiting example of details of computer system for implementing the machine learning training and inference methods used in the computing system, according to certain embodiments.

DETAILED DESCRIPTION

[0040] In the drawings, like reference numerals designate identical or corresponding parts throughout the several views. Further, as used herein, the words a, an and the like generally carry a meaning of one or more, unless stated otherwise.

[0041] Furthermore, the terms approximately, approximate, about, and similar terms generally refer to ranges that include the identified value within a margin of 20%, 10%, or preferably 5%, and any values therebetween.

[0042] Aspects of the present disclosure are directed to a system and method for affine formation maneuver control of nonlinear multi-agent systems, focusing particularly on resilience against actuator faults and deception attacks while maintaining prescribed performance during leader-follower maneuvers. Known formation maneuver control methods for multi-agent systems are limited by their inability to effectively address security threats such as deception attacks and physical faults in actuators, which significantly affect the stability and performance of the overall system.

[0043] For purposes of this disclosure, a multi-agent system can include an unmanned vehicle, particularly an unmanned aerial vehicle (UAV), as well as fleet vehicles that move in a coordinated fashion. The unmanned vehicle is not limited to an aerial vehicle, but can be an unmanned vehicle that travels under water, or in outer space, in a coordinated fashion. Also, the unmanned vehicles can include a combination of different types of unmanned vehicles, only limited by a capability to communicate with each other and perform manoeuvring operations using an embedded control mechanism. Hereinafter, multi-agent systems will be referred to as unmanned vehicles.

[0044] The present disclosure provides a system for controlling the trajectory of coordinated, time-varying maneuvers of a geometric formation of unmanned vehicles. The system comprises multiple unmanned vehicles, including at least one leader vehicle and follower vehicles, wherein each unmanned vehicle is equipped with communication circuitry, an actuator suite, a sensor suite, a flight controller, and processing circuitry. The leader vehicles define the trajectory, while the follower vehicles track the leader to maintain the desired geometric formation. The flight controller is configured to maintain stable maneuvering, even when the actuator suite is subjected to faults and the sensor suite experiences a deception attack.

[0045] The system includes a reinforcement learning framework configured to estimate the nonlinear movement of the unmanned vehicles, adjust the movement based on the estimated dynamics, and evaluate the adjusted movement based on feedback from the sensor suite, which may include corrupted signals due to deception attacks. By implementing the reinforcement learning framework, the system dynamically adapts to disturbances, ensuring stable formation maneuvers under adverse conditions.

[0046] FIG. 1A illustrates a system 100A for controlling a trajectory of coordinated time-varying maneuvers of a geometric formation of unmanned vehicles. The system 100A includes multiple unmanned vehicles 102, depicted as unmanned vehicle 102-1, unmanned vehicle 102-2, unmanned vehicle 102-3, unmanned vehicle 102-4, and unmanned vehicle 102-N, representing the multiple unmanned vehicles, which operate in a leader-follower configuration. Each of the unmanned vehicles 102 is configured with communication circuitry to communicate among the multiple unmanned vehicles 102, wherein a subset of the unmanned vehicles are designated as leader vehicles, while the remaining unmanned vehicles are follower vehicles.

[0047] Unmanned vehicles, including unmanned aerial vehicles (UAVs), drones, or autonomous vehicles, are robotic systems that operate without human intervention. These vehicles can perform a wide range of tasks, from surveillance and reconnaissance to package delivery and agricultural monitoring. In the present system, the unmanned vehicles 102 are designed to operate in a coordinated geometric formation, which may involve a variety of maneuvers such as translation, rotation, scaling, and complex trajectory following. Each unmanned vehicle 102 can be equipped with multiple rotors, such as top-mounted rotors for aerial vehicles to move the unmanned aerial vehicle forward, backward, left, and right by adjusting speed of each rotor. The top-mounted rotors are driven by the actuator suite, or other propulsion systems depending on the type of vehicle and application. Examples include quadcopters, fixed-wing UAVs, or ground-based autonomous rovers.

[0048] The communication circuitry in each unmanned vehicle 102 is configured for maintaining coordinated movement. Communication circuitry refers to the hardware and software components that facilitate data exchange between vehicles, ensuring all vehicles are aware of each other's positions, speed, and trajectory plans. This data exchange is essential for leader-follower coordination, where leader vehicles define the trajectory, and follower vehicles maintain their relative positions. Different types of communication can be implemented, including wireless communication protocols such as Wi-Fi, Zigbee, LoRa, and 5G cellular networks. For example, Wi-Fi may be employed for short-range, high-speed communication, whereas LoRa may be used for long-range communication with lower data rates in environments with limited infrastructure.

[0049] The user devices 104 and 106, depicted in FIG. 1A as a smartphone and a laptop respectively, are used by operators to remotely control and manage the geometric formation of the unmanned vehicles 102. These user devices are equipped with software interfaces that allow operators to adjust flight parameters, initiate trajectory changes, and monitor the overall status of the formation. For instance, user device 104 may be used to provide real-time instructions for changing the formation shape during an operation, while user device 106 can be used to monitor system status, view sensor data, and upload mission plans. Examples of user devices may also include tablets, desktop computers, specialized remote controllers, and wearable devices such as smartwatches or augmented reality (AR) headsets, depending on the operational requirements. The user devices are also capable of communicating with the unmanned vehicles via wireless communication channels to send commands or receive status updates.

[0050] The system 100A also includes a database 108, which is responsible for storing flight-related data, including mission parameters, sensor data, actuator performance metrics, and historical flight records. This data can be used for post-flight analysis, performance optimization, and improving the resilience of the system against faults or cyber-attacks. For example, data stored in database 108 may be used to analyze the effects of actuator faults on vehicle stability or to assess the effectiveness of deception attack countermeasures. The database 108 can be implemented using various types of storage, including cloud-based storage solutions, local servers, or distributed storage systems. Different types of memory that can be utilized for storing the data include non-volatile memory such as solid-state drives (SSD), hard disk drives (HDD), flash memory, and magnetic tape storage, as well as volatile memory like random access memory (RAM) for temporary data processing.

[0051] Each unmanned vehicle 102 is equipped with an actuator suite and a sensor suite. The actuator suite is used to maintain and adjust the direction and orientation of the respective unmanned vehicle. For aerial vehicles, this may involve adjusting the speed of rotors to change altitude or direction, while for ground vehicles, it could involve controlling wheel motors or steering mechanisms. The sensor suite includes various sensors such as gyroscopes, accelerometers, and magnetometers, which provide real-time feedback on the vehicle's position, orientation, and movement. This sensor data is critical for maintaining stability and ensuring that each vehicle accurately follows the intended trajectory, particularly during complex maneuvers or when subjected to external disturbances.

[0052] The system 100A further employs processing circuitry configured for controlling the leader-follower maneuvering of the geometric formation of the plurality of unmanned vehicles 102. In a preferred embodiment, processing circuitry integrates a reinforcement learning neural network, comprising an identifier radial basis function neural network, an actor radial basis function neural network, and a critic radial basis function neural network. The identifier neural network estimates the nonlinear movement dynamics of the unmanned vehicles, while the actor neural network adjusts the direction and orientation of the vehicles based on these estimates. The critic neural network evaluates the performance of the adjustments using feedback from the sensor suite, which may include corrupted signals due to deception attacks. I should be understood, that the reinforcement learning framework can be configured using other types of neural networks or machine learning algorithms, in part depending on limitations of the hardware implementation. In some embodiments, reinforcement learning can be performed using an optimization control function.

[0053] The leader-follower coordination ensures that the unmanned vehicles 102 maintain a cohesive formation during maneuvers, even in the presence of faults or attacks. The processing circuitry's use of reinforcement learning allows the system to dynamically adapt to changing environmental conditions or unforeseen disturbances, such as actuator faults or sensor deception attacks, thereby maintaining the stability of the formation. For example, if a deception attack alters the sensor feedback for one of the follower vehicles, the critic neural network can identify inconsistencies in the sensor data, allowing the processing circuitry to adjust the control signals to maintain proper formation.

[0054] The communication between unmanned vehicles 102, user devices 104 and 106, and database 108, along with the actuator and sensor suites, provides a comprehensive control system for coordinated time-varying maneuvers of unmanned vehicles. The system 100A, through its combination of leader-follower maneuvering, communication circuitry, reinforcement learning, and adaptive processing, ensures effective control of the geometric formation of unmanned vehicles in dynamic environments.

[0055] FIG. 1B illustrates an unmanned vehicle system 100B. The system is configured to control the trajectory of coordinated time-varying maneuvers of a geometric formation of unmanned vehicles. A geometric formation refers to the specific spatial arrangement or configuration that a group of objects, in this context unmanned vehicles, maintain relative to each other while moving. In a geometric formation, the vehicles (e.g., UAVs) are organized to form predetermined shapes, such as a line, triangle, square, or more complex arrangements. The vehicles coordinate their movements to maintain this specific formation while navigating, which can help in tasks like area coverage, surveillance, and efficient navigation through obstacles. The formation is typically managed by leader-follower dynamics, where leader vehicles set the trajectory, and follower vehicles adjust their positions to maintain the intended configuration. This type of formation allows for coordinated, precise, and adaptable group behavior in complex environments.

[0056] The system 100B includes multiple unmanned vehicles 102. For purposes of this disclosure, the unmanned vehicles are unmanned aerial vehicles (UAV) that do not carry persons. As noted above, the unmanned vehicles can also include vehicles that travel under water or in outer space. Each of the unmanned vehicles is configured with a communication circuitry 110, a sensor suite 112, an actuator suite 114, a flight controller 116, and a processing circuitry 118. The communication circuitry 110 is configured to facilitate communication between each unmanned vehicle in the system. The communication circuitry 110 is essential for the leader-follower configuration, where leader vehicles coordinate the movements of follower vehicles to maintain the geometric formation and achieve target positions along the desired trajectory. Communication between the unmanned vehicles 102 enables the unmanned vehicles 102 to adjust their positions in real-time based on flight control signals and maintain formation cohesion. The communication circuitry 110 may use wireless communication protocols such as Wi-Fi for short-range, high-speed communication, or LoRa for long-range communication in low-data-rate environments. The ability of the communication circuitry 110 to switch between different protocols based on environmental conditions ensures uninterrupted communication even in challenging terrains or when infrastructure is limited. For example, in an urban environment, Wi-Fi might be used to enable high-speed data exchange, whereas in rural or remote areas, LoRa can provide long-range connectivity with lower power consumption. Other communication technologies that may be used include Zigbee for mesh networking or 5G/6G cellular networks for high bandwidth and low latency.

[0057] The sensor suite 112 is integrated into the unmanned vehicle 102-1 and is configured for stabilizing and navigating the unmanned vehicle 102. The sensor suite 112 includes one or more sensors, such as gyroscopes, accelerometers, and magnetometers that gather real-time data about the vehicle's orientation, speed, and position. Gyroscopes provide data on rotational movements, accelerometers measure linear acceleration, and magnetometers help determine the direction relative to Earth's magnetic field. This sensor data is utilized for maintaining stable flight, particularly in environments where the vehicle may be subjected to actuator faults or deception attacks on the sensor inputs. Additionally, the sensor suite 112 may include obstacle detection sensors, such as LiDAR or ultrasonic sensors, to detect and avoid obstacles in the flight path, enhancing the safety and reliability of the formation. For example, a LiDAR sensor can create a 3D map of the surrounding environment to help the vehicle navigate complex terrains, while ultrasonic sensors can detect obstacles at short ranges, making them suitable for low-speed maneuvers or landing operations. Visual sensors, such as cameras, can also be part of the sensor suite, providing image data that can be used for visual navigation, obstacle avoidance, or target recognition.

[0058] The actuator suite 114 in the unmanned vehicle 102-1 is configured to maintain and adjust direction and orientation of a respective unmanned vehicle 102. The actuator suite 114 adjusts the movement mechanisms of the vehicle, such as its rotors or movable fins, to execute the maneuvers for maintaining formation and trajectory. For aerial vehicles, the actuator suite 114 may include multiple top-mounted rotors, which can adjust the vehicle's altitude and direction by varying the rotor speeds. For example, if the unmanned vehicle 102 is a quadcopter, the actuator suite controls each of the four rotors independently to achieve the desired maneuver. In another example, a fixed-wing UAV may have an actuator suite 114 that includes movable control surfaces such as ailerons, rudders, and elevators to control the roll, yaw, and pitch of the vehicle. The actuator suite 114 works in conjunction with the flight controller 116 to ensure the unmanned vehicle 102 performs as expected, even when actuator faults occur, such as a rotor failure or reduced thrust due to mechanical issues. In the case of ground vehicles, the actuator suite 114 may include motors for wheel control and steering mechanisms to navigate across different terrains.

[0059] The flight controller 116 is configured for sending control signals to the actuator suite 114 and receiving feedback signals from the sensor suite 112. By configuring the flight controller 116, the unmanned vehicle 102 maintains stable maneuvering under challenging conditions, such as when the sensor suite 112 is subjected to deception attacks, or when the actuator suite 114 experiences malfunctions. The flight controller 116 processes real-time data from both the sensor suite 112 and the actuator suite 114 to generate appropriate control signals that ensure stable flight. For instance, if the sensor suite 112 detects a sudden deviation from the intended trajectory, the flight controller 116 immediately adjusts the actuator suite 114 to compensate for the deviation and bring the vehicle back on course. This closed-loop control mechanism is crucial for maintaining stability in dynamic and unpredictable environments. For example, if the vehicle encounters a sudden gust of wind that pushes it off course, the flight controller 116 adjusts the rotor speeds to counteract the disturbance and restore the intended trajectory. The flight controller 116 of each of the unmanned vehicles 102 executes program instructions to use sensor suite data to adjust the unmanned vehicle positioning and rotor speeds.

[0060] The processing circuitry 118 is configured for controlling the leader-follower maneuvering of the geometric formation. The processing circuitry 118 is configured to implement a reinforcement learning neural network that includes an identifier radial basis function neural network, an actor radial basis function neural network, and a critic radial basis function neural network. The identifier radial basis function neural network estimates nonlinear movements of the unmanned vehicle 102 by analyzing sensor data and predicting the vehicle's dynamic behavior. With such estimation, the processing circuitry 118 understands the current state of the vehicle and makes informed decisions regarding its movement. For example, the identifier neural network may predict the vehicle's response to a sudden change in wind speed, allowing the processing circuitry to pre-emptively adjust the control signals to maintain stability.

[0061] The actor radial basis function neural network adjusts the direction and orientation of the vehicle using the actuator suite 114, based on the estimated dynamics provided by the identifier radial basis function neural network. The actor radial basis function neural network determines the optimal control actions needed to maintain the desired trajectory and formation. For example, if the vehicle needs to change altitude to avoid an obstacle, the actor radial basis function neural network will compute the necessary rotor speed adjustments and send these commands to the actuator suite 114. In another instance, if the vehicle is part of a formation that needs to rotate to change its orientation, the actor radial basis function neural network calculates the adjustments required for each rotor or control surface to achieve the coordinated maneuver.

[0062] The critic radial basis function neural network assess the adjusted direction and orientation of each of the unmanned vehicles based on a feedback signal by the sensor suite 112. This feedback may include corrupted signals resulting from deception attacks, where adversaries inject false data to disrupt the vehicle's control system. The critic radial basis function neural network analyzes discrepancies between expected and actual sensor readings to determine if the vehicle is deviating from its intended path. If inconsistencies are detected, the critic radial basis function neural network prompts the processing circuitry to adjust the control strategy, ensuring the vehicle continues to perform as required despite interference or faults. For example, if the sensor data suggests that the vehicle is drifting off course due to a deception attack, the critic radial basis function neural network identifies the anomaly and instruct the processing circuitry to apply corrective measures, such as recalibrating the control inputs.

[0063] The reinforcement learning neural network integrated within the processing circuitry 118 allows the unmanned vehicle 102 to adapt its behavior dynamically in response to changing environmental conditions. Adaption includes learning a performance function that resets a preassigned convergence time whenever a target formation maneuver changes, thereby maintaining the transient states of each leader-follower tracking error within a predefined range. Such adaptability is particularly useful during complex maneuvers in a coordinated time-varying formation, such as changing the formation shape, direction, rotation, scaling, or translation, where the leader vehicles dictate the maneuver and the follower vehicles track their positions to achieve the target formation. For instance, if the leader vehicle initiates a scaling maneuver to expand the formation, the processing circuitry in each follower vehicle adjusts its movement to maintain the new distances between vehicles, ensuring the geometric formation is preserved.

[0064] The reinforcement learning neural network is also configured to learn a performance function through a performance-constrained backstepping control methodology. The reinforcement learning neural network initially generates an intermediate control input, which serves as a primary input for achieving stable control within the system in which backstepping control is applied.

[0065] In the backstepping control method, the system is based on the stability provided by the intermediate control input to systematically stabilize each outer control input progressively. Specifically, the reinforcement learning neural network assesses the stability requirements of the intermediate control, By following this structured stabilization process, the system progressively aligns each outer control input to achieve overall system stability and enhanced performance across the control layers.

[0066] In addition to the onboard components, the system 100B may further comprise a ground-based controller configured with the processing circuitry 118 for centralized control of the leader-follower maneuvering. The ground-based controller can be used to set mission objectives, define trajectories, and provide real-time monitoring of the unmanned vehicles 102. Communication between the ground-based controller and the unmanned vehicles 102 may be implemented using wireless communication protocols, ensuring that mission-critical data is exchanged reliably. For example, the ground-based controller may be a laptop equipped with specialized software that allows the operator to input mission parameters, view real-time telemetry data, and make adjustments to the flight plan as needed. In scenarios where multiple unmanned vehicles 102 are deployed, the ground-based controller can also serve as a coordination hub, ensuring that all vehicles operate in sync to achieve the mission goals.

[0067] The unmanned vehicle 102 is also capable of storing flight-related data, such as sensor readings, actuator performance, and mission parameters, in onboard memory. This memory may include both volatile memory, such as RAM, for temporary data processing, and non-volatile memory, such as flash storage or SSDs, for long-term data retention. This stored data can be used for post-mission analysis, system diagnostics, and improving future mission performance through machine learning techniques. For example, after a mission, the data stored in non-volatile memory can be analyzed to identify patterns in actuator performance, which can then be used to optimize future control strategies. Additionally, flight data can be uploaded to a central database for further analysis, enabling the development of predictive maintenance schedules to reduce the likelihood of actuator or sensor failures during missions.

[0068] With integration of communication circuitry 110, sensor suite 112, actuator suite 114, flight controller 116, and processing circuitry 118, the unmanned vehicle 102 operates autonomously and maintains the intended formation, even under challenging conditions such as actuator faults, sensor deception attacks, or environmental disturbances. The use of reinforcement learning and adaptive control strategies further enhances the resilience and reliability of the unmanned vehicle system 100B, making it suitable for a wide range of applications, including surveillance, reconnaissance, and cooperative missions in dynamic environments. For example, in a surveillance mission, the unmanned vehicle can autonomously navigate through an urban area, avoiding obstacles and maintaining formation with other vehicles, while continuously transmitting live video feed to the ground-based controller for real-time monitoring.

[0069] FIG. 2A depicts a quad-rotor UAV, illustrating a configuration used for executing coordinated maneuvers in a geometric formation. The UAV includes a main body that houses the primary electronics, sensors, and cameras. The UAV in FIG. 2A features a plurality of top-mounted rotors, which are part of the actuator suite. These rotors generate lift and thrust, allowing the UAV to maintain altitude, change direction, and stabilize during flight.

[0070] Each rotor is attached to a motor that forms part of the actuator suite. When actuated, the motors spin the rotors, producing thrust that enables the UAV to execute precise maneuvers and maintain stability. The motors are controlled via an electronic speed controller (ESC), which regulates the power delivered to each motor, ensuring smooth operation and consistent performance during flight. The rotors are essential for stabilizing the UAV in various flight conditions, including sudden shifts in wind or other environmental factors.

[0071] The UAV depicted in FIG. 2A operates within a system where it may function as either a leader vehicle or a follower vehicle, depending on its role in the formation. Communication circuitry within the UAV facilitates communication with other UAVs in the formation, exchanging flight control signals to maintain the designated trajectory. The UAV's sensor suite provides real-time feedback on its position, orientation, and environmental conditions, enabling effective navigation.

[0072] Additionally, the UAV is equipped with a camera module that captures high-resolution visual data used for navigation, obstacle detection, or mission-specific tasks, such as surveillance or terrain mapping. The captured video or image data can be transmitted in real-time to a ground control station or user device for further analysis and control. This camera is integral for autonomous operations in complex environments, enabling the UAV to detect obstacles, adjust its flight path, and execute mission tasks without manual intervention.

[0073] FIG. 2B illustrates a single rotor unmanned aerial vehicle, depicting a single-rotor UAV with multiple movable fins. The single rotor provides the necessary thrust to lift the UAV off the ground, while the movable fins are driven by the actuator suite that controls the UAV's direction and orientation during flight. With such configuration, the UAV to operate efficiently in different flight modes, whether hovering or moving along a designated flight path.

[0074] In this configuration, the sensor suite within the UAV, including devices such as gyroscopes, accelerometers, and magnetometers, continuously monitors the UAV's orientation, altitude, and motion. This sensor data is used to adjust the rotor speed and fin positions, ensuring stable flight and precise maneuvering, even in the presence of actuator faults or external disturbances.

[0075] FIG. 3A represents the behavior of the tracking error .sup.i over time when a sudden change in the target trajectory p.sub.i occurs at t=4 seconds and t=8 seconds.

[0076] The curve 302 represents the initial transient behavior of the tracking error .sub.i at t=0t=0t=0. Initially, the error .sub.i is at .sub.i(0)-.sub.i0=6. As time progresses, the error decays and converges to the steady-state value .sub.is=0.3 at t=T.sub.s=0.5 seconds. The black curve shows the system's ability to stabilize the error within the specified settling time, demonstrating the initial performance of the proposed function in regulating the transient state.

[0077] The curve 304 shows the system's performance when a sudden decrease in the target trajectory occurs at t=4 seconds. At this point, the finite-time performance function resets the transient state to .sub.i(0)=.sub.i0 and begins to decay toward the steady-state value .sub.is. The red curve demonstrates the decay of the transient state after the reset, with the system settling within Ts=0.5 seconds. This behavior shows the ability of the proposed function to confine the new transient state within a predefined range and ensure convergence within the prescribed finite time.

[0078] The curve 306 illustrates the system's behavior when another sudden change in the target trajectory occurs at t=8 seconds. Once again, the performance function resets the transient state to .sub.i(0)=.sup.i0, and the error decays over time. Then the system settles at the steady-state value .sub.is=0.3 within T.sub.s=0.5 seconds after the transient state begins. This resetting behavior is analysed for ensuring that the system can handle multiple transient states in quick succession while maintaining accuracy and stability.

[0079] FIG. 3B represents the state trajectory pi tracking the target trajectory p.sub.i as it undergoes sudden changes in amplitude at t=4 seconds and t=8 seconds. The behavior of the trajectory is represented by two curves, curve 308 and curve 310.

[0080] The curve 308 represents the state trajectory p.sub.i as it tracks the target trajectory p.sub.i when there is a sudden change in amplitude at t=4 seconds. At this point, the system resets the transient state, and the proposed finite-time performance function ensures that the trajectory p.sub.i converges to the new steady-state value within the predefined settling time T.sub.s=0.5 seconds. The curve shows how the system reacts to the first change in the target trajectory and how the performance function handles this transition effectively.

[0081] The curve 310 illustrates the state trajectory pi when the target trajectory pi undergoes another sudden change at t=8 seconds. The performance function resets the transient state at this point, and the trajectory converges to the new steady-state within the finite time T.sub.s=0.5 seconds.

[0082] In the embodiment described with respect to FIG. 1A to 1C, graph theory is applied for modeling the communication and coordination among multiple agents in a multi-agent system (MAS) in two-dimensional space. The graph theory provides the mathematical foundation for defining and achieving the desired leader-follower formation topology and ensuring that agents can track their target positions while remaining coordinated in the presence of disturbances, faults, or changes in formation configuration. An undirected graph =(, ) is utilized to illustrate the communication among agents, which comprises a set of nodes ={v.sub.1, . . . , v.sub.N} and a set of edges .Math.. The set of neighbors of an i.sup.th agent is indicated by ={j: (i, j)}. For an MAS of N agents in two-dimensional space whose positions are represented by p.sub.i, i=1, 2, . . . , N, the formation of the agents is denoted by (, p) with

[00001]$p={\left[\begin{array}{cccc}{p}_1^T& p_2^T& .Math.& p_N^T\end{array}\right]}^T{}^{2N}$

being the configuration of the formation.

[0083] Define (, q) as the nominal formation of the agents, where

[00002]$q=[\begin{array}{cc}{q}_1^T& {q_2^T.Math.q_N^T]}^T\end{array}={\left[\begin{array}{cc}{q}_l^T& q_f^T\end{array}\right]}^T{}^{2N}$

is the constant nominal configuration that the agents desired to form. The target configuration of the agents p with various maneuvers is defined by:

[00003]$\begin{array}{cc}{p}^{}\left(t\right)=\left[I_N.Math.A\left(t\right)\right]q+1_N.Math.b\left(t\right)& \left(1\right)\end{array}$

where A(t) realizes the time-varying scaling and rotation maneuvers of the whole formation while b(t) realizes the translation maneuver of the whole formation with respect to the nominal configuration q. Then, the target position

[00004]$p_i^{}{}^2$

of each agent from (1) is thus:

[00005]$\begin{array}{cc}{p}_i^{}\left(t\right)=A\left(t\right)q_i+b\left(t\right)& \left(2\right)\end{array}$

[0084] To realize the affine transformation of the formation (, p), a stress .sub.ij=.sub.ji is assigned to the corresponding edge (i, j). If the stresses applied to the configuration are balanced, then the following relation holds.

[00006]$\begin{array}{cc}{.Math.}_{j{}_i}{}_{\mathrm{ij}}\left(p_j-p_i\right)=0,i& \left(3\right)\end{array}$

[0085] Therefore, the stress is called equilibrium stress. The stress matrix is denoted by and is defined as:

[00007]${\left[\right]}_{\mathrm{ij}}=\{\begin{array}{cc}0,& ij,\left(i,j\right).Math.\\ -{}_{\mathrm{ij}},& ij,\left(i,j\right)\\ \underset{k{}_i}{.Math.}{}_{\mathrm{ik}},& i=j\end{array}.$

[0086] The configuration of the agents can be divided into leaders and followers as:

[00008]$\begin{array}{cc}p={\left[\begin{array}{cc}{p}_l^T& p_f^T\end{array}\right]}^T& \left(4\right)\end{array}$

where

[00009]$p_l={\left[p_1^T{p}_2^T.Math.p_{N_l}^T\right]}^T$

is the group of leaders and

[00010]$p_f={\left[p_{N_l+1}^T{p}_{N_l+2}^T.Math.p_{N_l+N_f}^T\right]}^T$

is the group of followers, N.sub.l and N.sub.f=NN.sub.l are the numbers of leaders and followers, respectively.

[0087] The stress matrix can be partitioned according to the followers and leaders groups as:

[00011]$\begin{array}{cc}\stackrel{}{}=.Math.I_2=\left[\begin{array}{cc}{}_{\mathrm{ll}}& {}_{\mathrm{lf}}\\ {}_{\mathrm{fl}}& {}_{\mathrm{ff}}\end{array}\right].Math.I_2=\left[\begin{array}{cc}{\stackrel{\_}{}}_{\mathrm{ll}}& {\stackrel{\_}{}}_{\mathrm{lf}}\\ {\stackrel{\_}{}}_{\mathrm{fl}}& {\stackrel{\_}{}}_{\mathrm{ff}}\end{array}\right].& \left(5\right)\end{array}$

where .sub.u, .sub.ff and .sub.fl.

[0088] Definition 1: The nominal formation (, q) with q affinely span is said to be localizable if the target position of the followers

[00012]$p_f^{}$

can be uniquely obtained from p.sub.l as follows:

[00013]$\begin{array}{cc}{p}_f^{}=-\left({}_{\mathrm{ff}}^{-1}{}_{\mathrm{fl}}.Math.I_d\right)p_l.& \left(6\right)\end{array}$

[0089] For Definition 1 to be valid, the nominal formation (, q) is set such that its stress matrix is positive semi-definite, and satisfies rank()=Nd1 and .sub.ff is positive definite, with d=2 being the dimension of the agents in the Euclidean space.

[0090] In a first assumption,

[00014]$p^{}={\left[\begin{array}{cc}{p}_f^T& p_l^T\end{array}\right]}^T$

is the vector of the target formation maneuvers of the agents, with

[00015]$p_l^{}={\left[p_1^T{p}_2^T.Math.p_{N_l}^T\right]}^T\mathrm{and}{p}_f^{}={\left[p_{N_l+1}^T{p}_{N_l+2}^T.Math.p_{N_l+N_f}^T\right]}^T.$

In this disclosure, it can be assumed that the leaders have obtained the desired formation maneuvers i.e.

[00016]$p_l=p_l^{}\mathrm{and}{u}_l=u_l^{}.$

Therefore, the control design for the leaders will be ignored. The control aim is now to realize

[00017]$p_f.fwdarw.p_f^{}$

as t.fwdarw.T.sub.s, with T.sub.s is being a finite-time settling time.

[0091] The leader-follower MAS under consideration consists of N.sub.l leaders and N.sub.f followers. The leaders are governed by the following dynamic equations:

[00018]$\begin{array}{cc}\{\begin{array}{c}{\stackrel{}{p}}_i=v_i\\ {\stackrel{}{v}}_i=u_i,i=1,2,.Math.,N_l\end{array}& \left(7\right)\end{array}$

where p.sub.i, v.sub.i and u.sub.i, i=1,2 . . . , N.sub.l represent the positions, velocities, and control inputs of the leaders, respectively.

[0092] The followers are described by second-order nonlinear dynamic equations as follows:

[00019]$\begin{array}{cc}\{\begin{array}{c}{\stackrel{.}{p}}_i=v_i\\ {\stackrel{.}{v}}_i=g_i\left(p_i,v_i\right){\stackrel{}{u}}_i+h_i\left(p_i,v_i\right)+{}_i\left(t\right)\\ {\stackrel{\_}{p}}_i=p_i+{}_i\left(t,p_i\right)\\ {\stackrel{\_}{v}}_i=v_i+{}_i\left(t,v_i\right),i=N_l+1,.Math.,N_l+N_f\end{array}& \left(8\right)\end{array}$

where p.sub.i and v.sub.i represent the positions and velocities of the agents, respective, p.sub.i and v.sub.i are the positions and velocities of the agents under deception attacks, .sub.i(t, p.sub.i) and .sub.i(t, v.sub.i) are state-dependent deception attacks satisfying .sub.i(t, p.sub.i)=w.sub.p.sub.ip.sub.i and .sub.i(t, v.sub.i)=w.sub.v.sub.iv.sub.i, with w.sub.p.sub.i and w.sub.v.sub.i are time-varying and unknown, u.sub.i, .sub.i represent the faulty actuator, h.sub.i(p.sub.i, v.sub.i) is an unknown continuous nonlinear function, g.sub.i.sup.2 is a diagonal matrix of unknown input gains, .sub.i(t) are the external disturbances.

[0093] In second assumption, the deception attack coefficients are .sub.p.sub.i=(1+w.sub.p.sub.i) and .sub.v.sub.i=(1+w.sub.v.sub.i), 1+w.sub.p.sub.i0 and 1+w.sub.v.sub.i0 throughout the duration of the attack. The second assumption means that the system will remain controllable during the attack.

[0094] Let .sub.p.sub.i=(1+w.sub.p.sub.i) and .sub.v.sub.i=(1+w.sub.p.sub.i). Then,

[00020]$\begin{array}{cc}\{\begin{array}{c}{\stackrel{\_}{p}}_i={}_{p_i}{p}_i\\ {\stackrel{\_}{v}}_i={}_{v_i}{v}_i\end{array},i=N_l+1,.Math.,N_l+N_f& \left(9\right)\end{array}$

[0095] The model of the actuator fault .sub.i is described as:

[00021]$\begin{array}{cc}{\stackrel{}{u}}_i=m_i{u}_i+{}_i& \left(10\right)\end{array}$

where u.sub.i is the control signal of each agent, and .sub.i=[.sub.i1, .sub.i2].sup.T is the vector of bias fault, m.sub.i=diag{m.sub.i1, m.sub.i2} is a diagonal matrix of control effectiveness factors, and 0<m.sub.i1, m.sub.i21.

[0096] Considering (9) and (10), (8) can be expressed as follows:

[00022]$\begin{array}{cc}\{\begin{array}{cc}{\stackrel{.}{\stackrel{\_}{p}}}_i=& {}_{\mathrm{pi}}{}_{\mathrm{vi}}^{-1}{\stackrel{\_}{v}}_i+{\stackrel{.}{}}_{\mathrm{pi}}{}_{\mathrm{pi}}^{-1}{\stackrel{\_}{p}}_i\\ {\stackrel{.}{\stackrel{\_}{v}}}_i=& {}_{v_i}{h}_i+{}_{v_i}{m}_i{g}_i{u}_i+{}_{v_i}{g}_i{}_i+{}_{v_i}{}_i;+{\stackrel{.}{}}_{v_i}{}_{v_i}^{-1}{\stackrel{\_}{v}}_i,i=N_l+1,.Math.,N_l+N_f\end{array}& \left(11\right)\end{array}$

[0097] Defining:

[00023]$p_f={\left[{\stackrel{}{p}}_{N_l+1}^T{\stackrel{}{p}}_{N_l+2}^T.Math.{\stackrel{}{p}}_{N_l+N_f}^T\right]}^T$$v_f={\left[{\stackrel{}{v}}_{N_l+1}^T{\stackrel{}{v}}_{N_l+2}^T.Math.{\stackrel{\_}{v}}_{N_l+N_f}^T\right]}^T$$h={\left[h_{N_l+1}^T{h}_{N_l+2}^T.Math.h_{N_l+N_f}^T\right]}^T$$={\left[{}_{N_l+1}^T{}_{N_l+2}^T.Math.{}_{N_l+N_f}^T\right]}^T$$={\left[{}_{N_l+1}^T{}_{N_l+2}^T.Math.{}_{N_l+N_f}^T\right]}^T$${}_p=\mathrm{diag}\left\{{}_{p_{N_l+1}}{}_{p_{N_l+2}}.Math.{}_{p_{N_l+N_f}}\right\}$${}_v=\mathrm{diag}\left\{{}_{v_{N_l+1}}{}_{v_{N_l+2}}.Math.{}_{v_{N_l+N_f}}\right\}$$m=\mathrm{diag}\left\{m_{N_l+1}{m}_{N_l+2}.Math.m_{N_l+N_f}\right\}$$g=\mathrm{diag}\left\{g_{N_l+1}{g}_{N_l+2}.Math.g_{N_l+N_f}\right\}$

[0098] The global form of (11) is thus:

[00024]$\begin{array}{cc}\{\begin{array}{c}{\stackrel{.}{p}}_f={}_p{}_v^{-1}{v}_f+{\stackrel{.}{}}_p{}_p^{-1}{p}_f\\ {\stackrel{.}{v}}_f={}_v\mathrm{mgu}+{}_v+{}_{v}h+{}_v+{\stackrel{.}{}}_v{}_v^{-1}{v}_f\end{array}& \left(12\right)\end{array}$

[0099] Define the following error variables for the followers:

[00025]$\begin{array}{cc}{}_i=E_{j=1}^N{}_{\mathrm{ij}}\left(p_i-p_j\right),i=N_l+1,.Math.,N_l+N_f& \left(13\right)\end{array}$ [0100] and the compact form of (13) is given by:

[00026]$\begin{array}{cc}\begin{array}{c}{}_f={\stackrel{\_}{}}_{\mathrm{ff}}{p}_f+{\stackrel{\_}{}}_{\mathrm{fl}}{P}_l={\stackrel{\_}{}}_{\mathrm{ff}}\left[p_f-\left(-{}_{\mathrm{ff}}^{-1}{}_{\mathrm{fl}}.Math.I_d\right)P_l\right]\\ {\stackrel{\_}{}}_{\mathrm{ff}}\left[p_f-p_f^{}\right]\end{array}& \left(14\right)\end{array}$

where

[00027]${}_f={\left[{}_I^T{}_2^T.Math.{}_{N_f}^T\right]}^T$

and

[00028]$p_f^{}=\left(-{}_{\mathrm{ff}}^{-1}{}_{\mathrm{fl}}.Math.I_d\right)p_l$

as in (6).

[0101] Neural network approximations, specifically Radial Basis Function Neural Networks (RBFNNs), are implemented to manage the nonlinear functions that emerge from the design of the disclosed control strategy. In real-world multi-agent systems, it is challenging to represent complex nonlinear dynamics in closed-form equations suitable for real-time computation. RBFNNs approximate these functions, enabling accurate and efficient control.

[0102] The smooth and continuous nonlinear functions derived from the design of the disclosed control strategy are approximated with radial basis function neural networks (RBFNNs) as follows:

[00029]$\begin{array}{cc}{}_i=W_i^T{}_i\left(X_i\right)+{}_i\left(X_i\right)& \left(15\right)\end{array}$

where W.sub.i=[w.sub.i1 w.sub.i2 . . . w.sub.im].sup.T is the weight vector, m is the number of nodes, .sub.i(X.sub.i) is the RBFNN reconstruction error, error satisfying .sub.i(X.sub.i).sub.l, .sub.i(X.sub.i)=[.sub.i1(X.sub.i).sub.i2(X.sub.i) . . . .sub.im(X.sub.i)].sup.T is the vector of basis function with:

[00030]$\begin{array}{cc}{}_{\mathrm{ik}}\left(X_i\right)=\exp \left(-\frac{{.Math.X_i-{}_{1k}.Math.}^2}{{}_{2k}^2}\right),k=1,2,.Math.,m& \left(16\right)\end{array}$

where .sub.1k is the receptive center of the Gaussian function, .sub.2k is the Gaussian function width.

[0103] The specified performance is attained by confining each leader-follower tracking error .sub.i, i=N.sub.l+1, . . . , N.sub.l+N.sub.f within the following preassigned boundary.

[00031]$\begin{array}{cc}-\underset{\_}{{}_i}{}_i\left(t\right)<{}_i<{\stackrel{}{}}_i{}_i\left(t\right)& \left(17\right)\end{array}$

where .sub.i>0 and .sub.i are design constants, .sub.i(t):.fwdarw. is the prescribed performance function characterized as a positive, smooth, and decreasing function.

[0104] In second definition, a smooth function .sub.i(t): .fwdarw. is said to be a finite time prescribed performance function if it is characterized by a) .sub.i(t)>0; b) {dot over ()}.sub.i(t)<0; c) lim.sub.t.fwdarw.t.sub.s.sub.i(t)=.sub.is>0; and d) For any tt.sub.s, .sub.i(t)=.sub.is.

[0105] The finite time prescribed performance function proposed in this study is defined by:

[00032]$\begin{array}{cc}{}_i\left(t\right)=\{\begin{array}{cc}{}_{i0,}& t=t_n\\ & \left({}_{i0}-{}_{\mathrm{is}}\right)\exp \left(\frac{-K_i\left(t-t_n\right)}{T_s}\right)+{}_{\mathrm{is}},0<t-t_n<T_s\\ {}_{is,}& t-t_n{T}_s\\ & i=N_l+1,.Math.,N_l+N_f\end{array}& \left(18\right)\end{array}$ [0106] where .sub.i0>0, .sub.is>0, and .sub.i>0 are appropriately selected constants, T.sub.s is the preassigned finite settling time, and t.sub.n is the time at which the agents undergo new transient states due to the change in target formation maneuver.

[0107] The inequality (17) can be transformed into an equality form using the following error transformation:

[00033]$\begin{array}{cc}{e}_i=\left(\frac{{}_i}{{}_i}\right)& \left(19\right)\end{array}$

where e.sub.i is the transformed error, () is a smooth function and :(.sub.i, .sub.i).fwdarw.(, ). The transformation function ()isexpressedas:

[00034]$\begin{array}{cc}\left(\frac{{}_i}{{}_i}\right)=\frac{1}{2}\ln \left(\frac{\underset{\_}{{}_i}+\frac{{}_i}{{}_i}}{{\stackrel{\_}{}}_i-\frac{{}_i}{{}_i}}\right)& \left(20\right)\end{array}$

[0108] The time derivative of the transformed error yields

[00035]$\begin{array}{cc}{\stackrel{.}{e}}_i={}_i\left({\stackrel{}{}}_i-\frac{{\stackrel{.}{}}_i}{{}_i}{}_i\right)i=N_l+1,.Math.,N_l+N_f& \left(21\right)\end{array}$

where

[00036]${}_i=\frac{1}{{}_i}\frac{{}_i}{\left({}_i/{}_i\right)}.$

[0109] To better demonstrate the important features of the performance function, a simple example is presented in FIG. 1. This figure shows a state trajectory p.sub.i tracking a target trajectory with sudden change in amplitudes at t.sub.n=4 s and t.sub.n=8 s. The trajectory p.sub.i experienced new transient states at the time t=t.sub.n=4 s and t=t.sub.n=8 s. It is required that p.sub.i settles at T.sub.s seconds after every new transient state. The finite time performance function (18) can be reset and the new transient states of the tracking error E; can be constrained whenever the amplitude of the target trajectory changes. [0110] Initially, t=t.sub.n=0 and .sub.i(0)=.sub.i0=6. It is expected that as t grows, .sub.i(t) converges to .sub.is=0.3 at t=T.sub.s=0.5s. Then, .sub.i(t)=.sub.is for tT.sub.s. [0111] When there is a sudden decrease in the target trajectory at t=t.sub.n=4 s, tt.sub.n=0 and the finite time performance function is reset to .sub.i(0)=.sub.i0 to confine the new transient state within the predefined range. As t grows (i.e t4>0), the finite time performance function decays and settles at t4=0.5s. For t40.5s, .sub.i(t)=.sub.is. [0112] On the other hand, when the target trajectory suddenly increases at t=t.sub.n=8 s, tt.sub.n=0. At this moment, the finite time performance function is reset such that .sub.i(0)=.sub.i0 and begins decaying for t8>0 until t8=0.5s where it settles at .sub.i(t)=.sub.is. Subsequently, for t80.5s, .sub.i(t)=.sub.is.

[00037]$\begin{array}{cc}{}_i\left(t\right)=\left({}_{i0}-{}_i\right)e^{-{}_{i}t}+{}_i& \left(22\right)\end{array}$$\begin{array}{cc}{}_i\left(t\right)={}_{i0}\exp \left(\frac{-t}{{}_i}\right)+{}_i& \left(23\right)\end{array}$$\begin{array}{cc}{}_i\left(t\right)=\coth \left({}_{1i}t+{}_{i2}\right)-1+{}_i& \left(24\right)\end{array}$$\begin{array}{cc}{}_i\left(t\right)=\{\begin{array}{cc}\left({}_{i0}-{}_{\mathrm{is}}\right)\exp \left(\frac{-{}_i{T}_{s}t}{t-T_s}\right)+{}_{\mathrm{is}},& t<T_s\\ {}_{\mathrm{is}},& t{T}_s\end{array}& \left(25\right)\end{array}$$\begin{array}{cc}{}_i\left(t\right)=\{\begin{array}{cc}{}_{i0}\exp \left({}_i\left(1-\frac{T_s}{T_s-t}\right)\right){}_i+{}_{\mathrm{is}},& t<T_s\\ {}_i=\frac{1}{2}\cos \left(\frac{t}{T_s}\right)+\frac{1}{2}& \\ {}_{\mathrm{is}},& t{T}_s\end{array}& \left(26\right)\end{array}$$\begin{array}{cc}{}_i\left(t\right)=\{\begin{array}{cc}\left(1-\frac{t}{T_s}\right){\exp }^{1-\exp \left(\frac{T_s}{T_s-t}\right)}+{}_{\mathrm{is}},& t<T_s\\ {}_{\mathrm{is}},& t{T}_s\end{array}& \left(27\right)\end{array}$$\begin{array}{cc}{}_i\left(t\right)=\{\begin{array}{cc}\left({}_{i0}-\frac{t}{{}_i{T}_s}\right)& \\ \exp \left(1-\tan \left(\frac{t}{T_s-t}\right)\right)+{}_{\mathrm{is}},& t<T_s\\ {}_{\mathrm{is}},& t{T}_s\end{array}& \left(28\right)\end{array}$$\begin{array}{cc}{}_i\left(t\right)=\{\begin{array}{cc}\left({}_{i0}-{}_{\mathrm{is}}\right){\left(1-\frac{t}{T_s}\right)}^{{}_i}+{}_{\mathrm{is}},& t<T_s\\ {}_{\mathrm{is}},& t{T}_s\end{array}& \left(29\right)\end{array}$

[0113] FIG. 4A depicts the behavior 400A of the tracking error .sub.i under a sudden change in the target trajectory at t=4 seconds. The behavior of the system is represented by curve 402, curve 404, and curve 406.

[0114] Curve 402 represents the initial behavior of the tracking error .sub.i when the performance function is unable to reset the transient state after a sudden change in the target trajectory. Initially, the tracking error .sub.i follows the predefined trajectory, but at t=4 seconds, the system becomes unstable due to the inability of the performance function to adjust to the new transient state. Curve 402 shows the system's failure to maintain stability when faced with sudden reference signal changes.

[0115] Curve 404 shows the tracking error .sub.i experiencing instability after the sudden change at t=4 seconds. The lack of a reset mechanism in the existing performance functions prevents the system from controlling the new transient state, resulting in an erratic response. Curve 404 shows that the tracking error fails to converge to the desired steady-state after the target trajectory shifts, reinforcing the need for a modified performance function that can reset and accommodate multiple transient states.

[0116] Curve 406 further demonstrates the inability of the system to handle the sudden change in the target trajectory. The error remains outside the desired range, and the system cannot stabilize the transient state, leading to prolonged instability. Curve 406 indicates the limitations of existing performance functions (22)-(29) in handling sudden shifts in the target trajectory and tracking errors over time.

[0117] FIG. 4B illustrates an observation 400B of the state trajectory p.sub.i tracking the target trajectory p.sub.i when sudden changes in amplitude occur at t=4 seconds. The performance of the state trajectory is depicted by curve 408 and curve 410.

[0118] Curve 408 shows the behavior of the state trajectory pi as it initially tracks the target trajectory p.sub.i without issues. However, at t=4 seconds, the target trajectory p.sub.i changes abruptly. Due to the limitations of performance functions (22)-(29), the system cannot reset the transient state of the tracking error, leading to instability in the state trajectory. The inability to reset and stabilize is clearly demonstrated in this curve, as the trajectory pi becomes erratic after the sudden change.

[0119] Curve 410 represents the expected trajectory of p.sub.i had the system been able to properly handle the sudden change in the target trajectory at t=4 seconds. Curve 410 shows that the state trajectory would have stabilized if the performance function could reset to accommodate the new transient state. However, the existing performance functions fail to achieve this reset, resulting in divergence from the target trajectory. The divergence shows the core limitation of performance functions (22)-(29) in applications where mobile agents experience multiple transient states.

[0120] Optimal backstepping control laws are obtained from the approximate solution of the Hamilton-Jacobi-Bellman equation using reinforcement learning under the identifier-actor-critic neural networks. Subsequently, reinforcement learning-based optimized secure backstepping control can be realized for nonlinear leader-follower multi-agent systems with deception attacks and actuator faults to be resilient and realize various affine formation maneuvers. The objective of the controller is to ensure that the closed-loop system is bounded despite the actuator and the deceptive state signals injected by cyber-attackers.

Defining:

[00038]$e_f={\left[e_1^T{e}_2^T.Math.e_{N_f}^T\right]}^T$${}_f=\mathrm{diag}\left\{{\stackrel{.}{}}_1/{}_1{\stackrel{.}{}}_2/{}_2.Math.{\stackrel{.}{}}_{N_f}/{}_{N_f}\right\}$${}_f=\left\{{}_1{}_2.Math.{}_{N_f}\right\}$

The global form of (21) is given by:

[00039]$\begin{array}{cc}\begin{array}{c}{\stackrel{.}{e}}_f={}_f\left({\stackrel{.}{}}_f-{}_f{}_f\right)\\ ={}_f{\stackrel{\_}{}}_{\mathrm{ff}}\left({}_p{}_v^{-1}{v}_f+{\stackrel{.}{}}_p{}_p^{-1}{p}_f-{\stackrel{.}{p}}_f^{}\right)-{}_f{}_f{}_f\end{array}& \left(30\right)\end{array}$

According to the backstepping approach, v.sub.f will be treated as the intermediate control input. Let .sub.f be the virtual control. Then, define the error z.sub.f=v.sub.f.sub.f, and its time-derivative is calculated as:

[00040]$\begin{array}{cc}\begin{array}{c}{\stackrel{.}{z}}_f={\stackrel{.}{v}}_f-{\stackrel{.}{}}_f\\ ={}_v\mathrm{mgu}+{}_{v}g+{}_{v}h+{}_v+{\stackrel{.}{}}_v{}_v^{-1}{v}_f-{\stackrel{.}{}}_f\end{array}& \left(31\right)\end{array}$

[0121] For the purpose of a control scheme, the error dynamics (30) and (31) are transformed as follows:

[00041]$\begin{array}{cc}\begin{array}{c}{\stackrel{}{e}}_f=\left(p_f,v_f,{}_p,{}_v\right)+v_f\\ =\left(p_f,v_f,{}_p,{}_v\right)+z_f+{}_f\end{array}& \left(32\right)\end{array}$$\begin{array}{cc}{\stackrel{.}{z}}_f=F\left(p_f,v_f,{}_p,{}_v,u\right)+u-{\stackrel{.}{}}_f& \left(33\right)\end{array}$

where

[00042]$\left(p_f,v_f,{}_p,{}_v\right)={}_f{\stackrel{\_}{}}_{\mathrm{ff}}\left({}_p{}_v^{-1}{v}_f+{\stackrel{.}{}}_p{}_p^{-1}{p}_f-{\stackrel{.}{p}}_f^{}\right)-{}_f{}_f{}_f-v_f$$\mathrm{and}$$F\left(p_f,v_f,{}_p,{}_v,u\right)={}_v\mathrm{mgu}+{}_{v}g+{}_{v}h+{}_v+{\stackrel{.}{}}_v{}_v^{-1}{v}_f-u.$

A performance index function associated with es and as is defined as follows:

[00043]$\begin{array}{cc}{I}_p\left(e_f\right)={}_t^{}{r}_f\left(e_f\left(\right),{}_f\left(\right)\right)d& \left(34\right)\end{array}$

where

[00044]$r_f\left(e_f,{}_f\right)=e_f^T{e}_f+{}_f^T{}_f$

is the cost function. The optimal performance index I*(e.sub.f) associated with the optimal virtual controller

[00045]${}_f^{*}$

is given by:

[00046]$\begin{array}{cc}\begin{array}{c}{I}_p^{*}\left(e_f\right)=\underset{{}_f}{\min }\left({}_t^{}{r}_f\left(e_f\left(\right),{}_f\left(\right)\right)d\right)\\ ={}_t^{}\left(e_f^T{e}_f+{}_f^{*T}{}_f^{*}\right)d\end{array}& \left(35\right)\end{array}$

where is the set of admissible control inputs. From (35), the following Hamilton-Jacobi-Bellman (HJB) equation can be derived:

[00047]$\begin{array}{cc}{H}_p\left(e_f,{}_f^{*},\frac{I_p^{*}}{e_f}\right)=r_f\left(e_f,{}_f^{*}\right)+\frac{I_p^{*}}{e_f}{\stackrel{.}{e}}_f=e_f^T{e}_f+{}_f^{*T}{}_f^{*}+\frac{I_p^{*}}{e_f}\left(z_f+{}_f+-p_f^{}\right)=0& \left(36\right)\end{array}$

where

[00048]$\frac{I_p^{*}}{e_f}$

is the gradient of

[00049]$I_p^{*}$

along e.sub.f. Suppose the solution of (36) exists and is unique, the optimal control policy

[00050]${}_f^{*}$

can be achieved by computing

[00051]$H_p\left(e_f,{}_f^{*},\frac{I_p^{*}}{e_f}\right)/{}_f^{*}=0$

[00052]$\begin{array}{cc}{}_f^{*}=-\frac{1}{2}\frac{I_p^{*}\left(e_f\right)}{e_f}& \left(37\right)\end{array}$

Considering (36) and (37), it is clear that

[00053]$\frac{I_p^{*}}{e_f}$

is required to obtain the solution of (36). Nevertheless, due to the unknown deception attack signals and strong nonlinearities, it is impossible to solve (36). To achieve the control objective, RL actor-critic framework is employed to obtain the approximate solution online.
By using some mathematical manipulations,

[00054]$\frac{I_p^{*}}{e_f}$

can be expressed as follows:

[00055]$\begin{array}{cc}\frac{I_p^{*}\left(e_f\right)}{e_f}=2K_1{e}_f+2+I_p^0& \left(38\right)\end{array}$

where

[00056]$I_p^0=-2K_1{e}_f-2+\frac{I_p^{*}\left(e_f\right)}{e_f}.$

Inserting (38) into (37) gives:

[00057]$\begin{array}{cc}{}_f^{*}=-K_p{e}_f--\frac{1}{2}{I}_p^0& \left(39\right)\end{array}$

Since the continuous functions Y and

[00058]$I_p^0$

are unknown, RBFNNs are employed to approximate them in the following form:

[00059]$\begin{array}{cc}=W_{}^{*T}{}_{}\left(X_{}\right)+{}_{}\left(X_{}\right)& \left(40\right)\end{array}$$\begin{array}{cc}{I}_p^0=W_{I_p}^{*T}{}_{I_p}\left(X_{I_p}\right)+{}_{I_p}\left(X_{I_p}\right)& \left(41\right)\end{array}$

where

[00060]$W_{}^{*}\mathrm{and}{W}_{I_p}^{*}$

are the ideal weights, .sub.(X.sub.) and .sub.lp(X.sub.lp) are the vectors basis functions, and .sub.(X.sub.) and .sub.lp(X.sub.lp) are the RBFNN approximation errors.
Substituting (40) and (41) into (38) and (39), respectively:

[00061]$\begin{array}{cc}\frac{I_p^{*}\left(e_f\right)}{e_f}=2K_1{e}_f+2W_{}^{*T}{}_{}\left(X_{}\right)+W_{I_p}^{*T}{}_{I_p}\left(X_{I_p}\right)+{}_p& \left(42\right)\end{array}$$\begin{array}{cc}{}_f^{*}=-K_p{e}_f-W_{}^{*T}{}_{}\left(X_{}\right)-\frac{1}{2}{W}_{I_p}^{*T}{}_{I_p}\left(X_{I_p}\right)+{}_p& \left(43\right)\end{array}$

where .sub.p=.sub.(X.sub.)+.sub.lp(X.sub.lp).
The optimal control input (43) cannot be used because of the unknown weights

[00062]$W_{}^{*}\mathrm{and}{W}_{I_p}^{*}.$

The identifier RBFNN is used to estimate the unknown deceptive signals as:

[00063]$\begin{array}{cc}\hat{}={\hat{W}}_{}^T{}_{}\left(X_{}\right)& \left(44\right)\end{array}$

where .sub. is the weight of the identifier, and {circumflex over ()} is the output of the identifier. The RBFNN weight of the identifier

[00064]${\hat{W}}_{}^T$

is updated online by:

[00065]$\begin{array}{cc}{\stackrel{.}{\hat{W}}}_{}={}_p\left({}_{}\left(X_{}\right)e_f-{}_p{\hat{W}}_{}\right)& \left(45\right)\end{array}$

where .sub.p is a positive-definite matrix and .sub.p>0 is a small constant.
To obtain the optimized controller, the actor, critic, and identifier framework is developed:

[00066]$\begin{array}{cc}\frac{{\hat{I}}_p\left(e_f\right)}{e_f}=2K_1{e}_f+2{\hat{W}}_{}^{*T}{}_{}\left(X_{}\right)+{\hat{W}}_{c_p}^{*T}{}_{I_p}\left(X_{I_p}\right)& \left(46\right)\end{array}$$\begin{array}{cc}{\hat{}}_f^{*}=-K_p{e}_f-{\hat{W}}_{}^{*T}{}_{}\left(X_{}\right)-\frac{1}{2}{\hat{W}}_{a_p}^{*T}{}_{I_p}\left(X_{I_p}\right)& \left(47\right)\end{array}$

where

[00067]$\frac{{\hat{I}}_p\left(e_f\right)}{e_f}$

is the estimate of

[00068]$\frac{I_p\left(e_f\right)}{e_f},{\hat{W}}_{c_p}^{*}\mathrm{and}{\hat{W}}_{a_p}^{*}$

are the estimates of the critic and actor weights, respectively.
Adding (46) and (47) into (36), the approximated HJB equation is derived as:

[00069]$\begin{array}{cc}{H}_p\left(e_f,{\hat{}}_f^{*},\frac{{\hat{I}}_p}{e_f}\right)=& \left(48\right)\end{array}$$e_f^T{e}_f+{\left(-K_p{e}_f-{\hat{W}}_{}^{*T}{}_{}\left(X_{}\right)-\frac{1}{2}{\hat{W}}_{a_p}^{*T}{}_{I_p}\left(X_{I_p}\right)\right)}^T\left(-K_p{e}_f-{\hat{W}}_{}^{*T}{}_{}\left(X_{}\right)-\frac{1}{2}{\hat{W}}_{a_p}^{*T}{}_{I_p}\left(X_{I_p}\right)\right)+\left(2K_1{e}_f+2{\hat{W}}_{}^{*T}{}_{}\left(X_{}\right)+{\hat{W}}_{c_p}^{*T}{}_{I_p}\left(X_{I_p}\right)\right)$$\left(z_f+-K_p{e}_f-{\hat{W}}_{}^{*T}{}_{}\left(X_{}\right)-\frac{1}{2}{\hat{W}}_{a_p}^{*T}{}_{I_p}\left(X_{I_p}\right)\right)$

The Bellman's residual error {tilde over (H)}.sub.p is expressed as:

[00070]$\begin{array}{cc}{\stackrel{}{H}}_p=H_p\left(e_f,{\hat{}}_f^{*},\frac{{\hat{I}}_p}{e_f}\right)-H_p\left(e_f,{}_f^{*},\frac{I_p^{*}}{e_f}\right)& \left(49\right)\end{array}$

Considering (36), (49) becomes:

[00071]$\begin{array}{cc}{\stackrel{}{H}}_p=H_p\left(e_f,{\hat{}}_f^{*},\frac{{\hat{I}}_p}{e_f}\right)& \left(50\right)\end{array}$

With regards to (48), it is desired that

[00072]${\hat{}}_f^{*}$

realizes.

[00073]${\stackrel{}{H}}_p=H_p\left(e_f,{\hat{}}_f^{*},\frac{{\hat{I}}_p}{e_f}\right).fwdarw.0.$

If

[00074]$H_p\left(e_f,{\hat{}}_f^{*},\frac{{\hat{I}}_p}{e_f}\right)=0$

ensued, the following also hold:

[00075]$\begin{array}{cc}\frac{{}_f{H}_p\left(e_f,{\hat{}}_f^{*},\frac{{\hat{I}}_p}{e_f}\right)}{{\hat{W}}_{a_p}}=\frac{1}{2}{}_{I_p}^T{}_{I_p}\left({\hat{W}}_{a_p}-{\hat{W}}_{c_p}\right)=0& \left(51\right)\end{array}$

To derive the weight updating laws for .sub.a.sub.p and .sub.c.sub.p to guarantee (51), a positive definite function S.sub.p is defined as:

[00076]$\begin{array}{cc}{S}_p={\left({\hat{W}}_{a_p}-{\hat{W}}_{c_p}\right)}^T\left({\hat{W}}_{a_p}-{\hat{W}}_{c_p}\right)& \left(52\right)\end{array}$

Clearly, S.sub.p=0 is equivalent to (51). Noting that S.sub.p/.sub.a.sub.p=S.sub.p/.sub.c.sub.p=2 (.sub.a.sub.p.sub.c.sub.p), the time derivative of S.sub.p gives

[00077]$\begin{array}{cc}\begin{array}{c}\frac{d{S}_p}{\mathrm{dt}}=\frac{S_p}{{\hat{W}}_{c_p}^T}{\stackrel{.}{\hat{W}}}_{c_p}+\frac{S_p}{{\hat{W}}_{a_p}^T}{\stackrel{.}{\hat{W}}}_{a_p}=-k_c\frac{S_p}{{\hat{W}}_{c_p}^T}{}_{I_p}^T{}_{I_p}{\hat{W}}_{c_p}-\\ \frac{S_p}{{\hat{W}}_{a_p}^T}{}_{I_p}^T{}_{I_p}\left(k_a\left({\hat{W}}_{a_p}-{\hat{W}}_{c_p}\right)+k_c{\hat{W}}_{c_p}\right)\\ =-\frac{k_a}{2}\frac{S_p}{{\hat{W}}_{a_p}^T}{}_{I_p}^T{}_{I_p}\frac{S_p}{{\hat{W}}_{\mathrm{ai}}}0\end{array}& \left(53\right)\end{array}$

The RBFNN weight of the critic is updated as follows:

[00078]$\begin{array}{cc}{\stackrel{.}{\hat{W}}}_{c_p}=-{}_{c_p}{}_{I_p}^T{}_{I_p}{\hat{W}}_{c_p}& \left(54\right)\end{array}$

where .sub.c.sub.p>0 is a constant.
The RBFNN weight of the actor is updated as follows:

[00079]$\begin{array}{cc}{\stackrel{.}{\hat{W}}}_{a_p}=-{}_{I_p}^T{}_{I_p}\left({}_{a_p}\left({\hat{W}}_{a_p}-{\hat{W}}_{c_p}\right)+{}_{c_p}{\hat{W}}_{c_p}\right)& \left(55\right)\end{array}$

where .sub.a.sub.p>0 is a constant.
Using the approximate optimal controller (47), the error dynamics (32) can be rewritten as:

[00080]$\begin{array}{cc}{\stackrel{.}{e}}_f=z_f+{\hat{a}}_f^{*}+& \left(56\right)\end{array}$

A Lyapunov candidate function is selected for the er subsystem as:

[00081]$\begin{array}{cc}{L}_1=\frac{1}{2}{e}_f^T{e}_f+\frac{1}{2}{\tilde{W}}_{}^T{}_p^{-1}{\tilde{W}}_{}+\frac{1}{2}{\tilde{W}}_{c_p}^T{\tilde{W}}_{c_p}+\frac{1}{2}{\tilde{W}}_{a_p}^T{\tilde{W}}_{a_p}& \left(57\right)\end{array}$

where

[00082]${\tilde{W}}_{}={\hat{W}}_{}-W_{}^{*},{\tilde{W}}_{a_p}={\hat{W}}_{a_p}-W_{I_p}^{*},\mathrm{and}{\tilde{W}}_{c_p}={\hat{W}}_{c_p}-W_{I_p}^{*}$

are the RBFNN weight errors for the identifier, actor, and critic networks, respectively.

[0122] Differentiating L.sub.1 with respect to time and considering (45), (54), and (55), one gets:

[00083]$\begin{array}{cc}{\stackrel{.}{L}}_1=e_f\left(z_f+{\hat{}}_f^{*}+\right)+{\tilde{W}}_{}^T\left({}_{}^T{e}_f-{}_p{\hat{W}}_{}\right)-{}_{c_p}{\tilde{W}}_{c_p}^T{}_{I_p}^T{}_{I_p}{\hat{W}}_{c_p}-{\tilde{W}}_{a_p}^T{}_{I_p}^T{}_{I_p}\left({}_{a_p}\left({\hat{W}}_{a_p}-{\hat{W}}_{c_p}\right)+{}_{c_p}{\hat{W}}_{c_p}\right)& \left(58\right)\end{array}$

Inserting (40) and (47) into (58), one gets:

[00084]$\begin{array}{cc}{\stackrel{.}{L}}_1=e_f\left(-K_p{e}_f-\frac{1}{2}{\hat{W}}_{a_p}^T{}_{I_p}\left(X_{I_p}\right)-{\tilde{W}}_{}^T{}_{}\left(X_{}\right)+{}_{}+z_f\right)+{\tilde{W}}_{}^T\left({}_{}^T{e}_f-{}_p{\hat{W}}_{}\right)-{}_{c_p}{\tilde{W}}_{c_p}^T{}_{I_p}^T{}_{I_p}{\hat{W}}_{c_p}-{\tilde{W}}_{a_p}^T{}_{I_p}^T{}_{I_p}\left({}_{a_p}\left({\hat{W}}_{a_p}-{\hat{W}}_{c_p}\right)+{}_{c_p}{\hat{W}}_{c_p}\right)& \left(59\right)\end{array}$

Equation (59) can be re-expressed as follows:

[00085]$\begin{array}{cc}{\stackrel{.}{L}}_1=-e_f{K}_p{e}_f+e_f{}_{}+e_f{z}_f-\frac{1}{2}{e}_f^T{\hat{W}}_{a_p}^T{}_{I_p}-{}_p{\tilde{W}}_{}^T{\hat{W}}_{}-{}_{c_p}{\tilde{W}}_{c_p}^T{}_{I_p}^T{}_f{}_{\mathrm{ff}}{}_{I_p}{\hat{W}}_{c_p}-{\tilde{W}}_{a_p}^T{}_{I_p}^T{}_f{}_{\mathrm{ff}}{}_{I_p}\left({}_{a_p}\left({\hat{W}}_{a_p}-{\hat{W}}_{c_p}\right)+{}_{c_p}{\hat{W}}_{c_p}\right)& \left(60\right)\end{array}$

By utilizing Young's inequality, we have:

[00086]$\begin{array}{cc}\begin{array}{cc}{e}_f^T{}_{}& \frac{1}{2}{e}_f^T{e}_f+\frac{1}{2}{}_{}^T{}_{}\\ e_f^T{z}_f& \frac{1}{2}{e}_f^T{e}_f+\frac{1}{2}{z}_f^T{z}_f\\ -\frac{1}{2}{e}_f^T{\hat{W}}_{a_p}^T{}_{I_p}& \frac{1}{4}{e}_f^T{e}_f+\frac{1}{4}{\hat{W}}_{a_p}^T{}_{I_p}^T{}_{I_p}{\hat{W}}_{a_p}\end{array}& \left(61\right)\end{array}$

Substituting the Young's inequality (61) into (60) gives:

[00087]$\begin{array}{cc}{\stackrel{.}{L}}_1=-e_f\left(K_p-1.25\right)e_f-{}_p{\tilde{W}}_{}^T{\hat{W}}_{}-{}_{c_p}{\tilde{W}}_{c_p}^T{}_{I_p}^T{}_{I_p}{\hat{W}}_{c_p}-{}_{a_p}{\tilde{W}}_{a_p}^T{}_{I_p}^T{}_{I_p}{\hat{W}}_{a_p}+\left({}_{a_p}-{}_{a_c}\right){\tilde{W}}_{a_p}^T{}_{I_p}^T{}_{I_p}{\hat{W}}_{c_p}+\frac{1}{4}{\hat{W}}_{a_p}^T{}_{I_p}^T{}_{I_p}{\hat{W}}_{a_p}+\frac{1}{2}{}_{}^T+\frac{1}{2}{z}_f^T{z}_f& \left(62\right)\end{array}$