Nonlinear Filtering and Reinforcement Learning-based Smart Autonomous Multi-agent Systems

Abstract

<p>There is growing importance in complex engineering systems to operate autonomously, especially given potential malfunctions that may occur in the system, such as sensors, actuators, components, communication networks, and controllers. Fault detection, isolation, and reconstruction (FDIR) are crucial for autonomous systems. There is significant demand to evolve efficient intelligent systems to detect faults, isolate fault locations and autonomously reconstruct any component of a complex dynamical system. Hence, it is essential to detect, isolate, and reconstruct the faults efficiently and on time when the systems are in operation. This dissertation presents a novel methodology for developing smart autonomous multiagent systems (SAMAS). The SAMAS comprises several agents; some are homogeneous while others are heterogeneous. The proposed method involves developing a multi-agent systems (MAS) model and designing a decentralized smart control system. The MAS model contains homogeneous and heterogeneous agents, communicates among agents through an undirected connected graph, external disturbances, goals, and constraints. The sensor, actuator, communication, and controller faults are also modeled in the MAS model. A decentralized smart control system is designed to create a SAMAS model in the presence of uncertainty in each agents’ dynamics and faults located in the sensors, actuators, communication networks, and controllers. The proposed control method is based on non-linear filtering techniques, deep reinforcement learning, and robust control techniques. A Chebyshev neural network (CNN) is incorporated to learn the uncertain nonlinear functions in the agent dynamics of MAS. Additionally, robust control term using the hyperbolic tangent function is applied to counteract the neural network approximation errors. Meanwhile, a novel algorithm has been proposed which is employed to estimate the uncertain states of the agent dynamics and to train the internal parameters of the neural network given a set of prior measurements. Moreover, an adaptive threshold method has been proposed to detect any kinds of faults present in the system followed by a likelihood-based isolation method to locate each faulty agent. The novel algorithm is known as a reinforced unscented Kalman filter (RUKF). The primary purpose of the RUKF is to detect and isolate the faults, and to adapt the process and measurement noise covariance matrices to reconstruct the faults. To assess the performance of the proposed methodology, we developed a SAMAS consisting of six heterogeneous uncertain agent dynamics. The SAMAS model and the proposed control methodology are numerically simulated using MATLAB. A Monte-Carlo (MC) simulation was carried out to assess the performance of the proposed control methodology in the presence of uncertain agent dynamics and with the sensor, actuator, communication, and controller faults. The fault isolation results are summarized in confusion matrices for each faulty case. The stability of the RUKF, which ran in conjunction with a robust control method, has been proven using the Lyapunov stability approach. Extensive simulations were conducted to evaluate the performance of the proposed method. In this study, the proposed method showed superior performance to the standard unscented Kalman filter (UKF) and adaptive UKF (AUKF). The proposed fault isolation scheme isolated the faulty agents with over 96.68% success rate at the system level. Hence, the results of the numerical simulations show the feasibility of the proposed approach. The proposed RUKF is also computationally less expensive than the standard UKF and AUKF. The proposed approach can be considered a promising tool to evaluate fault detection, isolation, and reconstruction in complex engineering systems. Furthermore, the proposed approach can be extended to other deep space complex systems. The proposed approach can also be used for MAS problems for the financial sector, such as stock market prediction, economic time series, and multi-arm bandit problems.</p>