Resilient reinforcement learning and robust output regulation under denial-of-service attacks

Gao, Weinan; Deng, Chao; Yi, Jin; Jiang, Zhong‐Ping

doi:10.1016/j.automatica.2022.110366

Cited by 58 publications

(21 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on accessible data, these matrices have been estimated in [54] with the help of -learning and exploration noises (guaranteeing the PE condition). Then, a -learning algorithm combined with a dropout Smith predictor has been designed in [55] to solve the optimal control problems with network-induced dropouts. Very recently, a Bernoulli-driven HJB equation has been first developed in [53] to deal with optimal control problems without both a priori knowledge of system dynamics and the probability models of packet dropouts.…”

Section: Q Qmentioning

confidence: 99%

Adaptive Dynamic Programming for Networked Control Systems Under Communication Constraints: A Survey of Trends and Techniques

Wang¹,

Sun

Ding³

2022

IJNDI

129

View full text Add to dashboard Cite

Survey/review study Adaptive Dynamic Programming for Networked Control Systems under Communication Constraints: A Survey of Trends and Techniques Xueli Wang 1, Ying Sun 1,*, and Derui Ding 2 1 Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China 2 School of Software and Electrical Engineering, Swinburne University of Technology, Melbourne 3122, Australia * Correspondence: yingsun1991@163.com Received: 22 September 2022 Accepted: 28 November 2022 Published: 22 December 2022 Abstract: The adaptive dynamic programming (ADP) technology has been widely used benefiting from its recursive structure in forward and the prospective conception of reinforcement learning. Furthermore, ADP-based control issues with communication constraints arouse ever-increasing research consideration in theoretical analysis and engineering applications due mainly to the wide participation of digital communications in industrial systems. The latest development of ADP-based optimal control with communication constraints is systematically surveyed in this paper. To this end, the development of ADP-based dominant methods is first investigated from their structures and implementation. Then, technical challenges and corresponding approaches are comprehensively and thoroughly discussed and the existing results are reviewed according to the constraint types. Furthermore, some applications of the ADP method in practical systems are summarized. Finally, future topics are lighted on ADP-based control issues.

show abstract

Section: Q Qmentioning

confidence: 99%

Adaptive Dynamic Programming for Networked Control Systems Under Communication Constraints: A Survey of Trends and Techniques

Wang¹,

Sun

Ding³

2022

IJNDI

129

View full text Add to dashboard Cite

show abstract

“…8 DoS attacks affect the transmission channels blocking the communication between the controller, sensors, and actuators. 9,10 On the other hand, deception attacks affect data integrity by injecting false data into some components, such as actuators or sensors. 11 In particular, control strategies concerning deception attacks are still incipient due to the difficulties in accurately modeling those attacks, which may effectively be any type of signal injection.…”

Section: Introductionmentioning

confidence: 99%

Event‐triggered control of Takagi‐Sugeno fuzzy systems under deception attacks

Peixoto

Coutinho

Bessa

et al. 2023

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

This article handles the problem of periodic event-triggered control co-design for the stabilization of Takagi-Sugeno fuzzy models subject to stochastic deception attacks whose occurrence follows a given Bernoulli distribution.A novel delay-dependent condition is presented to simultaneously design the event-triggering mechanism and the state-feedback controller to ensure the local mean-square asymptotic stability of the closed-loop system. The co-design condition is derived as linear matrix inequalities by considering Lyapunov-Krasovskii stability arguments. In addition, an optimization procedure is provided to estimate an enlarged set of admissible initial conditions and the corresponding ultimate bound within the validity domain of the fuzzy model. Finally, numerical examples are provided to demonstrate the effectiveness of the proposed approach.

show abstract

“…In particular, ADP has gained numerous attentions recently and been applied to control both continuous-time systems [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26] and discrete-time systems. [27][28][29][30][31][32] Therefore, different studies have considered combining the theories of adaptive optimal control with the output regulation in order to achieve the adaptive optimal output regulation problem; see 13,[33][34][35][36][37][38][39][40][41][42][43][44] and references therein. Using reinforcement learning and Bellman's principle of optimality, 10 ADP methods, 16,24,34,[45][46][47][48][49][50][51][52][53]…”

Section: Introductionmentioning

confidence: 99%

“…[27][28][29][30][31][32] Therefore, different studies have considered combining the theories of adaptive optimal control with the output regulation in order to achieve the adaptive optimal output regulation problem; see 13,[33][34][35][36][37][38][39][40][41][42][43][44] and references therein. Using reinforcement learning and Bellman's principle of optimality, 10 ADP methods, 16,24,34,[45][46][47][48][49][50][51][52][53][54][55][56] which are essentially based on reinforcement learning, are developed such that the agent can learn towards the optimal control policy by interacting with its unknown environment. With this learning framework, see Fig.…”

Section: Introductionmentioning

confidence: 99%

Autonomous Satellite Docking via Adaptive Optimal Output Rregulation: A Reinforcement Learning Approach

Qasem¹,

Tiwari²,

Gutiérrez³

2023

Preprint

View full text Add to dashboard Cite

This paper describes an online off-policy data-driven reinforcement learning basedalgorithm to regulate and control the relative position of a deputy satellite in an autonomous satellite docking problem. The optimal control policy is learned under the framework of output regulation problem and adaptive dynamic programming (ADP) by considering the continuous-time linearized model of the satellite. The linearized model of relative motion is used to describe the motion between satellites, and the satellite docking problem is formulated as a linear optimal output regulation problem, in which the feedback-forward optimal controller is used to track a class of references and rejecting a class of disturbances while maintaining the overall system's closed-loop stability. The optimal control problem is presented using a data-driven reinforcement learning based method to regulate the relative position and velocity of the deputy to safely dock with the chief. Using the adaptive optimal output regulation framework, the learned optimal feedbackfeedforward gains guarantee optimal transient and steady state performances without any prior knowledge of the dynamics of the studied system. The states/input information of the underlying dynamical system are instead used to compute the approximated optimal feedback-feedforward control gain matrices. Reference tracking and disturbance rejection are achieved in an optimal sense without using any modelling information of the physics of the satellites. Simulation results are presented and demonstrate the efficacy of the proposed method.

show abstract

Resilient reinforcement learning and robust output regulation under denial-of-service attacks

Cited by 58 publications

References 31 publications

Adaptive Dynamic Programming for Networked Control Systems Under Communication Constraints: A Survey of Trends and Techniques

Adaptive Dynamic Programming for Networked Control Systems Under Communication Constraints: A Survey of Trends and Techniques

Event‐triggered control of Takagi‐Sugeno fuzzy systems under deception attacks

Autonomous Satellite Docking via Adaptive Optimal Output Rregulation: A Reinforcement Learning Approach

Contact Info

Product

Resources

About