Power System Security Under False Data Injection Attacks With Exploitation and Exploration Based on Reinforcement Learning

Wang, Zhisheng; Chen, Ying; Liu, Feng; Xia, Yunfeng; Zhang, Xuemin

doi:10.1109/access.2018.2856520

Cited by 25 publications

(21 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2) Select the appropriate value h k , compute∆ 22,k+1 , Λ 22,k+1 by (19), if∆ 22,k+1 < 0 and Λ 22,k+1 > 0,then the controller parametersK k can be solved by (20), and go to the next step, else jump to Step 7); 3) If the condition (21) is satisfied, go to the next step, else return to Step 2); 4) Solve the backward RDEs (17) to get P k and S k ; 5) If k = 0, set k = k − 1, and go back to Step 2), else turn to the next step; 6) If the condition P 0 < γ 2 W is satisfied, this algorithm is feasible, and output the results, else go to…”

Section: Resultsmentioning

confidence: 99%

H _∞ Control for Time-Varying Cyber-Physical System Under Randomly Occurring Hybrid Attacks: The Output Feedback Case

Liu

et al. 2020

IEEE Access

View full text Add to dashboard Cite

In this paper, the H ∞ control problem for a class of linear time-varying cyber-physical system (CPS) under randomly occurring hybrid attacks in a finite horizon is investigated. The hybrid attacks, including denial of service (DoS) attacks on both sensor-to-controller and controller-to-actuator communication channels and false data injection (FDI) attacks on sensors and actuators, aim to destroy the measurement data and control data in order to endanger the functionality of the closed-loop system. The purpose of this paper is to study the relationship between the attack injected signals and the controlled output, and to design the output feedback controller gains so that the H ∞ performance of the closed-loop system is guaranteed over a given finite horizon, meanwhile, the impact of attack signals in the worst case on the linear quadratic performance can be reduced. In order to solve the above problems, both the methods of stochastic analysis and completing squares are utilized to establish the sufficient conditions for the existence of the desired controller, and a finite-horizon controller design algorithm is presented by solving two coupled backward recursive Riccati difference equations (RDEs) subject to some scheduled conditions. At last, the numerical simulation and experimental results are given out to demonstrate the effectiveness of the proposed approach. INDEX TERMS Cyber-physical system, hybrid attacks, time-varying systems, H ∞ control, Riccati equations.

show abstract

Section: Resultsmentioning

confidence: 99%

H _∞ Control for Time-Varying Cyber-Physical System Under Randomly Occurring Hybrid Attacks: The Output Feedback Case

Liu

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…For discrete power control strategies, we propose the JID algorithm to achieve the general NE of the joint channel access and power optimization game. Besides, inspired by the process of exploration and exploitation in reinforcement learning algorithms [44], [45], the JIDEE algorithm is designed to achieve the optimal NE. Simultaneously, these two algorithms are extended to the case where the power control strategies are continuous.…”

Section: Distributed Algorithmsmentioning

confidence: 99%

“…Moreover, finding the optimal NE in a two-dimensional strategy space is even more difficult. Inspired by the thought of exploration and exploitation in reinforcement learning algorithms [44], [45], a joint-strategy iteration algorithm for discrete power control strategies based on exploration and exploitation (JIDEE) is proposed, which will eventually converge to the optimal NE after a sufficient number of iterations . The detail of JIDEE is listed in algorithm 3.…”

Section: Joint-strategy Iteration Algorithm For Discrete Power Conmentioning

confidence: 99%

A Distributed Collaborative Game-Theoretic Approach in Cognitive Satellite Communication Networks

et al. 2020

View full text Add to dashboard Cite

Due to the increasing spectrum scarcity in satellite communications and terrestrial communications, cognitive satellite communications have received widespread attention. In this paper, we take the inherent feature that the terrestrial cognitive users suffer from inter-beam interference. In particular, for the first time, we consider the coupling characteristics of channel access and power optimization in opportunistic spectrum access based on distributed frameworks. We first formulate a joint channel access and power optimization game, which is proven to be an exact potential game and accordingly has at least one pure Nash Equilibrium (NE) point. The sufficient conditions for interference-free between cognitive users and for maximization of system utility are given, respectively. Also, the lower bound of the system utility is deduced theoretically. For discrete power control strategies, we then propose a joint-strategy iteration algorithm (JID) to converge to the general NE in two-dimensional discrete strategy space. Especially, to solve the challenge of finding the optimal NE solution in two-dimensional strategy space, we propose a novel joint-strategy iteration algorithm based on exploration and exploitation (JIDEE). Simultaneously, these two algorithms are extended to the case where the power control strategies are continuous. Finally, simulations are conducted to confirm the effectiveness of the formulated game and the two proposed algorithms. INDEX TERMS Cognitive satellite communication, spectrum sharing, exact potential game, exploration and exploitation.

show abstract

“…An important aspect is cyber-security since cyber attacks can make best designed controls to malfunction or degrade their performances. Several works dealt with this problem [80][81][82][83][84][85][86][87][88]. Reference [80] considered cyber-physical systems security from systems and control perspective in general, and shortly discussed possibilities to use RL and DRL to this purpose.…”

Section: Power System Control-related Considerationsmentioning

confidence: 99%

“…Q-learning was proposed in [81] to analyze the transmission grid vulnerability under sequential topology attacks and identify critical attack sequences with consideration of physical system behaviors. A modified Q-learning (termed nearest sequence memory Q-learning) was adopted in [83] to evaluate threat imposed by false data injection attack on voltage control of a power system. Test results revealed if even a few substations are attacked a voltage collapse with its consequences can happen in the system.…”

Section: Power System Control-related Considerationsmentioning

confidence: 99%

(Deep) Reinforcement learning for electric power system control and related problems: A short review and perspectives

Glavić

2019

Annual Reviews in Control

111

View full text Add to dashboard Cite

This paper reviews existing works on (deep) reinforcement learning considerations in electric power system control. The works are reviewed as they relate to electric power system operating states (normal, preventive, emergency, restorative) and control levels (local, household, microgrid, subsystem, wide-area). Due attention is paid to the control-related problems considerations (cyber-security, big data analysis, short-term load forecast, and composite load modelling). Observations from reviewed literature are drawn and perspectives discussed. In order to make the text compact and as easy as possible to read, the focus is only on the works published (or "in press") in journals and books while conference publications are not included. Exceptions are several work available in open repositories likely to become journal publications in near future. Hopefully this paper could serve as a good source of information for all those interested in solving similar problems.

show abstract

Power System Security Under False Data Injection Attacks With Exploitation and Exploration Based on Reinforcement Learning

Cited by 25 publications

References 20 publications

H _∞ Control for Time-Varying Cyber-Physical System Under Randomly Occurring Hybrid Attacks: The Output Feedback Case

H _∞ Control for Time-Varying Cyber-Physical System Under Randomly Occurring Hybrid Attacks: The Output Feedback Case

A Distributed Collaborative Game-Theoretic Approach in Cognitive Satellite Communication Networks

(Deep) Reinforcement learning for electric power system control and related problems: A short review and perspectives

Contact Info

Product

Resources

About

Power System Security Under False Data Injection Attacks With Exploitation and Exploration Based on Reinforcement Learning

Cited by 25 publications

References 20 publications

H ∞ Control for Time-Varying Cyber-Physical System Under Randomly Occurring Hybrid Attacks: The Output Feedback Case

H ∞ Control for Time-Varying Cyber-Physical System Under Randomly Occurring Hybrid Attacks: The Output Feedback Case

A Distributed Collaborative Game-Theoretic Approach in Cognitive Satellite Communication Networks

(Deep) Reinforcement learning for electric power system control and related problems: A short review and perspectives

Contact Info

Product

Resources

About

H _∞ Control for Time-Varying Cyber-Physical System Under Randomly Occurring Hybrid Attacks: The Output Feedback Case

H _∞ Control for Time-Varying Cyber-Physical System Under Randomly Occurring Hybrid Attacks: The Output Feedback Case