Destabilizing Attack and Robust Defense for Inverter-Based Microgrids by Adversarial Deep Reinforcement Learning

Abstract

<p>The droop controllers of inverter based resources (IBRs) can be adjustable by grid operators to facilitate regulation services. Considering the increasing integration of IBRs in distribution systems, cyber-security is becoming a major concern. This paper investigates the data-driven destabilizing attack and robust defense strategy based on deep reinforcement learning for inverter-integrated distribution systems. Firstly, the full-order high-fidelity model and reduced-order small-signal model of typical multi-inverter systems are derived. Then the destabilizing attack on the droop control gains is analyzed, which reveals its impact to system stability. Finally, the attack and defense problems are formulated as Markov decision process (MDP) and adversarial MDP (AMDP). The problems are solved by twin delayed deep deterministic policy gradient (TD3) algorithm to find the least effort attack path of the system and obtain the corresponding robust defense strategy. The simulation studies are conducted in a multi-inverter microgrid system with 4 IBRs and IEEE 123-bus system with 10 IBRs to evaluate the proposed method.<br> </p>

Highly transferable adversarial attack against deep‐reinforcement‐learning‐based frequency control

“…Hackers hacked the information systems of three energy distribution companies in Ukraine, resulting in power grid vulnerabilities that affected 30 substations and approximately 230000 people. Therefore, once DRL models are deployed in the SCADA system, an adversarial attacker can design an attack algorithm based on several aspects of the DRL model, such as observation-oriented [15][16][17][18], reward-oriented [19], and environment-oriented attacks [20].…”

Deep reinforcement learning based voltage control revisited

“…Deep reinforcement learning based applications in power system are spreading widely because of its capability in solving challenges like decision making in complex and dynamic environment. Recent studies have demonstrated the effective use of DRL-based techniques in resolving various power system issues with satisfactory results, including grid operation [1,2], grid emergency control [3][4][5], energy trading [6][7][8], electricity markets [9], battery control [10,11], demand response [12], economic dispatch [13], cyber security [14][15][16], load-frequency control [17], and real-time topology control [18]. Some of the advantages of the DRL method over traditional methods include flexibility, continuous learning, and the elimination of the need for explicit models.…”