Understanding the Safety Requirements for Learning-based Power Systems Operations

Abstract

Recent advancements in machine learning and reinforcement learning have brought increased attention to their applicability in a range of decision-making tasks in the operations of power systems, such as short-term emergency control, Volt/VAr control, long-term residential demand response and battery energy management. Despite the promises of providing strong representation of complex system dynamics and fast, efficient learned operation strategies, the safety requirements of such learning paradigms are less discussed. This paper explores the design requirements on both data and model side of such learning algorithms by exploiting the impacts of adversarial attacks on safety critical system operations. Case studies performed on both voltage regulation and topology control tasks demonstrated the potential vulnerabilities of the standard reinforcement learning algorithms, and possible measures of machine learning robustness and security are discussed for power systems operation tasks.

Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training

“…Previous works in machine learning have demonstrated that standard RL agents are vulnerable to observation-noise attacks. With small perturbations over state space or minor alterations on the underlying dynamics, even fully-optimized RL agents will output non-optimal actions [14], [15]. Meanwhile, power grids have always been regarded as safety-critical infrastructures [16], and it is of top priority to validate the reliability of proposed algorithms under either system state uncertainties (e.g., renewable forecasting [17]) or system uncertainties (e.g., N-1 security criterion [6], [18]).…”

SAVER: Safe Learning-Based Controller for Real-Time Voltage Regulation

“…However, compared to the model-based counterparts [4], RL is trained to maximize the accumulated reward, while hard physical constraints such as feasible voltage magnitudes are mostly not guaranteed to be satisfied. Unsafe reactive power injections can cause severe impacts over the grids such as voltage collapse and load shedding [8], [9].…”