Stability Constrained Reinforcement Learning for Real-Time Voltage Control in Distribution Systems

Abstract

Deep Reinforcement Learning (RL) has been recognized as a promising tool to address the challenges in realtime control of power systems. However, its deployment in realworld power systems has been hindered by a lack of explicit stability and safety guarantees. In this paper, we propose a stability constrained reinforcement learning method for real-time voltage control in both single-phase and three-phase distribution grids and we prove that the proposed approach provides a voltage stability guarantee. The key idea underlying our approach is an explicitly constructed Lyapunov function that certifies stability. We demonstrate the effectiveness of our approach with both single-phase and three-phase IEEE test feeders, where the proposed method can reduce the transient control cost by more than 25% and shorten the response time by 21.5% on average compared to the widely used linear policy, while always achieving voltage stability. In contrast, standard RL methods often fail to achieve voltage stability.