Deep Reinforcement Learning-Based Voltage Control to Deal with Model Uncertainties in Distribution Networks

Abstract

This paper addresses the voltage control problem in medium-voltage distribution networks. The objective is to cost-efficiently maintain the voltage profile within a safe range, in presence of uncertainties in both the future working conditions, as well as the physical parameters of the system. Indeed, the voltage profile depends not only on the fluctuating renewable-based power generation and load demand, but also on the physical parameters of the system components. In reality, the characteristics of loads, lines and transformers are subject to complex and dynamic dependencies, which are difficult to model. In such a context, the quality of the control strategy depends on the accuracy of the power flow representation, which requires to capture the non-linear behavior of the power network. Relying on the detailed analytical models (which are still subject to uncertainties) introduces a high computational power that does not comply with the real-time constraint of the voltage control task. To address this issue, while avoiding arbitrary modeling approximations, we leverage a deep reinforcement learning model to ensure an autonomous grid operational control. Outcomes show that the proposed model-free approach offers a promising alternative to find a compromise between calculation time, conservativeness and economic performance.

Reinforcement learning for robust voltage control in distribution grids under uncertainties

“…Control of PV generation by DDPG was extended to static Var compensator in [14]. Coupling DDPG with a surrogate neural network (to learn the nonlinear mapping between the active/reactive power injections and the voltage magnitude at each node) allowed to control of distributed generation units as well as of tap transformer position in [15]. DDPG considers the limited knowledge of grid parameters, but an external predictor of load and generation is needed.…”

Real-time security margin control using deep reinforcement learning

“…Previous implementations of DRL in electric power system control have so far mainly been focused on emergency control, which has the role of controlling the system back into a stable state after a disturbance has already occurred [13]. Implementations include methods adapted for automatic voltage control [14]- [17], optimal load shedding [4], [18], [19], generator dynamic breaking [4], and oscillation damping [20]. DRL-based implementations adapted for preventive control found in the literature are few and in [13], the authors argue that the reason may be because these control problems have traditionally been formulated as static optimization problems.…”

Enhanced integration of flow-based market coupling in short-term adequacy assessment

“…Traditionally, the adequacy assessment has been carried out according to a deterministic formulation based on the amounts of peak load demands and available generations under conservative contingency assumptions. The growing integration of renewable-based generations into the power network has introduced new uncertainties in the electric power system [1], [2]. In addition, the liberalization of the electricity sector requires a cost-effective planning and operation of power network, which cannot be achieved through deterministic-based calculations [3], [4].…”

“…The FB domains are polytopes consisting of several hundred (up to 1000 and more) vertices. In order to calculate the dissimilarity between overall shapes of two FB domains, the final distance (dissimilarity) between two FB domains is calculated by aggregating all individual distances between each pair of their vertices (see (2)). Doing so, the final distance becomes too general such that different FB domains having various shapes can lead to a similar value as their differences can be covered in the final distance.…”