Data-Driven Dynamical Control for Bottom-up Energy Internet System

Abstract

With the increasing concern on climate change and global warming, the reduction of carbon emission becomes an important topic in many aspects of human society. The development of energy Internet (EI) makes it possible to achieve better utilization of distributed renewable energy sources with the power sharing functionality introduced by energy routers (ERs). In this paper, a bottom-up EI architecture is designed, and a novel data-driven dynamical control strategy is proposed. Intelligent controllers augmented by deep reinforcement learning (DRL) techniques are adopted for the operation of each microgrid independently in the bottom layer. Moreover, the concept of curriculum learning (CL) is integrated into DRL to improve the sample efficiency and accelerate the training process. Based on the power exchange plan determined in the bottom layer, considering the stochastic nature of electricity price in the future power market, the optimal power dispatching scheme in the upper layer is decided via model predictive control. The simulation has shown that, under the bottom-up architecture, compared with the conventional methods such as proportional integral and optimal power flow, the proposed method reduces overall generation cost by 7.1% and 37%, respectively. Meanwhile, the introduced CL-based training strategy can significantly speed up the convergence during the training of DRL. Last but not least, our method increases the profit of energy trading between ERs and the main grid.

Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach

“…Many of the RL applications for battery management define continuous action spaces, which motivates the selection of algorithms capable of handling continuous as well as discrete spaces, such as Several RL algorithms are available for optimizing the agent. Many of the RL applications for battery management define continuous action spaces, which motivates the selection of algorithms capable of handling continuous as well as discrete spaces, such as Advantage Actor-Critic (A2C) [77], Proximal Policy Optimization (PPO) [22], Deep Deterministic Policy Gradient (DDPG) [78,79] and Twin Delayed DDPG (TD3) [80] have been applied in the context of batteries. However, the task of the RL agent in this paper is to select the value for C. This selection must be made from a discrete set of possible values, due to the rules of the FCR-N market.…”

Enhanced Anomaly Detection System for IoT Based on Improved Dynamic SBPSO

“…The resulting solution was tested in a smart home scenario with eight IoT gadgets. In [22], the authors designed a bottom-up EI architecture and proposed novel data driven dynamical control strategy. Moreover, Intelligent controllers augmented by deep reinforcement learning (DRL) techniques are adopted and the concept of curriculum learning (CL) is integrated into DRL to improve the sample efficiency and accelerate the training process.…”

Multi‐level refined power system operation mode analysis: A data‐driven approach

“…The current data‐driven technology research is mainly divided into pattern recognition, prediction, and decision analysis. In terms of pattern recognition, data‐driven technology is mainly used in power system state estimation [6], topology recognition [7], and decision‐making [8], and so on. In [6], a method for online stability analysis of power systems using fuzzy pattern recognition was proposed to identify system stability and out‐of‐step units quickly.…”

“…In [7], a data‐driven method combining mathematical and physical models was used to identify the configuration of grid topology and corresponding line parameters. In [8], a hierarchical energy management scheme for energy Internet based on data‐driven stochastic optimization is proposed. The scheme can efficiently utilize the renewable energy of the power system and reduce power generation costs.…”