Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review

Cao, Dengqing; Hu, Weihao; Zhao, Junbo; Zhang, Guozhou; Zhang, Bin; Liu, Zhou; Chen, Zhe; Blaabjerg, Frede

doi:10.35833/mpce.2020.000552

Cited by 262 publications

(95 citation statements)

References 79 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…DRL is the combination of DNNs and RL [98]. The basic idea behind DRL consists of assigning rewards and punishments for an agent to shape its policies.…”

Section: ) Deep Reinforcement Learningmentioning

confidence: 99%

Deep Learning in Smart Grid Technology: A Review of Recent Advancements and Future Prospects

et al. 2021

View full text Add to dashboard Cite

The current electric power system witnesses a significant transition into Smart Grids (SG) as a promising landscape for high grid reliability and efficient energy management. This ongoing transition undergoes rapid changes, requiring a plethora of advanced methodologies to process the big data generated by various units. In this context, SG stands tied very closely to Deep Learning (DL) as an emerging technology for creating a more decentralized and intelligent energy paradigm while integrating high intelligence in supervisory and operational decision-making. Motivated by the outstanding success of DL-based prediction methods, this article attempts to provide a thorough review from a broad perspective on the state-of-the-art advances of DL in SG systems. Firstly, a bibliometric analysis has been conducted to categorize this review's methodology. Further, we taxonomically delve into the mechanism behind some of the trending DL algorithms. We then showcase the DL enabling technologies in SG, such as federated learning, edge intelligence, and distributed computing. Finally, challenges and research frontiers are provided to serve as guidelines for future work in the futuristic power grid domain. This study's core objective is to foster the synergy between these two fields for decision-makers and researchers to accelerate DL's practical deployment for SG systems. INDEX TERMSSmart grid, deep learning, deep neural networks, edge computing, distributed and federated learning, power systems. NOMENCLATURE Abbreviations DDL Distributed deep learning DL Deep learning DRL Deep reinforcement learning DRN Deep residual network EI Edge intelligence EPS Electric power systems FL Federated learning IoT Internet of things LSTM Long short-term memory neural network The associate editor coordinating the review of this manuscript and approving it for publication was Shadi Alawneh . NN Neural network PVPF Photovoltaic power forecasting D. RESEARCH METHODOLOGY AND SYSTEMATIC REVIEW PROTOCOLStarting from September 2019, the multiple-methods approach was conducted [24]. The collection of the mainstream research papers on SG/AI from Web of Science (WoS), Scopus, IEEE Xplore, Science Direct, and Google scholar was conducted as the largest databases of peerreviewed articles. Only peer-reviewed articles written in English, providing experimental results, and having a unique identifier from the mentioned databases were taken into consideration, including reviews, research articles, patent reports, and conference proceedings. The adopted methodology for conducting this review article employs a combination of keywords categorized into three main groups, specifically, 'Deep Learning', 'Smart Grid', and 'Prediction'. The search methodology focuses on the recent research articles from 2015-2020 to identify the comprehensive statues of the AI applications on SG. The filtering process results in 220 research papers from 600 related papers selected based on their relevance by reading the title, abstract, conclusion,

show abstract

“…DRL is the combination of DNNs and RL [98]. The basic idea behind DRL consists of assigning rewards and punishments for an agent to shape its policies.…”

Section: ) Deep Reinforcement Learningmentioning

confidence: 99%

Deep Learning in Smart Grid Technology: A Review of Recent Advancements and Future Prospects

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Various approaches have been proposed to solve the local optimal problems under multiple extremum points. Studies (Ram et al, 2017;Alik and Jusoh, 2018;Cao et al, 2020b) have presented the improved searching methods of GMPPT techniques under mismatch conditions based on fuzzy logic control, artificial neural network, and particle swarm optimization methods. Although GMPPT shows the advantages of implementation simplicity, reduced cost, and immediate adoption, the severe power loss caused by frequent mismatch conditions remains unresolved.…”

Section: Introductionmentioning

confidence: 99%

Modeling and Energy Generation Evaluations of Large-Scale Photovoltaic Plants Equipped With Panel-Level DC Optimizers

Wang

et al. 2022

Front. Energy Res.

View full text Add to dashboard Cite

The distributed maximum power point tracking (DMPPT) technology, based on a DC optimizer (DCO, a DC/DC micro-converter) for each single photovoltaic (PV) panel, is one of the most popular solutions to mitigating the waste of solar energy when suffering mismatch conditions. However, the trade-off between the additional costs of deploying the panel-level power electronic equipment and the improved generation benefits of a large-scale PV plant (LPP) remains to be further studied. This study presents a static modeling method for the DCO-based distributed LPPs to study the long-term energy generation characteristics based on historical hourly weather data and then evaluate the economic benefits. The operational characteristics of the PV strings equipped with series-connected DCOs for three different topologies (Boost, Buck, and Buck-boost) are investigated, and then the control strategies for the PV-DCO generation units are proposed to maximize the energy generation of LPPs under frequent mismatch conditions. Different mismatch scenarios caused by the panel aging, geographical location settings, and the partial shading in PV arrays are simulated in the model. Six typical centralized or distributed PV plant configurations are carried out for comparison in case studies, to explore the generation characteristics and the advantages of energy production for the DCO-based distributed LPPs. Besides, the Levelized cost of energy (LCOE) which considers both the energy generation benefits and investment costs is introduced to the economic evaluation of different structures of LPPs.

show abstract

“…In contrast, other disciplines have consolidated the obtained research findings of deep RL and highlighted its adaptive behaviour and the ability to generalise past experiences. This includes communications and networking (Luong et al 2019), cyber-physical-systems (Liu et al 2019), economic applications (Mosavi et al 2020), internet of things (Lei et al 2020), object grasping (Mohammed, Chung, and Chyi Chua 2020), power and energy systems (Cao et al 2020), robotics (Khan et al 2020), robotic manipulations tasks (Nguyen and La 2019), and dynamic task scheduling (Shyalika, Silva, and Karunananda 2020), which reflects the broad range of research and underlines the ongoing focus on implementing deep RL applications to significantly increase the adaptability and robustness of the respecting processes.…”

Section: Introductionmentioning

confidence: 99%

Deep reinforcement learning in production systems: a systematic literature review

Panzer

Bender

2021

International Journal of Production Research

126

View full text Add to dashboard Cite

Shortening product development cycles and fully customisable products pose major challenges for production systems. These not only have to cope with an increased product diversity but also enable high throughputs and provide a high adaptability and robustness to process variations and unforeseen incidents. To overcome these challenges, deep Reinforcement Learning (RL) has been increasingly applied for the optimisation of production systems. Unlike other machine learning methods, deep RL operates on recently collected sensor-data in direct interaction with its environment and enables real-time responses to system changes. Although deep RL is already being deployed in production systems, a systematic review of the results has not yet been established. The main contribution of this paper is to provide researchers and practitioners an overview of applications and to motivate further implementations and research of deep RL supported production systems. Findings reveal that deep RL is applied in a variety of production domains, contributing to data-driven and flexible processes. In most applications, conventional methods were outperformed and implementation efforts or dependence on human experience were reduced. Nevertheless, future research must focus more on transferring the findings to real-world systems to analyse safety aspects and demonstrate reliability under prevailing conditions.

show abstract

Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review

Cited by 262 publications

References 79 publications

Deep Learning in Smart Grid Technology: A Review of Recent Advancements and Future Prospects

Deep Learning in Smart Grid Technology: A Review of Recent Advancements and Future Prospects

Modeling and Energy Generation Evaluations of Large-Scale Photovoltaic Plants Equipped With Panel-Level DC Optimizers

Deep reinforcement learning in production systems: a systematic literature review

Contact Info

Product

Resources

About