Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: Simulations and experiments

Svetozarevic, Bratislav; Baumann, Cindy A; Muntwiler, Simon; Natale, Loris Di; Zeilinger, Melanie N.; Heer, Philipp

doi:10.1016/j.apenergy.2021.118127

Cited by 28 publications

(10 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most used RL technique is Q-learning [11,12]. Svetozarevic et al [13] presented a study where they implemented the RL method to smart home to control EV charging/discharging and room temperature while minimizing energy costs and maintaining thermal comfort simultaneously. They used historical data from a building and weather to obtain the optimal control policy without complex physical modelling of the building.…”

Section: Reinforcement Learningmentioning

confidence: 99%

A Review – Home Renewable Energy Management Systems in Smart Grids

Kallio

Siroux

2022

IOP Conf. Ser.: Earth Environ. Sci.

View full text Add to dashboard Cite

This paper presents a review on the Home Energy Management Systems (HEMS) for renewable energy production and used optimization methods. The HEMS is an important Smart Grid application. It is used to monitor and optimally manage the energy flows in buildings including renewable energy production, energy storage and smart home appliances. In this paper, two different methods for the optimal HEMS are selected and compared: Model Predictive Control (MPC) and Reinforcement Learning (RL). As a conclusion, the RL method can overcome the disadvantages of the MCP in the highly dynamic environment of buildings and renewable energies, and is a promising method for HEMS in Smart Grids. Finally, an experimental set-up of the hybrid renewable energy system is presented and its operation is discussed under the Time-of-Use energy management strategy.

show abstract

Section: Reinforcement Learningmentioning

confidence: 99%

A Review – Home Renewable Energy Management Systems in Smart Grids

Kallio

Siroux

2022

IOP Conf. Ser.: Earth Environ. Sci.

View full text Add to dashboard Cite

show abstract

“…Application Objective Building Type Algorithm [141], [142] Other/Mixed Cost Residential DQN [143] Cost & Load Balance [94] EV, ES, and RG Cost [144] Other [145] Cost & Comfort [146] HVAC, Fans, WH Cost [147] Other/Mixed Commercial [148] Cost & Comfort [149], [150] HVAC, Fans, WH Mixed/NA [151], [152] Other/Mixed Cost [153], [154] P2P Trading Other Mixed/NA [163] EV, ES, and RG [164] Other/Mixed Cost [165] Cost & Comfort Residential TRPO [51], [168], [169], [170] Other/Mixed [171], [172] Cost & Load Balance [173] Cost [174] EV, ES, and RG [175] Other/Mixed Cost & Comfort Academic [176] Other [177], [178] EV, ES, and RG Commercial [179], [180], [181] HVAC, Fans, WH Cost & Comfort Mixed/NA [182], [183], [184] EV, ES, and RG Other [185], [186] Other/Mixed Cost & Load Balance Residential SAC [187], [188] HVAC, Fans, WH Cost Commercial [189], [190],…”

Section: Referencementioning

confidence: 99%

Reinforcement Learning: Theory and Applications in HEMS

Alani¹,

Das²

2022

Preprint

View full text Add to dashboard Cite

The twin capabilities of learning from experience and learning at higher levels of abstraction, set reinforcement learning apart from other areas of machine learning and (within the broader context) all of artificial intelligence. It allows algorithmic agents to replace human beings in the real world, including in homes and buildings, in application domains that had hitherto been considered to be beyond today’s capabilities. This goal, specifically aimed at home energy automation that forms the backdrop of this article, which surveys the use of deep reinforcement learning in various HEMS applications. The article provides an overview of generic reinforcement learning. This is followed with discussions on the state-of-the-art methods for value based, policy gradient, and actor-critic methods in deep reinforcement learning. In order to make published literature in reinforcement learning more accessible to HEMS researchers, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using the same terminology as the machine learning community. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. The survey suggests that this research is still in its infancy.

show abstract

“…A safe DRL method for the EV charging/discharging problem was developed in [27], wherein the constrained optimal EV charging/discharging schedules are calculated using a deep neural network without defining a penalty term and adjusting its coefficient. In [28], a DRL model employing the deep deterministic policy gradient (DDPG) method was presented to jointly control the room temperature of households and bidirectional EV charging to minimize the electricity cost. A scalable DRL approach for EV routing was proposed in [29], wherein the EV routing problem within a time window was solved on the basis of an attention model incorporating a pointer network and a graph layer to parameterize the stochastic policy of a DRL agent.…”

Section: Introductionmentioning

confidence: 99%

Safety-Integrated Online Deep Reinforcement Learning for Mobile Energy Storage System Scheduling and Volt/VAR Control in Power Distribution Networks

2023

View full text Add to dashboard Cite

In coupled power distribution and transportation (CPT) system, a joint scheduling framework for mobile energy storage systems (MESSs) and Volt/VAR control (VVC) ensures reliable power distribution grid operations while supporting electric vehicle loads at electric vehicle charging stations (EVCSs). However, conventional model-based optimization methods for MESS scheduling and VVC may yield suboptimal solutions and greater computation times because of MESS operation and VVC in uncertain environment of CPT systems. To resolve this issue, this study proposes a model-free deep reinforcement learning (DRL) framework. In this framework, smart inverters of MESSs and solar photovoltaic (PV) systems cooperate to minimize the real power loss and mitigate the violations of both MESSs' state of charge (SOC) and voltage in the power distribution network, while MESSs travel via the transportation network to satisfy EV loads at EVCSs. A MESS routing algorithm based on Dijkstra's algorithm is developed to determine the optimal destinations of the MESSs. In addition, two safety modules are developed to ensure that neither SOC nor voltage violations occur by adjusting real and/or reactive power of MESSs and PV systems during the training process. The developed MESS routing algorithm and safety modules are integrated into the proposed DRL framework, wherein the DRL agent performs the desired MESS scheduling and VVC through safe exploration during the training procedure. The proposed approach is tested in coupled IEEE 33-bus power distribution and 15-node transportation systems and coupled IEEE 57bus power distribution and 42-node transportation systems. Numerical examples demonstrate the advantages of the proposed approach in terms of training convergence, real power loss, and SOC/voltage violation. INDEX TERMS Deep reinforcement learning, safe exploration, mobile energy storage system, Volt/VAR control, smart inverter, coupled power distribution and transportation system.

show abstract

Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: Simulations and experiments

Cited by 28 publications

References 47 publications

A Review – Home Renewable Energy Management Systems in Smart Grids

A Review – Home Renewable Energy Management Systems in Smart Grids

Reinforcement Learning: Theory and Applications in HEMS

Safety-Integrated Online Deep Reinforcement Learning for Mobile Energy Storage System Scheduling and Volt/VAR Control in Power Distribution Networks

Contact Info

Product

Resources

About