Optimising residential electric vehicle charging under renewable energy: Multi‐agent learning in software simulation and hardware‐in‐the‐loop evaluation

Abstract

Summary The integration of intermittent renewable energy sources coupled with the increasing demand of electric vehicles (EVs) poses new challenges to the electrical grid. To address this, many solutions based on demand response have been presented. These solutions are typically tested only in software‐based simulations. In this paper, we present the application in hardware‐in‐the‐loop (HIL) of a recently proposed algorithm for decentralised EV charging, prediction‐based multi‐agent reinforcement learning (P‐MARL), to the problem of optimal EV residential charging under intermittent wind power and variable household baseload demands. P‐MARL is an approach that can address EV charging objectives in a demand response aware manner, to avoid peak power usage while maximising the exploitation of renewable energy sources. We first train and test our algorithm in a residential neighbourhood scenario using GridLAB‐D, a software power network simulator. Once agents learn optimal behaviour for EV charging while avoiding peak power demand in the software simulator, we port our solution to HIL while emulating the same scenario, in order to decrease the effects of agent learning on power networks. Experimental results carried out in a laboratory microgrid show that our approach makes full use of the available wind power, and smooths grid demand while charging EVs for their next day's trip, achieving a peak‐to‐average ration of 1.67, down from 2.24 in the baseline case. We also provide an analysis of the additional demand response effects observed in HIL, such as voltage drops and transients, which can impact the grid and are not observable in the GridLAB‐D software simulation.

EV Smart Charging in Distribution Grids–Experimental Evaluation Using Hardware in the Loop Setup

“…The HIL simulation results show that an 88-97% network capacity usage rate is achieved. Similarly, a decentralised demand response based EV charging algorithm for residential community applications is proposed and inspected with both software simulation and HIL simulation in research [30]. The results not only prove the efficacy of the algorithm but also signify that the high resolution contributed by HIL can reveal phenomena that can hardly be observed in pure software simulations, such as power consumption difference due to hardware losses, voltage bounce and dip caused by power variation and transient features of the grid [30].…”

“…The application of HIL in investigating the innovative EV charging algorithm has become more popular as it bridges pure computer simulation and actual implementation. The HIL simulation offers a flexible platform for algorithm/hardware interface testing [30]. Placing the relevant hardware components in the HIL testbed introduces factors that can potentially alter the algorithm's performance.…”

Multiagent reinforcement learning for strictly constrained tasks based on Reward Recorder

“…5,6 Outstanding research has achieved good evaluations in hybrid environments 7 or multiobjective tasks, 8 but it is only in recent years that MARL has been used in engineering applications. They mainly focus on scheduling and optimization problems in multiple engineering domains, such as traffic control, 9 autonomous driving, 10,11 base station communication, 12 load frequency optimization, 13 electric vehicle charging and discharging planning, 14 power allocation, 15 and so forth. In addition, as more complex problems are considered, more applicable systems are modeled and analyzed, such as Markov Repairable Systems, 16 Markov Jumping Systems, 17 and so forth.…”