Effective Charging Planning Based on Deep Reinforcement Learning for Electric Vehicles

Zhang, Cong; Liu, Yuanan; Wu, Fan; Tang, Bihua; Fan, Wenhao

doi:10.1109/tits.2020.3002271

Cited by 96 publications

(29 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, compared to previous RL works, the prior was neglected in ( Zhang Q. et al, 2020 ). The latter was considered in ( Zhang C. et al, 2020 ) through estimating energy consumption from rarely available historical data. Second, a trained RL agent requires less computing effort and less memory space than model-based techniques and mixed integer non-linear programming formulations ( Mocanu et al, 2018 ) of the EV routing problem such as ( Pourazarm et al, 2014 ).…”

Section: Discussionmentioning

confidence: 99%

“…Additionally, researchers in ( Zhang Q. et al, 2020 ) used actor-critic RL to minimize the route’s energy consumption without recharging opportunities. In ( Zhang C. et al, 2020 ), a deep RL approach was proposed to reduce both travel time and distance while different charging modes and occupation of charging spots were considered. In this research, we formulate the EV-specific routing problem in a graph-theoretical setting as a Markov decision process (MDP) and suggest a possible model-free RL algorithm to solve it by generating energy feasible paths for EV from source to target.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Routing of Electric Vehicles With Intermediary Charging Stations: A Reinforcement Learning Approach

2021

View full text Add to dashboard Cite

In the past few years, the importance of electric mobility has increased in response to growing concerns about climate change. However, limited cruising range and sparse charging infrastructure could restrain a massive deployment of electric vehicles (EVs). To mitigate the problem, the need for optimal route planning algorithms emerged. In this paper, we propose a mathematical formulation of the EV-specific routing problem in a graph-theoretical context, which incorporates the ability of EVs to recuperate energy. Furthermore, we consider a possibility to recharge on the way using intermediary charging stations. As a possible solution method, we present an off-policy model-free reinforcement learning approach that aims to generate energy feasible paths for EV from source to target. The algorithm was implemented and tested on a case study of a road network in Switzerland. The training procedure requires low computing and memory demands and is suitable for online applications. The results achieved demonstrate the algorithm’s capability to take recharging decisions and produce desired energy feasible paths.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Routing of Electric Vehicles With Intermediary Charging Stations: A Reinforcement Learning Approach

2021

View full text Add to dashboard Cite

show abstract

“…Its core idea is to achieve the maximum return or achieve specific goals through learning strategies during the interaction between the agent and the environment, so that the agent has the ability to make optimal decisions. Considering that the transformation process of reinforcement learning between environments is very complicated, in order to simplify the reinforcement learning modeling problem, the Markov decision process (MDP) is proposed to describe and model the reinforcement learning process [39]. The principle of reinforcement learning is shown in Figure 2.…”

Section: Reinforcement Learningmentioning

confidence: 99%

“…The principle of reinforcement learning is shown in Figure 2. The specific description is as follows cated, in order to simplify the reinforcement learning modeling problem, the Markov decision process (MDP) is proposed to describe and model the reinforcement learning process [39]. The principle of reinforcement learning is shown in Figure 2.…”

Section: Reinforcement Learningmentioning

confidence: 99%

Improved Deep Q-Network for User-Side Battery Energy Storage Charging and Discharging Strategy in Industrial Parks

Chen

Jiang

et al. 2021

Entropy

View full text Add to dashboard Cite

Battery energy storage technology is an important part of the industrial parks to ensure the stable power supply, and its rough charging and discharging mode is difficult to meet the application requirements of energy saving, emission reduction, cost reduction, and efficiency increase. As a classic method of deep reinforcement learning, the deep Q-network is widely used to solve the problem of user-side battery energy storage charging and discharging. In some scenarios, its performance has reached the level of human expert. However, the updating of storage priority in experience memory often lags behind updating of Q-network parameters. In response to the need for lean management of battery charging and discharging, this paper proposes an improved deep Q-network to update the priority of sequence samples and the training performance of deep neural network, which reduces the cost of charging and discharging action and energy consumption in the park. The proposed method considers factors such as real-time electricity price, battery status, and time. The energy consumption state, charging and discharging behavior, reward function, and neural network structure are designed to meet the flexible scheduling of charging and discharging strategies, and can finally realize the optimization of battery energy storage benefits. The proposed method can solve the problem of priority update lag, and improve the utilization efficiency and learning performance of the experience pool samples. The paper selects electricity price data from the United States and some regions of China for simulation experiments. Experimental results show that compared with the traditional algorithm, the proposed approach can achieve better performance in both electricity price systems, thereby greatly reducing the cost of battery energy storage and providing a stronger guarantee for the safe and stable operation of battery energy storage systems in industrial parks.

show abstract

“…A distributed architecture to suggest a set of the nearest charging stations for EV recharging using a distributed ant system algorithm [11]. Zhang, et al [12], authors propose a formalization of the EV charging scheduling problem as a markov decision process and deep reinforcement learning algorithms to reduce the EVs total charging time. To minimize the total elapsed time by considering the traffic condition, the remaining energy in the battery, and the CS queuing length, the EV charging scheduling method is proposed in [13].…”

Section: Introductionmentioning

confidence: 99%

A smart management system of electric vehicles charging plans on the highway charging stations

El-Fedany

Kiouach

Alaoui

2021

IJEECS

View full text Add to dashboard Cite

Electric vehicles (EVs) are seen as one of the principal pillars of smart transportation to relieve the airborne pollution induced by fossil CO2 emissions. However, the battery limit, especially where the journey is with a long-distance road remains the most formidable obstacle to the large-scale use of EVs. To overcome the issue of prolonged waiting charging time due to the large number of EVs may have a charging plan at the same charging station (CS) along the highway, we propose a communication system to manage the EVs charging demands. The architecture system contains a smart scheduling algorithm to minimize trip time including waiting time, previous reservations and energyare needed to reach the destination. Moreover, an automatic mechanism for updating reservation is integrated to adjust the EVs charging plans. The results of the evaluation under the Moroccan highway scenario connecting Rabat and Agadir show the effectiveness of our proposal system.<br /><div> </div>

show abstract

Effective Charging Planning Based on Deep Reinforcement Learning for Electric Vehicles

Cited by 96 publications

References 40 publications

Routing of Electric Vehicles With Intermediary Charging Stations: A Reinforcement Learning Approach

Routing of Electric Vehicles With Intermediary Charging Stations: A Reinforcement Learning Approach

Improved Deep Q-Network for User-Side Battery Energy Storage Charging and Discharging Strategy in Industrial Parks

A smart management system of electric vehicles charging plans on the highway charging stations

Contact Info

Product

Resources

About