Navigating Electric Vehicles Along a Signalized Corridor via Reinforcement Learning: Toward Adaptive Eco-Driving Control

Abstract

One problem associated with the operation of electric vehicles (EVs) is the limited battery, which cannot guarantee their endurance. The increasing electricity consumption will also impose a burden on economy and ecology of the vehicles. To achieve energy saving, this paper proposes an adaptive eco-driving method in the environment of signalized corridors. The framework with adaptive and real-time control is implemented by the reinforcement learning technique. First, the operation of EVs in the proximity of intersections is defined as a Markov Decision Process (MDP) to apply the twin delayed deep deterministic policy gradient (TD3) algorithm, to deal with the decision process with continuous action space. Therefore, the speed of the vehicle can be adjusted continuously. Second, safety, traffic mobility, energy consumption, and comfort are all considered by designing a comprehensive reward function for the MDP. Third, the simulation study takes Aoti Street in Nanjing City with several consecutive signalized intersections as the research road network, and the state representation in MDP considers the information from consecutive downstream traffic signals. After the parameter tuning procedure, simulations are carried out for three typical eco-driving scenarios, including free flow, car following, and congestion flow. By comparing with default car-following behavior in the simulation platform SUMO and several state-of-the-art deep reinforcement learning algorithms, the proposed strategy shows a balanced and stable performance.

Eco-driving for Electric Connected Vehicles at Signalized Intersections: A Parameterized Reinforcement Learning approach

“…It means that the value of instantaneous energy consumption e i t is negative when the vehicle perform a braking operation, which presents challenges to searching a globally optimal eco-driving policy. The specific form and parameters of the energy model are given in our previous article (Zhang et al, 2022). The proposed PRL policy do not need to explicitly analysis the energy model, while it only receive a transient signal that indicates the energy consumption in a model-agnostic manner.…”