Joint Charging and Relocation Recommendation for E-Taxi Drivers via Multi-Agent Mean Field Hierarchical Reinforcement Learning

Wang, Enshu; Ding, Rong; Yang, Zhaoxing; Jin, Haiming; Miao, Chenglin; Lu, Shan; Zhang, Fan; Qiao, Chunming; Wang, Xinbing

doi:10.1109/tmc.2020.3022173

Cited by 37 publications

(20 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, EVs have become an emerging choice in the modern transportation system due to their low-carbon emission and energy efficiency. Some efforts [1,8,14,[35][36][37][38][39]44] have been made for charging station recommendation for EVs. In particular, most studies [1,8,35,38,39] focus on recommending charging station locations for EV drivers with the goal of time.…”

Section: Related Workmentioning

confidence: 99%

“…Cao et al [1] introduce the charging reservation information into the vehicle-to-vehicle system to facilitate the location recommendation. Different from charging location recommendation problem, another line of works [14,36,37,44] investigate to handle more complicated scenarios, especially considering commercial benefits. Yuan et al [44] propose a charging strategy that allows an electric taxi to get partially charged to meet the dynamic passenger demand.…”

Section: Related Workmentioning

confidence: 99%

“…Yuan et al [44] propose a charging strategy that allows an electric taxi to get partially charged to meet the dynamic passenger demand. In particular, with the help of deep reinforcement learning which has been widely adopted for solving sequential decision-making problems, Wang et al [36] design a multi-agent mean field hierarchical reinforcement learning framework by regarding each electric taxi as an individual agent to provide charging and relocation recommendations for electric taxi drivers, so as to maximize the cumulative rewards of the number of served orders. However, formulating each EV driver as an agent is not suitable for our problem, since most charging requests of a day in our task are ad-hoc and from non-repetitive drivers.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, the "single point of failure" [25] in the centralized approach may fail the whole system [15]. As an alternative, an existing approach Wang et al [36] tried to model a small set of vehicles as multiple agents and maximize the cumulative reward in terms of the number of served orders. However, in our charging recommendation task, most charging requests are ad-hoc and from non-repetitive drivers, which renders it impossible to learn a dedicated policy for each individual EV.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning

Zhang

Líu

Wang

et al. 2021

Proceedings of the Web Conference 2021

View full text Add to dashboard Cite

Electric Vehicle (EV) has become a preferable choice in the modern transportation system due to its environmental and energy sustainability. However, in many large cities, EV drivers often fail to find the proper spots for charging, because of the limited charging infrastructures and the spatiotemporally unbalanced charging demands. Indeed, the recent emergence of deep reinforcement learning provides great potential to improve the charging experience from various aspects over a long-term horizon. In this paper, we propose a framework, named Multi-Agent Spatio-Temporal Reinforcement Learning (Master), for intelligently recommending public accessible charging stations by jointly considering various long-term spatiotemporal factors. Specifically, by regarding each charging station as an individual agent, we formulate this problem as a multi-objective multi-agent reinforcement learning task. We first develop a multi-agent actor-critic framework with the centralized attentive critic to coordinate the recommendation between geo-distributed agents. Moreover, to quantify the influence of future potential charging competition, we introduce a delayed access strategy to exploit the knowledge of future charging competition during training. After that, to effectively optimize multiple learning objectives, we extend the centralized attentive critic to multi-critics and develop a dynamic gradient re-weighting strategy to adaptively guide the optimization direction. Finally, extensive experiments on two real-world datasets demonstrate that Master achieves the best comprehensive performance compared with nine baseline approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning

Zhang

Líu

Wang

et al. 2021

Proceedings of the Web Conference 2021

View full text Add to dashboard Cite

show abstract

“…Particularly, the RL-based techniques have been applied to solve the practical problems in the ridehailing system, such as order dispatching (Li et al 2019;Qin et al 2020) and joint order dispatch and vehicle repositioning (Zhou et al 2019;Holler et al 2019;Al-Abbasi, Ghosh, and Aggarwal 2019;Haliem et al 2021), and most recently, option-based vehicle repositioning (Tang et al 2019;Jiao et al 2021). For the vehicle repositioning problems, we briefly summarize three key components for the framework design: (1) improvement on the policy-/value-based network (Jin et al 2019;Holler et al 2019;Wang et al 2020;Jiao et al 2021), (2) reward shaping (Tang et al 2019;Shou and Di 2020), and (3) state representation design (Al-Abbasi, Ghosh, and Aggarwal 2019;Tang et al 2019;Schmoll and Schubert 2020). Interested readers are referred to (Qin, Zhu, and Ye 2021) for a detailed review on RL in vehicle repositioning problem.…”

Section: Related Workmentioning

confidence: 99%

DROP: Deep relocating option policy for optimal ride-hailing vehicle repositioning

Qian¹,

Guo²,

Aggarwal³

2021

Preprint

View full text Add to dashboard Cite

In a ride-hailing system, an optimal relocation of vacant vehicles can significantly reduce fleet idling time and balance the supply-demand distribution, enhancing system efficiency and promoting driver satisfaction and retention. Model-free deep reinforcement learning (DRL) has been shown to dynamically learn the relocating policy by actively interacting with the intrinsic dynamics in large-scale ride-hailing systems. However, the issues of sparse reward signals and unbalanced demand and supply distribution place critical barriers in developing effective DRL models. Conventional exploration strategy (e.g., the -greedy) may barely work under such an environment because of dithering in low-demand regions distant from high-revenue regions. This study proposes the deep relocating option policy (DROP) that supervises vehicle agents to escape from oversupply areas and effectively relocate to potentially underserved areas. We propose to learn the Laplacian embedding of a time-expanded relocation graph, as an approximation representation of the system relocation policy. The embedding generates task-agnostic signals, which in combination with task-dependent signals, constitute the pseudo-reward function for generating DROPs. We present a hierarchical learning framework that trains a highlevel relocation policy and a set of low-level DROPs. The effectiveness of our approach is demonstrated using a custombuilt high-fidelity simulator with real-world trip record data. We report that DROP significantly improves baseline models with 15.7% more hourly revenue and can effectively resolve the dithering issue in low-demand areas.

show abstract

A Review on Smart Charging Approaches for Electric Vehicle

Haldorai,

Murugan

et al. 2024

EAI/Springer Innovations in Communication and Computing

View full text Add to dashboard Cite

Joint Charging and Relocation Recommendation for E-Taxi Drivers via Multi-Agent Mean Field Hierarchical Reinforcement Learning

Cited by 37 publications

References 33 publications

Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning

Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning

DROP: Deep relocating option policy for optimal ride-hailing vehicle repositioning

A Review on Smart Charging Approaches for Electric Vehicle

Contact Info

Product

Resources

About