Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching

Abstract

Improving the e ciency of dispatching orders to vehicles is a research hotspot in online ride-hailing systems. Most of the existing solutions for order-dispatching are centralized controlling, which require to consider all possible matches between available orders and vehicles. For large-scale ride-sharing platforms, there are thousands of vehicles and orders to be matched at every second which is of very high computational cost. In this paper, we propose a decentralized execution order-dispatching method based on multi-agent reinforcement learning to address the large-scale order-dispatching problem. Di erent from the previous cooperative multi-agent reinforcement learning algorithms, in our method, all agents work independently with the guidance from an evaluation of the joint policy since there is no need for communication or explicit cooperation between agents. Furthermore, we use KL-divergence optimization at each time step to speed up the learning process and to balance the vehicles (supply) and orders (demand). Experiments on both the explanatory environment and real-world simulator show that the proposed method outperforms the baselines in terms of accumulated driver income (ADI) and Order Response Rate (ORR) in various tra c environments. Besides, with the support of the online platform of Didi Chuxing, we designed a hybrid system to deploy our model. CCS CONCEPTS•Computing methodologies → Multi-agent reinforcement learning; • eory of computation → Multi-agent reinforcement learning; •Applied computing → Transportation;

Zone-Agnostic Greedy Taxi Dispatch Algorithm Based on Contextual Matching Matrix for Efficient Maximization of Revenue and Profit

“…Most recent ride-hailing platforms either match a driver to the closest passenger call [8][9][10] or use queueing strategies for firstcome-first-served dispatching [6,7]. Zhou et al proposed a model which dispatches taxis in such a way that demands and supplies are balanced based on demand prediction [3]. Furthermore, Xu et al used a recurrent neural network (RNN) and captured complex spatio-temporal features to predict taxi demand [14].…”

“…Recent approaches divide a region into a grid of cells and match vacant taxis with the passenger calls that take place within the same cell to maximize the driver's profit and the match rate [2,3]. Liu et al proposed a road-connectivity-aware zone that was formed by clustering the road network graph as opposed to having a grid of rectangular and hexagonal cells [4].…”

“…Second, the service area can be divided into high-demand sub-area clusters, a grid of square or hexagonal cells, or administrative divisions. We can opt to match a taxi with passengers only within the same zone, which is a prevalent method in most of the recent works [2][3][4] implemented. Alternatively, we can have a zone-agnostic approach that does not limit the assignment of the taxis to the passenger calls within an arbitrary boundary.…”

Is Reinforcement Learning the Choice of Human Learners?

“…Urban Computing integrates urban sensing, data management, and data analytic as a unified process to explore, analyze, and solve problems related to people's everyday life [7, 10-12, 28, 30, 33, 34, 38, 53, 56, 58, 60]. In particular, a group of works have studied the topic of taxi operation [8,9,21,32,35,42,46,51], such as vehicle dispatching with reinforcement learning [17,18,23,24,27,39,43,46,49,61], and passenger-seeking strategies [14,19,36,54,55,57]. They aim to find optimal solutions to improve the revenue of individual taxi drivers as well as the entire fleet.…”

Multi-Objective Predictive Taxi Dispatch via Network Flow Optimization

“…We model an urban transportation system of interest as a flow network, which is a useful approach introduced to manage mobility-on-demand platforms [32,38,39]. In particular, we model the transportation system as a grid-world consisting of homogeneous regular hexagonal cells as in [18,27,40]. Each hexagonal cell represents the coverage of a vehicle in a unit time interval.…”