An Integrated Reinforcement Learning and Centralized Programming Approach for Online Taxi Dispatching

Abstract

Balancing the supply and demand for ride-sourcing companies is a challenging issue, especially with real-time requests and stochastic traffic conditions of large-scale congested road networks. To tackle this challenge, this paper proposes a robust and scalable approach that integrates reinforcement learning (RL) and a centralized programming (CP) structure to promote real-time taxi operations. Both real-time order matching decisions and vehicle relocation decisions at the microscopic network scale are integrated within a Markov decision process framework. The RL component learns the decomposed statevalue function, which represents the taxi drivers' experience, the off-line historical demand pattern, and the traffic network congestion. The CP component plans non-myopic decisions for drivers collectively under the prescribed system constraints to explicitly realize cooperation. Further, to circumvent sparse reward and sample imbalance problems over the microscopic road network, this paper proposed temporal-difference learning algorithm with prioritized gradient descent and adaptive exploration techniques.A simulator is built and trained with the Manhattan road network and New York City yellow taxi data to simulate the real-time vehicle dispatching environment. Both centralized and decentralized taxi dispatching policies are examined with the simulator. This case study shows that the proposed approach can further improve taxi drivers' profits while reducing customers' waiting times compared to several existing vehicle dispatching algorithms.

Multi-agent reinforcement learning for fleet management: a survey

“…Commonly, the map is discretized into hexagon girds, therefore the location of order or vehicle is expressed by the index of gird [13]. [14] applies a link-node-based micro-network representation, leveraging the heterogeneous traffic network topology. Besides, it's necessary to append the timestep t into the definition since the distribution of orders or vehicles may strongly have a temporal characteristic, e.g., morning rush hour and evening rush hour.…”

“…To be worse, the complicated road in urban may deteriorate the planning and lead to infeasible operation strategies. [14] builds a link-node-based micro-network representation and have successfully applied MARL on it, which is prospective.…”

“…According our survey, the learning and planning approach [8,9,15], and its variants [14,21], have been widely used in fleet management. Based on historical data, the off-line learning step enables the model to predict the distribution of demand and supply, therefore capable of arranging the fleets to fill the gap in advance.…”

Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey

“…Guo et al [8], [82] use a double DQN based framework to solve the fleet management problem ahead and leave the detailed Order Matching to the traditional Khun-Munkres (KM) algorithm. Liang et al [83] preserve the topology of the initial graph-based supply-demand distribution structure instead of discretizing them using a grid view. A special centralized programming planning module is developed to dispatch thousands of taxis on a real-time basis.…”

Sequential Information Design: Markov Persuasion Process and Its Efficient Reinforcement Learning

“…For example, a ride-sharing platform holds historical and real-time data on active riders and driver types in different locations, based on which they have developed centralized combinatorial optimization algorithms and reinforcement learning algorithms for vehicle repositioning, routing and order matching to optimize their operational efficiency and profit [33,42,34,43]. But the de facto decision makers are the drivers.…”