MOVI: A Model-Free Approach to Dynamic Fleet Management

Abstract

Modern vehicle fleets, e.g., for ridesharing platforms and taxi companies, can reduce passengers' waiting times by proactively dispatching vehicles to locations where pickup requests are anticipated in the future. Yet it is unclear how to best do this: optimal dispatching requires optimizing over several sources of uncertainty, including vehicles' travel times to their dispatched locations, as well as coordinating between vehicles so that they do not attempt to pick up the same passenger. While prior works have developed models for this uncertainty and used them to optimize dispatch policies, in this work we introduce a model-free approach. Specifically, we propose MOVI, a Deep Qnetwork (DQN)-based framework that directly learns the optimal vehicle dispatch policy. Since DQNs scale poorly with a large number of possible dispatches, we streamline our DQN training and suppose that each individual vehicle independently learns its own optimal policy, ensuring scalability at the cost of less coordination between vehicles. We then formulate a centralized receding-horizon control (RHC) policy to compare with our DQN policies. To compare these policies, we design and build MOVI as a large-scale realistic simulator based on 15 million taxi trip records that simulates policy-agnostic responses to dispatch decisions. We show that the DQN dispatch policy reduces the number of unserviced requests by 76% compared to without dispatch and 20% compared to the RHC approach, emphasizing the benefits of a model-free approach and suggesting that there is limited value to coordinating vehicle actions. This finding may help to explain the success of ridesharing platforms, for which drivers make individual decisions.

A Probabilistic Approach for Demand-Aware Ride-Sharing Optimization

“…Second, upon change in vehicle status, e.g., a user is delivered to the destination or an empty car picks up a new user, the system can re-optimize the routing decisions and request-vehicle assignments, so as to optimize the long-term ride-sharing performance in a greedy fashion. The overall solution can serve as a baseline for other demand-aware studies, e.g., the conceivable ones by extending the approach in [13] and [27] to the multi-rider setting and the recent one in [5]. We leave the performance analysis of the overall greedy solution, as well as developing solutions with optimized long-term performance, as interesting and important future directions.…”

DeepPool: Distributed Model-Free Algorithm for Ride-Sharing Using Deep Reinforcement Learning

“…Our design exploits these two features to manage dispatching vehicles and thus improving ridesharing services. Several previous studies show that it is possible to learn from past taxi data and thus organizing the taxi fleet and minimizing the wait-time for passengers and drivers [41], [42], [43], [27], [15], [44], [45]. Nishant et.…”

“…In [48], authors use Deep learning to solve traffic problems, such as travel mode choice predication. In [15], authors used DQN based approach for dynamic fleet management and show that distributed DQN based approaches can outperform the centralized ones. However, this approach is only for dispatching the vehicles and does not consider ride-sharing.…”

“…Note that the optimal route can be easily obtained. We note that in [15], there is no provision of car-pooling, hence a system optimizer does not need to consider the objective ∆ t as it is always zero. However, we have to judiciously select the action in order to minimize the objective.…”

“…DeepPool ensures scalability by adopting a modular architecture and using deep neural networks for training such that each individual vehicle learns its own optimal decision independently. • Evaluation results show the superiority of our approach (DeepPool) as compared to the state-of-the art algorithms, i.e., [15] and some considered baselines. Further, empirical results show that our approach can significantly reduce the number of operating vehicles (by at least 500 vehicles) for the same request acceptance rate and the same wait time of the passengers.…”