Optimal Routing for Autonomous Taxis using Distributed Reinforcement Learning

Abstract

In this paper, a learning-based optimal transportation algorithm for autonomous taxis and ridesharing vehicles is introduced. The goal is to design a mechanism to solve the routing problem for a fleet of autonomous vehicles in real-time in order to maximize the transportation company's profit. To solve this problem, the system is modeled as a Markov Decision Process (MDP) using past customers data. By solving the defined MDP, a centralized high-level planning recommendation is obtained, where this offline solution is used as an initial value for the realtime learning. Then, a distributed SARSA reinforcement learning algorithm is proposed to capture the model errors and the environment changes, such as variations in customer distributions in each area, traffic, and fares, thereby providing an accurate model and optimal policies in real-time. Agents are using only their local information and interaction, such as current passenger requests and estimates of neighbors' tasks and their optimal actions, to obtain the optimal policies in a distributed fashion. The agents use the estimated values of each action, provided by distributed SARSA reinforcement learning, in a distributed game-theory based task assignment to select their conflict-free customers. Finally, the customers data provided by the city of Chicago is used to validate the proposed algorithms.

Distributed Adaptive Reinforcement Learning: A Method for Optimal Routing

“…. This is trivially satisfied by the original Q t update equation (16). Now, let's rewrite (22) for a specific state-action pair (i, π[i]).…”

“…IV-B, we focus on ride sharing and courier taxi service routing. As compared to our preliminary work presented in an eight-page-long workshop article [16], this paper include many revision in all the sections, including two additional mathematically-rigorous proofs of convergence, more complete proofs of the theorems, and an appendix detailing some math used in the main proofs.…”

An Intelligent Offloading System Based on Multiagent Reinforcement Learning

“…Classical scheduling algorithms, such as greedy methods, are widely used in large companies, such as finding the nearest driver to serve customers [22], or using a first-in, first-out queue strategy [23]; although they are easy to dispatch, it only obtained nice profits in the short term; the spatiotemporal sequence does not match the supply-demand relationship in the long-term operation, which will lead to some suboptimal results [15]. Later, this dispatching process improved by using the central system through the taxi GPS trajectory and brute force method for the best path recommendation [24,25], considering whether the driver took the initiative to find hot spots to provide the scheduling strategy [11] and focusing on minimizing total customer waiting time by simultaneously scheduling multiple taxis and allowing taxis to exchange their booking tasks [9], taking into account the overall benefits of a more global and far-sighted approach [26].…”

Exploring the Ridesharing Efficiency of Taxi Services

“…Dynamic or demand-driven ridesharing services are paid more attention in recent studies. The dynamic ridesharing problem can be formulated as the dynamic vehicle routing problem (DVRP) [35] [36], in which the constraint that the order of the passing node in the current taxi schedule must keep intact when the next passenger joins in the taxi trip is required [11][37] [38]. Wang et al [33] proposed a ridesharing strategy that allowed a vehicle to change its route at most once while it was serving a passenger to respond to another ad hoc request.…”