Planning for Package Deliveries in Risky Environments Over Multiple Epochs

Abstract

We study a risk-aware robot planning problem where a dispatcher must construct a package delivery plan that maximizes the expected reward for a robot delivering packages across multiple epochs. Each package has an associated reward for delivery and a risk of failure. If the robot fails while delivering a package, no future packages can be delivered and the cost of replacing the robot is incurred. The package delivery plan takes place over the course of either a finite or an infinite number of epochs, denoted as the finite horizon problem and infinite horizon problem, respectively. The dispatcher has to weigh the risk and reward of delivering packages during any given epoch against the potential loss of any future epoch's reward. By using the ratio between a package's reward and its risk of failure, we prove an optimal, greedy solution to both the infinite and finite horizon problems. The finite horizon problem can be solved optimally in O(Kn log n) time where K is the number of epochs and n is the number of packages. We show an isomorphism between the infinite horizon problem and Markov Decision Processes to prove an optimal O(n) time algorithm for the infinite horizon problem.