Scheduling users in drive-thru Internet: a multi-armed bandit approach

Abstract

We consider the problem of allocating a wireless channel to mobile users moving on a straight road. The objective is to maximize a given function of the total data transmitted. We develop a model within the multi-armed bandit framework and we formulate an optimization problem under the constraint that only one user can be served at a time. We solve the relaxed optimization problem, in which one user is served on the average, for which the solution is given by Whittle's index policy, and we derive a heuristic policy for the original optimization problem using Whittle's index policy as well. We evaluate numerically to see how well the heuristic algorithm performs in various settings, including the dynamic scenario with arrivals of new users, and in the presence of heterogeneous users.Index terms-Markov Decision Process, restless multi-armed bandit problem, Whittle's index, scheduling, drive-thru internet * This is an extended version of the work that appears in Wiopt 2019.