A Restless Bandit Model for Resource Allocation, Competition, and Reservation

Abstract

In “A Restless Bandit Model for Resource Allocation, Competition and Reservation,” J. Fu, B. Moran, and P. G. Taylor study a resource allocation problem with varying requests and with resources of limited capacity shared by multiple requests. This problem is modeled as a set of heterogeneous restless multi-armed bandit problems (RMABPs) connected by constraints imposed by resource capacity. Following Whittle’s idea of relaxing the constraints and Weber and Weiss’s proof of asymptotic optimality, the authors propose an index policy and establish conditions for it to be asymptotically optimal in a regime where both arrival rates and capacities increase. In particular, they provide a simple sufficient condition for asymptotic optimality of the policy and, in complete generality, propose a method that generates a set of candidate policies for which asymptotic optimality can be checked. Via numerical experiments, they demonstrate the effectiveness of these results even in the pre-limit case.

Energy Efficient Priority-Based Task Scheduling for Computation Offloading in Fog Computing

“…As in [4, Corollary 1], an index policy that prioritizes the process with the highest indices is asymptotically optimal as h → ∞. The proposed PIER, while specifically designed for the TOSP, is in the same vein as priority-based policies proposed in [4,5,13], which have been shown to approach the optimal solution in stochastic optimization problems as the scale of the system is sufficiently large. Particularly, when J = 1, K = L (that is, there is only one ARC group in every destination area at the network edge), Γ 0 j,ℓ = 1 for all j ∈…”

“…We refer to the optimization problem with the objective function (9) and constraints (1) to ( 3) and ( 6) as the task offloading scheduling problem (TOSP). The TOSP forms an instance of the resource allocation problem discussed in [4], which consists of parallel restless multi-armed bandit problems (RMABPs) coupled by capacity constraints. A policy φ is considered better if it produces a smaller value of the objective function (9).…”

“…While similar ideas have been applied in our previous work [4,5,13], we now consider a tailor-made model for the offloading problem in fog computing, where a task may be offloaded to an FN at the network edge or the central cloud, subject to the availability of both computing resources and wireless transmission channels. We will also consider that the power consumption at the network edge is, in fact, a piece-wise function of the load, as an "idle power" would be needed to activate the hardware equipment supporting a resource group when at least one unit of the resources is occupied [9].…”

A Restless Bandit Model for Energy-Efficient Job Assignments in Server Farms

“…• Following the ideas of Whittle relaxation technique [30] and the asymptotic optimality proofs of [24], [37], when job sizes are exponentially distributed, under a mild condition related to the service and energy consumption rates of physical components, we prove that the index policy becomes optimal as the job arrival rates and the number of physical components in each cluster tend to be arbitrarily large proportionately; that is, it is asymptotically optimal. The asymptotic optimality is appropriate for computing/storage clusters with a large number of physical components.…”

“…Later in [37], Whittle index policy was proved to be asymptotically optimal under a non-trivial extra condition that requires the existence of a global attractor of a proposed process associated with the RMABP. Although the global attractor remains an open problem for a general RMABP, recently in [24], for a class of resource allocation problems, it was proved to exist and the Whittle index policy was proved to be asymptotically optimal. Along the same idea, in [21], a scalable job-assignment policy was proposed and proved to be asymptotically optimal for a server farm, where its servers were assumed to have only two power modes (corresponding to two power consumption rates).…”