Whittle Index Policy for Multichannel Scheduling in Queueing Systems

Abstract

In this paper, we consider a queueing system with multiple channels (or servers) and multiple classes of users. We aim at allocating the available channels among the users in such a way to minimize the expected total average queue length of the system. This known scheduling problem falls in the framework of Restless Bandit Problems (RBP) for which an optimal solution is known to be out of reach for the general case. The contributions of this paper are as follows. We rely on the Lagrangian relaxation method to characterize the Whittle index values and to develop an index-based heuristic for the original scheduling problem. The main difficulty lies in the fact that, for some queue states, deriving the Whittle's index requires introducing a new approach which consists in introducing a new expected discounted cost function and deriving the Whittle's index values with respect to the discount parameter β. We then deduce the Whittle's indices for the original problem (i.e. with total average queue length minimization) by taking the limit β → 1. The numerical results provided in this paper show that this policy performs very well and is very close to the optimal solution for high number of users.

On the Optimality of the Whittle’s Index Policy for Minimizing the Age of Information

“…Next, by replacing q k j,i (z(t)) with its value in (23), we can obtain the following relationship between z(t + 1) and z(t) for any z(t) ∈  w * . 1) k = m: In this case, z k i (t + 1) will have the following expression:…”

“…The Whittle's index policy revolves around: 1) assigning an index to each user based on its age and channel statistics, 2) scheduling the M users with the highest indices. This policy has been previously adopted for a large variety of scenarios (e.g., delay minimization [23], [24] and throughput maximization [25]- [27]). The main interest in this policy stems from its simplicity and notable performance [23]- [28].…”

“…This policy has been previously adopted for a large variety of scenarios (e.g., delay minimization [23], [24] and throughput maximization [25]- [27]). The main interest in this policy stems from its simplicity and notable performance [23]- [28]. However, the analysis of its performance is known to be challenging and, accordingly, remains elusive in the average age minimization framework.…”

Performance analysis of wireless network aided by discrete-phase-shifter IRS

“…To that extent, we suppose that the queue length of a given node does not exceed a certain constant denoted by L, which is in its turn less than the departure rate. Besides that, unlike our work in [1], we further prove the asymptotic optimality of our developed policy in the many user's regime.…”

“…We provide an explicit characterization of the Whittle's indices, develop a Whittle's index channel allocation policy for our problem. On the other hand, in contrast to previous work in [1], in this paper, we consider that the buffer size is very tight. The motivation behind that is that in the framework of IoT (Internet of things), the resources of the connected devices are very limited, especially the energy available and the backlog queue.…”