Delay-Aware Two-Hop Cooperative Relay Communications via Approximate MDP and Stochastic Learning

Wang, Rui; Lau, V.K.N.

doi:10.1109/tit.2013.2279895

Cited by 33 publications

(20 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The computation complexity of the value functions W k (k = 1, ..., L) defined in (7) is huge due to the following two reasons. First, the maximum possible stage number L is usually huge, and the value functions for all stages should be evaluated, unlike the infinite-horizon MDP considered in most of the existing literature [18], [20], [35]. Secondly, the lower-bounds are further decoupled for each file and each cache node via a novel linear approximation method in Section IV-B.…”

Section: Low-complexity Scheduling Policymentioning

confidence: 99%

Joint Optimization of File Placement and Delivery in Cache-Assisted Wireless Networks With Limited Lifetime and Cache Space

Wang

Cui

et al. 2020

IEEE Trans. Commun.

Self Cite

View full text Add to dashboard Cite

In this paper, the scheduling of downlink file transmission in one cell with the assistance of cache nodes with finite cache space is studied. Specifically, requesting users arrive randomly and the base station (BS) reactively multicasts files to the requesting users and selected cache nodes. The latter can offload the traffic in their coverage areas from the BS. We consider the joint optimization of the abovementioned file placement and delivery within a finite lifetime subject to the cache space constraint.Within the lifetime, the allocation of multicast power and symbol number for each file transmission at the BS is formulated as a dynamic programming problem with a random stage number. Note that there are no existing solutions to this problem. We develop an asymptotically optimal solution framework by transforming the original problem to an equivalent finite-horizon Markov decision process (MDP) with a fixed stage number. A novel approximation approach is then proposed to address the curse of dimensionality, where the analytical expressions of approximate value functions are provided. We also derive analytical bounds on the exact value function and approximation error. The approximate value functions depend on some system statistics, e.g., requesting users' distribution. One reinforcement learning algorithm is proposed for the scenario where these statistics are unknown.

show abstract

Section: Low-complexity Scheduling Policymentioning

confidence: 99%

Joint Optimization of File Placement and Delivery in Cache-Assisted Wireless Networks With Limited Lifetime and Cache Space

Wang

Cui

et al. 2020

IEEE Trans. Commun.

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, as we have already mentioned in abstract, MDP frameworks and Bellman equation are hard to solve them. [18] [3] try to minimize the average delay of the users' queues using stochastic learning algorithm. Indeed, the stochastic learning algorithm consumes lot of time and users memories.…”

Section: A Related Workmentioning

confidence: 99%

Whittle Index Policy for Multichannel Scheduling in Queueing Systems

Kriouile¹,

Larrañaga²,

Assaad³

2019

2019 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

In this paper, we consider a queueing system with multiple channels (or servers) and multiple classes of users. We aim at allocating the available channels among the users in such a way to minimize the expected total average queue length of the system. This known scheduling problem falls in the framework of Restless Bandit Problems (RBP) for which an optimal solution is known to be out of reach for the general case. The contributions of this paper are as follows. We rely on the Lagrangian relaxation method to characterize the Whittle index values and to develop an index-based heuristic for the original scheduling problem. The main difficulty lies in the fact that, for some queue states, deriving the Whittle's index requires introducing a new approach which consists in introducing a new expected discounted cost function and deriving the Whittle's index values with respect to the discount parameter β. We then deduce the Whittle's indices for the original problem (i.e. with total average queue length minimization) by taking the limit β → 1. The numerical results provided in this paper show that this policy performs very well and is very close to the optimal solution for high number of users.

show abstract

“…Remark 1. In most of the existing literature on wireless resource allocation with approximate MDP [12], [13], [14], the performance can hardly be bounded analytically. The novel solution framework proposed in this paper provides a low-complexity policy whose performance can be bounded analytically.…”

Section: B Scheduling With Approximate Value Functionmentioning

confidence: 99%

MDP-Based Scheduling Design for Mobile-Edge Computing Systems with Random User Arrival

Huang

Wang

2019

2019 IEEE Global Communications Conference (GLOBECOM)

Self Cite

View full text Add to dashboard Cite

In this paper, we investigate the scheduling design of a mobile-edge computing (MEC) system, where the random arrival of mobile devices with computation tasks in both spatial and temporal domains is considered. The binary computation offloading model is adopted. Every task is indivisible and can be computed at either the mobile device or the MEC server. We formulate the optimization of task offloading decision, uplink transmission device selection and power allocation in all the frames as an infinite-horizon Markov decision process (MDP). Due to the uncertainty in device number and location, conventional approximate MDP approaches to addressing the curse of dimensionality cannot be applied. A novel low-complexity suboptimal solution framework is then proposed. We first introduce a baseline scheduling policy, whose value function can be derived analytically. Then, one-step policy iteration is adopted to obtain a sub-optimal scheduling policy whose performance can be bounded analytically. Simulation results show that the gain of the sub-optimal policy over various benchmarks is significant.

show abstract

Delay-Aware Two-Hop Cooperative Relay Communications via Approximate MDP and Stochastic Learning

Cited by 33 publications

References 45 publications

Joint Optimization of File Placement and Delivery in Cache-Assisted Wireless Networks With Limited Lifetime and Cache Space

Joint Optimization of File Placement and Delivery in Cache-Assisted Wireless Networks With Limited Lifetime and Cache Space

Whittle Index Policy for Multichannel Scheduling in Queueing Systems

MDP-Based Scheduling Design for Mobile-Edge Computing Systems with Random User Arrival

Contact Info

Product

Resources

About