Multiuser Resource Control With Deep Reinforcement Learning in IoT Edge Computing

Abstract

By leveraging the concept of mobile edge computing (MEC), massive amount of data generated by a large number of Internet of Things (IoT) devices could be offloaded to MEC server at the edge of wireless network for further computational intensive processing. However, due to the resource constraint of IoT devices and wireless network, both the communications and computation resources need to be allocated and scheduled efficiently for better system performance. In this paper, we propose a joint computation offloading and multi-user scheduling algorithm for IoT edge computing system to minimize the long-term average weighted sum of delay and power consumption under stochastic traffic arrival. We formulate the dynamic optimization problem as an infinite-horizon average-reward continuous-time Markov decision process (CTMDP) model. One critical challenge in solving this MDP problem for the multi-user resource control is the curse-of-dimensionality problem, where the state space of the MDP model and the computation complexity increase exponentially with the growing number of users or IoT devices. In order to overcome this challenge, we use the deep reinforcement learning (RL) techniques and propose a neural network architecture to approximate the value functions for the post-decision system states. The designed algorithm to solve the CTMDP problem supports semi-distributed auction-based implementation, where the IoT devices submit bids to the BS to make the resource control decisions centrally. Simulation results show that the proposed algorithm provides significant performance improvement over the baseline algorithms, and also outperforms the RL algorithms based on other neural network architectures.

Deep reinforcement learning based computation offloading and resource allocation for low-latency fog radio access networks

“…In Ref. [23], a DRL scheme for IoT edge computing was proposed in consideration of joint computation offloading and multiuser scheduling algorithm to minimize the long-term average weighted sum of delay and power consumption under stochastic traffic arrival. In contrast to Refs.…”

“…In contrast to Refs. [11,23], Ref. [24] studied the double Deep Q-Network (DQN) for efficient computation offloading in ultradense Mobile Edge Computing (MEC) networks.…”

“…Manuscript received: 2020-10-17; revised: 2020-11-02; accepted: 2020- [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28] demand of these applications [3] . Furthermore, the fifthgeneration cellular systems have enabled the explosive growth of IoT devices, whose number can reach approximately 24 billion by 2020.…”

Deep Reinforcement Learning for Performance-Aware Adaptive Resource Allocation in Mobile Edge Computing

“…To relieve the above conflict, mobile edge computing (MEC), a novel computing paradigm whose objective is to bring the computing and storage capacities close to mobile devices and users, becomes attractive [3] [5]. Therefore, smart mobile devices can offload some of their mobile application workloads via a wireless channel to nearby edge servers, which significantly augment the capacities of mobile devices [6].…”

“…For energy consumption and latency-based resource allocation, the objective is to optimize the weighted sum of the energy consumption and latency. For example, [5] formulates the task offloading problem as a MDP and adopts a deep reinforcement learning technique to solve this problem. [20] jointly optimizes the offloading decision and computational resource allocation.…”