Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing

Huang, Liang; Xu, Feng; Zhang, Cheng; Qian, Liping; Wu, Yuan

doi:10.1016/j.dcan.2018.10.003

Cited by 228 publications

(124 citation statements)

References 10 publications

Supporting

Mentioning

121

Contrasting

Unclassified

Order By: Relevance

“…In particular, it relies on deep neural networks (DNNs) [17] to learn from the training data samples, and eventually produces the optimal mapping from the state space to the action space. There exists limited work on deep reinforcement learning-based offloading for MEC networks [18]- [22]. By taking advantage of parallel computing, [19] proposed a distributed deep learning-based offloading (DDLO) algorithm for MEC networks.…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks

Huang

Zhang

2020

IEEE Trans. on Mobile Comput.

Self Cite

850

355

View full text Add to dashboard Cite

Wireless powered mobile-edge computing (MEC) has recently emerged as a promising paradigm to enhance the data processing capability of low-power networks, such as wireless sensor networks and internet of things (IoT). In this paper, we consider a wireless powered MEC network that adopts a binary offloading policy, so that each computation task of wireless devices (WDs) is either executed locally or fully offloaded to an MEC server. Our goal is to acquire an online algorithm that optimally adapts task offloading decisions and wireless resource allocations to the time-varying wireless channel conditions. This requires quickly solving hard combinatorial optimization problems within the channel coherence time, which is hardly achievable with conventional numerical optimization methods. To tackle this problem, we propose a Deep Reinforcement learning-based Online Offloading (DROO) framework that implements a deep neural network as a scalable solution that learns the binary offloading decisions from the experience. It eliminates the need of solving combinatorial optimization problems, and thus greatly reduces the computational complexity especially in large-size networks. To further reduce the complexity, we propose an adaptive procedure that automatically adjusts the parameters of the DROO algorithm on the fly. Numerical results show that the proposed algorithm can achieve near-optimal performance while significantly decreasing the computation time by more than an order of magnitude compared with existing optimization methods. For example, the CPU execution latency of DROO is less than 0.1 second in a 30-user network, making real-time and optimal offloading truly viable even in a fast fading environment.Index Terms-Mobile-edge computing, wireless power transfer, reinforcement learning, resource allocation. ! • L. Huang is with the College

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks

Huang

Zhang

2020

IEEE Trans. on Mobile Comput.

Self Cite

850

355

View full text Add to dashboard Cite

show abstract

“…The last category of previous works studied the data offloading through machine learning [14][15]. In [14], Min et al provided a reinforcement learning based algorithm for energy harvesting devices.…”

Section: Related Workmentioning

confidence: 99%

“…The deep learning model learned the optimal offloading policy in respect to the device's internal conditions, transmission conditions, and the energy input. The authors in [15] proposed a Deep-Q-Network based resource allocation algorithm to manage an executionoffloading schedule for multiple users and devices with the objective of minimizing energy consumption and delay. In [15], a single computing task divided the total data into predetermined data blocks that had the option to be offloaded or executed locally.…”

Section: Related Workmentioning

confidence: 99%

“…The authors in [15] proposed a Deep-Q-Network based resource allocation algorithm to manage an executionoffloading schedule for multiple users and devices with the objective of minimizing energy consumption and delay. In [15], a single computing task divided the total data into predetermined data blocks that had the option to be offloaded or executed locally. Our work differs in two major aspects: 1) instead of using deep learning, we propose a FPTAS which solves the problem approximately within limited time; and 2) instead of considering energy and delay, our objective focuses on the finishing time which includes computing, communication, and moving time.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Mobile Edge Computing for Mobile Internet of Thing in 5G Networks

Chevalier

Wang

2020

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

We study the off-line efficient mobile edge computing (EMEC) problem for a joint computing to process a task both locally and remotely with the objective of minimizing the finishing time. When computing remotely, the time will include the communication and computing time. We first describe the time model, formulate EMEC, prove NPcompleteness of EMEC, and show the lower bound. We then provide an integer linear programming (ILP) based algorithm to achieve the optimal solution and give results for small-scale cases. A fully polynomialtime approximation scheme (FPTAS), named Approximation Partition (AP), is provided through converting ILP to the subset sum problem. Numerical results show that both the total data length and the movement have great impact on the time for mobile edge computing. Numerical results also demonstrate that our AP algorithm obtain the finishing time, which is close to the optimal solution.

show abstract

“…Specifically, DRL algorithms for multi-user MEC system have been considered in several existing works. [27] and [28] focus on the offloading and resource allocation problems under deterministic task models, where a fixed number of tasks per user need to be processed either locally or offloaded to the edge server. DQN based techniques are applied to solve the respectively problems.…”

Section: A Related Workmentioning

confidence: 99%

Multiuser Resource Control With Deep Reinforcement Learning in IoT Edge Computing

Lei

Xiong

et al. 2019

IEEE Internet Things J.

View full text Add to dashboard Cite

By leveraging the concept of mobile edge computing (MEC), massive amount of data generated by a large number of Internet of Things (IoT) devices could be offloaded to MEC server at the edge of wireless network for further computational intensive processing. However, due to the resource constraint of IoT devices and wireless network, both the communications and computation resources need to be allocated and scheduled efficiently for better system performance. In this paper, we propose a joint computation offloading and multi-user scheduling algorithm for IoT edge computing system to minimize the long-term average weighted sum of delay and power consumption under stochastic traffic arrival. We formulate the dynamic optimization problem as an infinite-horizon average-reward continuous-time Markov decision process (CTMDP) model. One critical challenge in solving this MDP problem for the multi-user resource control is the curse-of-dimensionality problem, where the state space of the MDP model and the computation complexity increase exponentially with the growing number of users or IoT devices. In order to overcome this challenge, we use the deep reinforcement learning (RL) techniques and propose a neural network architecture to approximate the value functions for the post-decision system states. The designed algorithm to solve the CTMDP problem supports semi-distributed auction-based implementation, where the IoT devices submit bids to the BS to make the resource control decisions centrally. Simulation results show that the proposed algorithm provides significant performance improvement over the baseline algorithms, and also outperforms the RL algorithms based on other neural network architectures.

show abstract

Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing

Cited by 228 publications

References 10 publications

Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks

Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks

Efficient Mobile Edge Computing for Mobile Internet of Thing in 5G Networks

Multiuser Resource Control With Deep Reinforcement Learning in IoT Edge Computing

Contact Info

Product

Resources

About