“DRL + FL”: An intelligent resource allocation model based on deep reinforcement learning for Mobile Edge Computing

Shan, Nanliang; Cui, Xiaopeng; Gao, Zhiqiang

doi:10.1016/j.comcom.2020.05.037

Cited by 40 publications

(19 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[14] designed three algorithms, including heuristic search, the reformulation linearization technique, and semi-definite relaxation, to jointly minimize latency and offloading failure probability. However, conventional methods face enormous challenges in responding to long-term benefits and a complex MEC environment [15]. The above methods that ensure system optimization at a specific status are not suitable for fast fading channels, as the dynamics of tasks and system environments are not considered.…”

Section: Related Workmentioning

confidence: 99%

Federated Deep Reinforcement Learning for Online Task Offloading and Resource Allocation in WPC-MEC Networks

2022

View full text Add to dashboard Cite

Mobile edge computing (MEC) is considered a more effective technological solution for developing the Internet of Things (IoT) by providing cloud-like capabilities for mobile users. This article combines wireless powered communication (WPC) technology with an MEC network, where a base station (BS) can transfer wireless energy to edge users (EUs) and execute computation-intensive tasks through task offloading. Traditional numerical optimization methods are time-consuming approaches for solving this problem in time-varying wireless channels, and centralized deep reinforcement learning (DRL) is not stable in large-scale dynamic IoT networks. Therefore, we propose a federated DRL-based online task offloading and resource allocation (FDOR) algorithm. In this algorithm, DRL is executed in EUs, and federated learning (FL) uses the distributed architecture of MEC to aggregate and update the parameters. To further solve the problem of the non-IID data of mobile EUs, we devise an adaptive method that automatically adjusts the FDOR algorithm's learning rate. Simulation results demonstrate that the proposed FDOR algorithm is superior to the traditional numerical optimization method and the existing DRL algorithm in four aspects: convergence speed, execution delay, overall calculation rate and stability in large-scale and dynamic IoT.INDEX TERMS Mobile edge computing, federated learning, deep reinforcement learning, online computing offload, wireless powered communication.

show abstract

Section: Related Workmentioning

confidence: 99%

Federated Deep Reinforcement Learning for Online Task Offloading and Resource Allocation in WPC-MEC Networks

2022

View full text Add to dashboard Cite

show abstract

“…Category Objectives QEEC [29] QL Response time, CPU utilization MDP_DT [164] QL Cost AGH+QL [114] QL Energy consumption, QoS MRLCO [26] MRL Network traffic, service latency DeepRM_Plus [46] DRL Turnaround time, cycling time DERP [160] DRL Automatic elasticity DPM [62] DRL Task latency, energy consumption DQST [17] DQL Makespan, load balancing MDRL [48] DDQN Energy consumption, response time RLTS [49] DDQN Makespan DRL-Cloud [115] DDQN Energy consumption, cost ADRL [13] DDQN Resource Utilization, response time IDRQN [60] DDQN Energy consumption, service latency MADRL [50] DDQN Computation delay, channel utilization DDQN [166] DDQN Service latency, system rewards DRL+FL [116] DDQN Energy consumption, load balancing AGH+QL [114], a novel revised Q-learning-based model, takes hash codes as input states with a reduced size of state space. DQST [17], deep Q-learning task scheduling, uses fully connected network to calculate the Q-values which can express the mapper of action decision.…”

Section: Algorithmmentioning

confidence: 99%

“…Actor network with two layer fully connected network is a mapper from state to action, and critic network with two fully connected network hidden layers and an output layer with one node is a mapper from state and action to Q-value. DRL+FL [116], based on DDQN, uses Federal Learning to accelerate the training of DRL agents. MDP_DT [164], a novel full-model based RL for elastic resource management, employs adaptive state space partitioning.…”

Section: Algorithmmentioning

confidence: 99%

“…Table 8: Scenario and Task/Server Nature of RL-based algorithms Algorithm Scenario Task/Server Nature QEEC [29] Online scheduling Heterogeneous servers MDP_DT [164] Dynamic scheduling Dynamic tasks AGH+QL [114] Scheduling in C-RANs Traffic demand MRLCO [26] Adaptive offloading Multiple tasks DeepRM_Plus [46] Online scheduling Independent tasks DERP [160] Dynamic scheduling Dynamic tasks DPM [62] Online scheduling Dynamic tasks DQST [17] Dynamic scheduling Non-preemptive task MDRL [48] Dynamic scheduling Heterogeneous servers RLTS [49] Dynamical scheduling Heterogeneous servers DRL-Cloud [115] Resource provisioning Depended Tasks ADRL [13] On-time scheduling Dynamic tasks IDRQN [60] Task offloading Heterogeneous servers MADRL [50] Multichannel access Joint multichannel DDQN [166] Online scheduling Delay-tolerant tasks DRL+FL [116] Dynamic scheduling Dynamic tasks…”

Section: Algorithmmentioning

confidence: 99%

“…Sadhasivam et al [147] designed an efficient Two-level Scheduler, Intra VMScheduler, for Cloud computing environment where the first level was a queue model and the second was a scheduler. Unlike First Come First Serve (FCFS) Policy and [9,12,21,29,33,35,36,48,[60][61][62] Makespan minimization (execution time minimization) [12, 17, 23, 49, 94-97, 99, 100, 105, 107, 109, 110, 117-136] Delay time (or delayed services) minimization [4,23,37,50,97,98,106,119,121] Response (or running) time minimization [9,13,19,29,30,33,46,63,94,108,124,137] Load balancing maximization [17,23,36,45,60,99,108,116,117,121,136] Reliability or Utilization maximization [13,19,…”

Section: Introduction Of Scheduling In Cloud Computingmentioning

confidence: 99%

See 2 more Smart Citations

Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directions

Zhou¹,

Tian²,

Buyya³

2021

Preprint

View full text Add to dashboard Cite

As the quantity and complexity of information processed by software systems increase, large-scale software systems have an increasing requirement for high-performance distributed computing systems. With the acceleration of the Internet in Web 2.0, Cloud computing as a paradigm to provide dynamic, uncertain and elastic services has shown superiorities to meet the computing needs dynamically. Without an appropriate scheduling approach, extensive Cloud computing may cause high energy consumptions and high cost, in addition that high energy consumption will cause massive carbon dioxide emissions. Moreover, inappropriate scheduling will reduce the service life of physical devices as well as increase response time to users' request. Hence, efficient scheduling of resource or optimal allocation of request, that usually a NP-hard problem, is one of the prominent issues in emerging trends of Cloud computing. Focusing on improving quality of service (QoS), reducing cost and abating contamination, researchers have conducted extensive work on resource scheduling problems of Cloud computing over years. Nevertheless, growing complexity of Cloud computing, that the super-massive distributed system, is limiting the application of scheduling approaches. Machine learning, a utility method to tackle problems in complex scenes, is used to resolve the resource scheduling of Cloud computing as an innovative idea in recent years. Deep reinforcement learning (DRL), a combination of deep learning (DL) and reinforcement learning (RL), is one branch of the machine learning and has a considerable prospect in resource scheduling of Cloud computing. This paper surveys the methods of resource scheduling with focus on DRL-based scheduling approaches in Cloud computing, also reviews the application of DRL as well as discusses challenges and future directions of DRL in scheduling of Cloud computing.

show abstract

Research on image processing of electric power system terminals based on reinforcement learning and mobile edge computing optimization

Zhou,

Yu,

Luo

et al. 2024

Adv Control Appl

View full text Add to dashboard Cite

This research is dedicated to the optimization of power system terminal image processing based on RL and MEC. With the continuous development of power system, the demand for image processing of terminal equipment is increasing day by day. However, traditional image processing methods have the problems of high computing complexity and real‐time and energy consumption. To solve this problem, this study introduces the idea of RL and MEC to improve the efficiency and performance of image processing of power system terminals. By modeling and optimizing the image processing task of the power system terminal equipment, the intelligent adjustment of the processing parameters is realized to adapt to the needs of different scenarios. MEC technology is introduced to move image processing tasks from the central server to the edge device, reducing data transmission delay and network burden, thus improving real‐time performance and reducing energy consumption. The experimental results show that the proposed optimization method based on RL and MEC has a significant performance improvement compared with the traditional method in the power system terminal image processing. The framework our proposed has achieved significant improvement in task completion latency, achieving higher system energy efficiency compared to traditional methods.

show abstract

“DRL + FL”: An intelligent resource allocation model based on deep reinforcement learning for Mobile Edge Computing

Cited by 40 publications

References 16 publications

Federated Deep Reinforcement Learning for Online Task Offloading and Resource Allocation in WPC-MEC Networks

Federated Deep Reinforcement Learning for Online Task Offloading and Resource Allocation in WPC-MEC Networks

Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directions

Research on image processing of electric power system terminals based on reinforcement learning and mobile edge computing optimization

Contact Info

Product

Resources

About