Fast Adaptive Task Offloading in Edge Computing Based on Meta Reinforcement Learning

Wang, Jin; Hu, Jia; Min, Geyong; Zomaya, Albert Y.; Georgalas, Nektarios

doi:10.1109/tpds.2020.3014896

Cited by 270 publications

(93 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hybrid DRL method: In [25], The task offloading method was proposed based on meta reinforcement learning, which can adapt fast to new environments with a small number of gradient updates and samples. In [26], an A3C deep reinforcement learning algorithm was introduced to obtain the resource pricing and allocation MEC enabled blockchain systems.…”

Section: Related Workmentioning

confidence: 99%

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multiagent Ensemble Deep Reinforcement Learning With Imitation Acceleration

Jiang

Dong

Wang

et al. 2022

IEEE Internet Things J.

View full text Add to dashboard Cite

In large-scale mobile edge computing (MEC) systems, the task latency and energy consumption are important for massive resource-consuming and delay-sensitive Internet of things devices (IoTDs). Against this background, we propose a distributed intelligent resource scheduling (DIRS) framework to minimize the sum of task latency and energy consumption for all IoTDs, which can be formulated as a mixed integer nonlinear programming. The DIRS framework includes centralized training relying on the global information and distributed decision making by each agent deployed in each MEC server. Specifically, we first introduce a novel multi-agent ensemble-assisted distributed deep reinforcement learning (DRL) architecture, which can simplify the overall neural network structure of each agent by partitioning the state space and also improve the performance of a single agent by combining decisions of all the agents. Secondly, we apply action refinement to enhance the exploration ability of the proposed DIRS framework, where the near-optimal state-action pairs are obtained by a novel Levy flight search. Finally, an imitation acceleration scheme is presented to pre-train all the agents, which can significantly accelerate the learning process of the proposed framework through learning the professional experience from a small amount of demonstration data. The simulation results in three typical scenarios demonstrate that the proposed DIRS framework is efficient and outperforms the existing benchmark schemes.

show abstract

Section: Related Workmentioning

confidence: 99%

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multiagent Ensemble Deep Reinforcement Learning With Imitation Acceleration

Jiang

Dong

Wang

et al. 2022

IEEE Internet Things J.

View full text Add to dashboard Cite

show abstract

“…Recently, Meta-DRL methods are starting to gain popularity due to the adaptive nature of these algorithms. For application of Meta-DRL schemes in edge computing and task offloading the reader can refer to [16], [17].…”

Section: Existing Work In Drl Algorithms For Edge Computing and Hardware Accelerationmentioning

confidence: 99%

Deep Reinforcement Learning Acceleration for Real-Time Edge Computing Mixed Integer Programming Problems

et al. 2022

View full text Add to dashboard Cite

In this work, we present the design and implementation of an ultra-low latency Deep Reinforcement Learning (DRL) FPGA based accelerator for addressing hard real-time Mixed Integer Programming problems. The accelerator exhibits ultra-low latency performance for both training and inference operations, enabled by training-inference parallelism, pipelined training, on-chip weights and replay memory, multi-level replication-based parallelism and DRL algorithmic modifications such as distribution of training over time. The design principles can be extended to support hardware acceleration for other relevant DRL algorithms (embedding the experience replay technique) with hard real time constraints. We evaluate the accuracy of the accelerator in a task offloading and resource allocation problem stemming from a Mobile Edge Computing (MEC/5G) scenario. The design has been implemented on a Xilinx Zynq Ultrascale+ MPSoC ZCU104 evaluation kit using High Level Synthesis. The accelerator achieves near optimal performance and exhibits a 10-fold decrease in training-inference execution latency when compared to a high-end CPU-based implementation.

show abstract

“…Meanwhile, Tang et al [43] proposed a distributed DRL solution to minimize the long-term cost for non-divisible and delay-sensitive tasks using MEC. Wang et al [44] proposed a meta reinforcement learning-based algorithm that leveraged recurrent neural networks for faster loss convergence, and and represented mobile applications as directed acyclic graphs for validation of the algorithm. Finally, Dai et al [45] proposed a DRL-based computation offloading and resource allocation algorithm to reduce overall energy consumption; it used a multi-user end-edge-cloud orchestrated network.…”

Section: A Related Workmentioning

confidence: 99%

A Deep Reinforcement Learning-Based Dynamic Computational Offloading Method for Cloud Robotics

Penmetcha

Min

2021

IEEE Access

View full text Add to dashboard Cite

Robots come with a variety of computing capabilities, and running computationallyintense applications on robots is sometimes challenging on account of limited onboard computing, storage, and power capabilities. Meanwhile, cloud computing provides on-demand computing capabilities, and thus combining robots with cloud computing can overcome the resource constraints robots face. The key to effectively offloading tasks is an application solution that does not underutilize the robot's own computational capabilities and makes decisions based on crucial cost parameters such as latency and CPU availability. In this paper, we formulate the application offloading problem as a Markovian decision process and propose a deep reinforcement learning-based deep Q-network (DQN) approach. The statespace is formulated with the assumption that input data size directly impacts application execution time. The proposed algorithm is designed as a continuous task problem with discrete action space; i.e., we apply a choice of action at each time step and use the corresponding outcome to train the DQN to acquire the maximum rewards possible. To validate the proposed algorithm, we designed and implemented a robot navigation testbed. The results demonstrated that for the given state-space values, the proposed algorithm learned to take appropriate actions to reduce application latency and also learned a policy that takes actions based on input data size. Finally, we compared the proposed DQN algorithm with a long short-term memory (LSTM) algorithm in terms of accuracy. When trained and validated on the same dataset, the proposed DQN algorithm obtained at least 9 percentage points greater accuracy than the LSTM algorithm.INDEX TERMS Cloud robotics, deep reinforcement learning, deep Q-networks (DQN), AWS, neural networks, application offloading, robot navigation.

show abstract

Fast Adaptive Task Offloading in Edge Computing Based on Meta Reinforcement Learning

Cited by 270 publications

References 26 publications

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multiagent Ensemble Deep Reinforcement Learning With Imitation Acceleration

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multiagent Ensemble Deep Reinforcement Learning With Imitation Acceleration

Deep Reinforcement Learning Acceleration for Real-Time Edge Computing Mixed Integer Programming Problems

A Deep Reinforcement Learning-Based Dynamic Computational Offloading Method for Cloud Robotics

Contact Info

Product

Resources

About