Dueling Deep Q-Network Learning Based Computing Offloading Scheme for F-RAN

Jiang, Fan; Ma, Rongxin; Sun, Changyin; Gu, Zesheng

doi:10.1109/pimrc48278.2020.9217355

Cited by 8 publications

(5 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e work in [12] introduces an empirical replay technique based on the reinforcement learning, which improves the convergence rate of the offloading algorithm. Similarly, the study in [13] adds a layer of the LSTM network to the deep Q network, which is used for predicting the amount of tasks to be received by the fog node at the next moment. Compared to the deep Q algorithm, the algorithm with the LSTM network can arrive at the optimal decision faster.…”

Section: Introductionmentioning

confidence: 99%

Randomization-Based Dynamic Programming Offloading Algorithm for Mobile Fog Computing

Bai

Yang

Zhang

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

Offloading to fog servers makes it possible to process heavy computational load tasks in local devices. However, since the generation problem of offloading decisions is an N-P problem, it cannot be solved optimally or traditionally, especially in multitask offloading scenarios. Hence, this paper has proposed a randomization-based dynamic programming offloading algorithm, based on genetic optimization theory, to solve the offloading decision generation problem in mobile fog computing. The algorithm innovatively designs a dynamic programming table-filling approach, i.e., iteratively generates a set of randomized offloading decisions. If some in these sets improve the decisions in the DP table, then they will be merged into the table. The iterated DP table is also used to improve the set of decisions generated in the iteration to obtain the optimal offloading approximate solution. Extensive simulations show that the proposed DPOA can generate decisions within 3 ms and the benefit is especially significant when users are in multitask offloading scenarios.

show abstract

Section: Introductionmentioning

confidence: 99%

Randomization-Based Dynamic Programming Offloading Algorithm for Mobile Fog Computing

Bai

Yang

Zhang

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Therefore, a preprocessing mechanism is implemented to reduce the complexity of the dueling DQN algorithm by immediately fulfilling some of the UE's task demands. Similarly, [115] proposed a new offloading policy in an F-RAN that uses the dueling DQN method to optimize the overall utility of UEs.…”

Section: ) Dueling Dqnmentioning

confidence: 99%

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

Abdulazeez

Askar

2023

IEEE Access

View full text Add to dashboard Cite

Fog computing has emerged as a computing paradigm for resource-restricted Internet of things (IoT) devices to support time-sensitive and computationally intensive applications. Offloading can be utilized to transfer resource-intensive tasks from resource-limited end devices to a resource-rich fog or cloud layer to reduce end-to-end latency and enhance the performance of the system. However, this advantage is still challenging to achieve in systems with a high request rate because it leads to long queues of tasks in fog nodes and reveals inefficiencies in terms of delays. In this regard, reinforcement learning (RL) is a well-known method for addressing such decision-making issues. However, in large-scale wireless networks, both action and state spaces are complex and extremely extensive. Consequently, reinforcement learning techniques may not be able to identify an efficient strategy within an acceptable time frame. Hence, deep reinforcement learning (DRL) was developed to integrate RL and deep learning (DL) to address this problem. This paper presents a systematic analysis of using RL or DRL algorithms to address offloadingrelated issues in fog computing. First, the taxonomy of fog computing offloading mechanisms based on RL and DRL algorithms was divided into three major categories: value-based, policy-based, and hybridbased algorithms. These categories were then compared based on important features, including offloading problem formulation, utilized techniques, performance metrics, evaluation tools, case studies, their strengths and drawbacks, offloading directions, offloading mode, SDN-based architecture, and offloading decisions. Finally, the future research directions and open issues are discussed thoroughly.INDEX TERMS Fog computing, Internet of Things (IoT), offloading, reinforcement learning, deep reinforcement learning.

show abstract

“…If the result can be directly obtained in the preprocessing stage, the maximum utility that UE n can obtain is expressed as However, if the result cannot be found in the pre-processing stage, the offloading procedure will be adopted. Since the BS server is equipped with a powerful computing server, the searching and delivery of the task result can be completed so fast that the delay to (19…”

Section: The Pre-processing Stagementioning

confidence: 99%

“…[18] effectively improved the offloading efficiency and decreasing the resource costing. Therefore, in our previous work [19], we studied a computing offloading policy for multiple user equipments (UEs) in F-RANs by using the DQN algorithm to optimize the total utility of UEs. However, the limitation of the work is that computational resource for FAPs has not been optimized.…”

mentioning

confidence: 99%

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

Jiang

Gao

et al. 2021

EURASIP J. Adv. Signal Process.

Self Cite

View full text Add to dashboard Cite

This paper investigates a computing offloading policy and the allocation of computational resource for multiple user equipments (UEs) in device-to-device (D2D)-aided fog radio access networks (F-RANs). Concerning the dynamically changing wireless environment where the channel state information (CSI) is difficult to predict and know exactly, we formulate the problem of task offloading and resource optimization as a mixed-integer nonlinear programming problem to maximize the total utility of all UEs. Concerning the non-convex property of the formulated problem, we decouple the original problem into two phases to solve. Firstly, a centralized deep reinforcement learning (DRL) algorithm called dueling deep Q-network (DDQN) is utilized to obtain the most suitable offloading mode for each UE. Particularly, to reduce the complexity of the proposed offloading scheme-based DDQN algorithm, a pre-processing procedure is adopted. Then, a distributed deep Q-network (DQN) algorithm based on the training result of the DDQN algorithm is further proposed to allocate the appropriate computational resource for each UE. Combining these two phases, the optimal offloading policy and resource allocation for each UE are finally achieved. Simulation results demonstrate the performance gains of the proposed scheme compared with other existing baseline schemes.

show abstract

Dueling Deep Q-Network Learning Based Computing Offloading Scheme for F-RAN

Cited by 8 publications

References 6 publications

Randomization-Based Dynamic Programming Offloading Algorithm for Mobile Fog Computing

Randomization-Based Dynamic Programming Offloading Algorithm for Mobile Fog Computing

Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment

A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN

Contact Info

Product

Resources

About